Lists of interview questions are easy to find and mostly useless, because the question is not the thing being assessed. An interviewer asking "what is RAG?" is not checking whether you can recite a definition. They are finding out how deep to go next.
Fundamentals: are you worth a harder question?
Opening questions are calibration. They are deliberately answerable, and the interviewer is listening to how you structure the answer rather than whether you know it.
- What is retrieval-augmented generation and what problem does it solve? — Testing whether you can explain a system to somebody without your context.
- What is a token, and why does it matter commercially? — Testing whether you have ever looked at a bill.
- What is the difference between fine-tuning and RAG? — Testing whether you reach for the expensive option by default.
- What is temperature? — Testing whether you have tuned anything, or only accepted defaults.
- Why do models hallucinate? — Testing whether you understand that generation is prediction, not retrieval.
Applied: have you actually built one?
This is where memorised answers come apart, because the follow-ups depend on your previous answer rather than on a script.
- Your RAG system returns confident wrong answers. How do you diagnose it? — The good answer separates retrieval failure from generation failure before touching the prompt.
- How would you chunk a 300-page technical manual? — Testing whether you know that fixed-size chunking destroys structure, and what you would do instead.
- How do you evaluate whether a change to the prompt made things better? — The most revealing question in the set. Most candidates have no answer beyond "it looked better".
- Your embedding search returns the right document but the model answers from the wrong part of it. What now? — Testing whether you understand what happens after retrieval.
- How would you stop the model answering questions outside its remit? — Testing whether you know that prompt instructions alone are not a control.
System design: can you reason about cost and failure?
Senior interviews spend most of their time here. The interviewer wants to see you make trade-offs out loud and name what you are trading away.
A typical prompt: design a support assistant over a company knowledge base, handling ten thousand queries a day. The answer they are hoping for covers retrieval, caching, evaluation, cost per query, what happens when the model is wrong, and what a human can override. The answer they usually get covers the architecture diagram and stops.
- Where does the cost actually go, and which step would you route to a cheaper model?
- What happens when the vector store is unavailable? Does the whole feature fail?
- How do you know it is working a month after launch, without reading transcripts by hand?
- What is the blast radius of a wrong answer, and does that change the design?
Behavioural questions are technical questions in disguise
Candidates prepare for the technical rounds and improvise the behavioural ones, which is backwards for this field. "Tell me about a time something you built failed" is not a personality test; it is a technical question wearing a soft jacket, and the interviewer is assessing whether you can reason about a system after the fact.
The weak answer describes a failure caused by somebody else and resolved by effort. The strong answer names what you did not know at the time, what signal you missed, and what you changed about how you work. That second answer is hard to fake because it requires having genuinely been through it.
- Tell me about a technical decision you would make differently now. — Testing self-assessment, and whether you have enough distance from your own work to critique it.
- Describe a time you disagreed with a technical approach. — Testing whether you can hold a position without being difficult about it.
- What is something you believed about LLMs six months ago that you no longer believe? — Increasingly common, and it separates people keeping up from people who read one article in 2024.
What interviewers notice that candidates do not
Two habits make a disproportionate difference and neither is about knowledge. The first is asking a clarifying question before answering a design prompt. Candidates worry it looks uncertain; it reads as experience, because nobody senior starts designing before establishing constraints.
The second is saying "I do not know" cleanly and then reasoning towards an answer anyway. An interviewer who asks something outside your experience is usually probing for the edge of it, and finding that edge honestly is a better outcome than a confident guess that unravels over the next two questions. The candidates who lose points here are not the ones who did not know — they are the ones who pretended.
How to prepare in a way that survives follow-ups
Memorising answers does not work, and it is audible within two follow-up questions. What works is having built something and being able to talk about the parts that went wrong.
A practical method: take one project you have actually done, and write out the three decisions you made that could have gone the other way. For each, why you chose what you chose, what it cost you, and what you would do differently. That preparation answers most applied and behavioural questions, because most of them are really asking for exactly this.
Then answer out loud, recorded, before reading anything about how to answer. The gap between knowing something and being able to say it under mild pressure is the thing interviews measure, and it is invisible until you hear yourself.
Key takeaways
- 1.Opening questions are calibration, not assessment. The interviewer is deciding how deep to go.
- 2.Evaluation is the question most candidates cannot answer, and the one that most distinguishes people who have shipped.
- 3.Senior interviews are about cost, failure and blast radius, not about architecture diagrams.
- 4.Preparation that survives follow-ups comes from one real project, not from a list of answers.
Do it now, free
AI Interview Question Generator
Tell it the role and the seniority and it produces twelve to fifteen questions you could genuinely be asked, grouped by type, each with a line on what the interviewer is actually testing.
No sign-up. The internship is free to join too, with real project briefs and reviewed submissions.