1. Explain how a large language model works - to a non-technical stakeholder.
The opener at most AI-role interviews, and the framing is the test: they want translation ability, not architecture lectures, because AI developers spend half their time explaining capabilities and limits to product people. A strong stakeholder version: "It has read an enormous amount of text and learned the patterns, so given some text it predicts what plausibly comes next - which turns out to be powerful enough to draft, summarise, and code. It doesn’t look facts up, so it can be confidently wrong, and everything we build around it accounts for that." Two sentences of capability, one of limitation - the shape signals you deploy these systems responsibly.
2. What is a hallucination, and how do you reduce hallucination in a product?
A hallucination is fluent, confident output that is factually wrong - a direct consequence of predicting plausible text rather than retrieving truth. The layered mitigations interviewers want to hear: ground the model in real data (retrieval/RAG so answers cite provided documents), constrain the task (extraction and summarisation hallucinate less than open generation), instruct for honesty ("say when the context doesn’t contain the answer"), validate outputs downstream (schemas, checks against source data), and design the UX for fallibility (citations, confidence signals, human review on high-stakes paths). Naming multiple layers matters - single-fix answers ("just use RAG") read as tutorial-depth.
3. Explain RAG - what problem does it solve and how does it work?
Retrieval-Augmented Generation solves the model’s two knowledge gaps: private data it never saw and events after its training. The pipeline: documents are chunked and embedded into vectors, stored in a vector database; a user’s question is embedded the same way; the most similar chunks are retrieved and injected into the prompt; the model answers FROM that context. The depth signals interviewers probe for: chunking strategy affects everything, retrieval quality is usually the bottleneck (garbage retrieved = garbage generated), and hybrid search (keyword + semantic) often beats pure vectors. Having built even one small RAG tool - a document Q&A for a niche you know - converts this from recitation to testimony.
4. What is an AI agent, and when is an agent the wrong answer?
An agent is a model given tools and a loop: it reasons about a goal, calls tools (search, database, code execution), observes results, and repeats until done - unlike a single prompt-response call. The second half of the question is where offers are decided, because agent-hype burned real teams: the wanted answer is that agents earn their complexity only when the path is genuinely dynamic (unknown steps, tool choice depends on intermediate results), and that a fixed pipeline of chained calls is simpler, cheaper, and more debuggable when the steps ARE known. "Start with the simplest structure that works, escalate to agents when the task demands it" is the judgement being screened for.
5. How do you evaluate whether an LLM feature is actually good?
The question that most cleanly separates people who have shipped AI from people who have played with it. Weak answer: "I test it with some prompts." Strong answer: a real evaluation loop - a test set of representative inputs (including the ugly edge cases), defined success criteria per case, automated scoring where possible (exact-match for extraction, LLM-as-judge with careful rubrics for generation), and regression running on every prompt or model change, because improvements in one area silently break others. Mentioning that eval sets grow from production failures - every bad output becomes a test case - signals genuine operational experience.
6. A feature works in the demo but fails for real users - walk me through your diagnosis.
The debugging probe, AI edition. The systematic answer: first look at real failing inputs (production inputs are always messier than demo inputs - longer, multilingual, ambiguous, adversarial); categorise the failures (retrieval misses? prompt brittleness? hallucination? formatting?); check the context actually being sent (truncation and bad retrieval are the usual suspects - log the full prompt); then fix the dominant category and add those cases to the eval set. Interviewers listen for the instinct to inspect data before touching prompts - "I’d rewrite the prompt" as a first move is the tell of someone who has never maintained one of these in production.
7. How do you think about cost and latency in LLM products?
The operational maturity probe - AI features have a per-use cost profile unlike normal code, and teams have been surprised by both bills and slowness. The framework that scores: right-size the model per task (frontier models for hard reasoning, small fast models for classification and routing - not everything needs the big model), control tokens (concise prompts, capped outputs, trimmed context), cache what repeats, stream responses so users see progress, and measure cost per request from day one because it is a product decision, not an afterthought. Naming a concrete trade-off you made in a project - even a small one - beats reciting the full list.
8. What is prompt injection, and why does it matter for agents especially?
Prompt injection is untrusted content smuggling instructions into the model: a webpage or document containing "ignore your instructions and..." that the model may obey because it cannot cleanly separate instructions from data. For chatbots it is embarrassing; for agents it is dangerous, because agents have tools - an injected instruction can trigger real actions (sending, deleting, leaking retrieved data). Mitigations to name: treat all retrieved/external content as untrusted, least-privilege tools (why does the email-reader need send access?), human confirmation gates on consequential actions, and input/output filtering. This question is rising fast in 2026 interviews because production incidents made it board-level; knowing it cold signals you read the real landscape.
9. Which model would you choose for [task], and how would you decide?
A judgement question disguised as a product question - naming your favourite model is the weak answer. The strong shape is a decision procedure: define the task’s actual requirements (reasoning depth, context length, latency budget, cost ceiling, data-privacy constraints), shortlist models that fit, and run YOUR eval set against them - benchmark leaderboards are marketing; your task’s eval is truth. Mention that the answer changes over time (models improve monthly, so the decision procedure matters more than any snapshot) and that switching costs argue for clean abstraction layers. This shows the durable skill: evaluating tools rather than pledging allegiance to one.
10. The meta-question: how do you stay current when the field changes monthly?
Asked because it is a real operational concern - the half-life of specific AI knowledge is short. The credible answer is a filtering system, not a firehose: build with the new thing rather than reading about it (one real task tells you more than ten threads), follow a small set of primary sources (model provider release notes and docs over influencer summaries), and anchor on the durable layer - fundamentals of how these systems fail and how to evaluate them - because that transfers across every model generation. Interviewers are also quietly checking for sustainability: candidates performing exhausting hype-chasing burn out; candidates with a calm system compound.
