The practice room
Put what you know to work.
Every interview question, code lab, design problem, and behavioral framework: searchable, filterable, completion synced with the per-topic pages.
- A batch X has shape 32 × 128 and a dense layer produces 64 features. What are the weight, bias and output shapes?
- For x = [1, 0] and y = [3, 4], compute the dot product and cosine similarity. What changes if y is doubled?
- Multiply X = [[2, 1], [0, 3]] by W = [[1, −1], [2, 0]]. How does this differ from element-wise multiplication?
- In one attention head, Q and K are 3 × 2 and V is 3 × 4. Explain every output shape in softmax(QKᵀ / √2)V.
- Why is [[1, 2], [2, 4]] rank-deficient, and what does that imply about recovering its input?
- Project y = [3, 1] onto the line spanned by u = [1, 1]. How can you check your answer?
- A covariance matrix is [[2, 1], [1, 2]]. Find its principal direction and the fraction of variance it explains.
- A 4096 × 4096 weight matrix uses a rank-8 LoRA update. How many trainable update parameters are needed, and what is the constraint?
- A fraud detector flags 90 of 100 frauds and 198 of 9,900 ordinary payments. Calculate recall, false-positive rate and precision.
- Does independence mean two events cannot happen together?
- X is 0, 1 or 2 with probabilities 0.25, 0.5 and 0.25. Find its mean and variance.
- Why is a probability density of 2 valid even though probabilities cannot exceed 1?
- Compare maximum likelihood and a Beta posterior after 3 successes in 3 Bernoulli trials.
- What does a 95% confidence interval say, and how does it differ from a credible interval?
- Why should two models evaluated on the same examples be compared with paired outcomes?
- Your A/B dashboard checks significance hourly and stops at p < 0.05. What is wrong with that rule?
- For L(w)=½(2w−6)² at w=1, calculate the gradient and one update with learning rate 0.1.
- What does the chain rule do in backpropagation?
- For Y=XW+b, derive the parameter gradients given G=∂L/∂Y.
- Why can a correct gradient step increase the loss?
- Does a zero gradient prove a global minimum?
- What makes a mini-batch gradient an unbiased estimate, and when might it not be?
- Why is AdamW different from adding L2 regularization to Adam?
- How would you check a suspected backpropagation bug?
- Compute the entropy of p=[0.5,0.25,0.25] in bits.
- For p=[0.5,0.25,0.25] and q=[0.25,0.5,0.25], find cross-entropy and KL.
- Why does cross-entropy punish a confident error so strongly?
- Is KL divergence a distance metric?
- The correct probabilities for two tokens are 0.5 and 0.25. Compute sequence perplexity.
- What is the mutual information of two identical fair bits, and of two independent fair bits?
- Does lower output entropy mean a model is more reliable?
- Why can perplexity comparisons across tokenizers be misleading?
- Why is softmax([1000,1001,1002]) dangerous to compute directly, and how do you fix it?
- How do FP16 and BF16 differ?
- What is the difference between an ill-conditioned problem and an unstable algorithm?
- Why can E[X²]−E[X]² be a poor way to compute variance?
- Why does making the finite-difference step smaller eventually make a gradient check worse?
- What is the correct relationship between loss scaling and gradient clipping?
- Why can an all-masked attention row still produce NaNs after subtracting the maximum?
- Two GPU reductions differ slightly. How would you decide whether it is a bug?
- State Bayes' theorem and explain each term.
- A medical test has 95% sensitivity and 95% specificity. Disease prevalence is 1%. If a patient tests positive, what is the probability they have the disease?
- Explain the relationship between L2 regularization and a Gaussian prior in Bayesian terms.
- Your fraud detection classifier shows 99% accuracy on the test set but catches almost zero fraudsters in production. What is happening and how do you fix it?
- How would you implement a Naive Bayes spam classifier from scratch? Walk through the key steps.
- What is calibration in ML model predictions, and why is it critical for safety-critical applications like autonomous driving? How do you measure and improve it?
- What is the bias-variance tradeoff?
- How do you diagnose underfitting vs overfitting, and what would you do about each?
- You have a model with high bias. Walk through 5 concrete things you would try to fix it, in order.
- Your model performs great on the test set but poorly in production. What could be happening beyond overfitting?
- What is double descent, and how does it relate to the bias-variance decomposition?
- You build a k-fold cross-validation pipeline for a churn model and get 88% AUC. In production it drops to 71%. What likely went wrong?
- When would you use leave-one-out cross-validation, and when would you avoid it?
- Walk through how you would design a cross-validation strategy for a daily demand-forecasting model.
- Explain L1 vs L2 regularization. Why does L1 produce sparsity but L2 does not?
- AdamW vs Adam with L2 regularization: what is the difference and when does it matter?
- Walk through how dropout works during training vs inference. What is inverted dropout?
- You are fine-tuning a 70B parameter foundation model on 5,000 domain-specific examples. What regularization strategies apply at this scale?
- Your fraud detection model has 99% accuracy on the test set. Your team is celebrating. Why are you skeptical?
- What can ROC-AUC miss in a rare-event problem, and why should you also inspect precision–recall behavior?
- A model has Brier score 0.05 and AUC 0.92, but the business says "the 80% confident predictions are wrong half the time." What is happening and how do you fix it?
- Design the evaluation stack for a new web search ranking system. Walk through your offline metrics, your online metrics, and how the two connect.
- When would you use the normal equation vs gradient descent to fit a linear regression?
- Why do we use cross-entropy loss for logistic regression instead of mean squared error? What goes wrong if you use MSE?
- Does optimizing log-loss guarantee that logistic regression is calibrated? How would you evaluate and improve calibration?
- When is logistic regression a useful component of large-scale ads CTR prediction, even when deep models are available?
- You're building a credit-scoring model for a regulated lender. The data science team proposes XGBoost; risk and legal push back and want logistic regression. Walk through the tradeoffs and how you would mediate.
- Why can the RBF kernel represent an infinite-dimensional feature space without ever computing it? Walk through the mechanics of the kernel trick.
- You have an RBF SVM that overfits badly: train accuracy 99%, validation accuracy 65%. Walk through how you would diagnose and fix this. What roles do C and γ play?
- How does one-class SVM work for anomaly detection, and when would you choose it over Isolation Forest or an autoencoder?
- Walk me through XGBoost's objective function. Why does the second-order Taylor expansion matter?
- What problem does CatBoost's ordered boosting solve, and how does it differ from standard GBDT?
- Design a fraud detection system for a payments platform processing 10M transactions/day with sub-100ms decision SLA.
- You have a CTR model with billions of training rows, 100M+ unique user IDs and 10M+ unique ad IDs. Would you use XGBoost? Why or why not?
- What limitation of static embeddings do contextual embeddings address, and when might static vectors still be sufficient?
- How does Word2Vec negative sampling reduce training cost, and how does its objective differ from full softmax?
- What problem does FastText solve that Word2Vec cannot, and how?
- A data scientist wants to cluster 5M customer records to find segments for a marketing campaign. Walk through your approach.
- Why does ReLU help with vanishing gradients compared to sigmoid?
- AdamW vs Adam: what changed and why does it matter for transformers?
- BatchNorm vs LayerNorm vs RMSNorm: when do you use each, and why do LLMs use LayerNorm/RMSNorm?
- Walk through implementing backprop for a 2-layer MLP from scratch. No autograd.
- You are training a 7B model and the loss spikes / NaNs out after 50K steps. Walk me through your debug strategy.
- Explain Xavier vs He initialization. When do you use each, and why does it matter?
- What does gradient clipping control, and when would you clip by norm or by value?
- Walk through mixed-precision training. Why is BF16 preferred over FP16 for LLM pretraining?
- Compare cross-entropy, MSE, and focal loss. When would you choose each?
- Explain KL divergence and its asymmetry. Why does direction matter in practice?
- How does a cosine learning-rate schedule with warmup work, and what needs tuning?
- Why do logits-based cross-entropy implementations improve numerical stability?
- Walk through the storyline from n-grams to attention. What problem did each step solve, and what new problem did it introduce?
- How does the LSTM forget gate help gradient flow, and what does it not guarantee?
- You're asked to build a next-word predictor for a mobile keyboard with a 10MB memory budget. Walk through the architectural options across the n-gram → RNN → attention spectrum and pick one with reasoning.
- What do locality, parameter sharing, and translation equivariance give a convolutional layer, and where do those assumptions help?
- Compute the receptive field of a 5-layer CNN with 3×3 kernels, stride 1, no dilation. Then redo it with stride 2 on layers 2 and 4. Then with dilation rates [1, 2, 4, 8, 16] and stride 1.
- Why do residual connections enable training 100+ layer networks? Explain at both the optimization and gradient-flow level, and describe the "degradation problem" they solved.
- Trace the LSTM gates mathematically. How does its cell-state path help gradients, and what are the limits?
- Compare LSTM and GRU. When would you choose one over the other?
- How does BPTT train an RNN, and what can gradient clipping and truncation change?
- Walk through how Bahdanau attention works in a seq2seq translation model. Why was this such a big deal in 2014?
- Which RNN operations depend sequentially on time, and how does that affect hardware utilization?
- Why did the transformer architecture replace LSTMs for language modeling? Be specific about what changed.
- How does ViT turn an image into tokens, and what affects its comparison with CNNs?
- When would you consider a CNN, RNN, or state-space model instead of an attention-based model?
- How do selective state-space models such as Mamba combine recurrence with efficient sequence processing?
- Compare BatchNorm, LayerNorm, GroupNorm, and InstanceNorm using explicit tensor axes.
- You have built a bidirectional LSTM for named entity recognition. A colleague needs it for a real-time chat moderation system. What problems will they hit and how would you redesign?
- You need to deploy an image classification model to a fleet of mobile devices with strict latency (<30ms) and binary size (<10MB) budgets. Walk through your model selection, compression, and deployment pipeline.
- How does U-Net preserve spatial detail for segmentation, and what determines whether it is a suitable choice?
- Why do we scale the dot product in self-attention by √d_k?
- What is the purpose of multi-head attention?
- Why is positional encoding necessary in Transformers?
- How does causal (masked) attention differ from bidirectional attention?
- What is the computational complexity of self-attention, and how can it be reduced?
- How do pre-norm and post-norm differ, and what does that change about optimization?
- Walk through implementing scaled dot-product attention from scratch in PyTorch. What are the key steps and common pitfalls?
- You need to serve a 70B parameter transformer model with sub-200ms latency for the first token. Walk through your architecture.
- What are the differences between encoder-only (BERT), decoder-only (GPT), and encoder-decoder (T5) architectures? When would you choose each?
- How does FlashAttention reduce attention memory traffic, and when can that improve speed?
- How would you design a tokenizer for a multilingual LLM? Discuss vocabulary size, training data balance, and the fertility-efficiency tradeoff.
- How would you build an NLP pipeline for content moderation on a social media platform where users post in 15+ Indian languages, including code-mixed text and transliterated scripts?
- Why did the transformer architecture replace LSTMs for language modeling?
- How does ViT adapt a transformer to images, and what controls its comparison with CNNs?
- What is the difference between a bi-encoder and a cross-encoder for retrieval?
- How does HNSW search a vector index, and how do you tune its costs and recall?
- What is Matryoshka Representation Learning, and why is it useful?
- How does hybrid lexical and dense search work, and how would you decide whether it helps?
- You need to build a semantic search system over 500 million product descriptions. Walk through your architecture decisions.
- What are hard negatives, and why do they matter for embedding model training?
- Compare pgvector, Qdrant, and Pinecone for a production retrieval system.
- What is ColBERT and how does it differ from standard bi-encoder retrieval?
- How do you evaluate and improve the quality of an embedding model for your domain?
- Explain the tradeoff between pre-filtering and post-filtering in vector search.
- You're building an enterprise search product. How do you select an embedding model, and what factors matter beyond benchmark scores?
- Compare HNSW and IVF-PQ for a billion-vector search index. When would you choose each, and what are the memory and latency implications?
- How would you fine-tune an embedding model for a specific domain? Walk through the data collection, training, and evaluation process.
- How would you design a product search system for an e-commerce platform with 100M+ SKUs where users search in mixed Hindi-English (Hinglish) and expect results across multiple product categories?
- Design a real-time restaurant and dish recommendation system for a food delivery app that must personalize results for 50M+ users while respecting delivery radius and restaurant availability constraints.
- Design a large video recommendation system that retrieves candidates from a corpus of 1B+ videos.
- How would you build a semantic product search system for an e-commerce platform where users search with natural language queries like "birthday gift for 5 year old under $30"? Cover intent understanding, retrieval, and ranking.
- How do you choose the right evaluation metric for a classification problem?
- What is MMLU, and when does it become less useful for comparing capable models?
- What are the pros and cons of using LLM-as-judge for evaluation?
- How do you evaluate a RAG system end-to-end?
- What is benchmark contamination and how do you detect it?
- You're shipping an LLM-powered support assistant. How would you design its evaluation strategy?
- You're evaluating an LLM for a customer support chatbot. Design a custom evaluation suite: what metrics and test cases do you include?
- Design an automated evaluation pipeline that runs nightly against your production LLM. Cover metric selection, alerting, and regression detection.
- How would you detect if a model's benchmark scores are inflated by data contamination? Walk through 3 techniques.
- What does `lm-evaluation-harness` actually provide that you'd otherwise have to rebuild? Where does Inspect AI fit alongside it?
- Design an LLM evaluation harness that runs ~100 benchmarks across ~20 model versions with reproducibility guarantees. What does "reproducible" actually mean here, and what infrastructure does it take?
- Design an evaluation harness that gates model deployments: what components does it need, and how do you prevent a regression from reaching production?
- How do you detect whether an LLM's training data has been contaminated with benchmark questions? Why does this matter?
- How would you evaluate and monitor a credit scoring ML model in production where ground-truth labels (default/no-default) arrive 6-12 months after the prediction? What do you track in the interim?
- How would you design an offline evaluation framework for a recommendation system that reliably predicts online A/B test outcomes? Why do offline metrics often fail to correlate with online gains?
- Explain how LoRA works and why it dramatically reduces memory requirements.
- When would you choose full fine-tuning over LoRA?
- What is catastrophic forgetting and how do you mitigate it during fine-tuning?
- How much data do you need for fine-tuning, and what matters more: quantity or quality?
- Walk me through QLoRA. What makes it work on consumer hardware?
- How would you fine-tune one model to handle multiple tasks?
- You have a pre-trained 7B model and 10,000 domain-specific examples. Walk through your end-to-end fine-tuning pipeline, including data prep, hyperparameter choices, and evaluation.
- Explain how you would implement LoRA fine-tuning from scratch. What matrices do you target and why?
- Your fine-tuned model performs well on your eval set but fails on real user queries. Walk through your debugging process.
- When should you use RL-based training (GRPO/PPO) vs supervised fine-tuning for teaching a model to reason?
- How would you implement and train a model that supports both extended-reasoning and direct-answer modes?
- How are synthetic data and model distillation used in modern LLM training pipelines?
- When would you reach for RAG vs. fine-tuning vs. both? Walk through your decision framework with concrete examples.
- A user asks: "Should I fine-tune, use RAG, or just prompt-engineer?" Walk through your decision framework.
- What makes training data "high quality" for SFT? How do you curate a dataset when you have 100K examples but suspect most are mediocre?
- You need to serve multiple LoRA adapters (one per customer) on the same base model. How does multi-adapter serving work, and what are the engineering challenges?
- What are the key components of a RAG pipeline?
- How does chunking strategy affect RAG quality?
- Why might you add a reranker to a RAG pipeline?
- What is hybrid search and when should you use it?
- How do you evaluate a RAG system?
- What is HyDE and when would you use it?
- Design a production RAG system for a legal firm with 10 million documents, strict accuracy requirements, and multi-user access control. Walk through your architecture.
- Your RAG system is returning irrelevant documents for 30% of queries. Walk through your debugging process step by step.
- Compare and contrast naive RAG, advanced RAG (with reranking and query transformation), and modular RAG / agentic RAG. When would you use each?
- What is GraphRAG and when does it outperform standard vector-based RAG?
- Design a production RAG system over a company's internal documents (Confluence, Google Docs, Slack, code repos). Cover ingestion, chunking, retrieval, and generation with source citations.
- What metrics should you use to evaluate a RAG pipeline, and how do you diagnose whether failures come from retrieval or generation?
- When should you use fine-tuning vs RAG vs a combination of both? Walk through the decision framework with concrete examples.
- Design a customer support automation system for a fintech app that handles 50K daily tickets across payments, loans, insurance, and credit cards. Users describe issues in mixed Hindi-English, and responses must be factually correct with zero tolerance for hallucinated policy information.
- What is the ReAct pattern and why is it effective for agents?
- How do you handle context window limits in long-running agents?
- What are the key considerations when designing tools for an agent?
- When would you use a multi-agent system vs a single agent?
- How do you evaluate an agentic AI system?
- What safety guardrails should an agentic system have?
- Design an autonomous customer support agent that can access user accounts, process refunds, and escalate to humans. What guardrails do you need?
- What are the key differences between ReAct, Plan-and-Execute, and LLM Compiler agent architectures? When would you pick each?
- Your deployed agent is stuck in infinite tool-calling loops 5% of the time. How do you diagnose and fix this?
- What is the Model Context Protocol (MCP) and how does it change the way AI agents interact with tools and data sources?
- Design a multi-agent system for a complex software engineering task (e.g., implementing a feature from a Jira ticket). How do you orchestrate the agents, handle failures, and ensure quality?
- You're deploying an AI agent that can execute code, make API calls, and modify files in production. What guardrails do you implement?
- What is the Agent-to-Agent (A2A) protocol and how does it complement MCP?
- Where does scaffolding stop and the agent start? Walk through the seam between a ReAct loop and the harness wrapping it.
- An agent calls a tool and gets an error or unexpected output. How should the agent recover, and what are the failure modes of naive retry strategies?
- How would you design a multi-agent system where agents with different specializations collaborate on a complex task? What coordination patterns exist?
- How do you evaluate an agentic AI system? What metrics matter beyond task success rate?
- What is the “harness gap,” and how would you measure the harness contribution to agent performance?
- What belongs in an `AGENTS.md` file and why is it treated as part of the codebase, not the prompt?
- How do effective harnesses prevent an agent from declaring victory too early on a coding task?
- Why does "one giant instruction file" fail, and how would you structure agent state across long-running sessions instead?
- Design a coding-agent harness for a monorepo team: environment surface, state files, verification gates, observability, session-cleanup discipline.
- Design the verification layer of a coding-agent harness: what runs, in what order, and how do you decide "done"?
- Implement a minimal harness loop in pseudocode: read task, plan, patch, run verification, self-check against acceptance criteria, fail loudly if not met.
- Design an evaluation harness for an autonomous coding agent. What environments, metrics, and safety boundaries would you build?
- How do you manage state in a multi-turn tool-use session where the agent makes sequential API calls, file edits, and shell commands?
- Walk through the PPO clipped surrogate objective. How does it differ from TRPO's KL constraint?
- What's the difference between value-based, policy-based, and actor-critic methods?
- Explain GRPO and the tradeoff between a learned critic and group-relative advantages.
- Why is exploration hard? Walk through ε-greedy, entropy bonus, and UCB tradeoffs.
- DESIGN: Design an RL loop to teach an LLM to use tools (search, code interpreter, calculator) effectively.
- What does the discount factor γ mean intuitively, and how do you choose it?
- Explain the Bellman optimality equation and why dynamic programming approaches don't scale to deep RL.
- What is reward hacking? Give three concrete failure modes and explain mitigation strategies.
- Off-policy vs on-policy learning: what does it mean and when do you choose each?
- Walk through the standard RLHF pipeline from a pretrained LLM to a deployed chat model.
- Explain advantage estimation and GAE. Why does it matter for PPO?
- How does AlphaZero combine RL with search, and why is the combination so much stronger than either alone?
- Walk me through the three stages of the standard RLHF pipeline.
- What is the alignment tax and why does it occur?
- Compare DPO and PPO. When would you choose each?
- What is reward hacking and how do you mitigate it in RLHF?
- How would you design a preference data collection pipeline for RLHF?
- Explain Constitutional AI and how it differs from standard RLHF.
- You're building a preference dataset for aligning a coding assistant. How do you collect and quality-control the preference pairs?
- Walk through the key implementation differences between PPO and DPO training loops. What are the memory and compute implications?
- Your RLHF-trained model is generating overly verbose, sycophantic responses. What's happening and how do you fix it?
- Compare GRPO and PPO. Why can group-relative optimization be useful for reasoning-model training?
- Explain the difference between Process Reward Models and Outcome Reward Models. When would you use each?
- DeepSeek-R1-Zero demonstrated emergent reasoning from pure RL without any SFT. Walk through how this works and what it tells us about the nature of LLM reasoning.
- After a fresh round of safety-focused RLHF, your model starts refusing benign prompts ("I can't help with that") at 3× the previous rate. Walk through how you would diagnose and fix this.
- What are the most important limitations of RLHF as a scalable alignment technique, and how do current alternatives try to address them?
- What is GRPO (Group Relative Policy Optimization), and how does critic-free optimization differ from eliminating a learned reward model? Compare it to PPO and DPO.
- What is the scalable oversight problem in RLHF, and what approaches exist for aligning models that surpass human evaluator capability?
- What is test-time compute scaling and how does it differ from training-time scaling?
- Explain the difference between Process Reward Models (PRMs) and Outcome Reward Models (ORMs).
- How does the DeepSeek-R1-Zero experiment differ from the full R1 pipeline, and what does its no-SFT result establish?
- Compare Best-of-N sampling, beam search with PRM, and MCTS for test-time reasoning.
- You're deploying a reasoning model for a math tutoring app. How do you manage thinking budgets?
- Design a system that dynamically allocates test-time compute based on query difficulty.
- Walk through how process reward models are trained and used for step-level verification.
- Your reasoning model produces correct final answers but with flawed intermediate reasoning steps. Diagnose and fix.
- When should you use a thinking model vs a standard model? Discuss the cost-accuracy tradeoff.
- How can reasoning models be trained? Explain documented RLVR and process-supervision approaches and their limits.
- How should you allocate thinking budget for reasoning models in production? Discuss the cost-latency-accuracy tradeoff and adaptive strategies.
- What is the difference between outer alignment and inner alignment?
- How does prompt injection work, and what defenses exist?
- Explain Constitutional AI and how it differs from standard RLHF.
- What is the safety-helpfulness tradeoff, and how do you manage it?
- Design a red teaming pipeline for a customer-facing LLM chatbot. What would you test and how?
- What are sleeper agent models, and why are they concerning?
- How do you handle safety in a multi-modal LLM (text + images)?
- What is Llama Guard and how does it fit into a safety stack?
- A deployed chatbot is found to be generating biased responses against a specific demographic group. Walk through your incident response.
- Compare rule-based, classifier-based, and LLM-as-judge approaches to content moderation. When would you use each?
- What are the main categories of prompt injection attacks against LLM applications, and how do you build a defense-in-depth strategy?
- How do you design and execute a red-teaming exercise for a foundation model? What methodology produces actionable findings vs just a list of jailbreaks?
- Explain Constitutional AI (CAI) and how it differs from standard RLHF for safety training. What are its advantages and limitations?
- Design a content integrity system for a social media platform that detects misinformation, hate speech, and coordinated inauthentic behavior across text, images, and video at 500M+ daily posts. How do you balance speed, accuracy, and free expression?
- What is the difference between data, tensor, and pipeline parallelism?
- Why does mixed-precision training use ~16 bytes per parameter?
- What do ZeRO stages 1, 2, and 3 shard, and how does stage 3 relate to FSDP?
- How is the pipeline bubble calculated, and how do you shrink it?
- How would you decide how to split, say, a 70B model across 32 GPUs?
- What is the core difference between distributed training in PyTorch and JAX?
- What is the difference between PTQ and QAT?
- Why is weight-only quantization useful for LLM serving?
- How does GPTQ work at a high level?
- What is the outlier problem in quantization?
- What is the typical quality impact of 4-bit vs 8-bit quantization?
- How do you choose between GPTQ, AWQ, and GGUF?
- You need to deploy a 70B model on a single A100 80GB GPU. Walk through your quantization options and how you'd choose.
- Explain the GPTQ quantization algorithm step by step. How does it differ from simple round-to-nearest quantization?
- Your INT4-quantized model passes all benchmarks but users complain about worse quality on long-form generation. How do you investigate?
- What are FP4 and MXFP4 formats, and how do they differ from INT4 quantization?
- Compare post-training quantization (PTQ) and quantization-aware training (QAT) for deploying an LLM to production. When is the extra cost of QAT justified?
- Explain FP8 and FP4 quantization formats on modern accelerators. How do they differ from integer quantization, and what does hardware-native support mean for inference cost?
- What is KV cache quantization, and why is it becoming critical for long-context LLM serving? What precision formats work well?
- What is the KV cache and why does it matter for LLM serving?
- Compare MHA, MQA, GQA, and MLA for KV-cache size. Why is GQA useful, and what does MLA do differently?
- What happens when the KV cache runs out of memory mid-generation, and how do serving systems handle it?
- What is the difference between TTFT and TPOT, and what determines each?
- Compare static batching with continuous (in-flight) batching.
- How would you optimize a serving stack that is hitting low GPU utilization but acceptable latency?
- Explain PagedAttention and what problem it solves.
- How does speculative decoding work, and when does it help vs hurt?
- Design the inference infrastructure for a ChatGPT-like service handling 10,000 concurrent users. Cover load balancing, GPU allocation, and autoscaling.
- Your LLM API's p99 latency just spiked from 2s to 15s. Walk through your debugging process.
- How would you implement a simple continuous batching scheduler? Walk through the key data structures and the scheduling loop.
- What is disaggregated serving (prefill-decode separation), and when is it useful for LLM inference?
- Explain speculative decoding. How does it speed up LLM inference without changing the output distribution?
- You're serving a 70B model and the KV cache is your memory bottleneck for 100 concurrent requests at 8K context. How would you calculate the cache size and choose optimization strategies?
- Design an asynchronous batch inference API where clients POST a single item or a batch and later poll for results. Cover request schema, idempotency, queueing, worker scaling, accelerator utilization, and partial-batch failures.
- Design a multi-GPU serving system for a 70B+ parameter model. Cover parallelism strategies, memory management, and how you would meet a 200ms TTFT SLA at 100 QPS.
- Explain speculative decoding. How does it preserve the target sampling distribution, and what are the practical tradeoffs in production?
- What is continuous batching and how does it differ from static batching? Explain preemption policies and their impact on tail latency.
- Design the ML inference infrastructure for a food delivery platform that needs real-time ETA predictions, dynamic pricing, and restaurant ranking, all within 50ms at 100K concurrent users during peak dinner hours.
- Your ride-hailing platform's surge pricing model runs inference in 200ms but business wants it under 20ms. The model is a 50-feature gradient-boosted tree ensemble with 500 trees. How do you optimize?
- Design the ML serving infrastructure for a social media News Feed that must rank 500+ candidate posts per user per request at 100K+ QPS with a 50ms latency budget. How do you structure the ranking pipeline?
- How would you deploy an on-device ML model for real-time camera features (e.g., portrait mode, object detection) on a smartphone with a 10ms latency budget and 200MB memory constraint?
- What is a Mixture of Experts model and what problem does it solve?
- How does the gating/router mechanism work in modern MoE Transformers?
- What is expert collapse and how is it prevented?
- How does an MoE model compare to a dense model of equivalent quality in terms of training cost, inference latency, and memory footprint?
- Walk through why serving an MoE model efficiently is harder than serving a dense model.
- Why might an MoE use many narrow experts and an always-on shared component instead of fewer wide experts?
- How would you implement a simple top-2 gating mechanism for an MoE layer? Walk through the forward pass.
- Design serving for a hypothetical 671B-total, 37B-active MoE. How do memory accounting and parallelism constrain the layout?
- Your MoE model shows that 90% of tokens are routed to just 2 out of 8 experts. What's happening and how do you fix it?
- How would you compare different releases in an MoE model family without mixing incompatible architecture or training claims?
- You're tasked with serving a 100B+ MoE model with 64 experts efficiently. What are the unique challenges compared to serving a dense model, and how do you address them?
- What is expert collapse in MoE training and what mechanisms prevent it?
- How do you handle load balancing across experts during MoE inference at scale, and what happens when routing skew causes hot-spot experts?
- What causes expert collapse during MoE training, and how do you tune the capacity factor and auxiliary loss to prevent it without degrading model quality?
- Explain the standard architecture of a modern VLM like LLaVA.
- What is the difference between SigLIP and CLIP as vision encoders for VLMs?
- How do VLMs handle high-resolution images?
- What causes visual hallucination in VLMs, and how do you mitigate it?
- Compare the cross-attention approach (Flamingo) with the visual token prepending approach (LLaVA). What are the tradeoffs?
- How do you extend a VLM to handle video?
- You need to build a document understanding system that can read invoices, contracts, and forms. What VLM architecture considerations matter?
- What is visual grounding, and how do current VLMs support it?
- Explain the two-stage training pipeline for VLMs and why each stage matters.
- How do multimodal embedding models differ from generative VLMs, and when would you use each?
- Compare different strategies for coupling a visual encoder with an LLM in a VLM architecture. What are the tradeoffs between linear projection, cross-attention, and Q-Former approaches?
- How do you reduce hallucination in visual question answering with VLMs? What are the main causes and mitigation strategies?
- Design a document understanding pipeline that handles PDFs with mixed text, tables, figures, and handwriting. How would you architect this for an enterprise search product?
- What practical advantages can diffusion models offer over GANs for image generation?
- Explain mode collapse in GANs. Why does it happen and how do approaches like WGAN address it?
- What is the difference between DDPM and DDIM sampling? Why is DDIM faster?
- Walk through classifier-free guidance. Why does it work, and what does the guidance scale parameter actually control?
- What does flow matching change about generative-model training and sampling?
- Derive the score-based / SDE formulation of diffusion. How does the probability-flow ODE relate to the reverse SDE, and why does this connect diffusion to flow matching?
- Design an image-generation API like Midjourney that serves thousands of QPS. Walk through model choice, serving architecture, cost optimization, and latency targets.
- What is a world model, and how does it differ from an LLM?
- Separate rendering, dynamics simulation and planning. Where does each function fit in a partially observable agent loop?
- Compare Genie 3, Marble, NVIDIA Cosmos, and V-JEPA. What are the core differences in output format, real-time capability, and intended use?
- Why predict latent features rather than reconstruct pixels, and how does JEPA approach this tradeoff?
- Design a world model to train an end-to-end autonomous driving policy. Walk through architecture, data, evaluation, and the sim-to-real gap.
- You have to pick an output representation (Gaussian splats, video frames, latent embeddings, or NeRFs) for a robot manipulation simulator. Walk through your decision.
- What makes long-horizon consistency hard in real-time world models like Genie 3?
- How does action conditioning work in a video-prediction-style world model? What is hard about it?
- Why is there no single sufficient world-model metric, and how would you evaluate one?
- What does the NVIDIA Cosmos project provide, and what would you check before adapting a world-model toolkit?
- Suppose Waymo wants to use a world model to test a new lane-change policy against rare scenarios (kid running into the road, cyclist falling). Design the pipeline.
- What is the relationship between world models and Vision-Language-Action (VLA) models? Could you train a VLA without a world model?
- If a scene system such as Marble exports Gaussian splats and a collision mesh, why are both useful, and how would you keep them consistent?
- What is a Vision-Language-Action (VLA) model and how is it different from a vision-language model (VLM)?
- Compare RT-2, OpenVLA, and π0. What are the key architectural and training differences, and what does each one teach us?
- Walk through how flow matching produces continuous robot actions in π0. What tradeoffs does it make relative to discrete-token decoding?
- You're designing a VLA training pipeline for a humanoid that needs to do kitchen tasks (cooking prep, loading a dishwasher, wiping counters). Walk through your data strategy, architecture, and evaluation plan.
- What is Open X-Embodiment and why is it called the "ImageNet moment" for robotics?
- What does "cross-embodiment transfer" mean and why is it considered the holy grail for VLAs?
- Compare single-model VLAs (OpenVLA, π0) with dual-model "System A / System B" architectures (Helix-style). When would you pick each?
- What do published VLA demonstrations establish, and what remains difficult to validate reliably?
- Why is evaluation so hard for VLAs, and what would a good benchmark look like?
- How do world models compose with VLAs? Why does this combination matter?
- You join a humanoid robotics company that wants to ship a VLA to customers within 12 months. They have $5M for data collection and 100 robots in the field. How do you allocate the budget?
- What's the role of Gemini Robotics and Gemini Robotics On-Device in the VLA landscape?
- Why are open-source small VLAs like SmolVLA important even if they're not state-of-the-art?
- Fix Feature Engineering Bug
- Fix Broken RAG Pipeline
- Optimize Slow Inference Pipeline
- Build a Coding Agent Harness
- Implement Attention in PyTorch
- Build a Transformer Encoder Block
- Implement LoRA from Scratch
- Implement KV Cache for Efficient Decoding
- Implement Beam Search Decoding
- Implement Rotary Position Embeddings
- LoRA Fine-tune a Text Classifier
- Build an LLM Evaluation Pipeline
- Implement a DPO Training Step
- Post-Training Quantization to INT8
- Deploy a Model with FastAPI + Batching
- Build a Vector Search Index
- Design a GRPO Post-Training Loop
- Build YouTube's Home Feed
- Build Airbnb Search Ranking
- Build Stripe Radar
- Build TikTok's Content Moderation Pipeline
- Build Google's Ad Exchange Bidder
- Build an Autonomous SWE Agent (Devin-style)
- Build OpenRouter's LLM Gateway
- Build DoorDash's Feature Store
- Build Datadog's ML Observability
- Build Statsig's Experimentation Platform
- Build Claude's MCP Tool Server
- Build Perplexity's Answer Engine
- Build an LLM Eval Platform (OpenAI-Evals-style)
- Build Claude's Skills Library
- Build LangSmith: Prompt Ops
- Build a Multi-Agent Coding Swarm (Devin-style)
- Tell me about a time you convinced a senior engineer or manager to change technical direction.
- Describe a time you led a project without formal authority.
- Tell me about a time you disagreed with a teammate about a technical decision.
- Describe a time you worked with a cross-functional partner (PM, designer, infra) who was blocking your work.
- Walk me through the most technically complex ML project you've shipped.
- Tell me about a project that didn't ship. What happened?
- Tell me about a time your model's offline metrics looked great but production behavior was bad.
- Describe a tradeoff between model quality and latency / cost / interpretability you had to make.
- Tell me about a time you had to choose between fine-tuning a model and prompt engineering / RAG.
- Tell me about the most significant feedback you've received from a manager or peer.
- Describe a time you were wrong about something important and changed your mind.