Practice bank
Every interview question, code lab, design problem, and behavioral framework: searchable, filterable, completion synced with the per-topic pages.
- State Bayes' theorem and explain each term.
- A medical test has 95% sensitivity and 95% specificity. Disease prevalence is 1%. If a patient tests positive, what is the probability they have the disease?
- Explain the relationship between L2 regularization and a Gaussian prior in Bayesian terms.
- Your fraud detection classifier shows 99% accuracy on the test set but catches almost zero fraudsters in production. What is happening and how do you fix it?
- How would you implement a Naive Bayes spam classifier from scratch? Walk through the key steps.
- What is calibration in ML model predictions, and why is it critical for safety-critical applications like autonomous driving? How do you measure and improve it?
- What is the bias-variance tradeoff?
- How do you diagnose underfitting vs overfitting, and what would you do about each?
- You have a model with high bias. Walk through 5 concrete things you would try to fix it, in order.
- Your model performs great on the test set but poorly in production. What could be happening beyond overfitting?
- What is double descent and why does it challenge classical bias-variance theory?
- You build a k-fold cross-validation pipeline for a churn model and get 88% AUC. In production it drops to 71%. What likely went wrong?
- When would you use leave-one-out cross-validation, and when would you avoid it?
- Walk through how you would design a cross-validation strategy for a daily demand-forecasting model.
- Explain L1 vs L2 regularization. Why does L1 produce sparsity but L2 does not?
- AdamW vs Adam with L2 regularization: what is the difference and when does it matter?
- Walk through how dropout works during training vs inference. What is inverted dropout?
- You are fine-tuning a 70B parameter foundation model on 5,000 domain-specific examples. What regularization strategies apply at this scale?
- Your fraud detection model has 99% accuracy on the test set. Your team is celebrating. Why are you skeptical?
- Why does ROC-AUC become misleading under extreme class imbalance, and what should you use instead?
- A model has Brier score 0.05 and AUC 0.92, but the business says "the 80% confident predictions are wrong half the time." What is happening and how do you fix it?
- Design the evaluation stack for a new web search ranking system. Walk through your offline metrics, your online metrics, and how the two connect.
- When would you use the normal equation vs gradient descent to fit a linear regression?
- Why do we use cross-entropy loss for logistic regression instead of mean squared error? What goes wrong if you use MSE?
- Why is logistic regression "naturally well-calibrated" while a random forest or a deep neural net is not? How would you fix calibration on a non-calibrated model?
- Why does Facebook / Google still use logistic regression for ads CTR prediction at scale in 2026, when deep models are available?
- You're building a credit-scoring model for a regulated lender. The data science team proposes XGBoost; risk and legal push back and want logistic regression. Walk through the tradeoffs and how you would mediate.
- Why can the RBF kernel represent an infinite-dimensional feature space without ever computing it? Walk through the mechanics of the kernel trick.
- You have an RBF SVM that overfits badly: train accuracy 99%, validation accuracy 65%. Walk through how you would diagnose and fix this. What roles do C and γ play?
- How does one-class SVM work for anomaly detection, and when would you choose it over Isolation Forest or an autoencoder?
- Walk me through XGBoost's objective function. Why does the second-order Taylor expansion matter?
- What problem does CatBoost's ordered boosting solve, and how does it differ from standard GBDT?
- Design a fraud detection system for a payments platform processing 10M transactions/day with sub-100ms decision SLA.
- You have a CTR model with billions of training rows, 100M+ unique user IDs and 10M+ unique ad IDs. Would you use XGBoost? Why or why not?
- Why did contextual embeddings replace static embeddings like Word2Vec?
- Explain how negative sampling works in Word2Vec and why it replaced the full softmax.
- What problem does FastText solve that Word2Vec cannot, and how?
- A data scientist wants to cluster 5M customer records to find segments for a marketing campaign. Walk through your approach.
- Why does ReLU help with vanishing gradients compared to sigmoid?
- AdamW vs Adam: what changed and why does it matter for transformers?
- BatchNorm vs LayerNorm vs RMSNorm: when do you use each, and why do LLMs use LayerNorm/RMSNorm?
- Walk through implementing backprop for a 2-layer MLP from scratch. No autograd.
- You are training a 7B model and the loss spikes / NaNs out after 50K steps. Walk me through your debug strategy.
- Explain Xavier vs He initialization. When do you use each, and why does it matter?
- Why is gradient clipping critical for transformer training, and should you clip by norm or by value?
- Walk through mixed-precision training. Why is BF16 preferred over FP16 for LLM pretraining?
- Compare cross-entropy, MSE, and focal loss. When would you choose each?
- Explain KL divergence and its asymmetry. Why does direction matter in practice?
- What is cosine LR schedule with warmup and why has it become the standard for LLM pretraining?
- Why is the combined softmax + cross-entropy gradient (p - y) numerically stable when computing them separately is not?
- Walk through the storyline from n-grams to attention. What problem did each step solve, and what new problem did it introduce?
- Why specifically did the LSTM forget gate solve the vanishing gradient problem that vanilla RNNs suffered from?
- You're asked to build a next-word predictor for a mobile keyboard with a 10MB memory budget. Walk through the architectural options across the n-gram → RNN → attention spectrum and pick one with reasoning.
- Explain the three core inductive biases of a convolutional layer: local receptive fields, parameter sharing, and translation equivariance. Why do they make CNNs so data-efficient compared to fully connected networks?
- Compute the receptive field of a 5-layer CNN with 3×3 kernels, stride 1, no dilation. Then redo it with stride 2 on layers 2 and 4. Then with dilation rates [1, 2, 4, 8, 16] and stride 1.
- Why do residual connections enable training 100+ layer networks? Explain at both the optimization and gradient-flow level, and describe the "degradation problem" they solved.
- Walk through what each LSTM gate does mathematically. Why does the LSTM architecture avoid the vanishing gradient problem that plagued vanilla RNNs?
- Compare LSTM and GRU. When would you choose one over the other?
- Explain BPTT (Backpropagation Through Time) and why gradient clipping is essential for training RNNs.
- Walk through how Bahdanau attention works in a seq2seq translation model. Why was this such a big deal in 2014?
- Why exactly do RNNs not parallelize over the time dimension during training, and why is this such a fatal weakness on modern hardware?
- Why did the transformer architecture replace LSTMs for language modeling? Be specific about what changed.
- Walk through how ViT adapted the transformer for images. Why did it eventually beat CNNs at scale?
- In 2026, when would you still pick a CNN over a ViT? And when would you pick an RNN/SSM over a Transformer?
- What is Mamba and why are people excited about state space models in 2025-2026?
- Compare BatchNorm, LayerNorm, GroupNorm, and InstanceNorm. When would you use each, and why has LayerNorm become dominant in modern architectures?
- You have built a bidirectional LSTM for named entity recognition. A colleague needs it for a real-time chat moderation system. What problems will they hit and how would you redesign?
- You need to deploy an image classification model to a fleet of mobile devices with strict latency (<30ms) and binary size (<10MB) budgets. Walk through your model selection, compression, and deployment pipeline.
- Explain U-Net and why it remains the default architecture for semantic segmentation, especially in medical imaging.
- Why do we scale the dot product in self-attention by √d_k?
- What is the purpose of multi-head attention?
- Why is positional encoding necessary in Transformers?
- How does causal (masked) attention differ from bidirectional attention?
- What is the computational complexity of self-attention, and how can it be reduced?
- Why has pre-norm become standard over post-norm?
- Walk through implementing scaled dot-product attention from scratch in PyTorch. What are the key steps and common pitfalls?
- You need to serve a 70B parameter transformer model with sub-200ms latency for the first token. Walk through your architecture.
- What are the differences between encoder-only (BERT), decoder-only (GPT), and encoder-decoder (T5) architectures? When would you choose each?
- Explain how FlashAttention achieves its speedup. What is IO-aware computation and why does it matter more than reducing FLOPs?
- How would you design a tokenizer for a multilingual LLM? Discuss vocabulary size, training data balance, and the fertility-efficiency tradeoff.
- How would you build an NLP pipeline for content moderation on a social media platform where users post in 15+ Indian languages, including code-mixed text and transliterated scripts?
- Why did the transformer architecture replace LSTMs for language modeling?
- Walk through how ViT adapted the transformer for images. Why did it eventually beat CNNs at scale?
- Why did diffusion models eclipse GANs for image generation by 2022?
- Explain mode collapse in GANs. Why does it happen and how do approaches like WGAN address it?
- What is the difference between DDPM and DDIM sampling? Why is DDIM faster?
- Walk through classifier-free guidance. Why does it work, and what does the guidance scale parameter actually control?
- Flow matching has largely replaced diffusion as the training objective for new image and video models in 2025+. Why?
- Derive the score-based / SDE formulation of diffusion. How does the probability-flow ODE relate to the reverse SDE, and why does this connect diffusion to flow matching?
- Design an image-generation API like Midjourney that serves thousands of QPS. Walk through model choice, serving architecture, cost optimization, and latency targets.
- What is a world model, and how does it differ from an LLM?
- Walk through Fei-Fei Li's three-function taxonomy of world models. Where does each function plug into the POMDP agent loop?
- Compare Genie 3, Marble, NVIDIA Cosmos, and V-JEPA. What are the core differences in output format, real-time capability, and intended use?
- Why does Yann LeCun argue that pixel-space world models like Genie are fundamentally wasteful, and how does JEPA address this?
- Design a world model to train an end-to-end autonomous driving policy. Walk through architecture, data, evaluation, and the sim-to-real gap.
- You have to pick an output representation (Gaussian splats, video frames, latent embeddings, or NeRFs) for a robot manipulation simulator. Walk through your decision.
- What makes long-horizon consistency hard in real-time world models like Genie 3?
- How does action conditioning work in a video-prediction-style world model? What is hard about it?
- Why is there "no BLEU for worlds"? How do practitioners evaluate world models in 2026?
- NVIDIA Cosmos is open infrastructure with 2M+ downloads. Why has it spread so fast, and what does it actually provide?
- Suppose Waymo wants to use a world model to test a new lane-change policy against rare scenarios (kid running into the road, cyclist falling). Design the pipeline.
- What is the relationship between world models and Vision-Language-Action (VLA) models? Could you train a VLA without a world model?
- Marble outputs Gaussian splats plus a collision mesh. Why both, and how are they generated together?
- What is the difference between a bi-encoder and a cross-encoder for retrieval?
- Explain HNSW and why it's the most popular ANN algorithm.
- What is Matryoshka Representation Learning, and why is it useful?
- How does hybrid search (dense + sparse) work, and when is it necessary?
- You need to build a semantic search system over 500 million product descriptions. Walk through your architecture decisions.
- What are hard negatives, and why do they matter for embedding model training?
- Compare pgvector, Qdrant, and Pinecone for a production retrieval system.
- What is ColBERT and how does it differ from standard bi-encoder retrieval?
- How do you evaluate and improve the quality of an embedding model for your domain?
- Explain the tradeoff between pre-filtering and post-filtering in vector search.
- You're building an enterprise search product. How do you select an embedding model, and what factors matter beyond benchmark scores?
- Compare HNSW and IVF-PQ for a billion-vector search index. When would you choose each, and what are the memory and latency implications?
- How would you fine-tune an embedding model for a specific domain? Walk through the data collection, training, and evaluation process.
- How would you design a product search system for an e-commerce platform with 100M+ SKUs where users search in mixed Hindi-English (Hinglish) and expect results across multiple product categories?
- Design a real-time restaurant and dish recommendation system for a food delivery app that must personalize results for 50M+ users while respecting delivery radius and restaurant availability constraints.
- Design a YouTube-scale video recommendation system that retrieves candidates from a corpus of 1B+ videos. How do you structure the candidate generation and ranking pipeline?
- How would you build a semantic product search system for an e-commerce platform where users search with natural language queries like "birthday gift for 5 year old under $30"? Cover intent understanding, retrieval, and ranking.
- What is a Mixture of Experts model and what problem does it solve?
- How does the gating/router mechanism work in modern MoE Transformers?
- What is expert collapse and how is it prevented?
- How does an MoE model compare to a dense model of equivalent quality in terms of training cost, inference latency, and memory footprint?
- Walk through why serving an MoE model efficiently is harder than serving a dense model.
- Why does DeepSeek-V3 use many small "fine-grained" experts plus a shared expert rather than the Mixtral-style few large experts?
- How would you implement a simple top-2 gating mechanism for an MoE layer? Walk through the forward pass.
- Design the serving infrastructure for a 671B parameter MoE model like DeepSeek-V3 (37B active). What parallelism strategies do you use?
- Your MoE model shows that 90% of tokens are routed to just 2 out of 8 experts. What's happening and how do you fix it?
- Walk through the Qwen 2.5/3 MoE architecture. What makes it one of the most competitive open-source MoE models?
- You're tasked with serving a 100B+ MoE model with 64 experts efficiently. What are the unique challenges compared to serving a dense model, and how do you address them?
- What is expert collapse in MoE training and what mechanisms prevent it?
- How do you handle load balancing across experts during MoE inference at scale, and what happens when routing skew causes hot-spot experts?
- What causes expert collapse during MoE training, and how do you tune the capacity factor and auxiliary loss to prevent it without degrading model quality?
- Explain the standard architecture of a modern VLM like LLaVA.
- What is the difference between SigLIP and CLIP as vision encoders for VLMs?
- How do VLMs handle high-resolution images?
- What causes visual hallucination in VLMs, and how do you mitigate it?
- Compare the cross-attention approach (Flamingo) with the visual token prepending approach (LLaVA). What are the tradeoffs?
- How do you extend a VLM to handle video?
- You need to build a document understanding system that can read invoices, contracts, and forms. What VLM architecture considerations matter?
- What is visual grounding, and how do current VLMs support it?
- Explain the two-stage training pipeline for VLMs and why each stage matters.
- How do multimodal embedding models differ from generative VLMs, and when would you use each?
- Compare different strategies for coupling a visual encoder with an LLM in a VLM architecture. What are the tradeoffs between linear projection, cross-attention, and Q-Former approaches?
- How do you reduce hallucination in visual question answering with VLMs? What are the main causes and mitigation strategies?
- Design a document understanding pipeline that handles PDFs with mixed text, tables, figures, and handwriting. How would you architect this for an enterprise search product?
- Explain how LoRA works and why it dramatically reduces memory requirements.
- When would you choose full fine-tuning over LoRA?
- What is catastrophic forgetting and how do you mitigate it during fine-tuning?
- How much data do you need for fine-tuning, and what matters more: quantity or quality?
- Walk me through QLoRA. What makes it work on consumer hardware?
- How would you fine-tune one model to handle multiple tasks?
- You have a pre-trained 7B model and 10,000 domain-specific examples. Walk through your end-to-end fine-tuning pipeline, including data prep, hyperparameter choices, and evaluation.
- Explain how you would implement LoRA fine-tuning from scratch. What matrices do you target and why?
- Your fine-tuned model performs well on your eval set but fails on real user queries. Walk through your debugging process.
- When should you use RL-based training (GRPO/PPO) vs supervised fine-tuning for teaching a model to reason?
- Qwen3 supports toggling between 'thinking' and 'non-thinking' modes. How would you implement and train a model with this capability?
- How are synthetic data and model distillation used in modern LLM training pipelines?
- When would you reach for RAG vs. fine-tuning vs. both? Walk through your decision framework with concrete examples.
- A user asks: "Should I fine-tune, use RAG, or just prompt-engineer?" Walk through your decision framework.
- What makes training data "high quality" for SFT? How do you curate a dataset when you have 100K examples but suspect most are mediocre?
- You need to serve multiple LoRA adapters (one per customer) on the same base model. How does multi-adapter serving work, and what are the engineering challenges?
- Walk through the PPO clipped surrogate objective: why does clipping replace TRPO's KL constraint?
- What's the difference between value-based, policy-based, and actor-critic methods?
- Explain GRPO and why it gained traction over PPO for LLM training in 2024 and 2025.
- Why is exploration hard? Walk through ε-greedy, entropy bonus, and UCB tradeoffs.
- DESIGN: Design an RL loop to teach an LLM to use tools (search, code interpreter, calculator) effectively.
- What does the discount factor γ mean intuitively, and how do you choose it?
- Explain the Bellman optimality equation and why dynamic programming approaches don't scale to deep RL.
- What is reward hacking? Give three real examples and explain the mitigation strategies.
- Off-policy vs on-policy learning: what does it mean and when do you choose each?
- Walk through the standard RLHF pipeline from a pretrained LLM to a deployed chat model.
- Explain advantage estimation and GAE. Why does it matter for PPO?
- How does AlphaZero combine RL with search, and why is the combination so much stronger than either alone?
- Walk me through the three stages of the standard RLHF pipeline.
- What is the alignment tax and why does it occur?
- Compare DPO and PPO. When would you choose each?
- What is reward hacking and how do you mitigate it in RLHF?
- How would you design a preference data collection pipeline for RLHF?
- Explain Constitutional AI and how it differs from standard RLHF.
- You're building a preference dataset for aligning a coding assistant. How do you collect and quality-control the preference pairs?
- Walk through the key implementation differences between PPO and DPO training loops. What are the memory and compute implications?
- Your RLHF-trained model is generating overly verbose, sycophantic responses. What's happening and how do you fix it?
- Compare GRPO and PPO. Why has GRPO become the dominant RL algorithm for training reasoning models?
- Explain the difference between Process Reward Models and Outcome Reward Models. When would you use each?
- DeepSeek-R1-Zero demonstrated emergent reasoning from pure RL without any SFT. Walk through how this works and what it tells us about the nature of LLM reasoning.
- After a fresh round of safety-focused RLHF, your model starts refusing benign prompts ("I can't help with that") at 3× the previous rate. Walk through how you would diagnose and fix this.
- What are the most important limitations of RLHF as a scalable alignment technique, and how do current alternatives try to address them?
- What is GRPO (Group Relative Policy Optimization) and how does it enable reward-free alignment? Compare it to PPO and DPO.
- What is the scalable oversight problem in RLHF, and what approaches exist for aligning models that surpass human evaluator capability?
- What are the key components of a RAG pipeline?
- How does chunking strategy affect RAG quality?
- Why might you add a reranker to a RAG pipeline?
- What is hybrid search and when should you use it?
- How do you evaluate a RAG system?
- What is HyDE and when would you use it?
- Design a production RAG system for a legal firm with 10 million documents, strict accuracy requirements, and multi-user access control. Walk through your architecture.
- Your RAG system is returning irrelevant documents for 30% of queries. Walk through your debugging process step by step.
- Compare and contrast naive RAG, advanced RAG (with reranking and query transformation), and modular RAG / agentic RAG. When would you use each?
- What is GraphRAG and when does it outperform standard vector-based RAG?
- Design a production RAG system over a company's internal documents (Confluence, Google Docs, Slack, code repos). Cover ingestion, chunking, retrieval, and generation with source citations.
- What metrics should you use to evaluate a RAG pipeline, and how do you diagnose whether failures come from retrieval or generation?
- When should you use fine-tuning vs RAG vs a combination of both? Walk through the decision framework with concrete examples.
- Design a customer support automation system for a fintech app that handles 50K daily tickets across payments, loans, insurance, and credit cards. Users describe issues in mixed Hindi-English, and responses must be factually correct with zero tolerance for hallucinated policy information.
- What is the ReAct pattern and why is it effective for agents?
- How do you handle context window limits in long-running agents?
- What are the key considerations when designing tools for an agent?
- When would you use a multi-agent system vs a single agent?
- How do you evaluate an agentic AI system?
- What safety guardrails should an agentic system have?
- Design an autonomous customer support agent that can access user accounts, process refunds, and escalate to humans. What guardrails do you need?
- What are the key differences between ReAct, Plan-and-Execute, and LLM Compiler agent architectures? When would you pick each?
- Your deployed agent is stuck in infinite tool-calling loops 5% of the time. How do you diagnose and fix this?
- What is the Model Context Protocol (MCP) and how does it change the way AI agents interact with tools and data sources?
- Design a multi-agent system for a complex software engineering task (e.g., implementing a feature from a Jira ticket). How do you orchestrate the agents, handle failures, and ensure quality?
- You're deploying an AI agent that can execute code, make API calls, and modify files in production. What guardrails do you implement?
- What is the Agent-to-Agent (A2A) protocol and how does it complement MCP?
- Where does scaffolding stop and the agent start? Walk through the seam between a ReAct loop and the harness wrapping it.
- An agent calls a tool and gets an error or unexpected output. How should the agent recover, and what are the failure modes of naive retry strategies?
- How would you design a multi-agent system where agents with different specializations collaborate on a complex task? What coordination patterns exist?
- How do you evaluate an agentic AI system? What metrics matter beyond task success rate?
- What is a Vision-Language-Action (VLA) model and how is it different from a vision-language model (VLM)?
- Compare RT-2, OpenVLA, and π0. What are the key architectural and training differences, and what does each one teach us?
- Walk through how flow matching is used to produce continuous robot actions in π0, and why it beats discrete-token decoding for dexterous manipulation.
- You're designing a VLA training pipeline for a humanoid that needs to do kitchen tasks (cooking prep, loading a dishwasher, wiping counters). Walk through your data strategy, architecture, and evaluation plan.
- What is Open X-Embodiment and why is it called the "ImageNet moment" for robotics?
- What does "cross-embodiment transfer" mean and why is it considered the holy grail for VLAs?
- Compare single-model VLAs (OpenVLA, π0) with dual-model "System A / System B" architectures (Helix-style). When would you pick each?
- What real-world tasks have VLAs reliably solved by 2026, and what's still hard?
- Why is evaluation so hard for VLAs, and what would a good benchmark look like?
- How do world models compose with VLAs? Why does this combination matter?
- You join a humanoid robotics company that wants to ship a VLA to customers within 12 months. They have $5M for data collection and 100 robots in the field. How do you allocate the budget?
- What's the role of Gemini Robotics and Gemini Robotics On-Device in the VLA landscape?
- Why are open-source small VLAs like SmolVLA important even if they're not state-of-the-art?
- What is test-time compute scaling and how does it differ from training-time scaling?
- Explain the difference between Process Reward Models (PRMs) and Outcome Reward Models (ORMs).
- How does DeepSeek-R1 achieve reasoning without supervised fine-tuning?
- Compare Best-of-N sampling, beam search with PRM, and MCTS for test-time reasoning.
- You're deploying a reasoning model for a math tutoring app. How do you manage thinking budgets?
- Design a system that dynamically allocates test-time compute based on query difficulty.
- Walk through how process reward models are trained and used for step-level verification.
- Your reasoning model produces correct final answers but with flawed intermediate reasoning steps. Diagnose and fix.
- When should you use a thinking model vs a standard model? Discuss the cost-accuracy tradeoff.
- How are reasoning models like o1 and DeepSeek-R1 trained? Explain RLVR (RL with Verifiable Rewards) and process reward models.
- How should you allocate thinking budget for reasoning models in production? Discuss the cost-latency-accuracy tradeoff and adaptive strategies.
- What is the difference between outer alignment and inner alignment?
- How does prompt injection work, and what defenses exist?
- Explain Constitutional AI and how it differs from standard RLHF.
- What is the safety-helpfulness tradeoff, and how do you manage it?
- Design a red teaming pipeline for a customer-facing LLM chatbot. What would you test and how?
- What are sleeper agent models, and why are they concerning?
- How do you handle safety in a multi-modal LLM (text + images)?
- What is Llama Guard and how does it fit into a safety stack?
- A deployed chatbot is found to be generating biased responses against a specific demographic group. Walk through your incident response.
- Compare rule-based, classifier-based, and LLM-as-judge approaches to content moderation. When would you use each?
- What are the main categories of prompt injection attacks against LLM applications, and how do you build a defense-in-depth strategy?
- How do you design and execute a red-teaming exercise for a foundation model? What methodology produces actionable findings vs just a list of jailbreaks?
- Explain Constitutional AI (CAI) and how it differs from standard RLHF for safety training. What are its advantages and limitations?
- Design a content integrity system for a social media platform that detects misinformation, hate speech, and coordinated inauthentic behavior across text, images, and video at 500M+ daily posts. How do you balance speed, accuracy, and free expression?
- What is the difference between data, tensor, and pipeline parallelism?
- Why does mixed-precision training use ~16 bytes per parameter?
- What do ZeRO stages 1, 2, and 3 shard, and how does stage 3 relate to FSDP?
- How is the pipeline bubble calculated, and how do you shrink it?
- How would you decide how to split, say, a 70B model across 32 GPUs?
- What is the core difference between distributed training in PyTorch and JAX?
- What is the difference between PTQ and QAT?
- Why is weight-only quantization the standard for LLMs?
- How does GPTQ work at a high level?
- What is the outlier problem in quantization?
- What is the typical quality impact of 4-bit vs 8-bit quantization?
- How do you choose between GPTQ, AWQ, and GGUF?
- You need to deploy a 70B model on a single A100 80GB GPU. Walk through your quantization options and how you'd choose.
- Explain the GPTQ quantization algorithm step by step. How does it differ from simple round-to-nearest quantization?
- Your INT4-quantized model passes all benchmarks but users complain about worse quality on long-form generation. How do you investigate?
- What are FP4 and MXFP4 formats, and how do they differ from INT4 quantization?
- Compare post-training quantization (PTQ) and quantization-aware training (QAT) for deploying an LLM to production. When is the extra cost of QAT justified?
- Explain FP8 and FP4 quantization formats on modern hardware (H100, Blackwell, TPU). How do they differ from integer quantization, and what does hardware-native support mean for inference cost?
- What is KV cache quantization, and why is it becoming critical for long-context LLM serving? What precision formats work well?
- What is the KV cache and why does it matter for LLM serving?
- What is the difference between TTFT and TPOT, and what determines each?
- Compare static batching with continuous (in-flight) batching.
- How would you optimize a serving stack that is hitting low GPU utilization but acceptable latency?
- Explain PagedAttention and what problem it solves.
- How does speculative decoding work, and when does it help vs hurt?
- Design the inference infrastructure for a ChatGPT-like service handling 10,000 concurrent users. Cover load balancing, GPU allocation, and autoscaling.
- Your LLM API's p99 latency just spiked from 2s to 15s. Walk through your debugging process.
- How would you implement a simple continuous batching scheduler? Walk through the key data structures and the scheduling loop.
- What is disaggregated serving (prefill-decode separation) and why is it becoming the standard architecture for LLM inference?
- Explain speculative decoding. How does it speed up LLM inference without changing the output distribution?
- You're serving a 70B model and the KV cache is your memory bottleneck: 100 concurrent users at 8K context consume 130GB of KV cache alone. Walk through the optimization strategies.
- Design an asynchronous batch inference API where clients POST a single item or a batch and later poll for results. Cover request schema, idempotency, queueing, worker scaling, accelerator utilization, and partial-batch failures.
- Design a multi-GPU serving system for a 70B+ parameter model. Cover parallelism strategies, memory management, and how you would meet a 200ms TTFT SLA at 100 QPS.
- Explain speculative decoding. How does it achieve speedup without changing model outputs, and what are the practical tradeoffs in production?
- What is continuous batching and how does it differ from static batching? Explain preemption policies and their impact on tail latency.
- Design the ML inference infrastructure for a food delivery platform that needs real-time ETA predictions, dynamic pricing, and restaurant ranking, all within 50ms at 100K concurrent users during peak dinner hours.
- Your ride-hailing platform's surge pricing model runs inference in 200ms but business wants it under 20ms. The model is a 50-feature gradient-boosted tree ensemble with 500 trees. How do you optimize?
- Design the ML serving infrastructure for a social media News Feed that must rank 500+ candidate posts per user per request at 100K+ QPS with a 50ms latency budget. How do you structure the ranking pipeline?
- How would you deploy an on-device ML model for real-time camera features (e.g., portrait mode, object detection) on a smartphone with a 10ms latency budget and 200MB memory constraint?
- How do you choose the right evaluation metric for a classification problem?
- What is MMLU and why is it less useful for frontier models in 2026?
- What are the pros and cons of using LLM-as-judge for evaluation?
- How do you evaluate a RAG system end-to-end?
- What is benchmark contamination and how do you detect it?
- You're shipping an LLM-powered support assistant. How would you design its evaluation strategy?
- You're evaluating an LLM for a customer support chatbot. Design a custom evaluation suite: what metrics and test cases do you include?
- Design an automated evaluation pipeline that runs nightly against your production LLM. Cover metric selection, alerting, and regression detection.
- How would you detect if a model's benchmark scores are inflated by data contamination? Walk through 3 techniques.
- What does `lm-evaluation-harness` actually provide that you'd otherwise have to rebuild? Where does Inspect AI fit alongside it?
- Design an LLM evaluation harness that runs ~100 benchmarks across ~20 model versions with reproducibility guarantees. What does "reproducible" actually mean here, and what infrastructure does it take?
- Design an evaluation harness that gates model deployments: what components does it need, and how do you prevent a regression from reaching production?
- How do you detect whether an LLM's training data has been contaminated with benchmark questions? Why does this matter?
- How would you evaluate and monitor a credit scoring ML model in production where ground-truth labels (default/no-default) arrive 6-12 months after the prediction? What do you track in the interim?
- How would you design an offline evaluation framework for a recommendation system that reliably predicts online A/B test outcomes? Why do offline metrics often fail to correlate with online gains?
- What is the "harness gap" and why has it become a first-class engineering discipline in 2025-2026?
- What belongs in an `AGENTS.md` file and why is it treated as part of the codebase, not the prompt?
- How do effective harnesses prevent an agent from declaring victory too early on a coding task?
- Why does "one giant instruction file" fail, and how would you structure agent state across long-running sessions instead?
- Design a coding-agent harness for a monorepo team: environment surface, state files, verification gates, observability, session-cleanup discipline.
- Design the verification layer of a coding-agent harness: what runs, in what order, and how do you decide "done"?
- Implement a minimal harness loop in pseudocode: read task, plan, patch, run verification, self-check against acceptance criteria, fail loudly if not met.
- Design an evaluation harness for an autonomous coding agent. What environments, metrics, and safety boundaries would you build?
- How do you manage state in a multi-turn tool-use session where the agent makes sequential API calls, file edits, and shell commands?
- Fix Feature Engineering Bug
- Fix Broken RAG Pipeline
- Optimize Slow Inference Pipeline
- Build a Coding Agent Harness
- Implement Scaled Dot-Product Attention
- Build a Transformer Encoder Block
- Implement LoRA from Scratch
- Implement KV Cache for Efficient Decoding
- Implement Beam Search Decoding
- Implement Rotary Position Embeddings
- LoRA Fine-tune a Text Classifier
- Build an LLM Evaluation Pipeline
- Implement a DPO Training Step
- Post-Training Quantization to INT8
- Deploy a Model with FastAPI + Batching
- Build a Vector Search Index
- Build YouTube's Home Feed
- Build Airbnb Search Ranking
- Build Stripe Radar
- Build TikTok's Content Moderation Pipeline
- Build Google's Ad Exchange Bidder
- Build an Autonomous SWE Agent (Devin-style)
- Build OpenRouter's LLM Gateway
- Build DoorDash's Feature Store
- Build Datadog's ML Observability
- Build Statsig's Experimentation Platform
- Build Claude's MCP Tool Server
- Build Perplexity's Answer Engine
- Build an LLM Eval Platform (OpenAI-Evals-style)
- Build Claude's Skills Library
- Build LangSmith: Prompt Ops
- Build a Multi-Agent Coding Swarm (Devin-style)
- Tell me about a time you convinced a senior engineer or manager to change technical direction.
- Describe a time you led a project without formal authority.
- Tell me about a time you disagreed with a teammate about a technical decision.
- Describe a time you worked with a cross-functional partner (PM, designer, infra) who was blocking your work.
- Walk me through the most technically complex ML project you've shipped.
- Tell me about a project that didn't ship. What happened?
- Tell me about a time your model's offline metrics looked great but production behavior was bad.
- Describe a tradeoff between model quality and latency / cost / interpretability you had to make.
- Tell me about a time you had to choose between fine-tuning a model and prompt engineering / RAG.
- Tell me about the most significant feedback you've received from a manager or peer.
- Describe a time you were wrong about something important and changed your mind.