pallavi-shekhar/ai-engineering-interview-questions-company-wise

AI Engineering Interview Questions Company Wise

AI Engineering Interview Questions Company Wise - Your Cheat Sheet For AI Engineering Interviews at Top AI Companies

> Real interview questions asked in AI Engineering interviews at 35 companies, organized company by company, with answers linked wherever we have them.

> These interview questions and answers are helpful for roles such as:

> - AI Engineer

- Gen AI Engineer

- LLM Engineer

- Agentic AI Engineer

- AI Agent Engineer

- Machine Learning Engineer

- Research Engineer

- Applied Scientist

- Forward Deployed Engineer

- AI Solutions Architect

- AI Platform Engineer

- Applied AI Engineer

- LLM Inference and Performance Engineer

- MLOps Engineer

- LLMOps Engineer

Table of Contents

• How to use this
• Common Questions Asked Across Companies
• LLM Internals and Architecture
• Inference, Serving and GPU Performance
• RAG and Retrieval
• Agents and Tool Use
• Fine-Tuning, Post-Training and Alignment
• Evaluation and Observability
• Safety, Security and Responsible AI
• Multimodal, Speech and Voice AI
• AI System Design
• Coding and Data Structures
• Frontier AI Labs
• Anthropic
• OpenAI
• Google DeepMind and Google AI
• Meta (Superintelligence Labs, FAIR, Llama)
• xAI
• Mistral AI
• Cohere
• DeepSeek
• Moonshot AI (Kimi)
• Zhipu AI (GLM)
• Alibaba (Qwen)
• Sarvam AI
• Big Tech AI Organizations
• Microsoft
• Amazon (AWS)
• Apple
• NVIDIA
• Tesla
• Consumer-Scale ML Companies (Uber, Netflix, LinkedIn, Airbnb, Pinterest, Spotify)
• AI Infrastructure and Platform Companies
• Databricks
• Groq
• Together AI
• Hugging Face
• Scale AI
• Perplexity
• AI-Native Product Companies
• Cursor (Anysphere)
• Cognition (Devin, Windsurf)
• Sierra
• Harvey
• Glean
• Character.AI
• ElevenLabs
• Abridge
• Figure AI
• Waymo
• Forward-Deployed and Enterprise AI
• Palantir

Prepared and maintained by Outcome School

AI and Machine Learning Program by Outcome School: AI and Machine Learning Program

Follow Outcome School

• YouTube
• X/Twitter
• LinkedIn
• GitHub

Note: We will keep updating this with new questions and answers.

> For topic-wise questions and answers, see AI Engineering Interview Questions and Answers.

---

How to use this

• Questions are compiled from publicly reported interview experiences. Nothing here is confidential. Interview loops change constantly and vary by team, level, and region, so treat each company section as a map of what that company cares about, not a script of what you will be asked.
• Start with Common Questions Asked Across Companies. These are the questions that recur across many companies. Each one is listed once, with the companies that ask it, so nothing is repeated in the company sections.
• Then go to your target companies. Each company section has the roles it covers, the interview loop as publicly reported, and the company-specific questions grouped by topic.
• Wherever we have an answer, it is linked right below the question. We will keep adding answers.

Common Questions Asked Across Companies

These questions come up in AI Engineering interviews at many companies. Each question is listed once here, with the companies where it (or a company-specific version of it) is asked. Work through these first.

LLM Internals and Architecture

• Explain scaled dot-product attention and why the 1/sqrt(d_k) scaling factor matters.
• Answer: Math behind √dₖ Scaling Factor in Attention and Math behind Attention - Q, K, and V
• What is the KV cache, and what are its memory implications at scale? Derive the formula.
• Asked at: OpenAI, xAI, Mistral AI, Amazon, Apple, NVIDIA, Together AI, Character.AI
• Answer: What is KV Cache in LLMs? and KV Cache Compression
• What are Multi-Query Attention (MQA) and Grouped-Query Attention (GQA), and what do they trade away?
• Asked at: Meta, Mistral AI
• Answer: Grouped Query Attention
• What is Multi-head Latent Attention (MLA) and why did DeepSeek introduce it?
• Asked at: DeepSeek, Moonshot AI
• Answer: KV Cache Compression
• Explain FlashAttention. It does not reduce FLOPs, so why is it faster?
• Asked at: Together AI
• Answer: Decoding Flash Attention in LLMs
• How does Byte Pair Encoding work, and what are its failure modes (numbers, code, non-Latin scripts)?
• Asked at: Alibaba, Sarvam AI, Hugging Face
• Answer: Byte Pair Encoding and Tokenization in Large Language Models (LLMs)
• What is positional encoding in transformers, and how has it evolved (sinusoidal → learned → RoPE → ALiBi)?
• Answer: Positional Embeddings in LLMs and Math Behind RoPE (Rotary Position Embedding)
• Explain RoPE and how position interpolation / YaRN extend context beyond the trained length.
• Asked at: Meta, Moonshot AI, Alibaba
• Answer: Math Behind RoPE (Rotary Position Embedding)
• What do the Chinchilla scaling laws say, and how do they differ from earlier scaling intuitions?
• Asked at: Anthropic
• What is a mixture-of-experts architecture and how does it scale capacity without scaling FLOPs?
• Asked at: Mistral AI, Cohere, DeepSeek, Moonshot AI, Zhipu AI, Alibaba
• Answer: Mixture of Experts Explained
• Explain the difference between pre-training, supervised fine-tuning and preference optimisation.
• Asked at: Meta, Scale AI
• Answer: Decoding InstructGPT and Reinforcement Learning from Human Feedback (RLHF)
• Compare greedy, beam search, top-k, top-p and temperature sampling. When does each fail?
• Asked at: Google DeepMind, Apple, Perplexity
• Answer: How does Temperature control LLM output? and How do Top-k and Top-p Sampling work?
• What is the lost-in-the-middle problem in long contexts and how do you address it?
• Asked at: Moonshot AI
• Answer: The Lost in the Middle Problem in LLMs
• Why is LayerNorm placed pre-block in modern transformers, and what is RMSNorm?
• Answer: RMSNorm (Root Mean Square Layer Normalization)
• Explain SwiGLU and why gated activations replaced ReLU/GELU in modern LLM MLP blocks.
• Asked at: Meta
• Answer: Feed-Forward Networks in LLMs
• Walk me through what happens, tensor by tensor, in one forward pass of a decoder-only transformer.
• Asked at: Anthropic
• Answer: Decoding Transformer Architecture

Inference, Serving and GPU Performance

• Explain the prefill and decode phases. Why is prefill compute-bound and decode memory-bandwidth-bound?
• Asked at: Moonshot AI, NVIDIA, Together AI
• Answer: Prefill vs Decode: LLM Inference Optimization
• What is continuous (in-flight) batching and why did it replace static batching?
• Asked at: Anthropic, xAI, Mistral AI, NVIDIA, Together AI
• Answer: Continuous Batching in LLMs
• How does PagedAttention work, and what problem of KV-cache fragmentation does it solve?
• Asked at: NVIDIA, Together AI
• Answer: Paged Attention in LLMs and How does vLLM work?
• What is speculative decoding? Why is output quality preserved, and when does it not help?
• Asked at: NVIDIA, Together AI
• Answer: Speculative Decoding
• Explain prefix caching / prompt caching. When should you use it, and what invalidates a cached prefix?
• Asked at: Moonshot AI, Character.AI
• Answer: How does Prompt Caching work?
• Compare FP16, BF16, FP8, INT8, INT4 and FP4 for serving. What breaks at each step down?
• Asked at: Mistral AI, Apple, NVIDIA, Together AI, Character.AI
• Answer: How does Model Quantization work?
• Compare tensor, pipeline, data, sequence and expert parallelism. When do you combine them?
• Asked at: Google DeepMind, Meta, Amazon, NVIDIA
• Estimate the GPU memory needed to serve a 70B model: weights, KV cache, activations, fragmentation.
• Asked at: NVIDIA
• Answer: What is KV Cache in LLMs? and Paged Attention in LLMs
• What are TTFT, TPOT, ITL and throughput, and how do they trade against each other?
• Asked at: Microsoft, Apple, Perplexity
• Answer: Prefill vs Decode: LLM Inference Optimization and The First-Token Latency Problem in LLMs
• Do the roofline maths: how many tokens/sec can one H100 produce for a 70B model at batch size 1?
• Asked at: NVIDIA, Together AI
• Answer: Prefill vs Decode: LLM Inference Optimization
• When would you choose vLLM vs SGLang vs TensorRT-LLM vs a custom stack?
• Asked at: NVIDIA, Together AI
• Answer: How does vLLM work?, How does SGLang work? and How does TensorRT-LLM work?
• How would you cut LLM serving cost by 10x? Enumerate every lever and rank them.
• Asked at: Microsoft, Amazon, NVIDIA, Cursor
• Answer: Explained in this video: LLM Inference Optimization and LLM Inference Optimization
• Your p99 latency doubled after a deploy with no model change. Walk through the diagnosis.
• Asked at: OpenAI, Amazon, Databricks, Perplexity
• What is chunked prefill, and why does it improve tail latency under mixed traffic?
• Explain disaggregated prefill/decode serving and when it pays for itself.
• Asked at: Moonshot AI, Groq
• Answer: Prefill-Decode Disaggregation in LLM Inference

RAG and Retrieval

• What chunking strategy would you use for a large technical documentation corpus, and why?
• Asked at: Glean
• Answer: Chunking Strategies for RAG
• How do you choose between a sparse retriever (BM25) and a dense retriever? When do you need both?
• Asked at: Microsoft, Perplexity, Glean
• Answer: How does Hybrid Search work?
• What is a reranker, when should you use one, and what does a cross-encoder cost you?
• Asked at: Cohere, Microsoft, Perplexity
• Answer: How does a Reranker work?
• How would you evaluate the quality of a RAG pipeline: retrieval and generation separately?
• Asked at: Cohere
• Answer: LLM Evaluation
• What is HyDE (hypothetical document embeddings) and when does it outperform standard dense retrieval?
• Answer: How does HyDE work in RAG?
• How does agentic RAG differ from standard RAG, and when is the extra complexity justified?
• Answer: Agentic RAG
• What causes semantic drift in embedding search and how do you detect it?
• Asked at: Cohere
• Design permission-aware retrieval: users must never see content they can't access in the source system.
• Asked at: Microsoft, Databricks, Glean, Palantir
• Compare HNSW, IVF-PQ and flat indexes. How do you pick, and what does recall@k cost in latency?
• Answer: How does Approximate Nearest Neighbor (ANN) search work? and How does a Vector Database work?
• How do you handle tables, figures and multi-column PDFs in a retrieval pipeline?
• How do you keep an index fresh when the underlying corpus changes continuously?
• Asked at: Perplexity, Cursor
• How do you attribute every claim in a generated answer to a specific retrieved span?
• Asked at: Perplexity, Harvey, Abridge

Agents and Tool Use

• Explain the ReAct pattern and what it solves over chain-of-thought alone.
• Answer: ReAct Agent and How does Chain-of-Thought (CoT) Prompting work?
• How do you handle tool-call errors, timeouts and retries in an agentic loop?
• Asked at: OpenAI, Cognition
• Answer: AI Agent Loop
• What is the difference between structured output and function calling?
• Asked at: Mistral AI, Apple
• Answer: How does Function Calling work in LLMs?
• What is MCP (Model Context Protocol) and how does it differ from traditional function calling?
• Asked at: Microsoft
• Answer: Explained in this video: AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, Quantization and What is MCP (Model Context Protocol)?
• How many tools is too many? How do you design tool schemas an LLM can actually use correctly?
• Asked at: Anthropic, Cognition
• Answer: Explained in this video: AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, Quantization
• How does multi-agent orchestration work, and when does it break down?
• Asked at: Cognition
• Answer: Multi-Agent Systems and AI Orchestration
• Design memory for a long-running agent: what do you store, where, and how do you retrieve it?
• Asked at: Anthropic
• Answer: AI Agent Memory
• How does an agent decide when to call a tool versus answer from its own knowledge?
• Answer: How does Function Calling work in LLMs?
• What makes an agent loop terminate correctly? How do you bound cost and steps?
• Answer: AI Agent Loop and Fix an infinite loop in an AI agent
• How do you make an agent's actions reversible, or at least auditable, in a production system?
• Asked at: Palantir
• Design human-in-the-loop approval for an agent that takes consequential actions.
• Asked at: OpenAI, Palantir
• Your agent drifts after a long run and confidently works on the wrong thing. Diagnose it.
• Asked at: Cognition

Fine-Tuning, Post-Training and Alignment

• Walk me through RLHF end to end: reward model, policy optimisation, KL penalty.
• Answer: Reinforcement Learning from Human Feedback (RLHF) and Proximal Policy Optimization (PPO)
• What is DPO and why did it displace PPO-based RLHF at many labs? When is online RL still better?
• Asked at: Hugging Face, Scale AI
• Answer: Direct Preference Optimization (DPO)
• Explain GRPO and why dropping the value network matters at scale.
• Asked at: DeepSeek
• Answer: Group Relative Policy Optimization (GRPO)
• Explain the LoRA decomposition mathematically. Why does it work, and how do you choose the rank r?
• Answer: LoRA - Low-Rank Adaptation of LLMs
• How does QLoRA achieve its memory reduction, and what are the quantization trade-offs?
• Asked at: Hugging Face
• Answer: Explained in this video: AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, Quantization and How does Model Quantization work?
• Compare LoRA, prefix tuning, prompt tuning and full fine-tuning. When would you choose each?
• Asked at: Sarvam AI, Apple
• Answer: How does fine-tuning work? and How does Prefix Tuning work?
• What is catastrophic forgetting and how do you mitigate it during fine-tuning?
• Asked at: Mistral AI
• Answer: Continual Learning in LLMs
• Prompting, RAG or fine-tuning: give me your decision framework with cost and latency attached.
• Asked at: OpenAI, Mistral AI, Cohere, Microsoft, Databricks, Glean
• Answer: Explained in this video: AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, Quantization
• Do the GPU memory maths for full fine-tuning a 7B model in bf16 with Adam. Now with LoRA.
• Asked at: Mistral AI, Hugging Face
• What is RLVR (RL with verifiable rewards) and where does it beat a learned reward model?
• Asked at: Zhipu AI, Alibaba, Sarvam AI, Scale AI
• Answer: Group Relative Policy Optimization (GRPO)
• Explain reward hacking in RLHF and how labs address it.
• Asked at: Scale AI
• Answer: Reinforcement Learning from Human Feedback (RLHF)
• What is distillation, and how do you build a strong small model from a large one?
• Asked at: Alibaba
• Answer: How does Knowledge Distillation work?

Evaluation and Observability

• Design an LLM-as-judge evaluation. What are its known biases and how do you correct for them?
• Asked at: Perplexity
• Answer: LLM as a Judge
• How do you build an eval set when there is no labelled ground truth and experts are expensive?
• Asked at: Cohere, Harvey
• How do you detect and measure hallucinations in a production RAG system?
• Asked at: Anthropic, OpenAI, Cursor
• Design the regression gate that decides whether a prompt or model change ships.
• Asked at: [