OpenAI's GPT-4 technical report, a large multimodal model accepting both text and image inputs that achieves human-level performance on a wide range of professional and academic benchmarks, marking a step change in LLM capability.
Key Idea
GPT-4 is a large multimodal model (accepts text and images, outputs text) that significantly advances the state of the art on academic and professional exams. Notably, the technical report deliberately withholds architectural and training details (model size, dataset, hardware) citing competitive and safety concerns, instead emphasizing capabilities, evaluation, and safety methodology.
Capabilities
- Multimodal input: Accepts interleaved text and images; outputs text only (image generation is handled by DALL·E, not GPT-4)
- Professional exams: Passes a simulated bar exam in the top 10% of test takers (vs GPT-3.5's bottom 10%); strong performance on LSAT, SAT, GRE, AP exams, and medical licensing
- MMLU: Scores 86.4% in English, vastly outperforming GPT-3.5 (~70%); maintains strong performance across 26 translated languages
- Coding: Significant improvements on HumanEval and other coding benchmarks
- Strong gains on reasoning-heavy tasks, while still exhibiting hallucinations and reasoning errors
Predictable Scaling
- OpenAI built infrastructure to predict GPT-4's final loss from smaller-scale runs (compute reduced by 1000-10000×)
- Demonstrates that loss curves and even certain capability metrics (like HumanEval pass rate) can be extrapolated reliably, enabling more efficient experimentation
- A practical answer to the question: "how do we know if this expensive training run will be worth it?"
Alignment & Safety
- RLHF pipeline similar to InstructGPT/ChatGPT, with significant scaling of human feedback collection
- Rule-based reward models (RBRMs): zero-shot GPT-4 classifiers used as additional reward signals during RLHF to steer the model toward refusing harmful requests
- System card approach: extensive evaluation of risks (hallucination, bias, disinformation, weapon synthesis, cybersecurity, emergent behaviors) with external red-teaming over 6+ months
- Reported residual issues: hallucinations, overconfidence, refusal calibration, and limited self-awareness of its own limitations
Why It Matters
- Set a new frontier benchmark that subsequent models (Claude, Gemini, Llama 3) explicitly target
- Multimodality at scale: opened the door to vision-language applications (document understanding, accessibility, visual reasoning)
- The "we won't disclose architecture" precedent influenced how frontier labs communicate going forward
- Predictable scaling validated the scaling-laws-driven approach to LLM development
Key Takeaways for Interviews
- GPT-4 = multimodal (text + image input), text output, frontier capability on exams and MMLU
- The paper is famously light on technical details, be ready to discuss why labs do this (safety, competitive, commercial)
- Predictable scaling is the meta-contribution worth knowing, extrapolating final loss from small runs
- RBRMs (GPT-4 itself as a reward signal) are an early instance of AI-feedback in alignment, presaging Constitutional AI and RLAIF
- The system-card methodology is now standard for frontier model releases