LLM Company Anatomy
21-slot reference atlas + provisional MZN map

• What is the actual scaling law for synthetic data quality vs. quantity? a synthetic-heavy small frontier model demonstrated existence proof but not optimal mix.
• Is there a multilingual scaling law analogous to Chinchilla? Adding 100 languages vs. 10 with same compute — no published frontier-scale result.
• How much of frontier model capability comes from data quality vs. quantity vs. mix curriculum? a leading open-weights model paper hints curriculum matters; no isolated study at scale.
• Copyright-clean training: can a frontier model be trained on only permissively-licensed data without significant capability loss? No public attempt at frontier scale.

Reference analyst note. Quality > quantity is now consensus, but the field has overcorrected — most labs underweight diversity in chase of curated quality. The 'textbook quality' direction (a synthetic-heavy small frontier model) is a local optimum, not a global one. Frontier 2026-2027 will rebalance toward curated-but-diverse, with synthetic data filling specific holes (math reasoning, agent traces) not as bulk replacement.

• Tokenizer-free architectures (MambaByte, MEGABYTE): why have they not matched BPE at frontier scale despite theoretical advantages? Compute hypothesis vs. fundamental limit unclear.
• Is there an optimal vocab size for a given (model size, data mix, target language portfolio)? Current choices are heuristic.
• Cross-lingual transfer in shared-vocab tokenizers: how much capability is shared vs. language-isolated? Limited mechanistic understanding.
• Multimodal tokenization: image tokens at 256 vs. 1024 per image — what is the actual quality-vs-cost frontier? AnyRes (LLaVA-NeXT) provides one data point, not a curve.

Reference analyst note. Tokenizer choice is a permanent commitment that constrains everything downstream. Frontier labs underinvest here — most use SentencePiece-defaults trained on subset of data. The next frontier capability gain may come from rethinking tokenization (entropy-aware dynamic tokenization, byte-level with efficient training). Anyone aiming for genuine multilingual frontier should treat tokenizer as a first-class capability investment.

• Is there a Pareto-better attention than MLA for long context? Several research efforts (Differential Attention, Lightning Attention) but no frontier context yet.
• Why does sliding window + global hybrid (a small open-weights model 2) underperform pure full-attention at frontier scale despite theoretical advantages? Empirical observation, not understood.
• Scaling laws for active parameters in MoE: if active=37B in an open-weights frontier model (V3 class) matches dense 70B-100B-class, what is the actual mapping? No published Chinchilla-equivalent for MoE.
• Reasoning models: does the architecture that's best for non-reasoning training remain optimal under RL post-training? o1/R1 suggest yes; theoretical reason absent.
• a state-space frontier architecture/State-Space Models at frontier: 2024 demonstrated competitive at 7-13B. Why has no lab pushed to 70B+ for serious comparison? Compute economics or architectural ceiling?

Reference analyst note. Dense architecture is dead at frontier scale by end of 2026. A leading open-weights flagship model is likely the last frontier-tier dense model. Either MoE (an open-weights frontier provider lineage, fine-grained 200+ experts) or new sparse paradigms wins. Architecture innovation is decoupling from scaling — RL post-training resets 'capability per parameter' such that smaller models with better post-training match much larger pre-train-only models. The bottleneck is shifting from architecture-quality to RL-environment-quality.

• Optimal LR schedule shape: cosine vs WSD vs constant-then-decay — which wins at 10T+ token scale? No frontier ablation published.
• Distributed Shampoo vs AdamW at frontier: a constitutional-methods frontier lab reportedly uses Shampoo; no public head-to-head exists at >100B scale.
• Training stability: are loss spikes random hardware artifacts, deterministic numerical issues, or signal of optimization pathology? Frontier labs disagree.
• Annealing phase impact: a leading open-weights model reports gains from final annealing; isolated effect vs. confound with high-quality data? Unclear.
• Cross-architecture parallelism transfer: knowledge of how to parallelize dense → MoE lossy transfer (expert parallelism is novel). An open-weights frontier provider had to develop new techniques.

Reference analyst note. an open-weights frontier model (V3 class)'s $5.6M-equivalent demonstrated the field has been overspending by 5-10×. The next 2 years will see massive efficiency gains as algorithmic improvements (FP8, fine-grained MoE, better parallelism, better data) compound. Frontier 'training compute' as the dominant moat is collapsing. The new moat is post-training infrastructure, RL environment quality, and inference-time compute scaling. Anyone with 1K H100s can now produce competitive models — the bottleneck has moved upstream of pre-training to data and downstream to RL.

• Is there a near-term alternative to a leading accelerator vendor hardware lock-in for training? AMD MI300X, a wafer-scale accelerator vendor CS-3, a multimodal frontier lab a custom-silicon accelerator competitive; software ecosystem gap remains the gating factor.
• Confidential compute (a leading accelerator vendor CC, a hyperscaler platform Nitro for GPU): production-ready or theatre? a constitutional-methods frontier lab uses a hyperscaler platform Nitro for third AI Safety Level-relevant workloads; performance overhead poorly characterized publicly.
• Optimal cluster size: when does adding GPUs hurt training (failure rate × MFU degradation)? Reported sweet spots vary 16K-32K.
• Power constraints will dominate by 2027-2028: cluster size limited not by capital but by available 100-500MW datacenter sites. Geographic distribution implications unclear.

Reference analyst note. Compute infrastructure is becoming a real estate / power infrastructure business as much as a hardware business. A synthetic-data-focused lab signing 20-year nuclear PPA with Three Mile Island, xAI building gas turbines on-site at Memphis, Stargate's $500B announcement — these reflect that the actual frontier constraint by 2027 is gigawatt-class power, not GPU supply. National strategic positioning of compute (US export controls on H800 to China, EU sovereign cloud requirements) is now first-order policy. Anyone serious about frontier needs to think 5+ years ahead about power and land, not just GPU procurement.

• What is the marginal value curve of SFT data? After ~100K well-curated, does the curve flatten or continue rising slowly?
• Synthetic vs human SFT data: where exactly do they diverge? Anecdotally synthetic struggles with creative tasks, edge cases — no rigorous study.
• SFT mixing ratios for multi-skill (chat + code + math + tool-use): no published ablation studies at scale.
• Does SFT actually teach new capability or just elicit / format pre-trained capability? Evidence (LIMA, Magpie) suggests mostly elicitation; deep SFT studies absent.

Reference analyst note. SFT is dramatically underrated and over-tuned. Most labs spend too much on SFT data scale (millions of examples) and not enough on quality + diversity. The optimal frontier SFT corpus is probably 100K-500K examples curated to within an inch of their lives. SFT-then-RL is the path; trying to push everything into SFT (Tulu approach) hits diminishing returns visible in current open community.

• Is RLHF (PPO-based) actually better than DPO at frontier scale? Open community converged on DPO; closed labs (a leading frontier lab, possibly a constitutional-methods frontier lab) retain PPO. No public head-to-head at 70B+ scale.
• Reward model scaling: does a 70B RM provide meaningfully better signal than 13B? Limited public ablation.
• Process Reward Models beyond math: PRMs work in math (verifiable steps); do they work in code, reasoning, writing? Active but unclear research area.
• RLVR generalization: an open-weights reasoning model trained on math/code generalized to other reasoning domains. Why? Mechanistic understanding absent.
• Constitutional methods: how much of its quality comes from the constitution document quality vs the RL-from-AI-Feedback (RLAIF) process? a constitutional-methods frontier lab's constitution is unusually detailed; lower-effort constitutions may not transfer.

Reference analyst note. RLHF as a method is mostly cargo-culted. The actual win at frontier comes from: (a) high-quality SFT, (b) RL-from-AI-Feedback (RLAIF) for breadth, (c) RLVR for verifiable tasks, (d) human RLHF only for irreducibly subjective categories. The DPO-vs-PPO debate is a sideshow — both work, choice is engineering preference. The real frontier shift in 2025-2026 is 'preference alignment' becoming 'reasoning alignment' — RL signal moving from human preference to verifiable correctness for hard tasks. This is the most important post-training shift since RLHF itself.

• Does explicit Constitution training actually shape behavior more than implicit RLHF preference? No clean ablation exists.
• Spec gaming: red teamers regularly find spec-compliant ways to produce undesired output. Is this a fundamental limit or a training quality issue?
• Authority hierarchy enforcement under prompt injection: Wallace 2024 trained for it, but persistent breakthroughs published monthly. Is this solvable in current paradigm?
• Open-weights specs: a model with public weights can be 'unspecced' via fine-tuning. Does specification have any role for open models?
• Does spec content matter, or just spec presence? Maybe any reasonable spec produces similar behavior given good training.

Reference analyst note. Specifications are operationally useful (alignment of human reviewers, regulatory clarity, dispute resolution) but their causal effect on model behavior is poorly understood. The a constitutional-methods frontier lab Constitution and a leading frontier lab a public alignment specification serve more as institutional artifacts than technical control mechanisms. The next frontier is 'specs the model can actually reason about' — current specs are read like training labels, not internalized reasoning frameworks. Constitutional Classifiers (2025) suggest a path: separate small model that explicitly checks against principles.

• Is there a fundamental limit to jailbreak resistance in autoregressive models? Several theoretical arguments (impossibility under attacker compute, no robust universal classifier) but no formal proof.
• Indirect prompt injection: is this solvable at the architectural level, or only via tool sandboxing? The pessimistic view is gaining ground.
• Red team coverage: how do you measure if you've tested 'enough'? Coverage metrics are arbitrary; novel attacks regularly surprise even mature programs.
• Distillation attacks: can a public API let an attacker steal the model via clever querying? Demonstrated at small scale (Carlini 2024); production-scale defense unclear.
• Reasoning-model jailbreaks: o1-style models can be jailbroken by manipulating their chain-of-thought directly. Whole new attack surface.

Reference analyst note. The field treats red teaming as a check-box rather than a primary capability. Strong red teams (a dedicated Frontier Red Team, Apollo Research, Lakera, an external evaluation organization) regularly find critical issues that internal teams missed. Industry should normalize 'find a flaw, get paid significantly' rather than current low-five-figure bounties. Indirect prompt injection in agent settings is the unsolved security problem of the next 2 years; no clean technical solution exists. The honest answer to 'is your AI agent secure' is 'no, build for graceful degradation.'

• Is there a saturation point for evaluation itself? When all standard benchmarks saturate, what replaces them?
• Contamination: how badly are public benchmarks contaminated in training data? Anecdotally severe; quantitative measures rare.
• Per-domain capability mapping: frontier models are 'generally capable' but per-task spread is huge. No good way to summarize.
• Long-tail capability: standard benchmarks measure central capabilities. The 'long tail' (rare tasks, novel domains, expert work) is where models actually fail.
• Reasoning eval: existing benchmarks (GSM8K → MATH → AIME → FrontierMath) chain. Is there a Pareto-frontier reasoning eval, or is it always 'next harder math'?

Reference analyst note. Standard benchmarks are entering crisis — saturation, contamination, gameability. The next 2 years will see shift to: (a) live arenas with continuous human ratings (lmarena), (b) frequently-refreshed benchmarks (LiveCodeBench), (c) expert-grade eval (GPQA, FrontierMath, HLE), (d) agent benchmarks measuring real task completion (SWE-bench, GAIA, OSWorld). The trend is from 'static MMLU score' to 'diverse evidence portfolio.' a constitutional-methods frontier lab system cards already do this; expect industry-wide adoption.

• What does 'CBRN uplift' actually mean operationally? Domain experts (virologists) review, but no agreed-upon threshold for 'meaningful uplift.'
• Sandbagging: can a model deliberately underperform on capability evals to avoid being flagged? Demonstrated possible (Apollo Research 2024). How do you eval against deception?
• Persuasion eval methodology: can persuasion be ethically and reliably measured? a constitutional-methods frontier lab's results are interesting but generalizability unclear.
• Bias evaluation framing: most bias benchmarks reflect US-centric demographic categories. Cross-cultural bias eval thin.
• Long-tail safety: standard benchmarks cover obvious harms. Subtle harms (gradual erosion of user agency, sycophancy) are real but unmeasured.

Reference analyst note. Safety evaluation is dramatically underdeveloped relative to capability evaluation. Capability has 50+ standard benchmarks; safety has maybe 15. We are flying blind on subtle harms (sycophancy, manipulation, deception under specific conditions). One lab's interpretabilityility work is the deepest probe; field-wide it's still surface-level. Expect frontier safety eval to expand 5-10× by 2027 driven by EU AI Act conformity and a national AI Safety Institute evaluations.

• Are Responsible Scaling Policy framework capability thresholds set rigorously enough? They're voluntary; no external oversight on threshold-setting.
• Eval validity: how do you prove that an eval correctly measures the capability it claims to? No formal verification.
• Pause discipline: would a frontier lab actually pause development if a threshold triggered, in face of competitive pressure? Untested.
• Capability surprise: capabilities emerge non-monotonically. Responsible Scaling Policy framework frameworks assume monotonic capability growth between evals. They might miss sharp jumps.
• Government takeover: if a lab triggers fourth AI Safety Level thresholds, what then? Frameworks are silent on government's role; geopolitically loaded.

Reference analyst note. Responsible Scaling Policies are useful coordination devices but their actual prophylactic power is untested. They've never paused a release. The optimistic read: capabilities haven't crossed thresholds. The pessimistic read: thresholds are calibrated to never bind. Truth probably mix. The next test will come when a model genuinely approaches third AI Safety Level cyber or CBRN — likely 2026-2027. Whether the framework holds under genuine commercial pressure is the real test.

• Watermark robustness against adversarial paraphrasing: a generative-content watermarking system demonstrated on benign paraphrasing; under active adversarial attack, removal is straightforward.
• Output classifier coverage: any classifier trained on a fixed taxonomy is gameable by attacks outside that taxonomy. The arms race is unwinnable in static defense.
• Multi-modal output safety: text classifiers mature; image generation safety (Diffusion model output filtering) less mature.
• Refusal style: 'sorry I can't help with that' refusals harm UX. Better refusal templates (offer alternative) under-deployed.
• Content provenance enforcement: C2PA only works if downstream platforms enforce it. They mostly don't. Adoption gap.

Reference analyst note. Output safety is the right architectural choice — input filtering is doomed because input space is unbounded, output space is comparatively constrained. output-conformance safety paradigm (egress filtering + cached refusal templates + classifier ensemble) is the production-ready answer. The remaining hard problem is multimodal output (image/video/audio) where classification is much harder than text. Watermarking is a useful piece but not a solution; treat it as evidence, not enforcement.

• Optimal serving stack at frontier: an open-source inference engine, a vendor inference stack, SGLang each have advantages. No standard 'best' — workload-specific.
• Multi-tenancy isolation: how strong is isolation between customer requests on shared GPU? Some side-channel concerns (timing, cache).
• Edge inference: a current-generation accelerator-class models on edge (laptops, phones) is the new frontier. A leading open-weights model.2-3B, a synthetic-heavy small frontier model-mini run on phone. Quality gap to frontier still substantial.
• Serving reasoning models: o1/R1-style models with hidden chains-of-thought have very different latency profiles (long initial thinking). UX patterns unclear.

Reference analyst note. Inference engineering is undervalued relative to training. A 5× throughput gain via better serving = 5× more users at same cost. Most labs underinvest. An open-source inference engine's PagedAttention was a paper; it should have been a unicorn. The next round of gains comes from: (a) speculative decoding everywhere (a draft-head speculative decoding technique-2, MTP), (b) FP8/FP4 inference on a next-generation accelerator, (c) cross-request KV cache (prefix caching), (d) serving optimizations specific to reasoning models. Anyone serving LLMs at scale who isn't doing all four is leaving 5-10× on the table.

• Optimal quantization at extreme low bit (W4A4, W2): research shows degradation; production deployment cautious.
• Speculative decoding for reasoning models: long-CoT outputs may have lower acceptance rates. Workload-specific tuning unclear.
• Hardware-software co-design: a next-generation accelerator NVLink Switch enables 72-GPU coherent domains. How much should serving stacks evolve to exploit this?
• Inference-time compute scaling (best-of-N, MCTS): how do you serve these? Same per-query infrastructure must scale 10-100× compute. Production patterns immature.

Reference analyst note. Inference optimization is solved at the kernel and batching levels — an open-source inference engine, a vendor inference stack, FlashAttention together cover most of the win. The remaining frontier is system-level: prefix caching at scale, speculative decoding for reasoning, multi-LoRA dispatch, hardware-aware kernel JIT. Frontier serving stacks in 2026 will look fundamentally different from 2024 in their handling of test-time-compute-scaling models — this transition is mid-progress and labs differ widely.

• Quality regression detection latency: how fast can you actually detect that a deployed model got slightly worse? Anecdotally: hours to days, depending on regression magnitude.
• Online eval reliability: LLM-as-judge has known biases (length, position, style). Online quality monitoring inherits these.
• User feedback signal: thumbs-up/down rates are 0.1-1% of interactions. How representative is this signal?
• Cost spike detection: distinguishing legitimate growth from abuse / attack / runaway agent loop is hard.

Reference analyst note. Production observability for LLMs is 5 years behind general SRE. LangSmith, Helicone, Langfuse are gradually catching up but lack maturity of Datadog/New Relic. The hard problem is quality monitoring — capability changes are subtle and statistical signals are noisy. Frontier labs maintain large internal observability teams; smaller deployments are largely flying blind. Expect this to be a major investment area 2025-2027.

• Quality regression detection in A/B: subjective quality is high-variance. Power analysis often insufficient.
• Model spec drift: as specs evolve, deployed model's spec adherence drifts. When do you re-train vs fine-tune vs just update?
• Multi-version cohabitation: does running 3+ generations of model in production degrade signal in monitoring?
• Forced upgrades: when API customers depend on specific behavior, version deprecation breaks them. Industry has no clean answer.

Reference analyst note. Deployment discipline is genuinely better than 5 years ago — frontier labs run staged rollouts, have rollback procedures, conduct A/B tests. But quality regression detection remains the soft underbelly. A model that's 5% worse on subjective metrics will pass safety / capability / SLO gates and ship. We're learning about quality regressions from arena ranking changes weeks after deployment. Better quality regression infrastructure is high-leverage but underinvested.

• Machine unlearning: how do you actually delete data from a trained model? Active research; no production-ready solution. Unlearning literature reports inconsistent outcomes.
• Training data summary specificity: EU AI Act Article 53 'sufficiently detailed' is undefined. Frontier labs publishing high-level summaries; regulators may demand more.
• Provenance enforcement: if no platform requires C2PA, does it matter that creators add it? Coordination problem.
• Cross-border data flows: EU adequacy decisions, US executive orders, China data localization create geopolitically fragmented governance regime.

Reference analyst note. Data governance is the frontier compliance bottleneck. The naive view ('we don't train on customer data') is insufficient — EU AI Act, copyright lawsuits (NYT v. A leading frontier lab), and emerging unlearning requirements force much deeper governance. Frontier labs that don't have hearing-grade data lineage today will spend 2025-2026 building it. The model card / system card transparency standard set by a constitutional-methods frontier lab is becoming default expectation.

• Indirect prompt injection: solvable in current architecture or requires fundamental redesign? Pessimistic camp ascendant.
• Confidential compute (a leading accelerator vendor CC, a hyperscaler platform Nitro) for inference: production-ready or theatre? Performance overhead poorly characterized publicly.
• Adversarial robustness vs security: how much overlap, how much divergence? Often confused; should be distinguished.
• Weight extraction via API: distillation attacks demonstrated at small scale. Production-scale defense unclear.

Reference analyst note. Security for LLMs is in a state similar to web security circa 2008 — patterns visible but practices immature. The frontier 2026 security stance: assume weights will eventually leak (insider, breach, gradual extraction); design for graceful degradation. The a constitutional-methods frontier lab third AI Safety Level framing (resist non-state actor) is appropriately calibrated; fourth AI Safety Level (resist state actor) is the next frontier and unsolved. Agent security is the unsolved problem of the next 2 years; current 'defenses' are mostly hopeful patterns, not robust controls.

• DP-SGD at frontier scale: too expensive (~5× compute overhead) currently. Does algorithmic improvement make it tractable by 2027?
• Membership inference defense: effective dedup helps, but theoretical worst-case bounds remain. Practical risk assessment unclear.
• Cross-border privacy regime: US-EU adequacy fragile, China PIPL strict, India DPDP emerging. Global compliance becomes per-jurisdiction.
• Inference-time privacy: TEE-based confidential inference deployed at a confidential-computing frontier lab; broader adoption depends on hardware availability and customer demand.

Reference analyst note. Privacy compliance is becoming a serious cost center. Frontier labs that haven't invested in privacy infrastructure (hearing-grade data governance, deletion processes, sectoral certifications) will face compounding regulatory costs 2025-2027. The technically-interesting frontier is private inference (TEE, private cloud compute, eventually homomorphic) — a confidential-computing frontier lab's deployment shows production viability. Differential privacy at training remains aspirational at frontier scale.

• EU AI Act enforcement intensity: untested. Will regulators interpret strictly or lightly?
• Cross-jurisdictional compliance: US, EU, UK, China each have differently-shaped frameworks. Global compliance becomes minimum-bar across all jurisdictions.
• Voluntary commitments: do they hold under competitive pressure? Untested at any frontier lab.
• Standards harmonization: NIST AI RMF, ISO/IEC 42001, EU AI Act, sectoral frameworks overlap and diverge. Industry is creating implicit standards via shared practice.

Reference analyst note. Compliance is becoming a strategic lever. A constitutional-methods frontier lab's investments in FedRAMP, ISO/IEC 42001, EU AI Act readiness give it enterprise customer access a leading frontier lab / a multimodal frontier lab catch up to slowly. The arbitrage is real: $50M+ in compliance investment can unlock $1B+ in regulated-industry revenue. The next 18 months will see frontier labs differentiate not on capability (saturating) but on compliance depth and trust signals.