Long ContextActive
Trust Score:
93
GDP.pdf
Professional-work benchmark over long PDF documents, evaluated by Artificial Analysis.
Launched: —Refresh: static
Status Assessment (active):
Actively evaluated by Artificial Analysis on current frontier models; no saturation evidence recorded.
Performance Timeline
Longitudinal progression of model scores against human baselines.Performance & Historical Trajectory
Empirical score progression across model release dates and evaluation rounds.
Independent Vendor-Reported
| Model | Score | Date | Source Type | Provenance |
|---|---|---|---|---|
| A.X-K2 | 6.4% | 2026-11-15 | independent | Source ↗ |
| Qwen3.7 Plus | 12.2% | 2026-11-15 | independent | Source ↗ |
| Ling 3.1 Flash | 10% | 2026-11-15 | independent | Source ↗ |
| NVIDIA Nemotron 3 Nano 4B | 0.4% | 2026-11-15 | independent | Source ↗ |
| K2 Horizon 0.9B | 0.8% | 2026-11-15 | independent | Source ↗ |
| Qwen3.8-Flash-Next | 15.6% | 2026-11-15 | independent | Source ↗ |
| Ministral 3 8B | 0.8% | 2026-11-15 | independent | Source ↗ |
| Llama 4 Maverick | 0.8% | 2026-11-15 | independent | Source ↗ |
| Qwen3.8 2.4T A95B | 15.6% | 2026-11-15 | independent | Source ↗ |
| Kimi K2.7 Code | 11.2% | 2026-11-15 | independent | Source ↗ |
| Gemini 4 Hydrogen | 27.6% | 2026-11-15 | independent | Source ↗ |
| DeepSeek V4-Pro | 11.4% | 2026-11-15 | independent | Source ↗ |
| Llama 4 Scout | 0% | 2026-11-15 | independent | Source ↗ |
| Celeris-1 | 1.2% | 2026-11-15 | independent | Source ↗ |
| Ling 3.0 Tiny | 0.8% | 2026-11-15 | independent | Source ↗ |
| Nemotron 3.5 Bolt 74B-A9B | 2.8% | 2026-11-15 | independent | Source ↗ |
| Muse Ember | 22.8% | 2026-11-15 | independent | Source ↗ |
| DeepSeek V4 Flash Vision | 12% | 2026-11-15 | independent | Source ↗ |
| Mercury 2.5 | 3.6% | 2026-11-15 | independent | Source ↗ |
| Gemini 3.8 Flash | 21% | 2026-11-15 | independent | Source ↗ |
| Inkling Small | 11.2% | 2026-11-15 | independent | Source ↗ |
| Ministral 3 14B | 1.2% | 2026-11-15 | independent | Source ↗ |
| Ling-3.0-flash-Fin | 5.4% | 2026-11-15 | independent | Source ↗ |
| Beam 1.0 | 11.6% | 2026-11-15 | independent | Source ↗ |
| GLM 5.3 | 11.2% | 2026-11-15 | independent | Source ↗ |
| Qwen3 Coder Next | 2.6% | 2026-11-15 | independent | Source ↗ |
| Quasar 438B | 5.4% | 2026-11-15 | independent | Source ↗ |
| Ring-2.6-1T | 3.8% | 2026-11-15 | independent | Source ↗ |
| Muse Wonder Full | 32.6% | 2026-11-15 | independent | Source ↗ |
| Inkling 2 Small | 25.2% | 2026-11-15 | independent | Source ↗ |
| Nemotron 3.5 Super VL | 8% | 2026-11-15 | independent | Source ↗ |
| Mistral Medium 3.5 | 2.8% | 2026-11-15 | independent | Source ↗ |
| Agnes 2.5 Pro Theta | 14.2% | 2026-11-15 | independent | Source ↗ |
| MiniMax M3.1 Flash | 20.8% | 2026-11-15 | independent | Source ↗ |
| Hy4 | 21.6% | 2026-11-15 | independent | Source ↗ |
| Gemma 4 31B | 6% | 2026-11-15 | independent | Source ↗ |
| Ministral 3 3B | 0% | 2026-11-15 | independent | Source ↗ |
| gpt-oss-120b | 4.2% | 2026-11-15 | independent | Source ↗ |
| GPT-5.5 Instant | 20.2% | 2026-11-15 | independent | Source ↗ |
| Gemini 3.1 Pro | 17.8% | 2026-11-15 | independent | Source ↗ |
| K2 Horizon 375B A23B | 7.4% | 2026-11-15 | independent | Source ↗ |
| Trinity Large Thinking | 1.2% | 2026-11-15 | independent | Source ↗ |
| JT-4.1 Flash 236B A21B | 6.6% | 2026-11-15 | independent | Source ↗ |
| Hy3 | 11.2% | 2026-11-15 | independent | Source ↗ |
| Qwen3.8 Max (0902) | 22.8% | 2026-11-15 | independent | Source ↗ |
| Motif 3 | 5.2% | 2026-11-15 | independent | Source ↗ |
| Granite 4.2 8B | 0.6% | 2026-11-15 | independent | Source ↗ |
| Granite 4.2 30B | 1.8% | 2026-11-15 | independent | Source ↗ |
| Grok 4.7 | 20% | 2026-11-15 | independent | Source ↗ |
| Claude Haiku 5.5 | 20.8% | 2026-11-15 | independent | Source ↗ |
| GPT-6 Astra | 31% | 2026-11-15 | independent | Source ↗ |
| North Mini Code | 1.8% | 2026-11-15 | independent | Source ↗ |
| GPT-6 Luna | 22.8% | 2026-11-15 | independent | Source ↗ |
| GLM 5.3 Flash | 15.4% | 2026-11-15 | independent | Source ↗ |
| K-EXAONE 2.0 | 4.2% | 2026-11-15 | independent | Source ↗ |
| Solar Pro 4 | 7% | 2026-11-15 | independent | Source ↗ |
| Mistral Small 4 | 0% | 2026-11-15 | independent | Source ↗ |
| MiMo-V2.6-Flash | 9% | 2026-11-15 | independent | Source ↗ |
| K2 Horizon 7B | 3.2% | 2026-11-15 | independent | Source ↗ |
| MiMo-V2.6-Pro | 19.2% | 2026-11-15 | independent | Source ↗ |
| Qwen3.5 397B A17B | 12.2% | 2026-11-15 | independent | Source ↗ |
| Inkling | 12.8% | 2026-11-15 | independent | Source ↗ |
| Gemini 4 Carbon | 28.4% | 2026-11-15 | independent | Source ↗ |
| GPT-6.1 Terra | 28.6% | 2026-11-15 | independent | Source ↗ |
| Claude Fable 5.2 | 24.2% | 2026-11-15 | independent | Source ↗ |
| Granite 4.2 3B | 0.2% | 2026-11-15 | independent | Source ↗ |
| Step 5 | 14.8% | 2026-11-15 | independent | Source ↗ |
| Gemini 3.5 Flash-Lite | 13.6% | 2026-11-15 | independent | Source ↗ |
| GPT-6.1 Sol | 31% | 2026-11-15 | independent | Source ↗ |
| Apodex 1.1 | 11.2% | 2026-11-15 | independent | Source ↗ |
| Ling-3.0-flash-VL | 8.4% | 2026-11-15 | independent | Source ↗ |
| Kimi K3.1 Pro | 31.4% | 2026-11-15 | independent | Source ↗ |
| MiniCPM5-2B | 1.4% | 2026-11-15 | independent | Source ↗ |
| Solar Mini 4 | 5% | 2026-11-15 | independent | Source ↗ |
| LongCat 2.0 | 6.4% | 2026-11-15 | independent | Source ↗ |
| Nemotron 3 Ultra | 5% | 2026-11-15 | independent | Source ↗ |
| Muse Glimmer | 10% | 2026-11-15 | independent | Source ↗ |
| DeepSeek V4.1 Flash 1017 | 17.2% | 2026-11-15 | independent | Source ↗ |
| K2 Horizon MoVA 36B A4B | 4.2% | 2026-11-15 | independent | Source ↗ |
| Ling 3.1 Flash 1019 | 12.6% | 2026-11-15 | independent | Source ↗ |
| Nanbeige4.3-33B | 11.2% | 2026-11-15 | independent | Source ↗ |
| Gemini 4 Argon | 21.8% | 2026-11-15 | independent | Source ↗ |
| Claude Sonnet 5.5 | 25.8% | 2026-11-15 | independent | Source ↗ |
| Nex-N2-Pro | 12.6% | 2026-11-15 | independent | Source ↗ |
| Muse Wonder | 30.6% | 2026-11-15 | independent | Source ↗ |
| Muse Spark Open | 18.6% | 2026-11-15 | independent | Source ↗ |
| Qwen3.8 27B | 16.4% | 2026-11-15 | independent | Source ↗ |
| K2 Horizon 3.7B | 3.2% | 2026-11-15 | independent | Source ↗ |
| Claude Opus 5.5 | 26.2% | 2026-11-15 | independent | Source ↗ |
| Command A+ | 1% | 2026-11-15 | independent | Source ↗ |
| Nemotron 3.5 Lightning | 3% | 2026-11-15 | independent | Source ↗ |
| Mistral Large 4 | 17.6% | 2026-11-15 | independent | Source ↗ |
| MiniMax M3 | 9.8% | 2026-11-15 | independent | Source ↗ |
Human Baseline & Difficulty Horizon
Calibrated human reference points, specialist benchmarks, and ceiling thresholds.Measured Human Scoren/aNot established
Baseline Protocol & Interpretation
No peer-reviewed human baseline has been published for this benchmark.
Metric & Scoring Methodology
Verification protocols, aggregation formulas, and specialized metric variants.Primary Metric:
accuracy (%)Scoring Engine:llm-judge
Dataset & Compute Cost
Evaluation volume, public availability, API pricing, and local hardware requirements.Total Dataset Size—Annotated evaluation items
Public Test SetPublicOpenly mirrored on repositories
Access GatingOpen AccessUnrestricted download
Evaluation LicenseOpen AccessDataset usage and redistribution terms
Frontier API Compute Cost:
$5 – $20 USD for full benchmark evaluation run on frontier APIs.
Recommended Local GPU Setup:
1x NVIDIA RTX 4090 (24GB) or A100 (40GB/80GB) via vLLM / SGLang
How to Run & Reproduce
Standardized evaluation protocols, CLI commands, and reproducible runner templates.Prompt Regimezero-shot
Reasoning Modedirect
Sampling Temp0
Pass@k Budgetk = 1
Tools & SandboxPure Text
Scoring Verifierllm-judge
Option AEleutherAI LM-Evaluation-Harness (Open-Weight Models)
lm_eval --model hf --model_args pretrained=<model_path> --tasks gdp-pdf --batch_size autoOption BOpenCompass Evaluation Framework
opencompass --datasets gdp-pdf --models <model_config>Python APIDeterministic Inference Loop Snippet
# Standard API Evaluation Loop
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
temperature=0.0,
)Standardized Reporting Requirement:
When publishing results for GDP.pdf, always report the exact prompt template, few-shot exemplar ordering, sampling temperature (temperature=0), maximum reasoning budget tokens, and the precise timestamped model snapshot ID.
Contamination & Memorization Analysis
Audit of pretraining exposure risks, memorization vectors, and refresh policies.Overall Contamination Risk:MEDIUM
Refresh Cadence:
Static fixed snapshot
Test Set Exposure:
Public on web / HuggingFace