TAU2
Measured May 14, 2026Source
Score
0.19
Models15 0 
Meta
Llama 3.1 Instruct 405B
MetaMetaUnited States
Llama 3.1 Instruct 405B is Meta's largest and most capable open-source language model, optimized for instruction following and dialogue. It features strong reasoning abilities, supports a 128K token context window, and excels at complex tasks including coding and multilingual understanding.
CodingReasoningLong context
Input / 1M tokens
$2.75Output / 1M tokens
$6.50Output tokens/s
44.62First-token seconds
0.62sSupported plans
0Benchmark history
Evaluations
Terminalbench Hard
Measured May 14, 2026Source
Score
0.07
Lcr
Measured May 14, 2026Source
Score
0.24
Ifbench
Measured May 14, 2026Source
Score
0.39
Aime 25
Measured May 14, 2026Source
Score
0.03
Aime
Measured May 14, 2026Source
Score
0.21
Math 500
Measured May 14, 2026Source
Score
0.7
Scicode
Measured May 14, 2026Source
Score
0.3
Livecodebench
Measured May 14, 2026Source
Score
0.31
Hle
Measured May 14, 2026Source
Score
0.04
Gpqa
Measured May 14, 2026Source
Score
0.52
Mmlu Pro
Measured May 14, 2026Source
Score
0.73
Artificial Analysis Math Index
Measured May 14, 2026Source
Score
3
Artificial Analysis Coding Index
Measured May 14, 2026Source
Score
14.5
Artificial Analysis Intelligence Index
Measured May 14, 2026Source
Score
17.4
Plan availability
Products and plans that support this model
No products or plans have been linked to this model yet.
Discussion
Thinking... Make sure you are connected to GitHub server