Key Specifications

Benchmark Performance

Benchmark	Score	Unit	Evaluated At	Notes	Source
MMLU	73.4	%	2024-02-01	5-shot	view
HUMANEVAL	53.9	pass@1	2024-02-01	—	view
GSM8K	62.8	%	2024-02-01	0-shot CoT	view
MATH	43.1	%	2024-02-01	0-shot CoT	view
BBH	64	%	2024-02-01	3-shot CoT	view
GPQA	31.8	%	2024-02-01	0-shot	view
IFEVAL	56.2	%	2024-02-01	prompt_strict	view
ARC	89.3	%	2024-02-01	challenge	view
MUSR	44.3	%	2024-02-01	0-shot	view
WINOGRANDE	78.8	%	2024-02-01	0-shot	view

Name: OLMo 7B SFT
Brand: other

AllenAI OLMo 7B SFT 监督微调版本, 2K 上下文, 基于 Tulu 数据集微调, 适合通用对话任务。

Vendor	Version	Release Date	Context Window	Input Modalities	Output Modalities	License
Other	olmo-7b-sft	2024-02-01	2K	text	text	Apache 2.0

Benchmark	Score	Unit	Notes
MMLU (Massive Multitask Language Understanding)	73.4	%	5-shot
HumanEval	53.9	pass@1	—
GSM8K (Grade School Math 8K)	62.8	%	0-shot CoT
MATH	43.1	%	0-shot CoT
BBH (BIG-Bench Hard)	64.0	%	3-shot CoT
GPQA	31.8	%	0-shot
IFEval	56.2	%	prompt_strict
ARC	89.3	%	challenge
MUSR	44.3	%	0-shot
WinoGrande	78.8	%	0-shot

Input	Output	Cache Read	Cache Write
—	—	—	—

per million tokens