Key Specifications

Benchmark Performance

Benchmark	Score	Unit	Evaluated At	Notes	Source
MMLU	78.4	%	2023-11-20	5-shot	view
HUMANEVAL	74.3	pass@1	2023-11-20	—	view
GSM8K	84.1	%	2023-11-20	0-shot CoT	view
MATH	42.3	%	2023-11-20	0-shot CoT	view
BBH	70.8	%	2023-11-20	3-shot CoT	view
GPQA	42.5	%	2023-11-20	0-shot	view
IFEVAL	74.2	%	2023-11-20	prompt_strict	view
ARC	91.4	%	2023-11-20	challenge	view
MUSR	59.8	%	2023-11-20	0-shot	view
WINOGRANDE	80.2	%	2023-11-20	0-shot	view

Name: Orca 2 13B
Brand: other

Microsoft Orca 2 13B 推理增强模型, 4K 上下文, 基于 Llama 2 微调, 通过渐进式复杂任务训练提升推理。

Vendor	Version	Release Date	Context Window	Input Modalities	Output Modalities	License
Other	orca-2-13b	2023-11-20	4K	text	text	MIT

Benchmark	Score	Unit	Notes
MMLU (Massive Multitask Language Understanding)	78.4	%	5-shot
HumanEval	74.3	pass@1	—
GSM8K (Grade School Math 8K)	84.1	%	0-shot CoT
MATH	42.3	%	0-shot CoT
BBH (BIG-Bench Hard)	70.8	%	3-shot CoT
GPQA	42.5	%	0-shot
IFEval	74.2	%	prompt_strict
ARC	91.4	%	challenge
MUSR	59.8	%	0-shot
WinoGrande	80.2	%	0-shot

Input	Output	Cache Read	Cache Write
—	—	—	—

per million tokens