GitHub热门项目: simple-evals

GitHub项目：simple-evals 仓库地址：https://github.com/openai/simple-evals Stars：4638 | 作者：openai 项目描述：暂无描述 ================================================== README 内容： # ⚠️ Deprecation Notice **July 2025**: `simple-evals` will no longer be updated for new models or benchmark results. The repo will continue to host reference implementations for **HealthBench**, **BrowseComp**, and **SimpleQA**. --- # Overview This repository contains a lightweight library for evaluating language models. We are open sourcing it so we can be transparent about the accuracy numbers we're publishing alongside our latest models. ## Benchmark Results | Model | Prompt | MMLU | GPQA [^8] | MATH [^6]| HumanEval | MGSM[^5] | DROP[^5]
(F1, 3-shot) | SimpleQA |:----------------------------:|:-------------:|:------:|:------:|:--------:|:---------:|:------:|:--------------------------:|:---------:| | **o3** | | | | | | | | | | | o3-high [^10] | n/a [^7] | 93.3 | 83.4 | 98.1 | 88.4 | 92.0 | 89.8 | 48.6 | | o3 [^9] [^10] | n/a | 92.9 | 82.8 | 97.8 | 87.4 | 92.3 | 80.6 | 49.4 | | o3-low [^10] | n/a | 92.8 | 78.6 | 96.9 | 87.3 | 91.9 | 82.3 | 49.4 | | **o4-mini** | | | | | | | | | | o4-mini-high [^9] [^10] | n/a | 90.3 | 81.3 | 98.2 | 99.3 | 93.5 | 78.1 | 19.3 | | o4-mini [^9] [^10] | n/a | 90.0 | 77.6 | 97.5 | 97.3 | 93.7 | 77.7 | 20.2 | | o4-mini-low [^10] | n/a | 89.5 | 73.6 | 96.2 | 95.9 | 93.0 | 76.0 | 20.2 | | **o3-mini** | | | | | | | | | | | o3-mini-high | n/a | 86.9 | 77.2 | 97.9 | 97.6 | 92.0 | 80.6 | 13.8 | | o3-mini | n/a | 85.9 | 74.9 | 97.3 | 96.3 | 90.8 | 79.2 | 13.4 | | o3-mini-low | n/a | 84.9 | 67.6 | 95.8 | 94.5 | 89.4 | 77.6 | 13.0 | | **o1** | | | | | | | | | | o1 | n/a | 91.8 | 75.7 | 96.4 | - | 89.3 | 90.2 | 42.6 | | o1-preview | n/a