HumanEval Benchmark: 164 Problems, pass@1 and HumanEval+
The HumanEval benchmark scores code models on 164 Python problems by pass@1. See how its tests accept wrong code, what HumanEval+ adds and why it is saturated.
TestMu AI Blog23 min read
1 story
The HumanEval benchmark scores code models on 164 Python problems by pass@1. See how its tests accept wrong code, what HumanEval+ adds and why it is saturated.
Showing 1 of 1 stories
You're all caught up.
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts