2 stories
With the goal of deploying generative AI systems, rater studies are becoming increasingly common and important. This means more and more researchers and engineers face the challenge of actually planning and conducting rater studies for AI systems.
Dealing with rater disagreement is becoming more important in AI, especially for LLMs and in specialized domains such as health. In the past year, I helped open source two datasets allowing to study rater disagreement in the health domain: a relabeling of MedQA, a key benchmark for evaluating…
Showing 2 of 2 stories
You're all caught up.
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts
