Know a great AI tool or AI news source? Submit it to AiByts
LLM-Assisted2 min read

When F1 Is Not Enough: Auditing Codebooks for LLM-Assisted Annotation

Large language models (LLMs) are increasingly used for text annotation in social science, but standard performance metrics do not explain why errors occur. Low F1 may reflect limitations of the annotation system, defects in the written codebook, or inconsistency in reference labels.

By Yuehong Cassandra Tai, Qunhua LiarXiv - cs.AI

Read the full story

Originally published by arXiv - cs.AI on Nov 9, 2026.

Read on arXiv - cs.AI(opens in a new tab)

Comments

Sign in to join the discussion.

No comments yet — be the first to share your thoughts.