RealCLI2 min read
RealCLI: Toward a Unified Evaluation of Human-Agent Collaboration for CLI Agents
Command-line interface (CLI) agents increasingly execute longer-horizon tasks, yet existing benchmarks either assume that user instructions remain fixed throughout execution or evaluate task outcomes without capturing runtime collaboration quality.
By Yunqi Liu, Peiyu Wang, Bingli Wang, Leo Luo, Xuanwu Song, Xudong Ma, Hongxin Li, Yaxin Du, Weicheng Gu, Fanglin Xu, Yi Xing, XiangLong Liu, Jujie He, Zhange Zhang, Huatong Song, Zhaoyi Li, Deyang Kong, Chuan Hao, Peihao Wu, Mingjie Tang, Ran Tao, Jian Yang, Wayne Xin Zhao, Bryan DaiarXiv - cs.CL (NLP)
Read the full story
Originally published by arXiv - cs.CL (NLP) on Oct 19, 2026.
Comments
Sign in to join the discussion.