CoEvoke: Evoking Multi-Agent Collaboration for Trajectory Collection
Training multi-agent collaboration requires trajectories of checking, feedback, and repair, yet task-solving rollouts may end in immediate approval or repeat unsuccessful attempts without useful correction.
This paper introduces CoEvoke, a framework for collecting trajectories to train multi-agent collaboration systems by simulating corrective feedback through controlled perturbations and reference-based guidance.
What happened
The article presents CoEvoke, a framework that creates recoverable difficulties through controlled perturbations and uses reference-based guidance to collect trajectories for multi-agent collaboration training. It demonstrates improved task-solving performance through experimental validation on software engineering and reasoning tasks.
Why it matters
The framework provides a method to obtain corrective supervision for multi-agent systems, addressing a critical challenge in training collaborative AI agents without requiring private-reference guidance or extensive retraining.
- CoEvoke's perturbation-based method significantly improves feedback effectiveness in multi-agent training
- The generated dataset provides valuable resources for studying collaborative AI systems
- Perturbation techniques outperform guidance-based approaches in performance gains
- Training improvements can be achieved without private-reference guidance
- The framework preserves task-solving capabilities with unperturbed inputs
The bigger picture
The approach enables more effective training of multi-agent systems by systematically generating corrective feedback data, potentially improving collaboration efficiency in software engineering and reasoning tasks without compromising task-solving topology.
CoEvoke offers a novel approach to trajectory collection for multi-agent collaboration by systematically introducing recoverable challenges and guided investigation, demonstrating measurable improvements in task-solving performance through controlled experiments.
Technical details
- Controlled perturbations for creating recoverable difficulties
- Restricted reference-based guidance for directing investigation
- ACPBench-Hard and SWE verifier benchmarks for evaluation
- Trajectory collection linking public interactions to evidence and task-native outcomes
- Frozen solving agents with perturbation-enriched judge training
FAQ
What problem does CoEvoke solve?
CoEvoke addresses the challenge of collecting effective trajectories for training multi-agent collaboration systems by creating recoverable difficulties and guiding investigation through controlled perturbations and reference-based methods.
What is the size of the dataset generated by CoEvoke?
The framework produces a dataset containing 4,567 reconstructable trajectories across 4,130 software-engineering and reasoning tasks.
How does CoEvoke improve task-solving performance?
By introducing controlled perturbations that create recoverable difficulties and using restricted reference-based guidance, CoEvoke generates corrective feedback that enhances training outcomes, as demonstrated by 6.73-point improvements in ACPBench-Hard success rates.
What are the key technical components of CoEvoke?
The framework uses controlled perturbations to create recoverable challenges, restricted reference-based guidance to direct investigation, and maintains task-solving topology through workflow evaluations without private-reference guidance.
What benchmarks were used to evaluate CoEvoke?
The framework was tested on ACPBench-Hard and SWE verifier benchmarks, showing significant improvements in task-solving performance through perturbation-enriched training.
This summary and analysis were generated with AI from the original article and may contain errors. Read the original for the full story.
Read the full story
Originally published by arXiv cs.LG on Oct 14, 2026.
Comments
Sign in to join the discussion.