智能AI
morning
是什么推动了反馈的互动改进?
2026-07-01
1 阅读
约4分钟阅读
Bart{\l}omiej Cupia{\l}, Jan {\L}ojek, Miko{\l}aj Garstecki, Szymon Pob{\l}ocki, Alicja Ziarko, Piotr Mi{\l}o\'s
字号:
arXiv:2606.30774v1 公告类型:新 摘要:我们研究自然语言反馈何时产生超越仅通过重复尝试获得的收益的改进。在多轮语言代理设置中,更高的最终精度可以反映有用的反馈,但它也可能来自重采样、格式校正或额外的测试时间计算。为了区分这些影响,我们引入了跨 Omni-MATH、Codeforces、BBEH Linguini 和 ARC-AGI1 的受控学生-教师协议,评估学生和教师角色的 13 个开放权重模型。 We compare external feedback, self-feedback, and unguided self-refinement, while varying interaction history, task difficulty, and teacher access to privileged task information. Across settings, we find that multi-turn improvement is often not evidence of feedback use: self-generated feedback adds little beyond unguided self-refinement, whereas the strongest external teachers produce substantially larger feedback-specific gains, suggesting that useful feedback must provide guidance beyond generic retry. Dense student-teacher interaction matrices further show that interactive gains are driven more by the student's ability to use feedback than by the teacher's identity, although teacher choice remains important for a fixed student. These results suggest that feedback-based agents should be evaluated against repeated-attempt baselines, and that ability to act on feedback, not merely feedback availability, is a central bottleneck for interactive improvement. We release our controlled student-teacher evaluation framework at https://j-lojek.github.io/feedback-generation-is-a-bottleneck/.
这篇文章对您有帮助吗?
订阅66必读
每日精选科技资讯,直达你的邮箱