Skip to content

EvoClaw: Evaluating AI Agents on Continuous Software Evolution

Authors
Gangda Deng, Zhaoling Chen, Zhongming Yu, Haoyang Fan, Yuhong Liu, Yuxin Yang, Dhruv Parikh, Rajgopal Kannan, Le Cong, Mengdi Wang, Qian Zhang, Viktor Prasanna, Xiangru Tang, Xingyao Wang
Year
2026
Preprint
Algorithm
EvoClaw
Identifiers
arXiv 2603.13428

Abstract

With AI agents increasingly deployed as long-running systems, it becomes essential to autonomously construct and continuously evolve customized software to enable interaction within dynamic environments. Yet, existing benchmarks evaluate agents on isolated, one-off coding tasks, neglecting the temporal dependencies and technical debt inherent in real-world software evolution. To bridge this gap, we introduce DeepCommit, an agentic pipeline that reconstructs verifiable Milestone DAGs from noisy commit logs, where milestones are defined as semantically cohesive development goals. These executable sequences enable EvoClaw, a novel benchmark that requires agents to sustain system integrity and limit error accumulation, dimensions of long-term software evolution largely missing from current benchmarks. Our evaluation of 12 frontier models across 4 agent frameworks reveals a critical vulnerability: overall performance scores drop significantly from >80% on isolated tasks to at most 38% in continuous settings, exposing agents' profound struggle with long-term maintenance and error propagation.

Knowledge Base

Map Tree Search

External

Document arXiv Abstract arXiv PDF arXiv HTML Google Scholar OpenAlex

Similar Papers

1
79%

Code as Agent Harness

('Xuying Ning', 'Katherine Tieu', 'Dongqi Fu', 'Tianxin Wei', 'Zihao Li', 'Yuanchen Bei', 'Jiaru Zou', 'Mengting Ai', 'Zhining Liu', 'Ting-Wei Li', 'Lingjie Chen', 'Yanjun Zhao', 'Ke Yang', 'Bingxuan Li', 'Cheng Qian', 'Gaotang Li', 'Xiao Lin', 'Zhichen Zeng', 'Ruizhong Qiu', 'Sirui Chen', 'Yifan Sun', 'Xiyuan Yang', 'Ruida Wang', 'Rui Pan', 'Chenyuan Yang', 'Dylan Zhang', 'Liri Fang', 'Zikun Cui', 'Yang Cao', 'Pan Chen', 'Dorothy Sun', 'Ren Chen', 'Mahesh Srinivasan', 'Nipun Mathur', 'Yinglong Xia', 'Hong Li', 'Hong Yan', 'Pan Lu', 'Lingming Zhang', 'Tong Zhang', 'Hanghang Tong', 'Jingrui He') / 2026
Detail Map Tree Search
2
72%

GLM-5: From Vibe Coding to Agentic Engineering

('Aohan Zeng', 'Xin Lv', 'Zhenyu Hou', 'Zhengxiao Du', 'Qinkai Zheng', 'Bin Chen', 'Da Yin', 'Chendi Ge', 'Chenghua Huang', 'Chengxing Xie', 'Chenzheng Zhu', 'Congfeng Yin', 'Cunxiang Wang', 'Gengzheng Pan', 'Hao Zeng', 'Haoke Zhang', 'Haoran Wang', 'Huilong Chen', 'Jiajie Zhang', 'Jian Jiao', 'Jiaqi Guo', 'Jingsen Wang', 'Jingzhao Du', 'Jinzhu Wu', 'Kedong Wang', 'Lei Li', 'Lin Fan', 'Lucen Zhong', 'Mingdao Liu', 'Mingming Zhao', 'Pengfan Du', 'Qian Dong', 'Rui Lu', 'Shuang Li', 'Shulin Cao', 'Song Liu', 'Ting Jiang', 'Xiaodong Chen', 'Xiaohan Zhang', 'Xuancheng Huang', 'Xuezhen Dong', 'Yabo Xu', 'Yao Wei', 'Yifan An', 'Yilin Niu', 'Yitong Zhu', 'Yuanhao Wen', 'Yukuo Cen', 'Yushi Bai', 'Zhongpei Qiao', 'Zihan Wang', 'Zikang Wang', 'Zilin Zhu', 'Ziqiang Liu', 'Zixuan Li', 'Bojie Wang', 'Bosi Wen', 'Can Huang', 'Changpeng Cai', 'Chao Yu', 'Chen Li', 'Chengwei Hu', 'Chenhui Zhang', 'Dan Zhang', 'Daoyan Lin', 'Dayong Yang', 'Di Wang', 'Ding Ai', 'Erle Zhu', 'Fangzhou Yi', 'Feiyu Chen', 'Guohong Wen', 'Hailong Sun', 'Haisha Zhao', 'Haiyi Hu', 'Hanchen Zhang', 'Hanrui Liu', 'Hanyu Zhang', 'Hao Peng', 'Hao Tai', 'Haobo Zhang', 'He Liu', 'Hongwei Wang', 'Hongxi Yan', 'Hongyu Ge', 'Huan Liu', 'Huanpeng Chu', "Jia'ni Zhao", 'Jiachen Wang', 'Jiajing Zhao', 'Jiamin Ren', 'Jiapeng Wang', 'Jiaxin Zhang', 'Jiayi Gui', 'Jiayue Zhao', 'Jijie Li', 'Jing An', 'Jing Li', 'Jingwei Yuan', 'Jinhua Du', 'Jinxin Liu', 'Junkai Zhi', 'Junwen Duan', 'Kaiyue Zhou', 'Kangjian Wei', 'Ke Wang', 'Keyun Luo', 'Laiqiang Zhang', 'Leigang Sha', 'Liang Xu', 'Lindong Wu', 'Lintao Ding', 'Lu Chen', 'Minghao Li', 'Nianyi Lin', 'Pan Ta', 'Qiang Zou', 'Rongjun Song', 'Ruiqi Yang', 'Shangqing Tu', 'Shangtong Yang', 'Shaoxiang Wu', 'Shengyan Zhang', 'Shijie Li', 'Shuang Li', 'Shuyi Fan', 'Wei Qin', 'Wei Tian', 'Weining Zhang', 'Wenbo Yu', 'Wenjie Liang', 'Xiang Kuang', 'Xiangmeng Cheng', 'Xiangyang Li', 'Xiaoquan Yan', 'Xiaowei Hu', 'Xiaoying Ling', 'Xing Fan', 'Xingye Xia', 'Xinyuan Zhang', 'Xinze Zhang', 'Xirui Pan', 'Xu Zou', 'Xunkai Zhang', 'Yadi Liu', 'Yandong Wu', 'Yanfu Li', 'Yidong Wang', 'Yifan Zhu', 'Yijun Tan', 'Yilin Zhou', 'Yiming Pan', 'Ying Zhang', 'Yinpei Su', 'Yipeng Geng', 'Yong Yan', 'Yonglin Tan', 'Yuean Bi', 'Yuhan Shen', 'Yuhao Yang', 'Yujiang Li', 'Yunan Liu', 'Yunqing Wang', 'Yuntao Li', 'Yurong Wu', 'Yutao Zhang', 'Yuxi Duan', 'Yuxuan Zhang', 'Zezhen Liu', 'Zhengtao Jiang', 'Zhenhe Yan', 'Zheyu Zhang', 'Zhixiang Wei', 'Zhuo Chen', 'Zhuoer Feng', 'Zijun Yao', 'Ziwei Chai', 'Ziyuan Wang', 'Zuzhou Zhang', 'Bin Xu', 'Minlie Huang', 'Hongning Wang', 'Juanzi Li', 'Yuxiao Dong', 'Jie Tang') / 2026
Detail Map Tree Search
4
70%

Kimi K2: Open Agentic Intelligence

Kimi K2

('Yifan Bai', 'Yiping Bao', 'Y. Charles', 'Cheng Chen', 'Guanduo Chen', 'Haiting Chen', 'Huarong Chen', 'Jiahao Chen', 'Ningxin Chen', 'Ruijue Chen', 'Yanru Chen', 'Yuankun Chen', 'Yutian Chen', 'Zhuofu Chen', 'Jialei Cui', 'Hao Ding', 'Mengnan Dong', 'Angang Du', 'Chenzhuang Du', 'Dikang Du', 'Yulun Du', 'Yu Fan', 'Yichen Feng', 'Kelin Fu', 'Bofei Gao', 'Chenxiao Gao', 'Hongcheng Gao', 'Peizhong Gao', 'Tong Gao', 'Yuyao Ge', 'Shangyi Geng', 'Qizheng Gu', 'Xinran Gu', 'Longyu Guan', 'Haiqing Guo', 'Jianhang Guo', 'Xiaoru Hao', 'Tianhong He', 'Weiran He', 'Wenyang He', 'Yunjia He', 'Chao Hong', 'Hao Hu', 'Yangyang Hu', 'Zhenxing Hu', 'Weixiao Huang', 'Zhiqi Huang', 'Zihao Huang', 'Tao Jiang', 'Zhejun Jiang', 'Xinyi Jin', 'Yongsheng Kang', 'Guokun Lai', 'Cheng Li', 'Fang Li', 'Haoyang Li', 'Ming Li', 'Wentao Li', 'Yang Li', 'Yanhao Li', 'Yiwei Li', 'Zhaowei Li', 'Zheming Li', 'Hongzhan Lin', 'Xiaohan Lin', 'Zongyu Lin', 'Chengyin Liu', 'Chenyu Liu', 'Hongzhang Liu', 'Jingyuan Liu', 'Junqi Liu', 'Liang Liu', 'Shaowei Liu', 'T. Y. Liu', 'Tianwei Liu', 'Weizhou Liu', 'Yangyang Liu', 'Yibo Liu', 'Yiping Liu', 'Yue Liu', 'Zhengying Liu', 'Enzhe Lu', 'Haoyu Lu', 'Lijun Lu', 'Yashuo Luo', 'Shengling Ma', 'Xinyu Ma', 'Yingwei Ma', 'Shaoguang Mao', 'Jie Mei', 'Xin Men', 'Yibo Miao', 'Siyuan Pan', 'Yebo Peng', 'Ruoyu Qin', 'Zeyu Qin', 'Bowen Qu', 'Zeyu Shang', 'Lidong Shi', 'Shengyuan Shi', 'Feifan Song', 'Jianlin Su', 'Zhengyuan Su', 'Lin Sui', 'Xinjie Sun', 'Flood Sung', 'Yunpeng Tai', 'Heyi Tang', 'Jiawen Tao', 'Qifeng Teng', 'Chaoran Tian', 'Chensi Wang', 'Dinglu Wang', 'Feng Wang', 'Hailong Wang', 'Haiming Wang', 'Jianzhou Wang', 'Jiaxing Wang', 'Jinhong Wang', 'Shengjie Wang', 'Shuyi Wang', 'Si Wang', 'Xinyuan Wang', 'Yao Wang', 'Yejie Wang', 'Yiqin Wang', 'Yuxin Wang', 'Yuzhi Wang', 'Zhaoji Wang', 'Zhengtao Wang', 'Zhengtao Wang', 'Zhexu Wang', 'Chu Wei', 'Qianqian Wei', 'Haoning Wu', 'Wenhao Wu', 'Xingzhe Wu', 'Yuxin Wu', 'Chenjun Xiao', 'Jin Xie', 'Xiaotong Xie', 'Weimin Xiong', 'Boyu Xu', 'Jinjing Xu', 'L. H. Xu', 'Lin Xu', 'Suting Xu', 'Weixin Xu', 'Xinran Xu', 'Yangchuan Xu', 'Ziyao Xu', 'Jing Xu', 'Jing Xu', 'Junjie Yan', 'Yuzi Yan', 'Hao Yang', 'Xiaofei Yang', 'Yi Yang', 'Ying Yang', 'Zhen Yang', 'Zhilin Yang', 'Zonghan Yang', 'Haotian Yao', 'Xingcheng Yao', 'Wenjie Ye', 'Zhuorui Ye', 'Bohong Yin', 'Longhui Yu', 'Enming Yuan', 'Hongbang Yuan', 'Mengjie Yuan', 'Siyu Yuan', 'Haobing Zhan', 'Dehao Zhang', 'Hao Zhang', 'Wanlu Zhang', 'Xiaobin Zhang', 'Yadong Zhang', 'Yangkun Zhang', 'Yichi Zhang', 'Yizhi Zhang', 'Yongting Zhang', 'Yu Zhang', 'Yutao Zhang', 'Yutong Zhang', 'Zheng Zhang', 'Haotian Zhao', 'Yikai Zhao', 'Zijia Zhao', 'Huabin Zheng', 'Shaojie Zheng', 'Longguang Zhong', 'Jianren Zhou', 'Xinyu Zhou', 'Zaida Zhou', 'Jinguo Zhu', 'Zhen Zhu', 'Weiyu Zhuang', 'Xinxing Zu') / 2025
Detail Map Tree Search
5
68%

A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence

('Huan-ang Gao', 'Jiayi Geng', 'Wenyue Hua', 'Mengkang Hu', 'Xinzhe Juan', 'Hongzhang Liu', 'Shilong Liu', 'Jiahao Qiu', 'Xuan Qi', 'Yiran Wu', 'Hongru Wang', 'Han Xiao', 'Yuhang Zhou', 'Shaokun Zhang', 'Jiayi Zhang', 'Jinyu Xiang', 'Yixiong Fang', 'Qiwen Zhao', 'Dongrui Liu', 'Qihan Ren', 'Cheng Qian', 'Zhenhailong Wang', 'Minda Hu', 'Huazheng Wang', 'Qingyun Wu', 'Heng Ji', 'Mengdi Wang') / 2025
Detail Map Tree Search