Skip to content

A Generalist Agent

Authors
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, Nando de Freitas
Year
2022
Preprint
Identifiers
arXiv 2205.06175

Abstract

Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato.

Knowledge Base

Map Tree Search

External

Document arXiv Abstract arXiv PDF arXiv HTML Google Scholar OpenAlex

Similar Papers

1
83%

RT-1: Robotics Transformer for Real-World Control at Scale

RT-1

('Anthony Brohan', 'Noah Brown', 'Justice Carbajal', 'Yevgen Chebotar', 'Joseph Dabis', 'Chelsea Finn', 'Keerthana Gopalakrishnan', 'Karol Hausman', 'Alex Herzog', 'Jasmine Hsu', 'Julian Ibarz', 'Brian Ichter', 'Alex Irpan', 'Tomas Jackson', 'Sally Jesmonth', 'Nikhil J Joshi', 'Ryan Julian', 'Dmitry Kalashnikov', 'Yuheng Kuang', 'Isabel Leal', 'Kuang-Huei Lee', 'Sergey Levine', 'Yao Lu', 'Utsav Malla', 'Deeksha Manjunath', 'Igor Mordatch', 'Ofir Nachum', 'Carolina Parada', 'Jodilyn Peralta', 'Emily Perez', 'Karl Pertsch', 'Jornell Quiambao', 'Kanishka Rao', 'Michael Ryoo', 'Grecia Salazar', 'Pannag Sanketi', 'Kevin Sayed', 'Jaspiar Singh', 'Sumedh Sontakke', 'Austin Stone', 'Clayton Tan', 'Huong Tran', 'Vincent Vanhoucke', 'Steve Vega', 'Quan Vuong', 'Fei Xia', 'Ted Xiao', 'Peng Xu', 'Sichun Xu', 'Tianhe Yu', 'Brianna Zitkovich') / 2022
Detail Map Tree Search
2
82%

RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation

RoboCat

('Konstantinos Bousmalis', 'Giulia Vezzani', 'Dushyant Rao', 'Coline Devin', 'Alex X. Lee', 'Maria Bauza', 'Todor Davchev', 'Yuxiang Zhou', 'Agrim Gupta', 'Akhil Raju', 'Antoine Laurens', 'Claudio Fantacci', 'Valentin Dalibard', 'Martina Zambelli', 'Murilo Martins', 'Rugile Pevceviciute', 'Michiel Blokzijl', 'Misha Denil', 'Nathan Batchelor', 'Thomas Lampe', 'Emilio Parisotto', 'Konrad Zolna', 'Scott Reed', 'Sergio Gomez Colmenarejo', 'Jon Scholz', 'Abbas Abdolmaleki', 'Oliver Groth', 'Jean-Baptiste Regli', 'Oleg Sushkov', 'Tom Rothorl', 'Jose Enrique Chen', 'Yusuf Aytar', 'Dave Barker', 'Joy Ortiz', 'Martin Riedmiller', 'Jost Tobias Springenberg', 'Raia Hadsell', 'Francesco Nori', 'Nicolas Heess') / 2023
Detail Map Tree Search
3
80%

Kimi K2: Open Agentic Intelligence

Kimi K2

('Yifan Bai', 'Yiping Bao', 'Y. Charles', 'Cheng Chen', 'Guanduo Chen', 'Haiting Chen', 'Huarong Chen', 'Jiahao Chen', 'Ningxin Chen', 'Ruijue Chen', 'Yanru Chen', 'Yuankun Chen', 'Yutian Chen', 'Zhuofu Chen', 'Jialei Cui', 'Hao Ding', 'Mengnan Dong', 'Angang Du', 'Chenzhuang Du', 'Dikang Du', 'Yulun Du', 'Yu Fan', 'Yichen Feng', 'Kelin Fu', 'Bofei Gao', 'Chenxiao Gao', 'Hongcheng Gao', 'Peizhong Gao', 'Tong Gao', 'Yuyao Ge', 'Shangyi Geng', 'Qizheng Gu', 'Xinran Gu', 'Longyu Guan', 'Haiqing Guo', 'Jianhang Guo', 'Xiaoru Hao', 'Tianhong He', 'Weiran He', 'Wenyang He', 'Yunjia He', 'Chao Hong', 'Hao Hu', 'Yangyang Hu', 'Zhenxing Hu', 'Weixiao Huang', 'Zhiqi Huang', 'Zihao Huang', 'Tao Jiang', 'Zhejun Jiang', 'Xinyi Jin', 'Yongsheng Kang', 'Guokun Lai', 'Cheng Li', 'Fang Li', 'Haoyang Li', 'Ming Li', 'Wentao Li', 'Yang Li', 'Yanhao Li', 'Yiwei Li', 'Zhaowei Li', 'Zheming Li', 'Hongzhan Lin', 'Xiaohan Lin', 'Zongyu Lin', 'Chengyin Liu', 'Chenyu Liu', 'Hongzhang Liu', 'Jingyuan Liu', 'Junqi Liu', 'Liang Liu', 'Shaowei Liu', 'T. Y. Liu', 'Tianwei Liu', 'Weizhou Liu', 'Yangyang Liu', 'Yibo Liu', 'Yiping Liu', 'Yue Liu', 'Zhengying Liu', 'Enzhe Lu', 'Haoyu Lu', 'Lijun Lu', 'Yashuo Luo', 'Shengling Ma', 'Xinyu Ma', 'Yingwei Ma', 'Shaoguang Mao', 'Jie Mei', 'Xin Men', 'Yibo Miao', 'Siyuan Pan', 'Yebo Peng', 'Ruoyu Qin', 'Zeyu Qin', 'Bowen Qu', 'Zeyu Shang', 'Lidong Shi', 'Shengyuan Shi', 'Feifan Song', 'Jianlin Su', 'Zhengyuan Su', 'Lin Sui', 'Xinjie Sun', 'Flood Sung', 'Yunpeng Tai', 'Heyi Tang', 'Jiawen Tao', 'Qifeng Teng', 'Chaoran Tian', 'Chensi Wang', 'Dinglu Wang', 'Feng Wang', 'Hailong Wang', 'Haiming Wang', 'Jianzhou Wang', 'Jiaxing Wang', 'Jinhong Wang', 'Shengjie Wang', 'Shuyi Wang', 'Si Wang', 'Xinyuan Wang', 'Yao Wang', 'Yejie Wang', 'Yiqin Wang', 'Yuxin Wang', 'Yuzhi Wang', 'Zhaoji Wang', 'Zhengtao Wang', 'Zhengtao Wang', 'Zhexu Wang', 'Chu Wei', 'Qianqian Wei', 'Haoning Wu', 'Wenhao Wu', 'Xingzhe Wu', 'Yuxin Wu', 'Chenjun Xiao', 'Jin Xie', 'Xiaotong Xie', 'Weimin Xiong', 'Boyu Xu', 'Jinjing Xu', 'L. H. Xu', 'Lin Xu', 'Suting Xu', 'Weixin Xu', 'Xinran Xu', 'Yangchuan Xu', 'Ziyao Xu', 'Jing Xu', 'Jing Xu', 'Junjie Yan', 'Yuzi Yan', 'Hao Yang', 'Xiaofei Yang', 'Yi Yang', 'Ying Yang', 'Zhen Yang', 'Zhilin Yang', 'Zonghan Yang', 'Haotian Yao', 'Xingcheng Yao', 'Wenjie Ye', 'Zhuorui Ye', 'Bohong Yin', 'Longhui Yu', 'Enming Yuan', 'Hongbang Yuan', 'Mengjie Yuan', 'Siyu Yuan', 'Haobing Zhan', 'Dehao Zhang', 'Hao Zhang', 'Wanlu Zhang', 'Xiaobin Zhang', 'Yadong Zhang', 'Yangkun Zhang', 'Yichi Zhang', 'Yizhi Zhang', 'Yongting Zhang', 'Yu Zhang', 'Yutao Zhang', 'Yutong Zhang', 'Zheng Zhang', 'Haotian Zhao', 'Yikai Zhao', 'Zijia Zhao', 'Huabin Zheng', 'Shaojie Zheng', 'Longguang Zhong', 'Jianren Zhou', 'Xinyu Zhou', 'Zaida Zhou', 'Jinguo Zhu', 'Zhen Zhu', 'Weiyu Zhuang', 'Xinxing Zu') / 2025
Detail Map Tree Search
4
80%

MolmoAct2: Action Reasoning Models for Real-world Deployment

MolmoAct2

('Haoquan Fang', 'Jiafei Duan', 'Donovan Clay', 'Sam Wang', 'Shuo Liu', 'Weikai Huang', 'Xiang Fan', 'Wei-Chuan Tsai', 'Shirui Chen', 'Yi Ru Wang', 'Shanli Xing', 'Jaemin Cho', 'Jae Sung Park', 'Ainaz Eftekhar', 'Peter Sushko', 'Karen Farley', 'Angad Wadhwa', 'Cole Harrison', 'Winson Han', 'Ying-Chun Lee', 'Eli VanderBilt', 'Rose Hendrix', 'Suveen Ellawela', 'Lucas Ngoo', 'Joyce Chai', 'Zhongzheng Ren', 'Ali Farhadi', 'Dieter Fox', 'Ranjay Krishna') / 2026
Detail Map Tree Search