Skip to content

PaliGemma: A Versatile 3B VLM for Transfer

Authors
Lucas Beyer, Andreas Steiner, Andre Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias Bauer, Matko Bosnjak, Xi Chen, Matthias Minderer, Paul Voigtlaender, Ioana Bica, Ivana Balazevic, Joan Puigcerver, Pinelopi Papalampidi, Olivier Henaff, Xi Xiong, Radu Soricut, Jeremiah Harmsen, Xiaohua Zhai
Year
2024
Preprint
Algorithm
PaliGemma
Source
arXiv
Identifiers
arXiv 2407.07726

Abstract

PaliGemma is an open Vision-Language Model (VLM) that is based on the SigLIP-So400m vision encoder and the Gemma-2B language model. It is trained to be a versatile and broadly knowledgeable base model that is effective to transfer. It achieves strong performance on a wide variety of open-world tasks. We evaluate PaliGemma on almost 40 diverse tasks including standard VLM benchmarks, but also more specialized tasks such as remote-sensing and segmentation.

Knowledge Base

Map Tree Search

External

Document arXiv Abstract arXiv PDF arXiv HTML Google Scholar OpenAlex

Tags

Language models Vision-language models Benchmarks PaliGemma

Similar Papers

2
78%

Kimi-VL Technical Report

Kimi-VL

('Angang Du', 'Bohong Yin', 'Bowei Xing', 'Bowen Qu', 'Bowen Wang', 'Cheng Chen', 'Chenlin Zhang', 'Chenzhuang Du', 'Chu Wei', 'Congcong Wang', 'Dehao Zhang', 'Dikang Du', 'Dongliang Wang', 'Enming Yuan', 'Enzhe Lu', 'Fang Li', 'Flood Sung', 'Guangda Wei', 'Guokun Lai', 'Han Zhu', 'Hao Ding', 'Hao Hu', 'Hao Yang', 'Hao Zhang', 'Haoning Wu', 'Haotian Yao', 'Haoyu Lu', 'Heng Wang', 'Hongcheng Gao', 'Huabin Zheng', 'Jiaming Li', 'Jianlin Su', 'Jianzhou Wang', 'Jiaqi Deng', 'Jiezhong Qiu', 'Jin Xie', 'Jinhong Wang', 'Jingyuan Liu', 'Junjie Yan', 'Kun Ouyang', 'Liang Chen', 'Lin Sui', 'Longhui Yu', 'Mengfan Dong', 'Mengnan Dong', 'Nuo Xu', 'Pengyu Cheng', 'Qizheng Gu', 'Runjie Zhou', 'Shaowei Liu', 'Sihan Cao', 'Tao Yu', 'Tianhui Song', 'Tongtong Bai', 'Wei Song', 'Weiran He', 'Weixiao Huang', 'Weixin Xu', 'Xiaokun Yuan', 'Xingcheng Yao', 'Xingzhe Wu', 'Xinhao Li', 'Xinxing Zu', 'Xinyu Zhou', 'Xinyuan Wang', 'Y. Charles', 'Yan Zhong', 'Yang Li', 'Yangyang Hu', 'Yanru Chen', 'Yejie Wang', 'Yibo Liu', 'Yibo Miao', 'Yidao Qin', 'Yimin Chen', 'Yiping Bao', 'Yiqin Wang', 'Yongsheng Kang', 'Yuanxin Liu', 'Yuhao Dong', 'Yulun Du', 'Yuxin Wu', 'Yuzhi Wang', 'Yuzi Yan', 'Zaida Zhou', 'Zhaowei Li', 'Zhejun Jiang', 'Zheng Zhang', 'Zhilin Yang', 'Zhiqi Huang', 'Zihao Huang', 'Zijia Zhao', 'Ziwei Chen', 'Zongyu Lin') / 2025
Detail Map Tree Search
4
75%

PaLM-E: An Embodied Multimodal Language Model

PaLM-E

('Danny Driess', 'Fei Xia', 'Mehdi S. M. Sajjadi', 'Corey Lynch', 'Aakanksha Chowdhery', 'Brian Ichter', 'Ayzaan Wahid', 'Jonathan Tompson', 'Quan Vuong', 'Tianhe Yu', 'Wenlong Huang', 'Yevgen Chebotar', 'Pierre Sermanet', 'Daniel Duckworth', 'Sergey Levine', 'Vincent Vanhoucke', 'Karol Hausman', 'Marc Toussaint', 'Klaus Greff', 'Andy Zeng', 'Igor Mordatch', 'Pete Florence') / 2023
Detail Map Tree Search
5
75%

PaLM: Scaling Language Modeling with Pathways

PaLM

('Aakanksha Chowdhery', 'Sharan Narang', 'Jacob Devlin', 'Maarten Bosma', 'Gaurav Mishra', 'Adam Roberts', 'Paul Barham', 'Hyung Won Chung', 'Charles Sutton', 'Sebastian Gehrmann', 'Parker Schuh', 'Kensen Shi', 'Sasha Tsvyashchenko', 'Joshua Maynez', 'Abhishek Rao', 'Parker Barnes', 'Yi Tay', 'Noam Shazeer', 'Vinodkumar Prabhakaran', 'Emily Reif', 'Nan Du', 'Ben Hutchinson', 'Reiner Pope', 'James Bradbury', 'Jacob Austin', 'Michael Isard', 'Guy Gur-Ari', 'Pengcheng Yin', 'Toju Duke', 'Anselm Levskaya', 'Sanjay Ghemawat', 'Sunipa Dev', 'Henryk Michalewski', 'Xavier Garcia', 'Vedant Misra', 'Kevin Robinson', 'Liam Fedus', 'Denny Zhou', 'Daphne Ippolito', 'David Luan', 'Hyeontaek Lim', 'Barret Zoph', 'Alexander Spiridonov', 'Ryan Sepassi', 'David Dohan', 'Shivani Agrawal', 'Mark Omernick', 'Andrew M. Dai', 'Thanumalayan Sankaranarayana Pillai', 'Marie Pellat', 'Aitor Lewkowycz', 'Erica Moreira', 'Rewon Child', 'Oleksandr Polozov', 'Katherine Lee', 'Zongwei Zhou', 'Xuezhi Wang', 'Brennan Saeta', 'Mark Diaz', 'Orhan Firat', 'Michele Catasta', 'Jason Wei', 'Kathy Meier-Hellstern', 'Douglas Eck', 'Jeff Dean', 'Slav Petrov', 'Noah Fiedel') / 2022
Detail Map Tree Search