Skip to content

A Cookbook of Self-Supervised Learning

Authors
Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yann LeCun, Micah Goldblum
Year
2023
Preprint
Source
arXiv
Identifiers
arXiv 2304.12210

Abstract

Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning. Yet, much like cooking, training SSL methods is a delicate art with a high barrier to entry. While many components are familiar, successfully training a SSL method involves a dizzying set of choices from the pretext tasks to training hyper-parameters. Our goal is to lower the barrier to entry into SSL research by laying the foundations and latest SSL recipes in the style of a cookbook. We hope to empower the curious researcher to navigate the terrain of methods, understand the role of the various knobs, and gain the know-how required to explore how delicious SSL can be.

Knowledge Base

Map Tree Search

External

Document arXiv Abstract arXiv PDF arXiv HTML Google Scholar OpenAlex

Tags

Self-supervised learning Supervised learning Learning

Similar Papers

1
84%

DINOv2: Learning Robust Visual Features without Supervision

DINOv2

('Maxime Oquab', 'Timothee Darcet', 'Theo Moutakanni', 'Huy Vo', 'Marc Szafraniec', 'Vasil Khalidov', 'Pierre Fernandez', 'Daniel Haziza', 'Francisco Massa', 'Alaaeldin El-Nouby', 'Mahmoud Assran', 'Nicolas Ballas', 'Wojciech Galuba', 'Russell Howes', 'Po-Yao Huang', 'Shang-Wen Li', 'Ishan Misra', 'Michael Rabbat', 'Vasu Sharma', 'Gabriel Synnaeve', 'Hu Xu', 'Herve Jegou', 'Julien Mairal', 'Patrick Labatut', 'Armand Joulin', 'Piotr Bojanowski') / 2023
Detail Map Tree Search
5
79%

DINOv3

('Oriane Simeoni', 'Huy V. Vo', 'Maximilian Seitzer', 'Federico Baldassarre', 'Maxime Oquab', 'Cijo Jose', 'Vasil Khalidov', 'Marc Szafraniec', 'Seungeun Yi', 'Michael Ramamonjisoa', 'Francisco Massa', 'Daniel Haziza', 'Luca Wehrstedt', 'Jianyuan Wang', 'Timothee Darcet', 'Theo Moutakanni', 'Leonel Sentana', 'Claire Roberts', 'Andrea Vedaldi', 'Jamie Tolan', 'John Brandt', 'Camille Couprie', 'Julien Mairal', 'Herve Jegou', 'Patrick Labatut', 'Piotr Bojanowski') / 2025
Detail Map Tree Search