We’ve found that the biggest contributions at OpenAI come from cross-functional experts, so we either need to hire them or grow them here. Before Learning Day, we very rarely saw people grow cross-functionally—for example, employees coming from a software background rarely picked up machine learning (something equally rare in other organizations except academia). Since Learning Day, this kind of growth has become very common.
On a typical learning day, people do things like:
- Reimplement papers.
- Follow deep learning tutorials.
- Play with new tools in cluster management, compilation, virtual world generation, or coding paradigms.
- Learn how to do research on bite-sized problems.
- Read about new developments in seemingly unrelated areas of AI.
We think Learning Day might be useful for other organizations, so we’d like to share how it started and works at OpenAI.
We first tried out Learning Day on our Robotics team. Here’s how our Head of Robotics, Wojciech Zaremba (Woj), came up with the idea:
In November 2018, I realized that I’d been stagnating in a number of areas because I was always overwhelmed with urgent tasks. These areas were becoming increasingly important for me to know. For example, I kept wanting to evaluate whether my team should switch deep learning frameworks, but I kept being interrupted after an hour or two of coding—which resulted in no forward progress. I kept hearing about research in other domains like causality or energy-based models which might be applicable to robotics, but I didn’t know anything about these fields—and reading about them for half an hour at a time wasn’t helpful.
I knew the best way to solve this problem would be to carve out a day a week for learning. But if this was what I needed to be more productive, it seemed likely that this would also be what my team needed. So I tried doing this for the whole team as an experiment.
I figured that we’d take a short-term productivity hit but see gains in one to two years. But within a month, I started to see better communication between researchers and engineers, with everyone starting to use jargon from each others’ specialty correctly (e.g. discounted reward, MAML, self-attention, container, SRAM, StatefulSet, Raft). Within half a year, I started to see researchers talking about restructuring our codebase using domain-driven design, and engineers picking up research tasks.
Though we encouraged self-study before, it never seemed to work. That’s different now—for example, one team member went from knowing nothing about machine learning to making computer vision contributions within three months. One very strong engineer studied RL for half a year, and now is producing outputs comparable to what I’d expect from an RL PhD.
How it works
Learning Day happens each Thursday. Woj wrote the following guidelines for the Robotics team, but we’ve adapted these principles across each team that has adopted Learning Day:
Learning day is a gift ❤️
Feel free to use learning day for:
- Reading AI papers
- Reimplementing AI papers
- Going over AI tutorials
- Having your tiny side research AI projects
- Learning about fundamentals: linear algebra, statistics etc.
- Learning about ML fundamentals: information theory, Bayesian inference etc.
- Learning about engineering. Read about new programming languages, frameworks (e.g. what is Rust?)
- Learning about management: self-management, prioritization, how renowned research labs works (e.g. Manhattan project, Bell labs)
- Learning about mechanical engineering or having your mechanical engineering research project (e.g. build a linear actuator)
- Learning about any skill that will level you up in your work (in my case, it’s better writing)
This learning day is a gift from Woj. Therefore, I kindly ask you to:
- Take advantage of this day, and really use it for learning. Don’t waste it on Netflix, or don’t use it as an extra weekend day.
- Please write what you are planning to learn, and please write summaries. This will help to motivate others to keep on pushing.
- Please don’t be shy in sharing what you are learning. It’s fine if you don’t know fundamentals, and it’s fine to say that you learned what is matrix multiplication. Everyone of us was there at some point. I would like to help you to level up.
- Please make sure that activities from the learning day don’t leak out to other non-learning days. For instance, it would suck if you would devote the entire week on learning fundamentals, or working on your side project.
To keep people accountable, we ask everyone to post in Slack what they learned that day.
What we learn on Learning Day
The following are examples of what people learn on a single Learning Day.
Deep learning reading
- “Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules”
- “Learning Domain Randomization Distributions for Transfer of Locomotion Policies”
- “Neural Graph Evolution: Towards Efficient Automatic Robot Design
Learning to Learn with Probabilistic Task Embeddings”
- “Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies”
- “Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables”
- “Does computer vision matter for action?”
- “WAIC, but Why? Generative Ensembles for Robust Anomaly Detection”
- “Weight Agnostic Neural Networks”
- “Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations”
- Deep unsupervised learning
- Deep RL Bootcamp
Deep learning coding
- Reptile and MAML
- Play with code in JAX
- Apply Sparse Transformers to vision tasks
- Implement LSTM and transformer from scratch; train them on Penn treebank
- Train a neural net to reproduce the behavior of a physical motor
Math and statistics
- Thanks for the Feedback: The Science and Art of Receiving Feedback Well
- Change Your Questions, Change Your Life: 10 Powerful Tools for Life and Work
Historical context on powerful technologies
- Dark Territory: The Secret History of Cyber War
- The Hacked World Order: How Nations Fight, Trade, Maneuver, and Manipulate in the Digital Age
- Technology Transfer to the USSR, 1928–1937 and 1966–1975: The Role of Western Technology in Soviet Economic Development
- The Turing Test: Verbal Behavior as the Hallmark of Intelligence
- The Information: A History, A Theory, A Flood
- Radical Markets: Uprooting Capitalism and Democracy for a Just Society
We also reimburse reasonable self-studying expenses such as books and tutors, used mostly to learn fundamentals in mathematics. These costs are very worthwhile investments!
How we sustain it
Learning Day’s impact comes from being rigorous about how people use it. It’s not a day for leisure, but instead a day for a specific kind of hard work. We see and try to counteract the following failure modes so that we can sustain it long term:
Learning Day could be used for work. Learning Day could turn into a normal working day because people may want to accomplish their main project faster (due to internal or external pressure). We prevent this by having Learning Day on the same day for every team. This creates positive peer pressure and encourages everyone to take advantage of Learning Day.
Learning Day could expand in scope to non–Learning Days. We actually haven’t yet observed this happen. Based on what we’ve seen with other organizations, we think this would most likely indicate that the person isn’t excited enough about their main project, and would be a sign to their manager that the person should switch teams or projects.
Learning Day could be used for leisure. Our solution is for every team member to share their progress on Slack via Geekbot. This keeps the excitement high and provides an accountability mechanism.
Learning Day beyond Robotics
We’ve recently expanded Learning Day from a subset of our technical teams to the entire company. It’s become a cultural staple—on our most recent internal survey, Learning Day was the aspect of our culture that people talked about the most. We’re excited to see its impact as we continue to evolve and support Learning Day in the future.