OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity.
We use Dota as a testbed for general-purpose AI systems which start to capture the messiness and continuous nature of the real world, such as teamwork, long time horizons, and hidden information. Our Dota training system showed that current AI algorithms can learn long-term planning with large but achievable scale. The system is not specific to Dota, and we’ve also used it to control a robotic hand—a previously unsolved problem in robotics.
Defeat the world’s top professionals at 1v1
Achieved August 11, 2017, this milestone showed our Dota system had learned the mechanical rules of Dota at world-competitive levels in this 1-on-1—using one of the three lanes, one of the three game phases, typically lasting 10 rather than 45 minutes, a single hero, and no neutral creeps, Roshan, warding, or invisibility.
Defeat five of the world’s top professionals
OpenAI Five lost two games against top Dota 2 players at The International 2018 in Vancouver with lightly restricted 5v5 (18 heroes, no summons/illusions, multiple banned items).
Dota is selected by looking down the list of games on Twitch, picking the most popular one that ran on Linux and had an API.
First commit in our Dota repository.
First commit in Rapid repository.
1v1 bot beats top professional Dota 2 players at The International 7
First game won by a Dota 2 professional by normal gameplay against final 1v1 bot (tried by dozens of pros for thousands of games).
First 5v5 results: OpenAI Five beats our scripted bot in exceedingly restricted 5v5 (playing to first tower death, with 5 invulnerable couriers, mirror match, five fixed heroes, no neutrals, runes, shrines, wards, Roshan, or invisibility).
OpenAI Five beats in-house OpenAI team at very restricted 5v5 (objective of max net worth at 7 minutes, with 5 invulnerable couriers, mirror match, five fixed heroes, no neutrals, runes, shrines, wards, Roshan, or invisibility).
OpenAI Five defeats in-house OpenAI team at fairly restricted 5v5 (5 invulnerable couriers, mirror match, five fixed heroes, no wards, Roshan, or invisibility)
OpenAI Five's parameters initialized.
OpenAI Five defeats popular casters at the Benchmark in front of a live audience and 100k livestream viewers, with somewhat restricted 5v5 (5 invulnerable couriers, 18 heroes)
Match vs Team Secret.
OpenAI Five plays a team of top professional Dota 2 players at The International 8.
Match vs Lithium (2–0).
Match vs SG e-sports (2–0).
Match vs Alliance (2–0).
OpenAI Five Finals, OG (2–0). OpenAI Five is the first AI to beat the world champions in an esports game, having won two back-to-back games versus the world champion Dota 2 team, OG, at Finals.
OpenAI Arena—we scaled up OpenAI Five to play the Internet, whether as a competitor or teammate. This final test let us answer an important research question—to what extent OpenAI Five is exploitable or can otherwise be reliably beaten—and be potentially the largest-ever deployment of a highly-competent deep reinforcement learning agent that people can knowingly interact with.
Our Dota 2 AI is undefeated against the world's best solo players: https://t.co/cUvTbGW7ya—OpenAI (@OpenAI) August 11, 2017
#AI bots just beat humans at the video game Dota 2. That’s a big deal, because their victory required teamwork and collaboration – a huge milestone in advancing artificial intelligence. https://t.co/UqIUhh9xFc—Bill Gates (@BillGates) June 26, 2018
Humans call gg for game 2!!!! OpenAI Five has won a best of three vs top 99.95th percentile of Dota players. Extremely good play by the human team. Our system is ready to take on the top professionals at The International later this month.—Greg Brockman (@gdb) August 5, 2018
How OpenAI Five Works
OpenAI Five is a team of five artificial neural networks, which you can think of as simulated “brains” which our team has designed to be well-shaped for learning Dota but start with no knowledge. OpenAI Five sees the world as a list of 20,000 numbers which encode the visible game state (limited to the information a human player is permitted to see), and chooses an action by emitting a list of 8 numbers. The OpenAI team writes code which maps between game state/actions and lists of numbers. Once trained, these neural networks are creatures of pure instinct—their neural networks implement memory but do not otherwise learn further. They play as a team, but we do not design special communication structures—only provide them with an incentive.
OpenAI Five’s neural networks start out with random parameters, and uses our general-purpose training system, Rapid, to learn better parameters. Rapid has OpenAI Five play copies of itself, generating 180 years of gameplay data each day across tens of thousands of simultaneous games, consuming 128,000 CPU cores and 256 GPUs. At each game frame, Rapid computes a numeric reward which is positive when something good has happened (e.g. an allied hero gained experience) and negative when something bad has happened (e.g. an allied hero was killed). Rapid then applies our Proximal Policy Optimization algorithm to update the parameters of the neural network—making actions which occurred soon before positive reward more likely and those soon before negative reward less likely.
Just like humans don’t plan out their muscle movements while planning out their day, the community (OpenAI included) had expected long-term planning to require algorithms which handle short-term and long-term plans separately—perhaps via a hierarchical reinforcement learning breakthrough. But despite its very simple underlying algorithm, OpenAI Five learns professional-level strategies from scratch—no human data provided.
- OpenAI's 'Dota 2' bot won 7,215 games against humans in three days April 23, 2019
- OpenAI’s Dota 2 bot defeated 99.4% of players in public matches April 22, 2019
- Eye on A.I.— Why Video Games Are Key to Better A.I. April 16, 2019
- OpenAI Five crushes Dota 2 world champs, and soon you can lose to it too April 13, 2019
- OpenAI bots just trounced pro players of “Dota 2.” Now what? April 14, 2019
- Humans Call GG! OpenAI Five Bots Beat Top Pros OG in Dota 2 April 13, 2019
- AI triumphs against the world’s top pro team in strategy game Dota 2 April 13, 2019
- OpenAI Five defeats professional Dota 2 team, twice April 13, 2019
- OpenAI’s Dota 2 AI steamrolls world champion e-sports team with back-to-back victories April 13, 2019
- OpenAI’s Dota 2 bots sets their sights on the big leagues June 25
- Can bots outwit humans in one of the biggest esports games? June 25
- OpenAI’s bot can now defeat skilled Dota 2 teams June 25
- OpenAI Five from a player’s perspective June 26
- OpenAI’s bot beat a human at video games last year. Now it will take on five at once. June 28
- The Science Behind OpenAI Five that just Produced One of the Greatest Breakthrough in the History of AI July 2
- Will OpenAI’s machine system defeat a pro team at Dota 2 in August? July 2
- What to pay attention to in the OpenAI Five Benchmark August 4
- Livetweets from the Benchmark August 5
- OpenAI bots thrash team of Dota 2 semi-pros, set eyes on mega-tourney August 6
- OpenAI & DOTA 2: Game Is Hard August 10
- To hone its collaborative skills, this AI is taking on the world’s top video game players August 15
- A team of Dota 2-playing AI bots beat the [experts] - and now they're gunning for more August 20
Special thanks to the numerous people across OpenAI that helped out at our Benchmark and Finals events.