Machine learning models have already been mastered in chess, Go, Atari games and more, but to take it to the next level, Facebook researchers intend to take AI to a different kind of game: the infamous Tough and infinitely complex nethack.
“We wanted to create the most accessible ‘grand challenge’ with this game. It wouldn’t be solution of AI, but it will unlock the way Better AI,” said Edward Grafenstet of Facebook AI Research. “Games are a good domain to explore our assumptions about what makes machines intelligent and break them.”
You may not be familiar with NetHack, but it is one of the most impressive games ever. You are an adventurer in a fantasy world that is different each time as you traverse the increasingly dangerous depths of a dungeon. You must fight demons, navigate traps and other dangers, and be on good terms with your lord in the meantime. It’s the first “roguelike” (after Rogue, its immediate and much simpler predecessor) and is arguably still the best – almost certainly the hardest.
(By the way, it’s free, and you can download and play it on almost any platform.)
Its simple ASCII graphics, AG for a ghost, an @ for the player, lines and dots for the level architecture, and so on, belie its incredible complexity. Because Nethack, which made its debut in 1987, has been under active development ever since, expanding its roster of objects and creatures, rules, and the countless, countless interactions between them all with its shifting team of developers.
And that’s part of what makes NetHack such a difficult and interesting challenge to AI: It’s so open-ended. Not only is the world different each time, but every object and creature can interact in new ways, most of them hand-coded over decades to cover every possible player choice.
“Atari, Dota 2, Starcraft 2… The solutions we found to progress there are very interesting. NetHack just presents different challenges. To play the game as a human you have to rely on human knowledge Will,” Greifenstadt said.
In these other games, there is a more or less clear strategy for winning. Of course it’s more complicated in a game like Dota 2 than in an Atari 800 game, but the idea is the same – player controls, a game board of environments, and win-win conditions to proceed. Similar is the case in NetHack, but it’s even weirder than that. For one thing, the game is different every time, and not just in the details.
“New dungeons, new worlds, new monsters and items, you have no save points. If you make a mistake and die you won’t get a second shot. It’s like real life,” Greifenstadt said. One has to learn from mistakes and come to new situations armed with that knowledge.”
Drinking a corrosive potion is definitely a bad idea, but what about throwing it at a monster? Coating your weapon with it? Putting it on the lock of the treasure chest? diluting it with water? We have intuitive ideas about these actions, but the AI playing the game doesn’t think like us.
According to Greifenstatt, the depth and complexity of the systems in NetHack are difficult to explain, but this variety and difficulty make the game an ideal candidate for a competition. “You have to rely on human wisdom to play the game,” he said.
People have been designing bots to play NetHack for many years that do not rely on neural networks but on complex decision trees in the form of the game. The team at Facebook Research hopes to generate a new approach by creating a training environment on which people can test machine learning-based game-playing algorithms.
The NetHack Learning Environment was actually put together last year, but the NetHack Challenge is just getting started. NLE is basically a version of a game embedded in a dedicated computing environment that lets the AI interact with it via text commands (actions such as directions, attacks or quaffs).
This is an attractive target for aspiring AI designers. While games like StarCraft 2 can enjoy a high profile in some ways, NetHack is legendary and the idea of building a model on completely different lines from those that dominate other games is an interesting challenge.
It’s also, as Greifenstatt explained, more accessible than many in the past. If you want to build AI for StarCraft 2, you’re going to need a lot of computing power to run the visual recognition engine on the imagery from the game. But in this case the entire game is transmitted via text, making it extremely efficient to work with. It can be played thousands of times faster than any human with the most basic computing setup. This leaves the challenge wide open to individuals and groups who do not have access to the high-powered setups needed to power other machine learning methods.
“We wanted to create a research environment that would have a lot of challenges for the AI community, but not just limit it to large academic labs,” he said.
For the next few months, NLE will be available for people to test, and competitors can build their bots or AI in basically any medium. But when the competition starts on October 15th, they’ll be limited to interacting with the game in their controlled environment through standard commands – no special access, no inspection RAM, etc.
The goal of the contest will be to complete the game, and the Facebook team will track how many times the agent “ascends” in a set amount of time, as NetHack says. But “we’re assuming it will be zero for everyone,” Greifenstadt acknowledged. After all, it’s one of the hardest games ever, and even humans who’ve played it for years have trouble winning it even once in a lifetime, let alone several times in a row. There will be other scoring metrics to judge the winners in many categories.
The hope is that this challenge provides the seed for a new approach to AI, one that is more fundamentally similar to real human thinking. Shortcuts, trial and error, score-hacking, and zerging won’t work here – the agent needs to learn the systems of logic and apply them flexibly and wisely, or die horribly at the hands of an angry centaur or owl.
You can find the rules and other specifics of the NetHack Challenge here. The results will be announced later this year at the NeuraIPS conference.