Machine Learning and the Future of Video Games

clemenswinter.com
7 min read
fairly difficult
The rapid progress in deep reinforcement learning (RL) over the last few years holds the promise of fixing the shortcomings of computer opponents in video games and of unlocking entirely new region…
The rapid progress in deep reinforcement learning (RL) over the last few years holds the promise of fixing the shortcomings of computer opponents in video games and of unlocking entirely new regions in game design space. However, the exorbitant engineering effort and hardware investments required to train neural networks that master complex real-time strategy games might lead to the impression that the commercial viability of deep RL is still far in the future. To the contrary, I will argue in Part 1 of this essay that these techniques are imminently practical and may see widespread adoption within the next decade. Part 2 presents a case study in which I use deep RL to improve the design of a real-time strategy game. Finally, in Part 3, I speculate about the many ways in which machine learning will impact video games in the years to come.

Forecast

The year 2019 saw an important milestone for large-scale RL systems with the defeat of top human players in Dota 2​1,2​ and StarCraft II​3​. These systems still have many limitations and there is an ongoing debate on how to make matches between computers and humans fair. Nonetheless, they serve as a compelling demonstration of the fact that we are now capable of creating AIs for complex video games that can beat the vast majority of human players. One apparent obstacle to the widespread adoption of these methods is their high computational cost, which has been estimated at tens of millions of dollars​4​. However, considering current trends in hardware cost, training efficiency, and the diminishing returns of scale, we can expect that the cost of comparable systems will decline by three orders of magnitude or more over the next decade.

Declining Hardware Cost

Machine learning is embarrassingly parallel and directly benefits from increasing transistor counts. While Moore's law​5​ has started to slow down, several die shrinks are currently underway​6–10​ and we can expect costs to decline for at least a few more…
Read full article