Meta's AI Director, Sergey Edunov, suggests two new nuclear plants can meet next year's AI power demand, emphasizing AI 'inference' efficiency and advanced GPUs.
VentureBeat presents: AI Unleashed - An exclusive executive event for enterprise data leaders. Hear from top industry leaders on Nov 15. Reserve your free pass Meta's director of engineering for Gen AI, Sergey Edunov, has a surprising answer to how much more power will be needed to handle the increasing demand for AI applications for the next year: just two new nuclear power plants. Edunov leads Meta's training efforts for its Llama 2 open-source foundation model, which is considered one of the leading models. Speaking during a panel session I moderated at the Digital Workers Forum last week in Silicon Valley, he said two power plants would seem to be enough to power humanity's AI needs for a year, and that this seemed to be acceptable. Referring to questions around whether the world has enough capacity to handle the growing AI power needs, especially given the rise of power-hungry generative AI applications, he said: "We can definitely solve this problem." Edunov made clear that he was working only from back-of-the-envelope math when preparing his answer, but said it provided a good ballpark estimate of how much power will be needed to do what is called AI "inference." Inference is the process when AI is deployed in an application, in order to respond to a question or to make a recommendation. Inference is distinct from AI model "training," which is when a model is trained on massive amounts of data in order for it to get ready to do inference. VB Event AI Unleashed Don't miss out on AI Unleashed on November 15! This virtual event will showcase exclusive insights and best practices from data leaders including Albertsons, Intuit, and more. Register for free here Training of large language models (LLMs) has gained scrutiny recently, because it requires massive processing, although only initially. Once a model has been trained, it can be used over and over for inference needs, which is where the real application of AI happens. Power needs for inference are…