How much energy and water does AI really use?

// This is a machine generated transcript. Please report any transcription errors to will-help@illinois.edu.

[00:00:00]
spk_0: The weather and climate often cause concern and curiosity, but understanding the facts behind the phenomena can be a challenge. We're here to help. From the University of Illinois, Urbana-Champaign, and Illinois Public Media, welcome to Weather Realness, a weekly podcast about the skies above Illinois and the world.

[00:00:27]
Trent Ford: I'm Trent Ford, Illinois state climatologist with the Prairie Research Institute. Generative AI is an increasingly present part of our lives, and it also has very high energy and water demands that we often don't think about. For example, researchers have estimated that a ChatGPT query consumes about five times more electricity than a simple web search. This high energy demand can result in large carbon dioxide emissions that warm our climate as well. This is already making waves in Illinois. Meta has entered a 20-year agreement with Constellation Energy to purchase electricity from the Clinton nuclear power plant in central Illinois starting in 2027 to support its AI energy demands.

[00:01:05]
spk_2: The investment will help Meta power their artificial intelligence projects which require a lot of electricity. Constellation says the deal will preserve 1,100 jobs and generate $13 million in tax revenue each year. The Clinton nuclear plant was in danger of closing just a few years ago.

[00:01:24]
Trent Ford: And of course AI also has large water demands as well. To talk about the environmental cost of AI and energy consumption, we're joined by Lav Varshney, professor in the Department of Electrical and Computer Engineering at Stony Brook University. Thanks for joining us, Lav.

[00:01:37]
Lav Varshney: Yeah, thanks for having me, Trent.

[00:01:39]
Trent Ford: Energy consumption of something like generative AI like ChatGPT or Gemini, that energy is consumed to produce a text or an image or other things and as far as we can understand it can be broken down in two components. One is creating and training the model itself, and the other one is answering questions from users. So using something like ChatGPT as an example, you know, Lav, can you kind of define what those two stages are and what the energy demands are at each of the stage?

[00:02:08]
Lav Varshney: Yeah, yeah, so like you described, the first phase is training for these large language models and other big AI models, and what you do there is you take a bunch of data which is often, you know, very large scale, and you train a big neural network, you adjust its weights by looking at the training data and these things are massive. They have often trillions of parameters and so it's a very complicated optimization problem that requires a lot of energy to execute and the big frontier labs don't release a lot of information on how big or how much energy it actually took, so it's hard to estimate. And then the second phase is what's often called inference, though there can be stages in between like things like post-training, which is where a lot of the guardrails for safety are introduced or fine tuning for specific purposes, and that requires a little bit of energy as well. But then the bigger thing at the end is inference, so that's actually answering the queries. And if it's a simple just kind of text response, it need not be that energy intensive. And in fact, people have developed hardware accelerators and other chips that are much more efficient than would be other ones. But when it starts to get very energy intensive is when we're using it in reasoning modes where there's a lot of complicated queries that are going on. You might have seen in the news that people are proving mathematical theorems using large language models and that's not a simple response. It's running numerous agents to do a variety of things and verifying the mathematical statements and that's actually quite energy intensive. And then on the inference side, the other reason for a significant amount of energy requirements is because of the scale, you know, it's millions of us, people who are asking queries and so you multiply that out and it becomes very large.

[00:04:07]
Trent Ford: And we've also seen, I think, right around the holidays last year, that the all the rage was those odd caricatures that people would put out of themselves with like Santa's kitchen. It's like, so like an image, things like that. How much more energy generally speaking, does an image like that or a soundbite take to create from one of these models versus just a text response?

[00:04:30]
Lav Varshney: Right, yeah, those like Studio Ghibli memes and all those kind of other images and videos, they're much more intensive than simple text. Again, assuming the text is fairly straightforward, you know, if you were trying to get the solution to the universe and you wanted to produce like a number like 42, that might require a lot of reasoning behind it, but, yeah, like images, videos, they can be, you know, thousands of times the energy requirements of text. I guess there's a famous saying, right, an image is worth about 1,000 words or something. I think the energy calculus actually supports that.

[00:05:08]
Trent Ford: We hear a lot across Illinois, but across other places as well about data centers, and I know the two are connected. But can you give us a real general concept of how data centers are connected with AI like what we've talked about with ChatGPT and others, and what the energy consumption of the data centers are and if that's related to or different from the energy and water consumption of AI.

[00:05:32]
Lav Varshney: People are building these AI factories as it were, these big old data centers. So for example, Meta built one in DeKalb, Illinois. It was a $1.2 billion project.

[00:05:43]
spk_4: Meta has made it clear they want to be a presence in DeKalb and with its community, bringing more than 1,000 jobs, utilizing environmentally friendly energy sources, and teaming up for a new program with Northern Illinois University.

[00:05:55]
spk_5: Today our state is the third largest data center market in the entire nation and fifth largest in the world.

[00:06:02]
spk_4: Connecting billions around the world, Meta turns on its data processing servers at the new DeKalb Data Center. The 2.3 million square foot campus supported more than 1,200 skilled trade workers during the facility's three-year construction and will maintain 200 jobs once fully operational.

[00:06:19]
Lav Varshney: When you build these data centers, they are where the training is happening. They are where the inference is happening. And they are factories for intelligence and they use incredible amounts of energy, often the full outputs of a nuclear power plant can go into running a data center.

[00:06:40]
Trent Ford: Would it be fair to say that the data centers are a necessary part of having a generative AI or some of the advanced AI models that we have like the ChatGPT or some of the more advanced versions of Copilot or Gemini.

[00:06:57]
Lav Varshney: Yeah, I mean the current approach is to build the centralized large data centers because of the efficiencies that come with scale, but there are alternatives, a more decentralized approach, and there's people talking about that from a national security perspective. As we've seen in recent hostilities in the Persian Gulf, data centers can actually be targets for attack, whereas if infrastructure is distributed, then it's not a target. And there's side benefits to that in terms of resilience in terms of infrastructure more broadly. So centralized data centers are more efficient, but that's not the only approach that's possible and if one did do things in a decentralized manner, it creates all kinds of new possibilities from a policy perspective. And one other thing to note is that, you know, ChatGPT and its ilk are not the only kind of AI that there is. A lot of people are starting to think of AI as this monolithic technology, but there's all kinds of alternative technologies that do intelligent things that require much less energy, much less water, much less land for data centers. As an example, information lattice learning is a technology that my group and [Kkcre] Inc. which is a startup that spun out of my research group, has been developing for many years and it requires much less energy and yet it's very performant and enabling creativity and scientific discovery. So it's important to recognize that, you know, there's different possibilities for AI technology as well. This hyperscaling is not the only paradigm.

[00:08:42]
Trent Ford: What are researchers doing to either minimize the impact of data center buildout and development with generative AI or seek, kind of like what you mentioned previously with alternative solutions to help provide the solutions and the advancements that AI and similar things can do, but maybe reduce the economic and environmental costs.

[00:09:06]
Lav Varshney: So there's many different things people are working on. One was the embodied carbon in the physical plant that I mentioned, so reducing the carbon of concrete. Another example is building chips that are much more efficient than older generations of chips for doing AI processing. But I think the most compelling is actually new kinds of algorithms. So [Kocre], which is a company that spun out of my research group that I'm leading, has been developing a technology called information lattice learning rather than using neural networks which are fairly inefficient. It's actually based on group theory and information theory foundations, and it allows people to understand exactly what's going on. It's much more data efficient. You can train a foundation model for music just on 370 chorales by Johann Sebastian Bach rather than stealing and scraping millions or billions of songs. And it's also much more energy efficient. So that's an example and there's a whole armamentarium of AI techniques that are being developed, which is not to say that LLMs and large reasoning models and other things are not amazing, they are, but these other possibilities are also very performant. So taking that portfolio approach makes sense.

[00:10:25]
Trent Ford: Especially with data center development is sort of a rapidly developing thing and here in Illinois but elsewhere as well. So where are we kind of in the state of policy and regulation of data center development? Are local, state, federal governments sort of right where they should be as far as regulation, way behind the curve, way ahead of the curve, kind of, yeah. Thoughts on that.

[00:10:53]
Lav Varshney: Yeah, yeah, so there's been a lot of efforts around AI regulation. So I myself actually served at the White House in 2022 and 2023, and so I was part of the team that developed the executive order on AI that President Biden signed in October of 2023 and various other things. So, you know, it's been front of mind for me for several years. And there's been a variety of things, so some are on the, you know, promoting AI for various purposes side including the [Genesis mission] which many of us are quite engaged in to drive U.S. competitiveness and AI for scientific discovery. And then on the regulatory side, there hasn't been too much national effort. There has been direction to various agencies to do certain things, but the various states have been pushing a variety of regulatory policy. In fact, in Illinois, there is the Power Act, which is, I guess it stands for Protecting our Water and Energy Resources Act, so that's working its way through the legislature right now, and that would be something that, you know, would be compelling to see in terms of environmental regulations. For me, you know, the biggest thing that would be compelling on the regulatory side is actually informational, so actually requiring transparency. And, you know, so when [one] actually knows how much energy or water or whatever other resources are being used by these AI factories just like there are similar regulations for steel production or coal plants. So I think that would be the most compelling and this also just kind of harkens back to some of my time in Washington. So again there were these transparency requirements, so for very large AI models there was a requirement to report things to the federal government. Also, one can also build in kind of know your customer kinds of requirements KYC, which can actually be very beneficial for national security, and it kind of can be part and parcel of the same transparency requirements for environmental considerations.

[00:13:15]
Trent Ford: Can you expand just a little bit on the know your customer? What is that referring to specifically?

[00:13:19]
Lav Varshney: Oh yeah, so know your customer requirements are when data centers must report who their clients are and what kinds of jobs are being executed, and that helps you prevent, you know, financial fraud from being run in your data center by nation-state actors or, you know, theft of cryptocurrency or you know, there's a variety of reasons why you would want to know your customers as a regulatory thing.

[00:13:46]
Trent Ford: AI has increasingly become incorporated in apps, services, websites, things like that. And the user consent is variable, we'll put it that way. Are there areas where folks may not know that AI is sort of working in the background when it actively is, either for good or not?

[00:14:12]
Lav Varshney: Yeah, I mean, I think you raise a great point about respect for human autonomy. In biomedical ethics, there's kind of four standard principles. One is justice, one is beneficence, one is non-maleficence, and the fourth is this respect for human autonomy, and that's why doctors always ask you before they treat you or medical researchers always get consent, because it's held that, you know, this is a significant form of human rights that you should know what's happening to you. And that hasn't been the case very much in AI. So AI is deployed in all kinds of settings where the people don't realize and from an ethic perspective, I think it's very important to have that respect for human autonomy. I think it's critical that we start to develop either through regulation or through kind of norms to do that. And also that broader question that's kind of underlying, I think what you're asking, I think people should have this right to AI or not to AI.

[00:15:15]
Trent Ford: Doctor Lav Varshney, he's professor in the Department of Electrical and Computer Engineering at Stony Brook University, also a former White House fellow and a co-developer, let's call it, of Chef Watson. Thanks again for joining us, really appreciate it.

[00:15:29]
Lav Varshney: Great, thanks so much, Trent.

[00:15:33]
Trent Ford: If you're just tuning in, you're listening to Weather Realness, the weekly podcast about weather and climate. We talked about the energy footprint of AI and now we're gonna talk about AI's water consumption and its environmental impacts. Speaking with us today is Praveen Kumar, the professor of civil and environmental engineering at the University of Illinois and executive director at the Prairie Research Institute. Thanks for joining us, Praveen.

[00:15:57]
Praveen Kumar: Thank you and thank you for this opportunity.

[00:16:01]
Trent Ford: So can we start by just kind of laying out why do data centers that feed AI, why do they need traditionally large amounts of water?

[00:16:15]
Praveen Kumar: OK, so this is a little bit more nuanced answer, because the key is that these data centers are essentially burning a lot of energy to create the, train the models, and consume all the data for training the models and so forth. So that energy which is being used by these chips creates heat and that has to be dissipated away. Water is one of the mechanisms for moving those heat outside the data centers and back into the environment, but there are other technologies as well. For example, it can be an air-cooled system. So water carries a lot of heat. I mean, if you boil water, it takes quite a bit of heat to boil water and hence when water moves at the temperature, it's carrying away a lot of heat. It's far more efficient than using air cooling because running blowers for air cooling requires additional energy, and that essentially you're substituting the role of water for energy, but that's the essential problem.

[00:17:27]
Trent Ford: Yeah, and our previous guest, Lav, he spoke to the energy demand of these data centers. And that in some cases, and including here in Illinois, there's proposals of building power plants, entire power plants for data centers, and of course those power plants oftentimes if they're nuclear or coal will also use water. So is there, in some cases, is there a water use tied to the energy production to feed those data centers as well?

[00:17:56]
Praveen Kumar: Yeah, so one of the ways you can think about it is what is the water footprint of running these data centers, right? So while there is a lot of water need to cool the environment and the chips within the data center, the energy itself is potentially using quite a bit of water depending on how it is being produced. So renewable energy will have almost a zero footprint. If you're using solar or wind, but if you're using hydropower or other natural gas, coal, or even nuclear, those will utilize water.

[00:18:37]
Trent Ford: Generally speaking, what are water needs for data centers? Do we have a good idea of that? And how does it compare with other sectors' water use? Like, you know, sometimes we think about, you know, the average American household uses X amount of water, and it sounds like a lot, but then you can compare it to, you know, a single acre of center pivot irrigation, and it just pales in capacity in comparison. So, you know, how much is any kind of data center using in comparison to, for example, agricultural water demands or your more traditional kind of industrial water demands, things like that.

[00:19:16]
Praveen Kumar: Yeah, so I think we can do some back of the envelope calculation on this, right? So I looked up the energy demand for my home last month, and it was roughly about 590 kilowatt hour. That translates to about 0.82 kilowatts. So if you have a 100 megawatt of energy usage in there, it would be like 120,000 homes like mine. I don't know if it's small, big or large, but something that [size]. So a 1 gigawatt plant would be 1.2 million homes equivalent, right? And I also looked at my water consumption and that was about 5,500 gallons of water last month. So that translates to about 183 gallons per day. It's pretty typical. 80, 90 gallons per person per day is a typical number, and there are two people in our household, so that matches. So 1 million gallons a day would translate to about 5,500 homes per day. So when you're thinking about saying, OK, this data center uses 1 million gallons a day, it's roughly about saying, OK, it's using something equivalent to 5,500 homes per day kind of a utilization. So that's easy to look at. It's a small town, rural Illinois, is what 1 million gallon translates to.

[00:20:42]
Trent Ford: So what are researchers, you know, what are some of the things researchers are doing to reduce the water demand of AI, and kind of the data centers that are being developed, the things that you've seen or done yourselves?

[00:20:58]
Praveen Kumar: I think the companies that are producing these chips that use all the energy in the water, to the best of my knowledge, they are very aware that they cannot be on an unsustainable path if they need to get these technologies in place. So they can use, in areas where there's water available, they can use water. In a place like Illinois, I think a combination of water and air cooling may be useful for seven to eight months of the year. We have very cool temperatures, so it's very easy to dissipate the heat through an air cooling rather than the water cooling. So it could be a combination. So it depends on what these companies are trying to do, and I was talking to somebody from one company and they say they use a biphasic [system]. So when water evaporates from water to vapor, it takes the most amount of heat. So they make sure that that happens when the water is in contact with the chip, and then it cools down, releasing a lot of the heat and goes into this closed loop. But then that heat has to be dissipated. But their companies are making substantial effort to reduce the water usage. Nvidia's new chip, I mean, they run the water through the chips at roughly 40 degrees Celsius, that's quite warm in there, so that they are not trying to bring down the water temperature to very cold temperatures.

[00:22:29]
Trent Ford: Generally speaking, does Illinois have the water resources to support the growth of data centers?

[00:22:36]
Praveen Kumar: I think the best way to ask this question or answer this question is not to think of Illinois as an entity as a whole, right, because water is very heterogeneously distributed across the state. There are several aquifers. There are several different types of aquifers across the state. So I think the better question to answer is if the data center is being sited at a location, does that location have enough water to support the data center? I don't think answering a question statewide is even feasible, because, I mean, some areas will have a lot more water to support a data center versus other areas.

[00:23:19]
Trent Ford: Our first section was about AI in weather and climate modeling, you know, weather forecasting and advancement. So [there's] some of this weird paradox where AI can both be a contributor to climate change through emissions, but also a benefit to studying climate change and modeling climate. And it seems like it could be the same way with water. You can correct me if I'm wrong, but, you know, can you maybe just briefly talk about some of the ways that either folks at Prairie Research Institute or others at university or beyond are using AI to better study water resources?

[00:23:57]
Praveen Kumar: Yeah, so, I mean, AI is going to be an extremely, is already an extremely valuable tool in terms of increasing productivity and answering questions, right? So when we're looking at understanding from data, so for example, PRI has data that goes back over 150 years and it goes beyond our capacity to put it all together and understand and ask questions, which may provide us insights going back decades and comparing what is going on with the past in a very very meaningful way. So I think that potential, we're looking into pretty hard as to figure out how we might use AI to answer questions which we may not have been able to answer, right? And that doesn't just pertain to water. I mean, these are questions related to ecosystems, biological processes, geological changes that have happened or geomorphological, so as to say. So there is tremendous opportunity for us to use these tools to understand based on the resources that the data resources that we already have. So the value is tremendous for the AI to be utilized in answering some scientific questions.

[00:25:24]
Trent Ford: So last question here, sort of big picture, how do you think about AI today and sort of the near future of how it will develop and the things we need to consider when we think about sustainable development of things like AI.

[00:25:41]
Praveen Kumar: Yeah, so I think that's a very deep question and examples that come to mind, I mean, after the industrial revolution, I mean, we created an aviation industry, we created a semiconductor industry, we have created internet connected economy. So I think AI falls into those revolutionary technologies. And the example I like to give is that in 1993 when the web browser was born, we would not have thought about companies like Amazon or Google and YouTube and you name it, right? And so AI is an outgrowth of that because there's so much information that is put out on the internet that could be mined to essentially learn from that. Trying to put out, think about what this might be in the future is really difficult because it basically opens up the possibility of people's imagination so much more. And so I'm not gonna even venture into that space except to say I am looking forward to seeing what comes out of it.

[00:27:01]
Trent Ford: Well, Praveen, really appreciate you joining us today.

[00:27:04]
Praveen Kumar: All right, thank you, Trent, for the opportunity.

[00:27:11]
Trent Ford: That's it for this week's Weather Realness. I'm Trent Ford, state climatologist from the Prairie Research Institute. If you have a weather or climate question you want us to tackle, leave us a voicemail at 217-333-2141, or you can email weatherrealness@illinois.edu.

[00:27:27]
Maddie Stover: Weather Realness is produced by Jeff Frame, Trent Ford, Steve Nesbitt, Reginald Hardwick, Stephanie Orellana, and myself, Maddie Stover. Funding for this podcast is partially provided by the [Backland] Charitable Trust. Weather Realness is produced by the Department of Climate, Meteorology and Atmospheric Sciences at the University of Illinois Urbana-Champaign and Illinois Public Media. We'll talk with you again next week.