Rebounds, rebounds, rebounds
Why environmental sustainability is like a game of whack-a-mole
This week I’m going to briefly talk about one of those frustrating and difficult impediments to improving environmental sustainability: the rebound effect.
The rebound effect, also known as Jevons paradox, was first talked about in the 1800s in relation to coal consumption. The basic idea is that when the efficiency of a particular resource is improved, i.e. you can get more out of that resource at less cost, the observed change in the use of that resource can be different from the expected change. For example, the use of that resource might actually increase as opposed to decrease. Basically, you expected that the use of the resource would go down because of an efficiency improvement, but in reality it just increased. Rebound effects can be positive or negative, but are more often than not talked about with respect to negative effects. Positive rebound effects are sometimes called spillover effects; this post will focus on negative rebound effects.
More concretely: say you want to reduce your energy consumption by installing an energy efficient air conditioning system. You install the system, and now when you cool your house it consumes some percentage of the amount of energy that it used to. You used to be somewhat conservative about when you ran your air conditioning, knowing that it consumed a lot of energy and was going to rack up your energy bill. But now, you decide that it’s probably fine to be liberal about when you run the cooling, and decide to run it the whole night. As a result, you consume more energy than you used to with your old inefficient cooling system.
Reasons for Rebound
Rebound effects are thought to be facilitated by economic, psychological, and behavioral factors [1]. Some prominently observed reasons are surveyed at a high level in [2], reproduced here:
Moral Licensing: The purchase or use of an efficiency-improved technology is perceived as a good deed that licenses increased preferences for purchase/use of that technology, or of other technologies.
Diffusion of Responsibility: Due to the purchase or use of an efficiency-improved technology, perceived responsibility for protecting the environment through frugal usage of that technology diffuses to other agents (e.g., engineers, policy makers, other consumers as potential adopters of efficient technologies), which leads to a decreased preference for frugal purchase/use of that technology, or of other technologies.
Attenuated Consequences: The purchase or use of an efficiency-improved technology leads to a reappraisal of personal monetary, social, or emotional consequences of purchasing or using an efficiency-improved technology, which entails an increased preference for purchasing/using that technology.
Improved Control: Due to an energy efficiency improvement, actual control over resources required to purchase or use a given technology is improved, and/or costs of using that technology compared to other goods and services decrease, which in turn leads to an increased purchase/use of that technology without a preference change towards that technology.
Some concrete examples of each, using machine learning model efficiency as an anchor point, are given here:
Moral licensing: I make my large model smaller, so now I feel okay to use a very overparameterized model for another task I’m working on, because I have potentially saved energy with my first model.
Diffusion of responsibility: I make my large model smaller, so now I feel fine to run it on super inefficient hardware, because its not my responsibility to make my hardware less energy intensive
Attenuated consequences: I make my large model smaller, so now I feel comfortable running a full grid search to get optimal hyperparameters for it
Improved control: I make my large model smaller, and since it runs more quickly, it handles many more requests when it is deployed
While this list isn’t comprehensive, it can provide a sense of why and how the rebound effect exists, and its potential to make improving the environmental sustainability of ML more difficult than it already is.
Rebounds in Machine Learning
Does the rebound effect actually exist in machine learning? It is difficult to disentangle all of the variables related to the energy and carbon footprint in machine learning, as we currently don’t have an excellent way to estimate this holistically. There are two studies, though, which provide some evidence indicating that there are likely rebound effects at play here.
The first is from Wu et al. 2022 [3]. The paper looks at overall energy consumption across Facebook from machine learning systems. They find that through optimizing the efficiency of their ML stack they have curtailed the total energy consumption of ML by 28.5% over a two year period, but despite that, the total energy consumption has still steadily increased.
The second is from Patterson et al. 2022 [4]. They measure the energy consumption due to machine learning at Google over a four year period with respect to the overall energy consumption at the company. They find that, while the percentage of the total energy footprint at the company remained steady at 10-15%, the energy consumption at Google still increased over that period, meaning that energy consumption due to ML increased.
We retroactively performed these calculations [of ML energy consumption] based on data for one week of April in 2019, 2020, and 2021. Each time, the ML portion was 10–15% of Google’s total energy consumption, despite ML representing 70–80% of the FLOPS at Google. While ML use certainly increased during those years, algorithmic and hardware improvements kept that expansion to a rate comparable with overall energy growth at the company.
Efficiency was necessary to ensure that the percentage didn’t skyrocket, but energy consumption from ML still increased overall.
Given this, it’s important to think about the rebound effect and its impact on environmental sustainability in machine learning. A holistic approach to sustainability will include it as one component, in order to maximize the gains from interventions which are designed to make ML more sustainable.
References
[1] Font Vivanco, David, Jaume Freire‐González, Ray Galvin, Tilman Santarius, Hans Jakob Walnum, Tamar Makov, and Serenella Sala. "Rebound effect and sustainability science: A review." Journal of Industrial Ecology 26, no. 4 (2022): 1543-1563.
[2] Santarius, Tilman, and Martin Soland. "How technological efficiency improvements change consumer preferences: towards a psychological theory of rebound effects." Ecological Economics 146 (2018): 414-424.
[3] Wu, Carole-Jean, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang et al. "Sustainable ai: Environmental implications, challenges and opportunities." Proceedings of Machine Learning and Systems 4 (2022): 795-813.
[4] Patterson, David, Joseph Gonzalez, Urs Hölzle, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David R. So, Maud Texier, and Jeff Dean. "The carbon footprint of machine learning training will plateau, then shrink." Computer 55, no. 7 (2022): 18-28.
It would seem intuitive that if a process can be done more efficiently, and the cost of using it goes down, it should increase its value relative to other alternatives leading to greater usage. Some might call the resulting increase in total energy consumed a "rebound", but another view is that this is just growth.
Sustainability shouldn't be a cap on growth. As long as human populations continue to grow, so will their energy needs. Sustainability must also meet the human need for increasing energy demand.
The air conditioning efficiency example has gotten more complicated because of advances in AI. The humans who think they are in control may want to run the air conditioning at night, however the smart thermostats have the final say. These devices "learn" the weather and home temperature patterns and independently determine when to turn on or off regardless of what the humans want. They also connect to the utility provider and can respond to usage alerts that affect the monthly cost of electricity.
I'm afraid we have passed the singularity for air conditioning at this point.