The rapid evolution of artificial intelligence (AI) is driving an unprecedented demand for computational capacity. As AI systems, especially Generative AI models, grow in complexity, they require vast amounts of processing power, storage, and high-speed connectivity.
This demand is reshaping the data center industry, pushing operators to expand their infrastructure while navigating a myriad of challenges, including land availability, power supply, and the need for greater fiber connectivity, both between data centers and into the fixed and mobile access networks where AI applications ultimately will be utilized.
AI Accelerates IT Capacity Demands
The global AI boom has significantly increased demand for data center capacity.
Much of this growth is fueled by hyperscale data centers operated by tech giants like Microsoft, Google, and Amazon Web Services (AWS). These facilities are essential for training large language models (LLMs) and running AI workloads that process very high volumes of data, measured in petabytes (that’s, 1,000,000 gigabytes!)
AI training and inference workloads are computationally intensive, often requiring specialized hardware such as graphics processing units (GPUs) and tensor processing units (TPUs). A TPU is an application-specific integrated circuit (ASIC) developed by Google for accelerating machine learning workloads.
These systems consume vast amounts of power, contributing to the surging energy demands of data centers. GPUs and TPUs also require extensive cooling, prompting data center operators to consider moving from air cooling to liquid cooling that is applied directly to the processor chips.
Challenges for Data Center Operators
The U.S. data center market comprises several large, concentrated campuses, or data center alleys, as the Northern Virginia area is dubbed. Northern Virginia is the largest data center market in the world according to global commercial real estate services and investment company, CBRE. Additional markets including Silicon Valley, CA, Hillsboro, OR, Dallas-Ft. Worth, and Chicago round out the top markets in the country.
Limits to Growth
Land Scarcity
In regions like Northern Virginia and Silicon Valley, available land is dwindling.
As the need for more data center capacity grows, finding sufficient land in desirable locations poses a major challenge. Prime sites near major urban centers often come with high costs and intense competition. Proximity to existing power and fiber networks further limits the pool of viable sites.
Power Shortages
Lack of adequate power is perhaps the most critical constraint on data center growth. AI-centric data centers often require hundreds of megawatts of power capacity to support their operations. However, power grids in major hubs are increasingly stressed, not so much from a power generation aspect, but rather the transmission and distribution (T&D) infrastructure needed to deliver power to the site. Utilities have raised the flag on expanding T&D capacity in key markets such as Northern Virginia and Silicon Valley, advising that upgrading the T&D infrastructure could take several years before adequate power connections are available.
Data center operators are leaning towards integrating renewable energy sources in key markets which only adds complexity to this issue. While many operators are committed to achieving carbon-neutral operations, sourcing renewable power at scale can be challenging, especially in urban areas where space to deploy solar and wind resources is limited.
Fiber Connectivity
Data centers require high-capacity fiber optic connectivity, typically with multi-strand dark fiber, to link them to other domestic data centers and into global networks. Finding locations with sufficient existing fiber infrastructure is becoming increasingly difficult, particularly as demand shifts toward secondary markets. Installing new fiber networks is capital-intensive with lengthy permitting and installation timelines that can affect data center projects, further complicating site selection.
Cooling
Conventional data centers use air to cool rows of racks of CPUs. Air is pumped under raised floors and through equipment racks in data center halls. A rack of CPUs consumes from 1 kW to 10 kW of power. High-powered GPUs can require several 10s of kW per rack. That increased power calls for a different approach to cooling the equipment. The latest cooling systems flow chilled water through manifolds that are mounted directly over the GPU processors. These new cooling systems will add to the data center construction and operating costs.
Addressing the Power Supply Challenge
Meeting the power demands of AI-focused data centers (which some industry pundits have dubbed, AI factories) has prompted data center operators to consider alternate solutions.
On-Site Renewable Energy Generation
Data centers are increasingly investing in on-site renewable energy projects, such as solar farms and wind turbines. These facilities require land but once operational can reduce reliance on traditional grids while aligning with sustainability goals. Google has invested in several solar farms to power its data centers in the U.S.
Energy Storage Systems
Battery energy storage systems are being deployed to enhance grid reliability and support integration of renewable power sources. These systems allow data centers to store excess energy generated from wind and solar sources and use it during peak demand periods. For example, Tesla’s Megapack batteries have been adopted by several data center operators for this purpose. The downside with batteries is that they are heavy and take up space, require continuous monitoring, must be regularly maintained and then properly disposed of at the end of their useful life.
Microgrids
Microgrids integrate distributed energy resources like solar panels, batteries, and backup generators. They offer a localized solution to power supply challenges. Microgrids can operate independently of the main power grid, ensuring uninterrupted operations even during outages.
Natural Gas Turbines
Natural gas turbine technology is well established and can be cost effective where the data center is in close proximity to a natural gas pipeline. These turbines can be deployed on the data center site and connect directly to the pipeline.
Hydrogen Fuel Cells
Hydrogen fuel cells represent another innovative solution. These systems produce electricity through a chemical reaction between hydrogen and oxygen, emitting only water as a byproduct. Data center operators, including Microsoft, are piloting hydrogen fuel cells to replace diesel generators for backup power.
Nuclear Energy
Small modular reactors (SMRs) are receiving more scrutiny as a potential long term solution for powering large data centers. SMRs offer a reliable and carbon-free energy source with a much smaller footprint than traditional nuclear plants. Microsoft, in collaboration with nuclear technology companies, has explored using SMRs for its future facilities. Assessing performance, safety, and reliability along with upfront and operating costs, likely will take a number of years before SMRs are widely used.
The Shift to Tier 2 Markets
As land, power, and connectivity challenges mount in major data center hubs, operators are increasingly turning to selective Tier 2 markets. These are smaller cities and regions outside major metro areas. Tier 2 markets offer several advantages: lower power costs, greater availability of land, and less congested infrastructure. These markets are expected to host substantial portions of the projected capacity additions through 2028.
Phoenix, AZ
Phoenix has emerged as a key Tier 2 market due to its affordable land, robust power infrastructure, and favorable climate for data center cooling. Major players like Google and Amazon have established significant operations in the area.
Atlanta, GA
Atlanta is the largest and most important data center hub in the southeastern U.S. It offers space to expand and sits at the junction of major fiber routes that extend west to Dallas and north to Northern Virginia. Atlanta is approaching such size as to be rated as a Tier 1 market.
Salt Lake City, UT
Salt Lake City offers a combination of low energy costs, abundant renewable resources, and is strategically located to major Western U.S. markets.
Columbus, OH
Columbus has attracted attention due to its central location between major markets in Northern Virginia, New York/New Jersey and Chicago. Columbus offers connectivity to fiber routes, and access to renewable energy. Google recently announced plans to build several large data centers in the region.
Austin, San Antonio and Houston, TX
These Texas cities are becoming attractive alternatives to the big Dallas data center hub. They each offer competitive land costs, reliable power grids, access to extensive fiber networks and are home to a growing tech savvy workforce.
Reno, NV
Reno benefits from Nevada’s business-friendly policies, low taxes, and access to renewable energy sources like geothermal. Its proximity to California also makes it an attractive option for operators looking to avoid the challenges of building in Silicon Valley.
Key Players Driving Investments
Hyperscale Cloud Providers
AWS, Microsoft Azure, and Google Cloud are expected to lead the charge. AWS, for example, announced investments of over $100 billion globally in data center infrastructure through 2030, with a significant portion of that investment focused on AI.
These companies are also building proprietary AI hardware, further enhancing their own data center capabilities.
Colocation Providers
Companies like Equinix, Digital Realty and DataBank are expanding their portfolios to support enterprises adopting AI. Digital Realty and DataBank plan to invest heavily in facilities to handle greater GPU workloads; Equinix is focusing on modular and edge data centers for AI.
Enterprise Investments
JLL Research suggests that Enterprises in finance, healthcare, and technology are expected to increase their spending on private AI infrastructure, especially for sensitive workloads that require on-premises or hybrid setups.
Challenges in Meeting Demand
In addition to power constraints, all the data center operators must deal with supply chain issues. These involve shortages of GPUs, semiconductors, and other critical hardware that could delay deployments. Certainly, all data center deployments face regulatory hurdles such as stringent zoning and environmental regulations, particularly concerning water usage in cooling systems, that could also impact construction timelines.
Future Data Center Developments
The next three to five years will see continued growth in AI processing demands, particularly in the U.S. market. This growth will necessitate innovations in both technology and site selection strategies. Operators must balance the need for scalability with sustainability goals and economic constraints.
Key Trends to Watch
Hybrid Approaches to Power Supply
Operators are likely to adopt a mix of renewable energy, energy storage, and emerging technologies like hydrogen and possibly, SMRs to meet sustainability goals.
Modular Data Centers
Smaller, distributed facilities will play a larger role in supporting AI workloads, particularly in regions with less developed infrastructure.
Collaboration with Utilities
Data center operators will increasingly partner with utility companies to ensure supply commitment and as a source to co-develop power solutions, including grid upgrades and renewable energy projects.
Focus on Energy Efficiency
Investments in energy-efficient hardware and cooling technologies will become more critical as operators seek to manage costs and reduce environmental impact.
Implications for the AI Digital Infrastructure Ecosystem
It all ties together. If you log on to the internet from a desktop computer in your office or home, stream a movie on a smart TV, scroll through social media or make a call from your smartphone, purchase an item or execute a transaction of any type from your laptop or smartphone, it all ends up in a data center … somewhere.
Indoors, those signals are carried either through a wired Ethernet connection or via a wireless WiFi connection that is routed over a fiber-to-the-home facility to the network core which is housed in a data center. Outdoors, our mobile devices (smartphones, laptops, wearables) connect to the nearest marco cell or small cell. That signal is then carried over a fiber backhaul connection to the network core which resides in a data center.
In most instances, we are downloading more than we are uploading. However, with the rise of AI inferencing and AI-based applications, we could see more uploading activity especially if large numbers of AI-enabled devices, like the new iPhone 16 with Apple Intelligence, are generating steady video content.
While the thesis of this article is around challenges to expand capacity at data centers, all aspects of the digital infrastructure ecosystem ultimately must expand to support that challenge. This means faster, high capacity fiber optic transport and more robust connectivity in both wireline and wireless access networks that support increasingly smarter, and more powerful, AI-enabled devices.
Data Center Capacity and Capex Growth Forecast
When looking ahead at the data center market, there are two key considerations: capacity growth, measured in GW; and, capital investment, measured in billions of dollars, needed to support that growth.
Capacity Growth
As AI adoption gains traction across industries, the demand for data center capacity in the U.S. is expected to grow at low to mid double-digit rates over the next 3–5 years. This trend is driven by generative AI applications, large language models, autonomous systems, and advanced analytics requiring substantial computing power and infrastructure. Meeting this rising demand will necessitate significant capital expenditure by cloud providers, colocation facilities, and enterprises alike.
The primary factor fueling the demand for AI data centers is the computational intensity of training and deploying large-scale AI models. AI workloads are growing rapidly, with data center operators estimating that the computational requirements of state-of-the-art models are doubling every 6–9 months. These workloads are being driven by:
- AI Model Growth: Training advanced AI models like GPT-4 or GPT-5 involve processing vast datasets and requires high-density GPU and AI-specific hardware. Each training session can consume tens of petaflops of compute power, and deployment at scale demands similar infrastructure.
- Generative AI and Enterprise Applications: AI adoption across sectors like healthcare, finance,retail, and manufacturing is accelerating. Enterprises are integrating AI for personalized customer services, predictive analytics, and process automation, further increasing reliance on data centers.
- Edge AI: Growth in edge computing for low-latency AI applications in IoT, automotive, and telecommunications is augmenting demand for modular data centers alongside hyperscale facilities.
Industry analysts project that AI-related workloads will constitute 25–30 percent of total data center workloads by 2028, up from 10–15 percent in 2023. To accommodate this, hyperscale cloud providers and colocation facilities must expand capacity significantly.
Starting from a base installed capacity that we estimate was approximately 27 GW in 2023, the next few years are expected to reflect major expansions, especially by hyperscalers, as AI workloads drive demand for higher-density installations, reaching an estimated 52 GW by the end of 2028.
At that point, the U.S. data center market will realize:
- A threefold increase in AI-optimized server racks to support higher-density, liquid-cooled GPUs and TPUs.
- An estimated 20–30 GW of total power capacity growth over the forecast period will be largely for AI-specific workloads. This marks a significant shift from traditional CPU workloads, which are less energy-intensive. Hyperscalers like Amazon, Microsoft, Google, and Meta are projecting collective capacity increases from 9 GW in 2023 to over 26 GW by 2028.
- Expansion of colocation facilities to cater to enterprises seeking to
- Challenges to achieving this capacity growth remain:
- Power supply constraints, particularly in major hubs like Northern Virginia, necessitate investments in renewable energy, microgrids, and innovative solutions like small modular reactors.
- New cooling systems capable of managing extreme power densities of 20-30 kW per rack (or more) are essential.
- The rise of Tier 2 and 3 markets addresses land and cost issues but requires new fiber and energy infrastructure.
Capital Investment
The U.S. data center industry is expected to reach a cumulative capital expenditure of $275-300 billion between 2024 and 2028 to meet AI-driven capacity demand. This investment includes:
- Infrastructure Expansion: Building new hyperscale facilities and retrofitting existing data centers to accommodate high-density AI hardware.
- Substantial Cost: Constructing a large hyperscale data center can cost upwards of $1 billion. Data centers operators report that expanding capacity in existing facilities by adding AI-optimized equipment racks is running at over $10 million per megawatt.
- Energy Solutions: Investments in renewable energy and grid optimization are needed to augment the power demands of AI workloads. For example, Google and Microsoft hyperscale facilities are prioritizing 100 percent renewable energy, requiring investment in on-site solar, wind farms, and battery storage.
- Cooling Systems: Liquid cooling systems are becoming essential for handling the heat generated by high-performance AI hardware. Cooling represents a significant and growing share of capex.
- Networking Infrastructure: AI workloads require high-speed, low-latency networking, necessitating significant upgrades in fiber connectivity and in-network storage solutions.
Key Takeaways
The U.S. is poised for a transformative expansion in AI data center capacity over the next 3–5 years, fueled by the relentless growth of AI applications and the associated computational demands.
While the challenges of land, power, and connectivity are significant, innovative solutions and strategic site selection can help operators meet future needs.
The shift toward Tier 2 markets, coupled with advancements in power technologies, promises to unlock new opportunities for sustainable growth. By embracing these trends, the data center industry can continue to support the AI revolution while navigating the complexities of an evolving infrastructure ecosystem.
With projected aggregate capex of up to $275-300 billion, this period will define the next generation of data center infrastructure. Hyperscale providers, colocation operators, and enterprises will drive this growth while navigating challenges around power, cooling, and supply chains.
For deep dives into the digital infrastructure ecosystem, equip your company with Intelligence, a quarterly market report by Inside Towers™. Subscribe Today
Reader Interactions