AI Data Center: A Complete Guide to AI Infrastructure, Power, Cooling, and Design
An AI data center is a data center designed to support high-performance AI workloads, including model training, inference, high-performance computing, and large-scale accelerated computing. Compared with conventional data centers, AI facilities must accommodate higher rack power, greater thermal density, faster capacity growth, and tighter integration between compute, power, cooling, and monitoring systems.
The defining difference is not simply the presence of GPUs. AI workloads change the infrastructure requirements around those GPUs. As compute density increases, power delivery and heat rejection become central design constraints, requiring a coordinated architecture rather than isolated upgrades to individual systems. Industry references describe this shift in terms of higher rack power, hybrid cooling architectures, and closer integration between infrastructure systems and workload requirements.
What Is an AI Data Center?
An AI data center is a purpose-designed or adapted data center infrastructure environment built to support AI and other accelerated-computing workloads.
Its core systems typically include:
- High-density compute infrastructure
- High-capacity power distribution and UPS systems
- Air or liquid cooling systems matched to the thermal load
- Network infrastructure
- Monitoring and control systems
- Physical security and fire protection
- Infrastructure designed for current and future workload density
An AI data center does not necessarily require one specific physical architecture. It can be built as a conventional facility, a modular or prefabricated data center, a containerized deployment, or a distributed AI infrastructure environment.
The architecture depends on the IT load, rack density, availability requirements, site power, cooling strategy, deployment schedule, and whether the project is a greenfield or brownfield deployment.
Why AI Workloads Change Data Center Design
Traditional data center design often starts with relatively predictable IT loads distributed across racks. AI infrastructure changes that assumption.
Accelerated-computing servers can concentrate a large amount of compute and power within a small physical footprint. A Schneider Electric industry reference notes that fully populated GPU racks have already reached power levels around 142 kW in some configurations, with higher-density systems expected to increase further. The same reference explains that direct-to-chip cooling addresses the primary heat sources but does not eliminate the need for supplementary air cooling for other components.
This creates four closely related design problems:
Power density. More electrical capacity must reach the rack without exceeding the limits of distribution equipment.
Thermal density. The heat generated by the IT load must be removed continuously and predictably.
Scalability. Infrastructure must support rapid increases in AI capacity without forcing every part of the facility to be rebuilt.
Operational integration. Power, cooling, monitoring, controls, and IT infrastructure increasingly need to be designed as one system.
The result is a shift from designing a room around IT equipment toward designing an integrated infrastructure around the workload.
AI Data Center Architecture
At a high level, an AI data center can be viewed as several interconnected infrastructure layers.
IT and Compute
The compute layer includes GPUs, CPUs, memory, storage, networking, and associated server infrastructure. AI training and inference workloads may produce much higher and more concentrated power and thermal loads than conventional enterprise applications.
The physical rack therefore becomes a critical engineering unit rather than simply a cabinet for mounting servers.
Power Infrastructure
Power infrastructure must deliver sufficient capacity to high-density racks while maintaining the required level of availability.
The power train can include:
- Utility or facility power
- Power conversion
- UPS systems
- Power distribution
- PDUs or rack-level distribution
- Backup power
- Monitoring and controls
There is no universal power architecture for AI facilities. The appropriate design depends on the IT load, availability target, rack density, expansion plan, and site infrastructure. Vertiv’s powertrain reference similarly emphasizes that data center power should be designed around both current and future requirements rather than a one-size-fits-all architecture.
Cooling Infrastructure
Cooling becomes one of the most significant differences between conventional and AI-oriented data centers.
Depending on rack density and facility conditions, an AI data center may use:
- Precision room cooling
- Row cooling
- Rack cooling
- Rear-door heat exchangers (RDHx)
- Direct-to-chip liquid cooling
- Immersion cooling
- Hybrid air and liquid cooling
The choice depends on the heat load, rack density, available water or heat-rejection infrastructure, facility design, and whether the project is new construction or a retrofit.
Monitoring and Control
AI infrastructure requires more than temperature monitoring.
A modern monitoring system may need to track:
- Power
- Cooling
- Temperature and humidity
- Water or leak detection
- Equipment alarms
- Environmental conditions
- Access and security
- Communication with BMS or other supervisory systems
ATTOM’s AgileView EMS, for example, supports UPS, cooling, smoke and water monitoring and can integrate with central BMS systems through Modbus-TCP or SNMP.
Power Architecture for AI Data Centers
Power design is often the first infrastructure constraint to become visible as AI rack density increases.
The challenge is not simply providing more electricity. The distribution system must deliver that capacity with the required redundancy, protection, monitoring, and room for future expansion.
For example, ATTOM’s AgileCore AI prefabricated data center platform includes configurations ranging from rack-based AI systems to larger containerized deployments. One documented high-density configuration provides 800 kW of IT load across 10 racks with direct-to-chip liquid cooling and two 1,000 kW modular UPS systems. Another configuration provides 500 kW across 10 racks using RDHx-based liquid cooling and two 600 kW modular UPS systems.
These examples illustrate an important design principle:
Power architecture and cooling architecture should be developed together.
Increasing IT capacity without accounting for the associated thermal load or distribution requirements simply moves the bottleneck elsewhere in the facility.
UPS and Power Distribution
UPS systems provide continuity during power disturbances and support the availability requirements of the IT load.
ATTOM’s AgilePower portfolio includes rack-mounted UPS systems from 1–60 kVA, as well as modular UPS systems from 20–600 kVA. The rack-mounted range includes online double-conversion designs and network management options such as SNMP.
For high-density AI infrastructure, the selection of UPS and distribution architecture should consider:
- Total IT load
- Rack density
- Redundancy requirements
- Future capacity
- Power quality
- Maintenance strategy
- Battery autonomy
- Available facility power
Cooling AI Data Centers
The thermal problem in AI infrastructure is fundamentally a heat-density problem.
As more compute is concentrated into each rack, simply supplying more room airflow does not necessarily provide the required thermal performance.
Liquid cooling addresses this by moving heat closer to the source rather than relying entirely on room air to transport it.
Direct-to-Chip Liquid Cooling
Direct-to-chip cooling transfers heat from high-power components through cold plates attached directly to the relevant chips or processors.
A typical system includes:
- Cold plates
- Manifolds
- Distribution hoses
- Coolant Distribution Units (CDUs)
- Heat exchangers
- Secondary cooling loops
- Heat-rejection equipment
ATTOM’s ByteCool system includes manifold kits designed to distribute coolant to multiple chips through individual lines. Its manifold uses stainless-steel construction and self-sealed quick connectors, while distribution hoses connect the manifold to individual cold plates.
At the CDU level, ATTOM’s documented portfolio includes rack-mounted units up to 100 kW and larger row-mounted configurations designed for significantly higher loads.
Direct-to-chip does not mean that every heat-producing component in a server becomes liquid cooled. Industry technical guidance notes that other components may continue to require supplemental air cooling, which is why hybrid cooling is common in high-density AI environments.
Rear-Door Heat Exchangers
RDHx provides another path for high-density cooling.
Instead of transferring heat directly from the chip, a rear-door heat exchanger removes heat from the hot exhaust air leaving the rack.
This approach can provide a transition between conventional air cooling and direct liquid cooling, depending on the rack density and facility architecture.
ATTOM’s SmoothAir platform includes DX and chilled-water RDHx configurations. Its cooling portfolio lists RDHx capacities in approximately the 25–30 kW range for split DX systems and 30–65 kW for chilled-water configurations.
Immersion Cooling
Immersion cooling places the IT equipment in direct contact with a dielectric coolant rather than using a cold plate attached only to selected components.
ATTOM’s OceanCool C Series uses a single-phase immersion architecture in which servers are immersed in dielectric liquid and heat is transferred through the liquid to a heat exchanger and then to heat-rejection equipment. The platform is designed for high-density applications and can be deployed as modular cooling cabinets with integrated CDU functions.
OceanCool’s R Series uses a modular immersion basin, in-rack coolant manifold and intelligent monitoring system, with leak detection and fluid-temperature monitoring built into the architecture.
Immersion cooling is therefore not simply a higher-capacity version of air cooling. It is a different thermal architecture with different hardware, fluid-management and operational requirements.
Choosing an AI Data Center Cooling Architecture
There is no single cooling technology that fits every AI deployment.
The appropriate architecture depends on several variables.
Rack Density
Lower-density AI workloads may still be served by advanced air cooling. As rack heat load increases, RDHx or liquid cooling becomes more relevant.
IT Load
Total IT load determines the amount of heat that must ultimately be rejected from the facility.
Existing Infrastructure
A greenfield AI data center can be designed around liquid cooling from the beginning. A brownfield site has to work within existing power, piping, cooling and space constraints. Schneider Electric’s guidance emphasizes auditing existing cooling capacity, current loads and future requirements before integrating liquid cooling into an existing facility.
Heat Rejection
Liquid cooling moves heat efficiently inside the IT environment, but the heat still has to leave the facility. The final architecture may therefore include chilled water, dry coolers, cooling towers, air-cooled chillers or other heat-rejection systems.
Availability
At high rack densities, cooling interruptions can quickly affect IT performance. Technical guidance for high-density liquid-cooled racks therefore emphasizes redundancy in CDUs, pumps and power supplies, together with leak detection and monitoring.
Prefabricated and Modular AI Data Centers
AI infrastructure does not always need to be built as a traditional large-scale facility.
Prefabricated and modular architectures can integrate IT infrastructure, power, cooling, monitoring and security into factory-engineered modules.
ATTOM’s AgileCore platform covers several deployment models, including rack-based AI micro data centers, liquid-cooled prefabricated systems, immersion-cooled systems and containerized AI infrastructure. The documented platform is designed around different workload scales rather than a single fixed architecture.
For example, ATTOM documents a rack-based configuration with 90 kW of IT load and 90 kW rack heat density using direct-chip liquid cooling, while another 50 kW configuration uses RDHx and water-cooled infrastructure.
At the larger end, the AgileCore containerized immersion platform includes 400 kW and 800 kW IT-load configurations at 100 kW per rack, with immersion cooling, air-cooled chillers, UPS and integrated monitoring.
The practical advantage of this approach is not simply speed. Factory integration can also make the relationship between power, cooling, monitoring and physical infrastructure more predictable before deployment.
Greenfield vs. Brownfield AI Data Centers
The design approach changes significantly depending on whether the facility is new or existing.
Greenfield AI Data Center
A greenfield project can establish:
- Utility capacity
- Electrical distribution
- Cooling topology
- Liquid cooling infrastructure
- Heat rejection
- Rack layout
- Monitoring
- Future expansion zones
from the beginning.
This provides greater freedom to design the facility around AI workload density.
Brownfield AI Upgrade
Existing facilities introduce additional constraints.
The project may need to work around:
- Existing power distribution
- Existing CRAC or CRAH equipment
- Existing piping
- Floor loading
- Rack space
- Network infrastructure
- Available electrical capacity
- Existing monitoring systems
This is why retrofitting AI workloads is not simply a matter of replacing servers.
The infrastructure must be evaluated as a system. Schneider Electric’s guidance specifically recommends auditing existing cooling capacity, current loads, future requirements and compatibility before introducing liquid cooling into an existing environment.
AI Data Center Monitoring and Operational Considerations
High-density AI infrastructure increases the value of real-time monitoring because the margin between normal operation and thermal or power constraints becomes smaller.
Monitoring should provide visibility into:
- IT load
- UPS status
- Cooling status
- Temperature
- Humidity
- Water
- Leaks
- Alarms
- Power consumption
- Environmental conditions
The monitoring layer should also integrate with the infrastructure it supervises.
ATTOM’s AgileView EMS supports Modbus-RTU, Modbus-TCP and SNMP, with integration to central BMS systems and third-party equipment such as UPS, generators and cooling systems.
For liquid-cooled AI environments, monitoring should extend into the cooling loop itself. Leak detection, fluid temperature, flow and pressure can become part of the operational picture.
What to Consider When Designing an AI Data Center
A practical AI data center design should answer several questions before equipment is selected.
How much IT power is required?
Start with the current workload, but also model the expected growth in rack density and total IT capacity.
How much heat must be removed?
Cooling capacity should be based on the actual IT load and the selected thermal architecture, not simply on room size.
Where should heat be removed?
The architecture may use room-level, row-level, rack-level or chip-level heat transfer.
Can the existing facility support the workload?
For brownfield projects, review power, cooling, space, piping, network and monitoring constraints before committing to the AI deployment.
What level of redundancy is required?
Power and cooling systems should be considered together with the required availability and maintenance strategy.
How will the infrastructure scale?
AI deployments can change quickly. Modular or prefabricated architectures can provide a way to add capacity in stages rather than sizing the entire infrastructure for the final workload on day one.
AI Data Center Efficiency: Beyond PUE
PUE remains a useful infrastructure metric, but it does not describe the full efficiency of an AI workload.
An AI data center can have a good PUE while still consuming substantial energy per unit of useful computation. Schneider Electric’s industry material argues for a broader view that connects chip performance, infrastructure performance and end-user value rather than relying on a single facility-level metric.
For AI infrastructure, efficiency should therefore be considered across:
- Compute utilization
- Power delivery
- Cooling efficiency
- Workload performance
- Infrastructure utilization
- Heat-rejection efficiency
- Facility energy consumption
The practical objective is not simply to reduce cooling power. It is to deliver the required AI workload with an infrastructure architecture that uses power and thermal resources effectively.
ATTOM AI Data Center Solutions
ATTOM approaches AI infrastructure as an integrated system rather than a standalone cooling or power product.
Its AI-oriented portfolio includes:
- Rack-based AI micro data centers
- Prefabricated AI data centers
- Containerized AI infrastructure
- Direct-to-chip liquid cooling
- RDHx cooling
- Immersion cooling
- Precision air cooling
- UPS and power distribution
- Environmental monitoring and BMS integration
The AgileCore platform documents configurations spanning rack-based systems, containerized liquid-cooled deployments and immersion-cooled AI infrastructure.
The appropriate configuration depends on the workload, rack density, power requirements, cooling architecture, site conditions and deployment model.
Frequently Asked Questions
What is an AI data center?
An AI data center is a data center designed or adapted to support AI and accelerated-computing workloads. Its infrastructure is typically designed around higher compute density, power requirements and thermal loads than conventional enterprise data center environments.
Why do AI data centers need liquid cooling?
High-density AI servers can generate more heat per rack than conventional air-cooling systems can efficiently remove. Liquid cooling transfers heat closer to the source and can support higher rack densities, although supplemental air cooling may still be required for components that are not liquid cooled.
What cooling technologies are used in AI data centers?
Depending on rack density and facility conditions, AI data centers may use precision air cooling, row cooling, rack cooling, RDHx, direct-to-chip liquid cooling, immersion cooling or hybrid combinations of these technologies.
How much power does an AI data center need?
There is no single power requirement. It depends on the number and type of AI servers, rack density, total IT capacity, redundancy and facility design. ATTOM’s documented AI prefabricated configurations range from rack-based systems to 400–800 kW containerized IT-load configurations.
Can an existing data center be upgraded for AI?
Yes, but the feasibility depends on available power, cooling capacity, space, piping, heat rejection, monitoring and other site constraints. Brownfield AI upgrades should begin with an assessment of existing infrastructure rather than starting with GPU deployment alone.
What is the role of a CDU in an AI data center?
A Coolant Distribution Unit manages the circulation and heat transfer of liquid within a liquid-cooling architecture. In direct-to-chip systems, the CDU provides an interface between the IT-side cooling loop and the facility-side heat-rejection system.
