A Step towards Decarbonizing Data Centers

On February 28, 2023, KDDI Corporation (KDDI), Mitsubishi Heavy Industries, Ltd. (MHI), and NEC Networks & System Integration Corporation (NESIC) conducted a demonstration testing of an immersion cooling system for potential applications of liquid cooling of IT components in a hyper-scale data center. The test protocol, targeting sustainable data centers that will contribute to decarbonization, successfully achieved Tier 4 (Tier is a measure of the assessed quality and rating level of a data center. Tier 4 indicates that, based on the redundancy of its auxiliary components, etc., a data center maintains a high level of quality) stable operation in a liquid-cooled unit. Compared to conventionally cooled data centers, immersion cooling reduced energy usage for cooling of the servers by 94% and achieved a PUE (Power Usage Effectiveness or PUE is a general measure of the energy efficiency of a data center), a metric indicating the energy efficiency of a data center, of 1.05. The demonstration testing was conducted at KDDI’s Oyama Network Center (NC).

KDDI, MHI and NESIC will commence provision of immersion cooling data centers within FY2023, anticipating a wide variety of applications from hyper-scale data centers to containerized data centers (Small-scale data centers offering mobility to locations closer to the end users).

Overview of the demonstration test

Background and challenges

Today, demand for cloud services and data centers is expanding rapidly as a result of global advances in digital transformation (DX), changes in workstyles amid the Covid-19 pandemic, and Business Continuity Planning (BCP) in preparation for natural disasters.

As IT components of data centers have become increasingly advanced in performance and density, they generate unprecedented volumes of heat, and dealing with this waste heat presents a challenge. Two factors are necessary in order to achieve sustainable data centers: reduced energy consumption and excellent waste heat efficiency. Calls are therefore being heard for data centers that consume less energy, to be achieved using high-efficiency cooling systems to cool the heat generated by their servers, and that have minimized impact on the environment.

By integrating the technologies and expertise of the three collaborating companies each specialized in its respective areas of social infrastructure, the testing confirmed the immersion cooling data center’s stability and viability.

Demonstration outline

On April 1, 2022, demonstration testing got underway at KDDI’s Oyama NC of a data center incorporating 100kVA class servers and other IT components and an immersion cooling system. Using a newly developed free cooling unit that performs optimized air cooling, the test demonstrated reduced energy consumption and improved waste heat treatment. In addition, high availability was obtained in the immersion cooling and free cooling units, confirming stable operation and viability of use of such equipment to achieve a Tier 4 level immersion cooling data center.

Results

Substantiation of cooling efficiency

Using the newly developed immersion cooling system, incorporating a free cooling unit to perform optimized cooling by outside air, and assuming its installation in a data center, the testing resulted in higher waste heat treatment capacity and less energy consumption. Energy consumed to cool the servers was cut by 94%, with a PUE of 1.05 (Compared to the total power consumed by the typical data centers currently in use – PUE: 1.7).

High availability

High availability was achieved in the immersion cooling and free cooling units, and engineering was developed for their installation in a Tier 4 immersion cooling data center. The demonstration substantiated their stable operation and viability. In addition, compared to air-cooled systems, noise generated by the IT components was reduced by approximately 35dB (dB: decibel is the unit used to measure noise level).

Preparation of maintenance manual in anticipation of commercialization

In anticipation of commercial use in the domestic market, a maintenance system was taken under consideration and a maintenance manual was  prepared.

Roles of the 3 companies

KDDI: Management to ensure smooth demonstration test operation.

• System engineering for applying immersion cooling to the data center, taking availability into account.
• Field work in preparation for installing IT components, and resolution of issues relating to maintenance and operation.

MHI: Development and trial production of the free cooling unit.

• Immersion cooling system engineering and fabrication.
• Immersion cooling system control and trial operation.
• Maintenance and operation engineering of the free cooling unit.

NESIC: Immersion cooling data center component deployment engineering, problem identification and improvement.

• Procurement of immersion cooling units, power supply, etc.; engineering; problem identification and improvement through construction.
• Verification of monitoring, management and control methods through SI engineering and fabrication of an integrated monitoring system.
• Establishment of optimal maintenance engineering, operation, and maintenance scheme.

Cooperating companies in the demonstrating test

The following companies were involved: AMD Japan, Ltd.; Arista Networks Japan LLC; Cisco Systems G.K.;DC ASIA Ltd.; Dell Technologies Japan Inc.; ENEOS Holdings, Inc.; FXC, Inc.; Giga Computing Technology Co., Ltd.; Hewlett Packard Japan, G.K.; Industrial Technology Research Institute (ITRI); Intel Corporation; MiTAC Computing Technology Corp.; Nihon Form Service Co., Ltd.; NVIDIA Corporation; Panduit Corporation Japan Branch; Solidigm Japan; Super Micro Computer, Inc.; and Western Digital Corporation.

Going forward, the three companies will apply the results of the demonstration testing as a way of contributing to the advancement of DX in Japan, decarbonization, and protection of the global environment.