IDataCool

Last updated

iDataCool is a high-performance computer cluster based on a modified IBM System x iDataPlex. The cluster serves as a research platform for cooling of IT equipment with hot water and efficient reuse of the waste heat. The project is carried out by the physics department of the University of Regensburg in collaboration with the IBM Research and Development Laboratory Böblingen and InvenSor. It is funded by the German Research Foundation (DFG), the German state of Bavaria, and IBM.

Contents

Overview

The iDataCool high-performance compute cluster is a research project on cooling with hot water and energy reuse in data centers. The waste heat of iDataCool is used to drive an adsorption chiller that generates chilled water. [1] The project pursues the following goals:

The iDataCool cluster has been operating with hot-water cooling since 2011. The infrastructure support for energy reuse was finished in 2012. Members of the project have also been active in other supercomputer projects such as QCDOC and QPACE. SuperMUC is based on the cooling technology invented for QPACE, Aquasar, and iDataCool. [2]

The iDataCool research project was presented at the International Supercomputing Conference in Leipzig, Germany, in 2013, [1] which led to it being featured in several articles. [3] [4]

Background and design target

Power and cooling of IT equipment are of major concern for modern data centers. Since 1996 the worldwide costs for power and cooling of IT infrastructure have increased by more than a factor of five. [5] Conventionally, data centers use air as the primary cooling medium for the IT equipment. While air cooling is simple and flexible it also has some disadvantages, e.g., limited packaging density and limited options for energy reuse. [6] Liquid cooling based on water as the coolant is another option. Since water has a very high heat capacity large amounts of heat can be removed from a system at moderate flow rates, thus allowing for a higher packaging density which in turn results in less floor space. Liquid cooling has recently resurfaced in the sector of high-performance computing. Since 2009 the Green500 list of the most energy-efficient supercomputers is dominated by liquid-cooled designs. [7]

If the design of the liquid-cooling system allows for high coolant temperatures, energy can be saved or even reused, depending on the climate conditions and on the local infrastructure. For example, free cooling is possible if the coolant temperature is higher than the ambient temperature. In that case the energy for chillers can be saved. If the coolant temperature is even higher, the waste heat from the compute equipment could be used for heating purposes or to drive an adsorption chiller to generate chilled water. The former option is implemented, e.g., by the Leibniz-Rechenzentrum in Germany, where SuperMUC drives the heating of the data center during winter with roughly 1 MW recovered from the compute equipment. The latter option, which is the design target of iDataCool, requires a high quality of the heat, which can only be achieved by direct hot-water cooling. One example for direct hot-water cooling is the Aquasar project at ETH Zürich, which is operated at coolant temperatures around 60°C. The aim of iDataCool was to achieve coolant temperatures of more than 65°C, at which commercially available adsorption chillers tend to become efficient, and to demonstrate the long-term stability of a large production machine under these conditions.

Architecture

The iDataCool installation at the University of Regensburg consists of three IBM System x iDataPlex [8] racks. Each rack contains 72 compute nodes. A compute node consists of two Intel Xeon Westmere server processors and is arranged as a distributed shared memory system with 24 GB of DDR3-SDRAM. Switched Infiniband is used for communication amongst the nodes. Gigabit Ethernet is used for disk I/O, system operation, and monitoring.

The original iDataPlex system is entirely cooled by air. The ambient air in the data center is drawn in through perforated front doors, and the hot air is blown back into the data center on the back side. Components that need cooling are the power supplies, network switches, and compute nodes. The power supplies and switches rely on built-in fans which generate the necessary air-flow, while fan blocks are used to draw the air over the compute nodes, which are equipped with passive heat sinks.

In a joint effort of the particle physics group of the University of Regensburg and the IBM Research and Development Laboratory Böblingen, Germany, a water-cooling solution for the compute nodes was developed which completely replaces the original fans and heat sinks. The processors are cooled by custom-designed copper heat sinks through which water flows directly. This minimizes the temperature difference between the compute cores and the coolant. A copper pipeline provides the water flow and is also thermally coupled to passive heat sinks for other components such as memory, chipset, and voltage converters.

All conversions of the original iDataPlex cluster were performed at the University of Regensburg. Newly developed parts were manufactured in the machine shop of the university's physics department. The data center of the university was extended to provide the liquid-cooling infrastructure. The system has been operating in stable production mode at coolant temperatures of up to 70°C since 2011.

Energy reuse

iDataCool allows for cooling with hot water at temperatures of up to 70°C. [1] The waste heat of iDataCool drives a low-temperature adsorption chiller (LTC 09 by InvenSor) that works efficiently already at temperatures around 65°C. The chiller generates chilled water that is used to cool other compute equipment in the data center. The installation was finished in the summer of 2012.

See also

Related Research Articles

<span class="mw-page-title-main">Supercomputer</span> Type of extremely powerful computer

A supercomputer is a type of computer with a high level of performance as compared to a general-purpose computer. The performance of a supercomputer is commonly measured in floating-point operations per second (FLOPS) instead of million instructions per second (MIPS). Since 2017, supercomputers have existed which can perform over 1017 FLOPS (a hundred quadrillion FLOPS, 100 petaFLOPS or 100 PFLOPS). For comparison, a desktop computer has performance in the range of hundreds of gigaFLOPS (1011) to tens of teraFLOPS (1013). Since November 2017, all of the world's fastest 500 supercomputers run on Linux-based operating systems. Additional research is being conducted in the United States, the European Union, Taiwan, Japan, and China to build faster, more powerful and technologically superior exascale supercomputers.

<span class="mw-page-title-main">Radiator</span> Type of heat exchanger; radiant body through water or other liquids

A radiator is a heat exchanger used to transfer thermal energy from one medium to another for the purpose of cooling and heating. The majority of radiators are constructed to function in cars, buildings, and electronics.

<span class="mw-page-title-main">Data center</span> Building or room used to house computer servers and related equipment

A data center or data centre is a building, a dedicated space within a building, or a group of buildings used to house computer systems and associated components, such as telecommunications and storage systems.

Internal combustion engine cooling uses either air or liquid to remove the waste heat from an internal combustion engine. For small or special purpose engines, cooling using air from the atmosphere makes for a lightweight and relatively simple system. Watercraft can use water directly from the surrounding environment to cool their engines. For water-cooled engines on aircraft and surface vehicles, waste heat is transferred from a closed loop of water pumped through the engine to the surrounding atmosphere by a radiator.

<span class="mw-page-title-main">Chiller</span> Machine that removes heat from a liquid coolant via vapor compression

A chiller is a machine that removes heat from a liquid coolant via a vapor-compression, adsorption refrigeration, or absorption refrigeration cycles. This liquid can then be circulated through a heat exchanger to cool equipment, or another process stream. As a necessary by-product, refrigeration creates waste heat that must be exhausted to ambience, or for greater efficiency, recovered for heating purposes. Vapor compression chillers may use any of a number of different types of compressors. Most common today are the hermetic scroll, semi-hermetic screw, or centrifugal compressors. The condensing side of the chiller can be either air or water cooled. Even when liquid cooled, the chiller is often cooled by an induced or forced draft cooling tower. Absorption and adsorption chillers require a heat source to function.

<span class="mw-page-title-main">Computer cooling</span> The process of removing waste heat from a computer

Computer cooling is required to remove the waste heat produced by computer components, to keep components within permissible operating temperature limits. Components that are susceptible to temporary malfunction or permanent failure if overheated include integrated circuits such as central processing units (CPUs), chipsets, graphics cards, hard disk drives, and solid state drives.

<span class="mw-page-title-main">Absorption refrigerator</span> Refrigerator that uses a heat source

An absorption refrigerator is a refrigerator that uses a heat source to provide the energy needed to drive the cooling process. Solar energy, burning a fossil fuel, waste heat from factories, and district heating systems are examples of convenient heat sources that can be used. An absorption refrigerator uses two coolants: the first coolant performs evaporative cooling and then is absorbed into the second coolant; heat is needed to reset the two coolants to their initial states. Absorption refrigerators are commonly used in recreational vehicles (RVs), campers, and caravans because the heat required to power them can be provided by a propane fuel burner, by a low-voltage DC electric heater or by a mains-powered electric heater. Absorption refrigerators can also be used to air-condition buildings using the waste heat from a gas turbine or water heater in the building. Using waste heat from a gas turbine makes the turbine very efficient because it first produces electricity, then hot water, and finally, air-conditioning—trigeneration.

Solar air conditioning, or "solar-powered air conditioning", refers to any air conditioning (cooling) system that uses solar power.

The IBM Intelligent Cluster was a cluster solution for x86-based high-performance computing composed primarily of IBM components, integrated with network switches from various vendors and optional high-performance InfiniBand interconnects.

<span class="mw-page-title-main">Server room</span> Room containing computer servers

A server room is a room, usually air-conditioned, devoted to the continuous operation of computer servers. An entire building or station devoted to this purpose is a data center.

Free cooling is an economical method of using low external air temperatures to assist in chilling water, which can then be used for industrial processes, or air conditioning systems. The chilled water can either be used immediately or be stored for the short- or long-term. When outdoor temperatures are lower relative to indoor temperatures, this system utilizes the cool outdoor air as a free cooling source. In this manner, the system replaces the chiller in traditional air conditioning systems while achieving the same cooling result. Such systems can be made for single buildings or district cooling networks.

QPACE is a massively parallel and scalable supercomputer designed for applications in lattice quantum chromodynamics.

<span class="mw-page-title-main">SciNet Consortium</span> Scientific research group between the University of Toronto and local hospitals

SciNet is a consortium of the University of Toronto and affiliated Ontario hospitals. It has received funding from both the federal and provincial government, Faculties at the University of Toronto, and affiliated hospitals.

SuperMUC was a supercomputer of the Leibniz Supercomputing Centre (LRZ) of the Bavarian Academy of Sciences. It was housed in the LRZ's data centre in Garching near Munich. It was decommissioned in January 2020, having been superseded by the more powerful SuperMUC-NG.

<span class="mw-page-title-main">Ice storage air conditioning</span>

Ice storage air conditioning is the process of using ice for thermal energy storage. The process can reduce energy used for cooling during times of peak electrical demand. Alternative power sources such as solar can also use the technology to store energy for later use. This is practical because of water's large heat of fusion: one metric ton of water can store 334 megajoules (MJ) of energy, equivalent to 93 kWh.

<span class="mw-page-title-main">Aquasar</span> Supercomputer system from IBM Research

Aquasar is a supercomputer prototype created by IBM Labs in collaboration with ETH Zurich in Zürich, Switzerland and ETH Lausanne in Lausanne, Switzerland. While most supercomputers use air as their coolant of choice, the Aquasar uses hot water to achieve its great computing efficiency. Along with using hot water as the main coolant, an air-cooled section is also included to be used to compare the cooling efficiency of both coolants. The comparison could later be used to help improve the hot water coolant's performance. The research program was first termed to be: "Direct use of waste heat from liquid-cooled supercomputers: the path to energy saving, emission-high performance computers and data centers." The waste heat produced by the cooling system is able to be recycled back in the building's heating system, potentially saving money. Beginning in 2009, the three-year collaborative project was introduced and developed in the interest of saving energy and being environmentally-safe while delivering top-tier performance.

<span class="mw-page-title-main">NCAR-Wyoming Supercomputing Center</span> High performance computing center in Wyoming, US

The NCAR-Wyoming Supercomputing Center (NWSC) is a high-performance computing (HPC) and data archival facility located in Cheyenne, Wyoming, that provides advanced computing services to researchers in the Earth system sciences.

<span class="mw-page-title-main">QPACE2</span> Massively parallel and scalable supercomputer

QPACE 2 is a massively parallel and scalable supercomputer. It was designed for applications in lattice quantum chromodynamics but is also suitable for a wider range of applications..

<span class="mw-page-title-main">Immersion cooling</span> IT cooling practice

Immersion cooling is an IT cooling practice by which complete servers are immersed in a dielectric, electrically non-conductive fluid that has significantly higher thermal conductivity than air. Heat is removed from a system by putting the coolant in direct contact with hot components, and circulating the heated liquid through heat exchangers. This practice is highly effective because liquid coolants can absorb more heat from the system, and are more easily circulated through the system, than air. Immersion cooling has many benefits, including but not limited to: sustainability, performance, reliability and cost.

<span class="mw-page-title-main">Green data center</span> Server facility which utilizes energy-efficient technologies

A green data center, or sustainable data center, is a service facility which utilizes energy-efficient technologies. They do not contain obsolete systems, and take advantage of newer, more efficient technologies.

References

  1. 1 2 3 N. Meyer et al., iDataCool: HPC with Hot-Water Cooling and Energy Reuse, Lecture Notes in Computer Science 7905 (2013) 383
  2. ↑ B. Michel et al., Aquasar: Der Weg zu optimal effizienten Rechenzentren [ permanent dead link ], 2011
  3. ↑ IEEE Spectrum, New Tech Keeps Data Centers Cool in Warm Climates, 26 June 2013
  4. ↑ R718.com, Waste heat driven adsorption chiller cools computing centre Archived 2014-01-17 at the Wayback Machine , 30 July 2013
  5. ↑ IDC, Worldwide Server Research, 2009
  6. ↑ N. Meyer et al., Data centre infrastructure requirements Archived 2017-06-03 at the Wayback Machine , European Exascale project DEEP, 2012
  7. ↑ The Green500 list, http://www.green500.org/ Archived 2016-08-26 at the Wayback Machine
  8. ↑ IBM System x iDataPlex dx360 M3,
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.