IDataCool

Last updated

iDataCool is a high-performance computer cluster based on a modified IBM System x iDataPlex. The cluster serves as a research platform for cooling of IT equipment with hot water and efficient reuse of the waste heat. The project is carried out by the physics department of the University of Regensburg in collaboration with the IBM Research and Development Laboratory Böblingen and InvenSor. It is funded by the German Research Foundation (DFG), the German state of Bavaria, and IBM.

Contents

Overview

The iDataCool high-performance compute cluster is a research project on cooling with hot water and energy reuse in data centers. The waste heat of iDataCool is used to drive an adsorption chiller that generates chilled water. [1] The project pursues the following goals:

The iDataCool cluster has been operating with hot-water cooling since 2011. The infrastructure support for energy reuse was finished in 2012. Members of the project have also been active in other supercomputer projects such as QCDOC and QPACE. SuperMUC is based on the cooling technology invented for QPACE, Aquasar, and iDataCool. [2]

The iDataCool research project was presented at the International Supercomputing Conference in Leipzig, Germany, in 2013, [1] which led to it being featured in several articles. [3] [4]

Background and design target

Power and cooling of IT equipment are of major concern for modern data centers. Since 1996 the worldwide costs for power and cooling of IT infrastructure have increased by more than a factor of five. [5] Conventionally, data centers use air as the primary cooling medium for the IT equipment. While air cooling is simple and flexible it also has some disadvantages, e.g., limited packaging density and limited options for energy reuse. [6] Liquid cooling based on water as the coolant is another option. Since water has a very high heat capacity large amounts of heat can be removed from a system at moderate flow rates, thus allowing for a higher packaging density which in turn results in less floor space. Liquid cooling has recently resurfaced in the sector of high-performance computing. Since 2009 the Green500 list of the most energy-efficient supercomputers is dominated by liquid-cooled designs. [7]

If the design of the liquid-cooling system allows for high coolant temperatures, energy can be saved or even reused, depending on the climate conditions and on the local infrastructure. For example, free cooling is possible if the coolant temperature is higher than the ambient temperature. In that case the energy for chillers can be saved. If the coolant temperature is even higher, the waste heat from the compute equipment could be used for heating purposes or to drive an adsorption chiller to generate chilled water. The former option is implemented, e.g., by the Leibniz-Rechenzentrum in Germany, where SuperMUC drives the heating of the data center during winter with roughly 1 MW recovered from the compute equipment. The latter option, which is the design target of iDataCool, requires a high quality of the heat, which can only be achieved by direct hot-water cooling. One example for direct hot-water cooling is the Aquasar project at ETH Zürich, which is operated at coolant temperatures around 60°C. The aim of iDataCool was to achieve coolant temperatures of more than 65°C, at which commercially available adsorption chillers tend to become efficient, and to demonstrate the long-term stability of a large production machine under these conditions.

Architecture

The iDataCool installation at the University of Regensburg consists of three IBM System x iDataPlex [8] racks. Each rack contains 72 compute nodes. A compute node consists of two Intel Xeon Westmere server processors and is arranged as a distributed shared memory system with 24 GB of DDR3-SDRAM. Switched Infiniband is used for communication amongst the nodes. Gigabit Ethernet is used for disk I/O, system operation, and monitoring.

The original iDataPlex system is entirely cooled by air. The ambient air in the data center is drawn in through perforated front doors, and the hot air is blown back into the data center on the back side. Components that need cooling are the power supplies, network switches, and compute nodes. The power supplies and switches rely on built-in fans which generate the necessary air-flow, while fan blocks are used to draw the air over the compute nodes, which are equipped with passive heat sinks.

In a joint effort of the particle physics group of the University of Regensburg and the IBM Research and Development Laboratory Böblingen, Germany, a water-cooling solution for the compute nodes was developed which completely replaces the original fans and heat sinks. The processors are cooled by custom-designed copper heat sinks through which water flows directly. This minimizes the temperature difference between the compute cores and the coolant. A copper pipeline provides the water flow and is also thermally coupled to passive heat sinks for other components such as memory, chipset, and voltage converters.

All conversions of the original iDataPlex cluster were performed at the University of Regensburg. Newly developed parts were manufactured in the machine shop of the university's physics department. The data center of the university was extended to provide the liquid-cooling infrastructure. The system has been operating in stable production mode at coolant temperatures of up to 70°C since 2011.

Energy reuse

iDataCool allows for cooling with hot water at temperatures of up to 70°C. [1] The waste heat of iDataCool drives a low-temperature adsorption chiller (LTC 09 by InvenSor) that works efficiently already at temperatures around 65°C. The chiller generates chilled water that is used to cool other compute equipment in the data center. The installation was finished in the summer of 2012.

See also

Related Research Articles

Water cooling Method of heat removal from components and industrial equipment

Water cooling is a method of heat removal from components and industrial equipment. Evaporative cooling using water is often more efficient than air cooling. Water is inexpensive and non-toxic however it can contain impurities and cause corrosion.

Data center Building or room used to house computer servers and related equipment

A data center or data centre is a building, dedicated space within a building, or a group of buildings used to house computer systems and associated components, such as telecommunications and storage systems.

Internal combustion engine cooling uses either air or liquid to remove the waste heat from an internal combustion engine. For small or special purpose engines, cooling using air from the atmosphere makes for a lightweight and relatively simple system. Watercraft can use water directly from the surrounding environment to cool their engines. For water-cooled engines on aircraft and surface vehicles, waste heat is transferred from a closed loop of water pumped through the engine to the surrounding atmosphere by a radiator.

Chiller

A chiller is a machine that removes heat from a liquid via a vapor-compression, Adsorption refrigeration, or absorption refrigeration cycles. This liquid can then be circulated through a heat exchanger to cool equipment, or another process stream. As a necessary by-product, refrigeration creates waste heat that must be exhausted to ambience, or for greater efficiency, recovered for heating purposes. Vapor compression chillers may use any of a number of different types of compressors. Most common today are the hermetic scroll, semi-hermetic screw, or centrifugal compressors. The condensing side of the chiller can be either air or water cooled. Even when water cooled, the chiller is often cooled by an induced or forced draft cooling tower. Absorption and adsorption chillers require a heat source to function.

Computer cooling

Computer cooling is required to remove the waste heat produced by computer components, to keep components within permissible operating temperature limits. Components that are susceptible to temporary malfunction or permanent failure if overheated include integrated circuits such as central processing units (CPUs), chipsets, graphics cards, and hard disk drives.

A coolant is a substance, typically liquid or gas, that is used to reduce or regulate the temperature of a system. An ideal coolant has high thermal capacity, low viscosity, is low-cost, non-toxic, chemically inert and neither causes nor promotes corrosion of the cooling system. Some applications also require the coolant to be an electrical insulator.

Absorption refrigerator

An absorption refrigerator is a refrigerator that uses a heat source to provide the energy needed to drive the cooling process. The system uses two coolants, the first of which performs evaporative cooling and is then absorbed into the second coolant; heat is needed to reset the two coolants to their initial states. The principle can also be used to air-condition buildings using the waste heat from a gas turbine or water heater. Using waste heat from a gas turbine makes the turbine very efficient because it first produces electricity, then hot water, and finally, air-conditioning—trigeneration. Absorption refrigerators are commonly used in recreational vehicles (RVs), campers, and caravans because they can be powered with propane fuel, rather than electricity. Unlike more common vapor-compression refrigeration systems, an absorption refrigerator can be produced with no moving parts other than the coolants.

Waste heat

Waste heat is heat that is produced by a machine, or other process that uses energy, as a byproduct of doing work. All such processes give off some waste heat as a fundamental result of the laws of thermodynamics. Waste heat has lower utility than the original energy source. Sources of waste heat include all manner of human activities, natural systems, and all organisms, for example, incandescent light bulbs get hot, a refrigerator warms the room air, a building gets hot during peak hours, an internal combustion engine generates high-temperature exhaust gases, and electronic components get warm when in operation.

Solar air conditioning refers to any air conditioning (cooling) system that uses solar power.

Server room Room containing computer servers

A server room is a room, usually air-conditioned, devoted to the continuous operation of computer servers. An entire building or station devoted to this purpose is a data center.

Free cooling is an economical method of using low external air temperatures to assist in chilling water, which can then be used for industrial processes, or air conditioning systems. The chilled water can either be used immediately or be stored for the short- or long-term. When outdoor temperatures are lower relative to indoor temperatures, this system utilizes the cool outdoor air as a free cooling source. In this manner, the system replaces the chiller in traditional air conditioning systems while achieving the same cooling result. Such systems can be made for single buildings or district cooling networks.

QPACE is a massively parallel and scalable supercomputer designed for applications in lattice quantum chromodynamics.

Radiator (engine cooling)

Radiators are heat exchangers used for cooling internal combustion engines, mainly in automobiles but also in piston-engined aircraft, railway locomotives, motorcycles, stationary generating plant or any similar use of such an engine.

SciNet Consortium

SciNet is a consortium of the University of Toronto and affiliated Ontario hospitals. It has received funding from both the federal and provincial government, Faculties at the University of Toronto, and affiliated hospitals.

HVAC is a major subdiscipline of mechanical engineering. The goal of HVAC design is to balance indoor environmental comfort with other factors such as installation cost, ease of maintenance, and energy efficiency. The discipline of HVAC includes a large number of specialized terms and acronyms, many of which are summarized in this glossary.

SuperMUC is a supercomputer of the Leibniz Supercomputing Centre (LRZ) of the Bavarian Academy of Sciences. It is housed in the LRZ's data centre in Garching near Munich.

Ice storage air conditioning is the process of using ice for thermal energy storage. This is practical because of water's large heat of fusion: one metric ton of water can store 334 megajoules (MJ) of energy, equivalent to 93 kWh.

Aquasar

Aquasar is a supercomputer prototype created by IBM Labs in collaboration with ETH Zurich in Zürich, Switzerland and ETH Lausanne in Lausanne, Switzerland. While most supercomputers use air as their coolant of choice, the Aquasar uses hot water to achieve its great computing efficiency. Along with using hot water as the main coolant, an air-cooled section is also included to be used to compare the cooling efficiency of both coolants. The comparison could later be used to help improve the hot water coolant's performance. The research program was first termed to be: "Direct use of waste heat from liquid-cooled supercomputers: the path to energy saving, emission-high performance computers and data centers." The waste heat produced by the cooling system is able to be recycled back in the building's heating system, potentially saving money. Beginning in 2009, the three-year collaborative project was introduced and developed in the interest of saving energy and being environmentally-safe while delivering top-tier performance.

Immersion cooling IT cooling practice

Immersion cooling is an IT cooling practice by which IT components and other electronics, including complete servers, are submerged in a thermally conductive dielectric liquid or coolant. Heat is removed from the system by circulating liquid into direct contact with hot components, then through cool heat exchangers. Fluids suitable for immersion cooling have very good insulating properties to ensure that they can safely come into contact with energized electronic components.

Green data center Server facility which utilizes energy-efficient technologies

A green data center, or sustainable data center, is a service facility which utilizes energy-efficient technologies. They do not contain obsolete systems, and take advantage of newer, more efficient technologies.

References

  1. 1 2 3 N. Meyer et al., iDataCool: HPC with Hot-Water Cooling and Energy Reuse, Lecture Notes in Computer Science 7905 (2013) 383
  2. ↑ B. Michel et al., Aquasar: Der Weg zu optimal effizienten Rechenzentren [ permanent dead link ], 2011
  3. ↑ IEEE Spectrum, New Tech Keeps Data Centers Cool in Warm Climates, 26 June 2013
  4. ↑ R718.com, Waste heat driven adsorption chiller cools computing centre, 30 July 2013
  5. ↑ IDC, Worldwide Server Research, 2009
  6. ↑ N. Meyer et al., Data centre infrastructure requirements, European Exascale project DEEP, 2012
  7. ↑ The Green500 list, http://www.green500.org/ Archived 2016-08-26 at the Wayback Machine
  8. ↑ IBM System x iDataPlex dx360 M3, http://www-03.ibm.com/systems/x/hardware/rack/dx360m3/index.html
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.