The National Center for Computational Sciences (NCCS) is a United States Department of Energy (DOE) Leadership Computing Facility that houses the Oak Ridge Leadership Computing Facility (OLCF), a DOE Office of Science User Facility charged with helping researchers solve challenging scientific problems of global interest with a combination of leading high-performance computing (HPC) resources and international expertise in scientific computing. [1]
The NCCS provides resources for calculation and simulation in fields including astrophysics, materials science, and climate research to users from government, academia, and industry who have many of the largest computing problems in science. [2]
The OLCF’s flagship supercomputer, the IBM AC922 Summit, is supported by advanced data management and analysis tools. The center hosted the Cray XK7 Titan system, one of the most powerful scientific tools of its time, from 2012 through its retirement in August 2019. The same year, construction began for Frontier, which is slated to debut as the OLCF’s first exascale system in 2021. [3]
On December 9, 1991, Congress signed the High-Performance Computing Act (HPCA) of 1991, created by Senator Al Gore. HPCA proposed a national information infrastructure to build communications networks and databases and also called for proposals to build new high-performance computing facilities to serve science. [4]
On May 24, 1992, ORNL was awarded a high-performance computing research center called the Center for Computational Sciences, or CCS, as part of HPCA. [5] ORNL also received a 66-processor, serial #1 Intel Paragon XP/S 5 for code development the same year. The system had a peak performance of 5 gigaflops (5 billion floating-point operations per second).
Oak Ridge National Laboratory (ORNL) joined with three other national laboratories and seven universities to submit the Partnership in Computational Science (PICS) proposal to the US Department of Energy as part of the High-Performance Computing and Communications Initiative. [6] [7]
With the High-End Computing Revitalization Act of 2004, CCS was tasked with carrying out the Leadership Computing Facility (LCF) Project at ORNL with the goal of developing and installing a petaflops-speed supercomputer by the end of 2008. [8] The center officially changed its name from the Center for Computational Sciences to NCCS the same year.
On December 9, 2019, Georgia Tourassi–who previously served as the director of ORNL's Health Data Sciences Institute and as group leader for ORNL’s Biomedical Sciences, Engineering, and Computing Group–was appointed to director of the NCCS, succeeding James Hack. [9]
The creation of the CCS in 1992 ushered in a series of Intel Paragon computers, including:
Eagle was a 184-node IBM RS/6000 SP operated by the Computer Science and Mathematics Division of ORNL. It had 176 Winterhawk-II “thin” nodes, each with four 375 MHz Power3-II processors and 2 GB of memory. Eagle also had eight Winterhawk-II “wide” nodes - each with two 375 MHz Power3-II processors and 2 GB of memory—for use as filesystem servers and other infrastructure tasks. Eagle’s estimated computational power was greater than 1 teraflop in the compute partition.
Falcon was a 64-node Compaq AlphaServer SC operated by the CCS and acquired as part of an early-evaluation project. It had four 667 MHz Alpha EV67 processors with 2 GB of memory per node and 2 TB of Fiber Channel disk attached, resulting in an estimated computational power of 342 gigaflops.
Cheetah was a 4.5 TF IBM pSeries System operated by the CCS. The compute partition of Cheetah included 27 p690 nodes, each with thirty-two 1.3 GHz Power4 processors. The login and I/O partitions together included 8 p655 nodes, each with four 1.7 GHz Power4 processors. All nodes were connected via IBM’s Federation interconnect.
The Power4 memory hierarchy consisted of three levels of cache. The first and second levels were on the Power4 chip (two processors to a chip). Level-1 instruction cache was 128 KB (64 KB per processor) and the data cache was 64 KB (32 KB per processor.) The level-2 cache was 1.5 MB shared between the two processors. The level 3 cache was 32 MB and was off-chip. There were 16 chips per node, or 32 processors.
Most of Cheetah’s compute nodes had 32 GB of memory. Five had 64 GB of memory and two had 128 GB of memory. Some of the nodes in Cheetah had approximately 160 GB of local disk space that could be used as temporary scratch space.
In June 2002, Cheetah was ranked the eighth-fastest computer in the world, according to TOP500, the semi-annual list of the world's top supercomputers. [19]
Ram was an SGI Altix supercomputer provided as a support system for the NCCS.
Ram was installed in 2003 and was used as a pre- and post-processing support system for allocated NCCS projects until 2007.
Ram had 256 Intel Itanium2 processors running at 1.5 GHz, each with 6 MB of L3 cache, 256K of L2 cache, and 32K of L1 cache. Ram had 8 GB of memory per processor for a total of 2 TB of shared memory. By contrast, the first supercomputer at ORNL, the Cray XMP installed in 1985, had one-millionth the memory of the SGI Altix.
Phoenix was a Cray X1E provided as a primary system in NCCS.
The original X1 was installed in 2003 and went through several upgrades, arriving at its final configuration in 2005. From October 2005 until 2008, it provided almost 17 million processor-hours. The system supported more than 40 large projects in research areas including climate, combustion, high energy physics, fusion, chemistry, computer science, materials science, and astrophysics.
At its final configuration, Phoenix had 1,024 multistreaming vector processors (MSPs). Each MSP had 2 MB of cache and a peak computation rate of 18 gigaflops. Four MSPs formed a node with 8 GB of shared memory. Memory bandwidth was very high, roughly half the cache bandwidth. The interconnect functioned as an extension of the memory system, offering each node direct access to memory on other nodes at high bandwidth and low latency.
Jaguar began as a 25-teraflop Cray XT3 in 2005. Later, it was upgraded to an XT4 containing 7,832 compute nodes, each containing a quad-core AMD Opteron 1354 processor running at 2.1 GHz, 8 GB of DDR2-800 memory (some nodes used DDR2-667 memory), and a SeaStar2 router. The resulting partition contained 31,328 processing cores, more than 62 TB of memory, more than 600 TB of disk space, and a peak performance of 263 teraflops (263 trillion floating point operations per second).
In 2008, Jaguar was upgraded to a Cray XT5 and became the first system to run a scientific application at a sustained petaflop. By the time of its ultimate transformation into Titan in 2012, [23] Jaguar contained nearly 300,000 processing cores and had a theoretical performance peak of 3.3 petaflops. Jaguar had 224,256 x86-based AMD Opteron processor cores and operated with a version of Linux called the Cray Linux Environment.
From November 2009 until November 2010, Jaguar was the world's most powerful computer.
Hawk was a 64-node Linux cluster dedicated to high-end visualization.
Hawk was installed in 2006 and was used as the Center’s primary visualization cluster until May 2008 when it was replaced by a 512-core system named Lens. [25]
Each node contained two single-core Opteron processors and 2 GB of memory. The cluster was connected by a Quadrics Elan3 network, providing high-bandwidth and low-latency communication. The cluster was populated with two flavors of NVIDIA graphics cards connected with AGP8x: 5900 and QuadroFX 3000G. Nodes with 3000G cards were directly connected to the EVEREST PowerWall and were reserved for PowerWall use.
Ewok was an Intel-based InfiniBand cluster running Linux. The system was provided as an end-to-end resource for center users. It was used for workflow automation for jobs running from the Jaguar supercomputer and for advanced data analysis. The system contained 81 nodes. Each node contained two 3.4 GHz Pentium IV processors, a 3.4 GHz Intel Xeon central processing unit (CPU), and 6 GB of memory. An additional node contained 4 dual-core AMD processors and 64 GB of memory. The system was configured with a 13 TB Lustre file system for scratch space.
Eugene was a 27-teraflop IBM Blue Gene/P System operated by NCCS. It provided approximately 45 million processor-hours yearly for ORNL staff and for the promotion of research collaborations between ORNL and its core university partner members.
The system consisted of 2,048 850 Mhz IBM quad-core 450d PowerPC processors and 2 GB of memory per each node. Eugene had 64 I/O nodes; each submitted job was required to use at least one I/O node. This means that each job consumed a minimum of 32 nodes per execution.
Eugene was officially decommissioned in October 2011. However, on December 13 of the same year, a portion of Eugene’s hardware was donated to Argonne Leadership Computing Facility (ALCF) at Argonne National Laboratory. [28]
Eos was a 736-node Cray XC30 cluster with a total of 47.104 TB of memory. Its processor was the Intel Xeon E5-2670. It featured 16 I/O service nodes and 2 external login nodes. Its compute nodes were organized in blades. Each blade contained 4 nodes. Every node had 2 sockets with 8 physical cores each. Intel’s HyperThreading (HT) Technology allowed each physical core to work as 2 logical cores so each node could function as if it had 32 cores. In total, the Eos compute partition contained 11,776 traditional processor cores (23,552 logical cores with HT Technology enabled). [29]
Eos provided a space for tool and application porting, small scale jobs to prepare capability runs on Titan, as well as software generation, verification, and optimization. [30]
Titan was a hybrid-architecture Cray XK7 system with a theoretical peak performance exceeding 27,000 trillion calculations per second (27 petaflops). It contained both advanced 16-core AMD Opteron CPUs and NVIDIA Kepler graphics processing units (GPUs). This combination allowed Titan to achieve 10 times the speed and 5 times the energy efficiency of its predecessor, the Jaguar supercomputer, while using only modestly more energy and occupying the same physical footprint. [31]
Titan featured 18,688 compute nodes, a total system memory of 710 TB, and Cray’s high-performance Gemini network. Its 299,008 CPU cores guided simulations and the accompanying GPUs handled hundreds of calculations simultaneously. The system provided decreased time to solution, increased complexity of models, and greater realism in simulations. [32] In November 2012, Titan received the Number 1 position on the TOP500 supercomputer list. [33]
After 7 years of service, Titan was decommissioned in August 2019 to make room for the Frontier supercomputer. [34]
The OLCF’s center-wide Lustre file system, called Spider, is the operational work file system for most OLCF computational resources. As an extremely high-performance system, Spider has over 20,000 clients, providing 32 PB of disk space, and it can move data at more than 1 TB/s. Spider comprises two filesystems, Atlas1 and Atlas2, in order to provide high availability and load balance across multiple metadata servers for increased performance. [35]
HPSS, ORNL’s archival mass-storage resource, consists of tape and disk storage components, Linux servers, and High Performance Storage System (HPSS) software. Tape storage is provided by StorageTek SL8500 robotic tape libraries, each of which can hold up to 10,000 cartridges. [36] Each library has 24 T10K-A drives, 60 T10K-B drives, 36 T10K-C drives, and 72 T10K-D drives. [37]
EVEREST (Exploratory Visualization Environment for Research in Science and Technology) is a large-scale venue for data exploration and analysis. EVEREST measures 30 feet long by 8 feet tall, and its main feature is a 27-projector PowerWall with an aggregate pixel count of 35 million pixels. The projectors are arranged in a 9×3 array, each providing 3,500 lumens for a very bright display.
Displaying 11,520 by 3,072 pixels, the wall offers a tremendous amount of visual detail. The wall is integrated with the rest of the computing center, creating a high-bandwidth data path between large-scale high-performance computing and large-scale data visualization.
EVEREST is controlled by a 14-node cluster. Each node contains four dual-core AMD Opteron processors. These 14 nodes have NVIDIA QuadroFX 3000G graphics cards connected to the projectors, providing a very-high-throughput visualization capability. The visualization lab acts as an experimental facility for development of future visualization capabilities. It houses a 12-panel tiled LCD display, test cluster nodes, interaction devices, and video equipment.
Rhea is a 521-node, commodity-type Linux cluster. Rhea provides a conduit for large-scale scientific discovery via pre- and post-processing of simulation data generated on the Titan supercomputer. Each of Rhea’s first 512 nodes contain two 8-core 2.0 GHz Intel Xeon processors with Intel’s HT Technology and 128 GB of main memory. Rhea also has nine large memory GPU nodes. These nodes each have 1 TB of main memory and two NVIDIA K80 GPUs with two 14-core 2.30 GHz Intel Xeon processors with HT Technology. Rhea is connected to the OLCF’s high performance Lustre filesystem, Atlas. [38]
Wombat is a single-rack cluster from HPE based on the 64-bit ARM architecture instead of traditional x86-based architecture. This system is available to support computer science research projects aimed at exploring the ARM architecture.
The Wombat cluster has 16 compute nodes, four of which have two AMD GPU accelerators attached (eight GPUs total in the system). Each compute node has two 28-core Cavium ThunderX2 processors, 256 GB RAM (16 DDR4 DIMMs) and a 480 GB SSD for node-local storage. Nodes are connected with EDR InfiniBand (~100 Gbit/s). [39]
The IBM AC922 Summit , or OLCF-4, is ORNL’s 200-petaflop flagship supercomputer. Summit was originally launched in June 2018, and as of the November 2019 TOP500 list, is the fastest computer in the world with a High Performance Linpack (HPL) performance of 148.6 petaflops. [40] Summit is also the first computer to reach exascale performance, achieving a peak throughput of 1.88 exaops through a mixture of single- and half-precision floating point operations. [41]
Like its predecessor Titan, Summit makes use of a hybrid architecture that integrates its 9,216 Power9 CPUs and 27,648 NVIDIA Volta V100 GPUs using NVIDIA’s NVLink. [42] Summit features 4,608 nodes (nearly a quarter of Titan’s 18,688 nodes), each with 512 GB of Double Data Rate 4 Synchronous Dynamic Random-Access Memory (DDR4) and 96 GB of High Bandwidth Memory (HBM2) per node, with a total storage capacity of 250 petabytes. [43]
Scheduled for delivery in 2021 with user access becoming available the following year, Frontier will be ORNL’s first sustainable exascale system, meaning it will be capable of performing one quintillion—one billion billion—operations per second. The system will be composed of more than 100 Cray Shasta cabinets with an anticipated peak performance around 1.5 exaflops. [44]
A supercomputer is a type of computer with a high level of performance as compared to a general-purpose computer. The performance of a supercomputer is commonly measured in floating-point operations per second (FLOPS) instead of million instructions per second (MIPS). Since 2017, supercomputers have existed, which can perform over 1017 FLOPS (a hundred quadrillion FLOPS, 100 petaFLOPS or 100 PFLOPS). For comparison, a desktop computer has performance in the range of hundreds of gigaFLOPS (1011) to tens of teraFLOPS (1013). Since November 2017, all of the world's fastest 500 supercomputers run on Linux-based operating systems. Additional research is being conducted in the United States, the European Union, Taiwan, Japan, and China to build faster, more powerful and technologically superior exascale supercomputers.
Oak Ridge National Laboratory (ORNL) is a federally funded research and development center in Oak Ridge, Tennessee, United States. Founded in 1943, the laboratory is now sponsored by the United States Department of Energy and administered by UT–Battelle, LLC.
Floating point operations per second is a measure of computer performance in computing, useful in fields of scientific computations that require floating-point calculations.
Blue Gene was an IBM project aimed at designing supercomputers that can reach operating speeds in the petaFLOPS (PFLOPS) range, with low power consumption.
Cray Inc., a subsidiary of Hewlett Packard Enterprise, is an American supercomputer manufacturer headquartered in Seattle, Washington. It also manufactures systems for data storage and analytics. Several Cray supercomputer systems are listed in the TOP500, which ranks the most powerful supercomputers in the world.
The Ohio Supercomputer Center (OSC) is a supercomputer facility located on the western end of the Ohio State University campus, just north of Columbus. Established in 1987, the OSC partners with Ohio universities, labs and industries, providing students and researchers with high performance computing, advanced cyberinfrastructure, research and computational science education services.
The Oak Ridge Leadership Computing Facility (OLCF), formerly the National Leadership Computing Facility, is a designated user facility operated by Oak Ridge National Laboratory and the Department of Energy. It contains several supercomputers, the largest of which is an HPE OLCF-5 named Frontier, which was ranked 1st on the TOP500 list of world's fastest supercomputers as of June 2023. It is located in Oak Ridge, Tennessee.
The TOP500 project ranks and details the 500 most powerful non-distributed computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year. The first of these updates always coincides with the International Supercomputing Conference in June, and the second is presented at the ACM/IEEE Supercomputing Conference in November. The project aims to provide a reliable basis for tracking and detecting trends in high-performance computing and bases rankings on HPL benchmarks, a portable implementation of the high-performance LINPACK benchmark written in Fortran for distributed-memory computers.
Jaguar or OLCF-2 was a petascale supercomputer built by Cray at Oak Ridge National Laboratory (ORNL) in Oak Ridge, Tennessee. The massively parallel Jaguar had a peak performance of just over 1,750 teraFLOPS. It had 224,256 x86-based AMD Opteron processor cores, and operated with a version of Linux called the Cray Linux Environment. Jaguar was a Cray XT5 system, a development from the Cray XT4 supercomputer.
The National Institute for Computational Sciences (NICS) is funded by the National Science Foundation and managed by the University of Tennessee. NICS was home to Kraken, the most powerful computer in the world managed by academia. The NICS petascale scientific computing environment is housed at Oak Ridge National Laboratory (ORNL), home to the world's most powerful computing complex. The mission of NICS, a member of the Extreme Science and Engineering Discovery Environment (XSEDE - formerly TeraGrid), is to enable the scientific discoveries of researchers nationwide by providing leading-edge computational resources, together with support for their effective use, and leveraging extensive partnership opportunities.
The National Computational Infrastructure is a high-performance computing and data services facility, located at the Australian National University (ANU) in Canberra, Australian Capital Territory. The NCI is supported by the Australian Government's National Collaborative Research Infrastructure Strategy (NCRIS), with operational funding provided through a formal collaboration incorporating CSIRO, the Bureau of Meteorology, the Australian National University, Geoscience Australia, the Australian Research Council, and a number of research intensive universities and medical research institutes.
Kraken was a Cray XT5 supercomputer that entered into full production mode on February 2, 2009. Kraken was operated by the University of Tennessee and was the most powerful computer in the world managed by academia at the time. It was housed in the Oak Ridge Leadership Computing facility at Oak Ridge National Laboratory. Kraken was decommissioned on April 30, 2014.
The history of supercomputing goes back to the 1960s when a series of computers at Control Data Corporation (CDC) were designed by Seymour Cray to use innovative designs and parallelism to achieve superior computational peak performance. The CDC 6600, released in 1964, is generally considered the first supercomputer. However, some earlier computers were considered supercomputers for their day such as the 1954 IBM NORC in the 1950s, and in the early 1960s, the UNIVAC LARC (1960), the IBM 7030 Stretch (1962), and the Manchester Atlas (1962), all of which were of comparable power.
Approaches to supercomputer architecture have taken dramatic turns since the earliest systems were introduced in the 1960s. Early supercomputer architectures pioneered by Seymour Cray relied on compact innovative designs and local parallelism to achieve superior computational peak performance. However, in time the demand for increased computational power ushered in the age of massively parallel systems.
Titan or OLCF-3 was a supercomputer built by Cray at Oak Ridge National Laboratory for use in a variety of science projects. Titan was an upgrade of Jaguar, a previous supercomputer at Oak Ridge, that uses graphics processing units (GPUs) in addition to conventional central processing units (CPUs). Titan was the first such hybrid to perform over 10 petaFLOPS. The upgrade began in October 2011, commenced stability testing in October 2012 and it became available to researchers in early 2013. The initial cost of the upgrade was US$60 million, funded primarily by the United States Department of Energy.
The high performance supercomputing program started in mid-to-late 1980s in Pakistan. Supercomputing is a recent area of Computer science in which Pakistan has made progress, driven in part by the growth of the information technology age in the country. Developing on the ingenious supercomputer program started in 1980s when the deployment of the Cray supercomputers was initially denied.
XK7 is a supercomputing platform, produced by Cray, launched on October 29, 2012. XK7 is the second platform from Cray to use a combination of central processing units ("CPUs") and graphical processing units ("GPUs") for computing; the hybrid architecture requires a different approach to programming to that of CPU-only supercomputers. Laboratories that host XK7 machines host workshops to train researchers in the new programming languages needed for XK7 machines. The platform is used in Titan, the world's second fastest supercomputer in the November 2013 list as ranked by the TOP500 organization. Other customers include the Swiss National Supercomputing Centre which has a 272 node machine and Blue Waters has a machine that has Cray XE6 and XK7 nodes that performs at approximately 1 petaFLOPS (1015 floating-point operations per second).
The Cray XC40 is a massively parallel multiprocessor supercomputer manufactured by Cray. It consists of Intel Haswell Xeon processors, with optional Nvidia Tesla or Intel Xeon Phi accelerators, connected together by Cray's proprietary "Aries" interconnect, stored in air-cooled or liquid-cooled cabinets. The XC series supercomputers are available with the Cray DataWarp applications I/O accelerator technology.
Summit or OLCF-4 is a supercomputer developed by IBM for use at Oak Ridge Leadership Computing Facility (OLCF), a facility at the Oak Ridge National Laboratory, capable of 200 petaFLOPS thus making it the 5th fastest supercomputer in the world after Frontier (OLCF-5), Fugaku, LUMI, and Leonardo, with Frontier being the fastest. It held the number 1 position from November 2018 to June 2020. Its current LINPACK benchmark is clocked at 148.6 petaFLOPS.
Hewlett Packard Enterprise Frontier, or OLCF-5, is the world's first exascale supercomputer. It is hosted at the Oak Ridge Leadership Computing Facility (OLCF) in Tennessee, United States and became operational in 2022. As of December 2023, Frontier is the world's fastest supercomputer. It is based on the Cray EX and is the successor to Summit (OLCF-4). Frontier achieved an Rmax of 1.102 exaFLOPS, which is 1.102 quintillion floating-point operations per second, using AMD CPUs and GPUs.
{{cite web}}
: CS1 maint: numeric names: authors list (link){{cite web}}
: CS1 maint: numeric names: authors list (link)