Titan (supercomputer)

This article is about the Titan supercomputer. For the Atlas 2 prototype computer, see Titan (1963 computer).

Titan

Active	Became operational October 29, 2012; decommissioned August 2, 2019
Sponsors	US DOE and NOAA (<10%)
Operators	Cray Inc.
Location	Oak Ridge National Laboratory
Architecture	18,688 AMD Opteron 6274 16-core CPUs 18,688 Nvidia Tesla K20X GPUs
Power	8.2 MW
Operating system	Cray Linux Environment
Space	404 m2 (4352 ft2)
Memory	693.5  TiB (584 TiB CPU and 109.5 TiB GPU)
Storage	40  PB, 1.4 TB/s IO Lustre filesystem
Speed	17.59 petaFLOPS (LINPACK) 27 petaFLOPS theoretical peak
Cost	US$97 million (equivalent to $124 million in 2022)
Ranking	TOP500 : 4th, June 2017 [1]
Purpose	Scientific research
Legacy	Ranked 1 on TOP500 when built. First GPU based supercomputer to perform over 10 petaFLOPS
Website	www.olcf.ornl.gov/titan/

Titan or OLCF-3 was a supercomputer built by Cray at Oak Ridge National Laboratory for use in a variety of science projects. Titan was an upgrade of Jaguar, a previous supercomputer at Oak Ridge, that uses graphics processing units (GPUs) in addition to conventional central processing units (CPUs). Titan was the first such hybrid to perform over 10  petaFLOPS. The upgrade began in October 2011, commenced stability testing in October 2012 and it became available to researchers in early 2013. The initial cost of the upgrade was US$60 million, funded primarily by the United States Department of Energy.

Titan was eclipsed at Oak Ridge by Summit in 2019, which was built by IBM and features fewer nodes with much greater GPU capability per node as well as local per-node non-volatile caching of file data from the system's parallel file system. ^[2]

Titan employed AMD Opteron CPUs in conjunction with Nvidia Tesla GPUs to improve energy efficiency while providing an order of magnitude increase in computational power over Jaguar. It used 18,688 CPUs paired with an equal number of GPUs to perform at a theoretical peak of 27 petaFLOPS; in the LINPACK benchmark used to rank supercomputers' speed, it performed at 17.59 petaFLOPS. This was enough to take first place in the November 2012 list by the TOP500 organization, but Tianhe-2 overtook it on the June 2013 list.

Titan was available for any scientific purpose; access depends on the importance of the project and its potential to exploit the hybrid architecture. Any selected programs must also be executable on other supercomputers to avoid sole dependence on Titan. Six vanguard programs were the first selected. They dealt mostly with molecular scale physics or climate models, while 25 others were queued behind them. The inclusion of GPUs compelled authors to alter their programs. The modifications typically increased the degree of parallelism, given that GPUs offer many more simultaneous threads than CPUs. The changes often yield greater performance even on CPU-only machines.

History

Plans to create a supercomputer capable of 20 petaFLOPS at the Oak Ridge Leadership Computing Facility (OLCF) at Oak Ridge National Laboratory (ORNL) originated as far back as 2005, when Jaguar was built. ^[3] Titan will itself be replaced by an approximately 200 petaFLOPS system in 2016 as part of ORNL's plan to operate an exascale (1000 petaFLOPS to 1 exaFLOPS) machine by 2020. ^{[3] [4] [5]} The initial plan to build a new 15,000 square meter (160,000 ft²) building for Titan, was discarded in favor of using Jaguar's existing infrastructure. ^[6] The precise system architecture was not finalized until 2010, although a deal with Nvidia to supply the GPUs was signed in 2009. ^[7] Titan was first announced at the private ACM/IEEE Supercomputing Conference (SC10) on November 16, 2010, and was publicly announced on October 11, 2011, as the first phase of the Titan upgrade began. ^{[4] [8]}

Jaguar had received various upgrades since its creation. It began with the Cray XT3 platform that yielded 25 teraFLOPS. ^[9] By 2008, Jaguar had been expanded with more cabinets and upgraded to the XT4 platform, reaching 263 teraFLOPS. ^[9] In 2009, it was upgraded to the XT5 platform, hitting 1.4 petaFLOPS. ^[9] Its final upgrades brought Jaguar to 1.76 petaFLOPS. ^[10]

Titan was funded primarily by the US Department of Energy through ORNL. Funding was sufficient to purchase the CPUs but not all of the GPUs so the National Oceanic and Atmospheric Administration agreed to fund the remaining nodes in return for computing time. ^{[11] [12]} ORNL scientific computing chief Jeff Nichols noted that Titan cost approximately $60 million upfront, of which the NOAA contribution was less than $10 million, but precise figures were covered by non-disclosure agreements. ^{[11] [13]} The full term of the contract with Cray included $97 million, excluding potential upgrades. ^[13]

The yearlong conversion began October 9, 2011. ^{[14] [15]} Between October and December, 96 of Jaguar's 200 cabinets, each containing 24  XT5 blades (two 6-core CPUs per node, four nodes per blade), were upgraded to XK7 blade (one 16-core CPU per node, four nodes per blade) while the remainder of the machine remained in use. ^[14] In December, computation was moved to the 96 XK7 cabinets while the remaining 104 cabinets were upgraded to XK7 blades. ^[14] ORNL's external ESnet connection was upgraded from 10  Gbit/s to 100 Gbit/s and the system interconnect (the network over which CPUs communicate with each other) was updated. ^{[14] [16]} The Seastar design used in Jaguar was upgraded to the Gemini interconnect used in Titan which connects the nodes into a direct 3D torus interconnect network. ^[17] Gemini uses wormhole flow control internally. ^[17] The system memory was doubled to 584  TiB. ^[15] 960 of the XK7 nodes (10 cabinets) were fitted with a Fermi based GPU as Kepler GPUs were not then available; these 960 nodes were referred to as TitanDev and used to test code. ^{[14] [15]} This first phase of the upgrade increased the peak performance of Jaguar to 3.3 petaFLOPS. ^[15] Beginning on September 13, 2012, Nvidia K20X GPUs were fitted to all of Jaguar's XK7 compute blades, including the 960 TitanDev nodes. ^{[14] [18] [19]} In October, the task was completed and the computer was finally renamed Titan. ^[14]

In March 2013, Nvidia launched the GTX Titan, a consumer graphics card that uses the same GPU die as the K20X GPUs in Titan. ^[20] Titan underwent acceptance testing in early 2013 but only completed 92% of the tests, short of the required 95%. ^{[14] [21]} The problem was discovered to be excess gold in the female edge connectors of the motherboards' PCIe slots causing cracks in the motherboards' solder. ^[22] The cost of repair was borne by Cray and between 12 and 16 cabinets were repaired each week. ^[22] Throughout the repairs users were given access to the available CPUs. ^[22] On March 11, they gained access to 8,972 GPUs. ^[23] ORNL announced on April 8 that the repairs were complete ^[24] and acceptance test completion was announced on June 11, 2013. ^[25]

Titan's hardware has a theoretical peak performance of 27  petaFLOPS with "perfect" software. ^[26] On November 12, 2012, the TOP500 organization that ranks the world's supercomputers by LINPACK performance, ranked Titan first at 17.59 petaFLOPS, displacing IBM Sequoia. ^{[27] [28]} Titan also ranked third on the Green500, the same 500 supercomputers ranked in terms of energy efficiency. ^[29] In the June 2013 TOP500 ranking, Titan fell to second place behind Tianhe-2 and to twenty-ninth on the Green500 list. ^{[30] [31]} Titan did not re-test for the June 2013 ranking, ^[30] because it would still have ranked second, at 27 petaFLOPS. ^[32]

Hardware

Titan uses Jaguar's 200 cabinets, covering 404 square meters (4,352 ft²), with replaced internals and upgraded networking. ^{[33] [34]} Reusing Jaguar's power and cooling systems saved approximately $20 million. ^[35] Power is provided to each cabinet at three-phase 480  V. This requires thinner cables than the US standard 208 V, saving $1 million in copper. ^[36] At its peak, Titan draws 8.2  MW, ^[37] 1.2 MW more than Jaguar, but runs almost ten times as fast in terms of floating point calculations. ^{[33] [36]} In the event of a power failure, carbon fiber flywheel power storage can keep the networking and storage infrastructure running for up to 16 seconds. ^[38] After 2 seconds without power, diesel generators fire up, taking approximately 7 seconds to reach full power. They can provide power indefinitely. ^[38] The generators are designed only to keep the networking and storage components powered so that a reboot is much quicker; the generators are not capable of powering the processing infrastructure. ^[38]

Titan has 18,688 nodes (4 nodes per blade, 24 blades per cabinet), ^[39] each containing a 16-core AMD Opteron 6274 CPU with 32 GB of DDR3 ECC memory and an Nvidia Tesla K20X GPU with 6 GB GDDR5 ECC memory. ^[40] There are a total of 299,008 processor cores, and a total of 693.6 TiB of CPU and GPU RAM. ^[36]

Initially, Titan used Jaguar's 10 PB of Lustre storage with a transfer speed of 240 GB/s, ^{[36] [41]} but in April 2013, the storage was upgraded to 40 PB with a transfer rate of 1.4 TB/s. ^[42] GPUs were selected for their vastly higher parallel processing efficiency over CPUs. ^[40] Although the GPUs have a slower clock speed than the CPUs, each GPU contains 2,688 CUDA cores at 732  MHz, ^[43] resulting in a faster overall system. ^{[34] [44]} Consequently, the CPUs' cores are used to allocate tasks to the GPUs rather than directly processing the data as in conventional supercomputers. ^[40]

Titan runs the Cray Linux Environment, a full version of Linux on the login nodes that users directly access, but a smaller, more efficient version on the compute nodes. ^[45]

Titan's components are air-cooled by heat sinks, but the air is chilled before being pumped through the cabinets. ^[46] Fan noise is so loud that hearing protection is required for people spending more than 15 minutes in the machine room. ^[47] The system has a cooling capacity of 23.2 MW (6600 tons) and works by chilling water to 5.5 °C (42 °F), which in turn cools recirculated air. ^[46]

Researchers also have access to EVEREST (Exploratory Visualization Environment for Research and Technology) to better understand the data that Titan outputs. EVEREST is a visualization room with a 10 by 3 meter (33 by 10 ft) screen and a smaller, secondary screen. The screens are 37 and 33  megapixels respectively with stereoscopic 3D capability. ^[48]

Projects

In 2009, the Oak Ridge Leadership Computing Facility that manages Titan narrowed the fifty applications for first use of the supercomputer down to six "vanguard" codes chosen for the importance of the research and for their ability to fully utilize the system. ^{[34] [49]} The six vanguard projects to use Titan were:

• S3D, a project that models the molecular physics of combustion, aims to improve the efficiency of diesel and biofuel engines. In 2009, using Jaguar, it produced the first fully resolved simulation of autoigniting hydrocarbon flames relevant to the efficiency of direct injection diesel engines. ^[49]
• WL-LSMS simulates the interactions between electrons and atoms in magnetic materials at temperatures other than absolute zero. An earlier version of the code was the first to perform at greater than one petaFLOPS on Jaguar. ^[49]
• Denovo simulates nuclear reactions with the aim of improving the efficiency and reducing the waste of nuclear reactors. ^[34] The performance of Denovo on conventional CPU-based machines doubled after the tweaks for Titan and it performs 3.5 times faster on Titan than it did on Jaguar. ^{[49] [50]}
• Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) is a molecular dynamics code that simulates particles across a range of scales, from quantum to relativistic, to improve materials science with potential applications in semiconductor, biomolecule and polymer development. ^[51]
• CAM-SE is a combination of two codes: Community Atmosphere Model, a global atmosphere model, and High Order Method Modeling Environment, a code that solves fluid and thermodynamic equations. CAM-SE will allow greater accuracy in climate simulations. ^[49]
• Non-Equilibrium Radiation Diffusion (NRDF) plots non-charged particles through supernovae with potential applications in laser fusion, fluid dynamics, medical imaging, nuclear reactors, energy storage and combustion. ^[49] Its Chimera code uses hundreds of partial differential equations to track the energy, angle, angle of scatter and type of each neutrino modeled in a star going supernova, resulting in millions of individual equations. ^[52] The code was named Chimera after the mythological creature because it has three "heads": the first simulates the hydrodynamics of stellar material, the second simulates radiation transport and the third simulates nuclear burning. ^[52]
• Bonsai is a gravitational tree code for n-body simulation. It has been used for the 2014 Gordon Bell prize nomination for simulating the Milky Way Galaxy on a star by star basis, with 200 billion stars. In this application the computer reached a sustained speed of 24.773 petaFlops. ^[53]
• VERA is a light-water reactor simulation written at the Consortium for Advanced Simulation of Light Water Reactors (CASL) on Jaguar. VERA allows engineers to monitor the performance and status of any part of a reactor core throughout the lifetime of the reactor to identify points of interest. ^[54] Although not one of the first six projects, VERA was planned to run on Titan after optimization with assistance from CAAR and testing on TitanDev. Computer scientist Tom Evans found that the adaption to Titan's hybrid architecture was more difficult than to previous CPU-based supercomputers. He aimed to simulate an entire reactor fuel cycle, an eighteen to thirty-six month-long process, in one week on Titan. ^[54]

In 2013 thirty-one codes were planned to run on Titan, typically four or five at any one time. ^{[47] [55]}

Code modifications

See also: GPGPU

The code of many projects has to be modified to suit the GPU processing of Titan, but each code is required to be executable on CPU-based systems so that projects do not become solely dependent on Titan. ^[49] OLCF formed the Center for Accelerated Application Readiness (CAAR) to aid with the adaptation process. It holds developer workshops at Nvidia headquarters to educate users about the architecture, compilers and applications on Titan. ^{[56] [57]} CAAR has been working on compilers with Nvidia and code vendors to integrate directives for GPUs into their programming languages. ^[56] Researchers can thus express parallelism in their code with their existing programming language, typically Fortran, C or C++, and the compiler can express it to the GPUs. ^[56] Dr. Bronson Messer, a computational astrophysicist, said of the task: "an application using Titan to the utmost must also find a way to keep the GPU busy, remembering all the while that the GPU is fast, but less flexible than the CPU." ^[56] Moab Cluster Suite is used to prioritize jobs to nodes to keep utilization high; it improved efficiency from 70% to approximately 95% in the tested software. ^{[58] [59]} Some projects found that the changes increased efficiency of their code on non-GPU machines; the performance of Denovo doubled on CPU-based machines. ^[49]

The amount of code alteration required to run on the GPUs varies by project. According to Dr. Messer of NRDF, only a small percentage of his code runs on GPUs because the calculations are relatively simple but processed repeatedly and in parallel. ^[60] NRDF is written in CUDA Fortran, a version of Fortran with CUDA extensions for the GPUs. ^[60] Chimera's third "head" was the first to run on the GPUs as the nuclear burning could most easily be simulated by GPU architecture. Other aspects of the code were planned to be modified in time. ^[52] On Jaguar, the project modeled 14 or 15 nuclear species but Messer anticipated simulating up to 200 species, allowing far greater precision when comparing the simulation to empirical observation. ^[52]

See also

• Jaguar (supercomputer) – OLCF-2
• Summit (supercomputer) – OLCF-4
• Oak Ridge Leadership Computing Facility

References

1. "TOP500 list refreshed, US edged out of third place", TOP500, 19 June 2017 (page visited on 19 June 2017).
2. "Summit FAQs". ornl.gov. Oak Ridge National Laboratory. November 14, 2014. Retrieved November 15, 2014.
3. "Discussing the ORNL Titan Supercomputer with ORNL's Jack Wells". The Exascale Report. November 2012. Archived from the original on May 25, 2013. Retrieved December 19, 2012.
4. Bland, Buddy (November 16, 2010). "Where do we go from here?" (PDF). Oak Ridge National Laboratory. Archived from the original (PDF) on March 3, 2012. Retrieved December 18, 2012.
5. Goldman, David (October 29, 2012). "Top U.S. supercomputer guns for fastest in world". CNN. Archived from the original on March 2, 2013. Retrieved March 31, 2013.
6. Munger, Frank (March 7, 2011). "Oak Ridge lab to add titanic supercomputer". Knox News. Archived from the original on July 4, 2012. Retrieved December 19, 2012.
7. Morgan, Timothy Prickett (October 1, 2009). "Oak Ridge goes gaga for Nvidia GPUs". The Register . Archived from the original on November 9, 2012. Retrieved December 19, 2012.
8. Levy, Dawn (October 11, 2011). "ORNL awards contract to Cray for Titan supercomputer". Oak Ridge National Laboratory. Archived from the original on February 26, 2013. Retrieved December 19, 2012.
9. "Jaguar: Oak ridge National Laboratory". TOP500. Archived from the original on March 17, 2013. Retrieved December 18, 2012.
10. "TOP500 List November 2011". TOP500. Archived from the original on January 21, 2013. Retrieved December 18, 2012.
11. Munger, Frank (November 26, 2012). "The ORNL and NOAA relationship". Knox News. Archived from the original on May 22, 2013. Retrieved December 20, 2012.
12. Munger, Frank (November 18, 2012). "The cost of Titan". Knox News. Archived from the original on May 22, 2013. Retrieved December 20, 2012.
13. Feldman, Michael (October 11, 2011). "GPUs Will Morph ORNL's Jaguar Into 20-Petaflop Titan". HPC Wire. Archived from the original on July 27, 2012. Retrieved October 29, 2012.
14. "Titan Project Timeline". Oak Ridge Leadership Computing Facility. Archived from the original on June 18, 2012. Retrieved December 18, 2012.
15. Brouner, Jennifer; McCorkle, Morgan; Pearce, Jim; Williams, Leo (2012). "ORNL Review Vol. 45" (PDF). Oak Ridge National Laboratory. Archived from the original (PDF) on March 4, 2013. Retrieved November 2, 2012.
16. "Superfast Titan, Superfast Network". Oak Ridge Leadership Computing Facility. December 17, 2012. Archived from the original on March 26, 2013. Retrieved December 18, 2012.
17. "The Gemini Network" (PDF). Cray Inc. 2010. Retrieved April 29, 2015.
18. Poeter, Damon (October 11, 2011). "Cray's Titan Supercomputer for ORNL Could Be World's Fastest". PC Magazine . Archived from the original on June 5, 2012. Retrieved October 29, 2012.
19. Jones, Gregory Scott (September 17, 2012). "Final Upgrade Underway". Oak Ridge Leadership Computing Facility. Archived from the original on March 26, 2013. Retrieved November 16, 2012.
20. Smith, Ryan (February 21, 2013). "Nvidia's GeForce GTX Titan Review, Part 2: Titan's Performance Unveiled". Anandtech. Archived from the original on February 23, 2013. Retrieved March 26, 2013.
21. Munger, Frank (February 20, 2013). "No. 1 Titan not yet living up to potential". Knox News. Archived from the original on March 27, 2013. Retrieved March 26, 2013.
22. Huotari, John (March 13, 2013). "Cray re-soldering Titan's connectors, supercomputer testing could be done in April". Oak Ridge Today. Archived from the original on March 18, 2013. Retrieved March 26, 2013.
23. Jones, Scott (March 26, 2013). "Titan Users Now Have Access to GPUs". Oak Ridge Leadership Computing Facility. Archived from the original on June 5, 2013. Retrieved March 26, 2013.
24. Huotari, John (April 8, 2013). "Titan repairs complete, ORNL preparing for second round of supercomputer testing". Oak Ridge Today. Retrieved April 8, 2013.
25. Munger, Frank. "Titan passes acceptance test, seals ORNL's supercomputer deal with Cray". Knox News. Retrieved July 2, 2013.
26. Jones, Gregory Scott (November 12, 2012). "ORNL Supercomputer Named World's Most Powerful". Oak Ridge National Laboratory. Archived from the original on February 22, 2013. Retrieved December 14, 2012.
27. "Oak Ridge Claims No. 1 Position on Latest TOP500 List with Titan". TOP500. November 12, 2012. Archived from the original on January 21, 2013. Retrieved November 15, 2012.
28. "US Titan supercomputer clocked as world's fastest". BBC. November 12, 2012. Archived from the original on February 3, 2013. Retrieved November 12, 2012.
29. Williams, Leo (November 14, 2012). "Titan is Also a Green Powerhouse". Oak Ridge Leadership Computing Facility. Archived from the original on February 16, 2013. Retrieved November 15, 2012.
30. "June 2013". TOP500. Retrieved July 2, 2013.
31. "The Green500 List - June 2013". Green500. Archived from the original on July 2, 2013. Retrieved July 2, 2013.
32. Munger, Frank. "Titan didn't re-test for TOP500, keeping last year's benchmark; ORNL's Jeff Nichols explains why". Knox News. Retrieved July 2, 2013.
33. Tibken, Shara (October 29, 2012). "Titan supercomputer debuts for open scientific research". CNET . Archived from the original on December 15, 2012. Retrieved October 29, 2012.
34. "Introducing Titan". Oak Ridge Leadership Computing Facility. Archived from the original on February 22, 2013. Retrieved October 29, 2012.
35. Munger, Frank (October 29, 2012). "Titan's ready to roll; ORNL supercomputer may become world's No. 1". Knox News. Archived from the original on July 5, 2013. Retrieved October 29, 2012.
36. Lal Shimpi, Anand (October 31, 2012). "Inside the Titan Supercomputer". Anandtech. p. 1. Archived from the original on January 25, 2013. Retrieved November 2, 2012.
37. "Heterogeneous Systems Re-Claim Green500 List Dominance". Green500. November 14, 2012. Archived from the original on February 5, 2013. Retrieved November 15, 2012.
38. Bland, Buddy; Lal Shimpi, Anand (October 30, 2012). "Oak Ridge National Laboratory Tour – Backup Power" (Youtube). Anandtech. Retrieved November 2, 2012.
39. Morgan, Timothy Prickett (October 11, 2011). "Oak Ridge changes Jaguar's spots from CPUs to GPUs". The Register . Archived from the original on October 15, 2012. Retrieved December 21, 2012.
40. "ORNL Debuts Titan Supercomputer" (PDF). Oak Ridge Leadership Computing Facility. Archived from the original (PDF) on February 26, 2013. Retrieved October 29, 2012.
41. Lal Shimpi, Anand (October 31, 2012). "Titan's storage array". Anandtech. Archived from the original on April 6, 2013. Retrieved December 18, 2012.
42. Santos, Alexis. "Titan supercomputer to be loaded with 'world's fastest' storage system". Engadget . Retrieved April 16, 2013.
43. Smith, Ryan (November 12, 2012). "NVIDIA Launches Tesla K20 & K20X: GK110 Arrives At Last". Anandtech. Archived from the original on January 24, 2013. Retrieved December 21, 2012.
44. Feldman, Michael (October 29, 2012). "Titan Sets High Water Mark for GPU Supercomputing". HPC Wire. Archived from the original on November 5, 2012. Retrieved October 30, 2012.
45. "Titan System Overview". Oak Ridge Leadership Computing Facility. Archived from the original on February 17, 2013. Retrieved December 21, 2012.
46. Bland, Buddy; Lal Shimpi, Anand (October 30, 2012). "Oak Ridge National Laboratory Tour – Cooling Requirements" (Youtube). Anandtech. Retrieved November 2, 2012.
47. Pavlus, John (October 29, 2012). "Building Titan: The 'world's fastest' supercomputer". BBC. Archived from the original on January 30, 2013. Retrieved January 8, 2013.
48. Munger, Frank (January 1, 2013). "ORNL visualization lab gets $2.5M makeover, adds 3D". Knox News. Archived from the original on May 24, 2013. Retrieved March 30, 2019.
49. "TITAN: Built for Science" (PDF). Oak Ridge Leadership Computing Facility. Archived from the original (PDF) on February 26, 2013. Retrieved October 29, 2012.
50. "Nuclear Energy – Supercomputer speeds path forward". Consortium for Advanced Simulation of LWRs. Archived from the original on February 14, 2013. Retrieved December 14, 2012.
51. Zybin, Sergey. "LAMMPS Molecular Dynamics Simulator". Sandia National Laboratories. Archived from the original on February 16, 2013. Retrieved October 29, 2012.
52. Messer, Bronson (October 30, 2012). "Using Titan to Model Supernovae" (Youtube). Anandtech. Retrieved November 15, 2012.
53. Bédorf, Jeroen; Gaburov, Evghenii; Fujii, Michiko S.; Nitadori, Keigo; Ishiyama, Tomoaki; Portegies Zwart, Simon (2014). "24.77 Pflops on a Gravitational Tree-Code to Simulate the Milky Way Galaxy with 18600 GPUs". SC14: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE. pp. 54–65. arXiv: 1412.0659 . Bibcode:2014hpcn.conf...54B. doi:10.1109/SC.2014.10. ISBN   978-1-4799-5500-8. S2CID   7008518.
54. Pearce, Jim. "VERA analyzes nuclear reactor designs in unprecedented detail". Oak Ridge National Laboratory. Archived from the original on February 15, 2013. Retrieved December 18, 2012.
55. "2013 INCITE Awards" (PDF). US Department of Energy. Archived from the original (PDF) on October 29, 2012. Retrieved January 17, 2013.
56. Williams, Leo. "Preparing users for Titan". Oak Ridge National Laboratory. Archived from the original on March 1, 2013. Retrieved November 19, 2012.
57. Rumsey, Jeremy (December 17, 2012). "Titan Trainers Take Road Trip". Oak Ridge Leadership Computing Facility. Archived from the original on March 26, 2013. Retrieved December 18, 2012.
58. "Supercomputing Leaders Choose Adaptive Computing to Complement Latest HPC Systems". Business Wire . January 30, 2013. Archived from the original on May 18, 2013. Retrieved January 31, 2013.
59. DuBois, Shelley (January 30, 2013). "The next revolution in cloud computing". Fortune Magazine . Archived from the original on April 2, 2013. Retrieved January 31, 2013.
60. Lal Shimpi, Anand (October 31, 2012). "Inside the Titan Supercomputer". Anandtech. p. 3. Archived from the original on April 5, 2013. Retrieved November 15, 2012.

External links

Listen to this article (20 minutes)

This audio file was created from a revision of this article dated 8 July 2016 (2016-07-08), and does not reflect subsequent edits.

(Audio help  · More spoken articles)

• Media related to Titan (supercomputer) at Wikimedia Commons
• Official website
• "GeForce GTX Titan". Nvidia. Archived from the original on February 24, 2013. Retrieved March 26, 2013.

Records
Preceded by IBM Sequoia 16.325 petaflops	World's most powerful supercomputer November 2012 – June 2013	Succeeded by Tianhe-2 33.9 petaflops