DiaGrid (distributed computing network)

Last updated

DiaGrid is a large, multicampus distributed research computing network utilizing the HTCondor system and centered at Purdue University in West Lafayette, Indiana. In 2012, it included nearly 43,000 processors representing 301 teraflops of computing power. DiaGrid received a Campus Technology Innovators Award from Campus Technology magazine [1] and an IDG InfoWorld 100 Award [2] in 2009 and was employed at the SC09 supercomputing conference in Portland, Ore., to capture nearly 150 days of compute time for science jobs. [3]

Contents

Partners

DiaGrid is a partnership with Purdue, Indiana University, Indiana State University, the University of Notre Dame, the University of Louisville, the University of Nebraska, the University of Wisconsin, Purdue's Calumet and North Central campuses, and Indiana University-Purdue University Fort Wayne. It is designed to accommodate computers at other campuses as new members join. The Purdue portion of the pool, named BoilerGrid, is the largest academic system of its kind.[ citation needed ]

Management

DiaGrid is managed by Information Technology at Purdue (ITaP), the central information technology organization at Purdue's West Lafayette campus, and ITaP's research computing unit the Rosen Center for Advanced Computing, which also operates the Steele, Coates, Rossmann, Hansen and Carter cluster supercomputers.[ citation needed ]

HTCondor

Through HTCondor, developed at the University of Wisconsin, DiaGrid harvests and manages computing cycles from idle or underused high-performance computing cluster nodes, servers, machines in campus computer and other labs, and office computers. Whenever a local user or scheduled job needs a given machine, the HTCondor job is stopped and automatically sent to another HTCondor node as soon as possible. While this "opportunistic" model limits the ability to do parallel processing and communications, a HTCondor pool can provide smaller, serial jobs vast numbers of cycles in a very short amount of time. HTCondor—and by extension, DiaGrid—is designed for high-throughput computing and is excellent for parameter sweeps, Monte Carlo simulation, or nearly any serial application. Some classes of parallel jobs (master-worker) may be run effectively via HTCondor as well.[ citation needed ]

Networking

To pool computational resources spread around Indiana and the Midwest, DiaGrid takes advantage of I-Light, the high-speed fiber-optic state network connecting Indiana campuses to each other, the Internet and national research networks such as the Internet2 and National LambdaRail. DiaGrid provides computational resources to researchers on both the Open Science Grid and the U.S. National Science Foundation's Extreme Science and Engineering Discovery Environment system (formerly TeraGrid).[ citation needed ]

Uses

DiaGrid and BoilerGrid have been used by researchers at Purdue and elsewhere for a variety of purposes, [1] such as imaging the structure of viruses at near-atomic resolutions, [4] [5] simulating the early stages of the Solar System's formation, projecting the reliability of Indiana's electrical supply, modeling the spread of water pollutants, discerning the structure of protein molecules and identifying millions of potential new forms of zeolites, silicate minerals widely used to catalyze chemical reactions on an industrial scale. [6] DiaGrid also is being used to develop data processing techniques for the Large Synoptic Survey Telescope. Purdue added a Web-based portal for BLAST processing with DiaGrid in 2011.

Related Research Articles

<span class="mw-page-title-main">Purdue University</span> American public university in West Lafayette, Indiana

Purdue University is a public land-grant research university in West Lafayette, Indiana, and the flagship campus of the Purdue University system. The university was founded in 1869 after Lafayette businessman John Purdue donated land and money to establish a college of science, technology, and agriculture in his name. The first classes were held on September 16, 1874, with six instructors and 39 students. It has been ranked as among the best public universities in the United States by major institutional rankings, and is renowned for its engineering program.

Grid computing is the use of widely distributed computer resources to reach a common goal. A computing grid can be thought of as a distributed system with non-interactive workloads that involve many files. Grid computing is distinguished from conventional high-performance computing systems such as cluster computing in that grid computers have each node set to perform a different task/application. Grid computers also tend to be more heterogeneous and geographically dispersed than cluster computers. Although a single grid can be dedicated to a particular application, commonly a grid is used for a variety of purposes. Grids are often constructed with general-purpose grid middleware software libraries. Grid sizes can be quite large.

HTCondor is an open-source high-throughput computing software framework for coarse-grained distributed parallelization of computationally intensive tasks. It can be used to manage workload on a dedicated cluster of computers, or to farm out work to idle desktop computers – so-called cycle scavenging. HTCondor runs on Linux, Unix, Mac OS X, FreeBSD, and Microsoft Windows operating systems. HTCondor can integrate both dedicated resources and non-dedicated desktop machines into one computing environment.

<span class="mw-page-title-main">TeraGrid</span>

TeraGrid was an e-Science grid computing infrastructure combining resources at eleven partner sites. The project started in 2001 and operated from 2004 through 2011.

The Pittsburgh Supercomputing Center (PSC) is a high performance computing and networking center founded in 1986 and one of the original five NSF Supercomputing Centers. PSC is a joint effort of Carnegie Mellon University and the University of Pittsburgh in Pittsburgh, Pennsylvania, United States.

The Texas Advanced Computing Center (TACC) at the University of Texas at Austin, United States, is an advanced computing research center that provides comprehensive advanced computing resources and support services to researchers in Texas and across the US. The mission of TACC is to enable discoveries that advance science and society through the application of advanced computing technologies. Specializing in high performance computing, scientific visualization, data analysis & storage systems, software, research & development and portal interfaces, TACC deploys and operates advanced computational infrastructure to enable computational research activities of faculty, staff, and students of UT Austin. TACC also provides consulting, technical documentation, and training to support researchers who use these resources. TACC staff members conduct research and development in applications and algorithms, computing systems design/architecture, and programming tools and environments.

<span class="mw-page-title-main">Purdue Research Park</span>

The Purdue Research Parks are a network of four research parks located in Indiana, United States. The 725-acre (2.93 km2) flagship West Lafayette park is located less than 2 miles (3 km) north of Purdue University's West Lafayette campus, and is the largest university-affiliated research park in the United States. The other facilities are located in Merrillville, Indianapolis, and New Albany. The parks were developed by the Purdue Research Foundation.

The Institute for Biocomputation and Physics of Complex Systems (BIFI) is a research center of the University of Zaragoza devoted to the study of complex systems from a multidisciplinary perspective. Biochemists, physicists, mathematicians, computer scientists and researchers from other fields study complex systems, as well as different phenomena and processes related to them (protein folding, interaction between diseases, the spread of epidemics, multi-layer networks, collective social phenomena, etc.). The ultimate goal is to unravel various aspects of complexity, promote basic science and assess the impact of applied research and possible benefits for society.

In computer science, high-throughput computing (HTC) is the use of many computing resources over long periods of time to accomplish a computational task.

Steele is a supercomputer that was installed at Purdue University on May 5, 2008. The high-performance computing cluster is operated by Information Technology at Purdue (ITaP), the university's central information technology organization. ITaP also operates clusters named Coates built in 2009, Rossmann built in 2010, and Hansen and Carter built in 2011. Steele was the largest campus supercomputer in the Big Ten outside a national center when built. It ranked 104th on the November 2008 TOP500 Supercomputer Sites list.

<span class="mw-page-title-main">Xgrid</span> Distributed computing protocol created by Apple

Xgrid is a proprietary program and distributed computing protocol developed by the Advanced Computation Group subdivision of Apple Inc that allows networked computers to contribute to a single task.

Clarence Leroy “Ben” Coates was an American computer scientist and engineer known for his work on waveform recognition devices, circuit gates and accumulators.

SiCortex was a supercomputer manufacturer founded in 2003 and headquartered in Clock Tower Place, Maynard, Massachusetts. On 27 May 2009, HPCwire reported that the company had shut down its operations, laid off most of its staff, and is seeking a buyer for its assets. The Register reported that Gerbsman Partners was hired to sell SiCortex's intellectual properties. While SiCortex had some sales, selling at least 75 prototype supercomputers to several large customers, the company had never produced an operating profit and ran out of venture capital. New funding could not be found.

Coates is a supercomputer installed at Purdue University on July 21, 2009. The high-performance computing cluster is operated by Information Technology at Purdue (ITaP), the university's central information technology organization. ITaP also operates clusters named Steele built in 2008, Rossmann built in 2010, and Hansen and Carter built in 2011. Coates was the largest campus supercomputer in the Big Ten outside a national center when built. It was the first native 10 Gigabit Ethernet (10GigE) cluster to be ranked in the TOP500 and placed 102nd on the June 2010 list.

<span class="mw-page-title-main">University of Minnesota Supercomputing Institute</span>

The Minnesota Supercomputing Institute (MSI) in Minneapolis, Minnesota is a core research facility of the University of Minnesota that provides hardware and software resources, as well as technical user support, to faculty and researchers at the university and at other institutions of higher education in Minnesota. MSI is located in Walter Library, on the university's Twin Cities campus.

<span class="mw-page-title-main">Supercomputing in Japan</span> Overview of supercomputing in Japan

Japan operates a number of centers for supercomputing which hold world records in speed, with the K computer becoming the world's fastest in June 2011. and Fugaku took the lead in June 2020, and furthered it, as of November 2020, to 3 times faster than number two computer.

<span class="mw-page-title-main">Supercomputing in Europe</span> Overview of supercomputing in Europe

Several centers for supercomputing exist across Europe, and distributed access to them is coordinated by European initiatives to facilitate high-performance computing. One such initiative, the HPC Europa project, fits within the Distributed European Infrastructure for Supercomputing Applications (DEISA), which was formed in 2002 as a consortium of eleven supercomputing centers from seven European countries. Operating within the CORDIS framework, HPC Europa aims to provide access to supercomputers across Europe.

Carter is a supercomputer installed at Purdue University in the fall of 2011 in a partnership with Intel. The high-performance computing cluster is operated by Information Technology at Purdue (ITaP), the university's central information technology organization. ITaP also operates clusters named Steele built in 2008, Coates built in 2009, Rossmann built in 2010, and Hansen built in the summer of 2011. Carter was the fastest campus supercomputer in the U.S. outside a national center when built. It was one of the first clusters to employ Intel's second generation Xenon E-5 "Sandy Bridge" processor and ranked 54th on the November 2011 TOP500 list, making it Purdue's first Top 100-ranked research computing system.

Rossmann is a supercomputer at Purdue University that went into production September 1, 2010. The high-performance computing cluster is operated by Information Technology at Purdue (ITaP), the university's central information technology organization. ITaP also operates clusters named Steele built in 2008, Coates built in 2009, Hansen built in the summer of 2011 and Carter built in the fall of 2012 in partnership with Intel. Rossmann ranked 126 on the November 2010 TOP500 list.

<span class="mw-page-title-main">Singularity (software)</span> Free, cross-platform and open-source computer program

Singularity is a free and open-source computer program that performs operating-system-level virtualization also known as containerization.

References

  1. 1 2 Grush, Mary; Villano, Matt (July 28, 2009). "Campus Technology Innovators Awards 2009: High-Performance Computing - Purdue University". Campus Technology.
  2. ↑ "The top 100 IT projects of 2009". InfoWorld, November 23, 2009.{{cite journal}}: Cite journal requires |journal= (help)
  3. ↑ "Cycle Computing and Purdue University to Power Dynamic Optimized Condor Pool at SuperComputing 2009" (Press release). Nov 13, 2009.
  4. ↑ Jiang, Wen; et al. (Feb 28, 2008). "Backbone structure of the infectious e15 virus capsid revealed by electron cryomicroscopy" (PDF). Nature. 451 (7182): 1130–1134. Bibcode:2008Natur.451.1130J. doi:10.1038/nature06665. PMID   18305544. S2CID   205212346.
  5. ↑ Wu, Weimin; Jiang, Wen (Apr 30, 2008). "Condor in Cryo-EM image processing".
  6. ↑ Pophale, Ramdas; Cheeseman, Phillip A.; Deem, Michael W. (2011). "A database of new zeolite-like materials". Physical Chemistry Chemical Physics. 13 (27): 12407–12412. Bibcode:2011PCCP...1312407P. doi:10.1039/C0CP02255A. PMID   21423937.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.