European Grid Infrastructure

Last updated

EGI
Company typeScientific support
Industry
Founded2010
HeadquartersScience Park 140 1098 XG,
Amsterdam
,
The Netherlands
Area served
Europe
Key people
Revenue3,972,057 Euro (2020)  OOjs UI icon edit-ltr-progressive.svg
Website egi.eu

European Grid Infrastructure (EGI) is a series of efforts to provide access to high-throughput computing resources across Europe using grid computing techniques. [1] The EGI links centres in different European countries to support international research in many scientific disciplines. Following a series of research projects such as DataGrid and Enabling Grids for E-sciencE, the EGI Foundation was formed in 2010 to sustain the services of EGI. [2]

Contents

Purpose

Science has become increasingly based on open collaboration between researchers across the world. It uses high-capacity computing to model complex systems and to process experimental results. In the early 21st century, grid computing became popular for scientific disciplines such as high-energy physics and bioinformatics to share and combine the power of computers and sophisticated, often unique, scientific instruments in a process known as e-Science. [2]

In addition to their scientific value, on 30 May 2008 The EU Competitiveness Council promoted "the essential role of e-infrastructures as an integrating mechanism between Member States, regions as well as different scientific disciplines, also contributing to overcoming digital divides." [3]

EGI is partially supported by the EGI-InSPIRE EC project.

History

DataGrid logo 2001-2004 DataGrid logo.gif
DataGrid logo 2001–2004

The European DataGrid project was first funded in 2001 for three years as one of the Framework Programmes for Research and Technological Development series. [4] [5] Fabrizio Gagliardi was project manager of DataGrid and its budget was about 12 million euro, with the full project named "Research and Technological Development for an International Data Grid". [6]

A major motivation behind the concept was the massive data requirements of CERN's LHC (Large Hadron Collider) project. [7]

EGEE

On 1 April 2004 the Enabling Grids for E-Science in Europe (EGEE) project was funded by the European Commission through the Directorate-General for Information Society and Media, led by the information technology division of CERN. [8] This 24-month project of the Sixth Framework Programme had a cost of over 46 million euro. The consortium included 70 institutions in 27 countries. [8] The LHC Computing Grid continued to be a major application of EGEE technology. [9] By 1 April 2006 the "in Europe" was dropped from the project name, but the acronym was kept as EGEE-II for Enabling Grids for E-sciencE. This two-year phase cost about 52.6 million euro. [10] The new name reflected a more global extent, such as a cluster of computers at the Institute of Microelectronic Systems in Malaysia. [11] By 2007 the EGI was supported by 36 countries. [12]

Logo 2004-2010 Enabling Grids for E-sciencE logo.jpg
Logo 2004–2010

A middleware software package known as gLite was developed for EGEE. [13]

A third two-year project phase was called EGEE-III, running from 2008 to 2010. On 30 April 2010 the EGEE project ended. [14] By 2009 the governance model evolved towards a European Grid Initiative (EGI), building upon National Grid Initiatives (NGIs). [15]

DILIGENT & D4Science

A project called Diligent [16] [17] [18] (DIgital LIbrary Infrastructure on Grid ENabled Technology) was funded from 1 September 2004 to 30 November 2007, and developed an open source software called gCube system [19] conceived to be interoperable with the EGEE technology. This project cost more than 8.9 million euro. Follow-on projects called distributed collaboratories infrastructure on grid enabled technology 4 science (D4Science) cost about 3.9 million euro through the end of 2009, [20] and 5.4 million euro until September 2011. [21] These projects initiatied the development of the D4Science organization and its Data Infrastructure that is interoperable with the EGI services.

BEinGRID

A project on Business Experiments in GRID (BEinGRID) ran from 1 June 2006, through November 2009, with a cost estimated 23.6 million euro. [22] The project published 25 case studies. [23] One participant observed in 2008 "a reluctant and slow take-off of Grid technology by the industry." [24]

Yaim

YAIM was initially called Yet Another Installation Method in 2007 when developed at CERN. [25]

YAIM was implemented to configure grid services, but also can be a general purpose configuration tool independent from the grid middleware. YAIM aimed to provide simple configuration methods that can be used to set up uniform grid sites but can also be easily adapted to meet the needs of larger sites. To adapt to local requirements, it was implemented as a set of bash scripts. To support EGEE's component based release model YAIM 4 was modularized and a YAIM core is supplemented by component specific scripts, distributed as separate rpms.

YAIM's modular structure allows distributed and asynchronous development for the quickly changing configuration requirements of the Grid Middleware.

The hierarchical configuration storage - which is to reflect the architecture of a grid site - and the configurable function behavior implement the local settings along with YAIM's default configuration.

Design study

The EGI Design Study (EGI_DS) project was launched in September 2008 and continued until the end of December 2009. The project was partially funded by the European Commission's 7th Framework Programme in order to: evaluate requirements and use cases, identify processes and mechanisms for establishment, define the structure, and initiate the organization. [26] [27]

The study was directed by Dieter Kranzmüller, and cost about 3.9 million euro. [1] [28] In October 2008 site selection for EGI was begun. [29] Participants included 37 national projects. [30]

Structure

In March 2009, the policy board of the EGI announced it would be hosted in Amsterdam, the Netherlands at the Science Park Amsterdam. [31] The EGI.eu foundation was officially formed on 8 February 2010 in Amsterdam. [32] The name change included using infrastructure as the third word for the acronym, to reflect the transition from a series of short-term research projects to a more sustainable service. [33]

National Grid Initiatives (NGI) support scientific disciplines for computational resources within individual countries. The EGI is governed by a Council of representatives from each member NGI, which controls an executive that manages the international collaboration between NGI, so that individual researchers can share and combine computing resources in international collaborative research projects.

The governance model of the EGI coordinating the collaboration of National Grid Initiatives uses general policies of co-operation and subsidiarity adopted in the European Research Area.

A 32-million-euro project named the EGI-Integrated Sustainable Pan-European Infrastructure for Research in Europe (EGI.INSPIRE) was funded in September 2010 under direction of Steven Newhouse. [34] A 1.5-million-euro project called e-ScienceTalk was funded in 2010 for 33 months to support websites and publications covering the EGI. [35] It followed an earlier programme known as GridTalk that was funded from 2008 to 2010. [36] [37]

See also

Related Research Articles

Grid computing is the use of widely distributed computer resources to reach a common goal. A computing grid can be thought of as a distributed system with non-interactive workloads that involve many files. Grid computing is distinguished from conventional high-performance computing systems such as cluster computing in that grid computers have each node set to perform a different task/application. Grid computers also tend to be more heterogeneous and geographically dispersed than cluster computers. Although a single grid can be dedicated to a particular application, commonly a grid is used for a variety of purposes. Grids are often constructed with general-purpose grid middleware software libraries. Grid sizes can be quite large.

E-Science or eScience is computationally intensive science that is carried out in highly distributed network environments, or science that uses immense data sets that require grid computing; the term sometimes includes technologies that enable distributed collaboration, such as the Access Grid. The term was created by John Taylor, the Director General of the United Kingdom's Office of Science and Technology in 1999 and was used to describe a large funding initiative starting in November 2000. E-science has been more broadly interpreted since then, as "the application of computer technology to the undertaking of modern scientific investigation, including the preparation, experimentation, data collection, results dissemination, and long-term storage and accessibility of all materials generated through the scientific process. These may include data modeling and analysis, electronic/digitized laboratory notebooks, raw and fitted data sets, manuscript production and draft versions, pre-prints, and print and/or electronic publications." In 2014, IEEE eScience Conference Series condensed the definition to "eScience promotes innovation in collaborative, computationally- or data-intensive research across all disciplines, throughout the research lifecycle" in one of the working definitions used by the organizers. E-science encompasses "what is often referred to as big data [which] has revolutionized science... [such as] the Large Hadron Collider (LHC) at CERN... [that] generates around 780 terabytes per year... highly data intensive modern fields of science...that generate large amounts of E-science data include: computational biology, bioinformatics, genomics" and the human digital footprint for the social sciences.

UNICORE (UNiform Interface to COmputing REsources) is a grid computing technology for resources such as supercomputers or cluster systems and information stored in databases. UNICORE was developed in two projects funded by the German ministry for education and research (BMBF). In European-funded projects UNICORE evolved to a middleware system used at several supercomputer centers. UNICORE served as a basis in other research projects. The UNICORE technology is open source under BSD licence and available at SourceForge.

<span class="mw-page-title-main">TeraGrid</span>

TeraGrid was an e-Science grid computing infrastructure combining resources at eleven partner sites. The project started in 2001 and operated from 2004 through 2011.

<span class="mw-page-title-main">NorduGrid</span> Grid computing project

NorduGrid is a collaboration aiming at development, maintenance and support of the free Grid middleware, known as the Advanced Resource Connector (ARC).

<span class="mw-page-title-main">Advanced Resource Connector</span> Grid computing software

Advanced Resource Connector (ARC) is a grid computing middleware introduced by NorduGrid. It provides a common interface for submission of computational tasks to different distributed computing systems and thus can enable grid infrastructures of varying size and complexity. The set of services and utilities providing the interface is known as ARC Computing Element (ARC-CE). ARC-CE functionality includes data staging and caching, developed in order to support data-intensive distributed computing. ARC is an open source software distributed under the Apache License 2.0.

The D-Grid Initiative was a government project to fund computer infrastructure for education and research (e-Science) in Germany. It uses the term grid computing. D-Grid started September 1, 2005 with six community projects and an integration project (DGI) as well as several partner projects.

<span class="mw-page-title-main">Open Grid Forum</span> Computing standards organization

The Open Grid Forum (OGF) is a community of users, developers, and vendors for standardization of grid computing. It was formed in 2006 in a merger of the Global Grid Forum and the Enterprise Grid Alliance. The OGF models its process on the Internet Engineering Task Force (IETF), and produces documents with many acronyms such as OGSA, OGSI, and JSDL.

<span class="mw-page-title-main">University Computing Centre</span>

The University Computing Centre in Zagreb has a long tradition in the area of information and communication technologies. It was founded in 1971 within the University of Zagreb, the only Croatian university at the time, with the purpose to enhance the implementation of information technologies in the academic community as well as in Croatia in general.

The Nordic Data Grid Facility, or NDGF, is a common e-Science infrastructure provided by the Nordic countries for scientific computing and data storage. It is the first and so far only internationally distributed WLCG Tier1 center, providing computing and storage services to experiments at CERN.

The INFN Grid project was an initiative of the Istituto Nazionale di Fisica Nucleare (INFN) —Italy's National Institute for Nuclear Physics—for grid computing. It was intended to develop and deploy grid middleware services to allow INFN's users to transparently and securely share the computing and storage resources together with applications and technical facilities for scientific collaborations.

gLite Grid computing software

gLite is a middleware computer software project for grid computing used by the CERN LHC experiments and other scientific domains. It was implemented by collaborative efforts of more than 80 people in 12 different academic and industrial research centers in Europe. gLite provides a framework for building applications tapping into distributed computing and storage resources across the Internet. The gLite services were adopted by more than 250 computing centres, and used by more than 15000 researchers in Europe and around the world.

<span class="mw-page-title-main">P-GRADE Portal</span> Grid computing software

The P-GRADE Grid Portal was software for web portals to manage the life-cycle of executing a parallel application in grid computing. It was developed by the MTA SZTAKI Laboratory of Parallel and Distributed Systems (LPDS) at the Hungarian Academy of Sciences, Hungary, from around 2005 through 2010.

gUSE Grid computing framework

The Grid and Cloud User Support Environment (gUSE), also known as WS-PGRADE /gUSE, is an open source science gateway framework that enables users to access grid and cloud infrastructures. gUSE is developed by the Laboratory of Parallel and Distributed Systems (LPDS) at Institute for Computer Science and Control (SZTAKI) of the Hungarian Academy of Sciences.

The SHIWA project within grid computing was a project led by the LPDS of MTA Computer and Automation Research Institute. The project coordinator was Prof. Dr. Peter Kacsuk. It started on 1 July 2010 and lasted two years. SHIWA was supported by a grant from the European Commission's FP7 INFRASTRUCTURES-2010-2 call under grant agreement n°261585.

Polish Grid Infrastructure PL-Grid, a nationwide computing structure, built in 2009-2011, under the scientific project PL-Grid – Polish Infrastructure for Supporting Computational Science in the European Research Space. Its purpose was to enable scientific research based on advanced computer simulations and large-scale computations using the computer clusters, and to provide convenient access to the computer resources for research teams, also outside the communities, in which the high performance computing centers operate.

<span class="mw-page-title-main">Supercomputing in Europe</span> Overview of supercomputing in Europe

Several centers for supercomputing exist across Europe, and distributed access to them is coordinated by European initiatives to facilitate high-performance computing. One such initiative, the HPC Europa project, fits within the Distributed European Infrastructure for Supercomputing Applications (DEISA), which was formed in 2002 as a consortium of eleven supercomputing centers from seven European countries. Operating within the CORDIS framework, HPC Europa aims to provide access to supercomputers across Europe.

GridPP is a collaboration of particle physicists and computer scientists from the United Kingdom and CERN. They manage and maintain a distributed computing grid across the UK with the primary aim of providing resources to particle physicists working on the Large Hadron Collider (LHC) experiments at CERN. They are funded by the UK's Science and Technology Facilities Council. The collaboration oversees a major computing facility called the Tier1 at the Rutherford Appleton Laboratory (RAL) along with the four Tier 2 organisations of ScotGrid, NorthGrid, SouthGrid and LondonGrid. The Tier 2s are geographically distributed and are composed of computing clusters at multiple institutes.

<span class="mw-page-title-main">European Middleware Initiative</span>

The European Middleware Initiative (EMI) is a computer software platform for high performance distributed computing. It is developed and distributed directly by the EMI project. It is the base for other grid middleware distributions used by scientific research communities and distributed computing infrastructures all over the world especially in Europe, South America and Asia. EMI supports broad scientific experiments and initiatives, such as the Worldwide LHC Computing Grid.

D4Science is an organisation operating a Data Infrastructure offering a rich array of services by community-driven virtual research environments. In particular, it supports communities of practice willing to implement open science practices. The infrastructure follows the system of systems approach, where the constituent systems offer “resources” assembled together to implement the overall set of D4Science services. In particular, D4Science aggregates “domain agnostic” service providers as well as community-specific ones to build a unifying space where the aggregated resources can be exploited via Virtual research Environments and their services.

References

  1. 1 2 Christopher Lazou (14 April 2008). "Grid Infrastructure Organization". European Parliament Magazine. Vol. 265. pp. 8–9. Retrieved 2 October 2011.
  2. 1 2 "History of EGI". EGI.eu website. Archived from the original on 2 March 2012. Retrieved 4 October 2011.
  3. ↑ European Competitiveness Council, Conclusions on European Research Infrastructures and their regional dimension, Brussels, 3 June 2008
  4. ↑ "EGEE‐III Project Publishable Summary" (PDF). Enabling Grids for E-sciencE. 8 June 2010. Archived from the original (PDF) on 6 April 2013. Retrieved 2 October 2011.
  5. ↑ "The DataGrid Project". CERN. March 2004. Archived from the original on 31 August 2016. Retrieved 2 October 2011.
  6. ↑ "Research and Technological Development for an International Data Grid". Project IST-2000-25182 description. European Community Research and Development Information Service. Retrieved 7 October 2011.
  7. ↑ Robert J. Wilson (22 October 2001). "The European DataGrid Project" (PDF). Snowmass 2001. American Physical Society. Retrieved 2 October 2011.
  8. 1 2 "Enabling grids For E-science". Project description. European Community Research and Development Information Service. Retrieved 2 October 2011.
  9. ↑ LHC Computing Grid: Technical Design Report (PDF). The LCG TDR Editorial Board. 20 June 2005. ISBN   92-9083-253-3 . Retrieved 3 October 2011.{{cite book}}: |work= ignored (help)
  10. ↑ "Enabling grids for E-sciencE-II". Project description. European Community Research and Development Information Service. Retrieved 2 October 2011.
  11. ↑ Jo Timbuong (24 April 2008). "Computer grid grows in power". The Star Online. Kuala Lumpur. Archived from the original on 13 July 2012. Retrieved 2 October 2011.
  12. ↑ Leena Jukka (5 September 2007). "36 national grids initiatives in Europe support the EGI concept". Innovations Report. Retrieved 3 October 2011.
  13. ↑ "gLite – Lightweight Middleware for Grid Computing". CERN. Retrieved 3 October 2011.
  14. ↑ "EGEE Portal: Enabling Grids for E-sciencE". Official web site. Archived from the original on 19 June 2010. Retrieved 2 October 2011.
  15. ↑ "ICT Infrastructures for e-Science". Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee of the Regions. Brussels. 5 March 2009. Retrieved 2 October 2011.
  16. ↑ "Diligent Project". Defunct website. Archived from the original on 4 November 2008. Retrieved 3 October 2011.
  17. ↑ "A testbed digital library infrastructure on grid enabled technology". Project description. European Community Research and Development Information Service. Retrieved 3 October 2011.
  18. ↑ Candela, L.; Akal, F.; Avancini, H.; Castelli, D.; Fusco, L.; Guidetti, V.; Langguth, C.; Manzi, A.; Pagano, P.; Schuldt, H.; Simi, M.; Springmann, M.; Voicu, L. (2007). "DILIGENT: integrating digital library and Grid technologies for a new Earth observation research infrastructure". International Journal on Digital Libraries. 7 (1–2): 59–80. doi:10.1007/s00799-007-0023-8. S2CID   29730933.
  19. ↑ Assante, M.; Candela, L.; Castelli, D.; Cirillo, R.; Coro, G.; Frosini, L.; Lelii, L.; Mangiacrapa, F.; Marioli, V.; Pagano, P.; Panichi, G.; Perciante, C.; Sinibaldi, F. (2019). "The gCube system: Delivering Virtual Research Environments as-a-Service". Future Generation Computer Systems. 95: 445–453. doi:10.1016/j.future.2018.10.035.
  20. ↑ "DIstributed colLaboratories infrastructure on Grid enabled technology 4 science". Project description. European Community Research and Development Information Service. Retrieved 3 October 2011.
  21. ↑ "Data Infrastructure Ecosystem for Science". Project description. European Community Research and Development Information Service. Retrieved 3 October 2011.
  22. ↑ "Business experiments in GRID". Project 034702 fact sheet. European Community Research and Development Information Service. Retrieved 9 October 2011.
  23. ↑ "Business Experiments in Grid: BEinGRID". Project web site. European Community Research and Development Information Service. Archived from the original on 23 July 2011. Retrieved 9 October 2011.
  24. ↑ Claire Gresty; George Thanos; Theo Dimitrakos; Paul Warren (January 2008). "Service oriented infrastructure: Meeting customer needs". BT Technology Journal. 26 (1). Retrieved 9 October 2011.
  25. ↑ "YAIM 4 reference guide" (PDF). CERN. 30 August 2007. Retrieved 3 October 2011.
  26. ↑ "European Grid Initiative Design Study". Official website. 2009. Archived from the original on 20 September 2011. Retrieved 2 October 2011.
  27. ↑ "Europeans connecting through the Grid". European Research Headlines. European Commission. 14 September 2007. Retrieved 3 October 2011.
  28. ↑ "European grid initiative design study". Project 211693 description. European Community Research and Development Information Service. Retrieved 2 October 2011.
  29. ↑ "Euroopan grid-organisaatio hakee kotipaikkaa". Digitoday (in Finnish). 17 October 2008. Retrieved 2 October 2011.
  30. ↑ "Scientists join forces to create European grid computing infrastructure". News release. European Community Research and Development Information Service. 25 September 2007. Retrieved 2 October 2011.
  31. ↑ "European Grid Initiative Head Office comes to Science Park Amsterdam". News release. University of Amsterdam. 23 March 2009. Retrieved 2 October 2011.
  32. ↑ Rob van der Meer (11 February 2010). "EGI.eu founded". News release. EGI.eu. Archived from the original on 5 March 2012. Retrieved 3 October 2011.
  33. ↑ "Establishing EGI.eu". Projects. British Publishers. p. 26. Retrieved 3 October 2011.
  34. ↑ "Work on pan-European grid infrastructure moves to next level". News release. European Community Research and Development Information Service. 16 September 2010. Retrieved 2 October 2011.
  35. ↑ "e-ScienceTalk: Supporting Grid and High Performance Computing reporting across Europe". Project 260733 summary. European Community Research and Development Information Service. 1 September 2010. Retrieved 4 October 2011.
  36. ↑ "GridTalk: co-ordinating grid reporting across Europe". Project 223534 summary. European Community Research and Development Information Service. 1 May 2010. Retrieved 4 October 2011.
  37. ↑ "About GridTalk". Official website. 2010. Archived from the original on 25 April 2012. Retrieved 4 October 2011.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.