UNICORE

Last updated

UNICORE (UNiform Interface to COmputing REsources) is a grid computing technology for resources such as supercomputers or cluster systems and information stored in databases. UNICORE was developed in two projects funded by the German ministry for education and research (BMBF). In European-funded projects UNICORE evolved to a middleware system used at several supercomputer centers. UNICORE served as a basis in other research projects. The UNICORE technology is open source under BSD licence and available at SourceForge.

Contents

History

The concept of grid computing was first introduced in the book "The Grid: Blueprint for a New Computing Infrastructure" at the end of 1998. By 1997, the development of UNICORE was initiated for German supercomputer centers as an alternative for the Globus Toolkit. The first prototype was developed in the German UNICORE project, while the foundations for the production version were laid in the follow-up project UNICORE Plus, which ended in 2002.

Follow-up European projects extended the functionality and worked towards providing implementations of Open Grid Forum standards. These resulted in the release of UNICORE 6 on 28 August 2007.

Architecture

UNICORE consists of three layers: a user, server, and target system tier. The user tier is represented by various clients. The primary clients are the UNICORE Rich Client, a graphical user interface based on the Eclipse framework, and the UNICORE commandline client (UCC). The clients use SOAP Web services to communicate with the server tier. XML documents are used to transmit platform and site independent descriptions of computational and data related tasks, resource information, and workflow specifications between client and server. The servers are accessible only via the Secure Sockets Layer protocol.

As the single secure entry point to a UNICORE site, the Gateway accepts and authenticates all requests, and forwards them to the target service. A further server, UNICORE/X, is used to access a particular set of Grid resources at a site. UNICORE supports many different system architectures and ensures that organization full control over its resources. UNICORE/X servers may be used to access a supercomputer, a Linux cluster or a single PC. The UNICORE/X server creates concrete target system specific actions from the XML job description (Abstract Job Objects, AJO [1] ) received from the client. Available UNICORE services include job submission and job management, file access, file transfer (both client-server and server-server), storage operations (mkdir, ls, etc.), and workflow submission and management. The target system tier consists of the Target System Interface (TSI), which directly interfaces with the underlying local operating system and resource management system.

Security model

The security within UNICORE relies on the usage of permanent X.509 certificates issued by a trusted Certificate Authority (CA). These certificates are used to provide a single sign-on in the UNICORE client, i.e. no further password requests are handed to the user. In addition the certificates are used for authentication and authorization, including the mapping of UNICORE user certificates to local accounts, e.g. Unix uid/gid, and for signing XML requests, which are sent over SSL based communication channels across 'insecure' internet links. Using X.509 certificates is one example for the consideration of well-known standards, e.g. released by the Global Grid Forum (GGF), within the UNICORE architecture. For trust delegation, UNICORE uses signed SAML assertions, while local authorisation is controlled by XACML policies.

Licensing

All components of the UNICORE technology are open source software under BSD license and can be downloaded from the SourceForge repository.

UNICORE in research & production

Many European and international research projects base their Grid software implementations on UNICORE, e.g. EUROGRID, GRIP, OpenMolGRID, VIOLA, or the Japanese NaReGI project. These projects extended or are extending the set of core UNICORE functions, including new features specific to their research or project focus. The goals of such projects are not only limited to the computer science community. Other scientific domains such as bioengineering or computational chemistry are also using UNICORE as the basis for their work and research, like in the OpenMolGRID or Chemomentum projects. Within the European DEISA project leading HPC centers in Europe joined to deploy and operate a pervasive, distributed, heterogeneous, multi-tera-scale supercomputing platform. UNICORE was used as the Grid middleware to access the DEISA resources.

UNICORE is deployed in distributed computing infrastructures in Europe (PRACE, the European Grid Infrastructure EGI) and is foreseen to be deployed in the upcoming Extreme Science and Engineering Discovery Environment (XSEDE) infrastructure in the United States.

UNICORE is successfully used in production environments, e.g. within the John von Neumann-Institute for Computing (NIC) to access the 294912 core "JUGENE" IBM BlueGene/P supercomputer and the 26304 core "JUROPA" cluster. The users of these resources come from a broad field of scientific domains including e.g. astrophysics, quantum physics, medicine, biology, computational chemistry, and climatology.

Related Research Articles

Grid computing is the use of widely distributed computer resources to reach a common goal. A computing grid can be thought of as a distributed system with non-interactive workloads that involve many files. Grid computing is distinguished from conventional high-performance computing systems such as cluster computing in that grid computers have each node set to perform a different task/application. Grid computers also tend to be more heterogeneous and geographically dispersed than cluster computers. Although a single grid can be dedicated to a particular application, commonly a grid is used for a variety of purposes. Grids are often constructed with general-purpose grid middleware software libraries. Grid sizes can be quite large.

HTCondor is an open-source high-throughput computing software framework for coarse-grained distributed parallelization of computationally intensive tasks. It can be used to manage workload on a dedicated cluster of computers, or to farm out work to idle desktop computers – so-called cycle scavenging. HTCondor runs on Linux, Unix, Mac OS X, FreeBSD, and Microsoft Windows operating systems. HTCondor can integrate both dedicated resources and non-dedicated desktop machines into one computing environment.

Grid MP is a commercial distributed computing software package developed and sold by Univa, a privately held company based primarily in Austin, Texas. It was formerly known as the MetaProcessor prior to the release of version 4.0, however the letters MP in Grid MP do not officially stand for anything.

NorduGrid

NorduGrid is a collaboration aiming at development, maintenance and support of the free Grid middleware, known as the Advanced Resource Connector (ARC).

Advanced Resource Connector

Advanced Resource Connector (ARC) is a grid computing middleware introduced by NorduGrid. It provides a common interface for submission of computational tasks to different distributed computing systems and thus can enable grid infrastructures of varying size and complexity. The set of services and utilities providing the interface is known as ARC Computing Element (ARC-CE). ARC-CE functionality includes data staging and caching, developed in order to support data-intensive distributed computing. ARC is an open source software distributed under the Apache License 2.0.

Volunteer computing is a type of distributed computing in which people donate their computers' unused resources to a research-oriented project. The fundamental idea behind it is that a modern desktop computer is sufficiently powerful to perform billions of operations a second, but for most users only between 10-15% of its capacity is used. Typical uses like basic word processing or web browsing leave the computer mostly idle.

The Simple API for Grid Applications (SAGA) is a family of related standards specified by the Open Grid Forum to define an application programming interface (API) for common distributed computing functionality.

Xgrid

Xgrid is a proprietary program and distributed computing protocol developed by the Advanced Computation Group subdivision of Apple Inc that allows networked computers to contribute to a single task.

Computer cluster

A computer cluster is a set of computers that work together so that they can be viewed as a single system. Unlike grid computers, computer clusters have each node set to perform the same task, controlled and scheduled by software.

gLite

gLite is a middleware computer software project for grid computing used by the CERN LHC experiments and other scientific domains. It was implemented by collaborative efforts of more than 80 people in 12 different academic and industrial research centers in Europe. gLite provides a framework for building applications tapping into distributed computing and storage resources across the Internet. The gLite services were adopted by more than 250 computing centres, and used by more than 15000 researchers in Europe and around the world.

Techila Distributed Computing Engine is a commercial grid computing software product. It speeds up simulation, analysis and other computational applications by enabling scalability across the IT resources in user's on-premises data center and in the user's own cloud account. Techila Distributed Computing Engine is developed and licensed by Techila Technologies Ltd, a privately held company headquartered in Tampere, Finland. The product is also available as an on-demand solution in Google Cloud Launcher, the online marketplace created and operated by Google. According to IDC, the solution enables organizations to create HPC infrastructure without the major capital investments and operating expenses required by new HPC hardware.

MTA SZTAKI Laboratory of Parallel and Distributed Systems

The Laboratory of Parallel and Distributed Systems (LPDS), as a department of MTA SZTAKI, is a research laboratory in distributed grid and cloud technologies. LPDS is a founding member of the Hungarian Grid Competence Centre, the Hungarian National Grid Initiative and the Hungarian OpenNebula Community and also coordinates several European grid/cloud projects.

gUSE

The Grid and Cloud User Support Environment (gUSE), also known as WS-PGRADE /gUSE, is an open source science gateway framework that enables users to access grid and cloud infrastructures. gUSE is developed by the Laboratory of Parallel and Distributed Systems (LPDS) at Institute for Computer Science and Control (SZTAKI) of the Hungarian Academy of Sciences.

The SHIWA is a project led by the LPDS of MTA Computer and Automation Research Institute. The project coordinator is Prof. Dr. Peter Kacsuk, it started on 1 July 2010 and lasted two years. SHIWA was supported by a Grant from the European Commission's FP7 INFRASTRUCTURES-2010-2 call under grant agreement n°261585.

GridRPC is Remote Procedure Call over a grid. This paradigm has been proposed by the GridRPC working group of the Open Grid Forum (OGF), and an API has been defined in order for clients to access remote servers as simply as a function call. It is used among numerous Grid middleware for its simplicity of implementation, and has been standardized by the OGF in 2007. For interoperability reasons between the different existing middleware, the API has been followed by a document describing good use and behavior of the different GridRPC API implementations. Works have then been conducted on the GridRPC Data Management, which has been standardized in 2011.

DIET

DIET is a software for grid-computing. As middleware, DIET sits between the operating system and the application software. DIET was created in 2000. It was designed for high-performance computing. It is currently developed by INRIA, École Normale Supérieure de Lyon, CNRS, Claude Bernard University Lyon 1, SysFera. It is open-source software released under the CeCILL license.

Supercomputing in Europe Overview of supercomputing in Europe

Several centers for supercomputing exist across Europe, and distributed access to them is coordinated by European initiatives to facilitate high-performance computing. One such initiative, the HPC Europa project, fits within the Distributed European Infrastructure for Supercomputing Applications (DEISA), which was formed in 2002 as a consortium of eleven supercomputing centers from seven European countries. Operating within the CORDIS framework, HPC Europa aims to provide access to supercomputers across Europe.

Quasi-opportunistic supercomputing

Quasi-opportunistic supercomputing is a computational paradigm for supercomputing on a large number of geographically disperse computers. Quasi-opportunistic supercomputing aims to provide a higher quality of service than opportunistic resource sharing.

European Middleware Initiative

The European Middleware Initiative (EMI) is a computer software platform for high performance distributed computing. It is developed and distributed directly by the EMI project. It is the base for other grid middleware distributions used by scientific research communities and distributed computing infrastructures all over the world especially in Europe, South America and Asia. EMI supports broad scientific experiments and initiatives, such as the Worldwide LHC Computing Grid.

Science gateways provide access to advanced resources for science and engineering researchers, educators, and students. Through streamlined, online, user-friendly interfaces, gateways combine a variety of cyberinfrastructure (CI) components in support of a community-specific set of tools, applications, and data collections.: In general, these specialized, shared resources are integrated as a Web portal, mobile app, or a suite of applications. Through science gateways, broad communities of researchers can access diverse resources which can save both time and money for themselves and their institutions. As listed below, functions and resources offered by science gateways include shared equipment and instruments, computational services, advanced software applications, collaboration capabilities, data repositories, and networks.

References

  1. Laurence T. Yang; Minyi Guo (2005). High-Performance Computing: Paradigm and Infrastructure. John Wiley & Sons. p. 438. ISBN   978-0-471-73270-9.