3G Bridge

Last updated

The Generic Grid-Grid (3G) Bridge is an open-source core job bridging component between different grid infrastructures. Its development started in 2008 within the CancerGrid and EDGeS projects. The aim was to create a generic bridge component that can be used in different grid interoperability scenarios. The 3G Bridge used within the EDGeS project that provides the core component of the Service Grid - Desktop Grid interoperability solution. 3G Bridge helps to connect user communities of different grid systems. For example, communities working on parameter sweep problems (physics, biologists, ...) and using service grid infrastructures can migrate their applications to the more adequate desktop grid platform using the 3G Bridge technology, resulting in an accelerated research.

Contents

The project, started in 2008, is technically coordinated by Prof. Peter Kacsuk, head of the Laboratory of Parallel and Distributed Systems and Msc. Zoltan Farkas, research leader of LPDS at MTA-SZTAKI, Hungary.

Architecture

3G Bridge is currently actively used in the EDGI and DEGSICO projects as the job bridging technology between service and desktop grids. 3G Bridge enables interconnecting different grid technologies by migrating jobs between them. This way it allows to move specific type of applications to an adequate execution platform, and enables load sharing among different grid infrastructures, resulting in increased job capacity and faster execution.

The 3G Bridge has the following main components:

Job Handler Interface: this is a very simple MySQL interface for handling jobs in the 3G Bridge Job Database. Can be used by local MySQL clients, or a very simple application shipped with the 3G Bridge called 'injector'.

WSSubmitter: this is a Web Service interface above the 3G Bridge Job Database. Using this interface, remote clients can add jobs to the 3G Bridge for execution, or check the status of already submitted ones.

3G Bridge Job Database: this is a MySQL database that contains the set of jobs the 3G Bridge should handle. Important properties of the jobs, like executable name, command-line arguments, list of files used is stored within the database.

Download Manager: this component is responsible for fetching the input files belonging to jobs submitted in a remote manner.

Queue Manager: this component arranges the execution and management of 3G Bridge jobs, periodically calls the different grid plugins in order to manage jobs assigned to them.

Grid Handler Interface: this is a generic interface above the grid plugins. By using this generic layer, no 3G Bridge components other than the grid plugins must contain grid-specific code. The methods a grid plugin should implement is: submission and status update.

Grid Plugin: grid plugins are responsible for managing jobs in destination grids. For each plugin, there can be a number of instances, for example for submitting jobs to different EGEE VOs. The 3G Bridge supports the following grids: EGEE, BOINC, XtremWeb-HEP, and OurGrid.

HTTPD: is a web server so remote clients can fetch output files of finished jobs from the 3G Bridge server. The main programming language is C++, however 3G Bridge supports implementing grid plugins in Java. Web service interfaces are implemented using gSOAP.

Destination plugins

communication ratio.

See also

Related Research Articles

Grid computing is the use of widely distributed computer resources to reach a common goal. A computing grid can be thought of as a distributed system with non-interactive workloads that involve many files. Grid computing is distinguished from conventional high-performance computing systems such as cluster computing in that grid computers have each node set to perform a different task/application. Grid computers also tend to be more heterogeneous and geographically dispersed than cluster computers. Although a single grid can be dedicated to a particular application, commonly a grid is used for a variety of purposes. Grids are often constructed with general-purpose grid middleware software libraries. Grid sizes can be quite large.

WebObjects Java web application server and framework originally developed by NeXT Software

WebObjects is a Java web application server and a server-based web application framework originally developed by NeXT Software, Inc. As of 2009 the software has been independently maintained by a volunteer community.

HTCondor is an open-source high-throughput computing software framework for coarse-grained distributed parallelization of computationally intensive tasks. It can be used to manage workload on a dedicated cluster of computers, or to farm out work to idle desktop computers – so-called cycle scavenging. HTCondor runs on Linux, Unix, Mac OS X, FreeBSD, and Microsoft Windows operating systems. HTCondor can integrate both dedicated resources and non-dedicated desktop machines into one computing environment.

A GIS software program is a computer program to support the use of a geographic information system, providing the ability to create, store, manage, query, analyze, and visualize geographic data, that is, data representing phenomena for which location is important. The GIS software industry encompasses a broad range of commercial and open-source products that provide some or all of these capabilities within various information technology architectures.

UNICORE is a grid computing technology for resources such as supercomputers or cluster systems and information stored in databases. UNICORE was developed in two projects funded by the German ministry for education and research (BMBF). In European-funded projects UNICORE evolved to a middleware system used at several supercomputer centers. UNICORE served as a basis in other research projects. The UNICORE technology is open source under BSD licence and available at SourceForge.

Catalyst (software)

Catalyst is an open source web application framework written in Perl, that closely follows the model–view–controller (MVC) architecture, and supports a number of experimental web patterns. It is written using Moose, a modern object system for Perl. Its design is heavily inspired by such frameworks as Ruby on Rails, Maypole, and Spring.

SZTAKI Desktop Grid (SzDG) was a BOINC project located in Hungary run by the Computer and Automation Research Institute (SZTAKI) of the Hungarian Academy of Sciences. It closed on June 21, 2018.

Distributed Resource Management Application API (DRMAA) is a high-level Open Grid Forum (OGF) API specification for the submission and control of jobs to a distributed resource management (DRM) system, such as a cluster or grid computing infrastructure. The scope of the API covers all the high level functionality required for applications to submit, control, and monitor jobs on execution resources in the DRM system.

Open Grid Forum Organization

The Open Grid Forum (OGF) is a community of users, developers, and vendors for standardization of grid computing. It was formed in 2006 in a merger of the Global Grid Forum and the Enterprise Grid Alliance. The OGF models its process on the Internet Engineering Task Force (IETF), and produces documents with many acronyms such as OGSA, OGSI, and JSDL.

BOINC client–server technology

BOINC client–server technology refers to the model under which BOINC works. The BOINC framework consists of two layers which operate under the client–server architecture. Once the BOINC software is installed in a machine, the server starts sending tasks to the client. The operations are performed client-side and the results are uploaded to the server-side.

The Base One Foundation Component Library (BFC) is a rapid application development toolkit for building secure, fault-tolerant, database applications on Windows and ASP.NET. In conjunction with Microsoft's Visual Studio integrated development environment, BFC provides a general-purpose web application framework for working with databases from Microsoft, Oracle, IBM, Sybase, and MySQL, running under Windows, Linux/Unix, or IBM iSeries or z/OS. BFC also includes facilities for distributed computing, batch processing, queuing, and database command scripting, and these run under Windows or Linux with Wine.

gLite

gLite is a middleware computer software project for grid computing used by the CERN LHC experiments and other scientific domains. It was implemented by collaborative efforts of more than 80 people in 12 different academic and industrial research centers in Europe. gLite provides a framework for building applications tapping into distributed computing and storage resources across the Internet. The gLite services were adopted by more than 250 computing centres, and used by more than 15000 researchers in Europe and around the world.

Vine Toolkit is an open source software framework that is used to create Grid-aware web applications.

P-GRADE Portal

The P-GRADE Grid Portal was software for web portals to manage the life-cycle of executing a parallel application in grid computing. It was developed by the Laboratory of Parallel and Distributed Systems (LPDS) at the Hungarian Academy of Sciences, Hungary.

MTA SZTAKI Laboratory of Parallel and Distributed Systems

The Laboratory of Parallel and Distributed Systems (LPDS), as a department of MTA SZTAKI, is a research laboratory in distributed grid and cloud technologies. LPDS is a founding member of the Hungarian Grid Competence Centre, the Hungarian National Grid Initiative and the Hungarian OpenNebula Community and also coordinates several European grid/cloud projects.

gUSE

The Grid and Cloud User Support Environment (gUSE), also known as WS-PGRADE /gUSE, is an open source science gateway framework that enables users to access grid and cloud infrastructures. gUSE is developed by the Laboratory of Parallel and Distributed Systems (LPDS) at Institute for Computer Science and Control (SZTAKI) of the Hungarian Academy of Sciences.

Róbert Lovas

Róbert Lovas is a Hungarian computer scientist at SZTAKI, Budapest, Hungary.

SLinCA@Home

SLinCA@Home was a research project that uses Internet-connected computers to do research in fields such as physics and materials science.

The SHIWA is a project led by the LPDS of MTA Computer and Automation Research Institute. The project coordinator is Prof. Dr. Peter Kacsuk, it started on 1 July 2010 and lasted two years. SHIWA was supported by a Grant from the European Commission's FP7 INFRASTRUCTURES-2010-2 call under grant agreement n°261585.

XQuery API for Java

XQuery API for Java (XQJ) refers to the common Java API for the W3C XQuery 1.0 specification.

References