GStat

Last updated
GStat
Developer(s) Joanna Huang (Academia Sinica), Laurence Field, David Horat (CERN)
Stable release
2.0 RC 2 / 19 February 2010
Operating system Scientific Linux 5
Type Grid computing
License Apache License, Version 2.0
Website http://cern.ch/gridinfo

GStat is a web application which is aimed at displaying information about grid services, the grid information system itself and related metrics. The system is designed in modular way so that the software can be reused in different application scenarios. [1]

Contents

History

Gstat has evolved over the past few years from a simple CGI script that displayed the summary of a grid infrastructure to a production quality service providing rich features such as information content testing and infrastructure monitoring. An evolutionary approach to its development has enabled GStat to add functionally in response to real use cases and to become a key operational tool. GStat 2.0 is a major redesign of the original version which will ensure that it will meet the future demands of an evolving infrastructure and easily integrate with other operational tools.

Computer-generated imagery application of computer graphics to create or contribute to images

Computer-generated imagery (CGI) is the application of computer graphics to create or contribute to images in art, printed media, video games, films, television programs, shorts, commercials, videos, and simulators. The visual scenes may be dynamic or static and may be two-dimensional (2D), though the term "CGI" is most commonly used to refer to 3D computer graphics used for creating scenes or special effects in films and television. Additionally, the use of 2D CGI is often mistakenly referred to as "traditional animation", most often in the case when dedicated animation software such as Adobe Flash or Toon Boom is not used or the CGI is hand drawn using a tablet and mouse.

GStat is the result of a collaboration between Academia Sinica and the Grid Technology Group at CERN. The main purpose of the joint project is to align GStat with direction taken by the WLCG monitoring group with respect to operational tools and in addition ensure that GStat can make a contribution to middleware certification and site validation.

Academia Sinica National Academy of Taiwan

Academia Sinica, headquartered in Nangang District, Taipei, is the national academy of Taiwan. It supports research activities in a wide variety of disciplines, ranging from mathematical and physical sciences, to life sciences, and to humanities and social sciences. As an educational institute, it provides PhD training and scholarship through its English-language Taiwan International Graduate Program in biology, agriculture, chemistry, physics, informatics, and earth and environmental sciences. Academia Sinica is ranked 144th in Nature Publishing Index - 2014 Global Top 200 and 22nd in Reuters World's Most Innovative Research Institutions. The current president since 2016 is James C. Liao, an expert in metabolic engineering, systems biology and synthetic biology.

CERN International organization which operates the worlds largest particle physics laboratory

The European Organization for Nuclear Research, known as CERN, is a European research organization that operates the largest particle physics laboratory in the world. Established in 1954, the organization is based in a northwest suburb of Geneva on the Franco–Swiss border and has 23 member states. Israel is the only non-European country granted full membership. CERN is an official United Nations Observer.

GStat is compatible with version 1.3 of the Grid Laboratory Uniform Environment data model, taking the information of existing Berkeley Database Information Index instances. Currently there is an effort on developing compatibility with version 2.0 of the Grid Laboratory Uniform Environment data model.

The initial version of GStat were designed and developed by Min Tsai. The current team members can be contacted in the Grid Information Product Team webpage.

High-level system architecture

GStat provides a method to visualize a grid infrastructure from an operational perspective based on information found in the grid information system. Even in the absence of an information system, information about the existence of grid services needs to be communicated. The existence of grid services and the communication of their existence defines the grid infrastructures and as such one of the main concepts in GStat 2.0 is this is should be bootstrapped by the information system endpoint that defines the view of the grid infrastructure. It periodically takes a snapshot of the information system and maintains a cache of the main entities found in the infrastructure which provides the basic structure for the visualization. The main entities cache is also used to configure monitoring framework that monitors the information system and reports the health of the various components from which the information system is composed along with further metrics about the performance. The resulting information from both the information system itself and the monitoring thereof is used to produce various displays that address specific use cases.

The GStat architecture makes a clear separation between data, infrastructure monitoring, content validation and visualization. At the core is the data model used to maintain a snapshot of the information system and a cache of the main entities. Probes are used to monitor the information system components and validation checks are used ensure that the information content is correct. A visualization framework is used for displaying the resulting data. The modular approach enables the software to be reused in other application scenarios.

Gstat is uniquely positioned to support modern day Big Data initiatives.

Implementation

The GStat architecture is implemented using two main frameworks; Django and Nagios. Django is an open source web application framework, written in Python, which follows the model–view–controller architectural pattern. Django models are used to provide the core data model of the system. The snapshot script takes a snapshot of the information system and uses the Django framework to store the information. The import-entities script extracts the main entities, such as Sites and Services, from the snapshot and maintains a cache of entities. In addition, certain attributes are extracted from the snapshot and stored in rrd databases using the gstat-update-rrd script. Nagios is an open source monitoring framework and is used in GStat to both monitor the information system components and validate the information content via the use of custom probes. These monitoring probes can be re-used by other Nagios based monitoring tools and also executed on the command line, which enables them to be easily incorporated in other test suites. Django is also leveraged for the visualization aspects of GStat. The entity cache is used to provide the main structure for the displays. The snapshot and result from testing are used to provide more detailed information.

Django (web framework) Python web framework

Django is a Python-based free and open-source web framework, which follows the model-view-template (MVT) architectural pattern. It is maintained by the Django Software Foundation (DSF), an independent organization established as a 501(c)(3) non-profit.

Nagios computer system and network monitoring application software

Nagios, now known as Nagios Core, is a free and open source computer-software application that monitors systems, networks and infrastructure. Nagios offers monitoring and alerting services for servers, switches, applications and services. It alerts users when things go wrong and alerts them a second time when the problem has been resolved.

Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum and first released in 1991, Python has a design philosophy that emphasizes code readability, notably using significant whitespace. It provides constructs that enable clear programming on both small and large scales. Van Rossum led the language community until stepping down as leader in July 2018.

See also

Related Research Articles

RRDtool free software

RRDtool aims to handle time series data such as network bandwidth, temperatures or CPU load. The data is stored in a circular buffer based database, thus the system storage footprint remains constant over time.

NetApp company

NetApp, Inc. is a hybrid cloud data services and data management company headquartered in Sunnyvale, California. It has ranked in the Fortune 500 since 2012. Founded in 1992 with an IPO in 1995, NetApp offers hybrid cloud data services for management of applications and data across cloud and on-premises environments.

Quattor is a generic open-source tool-kit used to install, configure, and manage computers. Quattor was originally developed in the framework of European Data Grid project (2001-2004). Since its first release in 2003, Quattor has been maintained and extended by a volunteer community of users and developers, primarily from the community of grid system administrators. The Quattor tool-kit, like other configuration management systems, reduces the manpower required to maintain a cluster and facilitates reliable change management. However, three unique features make it particularly attractive for managing grid resources:

3D city models

3D city models are digital models of urban areas that represent terrain surfaces, sites, buildings, vegetation, infrastructure and landscape elements in three-dimensional scale as well as related objects belonging to urban areas. Their components are described and represented by corresponding two-dimensional and three-dimensional spatial data and geo-referenced data. 3D city models support presentation, exploration, analysis, and management tasks in a large number of different application domains. In particular, 3D city models allow "for visually integrating heterogeneous geoinformation within a single framework and, therefore, create and manage complex urban information spaces."

Core architecture data model

Core architecture data model (CADM) in enterprise architecture is a logical data model of information used to describe and build architectures.

gLite

gLite is a middleware computer software project for grid computing used by the CERN LHC experiments and other scientific domains. It was implemented by collaborative efforts of more than 80 people in 12 different academic and industrial research centers in Europe. gLite provides a framework for building applications tapping into distributed computing and storage resources across the Internet. The gLite services were adopted by more than 250 computing centres and used by more than 15000 researchers in Europe and around the world.

Opsview software house

Opsview is a software company specializing in enterprise systems monitoring software for physical, virtual, and cloud-based IT infrastructures. Its headquarters is in Reading, England, with a US office in Woburn, Massachusetts. Its product, Opsview Monitor, provides a single, unified view of IT operations.

Shinken (software)

Shinken is an open source computer system and network monitoring software application compatible with Nagios. It watches hosts and services, gathers performance data and alerts users when error conditions occur and again when the conditions clear.

FEMtools is a multi-functional, cross-platform and solver-independent family of CAE software programs providing analysis and scripting solutions for many different types of applications. The program is developed, supported and licensed by Dynamic Design Solutions ("DDS") NV.

Icinga monitoring software

Icinga is an open-source computer system and network monitoring application. It was originally created as a fork of the Nagios system monitoring application in 2009.

Moose (analysis)

Moose is a free and open source platform for software and data analysis built in Pharo.

GridPP is a collaboration of particle physicists and computer scientists from the United Kingdom and CERN. They manage and maintain a distributed computing grid across the UK with the primary aim of providing resources to particle physicists working on the Large Hadron Collider experiments at CERN. They are funded by the UK's Science and Technology Facilities Council. The collaboration oversees a major computing facility called the Tier1 at the Rutherford Appleton Laboratory (RAL) along with the four Tier 2 organisations of ScotGrid, NorthGrid, SouthGrid and LondonGrid. The Tier 2s are geographically distributed and are composed of computing clusters at multiple institutes.

Check_MK is an IT Monitoring system. It started originally as an extension to the Nagios monitoring system that allows creating rule-based configuration using Python and offloading work from the Nagios core to make it scale better, allowing more systems to be monitored from a single Nagios server.

PRADO (framework)

PRADO is an open source, object-oriented, event-driven, component-based PHP web framework. PRADO's name is an acronym derived from "PHP Rapid Application Development Object-oriented".

Graphite (software) tool that monitors and graphs the performance of computer systems

Graphite is a free open-source software (FOSS) tool that monitors and graphs numeric time-series data such as the performance of computer systems. Graphite was developed by Orbitz Worldwide, Inc and released as open source software in 2008.

SNAMP

SNAMP is an open-source, cross-platform software platform for telemetry, tracing and elasticity management of distributed applications.

Octopussy (software)

Octopussy, also known as 8Pussy, is a free and open-source computer-software which monitors systems, by constantly analyzing the syslog data they generate and transmit to such a central Octopussy server. Therefore, software like Octopussy plays an important role in maintaining an ISMS within ISO/IEC 27001-compliant environments.

References

  1. GStat 2.0 Overview