LONI Pipeline

Last updated
LONI Pipeline
LONI Pipeline Logo faucet.png
Pipeline workflow.png
Pipeline Environment
Developer(s) Samuel Hobel
Stable release
7.0.3 / March 3, 2020 (2020-03-03)
Written in Java
Operating system Linux, Mac OS X, Microsoft Windows
Type Scientific workflow system, Workflow processing environments
License LONI License
Website pipeline.loni.usc.edu

The LONI Pipeline is a free distributed system for designing, executing, monitoring and sharing scientific workflows [1] [2] on grid computing architectures. Pipeline allows users to connect and run any number of different software tools, and conveniently visualize and download the results.

Contents

Unlike other workflow processing environments, Pipeline does not require new tools and services to include or be built against the core Pipeline libraries. The Pipeline environment references all data, services and tools as external objects. This allows the Pipeline to run as a light-weight middleware, but at the same time, restrict the scope of its applications. For example, the Pipeline does not provide a set of internal core libraries, filters, and processes for rudimentary image processing (e.g., image addition). All tools necessary to complete an analysis protocol must first be built as external stand-alone applications or services, whose interface methods are then described in the Pipeline XML language. Users can connect to the LONI Cranium server to gain quick access to a wide array of pre-built software applications, such as FSL, AFNI, and FreeSurfer already described in XML as modules and workflows. Pipeline allows users to create new workflow descriptions, edit existing ones, and share their work with others.

Typical pipeline server installations include a suite of core resources that are available to all users with access to the specific server, however, different servers will have different suites of default module and module-group (pipeline) definitions. The previous release (version 5) of the LONI Pipeline [3] provided a mechanism for integrating heterogeneous and incongruous data including images, clinical charts and demographic meta-data.

The LONI Pipeline has hundreds of users in a variety of fields (e.g., genomics, [4] neuro-imaging, [5] and Biomedical Informatics [6] ) from academic institutions around the world.

Features

Pipeline has cross platform compatibility, and the ability to connect from your local client to a remote server for executing processing and analysis on other operating systems.

Pipeline grants developers the opportunity to create their own plugins to communicate with various grid managers. The default Pipeline package includes the JGDIPlugin and the DRMAAPlugin plugins created for Sun Grid Engine but they may work with Oracle Grid Engine, Univa Grid Engine or Son of Grid Engine. Both plugins are housed under the gridplugins directory which is parented under the dist directory in the installed package of Pipeline. All additional plugins you wish to employ can be downloaded separately.

The Pipeline Library grants users access to hundreds of predefined neuroimaging solutions, including data, modules and workflows that are regularly updated.

Other integral features of the LONI Pipeline are:

Developers

Present:

Past:

See also

Related Research Articles

A web content management system is a software content management system (CMS) specifically for web content. It provides website authoring, collaboration, and administration tools that help users with little knowledge of web programming languages or markup languages create and manage website content. A WCMS provides the foundation for collaboration, providing users the ability to manage documents and output for multiple author editing and participation. Most systems use a content repository or a database to store page content, metadata, and other information assets the system needs.

Statistics Online Computational Resource

The Statistics Online Computational Resource (SOCR) is an online multi-institutional research and education organization. SOCR designs, validates and broadly shares a suite of online tools for statistical computing, and interactive materials for hands-on learning and teaching concepts in data science, statistical analysis and probability theory. The SOCR resources are platform agnostic based on HTML, XML and Java, and all materials, tools and services are freely available over the Internet.

FreeSurfer Brain imaging software package

FreeSurfer is a brain imaging software package originally developed by Bruce Fischl, Anders Dale, Martin Sereno, and Doug Greve. Development and maintenance of FreeSurfer is now the primary responsibility of the Laboratory for Computational Neuroimaging at the Athinoula A. Martinos Center for Biomedical Imaging. FreeSurfer contains a set of programs with a common focus of analyzing magnetic resonance imaging (MRI) scans of brain tissue. It is an important tool in functional brain mapping and contains tools to conduct both volume based and surface based analysis. FreeSurfer includes tools for the reconstruction of topologically correct and geometrically accurate models of both the gray/white and pial surfaces, for measuring cortical thickness, surface area and folding, and for computing inter-subject registration based on the pattern of cortical folds.

GenePattern is a freely available computational biology open-source software package originally created and developed at the Broad Institute for the analysis of genomic data. Designed to enable researchers to develop, capture, and reproduce genomic analysis methodologies, GenePattern was first released in 2004. GenePattern is currently developed at the University of California, San Diego.

Apache Taverna

Apache Taverna is an open source software tool for designing and executing workflows, initially created by the myGrid project under the name Taverna Workbench, now a project under the Apache incubator. Taverna allows users to integrate many different software components, including WSDL SOAP or REST Web services, such as those provided by the National Center for Biotechnology Information, the European Bioinformatics Institute, the DNA Databank of Japan (DDBJ), SoapLab, BioMOBY and EMBOSS. The set of available services is not finite and users can import new service descriptions into the Taverna Workbench.

VisIt

VisIt is an open-source interactive parallel visualization and graphical analysis tool for viewing scientific data. It can be used to visualize scalar and vector fields defined on 2D and 3D structured and unstructured meshes. VisIt was designed to handle very large data set sizes in the terascale range and yet can also handle small data sets in the kilobyte range.

Kepler is a free software system for designing, executing, reusing, evolving, archiving, and sharing scientific workflows. Kepler's facilities provide process and data monitoring, provenance information, and high-speed data movement. Workflows in general, and scientific workflows in particular, are directed graphs where the nodes represent discrete computational components, and the edges represent paths along which data and results can flow between components. In Kepler, the nodes are called 'Actors' and the edges are called 'channels'. Kepler includes a graphical user interface for composing workflows in a desktop environment, a runtime engine for executing workflows within the GUI and independently from a command-line, and a distributed computing option that allows workflow tasks to be distributed among compute nodes in a computer cluster or computing grid. The Kepler system principally targets the use of a workflow metaphor for organizing computational tasks that are directed towards particular scientific analysis and modeling goals. Thus, Kepler scientific workflows generally model the flow of data from one step to another in a series of computations that achieve some scientific goal.

VisTrails

VisTrails is a scientific workflow management system developed at the Scientific Computing and Imaging Institute at the University of Utah that provides support for data exploration and visualization. It is written in Python and employs Qt via PyQt bindings. The system is open source, released under the GPL v2 license. The pre-compiled versions for Windows, Mac OS X, and Linux come with an installer and several packages, including VTK, matplotlib, and ImageMagick. VisTrails also supports user-defined packages.

iTools Resourceome

iTools is a distributed infrastructure for managing, discovery, comparison and integration of computational biology resources. iTools employs Biositemap technology to retrieve and service meta-data about diverse bioinformatics data services, tools, and web-services. iTools is developed by the National Centers for Biomedical Computing as part of the NIH Road Map Initiative.

KNIME, the Konstanz Information Miner, is a free and open-source data analytics, reporting and integration platform. KNIME integrates various components for machine learning and data mining through its modular data pipelining "Lego of Analytics" concept. A graphical user interface and use of JDBC allows assembly of nodes blending different data sources, including preprocessing, for modeling, data analysis and visualization without, or with only minimal, programming.

Discovery Net is one of the earliest examples of a scientific workflow system allowing users to coordinate the execution of remote services based on Web service and Grid Services standards. The system was designed and implemented at Imperial College London as part of the Discovery Net pilot project funded by the UK e-Science Programme. Many of the concepts pioneered by Discovery Net have been later incorporated into a variety of other scientific workflow systems.

A scientific workflow system is a specialized form of a workflow management system designed specifically to compose and execute a series of computational or data manipulation steps, or workflow, in a scientific application.

LabKey Server is a software suite available for scientists to integrate, analyze, and share biomedical research data. The platform provides a secure data repository that allows web-based querying, reporting, and collaborating across a range of data sources. Specific scientific applications and workflows can be added on top of the basic platform and leverage a data processing pipeline.

TACTIC (web framework)

TACTIC is a web-based, open source workflow platform and digital asset management system supported by Southpaw Technology in Toronto, ON. Designed to optimize busy production environments with high volumes of content traffic, TACTIC applies business or workflow logic to combined database and file system management. Using elements of digital asset management, production asset management and workflow management, TACTIC tracks the creation and development of digital assets through production pipelines. TACTIC is available under both commercial and open-source licenses, and also as a hosted cloud service through Amazon Web Services Marketplace.

Neuroimaging Informatics Tools and Resources Clearinghouse

The Neuroimaging Tools and Resources Collaboratory is a neuroimaging informatics knowledge environment for MR, PET/SPECT, CT, EEG/MEG, optical imaging, clinical neuroinformatics, imaging genomics, and computational neuroscience tools and resources.

CONN (functional connectivity toolbox)

CONN is a Matlab-based cross-platform imaging software for the computation, display, and analysis of functional connectivity in fMRI in the resting state and during task.

References

  1. Rex, D. E., Ma, J.Q., and Toga, A.W. (2003). "The LONI Pipeline Processing Environment." Neuroimage, 19(3), 1033-48.
  2. Rex, D. E., Shattuck, D. W., Woods, R. P., Narr, K. L., Luders, E., Rehm, K., Stolzner, S. E., Rottenberg, D. E., and Toga, A. W. (2004). "A meta-algorithm for brain extraction in MRI." NeuroImage, 23(2), 625–637
  3. Dinov ID, Lozev K, Petrosyan P, Liu Z, Eggert P, Pierce, J, Zamanyan, A, Chakrapani, S, Van Horn, JD, Parker, DS, Magsipoc, R, Leung, K, Gutman, B, Woods, RP, Toga, AW. (2010). "Neuroimaging Study Designs, Computational Analyses and Data Provenance Using the LONI Pipeline." PLoS ONE 5(9): e13070. doi : 10.1371/journal.pone.0013070.
  4. Torri, F., Dinov, ID, Zamanyan, A, Hobel, S, Genco, A, Petrosyan, P, Clark, AP, Liu, Z, Eggert, P, Pierce, J, Knowles, JA, Ames, J, Kesselman, C, Toga, AW, Potkin, SG, Vawter, MP, Macciardi, F. (2012) Next Generation Sequence Analysis and Computational Genomics Using Graphical Pipeline Workflows, Genes, 3(3):545-575; doi:10.3390/genes3030545.
  5. Woo MS, Dinov, ID, Hobel, S, Zamanyan, A, Choi, YC, Thompson, PM, Toga, AW and Alzheimer’s Disease Neuroimaging Initiative (ADNI) (2015) Structural Brain Changes in Early-Onset Alzheimer’s Disease Subjects Using the LONI Pipeline Environment. Journal of Neuroimaging., in press. DOI: 10.1111/jon.12252
  6. Toga, WA, Dinov, ID. (2015) Sharing big biomedical data. Journal of Big Data., 2(7):1-12. DOI: 10.1186/s40537-015-0016-1.