XMDF

Last updated
eXtensible Model Data Format
Initial release2005
Latest release
2.2
January 20, 2017
Type of format scientific data format
Container for HDF5
Website http://www.xmdf.org

XMDF (eXtensible Model Data Format) is a library providing a standard format for the geometric data storage of river cross-sections, 2D/3D structured and unstructured meshes, geometric paths through space, and associated time data. XMDF uses HDF5 for cross-platform data storage and compression. It was initiated in Engineer Research and Development Center (ERDC) and is developed by Aquaveo (formerly the Environmental Modeling Research Laboratory (EMRL) at Brigham Young University). API includes interfaces for C/C++ and Fortran.

An overview of the format was published by the ERDC in 2007. [1] More recent documentation is available online at https://www.xmswiki.com/wiki/XMDF. A binary format was chosen for the data in order to improve performance and data storage as compared with more traditional ASCII grids. The group considered both the NetCDF and HDF5 data formats, and chose HDF5 because it has more flexibility for data storage, compression and data mining. It also supports data folders and data structures, making it more customizable.

The modeling format is used by hydrologic modeling software such as TUFLOW, [2] SMS, GMS, and WMS.

See also

Related Research Articles

In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compression reduces bits by identifying and eliminating statistical redundancy. No information is lost in lossless compression. Lossy compression reduces bits by removing unnecessary or less important information. Typically, a device that performs data compression is referred to as an encoder, and one that performs the reversal of the process (decompression) as a decoder.

<span class="mw-page-title-main">Computer-aided design</span> Constructing a product by means of computer

Computer-aided design (CAD) is the use of computers to aid in the creation, modification, analysis, or optimization of a design. This software is used to increase the productivity of the designer, improve the quality of design, improve communications through documentation, and to create a database for manufacturing. Designs made through CAD software help protect products and inventions when used in patent applications. CAD output is often in the form of electronic files for print, machining, or other manufacturing operations. The terms computer-aided drafting (CAD) and computer-aided design and drafting (CADD) are also used.

A GIS file format is a standard for encoding geographical information into a computer file, as a specialized type of file format for use in geographic information systems (GIS) and other geospatial applications. Since the 1970s, dozens of formats have been created based on various data models for various purposes. They have been created by government mapping agencies, GIS software vendors, standards bodies such as the Open Geospatial Consortium, informal user communities, and even individual developers.

<span class="mw-page-title-main">Hierarchical Data Format</span> Set of file formats

Hierarchical Data Format (HDF) is a set of file formats designed to store and organize large amounts of data. Originally developed at the U.S. National Center for Supercomputing Applications, it is supported by The HDF Group, a non-profit corporation whose mission is to ensure continued development of HDF5 technologies and the continued accessibility of data stored in HDF.

Common Data Format (CDF) is a library and toolkit that was developed by the National Space Science Data Center (NSSDC) at NASA started in 1985. The software is an interface for the storage and manipulation of multi-dimensional data sets.

Mass spectrometry is a scientific technique for measuring the mass-to-charge ratio of ions. It is often coupled to chromatographic techniques such as gas- or liquid chromatography and has found widespread adoption in the fields of analytical chemistry and biochemistry where it can be used to identify and characterize small molecules and proteins (proteomics). The large volume of data produced in a typical mass spectrometry experiment requires that computers be used for data storage and processing. Over the years, different manufacturers of mass spectrometers have developed various proprietary data formats for handling such data which makes it difficult for academic scientists to directly manipulate their data. To address this limitation, several open, XML-based data formats have recently been developed by the Trans-Proteomic Pipeline at the Institute for Systems Biology to facilitate data manipulation and innovation in the public sector. These data formats are described here.

JT is an openly-published ISO-standardized 3D CAD data exchange format used for product visualization, collaboration, digital mockups, and other purposes. It was developed by Siemens.

JTS Topology Suite is an open-source Java software library that provides an object model for Euclidean planar linear geometry together with a set of fundamental geometric functions. JTS is primarily intended to be used as a core component of vector-based geomatics software such as geographical information systems. It can also be used as a general-purpose library providing algorithms in computational geometry.

<span class="mw-page-title-main">Tecplot</span>

Tecplot is the name of a family of visualization & analysis software tools developed by American company Tecplot, Inc., which is headquartered in Bellevue, Washington. The firm was formerly operated as Amtec Engineering. In 2016, the firm was acquired by Vela Software, an operating group of Constellation Software, Inc. (TSX:CSU).

<span class="mw-page-title-main">GMS (software)</span>

GMS is water modeling application for building and simulating groundwater models from Aquaveo. It features 2D and 3D geostatistics, stratigraphic modeling and a unique conceptual model approach. Currently supported models include MODFLOW, MODPATH, MT3DMS, RT3D, FEMWATER, SEEP2D, and UTEXAS.

Silo is a computer data format and library developed at Lawrence Livermore National Laboratory (LLNL) for storing rectilinear, curvilinear, unstructured, or point meshes in 2D and 3D. It supports data upon those meshes, including scalar, vector, and tensor variables; volume fraction-based materials; and mass fraction-based species. It fully supports block structured adaptive mesh refinement (AMR) meshes by way of mesh blocks structured in a hierarchy. Silo sits on top of other low-level storage libraries such as PDB, NetCDF, and HDF5.

SMS is a complete program for building and simulating surface water models from Aquaveo. It features 1D and 2D modeling and a unique conceptual model approach. Currently supported models include ADCIRC, CMS-FLOW2D, FESWMS, TABS, TUFLOW, BOUSS-2D, CGWAVE, STWAVE, CMS-WAVE (WABED), GENESIS, PTM, and WAM.

CGNS stands for CFD General Notation System. It is a general, portable, and extensible standard for the storage and retrieval of CFD analysis data. It consists of a collection of conventions, and free and open software implementing those conventions. It is self-descriptive, cross-platform also termed platform or machine independent, documented, and administered by an international steering committee. It is also an American Institute of Aeronautics and Astronautics (AIAA) recommended practice. The CGNS project originated in 1994 as a joint effort between Boeing and NASA, and has since grown to include many other contributing organizations worldwide. In 1999, control of CGNS was completely transferred to a public forum known as the CGNS Steering CommitteeArchived 2007-06-24 at the Wayback Machine. This Committee is made up of international representatives from government and private industry.

XDMF provides a standard way to access data produced by HPC codes. Data format refers to the raw data to be manipulated, the description of the data is separate from the values themselves. It distinguishes the metadata and the values themselves. Light data is stored using XML, Heavy data is stored using HDF5, so some information is stored redundantly in both XML and HDF5. APIs to read and write XDMF exists for multiple programing languages.

<span class="mw-page-title-main">Ocarina Networks</span>

Ocarina Networks was a technology company selling a hardware/software solution designed to reduce data footprints with file-aware storage optimization. A subsidiary of Dell, their flagship product, the Ocarina Appliance/Reader, released in April 2008, uses patented data compression techniques incorporating such methods as record linkage and context-based lossless data compression. The product includes the hardware-appliance-based compressor, the Ocarina Optimizer and a real-time decompressor, the software-based Ocarina Reader.

A geographic data model, geospatial data model, or simply data model in the context of geographic information systems, is a mathematical and digital structure for representing phenomena over the Earth. Generally, such data models represent various aspects of these phenomena by means of geographic data, including spatial locations, attributes, change over time, and identity. For example, the vector data model represents geography as collections of points, lines, and polygons, and the raster data model represent geography as cell matrices that store numeric values. Data models are implemented throughout the GIS ecosystem, including the software tools for data management and spatial analysis, data stored in a variety of GIS file formats, specifications and standards, and specific designs for GIS installations.

NetCDF is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. The project homepage is hosted by the Unidata program at the University Corporation for Atmospheric Research (UCAR). They are also the chief source of netCDF software, standards development, updates, etc. The format is an open standard. NetCDF Classic and 64-bit Offset Format are an international standard of the Open Geospatial Consortium.

<span class="mw-page-title-main">Apple Disk Image</span> File format developed by Apple and used by macOS

AppleDisk Image is a disk image format commonly used by the macOS operating system. When opened, an Apple Disk Image is mounted as a volume within the Finder.

Apache Parquet is a free and open-source column-oriented data storage format in the Apache Hadoop ecosystem. It is similar to RCFile and ORC, the other columnar-storage file formats in Hadoop, and is compatible with most of the data processing frameworks around Hadoop. It provides efficient data compression and encoding schemes with enhanced performance to handle complex data in bulk.

References

  1. Butler, Cary D., et al. eXtensible Model Data Format (XMDF). System-Wide water Resources Program. US Army Corps of Engineers. ERDC SR-07-1. January 2007. PDF
  2. "TUFLOW – Flood and Coastal Simulation Software". BMT Limited.