IDL (programming language)

Last updated
IDL (Interactive Data Language)
Paradigm vector-oriented programming
Designed by David Stern
Developer David Stern & ITT Visual Information Solutions (ITT VIS)
First appeared1977
Stable release
IDL 9.0 / November 30, 2023
Typing discipline Dynamic
Website www.harrisgeospatial.com/Software-Technology/IDL
Major implementations
IDL, GNU Data Language, Fawlty Language

IDL, short for Interactive Data Language, is a programming language used for data analysis. It is popular in particular areas of science, such as astronomy, atmospheric physics and medical imaging.[ citation needed ] IDL shares a common syntax with PV-Wave and originated from the same codebase, though the languages have subsequently diverged in detail. There are also free or costless implementations, such as GNU Data Language (GDL) and Fawlty Language (FL).

Contents

Overview

IDL is vectorized, numerical, and interactive, and is commonly used for interactive processing of large amounts of data (including image processing). The syntax includes many constructs from Fortran and some from C.

IDL originated from early VMS Fortran, and its syntax still shows its heritage:

x=findgen(100)/10y=sin(x)/x plot,x,y 

The findgen function in the above example returns a one-dimensional array of floating point numbers, with values equal to a series of integers starting at 0.

Note that the operation in the second line applies in a vectorized manner to the whole 100-element array created in the first line, analogous to the way general-purpose array programming languages (such as APL, J or K) would do it. This example contains a division by zero; IDL will report an arithmetic overflow, and store a NaN value in the corresponding element of the y array (the first one), but the other array elements will be finite. The NaN is excluded from the visualization generated by the plot command.

As with most other array programming languages, IDL is very fast at doing vector operations (sometimes as fast as a well-coded custom loop in Fortran or C) but quite slow if elements need processing individually. Hence part of the art of using IDL (or any other array programming language, for that matter) for numerically heavy computations is to make use of the built-in vector operations.

History

The predecessor versions of IDL were developed in the 1970s at the Laboratory for Atmospheric and Space Physics (LASP) at the University of Colorado at Boulder. At LASP, David Stern was involved in efforts to allow scientists to test hypotheses without employing programmers to write or modify individual applications. The first program in the evolutionary chain to IDL that Stern developed was named Rufus; it was a simple vector-oriented calculator that ran on the PDP-12. It accepted two-letter codes that specified an arithmetic operation, the input registers to serve as operands, and the destination register. A version of Rufus developed on the PDP-8 was the Mars Mariner Spectrum Editor (MMED). MMED was used by LASP scientists to interpret data from Mariner 7 and Mariner 9. Later, Stern wrote a program named SOL, which also ran on the PDP-8. Unlike its predecessors, it was a true programming language with a FORTRAN-like syntax. SOL was an array-oriented language with some primitive graphics capabilities. [1]

Stern left LASP to found Research Systems Inc. (RSI) in 1977. The first RSI product was IDL for the PDP-11. [1] In this release, the graphics supported by IDL were primarily Tektronix terminals and raster graphics displays. RSI sold its first IDL licenses to NASA's Goddard Space Flight Center and Ball Aerospace & Technologies Corp. in 1979. Two years later RSI released an initial VAX/VMS version of IDL, which was written in VAX MACRO and FORTRAN. It took advantage of the VAX virtual memory and 32-bit address space. [1] The National Center for Atmospheric Research (NCAR), the University of Michigan, the University of Colorado, and the Naval Research Laboratory started to use IDL with this version.

In 1987 RSI shifted development work of IDL to the Unix environment, which required a complete re-write of the code in C rather than a port of the existing version of VAX IDL. [1] Stern and Ali Bahrami rewrote IDL for Unix on the Sun 3, taking advantage of the re-write to extend and improve the language. Subsequently, IDL was further expanded and ported to several variants of Unix, VMS, Linux, Microsoft Windows (1992), and Mac OS (1994).

Widgets were added to IDL in 1992, providing event-driven programming with graphical user interfaces. In 1997 ION (IDL On the Net), a web server-based system, was commercially released. The first version of ENVI, an application for remote sensing multispectral and hyperspectral image analysis written in IDL, was released in 1994. ENVI was created, developed and owned by Better Solutions Consulting, LLC, until it was purchased from BSC in October 2000 by Eastman Kodak coincident with their purchase of RSI. RSI sold, marketed and supported ENVI under the terms of a license agreement with BSC, LLC from 1994 through October 2000. New object and pointer types, and limited object-oriented programming capabilities, were added to IDL in 1997.

IDL has been applied widely in space science, for example in solar physics. The European Space Agency used IDL to process almost all of the pictures of Halley's Comet taken by the Giotto spacecraft. The team repairing the Hubble Space Telescope used IDL to help them diagnose anomalies in the main mirror. In 1995, astronauts on board a Space Shuttle used IDL loaded on a laptop to study ultraviolet radiation. Currently, amongst other applications, IDL is being used for most of the analysis of the SECCHI part of the STEREO mission at NRL, USA, and at the Rutherford Appleton Laboratory, UK.

RSI became a wholly owned subsidiary of ITT Industries in March 2004. As of 15 May 2006, RSI began doing business as ITT Visual Information Solutions. Effective 31 October 2011, as a result of restructuring, that company became Exelis Visual Information Solutions. In 2015, IDL was purchased by Harris Geospatial Solutions which later became L3Harris Geospatial Solutions. In April 2023, IDL was acquired by NV5 Geospatial.

Features

As a computer language, IDL:

Problems

Some of these features, which make IDL very simple to use interactively, also cause difficulties when building large programs. The single namespace is particularly problematic; for example, language updates that include new built-in functions have on occasion invalidated large scientific libraries. [2]

Arrays are passed by reference, and this mechanism is an advertised feature of the language to pass data back out of a subroutine – in contrast, array slices are copied before being passed, so that data modifications do not flow back into array ranges (after the subroutine exits), violating the principle of least surprise.

Many historical irregularities survive from the early heritage of the language, requiring individual workarounds by the programmer. As an example:

The preceding issue can be alleviated using this compiler option:

COMPILE_OPT STRICTARR

ITT Visual Information Solutions (ITT VIS), the developers of IDL, have taken explicit steps to prevent bytecode compatibility with other environments. Files containing compiled routines use a binary tagged-data-structure format that has not been officially published but has been investigated and documented by users [3] but also contain the following notice as ASCII text embedded within each saved file:

IDL Save/Restore files embody unpublished proprietary information about the IDL program. Reverse engineering of this file is therefore forbidden under the terms of the IDL End User License Agreement (IDL EULA). All IDL users are required to read and agree to the terms of the IDL EULA at the time that they install IDL. Software that reads or writes files in the IDL Save/Restore format must have a license from NV5 Geospatial Solutions, Inc. explicitly granting the right to do so. In this case, the license will be included with the software for your inspection. Please report software that does not have such a license to your account manager or sales representative.

As of August 2023, the statement has not been tested in a court of law. Also, that provision of the IDL EULA has no effect in Australia, as a result of sections 47D and 47H of that country's Copyright Act.

Examples

The following graphics were created with IDL (source code included):

See also

Related Research Articles

<span class="mw-page-title-main">Fortran</span> General-purpose programming language

Fortran is a third generation, compiled, imperative programming language that is especially suited to numeric computation and scientific computing.

In computer science, imperative programming is a programming paradigm of software that uses statements that change a program's state. In much the same way that the imperative mood in natural languages expresses commands, an imperative program consists of commands for the computer to perform. Imperative programming focuses on describing how a program operates step by step, rather than on high-level descriptions of its expected results.

DIBOL or Digital's Business Oriented Language is a general-purpose, procedural, imperative programming language that was designed for use in Management Information Systems (MIS) software development. It was developed from 1970 to 1993.

In computer programming, array slicing is an operation that extracts a subset of elements from an array and packages them as another array, possibly in a different dimension from the original.

Addressing modes are an aspect of the instruction set architecture in most central processing unit (CPU) designs. The various addressing modes that are defined in a given instruction set architecture define how the machine language instructions in that architecture identify the operand(s) of each instruction. An addressing mode specifies how to calculate the effective memory address of an operand by using information held in registers and/or constants contained within a machine instruction or elsewhere.

In computer science, array programming refers to solutions that allow the application of operations to an entire set of values at once. Such solutions are commonly used in scientific and engineering settings.

Coarray Fortran (CAF), formerly known as F--, started as an extension of Fortran 95/2003 for parallel processing created by Robert Numrich and John Reid in the 1990s. The Fortran 2008 standard now includes coarrays, as decided at the May 2005 meeting of the ISO Fortran Committee; the syntax in the Fortran 2008 standard is slightly different from the original CAF proposal.

Perl Data Language is a set of free software array programming extensions to the Perl programming language. PDL extends the data structures built into Perl, to include large multidimensional arrays, and adds functionality to manipulate those arrays as vector objects. It also provides tools for image processing, machine learning, computer modeling of physical systems, and graphical plotting and presentation. Simple operations are automatically vectorized across complete arrays, and higher-dimensional operations are supported.

In computer programming, a return statement causes execution to leave the current subroutine and resume at the point in the code immediately after the instruction which called the subroutine, known as its return address. The return address is saved by the calling routine, today usually on the process's call stack or in a register. Return statements in many programming languages allow a function to specify a return value to be passed back to the code that called the function.

In computer programming, a sigil is a symbol affixed to a variable name, showing the variable's datatype or scope, usually a prefix, as in $foo, where $ is the sigil.

This comparison of programming languages (array) compares the features of array data structures or matrix processing for various computer programming languages.

PV-WAVE is an array oriented fourth-generation programming language used by engineers, scientists, researchers, business analysts and software developers to build and deploy visual data analysis applications. In January 2019, PV-Wave parent Rogue Wave Software was acquired by Minneapolis, Minnesota-based application software developer Perforce.

NetCDF is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. The project homepage is hosted by the Unidata program at the University Corporation for Atmospheric Research (UCAR). They are also the chief source of netCDF software, standards development, updates, etc. The format is an open standard. NetCDF Classic and 64-bit Offset Format are an international standard of the Open Geospatial Consortium.

NV5 Geospatial Solutions develops products for the visualization, analysis, and management of geospatial imagery and scientific data. The company develops products such as IDL, ENVI, Jagwire, and Helios which are used in a variety of industries including defense and intelligence, environmental, engineering, aerospace, medical imaging, federal and civil governments, precision agriculture and academia worldwide.

In computer science, array is a data type that represents a collection of elements, each selected by one or more indices that can be computed at run time during program execution. Such a collection is usually called an array variable or array value. By analogy with the mathematical concepts vector and matrix, array types with one and two indices are often called vector type and matrix type, respectively. More generally, a multidimensional array type can be called a tensor type, by analogy with the physical concept, tensor.

<span class="mw-page-title-main">Speakeasy (computational environment)</span> Computer software environment with own programming language

Speakeasy was a numerical computing interactive environment also featuring an interpreted programming language. It was initially developed for internal use at the Physics Division of Argonne National Laboratory by the theoretical physicist Stanley Cohen. He eventually founded Speakeasy Computing Corporation to make the program available commercially.

Many programming languages and other computer files have a directive, often called include, import, or copy, that causes the contents of the specified file to be inserted into the original file. These included files are called header files or copybooks. They are often used to define the physical layout of program data, pieces of procedural code, and/or forward declarations while promoting encapsulation and the reuse of code or data.

In computer programming, a function is a callable unit of software logic that has a well-defined interface and behavior and can be invoked multiple times.

References

  1. 1 2 3 4 Schienle, Mike (1991-01-19). "IDL FAQ" . Retrieved 8 February 2019.
  2. Fanning, David. "Program Naming Conflicts in IDL 8". Archived from the original on 6 March 2014. Retrieved 30 September 2014.
  3. Markwardt, Craig (2011-12-21). "Unofficial Format Specification of the IDL "SAVE" File" . Retrieved 2013-02-13.