Code examples
Hello World
publicclassHelloWorld{publicstaticvoidmain(String[]args){[||System.out.println("Hello");||System.out.println("World");]}}Each || symbol introduces a parallel branch. Running this program will print either
Hello World
or
World Hello
depending on how the parallel branches happen to be scheduled.
Data parallelism
[||(inti:array.length)array[i]++;]
The quantification (int i : N) creates one parallel branch for each value of i. The effect of this code is to increment all elements of array in parallel. This code is equivalent to
[||array[0]++;||array[1]++;...||array[array.length-1]++;]
More complex quantifications are possible. The following example quantifies over the upper left triangle of a square matrix:
[||(inti:N,intj:N,ifi+j<N)matrix[i][j]++;]
Code that performs a similar and typically small operation on a large collection of elements is called data parallel, and appears often in high-performance scientific applications. A typical representative of data-parallel languages for the C/C++ or Fortran ecosystems is OpenMP.
Data parallelism features can also be implemented by libraries using dedicated data structures, such as parallel arrays.
Task parallelism
The term task parallelism is used when work can conceptually be decomposed into a number of logical tasks. In this example, tasks are created recursively:
intfib(intn){if(n<=1)return1;intfib1,fib2;// recursively create parallel branches[||fib1=fib(n-1);||fib2=fib(n-2);]returnfib1+fib2;}Task parallelism is implemented in languages such as Cilk, and in libraries similar to the fork/join pair of Unix system calls.
Message-passing
Parallel branches have two ways of communicating; either by concurrently reading and writing shared variables, or by sending explicit messages. The operators ! and ? respectively send and receive a message on a channel.
In this example, two parallel branches communicate via explicit message passing:
Chan<String>chan=newChan<String>();[// branch 1 sends a value over the channel||chan!"Hello";// branch 2 receives a value from the channel and prints it||chan?s;System.out.println(s);]
Data-flow
A program is said to be data-flow when computation is initiated and synchronized by the availability of data in a flow. A typical example is an adder: it has two inputs, one output, and whenever the two inputs are ready, it sends their sum on the output.
voidadder(Chan<Integer>in1,Chan<Integer>in2,Chan<Integer>out){for(;;){intvalue1,value2;[in1?value1;||in2?value2;];out!value1+value2;}}Note the parallel read [ in1 ? value1; || in2 ? value2; ]. It means that the two input values can come in any order. Without it, the code may deadlock if values were coming in the wrong order. This shows that parallel primitives in a programming language are not only about performance, but also about the behavior of programs.
The adder by itself doesn't do anything, since it reacts on input data. It needs to be put in a context where other parts feed input values and read output values. The way to express this is to compose all pieces in a large parallel block:
[||source(c1);// generates values on c1||source(c2);// generates values on c2||adder(c1,c2,c3);||sink(c3);// read values from c3]
Anything that can be thought of as a combination of logical gates or electrical circuits can readily be expressed in this way as a data-flow program.
In computer science, an array is a data structure consisting of a collection of elements, of same memory size, each identified by at least one array index or key. An array is stored such that the position of each element can be computed from its index tuple by a mathematical formula. The simplest type of data structure is a linear array, also called a one-dimensional array.
In programming language theory, lazy evaluation, or call-by-need, is an evaluation strategy which delays the evaluation of an expression until its value is needed and which avoids repeated evaluations.
Templates are a feature of the C++ programming language that allows functions and classes to operate with generic types. This allows a function or class declaration to reference via a generic variable another different class without creating full declaration for each of these different classes.
OCaml is a general-purpose, high-level, multi-paradigm programming language which extends the Caml dialect of ML with object-oriented features. OCaml was created in 1996 by Xavier Leroy, Jérôme Vouillon, Damien Doligez, Didier Rémy, Ascánder Suárez, and others.
In computer programming, a parameter or a formal argument is a special kind of variable used in a subroutine to refer to one of the pieces of data provided as input to the subroutine. These pieces of data are the values of the arguments with which the subroutine is going to be called/invoked. An ordered list of parameters is usually included in the definition of a subroutine, so that, each time the subroutine is called, its arguments for that call are evaluated, and the resulting values can be assigned to the corresponding parameters.
In computer science, a generator is a routine that can be used to control the iteration behaviour of a loop. All generators are also iterators. A generator is very similar to a function that returns an array, in that a generator has parameters, can be called, and generates a sequence of values. However, instead of building an array containing all the values and returning them all at once, a generator yields the values one at a time, which requires less memory and allows the caller to get started processing the first few values immediately. In short, a generator looks like a function but behaves like an iterator.
In computer programming, foreach loop is a control flow statement for traversing items in a collection. foreach is usually used in place of a standard for loop statement. Unlike other for loop constructs, however, foreach loops usually maintain no explicit counter: they essentially say "do this to everything in this set", rather than "do this x times". This avoids potential off-by-one errors and makes code simpler to read. In object-oriented languages, an iterator, even if implicit, is often used as the means of traversal.
Cilk, Cilk++, Cilk Plus and OpenCilk are general-purpose programming languages designed for multithreaded parallel computing. They are based on the C and C++ programming languages, which they extend with constructs to express parallel loops and the fork–join idiom.
The syntax of Java is the set of rules defining how a Java program is written and interpreted.
In computer programming, an entry point is the place in a program where the execution of a program begins, and where the program has access to command line arguments.
In mathematics and in computer programming, a variadic function is a function of indefinite arity, i.e., one which accepts a variable number of arguments. Support for variadic functions differs widely among programming languages.
Boundary-value analysis is a software testing technique in which tests are designed to include representatives of boundary values in a range. The idea comes from the boundary. Given that there is a set of test vectors to test the system, a topology can be defined on that set. Those inputs which belong to the same equivalence class as defined by the equivalence partitioning theory would constitute the basis. Given that the basis sets are neighbors, there would exist a boundary between them. The test vectors on either side of the boundary are called boundary values. In practice, this would require that the test vectors can be ordered, and that the individual parameters follows some kind of order.
In computer science, stream processing is a programming paradigm which views streams, or sequences of events in time, as the central input and output objects of computation. Stream processing encompasses dataflow programming, reactive programming, and distributed data processing. Stream processing systems aim to expose parallel processing for data streams and rely on streaming algorithms for efficient implementation. The software stack for these systems includes components such as programming models and query languages, for expressing computation; stream management systems, for distribution and scheduling; and hardware components for acceleration including floating-point units, graphics processing units, and field-programmable gate arrays.
Automatic vectorization, in parallel computing, is a special case of automatic parallelization, where a computer program is converted from a scalar implementation, which processes a single pair of operands at a time, to a vector implementation, which processes one operation on multiple pairs of operands at once. For example, modern conventional computers, including specialized supercomputers, typically have vector operations that simultaneously perform operations such as the following four additions :
The conditional operator is supported in many programming languages. This term usually refers to ?: as in C, C++, C#, and JavaScript. However, in Java, this term can also refer to && and ||.
Go is a statically typed, compiled high-level general purpose programming language. It is known for the simplicity of its syntax and the efficiency of development that it enables by the inclusion of a large standard library supplying many needs for common projects. It was designed at Google in 2009 by Robert Griesemer, Rob Pike, and Ken Thompson. It is syntactically similar to C, but also has memory safety, garbage collection, structural typing, and CSP-style concurrency. It is often referred to as Golang to avoid ambiguity and because of its former domain name, golang.org, but its proper name is Go.
This article describes the features in the programming language Haskell.
OptimJ is an extension for Java with language support for writing optimization models and abstractions for bulk data processing. The extensions and the proprietary product implementing the extensions were developed by Ateji which went out of business in September 2011. OptimJ aims at providing a clear and concise algebraic notation for optimization modeling, removing compatibility barriers between optimization modeling and application programming tools, and bringing software engineering techniques such as object-orientation and modern IDE support to optimization experts.
In software engineering, the module pattern is a design pattern used to implement the concept of software modules, defined by modular programming, in a programming language with incomplete direct support for the concept.
X macros are an idiomatic usage of programming language macros for generating list-like structures of data or code. They are most useful when at least some of the lists cannot be composed by indexing, such as compile time. They provide reliable maintenance of parallel lists whose corresponding items must be declared or executed in the same order.
