Code examples
Hello World
publicclassHelloWorld{publicstaticvoidmain(String[]args){[||System.out.println("Hello");||System.out.println("World");]}}Each || symbol introduces a parallel branch. Running this program will print either
Hello World
or
World Hello
depending on how the parallel branches happen to be scheduled.
Data parallelism
[||(inti:array.length)array[i]++;]
The quantification (int i : N) creates one parallel branch for each value of i. The effect of this code is to increment all elements of array in parallel. This code is equivalent to
[||array[0]++;||array[1]++;...||array[array.length-1]++;]
More complex quantifications are possible. The following example quantifies over the upper left triangle of a square matrix:
[||(inti:N,intj:N,ifi+j<N)matrix[i][j]++;]
Code that performs a similar and typically small operation on a large collection of elements is called data parallel, and appears often in high-performance scientific applications. A typical representative of data-parallel languages for the C/C++ or Fortran ecosystems is OpenMP.
Data parallelism features can also be implemented by libraries using dedicated data structures, such as parallel arrays.
Task parallelism
The term task parallelism is used when work can conceptually be decomposed into a number of logical tasks. In this example, tasks are created recursively:
intfib(intn){if(n<=1)return1;intfib1,fib2;// recursively create parallel branches[||fib1=fib(n-1);||fib2=fib(n-2);]returnfib1+fib2;}Task parallelism is implemented in languages such as Cilk, and in libraries similar to the fork/join pair of Unix system calls.
Message-passing
Parallel branches have two ways of communicating; either by concurrently reading and writing shared variables, or by sending explicit messages. The operators ! and ? respectively send and receive a message on a channel.
In this example, two parallel branches communicate via explicit message passing:
Chan<String>chan=newChan<String>();[// branch 1 sends a value over the channel||chan!"Hello";// branch 2 receives a value from the channel and prints it||chan?s;System.out.println(s);]
Data-flow
A program is said to be data-flow when computation is initiated and synchronized by the availability of data in a flow. A typical example is an adder: it has two inputs, one output, and whenever the two inputs are ready, it sends their sum on the output.
voidadder(Chan<Integer>in1,Chan<Integer>in2,Chan<Integer>out){for(;;){intvalue1,value2;[in1?value1;||in2?value2;];out!value1+value2;}}Note the parallel read [ in1 ? value1; || in2 ? value2; ]. It means that the two input values can come in any order. Without it, the code may deadlock if values were coming in the wrong order. This shows that parallel primitives in a programming language are not only about performance, but also about the behavior of programs.
The adder by itself doesn't do anything, since it reacts on input data. It needs to be put in a context where other parts feed input values and read output values. The way to express this is to compose all pieces in a large parallel block:
[||source(c1);// generates values on c1||source(c2);// generates values on c2||adder(c1,c2,c3);||sink(c3);// read values from c3]
Anything that can be thought of as a combination of logical gates or electrical circuits can readily be expressed in this way as a data-flow program.
In programming language theory, lazy evaluation, or call-by-need, is an evaluation strategy which delays the evaluation of an expression until its value is needed and which also avoids repeated evaluations (sharing).
OCaml is a general-purpose, high-level multi-paradigm programming language which extends the Caml dialect of ML with object-oriented features. OCaml was created in 1996 by Xavier Leroy, Jérôme Vouillon, Damien Doligez, Didier Rémy, Ascánder Suárez, and others.
OpenMP is an application programming interface (API) that supports multi-platform shared-memory multiprocessing programming in C, C++, and Fortran, on many platforms, instruction-set architectures and operating systems, including Solaris, AIX, FreeBSD, HP-UX, Linux, macOS, and Windows. It consists of a set of compiler directives, library routines, and environment variables that influence run-time behavior.
In computer science, a generator is a routine that can be used to control the iteration behaviour of a loop. All generators are also iterators. A generator is very similar to a function that returns an array, in that a generator has parameters, can be called, and generates a sequence of values. However, instead of building an array containing all the values and returning them all at once, a generator yields the values one at a time, which requires less memory and allows the caller to get started processing the first few values immediately. In short, a generator looks like a function but behaves like an iterator.
In computer programming, foreach loop is a control flow statement for traversing items in a collection. foreach is usually used in place of a standard for loop statement. Unlike other for loop constructs, however, foreach loops usually maintain no explicit counter: they essentially say "do this to everything in this set", rather than "do this x times". This avoids potential off-by-one errors and makes code simpler to read. In object-oriented languages, an iterator, even if implicit, is often used as the means of traversal.
Cilk, Cilk++, Cilk Plus and OpenCilk are general-purpose programming languages designed for multithreaded parallel computing. They are based on the C and C++ programming languages, which they extend with constructs to express parallel loops and the fork–join idiom.
The syntax of Java refers to the set of rules defining how a Java program is written and interpreted.
The TPK algorithm is a simple program introduced by Donald Knuth and Luis Trabb Pardo to illustrate the evolution of computer programming languages. In their 1977 work "The Early Development of Programming Languages", Trabb Pardo and Knuth introduced a small program that involved arrays, indexing, mathematical functions, subroutines, I/O, conditionals and iteration. They then wrote implementations of the algorithm in several early programming languages to show how such concepts were expressed.
In computer programming, an entry point is the place in a program where the execution of a program begins, and where the program has access to command line arguments.
In mathematics and in computer programming, a variadic function is a function of indefinite arity, i.e., one which accepts a variable number of arguments. Support for variadic functions differs widely among programming languages.
In computer science, stream processing is a programming paradigm which views data streams, or sequences of events in time, as the central input and output objects of computation. Stream processing encompasses dataflow programming, reactive programming, and distributed data processing. Stream processing systems aim to expose parallel processing for data streams and rely on streaming algorithms for efficient implementation. The software stack for these systems includes components such as programming models and query languages, for expressing computation; stream management systems, for distribution and scheduling; and hardware components for acceleration including floating-point units, graphics processing units, and field-programmable gate arrays.
In computer programming, an anonymous function is a function definition that is not bound to an identifier. Anonymous functions are often arguments being passed to higher-order functions or used for constructing the result of a higher-order function that needs to return a function. If the function is only used once, or a limited number of times, an anonymous function may be syntactically lighter than using a named function. Anonymous functions are ubiquitous in functional programming languages and other languages with first-class functions, where they fulfil the same role for the function type as literals do for other data types.
Generics are a facility of generic programming that were added to the Java programming language in 2004 within version J2SE 5.0. They were designed to extend Java's type system to allow "a type or method to operate on objects of various types while providing compile-time type safety". The aspect compile-time type safety was not fully achieved, since it was shown in 2016 that it is not guaranteed in all cases.
Loop-level parallelism is a form of parallelism in software programming that is concerned with extracting parallel tasks from loops. The opportunity for loop-level parallelism often arises in computing programs where data is stored in random access data structures. Where a sequential program will iterate over the data structure and operate on indices one at a time, a program exploiting loop-level parallelism will use multiple threads or processes which operate on some or all of the indices at the same time. Such parallelism provides a speedup to overall execution time of the program, typically in line with Amdahl's law.
Software is said to exhibit scalable parallelism if it can make use of additional processors to solve larger problems, i.e. this term refers to software for which Gustafson's law holds. Consider a program whose execution time is dominated by one or more loops, each of that updates every element of an array --- for example, the following finite difference heat equation stencil calculation:
for t := 0 to T dofor i := 1 to N-1 do new(i) := * .25 // explicit forward-difference with R = 0.25 endfor i := 1 to N-1 do A(i) := new(i) endend
Go is a statically typed, compiled high-level programming language designed at Google by Robert Griesemer, Rob Pike, and Ken Thompson. It is syntactically similar to C, but with memory safety, garbage collection, structural typing, and CSP-style concurrency. It is often referred to as Golang because of its former domain name, golang.org, but its proper name is Go.
This article describes the features in Haskell.
OptimJ is an extension for Java with language support for writing optimization models and abstractions for bulk data processing. The extensions and the proprietary product implementing the extensions were developed by Ateji which went out of business in September 2011. OptimJ aims at providing a clear and concise algebraic notation for optimization modeling, removing compatibility barriers between optimization modeling and application programming tools, and bringing software engineering techniques such as object-orientation and modern IDE support to optimization experts.
In computer programming, a function or subroutine is a sequence of program instructions that performs a specific task, packaged as a unit. This unit can then be used in programs wherever that particular task should be performed.
In software engineering, the module pattern is a design pattern used to implement the concept of software modules, defined by modular programming, in a programming language with incomplete direct support for the concept.
