Unums (universal numbers [1] ) are a family of number formats and arithmetic for implementing real numbers on a computer, proposed by John L. Gustafson in 2015. [2] They are designed as an alternative to the ubiquitous IEEE 754 floating-point standard. The latest version is known as posits. [3]
The first version of unums, formally known as Type I unum, was introduced in Gustafson's book The End of Error as a superset of the IEEE-754 floating-point format. [2] The defining features of the Type I unum format are:
For computation with the format, Gustafson proposed using interval arithmetic with a pair of unums, what he called a ubound, providing the guarantee that the resulting interval contains the exact solution.
William M. Kahan and Gustafson debated unums at the Arith23 conference. [4] [5] [6] [7]
Type II Unums were introduced in 2016 [8] as a redesign of Unums that broke IEEE-754 compatibility.
This section needs expansion. You can help by adding to it. (June 2021) |
In February 2017, Gustafson officially introduced Type III unums (posits), for fixed floating-point-like values and valids for interval arithmetic. [3] In March 2022, a standard was ratified and published by the Posit Working Group. [9]
Posits [3] [10] [11] are a hardware-friendly version of unum where difficulties faced in the original type I unum due to its variable size are resolved. Compared to IEEE 754 floats of similar size, posits offer a bigger dynamic range and more fraction bits for values with magnitude near 1 (but fewer fraction bits for very large or very small values), and Gustafson claims that they offer better accuracy. [12] [13] Studies [14] [15] confirm that for some applications, posits with quire out-perform floats in accuracy. Posits have superior accuracy in the range near one, where most computations occur. This makes it very attractive to the current trend in deep learning to minimize the number of bits used. It potentially helps any application to accelerate by enabling the use of fewer bits (since it has more fraction bits for accuracy) reducing network and memory bandwidth and power requirements.
The format of an n-bit posit is given a label of "posit" followed by the decimal digits of n (e.g., the 16-bit posit format is "posit16") and consists of four sequential fields:
The regime field uses unary coding of k identical bits, followed by a bit of opposite value if any remaining bits are available, to represent an unsigned integer r that is −k if the first bit is 0 or k − 1 if the first bit is 1. The sign, exponent, and fraction fields are analogous to IEEE 754 sign, exponent, and significand fields (respectively), except that the posit exponent and fraction fields may be absent or truncated and implicitly extended with zeroes—an absent exponent is treated as 00
2 (representing 0), a one-bit exponent E1 is treated as E10
2 (representing the integer 0 if E1 is 0 or 2 if E1 is 1), and an absent fraction is treated as 0.
The two encodings in which all non-sign bits are 0 have special interpretations:
NaR
("not a real")sign
function returns 0)Otherwise, the posit value is equal to , in which r scales by powers of 16, e scales by powers of 2, f distributes values uniformly between adjacent combinations of (r, e), and s adjusts the sign symmetrically about 0.
type (positn) | Binary | Value | Notes |
---|---|---|---|
Any | 10… | NaR | anything not mathematically definable as a unique real number [9] |
Any | 00… | 0 | |
Any | 010… | 1 | |
Any | 110… | −1 | |
Any | 00111 0… | 0.5 | |
Any | 00…1 | smallest positive value | |
Any | 01… | largest positive value | |
posit8 | 00000001 | smallest positive value | |
posit8 | 01111111 | largest positive value | |
posit16 | 0000000000000001 | smallest positive value | |
posit16 | 0111111111111111 | largest positive value | |
posit32 | 00000000000000000000000000000001 | smallest positive value | |
posit32 | 01111111111111111111111111111111 | largest positive value |
Note: 32-bit posit is expected to be sufficient to solve almost all classes of applications[ citation needed ].
For each positn type of precision , the standard defines a corresponding "quire" type quiren of precision , used to accumulate exact sums of products of those posits without rounding or overflow in dot products for vectors of up to 231 or more elements (the exact limit is ). The quire format is a two's complement signed integer, interpreted as a multiple of units of magnitude except for the special value with a leading sign bit of 1 and all other bits equal to 0 (which represents NaR
). Quires are based on the work of Ulrich W. Kulisch and Willard L. Miranker. [16]
Valids are described as a Type III Unum mode that bounds results in a given range. [3]
Several software and hardware solutions implement posits. [14] [17] [18] [19] [20] The first complete parameterized posit arithmetic hardware generator was proposed in 2018. [21]
Unum implementations have been explored in Julia [22] [23] [24] [25] [26] [27] and MATLAB. [28] [29] A C++ version [30] with support for any posit sizes combined with any number of exponent bits is available. A fast implementation in C, SoftPosit, [31] provided by the NGA research team based on Berkeley SoftFloat adds to the available software implementations.
Project author | Type | Precisions | Quire Support? | Speed | Testing | Notes |
---|---|---|---|---|---|---|
GP-GPU VividSparks | World's first FPGA GPGPU | 32 | Yes | ~3.2 TPOPS | Exhaustive. No known bugs. | RacEr GP-GPU has 512 cores |
SoftPosit A*STAR | C library based on Berkeley SoftFloat C++ wrapper to override operators Python wrapper using SWIG of SoftPosit | 8, 16, 32 published and complete; | Yes | ~60 to 110 MPOPS on x86 core (Broadwell) | 8: Exhaustive; 16: Exhaustive except FMA, quire 32: Exhaustive test is still in progress. No known bugs. | Open source license. Fastest and most comprehensive C library for posits presently. Designed for plug-in comparison of IEEE floats and posits. |
posit4.nb A*STAR | Mathematica notebook | All | Yes | < 80 KPOPS | Exhaustive for low precisions. No known bugs. | Open source (MIT license). Original definition and prototype. Most complete environment for comparing IEEE floats and posits. Many examples of use, including linear solvers |
posit-javascript A*STAR | JavaScript widget | Convert decimal to posit 6, 8, 16, 32; generate tables 2–17 with es 1–4. | N/A | N/A; interactive widget | Fully tested | Table generator and conversion |
Universal Stillwater Supercomputing, Inc | C++ template library C library Python wrapper Golang library | Arbitrary precision posit float valid (p) Unum type 1 (p) Unum type 2 (p) | Arbitrary quire configurations with programmable capacity | posit<4,0> 1 GPOPS posit<8,0> 130 MPOPS posit<16,1> 115 MPOPS posit<32,2> 105 MPOPS posit<64,3> 50 MPOPS posit<128,4> 1 MPOPS posit<256,5> 800 KPOPS | Complete validation suite for arbitrary posits Randoms for large posit configs. Uses induction to prove nbits+1 is correct no known bugs | Open source. MIT license. Fully integrated with C/C++ types and automatic conversions. Supports full C++ math library (native and conversion to/from IEEE). Runtime integrations: MTL4/MTL5, Eigen, Trilinos, HPR-BLAS. Application integrations: G+SMO, FDBB, FEniCS, ODEintV2, TVM.ai. Hardware accelerator integration (Xilinx, Intel, Achronix). |
Speedgo Chung Shin Yee | Python library | All | No | ~20 MPOPS | Extensive; no known bugs | Open source (MIT license) |
softposit-rkt David Thien | SoftPosit bindings for Racket | All | Yes | Unknown | Unknown | |
sfpy Bill Zorn | SoftPosit bindings for Python | All | Yes | ~20–45 MPOPS on 4.9 GHz Skylake core | Unknown | |
positsoctave Diego Coelho | Octave implementation | All | No | Unknown | Limited Testing; no known bugs | GNU GPL |
Sigmoid Numbers Isaac Yonemoto | Julia library | All <32, all ES | Yes | Unknown | No known bugs (posits). Division bugs (valids) | Leverages Julia's templated mathematics standard library, can natively do matrix and tensor operations, complex numbers, FFT, DiffEQ. Support for valids |
FastSigmoid Isaac Yonemoto | Julia and C/C++ library | 8, 16, 32, all ES | No | Unknown | Known bug in 32-bit multiplication | Used by LLNL in shock studies |
SoftPosit.jl Milan Klöwer | Julia library | Based on softposit; 8-bit (es=0..2) 16-bit (es=0..2) 24-bit (es=1..2) 32-bit (es=2) | Yes | Similar to A*STAR "SoftPosit" (Cerlane Leong) | Yes: Posit (8,0), Posit (16,1), Posit (32,2) Other formats lack full functionality | Open source. Issues and suggestions on GitHub. This project was developed due to the fact that SigmoidNumbers and FastSigmoid by Isaac Yonemoto is not maintained currently. Supports basic linear algebra functions in Julia (Matrix multiplication, Matrix solve, Elgen decomposition, etc.) |
PySigmoid Ken Mercado | Python library | All | Yes | < 20 MPOPS | Unknown | Open source (MIT license). Easy-to-use interface. Neural net example. Comprehensive functions support. |
cppPosit Federico Rossi, Emanuele Ruffaldi | C++ library | 4 to 64 (any es value); "Template version is 2 to 63 bits" | No | Unknown | A few basic tests | 4 levels of operations working with posits. Special support for NaN types (non-standard) |
bfp:Beyond Floating Point Clément Guérin | C++ library | Any | No | Unknown | Bugs found; status of fixes unknown | Supports + – × ÷ √ reciprocal, negate, compare |
Verilog.jl Isaac Yonemoto | Julia and Verilog | 8, 16, 32, ES=0 | No | Unknown | Comprehensively tested for 8-bit, no known bugs | Intended for Deep Learning applications Addition, Subtraction and Multiplication only. A proof of concept matrix multiplier has been built, but is off-spec in its precision |
Lombiq Arithmetics Lombiq Technologies | C# with Hastlayer for hardware generation | 8, 16, 32. (64bits in progress) | Yes | 10 MPOPS Click here for more | Partial | Requires Microsoft .Net APIs |
Deepfloat Jeff Johnson, Facebook | SystemVerilog | Any (parameterized SystemVerilog) | Yes | N/A (RTL for FPGA/ASIC designs) | Limited | Does not strictly conform to posit spec. Supports +,-,/,*. Implements both logarithmic posit and normal, "linear" posits License: CC-BY-NC 4.0 at present |
Tokyo Tech | FPGA | 16, 32, extendable | No | "2 GHz", not translated to MPOPS | Partial; known rounding bugs | Yet to be open-source |
PACoGen: Posit Arthmetic Core Generator Manish Kumar Jaiswal | Verilog HDL for Posit Arithmetic | Any precision. Able to generate any combination of word-size (N) and exponent-size (ES) | No | Speed of design is based on the underlying hardware platform (ASIC/FPGA) | Exhaustive tests for 8-bit posit. Multi-million random tests are performed for up to 32-bit posit with various ES combinations | It supports rounding-to-nearest rounding method. |
Vinay Saxena, Research and Technology Centre, Robert Bosch, India (RTC-IN) and Farhad Merchant, RWTH Aachen University | Verilog generator for VLSI, FPGA | All | No | Similar to floats of same bit size | N=8 - ES=2 | N=7,8,9,10,11,12 Selective (20000*65536) combinations for - ES=1 | N=16 | To be used in commercial products. To the best of our knowledge. ***First ever integration of posits in RISC-V*** |
Posit-enabled RISC-V core (Sugandha Tiwari, Neel Gala, Chester Rebeiro, V.Kamakoti, IIT MADRAS) | BSV (Bluespec System Verilog) Implementation | 32-bit posit with (es=2) and (es=3) | No | — | Verified against SoftPosit for (es=2) and tested with several applications for (es=2) and (es=3). No known bugs. | First complete posit-capable RISC-V core. Supports dynamic switching between (es=2) and (es=3). More info here. |
PERCIVAL David Mallasén | Open-Source Posit RISC-V Core with Quire Capability | Posit<32,2> with 512-bit quire | Yes | Speed of design is based on the underlying hardware platform (ASIC/FPGA) | Functionality testing of each posit instruction. | Application-level posit-capable RISC-V core based on CVA6 that can execute all posit instructions, including the quire fused operations. PERCIVAL is the first work that integrates the complete posit ISA and quire in hardware. It allows the native execution of posit instructions as well as the standard floating-point ones simultaneously. |
LibPosit Chris Lomont | Single file C# MIT Licensed | Any size | No | Extensive; no known bugs | Ops: arithmetic, comparisons, sqrt, sin, cos, tan, acos, asin, atan, pow, exp, log | |
unumjl REX Computing | FPGA version of the "Neo" VLIW processor with posit numeric unit | 32 | No | ~1.2 GPOPS | Extensive; no known bugs | No divide or square root. First full processor design to replace floats with posits. |
PNU: Posit Numeric Unit Calligo Tech |
|
| Yes - Fully supported. | 500 MHz * 8 Cores | Exhaustive tests completed for 32 bits and 64 bits with Quire support completed. Applications tested and being made available for seamless adoption www.calligotech.com | Fully integrated with C/C++ types and automatic conversions. Supports full C++ math library (native and conversion to/from IEEE). Runtime integrations: GNU Utils, OpenBLAS, CBLAS. Application integrations: in progress. Compiler support extended: C/C++, G++, GFortran & LLVM (in progress). |
IBM-TACC Jianyu Chen | Specific-purpose FPGA | 32 | Yes | 16–64 GPOPS | Only one known case tested | Does 128-by-128 matrix-matrix multiplication (SGEMM) using quire. |
Deep PeNSieve Raul Murillo | Python library (software) | 8, 16, 32 | Yes | Unknown | Unknown | A DNN framework using posits |
Gosit Jaap Aarts | Pure Go library | 16/1 32/2 (included is a generic 32/ES for ES<32)[ clarification needed ] | No | 80 MPOPS for div32/2 and similar linear functions. Much higher for truncate and much lower for exp. | Fuzzing against C softposit with a lot of iterations for 16/1 and 32/2. Explicitly testing edge cases found. | (MIT license) The implementations where ES is constant the code is generated. The generator should be able to generate for all sizes {8,16,32} and ES below the size. However, the ones not included into the library by default are not tested, fuzzed, or supported. For some operations on 32/ES, mixing and matching ES is possible. However, this is not tested. |
SoftPosit [31] is a software implementation of posits based on Berkeley SoftFloat. [32] It allows software comparison between posits and floats. It currently supports
It works for 16-bit posits with one exponent bit and 8-bit posit with zero exponent bit. Support for 32-bit posits and flexible type (2-32 bits with two exponent bits) is pending validation. It supports x86_64 systems. It has been tested on GNU gcc (SUSE Linux) 4.8.5 Apple LLVM version 9.1.0 (clang-902.0.39.2).
Add with posit8_t
#include"softposit.h"intmain(intargc,char*argv[]){posit8_tpA,pB,pZ;pA=castP8(0xF2);pB=castP8(0x23);pZ=p8_add(pA,pB);// To check answer by converting it to doubledoubledZ=convertP8ToDouble(pZ);printf("dZ: %.15f\n",dZ);// To print result in binary (warning: non-portable code)uint8_tuiZ=castUI8(pZ);printBinary((uint64_t*)&uiZ,8);return0;}
Fused dot product with quire16_t
// Convert double to positposit16_tpA=convertDoubleToP16(1.02783203125);posit16_tpB=convertDoubleToP16(0.987060546875);posit16_tpC=convertDoubleToP16(0.4998779296875);posit16_tpD=convertDoubleToP16(0.8797607421875);quire16_tqZ;// Set quire to 0qZ=q16_clr(qZ);// Accumulate products without roundingsqZ=q16_fdp_add(qZ,pA,pB);qZ=q16_fdp_add(qZ,pC,pD);// Convert back to positposit16_tpZ=q16_to_p16(qZ);// To check answerdoubledZ=convertP16ToDouble(pZ);
William M. Kahan, the principal architect of IEEE 754-1985 criticizes type I unums on the following grounds (some are addressed in type II and type III standards): [6] [33]
In computing, floating-point arithmetic (FP) is arithmetic that represents subsets of real numbers using an integer with a fixed precision, called the significand, scaled by an integer exponent of a fixed base. Numbers of this form are called floating-point numbers. For example, 12.345 is a floating-point number in base ten with five digits of precision:
IEEE 754-1985 is a historic industry standard for representing floating-point numbers in computers, officially adopted in 1985 and superseded in 2008 by IEEE 754-2008, and then again in 2019 by minor revision IEEE 754-2019. During its 23 years, it was the most widely used format for floating-point computation. It was implemented in software, in the form of floating-point libraries, and in hardware, in the instructions of many CPUs and FPUs. The first integrated circuit to implement the draft of what was to become IEEE 754-1985 was the Intel 8087.
A computer number format is the internal representation of numeric values in digital device hardware and software, such as in programmable computers and calculators. Numerical values are stored as groupings of bits, such as bytes and words. The encoding between numerical values and bit patterns is chosen for convenience of the operation of the computer; the encoding used by the computer's instruction set generally requires conversion for external use, such as for printing and display. Different types of processors may have different internal representations of numerical values and different conventions are used for integer and real numbers. Most calculations are carried out with number formats that fit into a processor register, but some software systems allow representation of arbitrarily large numbers using multiple words of memory.
In computing, NaN, standing for Not a Number, is a particular value of a numeric data type which is undefined as a number, such as the result of 0/0. Systematic use of NaNs was introduced by the IEEE 754 floating-point standard in 1985, along with the representation of other non-finite quantities such as infinities.
Double-precision floating-point format is a floating-point number format, usually occupying 64 bits in computer memory; it represents a wide range of numeric values by using a floating radix point.
The IEEE Standard for Floating-Point Arithmetic is a technical standard for floating-point arithmetic originally established in 1985 by the Institute of Electrical and Electronics Engineers (IEEE). The standard addressed many problems found in the diverse floating-point implementations that made them difficult to use reliably and portably. Many hardware floating-point units use the IEEE 754 standard.
The significand is the first (left) part of a number in scientific notation or related concepts in floating-point representation, consisting of its significant digits. Depending on the interpretation of the exponent, the significand may represent an integer or a fractional number.
The term arithmetic underflow is a condition in a computer program where the result of a calculation is a number of more precise absolute value than the computer can actually represent in memory on its central processing unit (CPU).
Signed zero is zero with an associated sign. In ordinary arithmetic, the number 0 does not have a sign, so that −0, +0 and 0 are equivalent. However, in computing, some number representations allow for the existence of two zeros, often denoted by −0 and +0, regarded as equal by the numerical comparison operations but with possible different behaviors in particular operations. This occurs in the sign-magnitude and ones' complement signed number representations for integers, and in most floating-point number representations. The number 0 is usually encoded as +0, but can still be represented by +0, −0, or 0.
In C and related programming languages, long double
refers to a floating-point data type that is often more precise than double precision though the language standard only requires it to be at least as precise as double
. As with C's other floating-point types, it may not necessarily map to an IEEE format.
Extended precision refers to floating-point number formats that provide greater precision than the basic floating-point formats. Extended precision formats support a basic format by minimizing roundoff and overflow errors in intermediate values of expressions on the base format. In contrast to extended precision, arbitrary-precision arithmetic refers to implementations of much larger numeric types using special software.
Decimal floating-point (DFP) arithmetic refers to both a representation and operations on decimal floating-point numbers. Working directly with decimal (base-10) fractions can avoid the rounding errors that otherwise typically occur when converting between decimal fractions and binary (base-2) fractions.
IEEE 754-2008 is a revision of the IEEE 754 standard for floating-point arithmetic. It was published in August 2008 and is a significant revision to, and replaces, the IEEE 754-1985 standard. The 2008 revision extended the previous standard where it was necessary, added decimal arithmetic and formats, tightened up certain areas of the original standard which were left undefined, and merged in IEEE 854 . In a few cases, where stricter definitions of binary floating-point arithmetic might be performance-incompatible with some existing implementation, they were made optional. In 2019, it was updated with a minor revision IEEE 754-2019.
Offset binary, also referred to as excess-K, excess-N, excess-e, excess code or biased representation, is a method for signed number representation where a signed number n is represented by the bit pattern corresponding to the unsigned number n+K, K being the biasing value or offset. There is no standard for offset binary, but most often the K for an n-bit binary word is K = 2n−1 (for example, the offset for a four-digit binary number would be 23=8). This has the consequence that the minimal negative value is represented by all-zeros, the "zero" value is represented by a 1 in the most significant bit and zero in all other bits, and the maximal positive value is represented by all-ones (conveniently, this is the same as using two's complement but with the most significant bit inverted). It also has the consequence that in a logical comparison operation, one gets the same result as with a true form numerical comparison operation, whereas, in two's complement notation a logical comparison will agree with true form numerical comparison operation if and only if the numbers being compared have the same sign. Otherwise the sense of the comparison will be inverted, with all negative values being taken as being larger than all positive values.
In computing, half precision is a binary floating-point computer number format that occupies 16 bits in computer memory. It is intended for storage of floating-point values in applications where higher precision is not essential, in particular image processing and neural networks.
In computing, quadruple precision is a binary floating-point–based computer number format that occupies 16 bytes with precision at least twice the 53-bit double precision.
Single-precision floating-point format is a computer number format, usually occupying 32 bits in computer memory; it represents a wide dynamic range of numeric values by using a floating radix point.
In computing, Microsoft Binary Format (MBF) is a format for floating-point numbers which was used in Microsoft's BASIC languages, including MBASIC, GW-BASIC and QuickBASIC prior to version 4.00.
Floating-point error mitigation is the minimization of errors caused by the fact that real numbers cannot, in general, be accurately represented in a fixed space. By definition, floating-point error cannot be eliminated, and, at best, can only be managed.
The bfloat16 floating-point format is a computer number format occupying 16 bits in computer memory; it represents a wide dynamic range of numeric values by using a floating radix point. This format is a shortened (16-bit) version of the 32-bit IEEE 754 single-precision floating-point format (binary32) with the intent of accelerating machine learning and near-sensor computing. It preserves the approximate dynamic range of 32-bit floating-point numbers by retaining 8 exponent bits, but supports only an 8-bit precision rather than the 24-bit significand of the binary32 format. More so than single-precision 32-bit floating-point numbers, bfloat16 numbers are unsuitable for integer calculations, but this is not their intended use. Bfloat16 is used to reduce the storage requirements and increase the calculation speed of machine learning algorithms.
JG: The word "unum" is short for "universal number," the same way the word "bit" is short for "binary digit."
I started out calling them "unums 2.0," which seemed to be as good a name for the concept as any, but it is really not a "latest release" so much as it is an alternative.