SPECint

Last updated

SPEC INT is a computer benchmark specification for CPU integer processing power. It is maintained by the Standard Performance Evaluation Corporation (SPEC). SPEC INT is the integer performance testing component of the SPEC test suite. The first SPEC test suite, CPU92, was announced in 1992. It was followed by CPU95, CPU2000, and CPU2006. The latest standard is SPEC CPU 2017 and consists of SPEC speed and SPEC rate (aka SPECCPU_2017).

Contents

SPEC INT 2006

CPU2006 is a set of benchmarks designed to test the CPU performance of a modern server computer system. It is split into two components, the first being CINT2006, the other being CFP2006 (SPECfp), for floating point testing.

SPEC defines a base runtime for each of the 12 benchmark programs. For SPECint2006, that number ranges from 1000 to 3000 seconds. The timed test is run on the system, and the time of the test system is compared to the reference time, and a ratio is computed. That ratio becomes the SPEC INT score for that test. (This differs from the rating in SPECINT2000, which multiplies the ratio by 100.)

As an example for SPECint2006, consider a processor which can run 400.perlbench in 2000 seconds. The time it takes the reference machine to run the benchmark is 9770 seconds. [1] Thus the ratio is 4.885. Each ratio is computed, and then the geometric mean of those ratios is computed to produce an overall value.

Background

For a fee, SPEC distributes source code files to users wanting to test their systems. These files are written in a standard programming language, which is then compiled for each particular CPU architecture and operating system. Thus, the performance measured is that of the CPU, RAM, and compiler, and does not test I/O, networking, or graphics.

Two metrics are reported for a particular benchmark, "base" and "peak". Compiler options account for the difference between the two numbers. As the SPEC benchmarks are distributed as source code, it is up to the party performing the test to compile this code. There is agreement that the benchmarks should be compiled in the same way as a user would compile a program, but there is no consistent method for user compilation, it varies system by system. SPEC, in this case, defines two reference points, "base" and "peak". Base has a more strict set of compilation rules than peak. Less optimization can be done, the compiler flags must be the same for each benchmark, in the same order, and there must be a limited number of flags. Base, then, is closest to how a user would compile a program with standard flags. The 'peak' metric can be performed with maximum compiler optimization, even to the extent of different optimizations for each benchmark. This number represents maximum system performance, achieved by full compiler optimization.

SPEC INT tests are carried out on a wide range of hardware, with results typically published for the full range of system-level implementations employing the latest CPUs. For SPECint2006, the CPUs include Intel and AMD x86 & x86-64 processors, Sun SPARC CPUs, IBM Power CPUs, and IA-64 CPUs. This range of capabilities, specifically in this case the number of CPUs, means that the SPEC INT benchmark is usually run on only a single CPU, even if the system has many CPUs. If a single CPU has multiple cores, only a single core is used; hyper-threading is also typically disabled,

A more complete system-level benchmark that allows all CPUs to be used is known as SPECint_rate2006, also called "CINT2006 Rate".

Benchmarks

The SPECint2006 test suite consists of 12 benchmark programs, designed to test exclusively the integer performance of the system.

The benchmarks are: [2]

Benchmark Language CategoryDescription
400.perlbenchCPerl Programming LanguageDerived from Perl V5.8.7. The workload includes SpamAssassin, MHonArc (an email indexer), and specdiff (SPEC's tool that checks benchmark outputs).
401.bzip2CCompressionJulian Seward's bzip2 version 1.0.3, modified to do most work in memory, rather than doing I/O.
403.gccCC CompilerBased on gcc Version 3.2, generates code for Opteron.
429.mcfCCombinatorial OptimizationVehicle scheduling. Uses a network simplex algorithm (which is also used in commercial products) to schedule public transport.
445.gobmkCArtificial Intelligence: go playingPlays the game of Go, a simply described but deeply complex game.
456.hmmerCSearch Gene SequenceProtein sequence analysis using profile hidden Markov models (profile HMMs)
458.sjengCArtificial Intelligence: chess playingA highly-ranked chess program that also plays several chess variants.
462.libquantumCPhysics: Quantum ComputingSimulates a quantum computer, running Shor's polynomial-time factorization algorithm.
464.h264refCVideo CompressionA reference implementation of H.264/AVC, encodes a videostream using 2 parameter sets. The H.264/AVC standard is expected to replace MPEG2
471.omnetppC++Discrete Event SimulationUses the OMNet++ discrete event simulator to model a large Ethernet campus network.
473.astarC++Path-finding AlgorithmsPathfinding library for 2D maps, including the well known A* algorithm.
483.xalancbmkC++XML ProcessingA modified version of Xalan-C++, which transforms XML documents to other document types.

See also

Related Research Articles

Processor design is a subfield of computer science and computer engineering (fabrication) that deals with creating a processor, a key component of computer hardware.

Dhrystone is a synthetic computing benchmark program developed in 1984 by Reinhold P. Weicker intended to be representative of system (integer) programming. The Dhrystone grew to become representative of general processor (CPU) performance. The name "Dhrystone" is a pun on a different benchmark algorithm called Whetstone, which emphasizes floating point performance.

In computing, an optimizing compiler is a compiler that tries to minimize or maximize some attributes of an executable computer program. Common requirements are to minimize a program's execution time, memory footprint, storage size, and power consumption.

x86 assembly language is the name for the family of assembly languages which provide some level of backward compatibility with CPUs back to the Intel 8008 microprocessor, which was launched in April 1972. It is used to produce object code for the x86 class of processors.

In computer science, program optimization, code optimization, or software optimization is the process of modifying a software system to make some aspect of it work more efficiently or use fewer resources. In general, a computer program may be optimized so that it executes more rapidly, or to make it capable of operating with less memory storage or other resources, or draw less power.

<span class="mw-page-title-main">OpenMP</span> Open standard for parallelizing

OpenMP is an application programming interface (API) that supports multi-platform shared-memory multiprocessing programming in C, C++, and Fortran, on many platforms, instruction-set architectures and operating systems, including Solaris, AIX, FreeBSD, HP-UX, Linux, macOS, and Windows. It consists of a set of compiler directives, library routines, and environment variables that influence run-time behavior.

In computer programming, undefined behavior (UB) is the result of executing a program whose behavior is prescribed to be unpredictable, in the language specification to which the computer code adheres. This is different from unspecified behavior, for which the language specification does not prescribe a result, and implementation-defined behavior that defers to the documentation of another component of the platform.

<span class="mw-page-title-main">Action! (programming language)</span> Atari 8-bit family programming language

Action! is a procedural programming language and integrated development environment written by Clinton Parker for the Atari 8-bit family. The language, which is similar to ALGOL, compiles to high-performance code for the MOS Technology 6502 of the Atari computers. Action! was distributed on ROM cartridge by Optimized Systems Software starting in 1983. It was one of the company's first bank-switched 16 kB "Super Cartridges". The runtime library is stored in the cartridge; to make a standalone application requires the Action! Toolkit which was sold separately by OSS.

<span class="mw-page-title-main">Benchmark (computing)</span> Comparing the relative performance of computers by running the same program on all of them

In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it.

SPECfp is a computer benchmark designed to test the floating-point performance of a computer. It is managed by the Standard Performance Evaluation Corporation. SPECfp is the floating-point performance testing component of the SPEC CPU testing suit. The first standard SPECfp was released in 1989 as SPECfp89. Later it was replaced by SPECfp92, then SPECfp95, then SPECfp2000, then SPECfp2006, and finally SPECfp2017.

Intel oneAPI DPC++/C++ Compiler and Intel C++ Compiler Classic are Intel’s C, C++, SYCL, and Data Parallel C++ (DPC++) compilers for Intel processor-based systems, available for Windows, Linux, and macOS operating systems.

In computing, computer performance is the amount of useful work accomplished by a computer system. Outside of specific contexts, computer performance is estimated in terms of accuracy, efficiency and speed of executing computer program instructions. When it comes to high computer performance, one or more of the following factors might be involved:

AMD FireStream was AMD's brand name for their Radeon-based product line targeting stream processing and/or GPGPU in supercomputers. Originally developed by ATI Technologies around the Radeon X1900 XTX in 2006, the product line was previously branded as both ATI FireSTREAM and AMD Stream Processor. The AMD FireStream can also be used as a floating-point co-processor for offloading CPU calculations, which is part of the Torrenza initiative. The FireStream line has been discontinued since 2012, when GPGPU workloads were entirely folded into the AMD FirePro line.

<span class="mw-page-title-main">Computer architecture</span> Set of rules describing computer system

In computer science and computer engineering, computer architecture is a description of the structure of a computer system made from component parts. It can sometimes be a high-level description that ignores details of the implementation. At a more detailed level, the description may include the instruction set architecture design, microarchitecture design, logic design, and implementation.

NBench, short for Native mode Benchmark and later known as BYTEmark, is a synthetic computing benchmark program developed in the mid-1990s by the now defunct BYTE magazine intended to measure a computer's CPU, FPU, and Memory System speed.

FastCode is an open source programming project that aims to provide optimized runtime library routines for Embarcadero Delphi and C++ Builder. This community-driven project was started in 2003 by Dennis Kjaer Christensen and has since contributed optimized functionality to the 32-bit Delphi runtime library (RTL).

The x32 ABI is an application binary interface (ABI) and one of the interfaces of the Linux kernel. The x32 ABI provides 32-bit integers, long and pointers (ILP32) on Intel and AMD 64-bit hardware. The ABI allows programs to take advantage of the benefits of x86-64 instruction set while using 32-bit pointers and thus avoiding the overhead of 64-bit pointers.

The LINPACK Benchmarks are a measure of a system's floating-point computing power. Introduced by Jack Dongarra, they measure how fast a computer solves a dense n by n system of linear equations Ax = b, which is a common task in engineering.

<span class="mw-page-title-main">Titan (supercomputer)</span> American supercomputer

Titan or OLCF-3 was a supercomputer built by Cray at Oak Ridge National Laboratory for use in a variety of science projects. Titan was an upgrade of Jaguar, a previous supercomputer at Oak Ridge, that uses graphics processing units (GPUs) in addition to conventional central processing units (CPUs). Titan was the first such hybrid to perform over 10 petaFLOPS. The upgrade began in October 2011, commenced stability testing in October 2012 and it became available to researchers in early 2013. The initial cost of the upgrade was US$60 million, funded primarily by the United States Department of Energy.

The Rugg/Feldman benchmarks are a series of seven short BASIC programming language programs that are used to test the performance of BASIC implementations on various microcomputers. They were published by Tom Rugg and Phil Feldman in the June 1977 issue of the US computer magazine, Kilobaud.

References

  1. "The SPEC Benchmarks". 2003-02-03. Retrieved 2008-09-01.
  2. "SPEC CINT2006 Benchmarks".