Hoard memory allocator

Last updated
Hoard
Original author(s) Emery Berger, Kathryn S. McKinley, Robert D. Blumofe, Paul R. Wilson
Developer(s) Emery Berger
Initial releaseSeptember 29, 1999 (1999-09-29)
Stable release
3.13 / January 1, 2019;3 years ago (2019-01-01)
Repository
Operating system Linux, OS X, Microsoft Windows
Available in C++
Type Memory allocation
License Apache License v2
Website hoard.org

The Hoard memory allocator, or Hoard, is a memory allocator for Linux, OS X, and Microsoft Windows. Hoard is designed to be efficient when used by multithreaded applications on multiprocessor computers. Hoard is distributed under the Apache License, version 2.0.

Contents

History

In 2000, its author Emery Berger benchmarked some famous memory allocators and stated Hoard improves the performance of multithreaded applications by providing fast, scalable memory management functions (malloc and free). In particular, it reduces contention for the heap (the central data structure used in dynamic memory allocation) caused when multiple threads allocate or free memory, and avoids the false sharing that can be introduced by memory allocators. At the same time, Hoard has strict bounds on fragmentation. [1]

Hoard continues to be maintained and improved, and is in use by a number of open source and commercial projects. [2] [3]

It has also inspired changes to other memory allocators such as the one in OS X since February 2008 (first released in Mac OS X Snow Leopard). [4] [5]

See also

Related Research Articles

C (programming language) General-purpose programming language

C is a general-purpose computer programming language. It was created in the 1970s by Dennis Ritchie, and remains very widely used and influential. By design, C's features cleanly reflect the capabilities of the targeted CPUs. It has found lasting use in operating systems, device drivers, protocol stacks, though decreasingly for application software, and is common in computer architectures that range from the largest supercomputers to the smallest microcontrollers and embedded systems.

Garbage collection (computer science) Form of automatic memory management

In computer science, garbage collection (GC) is a form of automatic memory management. The garbage collector attempts to reclaim memory which was allocated by the program, but is no longer referenced—also called garbage. Garbage collection was invented by American computer scientist John McCarthy around 1959 to simplify manual memory management in Lisp.

Memory management Computer memory management methodology

Memory management is a form of resource management applied to computer memory. The essential requirement of memory management is to provide ways to dynamically allocate portions of memory to programs at their request, and free it for reuse when no longer needed. This is critical to any advanced computer system where more than a single process might be underway at any time.

Classic Mac OS memory management

Historically, the classic Mac OS used a form of memory management that has fallen out of favor in modern systems. Criticism of this approach was one of the key areas addressed by the change to Mac OS X.

C dynamic memory allocation refers to performing manual memory management for dynamic memory allocation in the C programming language via a group of functions in the C standard library, namely malloc, realloc, calloc and free.

Valgrind Programming tool for profiling, memory debugging and memory leak detection

Valgrind is a programming tool for memory debugging, memory leak detection, and profiling.

Address space layout randomization (ASLR) is a computer security technique involved in preventing exploitation of memory corruption vulnerabilities. In order to prevent an attacker from reliably jumping to, for example, a particular exploited function in memory, ASLR randomly arranges the address space positions of key data areas of a process, including the base of the executable and the positions of the stack, heap and libraries.

Memory pools, also called fixed-size blocks allocation, is the use of pools for memory management that allows dynamic memory allocation comparable to malloc or C++'s operator new. As those implementations suffer from fragmentation because of variable block sizes, it is not recommendable to use them in a real time system due to performance. A more efficient solution is preallocating a number of memory blocks with the same size called the memory pool. The application can allocate, access, and free blocks represented by handles at run time.

Stack-based memory allocation

Stacks in computing architectures are regions of memory where data is added or removed in a last-in-first-out (LIFO) manner.

In the C++ programming language, new and delete are a pair of language constructs that perform dynamic memory allocation, object construction and object destruction.

In computer science, manual memory management refers to the usage of manual instructions by the programmer to identify and deallocate unused objects, or garbage. Up until the mid-1990s, the majority of programming languages used in industry supported manual memory management, though garbage collection has existed since 1959, when it was introduced with Lisp. Today, however, languages with garbage collection such as Java are increasingly popular and the languages Objective-C and Swift provide similar functionality through Automatic Reference Counting. The main manually managed languages still in widespread use today are C and C++ – see C dynamic memory allocation.

Dynamic program analysis is the analysis of computer software that is performed by executing programs on a real or virtual processor. For dynamic program analysis to be effective, the target program must be executed with sufficient test inputs to cover almost all possible outputs. Use of software testing measures such as code coverage helps ensure that an adequate slice of the program's set of possible behaviors has been observed. Also, care must be taken to minimize the effect that instrumentation has on the execution of the target program. Dynamic analysis is in contrast to static program analysis. Unit tests, integration tests, system tests and acceptance tests use dynamic testing.

Libumem is a memory allocator userspace library used to manage memory allocation, and may be used to detect memory management bugs in applications. It is based on the Slab allocator concept. Libumem is available as a standard part of Solaris from Solaris 9 Update 3 onwards.

mtrace is the memory debugger included in the GNU C Library.

oneAPI Threading Building Blocks, is a C++ template library developed by Intel for parallel programming on multi-core processors. Using TBB, a computation is broken down into tasks that can run in parallel. The library manages and schedules threads to execute these tasks.

ChibiOS/RT Real-time operating system

ChibiOS/RT is a compact and fast real-time operating system supporting multiple architectures and released under a mix of the GNU General Public License version 3 (GPL3) and the Apache License 2.0. It is developed by Giovanni Di Sirio.

brk and sbrk are basic memory management system calls used in Unix and Unix-like operating systems to control the amount of memory allocated to the data segment of the process. These functions are typically called from a higher-level memory management library function such as malloc. In the original Unix system, brk and sbrk were the only ways in which applications could acquire additional data space; later versions allowed this to also be done using the mmap call.

In operating systems, memory management is the function responsible for managing the computer's primary memory.

Kathryn S. McKinley American computer scientist

Kathryn S. McKinley is an American computer scientist noted for her research on compilers, runtime systems, and computer architecture. She is also known for her leadership in broadening participation in computing. McKinley was co-chair of CRA-W from 2011 to 2014.

mimalloc is a free and open-source compact general-purpose memory allocator developed by Microsoft with focus on performance characteristics. The library is about 11000 lines of code and works as a drop-in replacement for malloc of the C standard library and requires no additional code changes. mimalloc was initially developed for the run-time systems of the Lean and Koka languages. Notable design aspects include free list sharding, eager page reset, first-class heaps. It can co-exist with other memory allocators linked to the same program. mimalloc is available on Windows, Mac OS X, Linux and *BSD. The source code is licensed under MIT License and available on GitHub.

References

  1. Berger, E. D.; McKinley, K. S.; Blumofe, R. D.; Wilson, P. R. (November 2000). Hoard: A Scalable Memory Allocator for Multithreaded Applications (PDF). ASPLOS-IX. Proceedings of the ninth international conference on Architectural support for programming languages and operating systems. pp. 117–128. CiteSeerX   10.1.1.1.4174 . doi:10.1145/378993.379232. ISBN   1-58113-317-0.
  2. "An alternative Memory Allocator for the standard glibc". 2007-09-16. Archived from the original on 2011-10-07.
  3. "GNU Common C++ Downloading".
  4. "I'm a Mac (or, "Emery Inside")". 26 October 2010.
  5. "A look at how malloc works on the Mac".