Virtual address space

Last updated
Virtual address space and physical address space relationship.svg

In computing, a virtual address space (VAS) or address space is the set of ranges of virtual addresses that an operating system makes available to a process. [1] The range of virtual addresses usually starts at a low address and can extend to the highest address allowed by the computer's instruction set architecture and supported by the operating system's pointer size implementation, which can be 4 bytes for 32-bit or 8 bytes for 64-bit OS versions. This provides several benefits, one of which is security through process isolation assuming each process is given a separate address space.

Contents

Example

In the following description, the terminology used will be particular to the Windows NT operating system, but the concepts are applicable to other virtual memory operating systems.

When a new application on a 32-bit OS is executed, the process has a 4 GiB VAS: each one of the memory addresses (from 0 to 232 − 1) in that space can have a single byte as a value. Initially, none of them have values ('-' represents no value). Using or setting values in such a VAS would cause a memory exception.

           0                                           4 GiB VAS        |----------------------------------------------|

Then the application's executable file is mapped into the VAS. Addresses in the process VAS are mapped to bytes in the exe file. The OS manages the mapping:

           0                                           4 GiB VAS        |---vvvvvvv------------------------------------| mapping        |-----| file bytes      app.exe

The v's are values from bytes in the mapped file. Then, required DLL files are mapped (this includes custom libraries as well as system ones such as kernel32.dll and user32.dll):

           0                                           4 GiB VAS        |---vvvvvvv----vvvvvv---vvvv-------------------| mapping        |||||||    ||||||   |||| file bytes      app.exe    kernel   user

The process then starts executing bytes in the exe file. However, the only way the process can use or set '-' values in its VAS is to ask the OS to map them to bytes from a file. A common way to use VAS memory in this way is to map it to the page file. The page file is a single file, but multiple distinct sets of contiguous bytes can be mapped into a VAS:

           0                                           4 GiB VAS        |---vvvvvvv----vvvvvv---vvvv----vv---v----vvv--| mapping        |||||||    ||||||   ||||    ||   |    ||| file bytes      app.exe    kernel   user   system_page_file

And different parts of the page file can map into the VAS of different processes:

           0                                           4 GiB VAS 1      |---vvvv-------vvvvvv---vvvv----vv---v----vvv--| mapping        ||||       ||||||   ||||    ||   |    ||| file bytes      app1 app2  kernel   user   system_page_file mapping             ||||  ||||||   ||||       ||   | VAS 2      |--------vvvv--vvvvvv---vvvv-------vv---v------|

On Microsoft Windows 32-bit, by default, only 2 GiB are made available to processes for their own use. [2] The other 2 GiB are used by the operating system. On later 32-bit editions of Microsoft Windows it is possible to extend the user-mode virtual address space to 3 GiB while only 1 GiB is left for kernel-mode virtual address space by marking the programs as IMAGE_FILE_LARGE_ADDRESS_AWARE and enabling the /3GB switch in the boot.ini file. [3] [4]

On Microsoft Windows 64-bit, in a process running an executable that was linked with /LARGEADDRESSAWARE:NO, the operating system artificially limits the user mode portion of the process's virtual address space to 2 GiB. This applies to both 32- and 64-bit executables. [5] [6] Processes running executables that were linked with the /LARGEADDRESSAWARE:YES option, which is the default for 64-bit Visual Studio 2010 and later, [7] have access to more than 2 GiB of virtual address space: Up to 4 GiB for 32-bit executables, up to 8 TiB for 64-bit executables in Windows through Windows 8, and up to 128 TiB for 64-bit executables in Windows 8.1 and later. [4] [8]

Allocating memory via C's malloc establishes the page file as the backing store for any new virtual address space. However, a process can also explicitly map file bytes.

Linux

For x86 CPUs, Linux 32-bit allows splitting the user and kernel address ranges in different ways: 3G/1G user/kernel (default), 1G/3G user/kernel or 2G/2G user/kernel. [9]

See also

Notes

  1. IBM Corporation. "What is an address space?" . Retrieved August 24, 2013.
  2. "Virtual Address Space". MSDN . Microsoft.
  3. "LOADED_IMAGE structure". MSDN . Microsoft.
  4. 1 2 "4-Gigabyte Tuning: BCDEdit and Boot.ini". MSDN . Microsoft.
  5. "/LARGEADDRESSAWARE (Handle Large Addresses)". MSDN . Microsoft.
  6. "Virtual Address Space". MSDN . Microsoft.
  7. "/LARGEADDRESSAWARE (Handle Large Addresses)". MSDN . Microsoft.
  8. "/LARGEADDRESSAWARE (Handle Large Addresses)". MSDN . Microsoft.
  9. "Linux kernel - x86: Memory split".

Related Research Articles

The Portable Executable (PE) format is a file format for executables, object code, DLLs and others used in 32-bit and 64-bit versions of Windows operating systems. The PE format is a data structure that encapsulates the information necessary for the Windows OS loader to manage the wrapped executable code. This includes dynamic library references for linking, API export and import tables, resource management data and thread-local storage (TLS) data. On NT operating systems, the PE format is used for EXE, DLL, SYS, MUI and other file types. The Unified Extensible Firmware Interface (UEFI) specification states that PE is the standard executable format in EFI environments.

The Windows API, informally WinAPI, is Microsoft's core set of application programming interfaces (APIs) available in the Microsoft Windows operating systems. The name Windows API collectively refers to several different platform implementations that are often referred to by their own names ; see the versions section. Almost all Windows programs interact with the Windows API. On the Windows NT line of operating systems, a small number use the Native API.

In computer architecture, 64-bit integers, memory addresses, or other data units are those that are 64 bits wide. Also, 64-bit central processing unit (CPU) and arithmetic logic unit (ALU) architectures are those that are based on processor registers, address buses, or data buses of that size. 64-bit microcomputers are computers in which 64-bit microprocessors are the norm. From the software perspective, 64-bit computing means the use of machine code with 64-bit virtual memory addresses. However, not all 64-bit instruction sets support full 64-bit virtual memory addresses; x86-64 and ARMv8, for example, support only 48 bits of virtual address, with the remaining 16 bits of the virtual address required to be all 0's or all 1's, and several 64-bit instruction sets support fewer than 64 bits of physical memory address.

x86-64 Type of instruction set which is a 64-bit version of the x86 instruction set

x86-64 is a 64-bit version of the x86 instruction set, first released in 1999. It introduced two new modes of operation, 64-bit mode and compatibility mode, along with a new 4-level paging mode.

In computing, Physical Address Extension (PAE), sometimes referred to as Page Address Extension, is a memory management feature for the x86 architecture. PAE was first introduced by Intel in the Pentium Pro, and later by AMD in the Athlon processor. It defines a page table hierarchy of three levels (instead of two), with table entries of 64 bits each instead of 32, allowing these CPUs to directly access a physical address space larger than 4 gigabytes (232 bytes).

Windows 9x Series of Microsoft Windows computer operating systems

Windows 9x is a generic term referring to a series of Microsoft Windows computer operating systems produced from 1995 to 2000, which were based on the Windows 95 kernel and its underlying foundation of MS-DOS, both of which were updated in subsequent versions. The first version in the 9x series was Windows 95, which was succeeded by Windows 98 and then Windows Me, which was the last version of Windows on the 9x line.

PaX

PaX is a patch for the Linux kernel that implements least privilege protections for memory pages. The least-privilege approach allows computer programs to be able to restrict the set of operations they are allowed to perform–in the case of PaX, the ability to execute data as code, which is generally not applicable outside of certain kinds of programs but is often used for exploitation. PaX was first released in 2000.

Windows Registry Database for Microsoft Windows

The Windows Registry is a hierarchical database that stores low-level settings for the Microsoft Windows operating system and for applications that opt to use the registry. The kernel, device drivers, services, Security Accounts Manager, and user interfaces can all use the registry. The registry also allows access to counters for profiling system performance.

Architecture of Windows NT Overview of the architecture of the Microsoft Windows NT line of operating systems

The architecture of Windows NT, a line of operating systems produced and sold by Microsoft, is a layered design that consists of two main components, user mode and kernel mode. It is a preemptive, reentrant multitasking operating system, which has been designed to work with uniprocessor and symmetrical multiprocessor (SMP)-based computers. To process input/output (I/O) requests, they use packet-driven I/O, which utilizes I/O request packets (IRPs) and asynchronous I/O. Starting with Windows XP, Microsoft began making 64-bit versions of Windows available; before this, there were only 32-bit versions of these operating systems.

Dynamic-link library (DLL) is Microsoft's implementation of the shared library concept in the Microsoft Windows and OS/2 operating systems. These libraries usually have the file extension DLL, OCX, or DRV . The file formats for DLLs are the same as for Windows EXE files – that is, Portable Executable (PE) for 32-bit and 64-bit Windows, and New Executable (NE) for 16-bit Windows. As with EXEs, DLLs can contain code, data, and resources, in any combination.

The Windows NT startup process is the process by which Windows NT 4.0, Windows 2000, Windows XP and Windows Server 2003 operating systems initialize. In Windows Vista and later, this process has changed significantly; see Windows Vista startup process for information about what has changed.

In computing ntoskrnl.exe, also known as kernel image, provides the kernel and executive layers of the Microsoft Windows NT kernel space, and is responsible for various system services such as hardware abstraction, process and memory management, thus making it a fundamental part of the system. It contains the cache manager, the executive, the kernel, the security reference monitor, the memory manager, and the scheduler (Dispatcher).

Session Manager Subsystem, or smss.exe, is a component of the Microsoft Windows NT family of operating systems, starting in Windows NT 3.1. It is executed during the startup process of those operating systems, at which time it performs the following tasks:

The Microsoft Windows operating system supports a form of shared libraries known as "dynamic-link libraries", which are code libraries that can be used by multiple processes while only one copy is loaded into memory. This article provides an overview of the core libraries that are included with every modern Windows installation, on top of which most Windows applications are built.

Address Windowing Extensions (AWE) is a Microsoft Windows application programming interface that allows a 32-bit software application to access more physical memory than it has virtual address space, even in excess of the 4 GB limit. The process of mapping an application's virtual address space to physical memory under AWE is known as "windowing", and is similar to the overlay concept of other environments. AWE is beneficial to certain data-intensive applications, such as database management systems and scientific and engineering software, that need to manipulate very large data sets while minimizing paging.

A memory-mapped file is a segment of virtual memory that has been assigned a direct byte-for-byte correlation with some portion of a file or file-like resource. This resource is typically a file that is physically present on disk, but can also be a device, shared memory object, or other resource that the operating system can reference through a file descriptor. Once present, this correlation between the file and the memory space permits applications to treat the mapped portion as if it were primary memory.

In computing on Microsoft platforms, WoW64 is a subsystem of the Windows operating system capable of running 32-bit applications on 64-bit Windows. It is included in all 64-bit versions of Windows—including Windows XP Professional x64 Edition, IA-64 and x64 versions of Windows Server 2003, as well as x64 versions of Windows Vista, Windows Server 2008, Windows 7, Windows 8, Windows Server 2012, Windows 8.1, Windows 10, Windows Server 2016 and Windows Server 2019, as well as ARM64 versions of Windows 10 and Windows Server 2022. In Windows Server 2008 R2 Server Core, it is an optional component. It is not available in Windows Nano Server variants. WoW64 aims to take care of many of the differences between 32-bit Windows and 64-bit Windows, particularly involving structural changes to Windows itself.

Architecture of Windows 9x

The Windows 9x series of operating systems refers to the kernel which lies at the heart of Windows 9x. Its architecture is monolithic.

In computing the Process Environment Block is a data structure in the Windows NT operating system family. It is an opaque data structure that is used by the operating system internally, most of whose fields are not intended for use by anything other than the operating system. Microsoft notes, in its MSDN Library documentation — which documents only a few of the fields — that the structure "may be altered in future versions of Windows". The PEB contains data structures that apply across a whole process, including global context, startup parameters, data structures for the program image loader, the program image base address, and synchronization objects used to provide mutual exclusion for process-wide data structures.

Windows Subsystem for Linux Compatibility layer for running Linux binary executables natively on Windows

Windows Subsystem for Linux (WSL) is a compatibility layer for running Linux binary executables natively on Windows 10 and Windows Server 2019.

References