Physical Address Extension

In computing, Physical Address Extension (PAE), sometimes referred to as Page Address Extension, ^[1] is a memory management feature for the x86 architecture. PAE was first introduced by Intel in the Pentium Pro, and later by AMD in the Athlon processor. ^[2] It defines a page table hierarchy of three levels (instead of two), with table entries of 64 bits each instead of 32, allowing these CPUs to directly access a physical address space larger than 4  gigabytes (2³² bytes).

The page table structure used by x86-64 CPUs when operating in long mode further extends the page table hierarchy to four or more levels, extending the virtual address space, and uses additional physical address bits at all levels of the page table, extending the physical address space. It also uses the topmost bit of the 64-bit page table entry as a no-execute or "NX" bit, indicating that code cannot be executed from the associated page. The NX feature is also available in protected mode when these CPUs are running a 32-bit operating system, provided that the operating system enables PAE.

History

PAE was first implemented in the Intel Pentium Pro in 1995, ^[3] although the accompanying chipsets usually lacked support for the required extra address bits. ^[4]

PAE is supported by the Pentium Pro, Pentium II, Pentium III, and Pentium 4 processors. The first Pentium M family processors ("Banias") introduced in 2003 also support PAE; however, they do not show the PAE support flag in their CPUID information. ^[5] This was remedied in a later revision of the "Dothan" core in 2005. It was also available on AMD processors including the AMD Athlon ^{[6] [7]} (although the chipsets are limited to 32-bit addressing ^[8] ) and later AMD processor models.

When AMD defined their 64-bit extension of the industry standard x86 architecture, AMD64 or x86-64, they also enhanced the paging system in "long mode" based on PAE. ^[9] It supports 64-bit virtual addresses ^{[10] : 24} (as of July 2023, ^[update] 48 bits are implemented on some processors and 57 bits are implemented on others ^{[10] : 139, 141–143  [11]} ), 52-bit physical addresses, ^{[10] : 24} and includes NX bit functionality. When the x86-64 processor is initialized, the PAE feature is required to be enabled before the processor is switched from Legacy Mode to Long Mode. ^[9]

Design

With PAE, the page table entry of the x86 architecture is enlarged from 32 to 64 bits. This allows more room for the physical page address, or "page frame number" field, in the page table entry. In the initial implementations of PAE the page frame number field was expanded from 20 to 24 bits. The size of the "byte offset" from the address being translated is still 12 bits, so total physical address size increases from 32 bits to 36 bits (i.e. from 20+12 to 24+12). This increased the physical memory that is theoretically addressable by the CPU from 4 GB to 64 GB.

In the first processors that supported PAE, support for larger physical addresses is evident in their package pinout, with address pin designations going up to A35 instead of stopping at A31. ^[12] Later processor families use interconnects such as Hypertransport or QuickPath Interconnect, which lack dedicated memory address signals, so this relationship is less apparent.

The 32-bit size of the virtual address is not changed, so regular application software continues to use instructions with 32-bit addresses and (in a flat memory model) is limited to 4 gigabytes of virtual address space. Operating systems supporting this mode use page tables to map the regular 4 GB virtual address space into the physical memory, which, depending on the operating system and the rest of the hardware platform, may be as big as 64 GB. The mapping is typically applied separately for each process, so that the additional RAM is useful even though no single process can access it all simultaneously.

Later work associated with AMD's development of x86-64 architecture expanded the theoretical possible size of physical addresses to 52 bits. ^{[10] : 24}

Page table structures

32-bit paging, 4 KiB pages, without PAE

No PAE, 4 KB pages

In protected mode with paging enabled (bit 31, PG, of control register CR0 is set), but without PAE, x86 processors use a two-level page translation scheme. Control register CR3 holds the page-aligned physical address of a single 4 KB long page directory. This is divided into 1024 four-byte page directory entries that in turn, if valid, hold the page-aligned physical addresses of page tables, each 4 KB in size. These similarly consist of 1024 four-byte page table entries which, if valid, hold the page-aligned physical addresses of 4 KB long pages of physical memory (RAM).

32-bit paging, 4 MiB pages, without PAE

No PAE, 4 MB pages

The entries in the page directory have an additional flag in bit 7, named PS (for page size). If the system has set this bit to 1, the page directory entry does not point to a page table but to a single, large 4 MB page (Page Size Extension).

32-bit paging, 4 KiB pages, with PAE

With PAE; 4 KB pages

Enabling PAE (by setting bit 5, PAE, of the system register CR4) causes major changes to this scheme. By default, the size of each page remains as 4 KB. Each entry in the page table and page directory becomes 64 bits long (8 bytes), instead of 32 bits, to allow for additional address bits. However, the size of each table does not change, so both table and directory now have only 512 entries. Because this allows only one half of the entries of the original scheme, an extra level of hierarchy has been added, so the system register CR3 now points physically to a Page Directory Pointer Table, a short table containing four pointers to page directories.

Supporting 64 bit addresses in the page-table is a significant change as this enables two changes to the processor addressing. Firstly, the page table walker, which uses physical addresses to access the page table and directory, can now access physical addresses greater than the 32-bit physical addresses supported in systems without PAE. From CR3, the page table walker can access page directories and tables that are beyond the 32-bit range. Secondly, the physical address for the data being accessed (stored in the page table) can be represented as a physical address larger than the 32-bit addresses supported in a system without PAE. Again, this allows data accesses to access physical memory regions beyond the 32-bit range. ^[13]

32-bit paging, 2 MiB pages, with PAE

With PAE; 2 MB pages

The entries in the page directory have an additional flag in bit 7, named PS (for page size). If the system has set this bit to 1, the page directory entry does not point to a page table but to a single, large 2 MB page (Page Size Extension).

Summary of 32-bit paging

In all page table formats supported by IA-32 and x86-64, the 12 least significant bits of the page table entry are either interpreted by the memory management unit or are reserved for operating system use. In processors that implement the "no-execute" or "execution disable" feature, the most significant bit (bit 63) is the NX bit. The next eleven most significant bits (bits 52 through 62) are reserved for operating system use by both Intel and AMD's architecture specifications. Thus, from 64 bits in the page table entry, 12 low-order and 12 high-order bits have other uses, leaving 40 bits (bits 12 though 51) for the physical page number. Combined with 12 bits of "offset within page" from the linear address, a maximum of 52 bits are available to address physical memory. This allows a maximum RAM configuration of 2⁵² bytes, or 4 petabytes (about 4.5×10¹⁵ bytes).

x86-64 paging

On x86-64 processors in native long mode, the address translation scheme uses PAE but adds a fourth table, the 512-entry page-map level 4 table, and extends the page directory pointer table to 512 entries instead of the original 4 entries it has in protected mode. This means that 48 bits of virtual page number are translated, giving a virtual address space of up to 256 TB. For some processors, a mode can be enabled with a fifth table, the 512-entry page-map level 5 table; this means that 57 bits of virtual page number are translated, giving a virtual address space of up to 128 PB. ^{[10] : 141–153} In the page table entries, in the original specification, 40 bits of physical page number are implemented.

Hardware support

See also: CPUID

Software can identify via the CPUID flag PAE whether a CPU supports PAE mode or not. A free-of-charge program for Microsoft Windows is available which will list many processor capabilities, including PAE support. ^[14] In Linux, commands such as cat /proc/cpuinfo can list the pae flag when present, ^[15] as well as other tools such as the SYSLINUX Hardware Detection Tool.

To run the processor in PAE mode, operating system support is required. To use PAE to access more than 4 GB of RAM, further support is required in the operating system, in the chipset, and on the motherboard. Some chipsets do not support physical memory addresses above 4 GB (FFFFFFFF in hexadecimal), and some motherboards simply do not have enough RAM sockets to allow the installation of more than 4 GB of RAM. Nevertheless, even if no more than 4 GB of RAM is available and accessible, a PAE-capable CPU may be run in PAE mode, for example to allow use of the No execute feature.

Operating system support

Microsoft Windows

32-bit versions of Microsoft Windows support PAE if booted with the appropriate option. According to Microsoft Technical Fellow Mark Russinovich, some drivers were found to be unstable when encountering physical addresses above 4GB. ^[16]

The following table shows the memory limits for 32-bit versions of Microsoft Windows:

Memory limits on 32-bit editions of Microsoft Windows,
with PAE support ^{[17] [18] [19]}

Windows version	Memory limit
Windows 2000 Professional, Server	4 GB
Windows 2000 Advanced Server	8 GB
Windows 2000 Datacenter	32 GB
Windows XP Starter	0.5 GB
Windows XP (other editions)	4 GB
Windows Server 2003 Web SP2	2 GB
Windows Server 2003 Standard SP2	4 GB
Windows Server 2003 Enterprise/Datacenter SP2	64 GB
Windows Storage Server 2003 Enterprise	8 GB
Windows Storage Server 2003 (other editions)	4 GB
Windows Home Server	4 GB
Windows Vista Starter	1 GB
Windows Vista (other editions)	4 GB
Windows Server 2008 Standard, Web	4 GB
Windows Server 2008 Enterprise, Datacenter	64 GB
Windows 7 Starter	2 GB
Windows 7 (other editions)	4 GB
Windows 8 (all editions)	4 GB
Windows 10 (all editions)	4 GB

The original releases of Windows XP and Windows XP SP1 used PAE mode to allow RAM to extend beyond the 4 GB address limit. However, it led to compatibility problems with 3rd party drivers which led Microsoft to remove this capability in Windows XP Service Pack 2. Windows XP SP2 and later, by default, on processors with the no-execute (NX) or execute-disable (XD) feature, runs in PAE mode in order to allow NX. ^[20] The NX bit resides in bit 63 of the page table entry and, without PAE, page table entries on 32-bit systems have only 32 bits; therefore PAE mode is required in order to exploit the NX feature. However, "client" versions of 32-bit Windows (Windows XP SP2 and later, Windows Vista, Windows 7) limit physical address space to the first 4 GB for driver compatibility ^[16] even though these versions do run in PAE mode if NX support is enabled.

Windows 8 and later releases will only run on processors which support PAE, in addition to NX and SSE2. ^{[21] [22]}

macOS

Mac OS X Tiger through Mac OS X Snow Leopard support PAE and the NX bit on IA-32 processors; Snow Leopard was the last version to support IA-32 processors. On x86-64 processors, all versions of macOS use 4-level paging (IA-32e paging rather than PAE) to address memory above 4GB. Mac Pro and Xserve systems can use up to 64 GB of RAM. ^[23]

Linux

See also: Executable space protection § Linux

The Linux kernel includes full PAE-mode support starting with version 2.3.23, ^[24] in 1999 enabling access of up to 64 GB of memory on 32-bit machines. A PAE-enabled Linux kernel requires that the CPU also support PAE. The Linux kernel supports PAE as a build option and major distributions provide a PAE kernel either as the default or as an option.

The NX bit feature requires a kernel built with PAE support. ^[25]

Linux distributions now commonly use a PAE-enabled kernel as the default, a trend that began in 2009. ^[26] As of 2012 ^[update] many, including Ubuntu (and derivatives like Xubuntu and Linux Mint), ^{[27] [28]} Red Hat Enterprise Linux 6.0, ^[29] and CentOS, have stopped distributing non-PAE kernels, thus making PAE-supporting hardware mandatory. Linux distributions that require PAE may refuse to boot on Pentium M family processors because they do not show the PAE support flag in their CPUID information (even though it is supported internally). ^[5] However, this can be easily bypassed with the forcepae option. ^[30]

Distributions that still provide a non-PAE option, including Debian (and derivatives like LMDE 2 (Linux Mint Debian Edition) ^[31] ), Slackware, and LXLE, typically do so with "i386", "i486", or "retro" labels. ^{[32] [33]} The article Lightweight Linux distribution does list some others, allowing to install Linux onto old computers.

Others

FreeBSD and NetBSD also support PAE as a kernel build option. FreeBSD supports PAE in the 4.x series starting with 4.9, in the 5.x series starting with 5.1, and in all 6.x and later releases. Support requires the kernel PAE configuration-option. Loadable kernel modules can only be loaded into a kernel with PAE enabled if the modules were built with PAE enabled; the binary modules in FreeBSD distributions are not built with PAE enabled, and thus cannot be loaded into PAE kernels. Not all drivers support more than 4 GB of physical memory; those drivers won't work correctly on a system with PAE. ^[34]

OpenBSD has had support for PAE since 2006 with the standard GENERIC i386 kernel. GeNUA mbH supported the initial implementation. ^[35] Since release 5.0 PAE has had a series of changes, in particular changes to i386 MMU processing for PMAP, see pmap(9). ^{[36] [ failed verification ]}

Solaris supports PAE beginning with Solaris version 7. However, third-party drivers used with version 7 which do not specifically include PAE support may operate erratically or fail outright on a system with PAE. ^[37]

Haiku added initial support for PAE sometime after the R1 Alpha 2 release. With the release of R1 Alpha 3 PAE is now officially supported.

ArcaOS has limited support for PAE for the purpose of creating RAM disks above the 4 GB boundary. ^[38]

See also

• RAM limit
• Page Size Extension
• PCI hole
• PSE-36
• Architecture of Windows NT
• 3 GB barrier
• Large Physical Address Extension (LPAE) – in the ARM architecture
• Intel 5-level paging

References

1. Dual-Core Intel® Xeon® Processor 2.80 GHz Specification Update (PDF). Intel Corporation. October 2006. p. 18.
2. "Appendix E". AMD Athlon™ Processor x86 Code Optimization Guide (PDF) (Revision K ed.). AMD, Inc. February 2002. p. 250. Retrieved 2017-04-13. A 2-bit index consisting of PCD and PWT bits of the page table entry is used to select one of four PAT register fields when PAE (page address extensions) is enabled, or when the PDE doesn't describe a large page.
3. T. Shanley (1998). Pentium Pro and Pentium II System Architecture. Addison-Wesley Professional. p. 439. ISBN   978-0-201-30973-7.
4. "Operating Systems and PAE Support". Hardware Developers Center. 1 June 2017. Retrieved 11 July 2023.
5. "PAE - Ubuntu Community Help Wiki".
6. "Appendix E". AMD Athlon™ Processor x86 Code Optimization Guide (PDF) (Revision K ed.). AMD, Inc. February 2002. p. 250. Retrieved 2017-04-13. A 2-bit index consisting of PCD and PWT bits of the page table entry is used to select one of four PAT register fields when PAE (page address extensions) is enabled, or when the PDE doesn't describe a large page.
7. "AMD Athlon 500 - AMD-K7500MTR51B C". Cpu-world.com. 26 March 2014. Retrieved 11 July 2023.
8. "AMD-762 System Controller" (PDF). p. 2. Supports up to 4 Gbytes of memory
9. AMD Corporation (June 2023). "Volume 2: System Programming" (PDF). AMD64 Architecture Programmer's Manual. AMD Corporation. p. 139. Retrieved 2023-07-11. Long-mode page translation requires the use of physical-address extensions (PAE). Before activating long mode, PAE must be enabled by setting CR4.PAE to 1. Activating long mode before enabling PAE causes a general-protection exception (#GP) to occur.
10. AMD Corporation (June 2023). "Volume 2: System Programming" (PDF). AMD64 Architecture Programmer's Manual. AMD Corporation. Retrieved 2023-07-11.
11. "Volume 3 (3A, 3B, 3C & 3D): System Programming Guide". Intel 64 and IA-32 Architectures Software Developer’s Manual. Intel. June 2023. p. 4-7. Retrieved 2023-07-11.
12. Pentium® III Xeon™ Processor at 500 and 550 MHz Datasheet. Intel Corporation. February 2000. p. 86. 245094-002. A[35:03]# (I/O): The A[35:3]# (Address) signals define a 2-to-the-36-byte physical memory address space.
13. "4.4 Paging". Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3A. Intel . Retrieved 28 October 2023.
14. "Coreinfo - Sysinternals". Windows Sysinternals. Microsoft. 7 June 2023. Retrieved 11 July 2023.
15. "Detecting your Hardware". Gentoo. October 8, 2008. Archived from the original on 2013-05-03. Retrieved 2013-04-28.
16. Mark Russinovich (2008-07-21). "Pushing the Limits of Windows: Physical Memory". Archived from the original on 2008-07-25. Retrieved 2010-07-11.
17. "Memory Limits for Windows releases". MSDN . Microsoft. December 5, 2007. Archived from the original on December 17, 2007. Retrieved 2015-11-16.
18. "Intel Physical Addressing Extensions (PAE) in Windows 2000". Support. Microsoft. October 26, 2007. Archived from the original on 2008-01-01. Retrieved 2007-12-29.
19. "Overview of Windows Server 2003 R2 Datacenter Edition". TechNet . Microsoft. Archived from the original on 2011-11-28. Retrieved 2009-05-15.
20. "The RAM reported by the System Properties dialog box and the System Information tool is less than you expect in Windows Vista or in Windows XP Service Pack 2 or later version (MSKB 888137)". Knowledge Base. Microsoft. Archived from the original on 2009-02-04. Retrieved 2009-01-30.
21. Khurshid, Usman (2 November 2012). "How To Check If Your Processor Supports PAE, NX And SSE2 For Windows 8 Installation". technize.net. Technize. Retrieved 20 April 2014.
22. "PAE/NX/SSE2 Support Requirement Guide for Windows 8". Microsoft Docs. 10 February 2014. Retrieved 11 July 2023.
23. "Road to Mac OS X 10.6 Snow Leopard: 64-Bits". 2008-09-26. Retrieved 2008-09-26.
24. Molnar, Ingo (20 October 1999). "2.3.23-pre4 x86 64 GB RAM changes [HIGHMEM patch] explained a bit". linux-kernel (Mailing list).
25. Mauerer, Wolfgang (11 March 2010). Professional Linux Kernel Architecture. Figure 3.16 Code flow for paging_init. ISBN   978-1-118-07991-1. Execute Disable Protection is also enabled if supported by processor and if the kernel was compiled with PAE support; unfortunately, the feature is otherwise not available.
26. "x86 Specifics for Fedora 11". Archived from the original on 2010-07-04.
27. "Xubuntu 12.04 released". Xubuntu.org. April 26, 2012. Retrieved 24 October 2015. The non-PAE kernel will not be available in future Xubuntu releases.
28. "PAE". Ubuntu Community Help Wiki. Retrieved 2023-07-11.
29. "RHEL 6 Release Notes, 12.6. General Kernel Updates 12.6.1. Physical Address Extension (PAE)". RedHat. Retrieved 27 November 2013.
30. "The kernel's command-line parameters". The Linux Kernel documentation.
31. "Known problems in Linux Mint Debian". Archived from the original on 2015-10-16. Retrieved 2015-10-24. To guarantee compatibility with non-PAE processors, the 32-bit versions of Linux Mint Debian come with a 486 kernel by default.
32. "Precise Puppy". puppylinux.org. PuppyLinux. Archived from the original on 2014-08-13. Retrieved 20 April 2014.
33. "2.1. Supported Hardware". Debian GNU/Linux Installation Guide. SPI. Archived from the original on 2014-05-13. Retrieved 20 April 2014.
34. "FreeBSD i386 5.5-RELEASE PAE(4) man page". 2003-04-08. Retrieved 2023-07-11.
35. "PAE for OpenBSD/i386 by Michael Shalayeff, NYC". 2006. Retrieved 2018-02-03.
36. pmap(9)  –  OpenBSD Kernel Developer's Manual
37. "Added Support for Physical Address Extension (PAE) Mode". Solaris 7 5/99 Release Notes (Intel Platform Edition), Appendix B: Hardware Compatibility List and Device Configuration Guide (Intel Platform Edition) 5/99. 1999. Retrieved 2018-03-23.
38. "ArcaOS 5.0 from Arca Noae is the new release of OS/2 for the 21st Century". 2017. Retrieved 2019-12-16.

Further reading

• "Intel 64 and IA-32 Architectures Software Developer's Manual, Volume 3A: System Programming Guide, Part 1". Intel. 11 February 2014.
• "Physical Address Extension". Microsoft Docs . 7 January 2021. Retrieved 2023-07-11.