High memory area

Last updated

The high memory area is highlighted. IBM PC Memory areas.svg
The high memory area is highlighted.

In DOS memory management, the high memory area (HMA) is the RAM area consisting of the first 65520 bytes above the one megabyte in an IBM AT or compatible computer.

Contents

In real mode, the segmentation architecture of the Intel 8086 and subsequent processors identifies memory locations with a 16-bit segment and a 16-bit offset, which is resolved into a physical address via (segment) × 16 + (offset). Although intended to address only 1  Megabyte (MB) (220 bytes) of memory, segment:offset addresses at FFFF:0010 and beyond reference memory beyond 1 MB (FFFF0 + 0010 = 100000). So, on an 80286 and subsequent processors, this mode can actually address the first 65520 bytes of extended memory as part of the 64 KB range starting 16 bytes before the 1 MB mark—FFFF:0000 (0xFFFF0) to FFFF:FFFF (0x10FFEF). The Intel 8086 and 8088 processors, with only 1 MB of memory and only 20  address lines, wrapped around at the 20th bit, so that address FFFF:0010 was equivalent to 0000:0000. [1]

To allow running existing DOS programs which relied on this feature to access low memory on their newer IBM PC AT computers, IBM added special circuitry on the motherboard to simulate the wrapping around. This circuit was a simple logic gate which could disconnect the microprocessor's 21st addressing line, A20, from the rest of the motherboard. This gate could be controlled, initially through the keyboard controller, to allow running programs which wanted to access the entire RAM. [1]

So-called A20 handlers could control the addressing mode dynamically, [1] thereby allowing programs to load themselves into the 1024–1088 KB region and run in real mode. [1]

Code suitable to be executed in the HMA must either be coded to be position-independent (using only relative references), [2] [1] be compiled to work at the specific addresses in the HMA (typically allowing only one or at most two pieces of code to share the HMA), or it must be designed to be paragraph boundary or even offset relocatable (with all addresses being fixed up during load). [2] [1]

Before code (or data) in the HMA can be addressed by the CPU, the corresponding driver must ensure that the HMA is mapped in. This requires that any such requests are tunneled through a stub remaining in memory outside the HMA, which would invoke the A20 handler in order to (temporarily) enable the A20 gate. [2] [1] If the driver does not exhibit any public data structures and only uses interrupts or calls already controlled by the underlying operating system, it might be possible to register the driver with the system in a way so that the system will take care of A20 itself thereby eliminating the need for a separate stub. [1] [nb 1]

The first user of the HMA among Microsoft products was Windows/286  2.1 in 1988, which introduced the HIMEM.SYS device driver. Starting in 1990 with Digital Research's DR DOS 5.0 [3] (via HIDOS.SYS /BDOS=FFFF [4] and CONFIG.SYS HIDOS=ON) and since 1991 with MS-DOS 5.0 [3] (via DOS=HIGH), parts of the operating system's BIOS and kernel could be loaded into the HMA as well, [3] [5] freeing up to 46 KB of conventional memory. [1] Other components, such as device drivers and terminate-and-stay-resident programs (TSRs), could at least be loaded into the upper memory area (UMA), but not into the HMA. Under DOS 5.0 and higher, with DOS=HIGH, the system additionally attempted to move the disk buffers into the HMA. [5] Under DR DOS 6.0 (1991) and higher, the disk buffers (via HIBUFFERS , and later also BUFFERSHIGH ), parts of the command processor COMMAND.COM as well as several special self-relocating drivers like KEYB, NLSFUNC and SHARE could load into the HMA as well (using their /MH option), thereby freeing up even more conventional memory and upper memory for conventional DOS software to work with. [1] TASKMAX seems to have relocated parts of itself into the HMA as well. [6] [7] Novell's NLCACHE from NetWare Lite and early versions of NWCACHE from Personal NetWare and Novell DOS 7 could utilize the HMA as well. [8] [9] [7] Under MS-DOS/PC DOS, a ca. 2 KB shared portion of COMMAND.COM can be relocated into the HMA, [10] as well as DISPLAY.SYS bitmaps for prepared codepages. [10] [11] Under MS-DOS 6.2 (1993) and higher, a ca. 5 KB portion of DBLSPACE.BIN/DRVSPACE.BIN can coexist with DOS in the HMA (unless DBLSPACE/DRVSPACE /NOHMA is invoked). [5] [12] Under PC DOS 7.0 (1995) and 2000, DOSKEY loads into the HMA (if available), [13] and SHARE can be loaded into the HMA as well (unless its /NOHMA option is given). [13] Under MS-DOS 7.0 (1995) to 8.0 (2000), parts of the HMA are also used as a scratchpad to hold a growing data structure recording various properties of the loaded real-mode drivers. [7] [14] [15]

See also

Notes

  1. A stub is not necessary for the resident system extensions SHARE and NLSFUNC under DR DOS 6.0 and higher, because they only hook the multiplex interrupt INT 2Fh and can therefore utilize a backdoor interface to hook into the interrupt chain in kernel space so that the kernel's gate A20 handler will provide the functionality of the stub.

Related Research Articles

<span class="mw-page-title-main">Drive letter assignment</span> Alphabetical assignment to logical drives on computers (e.g., C:\)

In computer data storage, drive letter assignment is the process of assigning alphabetical identifiers to volumes. Unlike the concept of UNIX mount points, where volumes are named and located arbitrarily in a single hierarchical namespace, drive letter assignment allows multiple highest-level namespaces. Drive letter assignment is thus a process of using letters to name the roots of the "forest" representing the file system; each volume holds an independent "tree".

<span class="mw-page-title-main">DR-DOS</span> MS-DOS-like operating system

DR-DOS is a disk operating system for IBM PC compatibles. Upon its introduction in 1988, it was the first DOS attempting to be compatible with IBM PC DOS and MS-DOS.

A terminate-and-stay-resident program is a computer program running under DOS that uses a system call to return control to DOS as though it has finished, but remains in computer memory so it can be reactivated later. This technique partially overcame DOS's limitation of executing only one program, or task, at a time. TSRs are used only in DOS, not in Windows.

Real mode, also called real address mode, is an operating mode of all x86-compatible CPUs. The mode gets its name from the fact that addresses in real mode always correspond to real locations in memory. Real mode is characterized by a 20-bit segmented memory address space and unlimited direct software access to all addressable memory, I/O addresses and peripheral hardware. Real mode provides no support for memory protection, multitasking, or code privilege levels.

<span class="mw-page-title-main">A20 line</span> Signal in the system bus of an x86-based computer system

The A20, or address line 20, is one of the electrical lines that make up the system bus of an x86-based computer system. The A20 line in particular is used to transmit the 21st bit on the address bus.

<span class="mw-page-title-main">DOS memory management</span>

In IBM PC compatible computing, DOS memory management refers to software and techniques employed to give applications access to more than 640 kibibytes (KiB) of "conventional memory". The 640 KiB limit was specific to the IBM PC and close compatibles; other machines running MS-DOS had different limits, for example the Apricot PC could have up to 768 KiB and the Sirius Victor 9000, 896 KiB. Memory management on the IBM family was made complex by the need to maintain backward compatibility to the original PC design and real-mode DOS, while allowing computer users to take advantage of large amounts of low-cost memory and new generations of processors. Since DOS has given way to Microsoft Windows and other 32-bit operating systems not restricted by the original arbitrary 640 KiB limit of the IBM PC, managing the memory of a personal computer no longer requires the user to manually manipulate internal settings and parameters of the system.

<span class="mw-page-title-main">Conventional memory</span> First 640K of RAM under DOS

In DOS memory management, conventional memory, also called base memory, is the first 640 kilobytes of the memory on IBM PC or compatible systems. It is the read-write memory directly addressable by the processor for use by the operating system and application programs. As memory prices rapidly declined, this design decision became a limitation in the use of large memory capacities until the introduction of operating systems and processors that made it irrelevant.

In compiler theory, dead-code elimination is a compiler optimization to remove dead code. Removing such code has several benefits: it shrinks program size, an important consideration in some contexts, and it allows the running program to avoid executing irrelevant operations, which reduces its running time. It can also enable further optimizations by simplifying program structure. Dead code includes code that can never be executed, and code that only affects dead variables, that is, irrelevant to the program.

CONFIG.SYS is the primary configuration file for the DOS and OS/2 operating systems. It is a special ASCII text file that contains user-accessible setup or configuration directives evaluated by the operating system's DOS BIOS during boot. CONFIG.SYS was introduced with DOS 2.0.

Quarterdeck Expanded Memory Manager (QEMM) is a memory manager produced by Quarterdeck Office Systems in the late 1980s through the late 1990s. It was the most popular third-party memory manager for the MS-DOS and other DOS operating systems.

AUTOEXEC.BAT is a system file that was originally on DOS-type operating systems. It is a plain-text batch file in the root directory of the boot device. The name of the file is an abbreviation of "automatic execution", which describes its function in automatically executing commands on system startup; the filename was coined in response to the 8.3 filename limitations of the FAT file system family.

<span class="mw-page-title-main">IBMBIO.COM</span> System file (DOS BIOS) in PC DOS and DR-DOS

IBMBIO.COM is a system file in many DOS operating systems. It contains the system initialization code and all built-in device drivers. It also loads the DOS kernel (IBMDOS.COM) and optional pre-loadable system components, displays boot menus, processes configuration files and launches the shell.

<span class="mw-page-title-main">Upper memory area</span> Physical memory region on IBM PC compatibles

In DOS memory management, the upper memory area (UMA) is the memory between the addresses of 640 KB and 1024 KB (0xA0000–0xFFFFF) in an IBM PC or compatible. IBM reserved the uppermost 384 KB of the 8088 CPU's 1024 KB address space for BIOS ROM, Video BIOS, Option ROMs, video RAM, RAM on peripherals, memory-mapped I/O, and obsoleted ROM BASIC.

.sys is a filename extension used in MS-DOS applications and Microsoft Windows operating systems. They are system files that contain device drivers or hardware configurations for the system.

Computer hardware or software is said to be bug compatible if it exactly replicates even an undesirable feature of a previous version. The phrase is found in the Jargon File.

<span class="mw-page-title-main">MSCDEX</span>

MSCDEX or Microsoft CD-ROM Extensions is a software program produced by Microsoft and included with MS-DOS 6.x and certain versions of Windows to provide CD-ROM support. Earlier versions of MSCDEX since 1986 were installable add-ons for MS-DOS 3.1 and higher.

In computing, LOADHIGH is an internal DOS command in COMMAND.COM that is used to load a program into the upper memory area (UMA) instead of conventional memory.

In computer programming, a self-relocating program is a program that relocates its own address-dependent instructions and data when run, and is therefore capable of being loaded into memory at any address. In many cases, self-relocating code is also a form of self-modifying code.

<span class="mw-page-title-main">Architecture of Windows 9x</span>

The Windows 9x series of operating systems refers to the kernel which lies at the heart of Windows 9x. Its architecture is monolithic.

DOS Protected Mode Services (DPMS) is a set of extended DOS memory management services to allow DPMS-enabled DOS drivers to load and execute in extended memory and protected mode.

References

  1. 1 2 3 4 5 6 7 8 9 10 Paul, Matthias R. (2002-02-02). "Treiber dynamisch nachladen (Intra-Segment-Offset-Relokation zum Laden von TSRs in die HMA)" [Loading drivers dynamically (Intra-segment offset relocation to load TSRs into the HMA)] (in German). Newsgroup:  de.comp.os.msdos. Archived from the original on 2017-09-09. Retrieved 2017-07-02. (NB. Gives a comprehensive overview on the history and "nature" of the HMA and the non-obvious design constraints to be observed when developing resident system extensions to be loaded into the HMA. It also describes how to address these issues using stubs, backdoors, and intra-segment offset relocation, a method used by DR-DOS drivers capable of relocating into the HMA and similar to a (more sophisticated) method used as the basis for the dynamic dead code elimination in the author's FreeKEYB driver.)
  2. 1 2 3 Ingenoso, Tony (1998-12-20). "Chapter 13 - The A20 gate and the HMA". Making Code Work Better - How to minimize the size of 80x86 code and sometimes make it faster (e-book). Archived from the original on 2019-11-18. Retrieved 2019-11-18.
  3. 1 2 3 Dryfoos, Mike, ed. (1991-09-18) [1991-07-19]. "MS-DOS 5.0 Development Post-Mortem Report" (PDF) (mail as court document). Microsoft. p. 10. MS-PCA1179169 (MS-PCA1179159-MS-PCA1179191). MS7020988 (MS7020978-MS7021010). Depo. Ex. 1109. Comes v Microsoft Plaintiff's Exhibit 3473. CA.No.2:96CV645B Plaintiff's Exhibit 477. Archived (PDF) from the original on 2019-04-02. Retrieved 2019-07-22. […] One of the most important stimulanta for adding features was competitive pressure from DRDOS 5.0, which we first learnt of in the spring of 1990. The DRDOS feature set led us to add UMB support, task swapping, and Undelete. […] Considerable amounts of the team's management attention was diverted to new features such as file transfer software, undelete and network installation […] Eventually this situation reached a crisis point at the end of July 1990, and, led by BradS, the team's management spent an arduous series of meetings nailing down a schedule and process for closing the project down […] (1+32 pages)
  4. Banta, K.; Partridge, D. (1994-08-18). "Third Party Memory Managers". DR DOS 6.0 (Technical information document). Revision A. Novell. TID800074 (replaces FYI-M-1303). Archived from the original on 2021-12-15. Retrieved 2021-12-15.
  5. 1 2 3 Schulman, Andrew; Brown, Ralf D.; Maxey, David; Michels, Raymond J.; Kyle, Jim (1994) [November 1993]. Williams, Andrew (ed.). Undocumented DOS: A programmer's guide to reserved MS-DOS functions and data structures - expanded to include MS-DOS 6, Novell DOS and Windows 3.1 . The Andrew Schulman Programming Series (1st printing, 2nd ed.). Reading, Massachusetts, USA: Addison Wesley Publishing Company. pp. 42, 349–350, 437–438. ISBN   0-201-63287-X. ISBN   978-0-201-63287-3. (xviii+856+vi pages, 3.5"-floppy ) Errata:
  6. "Format of HMA Memory Block (DR DOS 6.0 kernel loaded in HMA)". RBIL . 2000. Archived from the original on 2020-02-18. Retrieved 2020-02-18.
  7. 1 2 3 Paul, Matthias R. (2002-04-10). "[fd-dev] HMA access from TSR". freedos-dev. Archived from the original on 2017-09-09. Retrieved 2017-09-09. […] MS-DOS 7.0+ adds INT 21h/AX=4A03h and INT 21h/AX=4A04h. RBIL61 INT 21h/AH=52h has some info on the MS-DOS 7.0+ HMA MCB chain […] HMA relocation for TSRs makes much sense for DR-DOS: Although you can load large parts of the BIOS and BDOS, the resident part of the shell, the BUFFERS, and DR-DOS TSRs like SHARE, KEYB, and NLSFUNC (and in some issues parts of TASKMGR and NWCACHE) into the HMA, there is usually still free space available, typically around 10 Kb (up to ca. 20 Kb when you use a 3rd party shell). It also makes sense for MS-DOS 5.0 - 6.22 and PC DOS up to 2000, which typically leave 4 - 7 Kb of the HMA memory unused (SHARE, KEYB, and NLSFUNC cannot load into the HMA, but DBLSPACE and HIMEM can to some extent). Available HMA space can be rather tight with MS-DOS 7.0+, since this issue introduced a new and for the most part undocumented RMD data structure usually located in the HMA. The kernel collects and records configuration and Real Mode Driver data during boot (type of driver, interrupts hooked by driver, CONFIG.SYS line of invocation, etc.) and stores this information in an […] complicated […] and […] growing data structure. Presumably this info is meant to be used by the Windows core to get a better picture of the loaded Real Mode drivers instead of treating DOS as a monolithic block, or even […] attempt to unhook or unload some of them, however, it is only used to a very limited extent (for example you can see some of the info reflected in the log files created on Windows 9x startup, and some parts of the Windows configuration manager also make use of it), leaving room for speculation much beyond the technical side - in particular because nothing of the interesting stuff is documented… […]
  8. Paul, Matthias R. (1997-07-30) [1994-05-01]. "II.4. Undokumentierte Eigenschaften externer Kommandos". NWDOS-TIPs Tips & Tricks rund um Novell DOS 7, mit Blick auf undokumentierte Details, Bugs und Workarounds. MPDOSTIP (in German) (3 ed.). Archived from the original on 2016-11-05. Retrieved 2012-01-11. (NB. NWDOSTIP.TXT is a comprehensive work on Novell DOS 7 and OpenDOS 7.01, including the description of many undocumented features and internals. It is part of the author's yet larger MPDOSTIP.ZIP collection maintained up to 2001 and distributed on many sites at the time. The provided link points to a HTML-converted older version of the file.)
  9. Paul, Matthias R. (2001-04-09). "II.4. Undokumentierte Eigenschaften externer Kommandos". NWDOS-TIPs Tips & Tricks rund um Novell DOS 7, mit Blick auf undokumentierte Details, Bugs und Workarounds. MPDOSTIP (in German) (3 ed.).
  10. 1 2 Chappell, Geoff (January 1994). Schulman, Andrew; Pedersen, Amorette (eds.). DOS Internals. The Andrew Schulman Programming Series (1st printing, 1st ed.). Addison Wesley Publishing Company. pp. 4, 21, 100–106, 127–129. ISBN   978-0-201-60835-9. ISBN   0-201-60835-9. (xxvi+738+iv pages, 3.5"-floppy ) Errata:
  11. Paul, Matthias R. (2002-12-04). "[fd-dev] DISPLAY CON?". freedos-dev. Archived from the original on 2021-12-08. […] some issues of DISPLAY.SYS (of PC DOS 7/2000, for example) store the currently unused fonts in XMS memory. Some earlier issues of MS-DOS/PC DOS DISPLAY.SYS seem to have had a facility to store them in the HMA […]
  12. Cooper, Jim (2002). Using MS-DOS 6.22 (special 3rd ed.). Que Publishing. p. 669. ISBN   0-78972573-8. ISBN   978-0-78972573-8. Archived from the original on 2020-02-18. Retrieved 2020-02-18.
  13. 1 2 Brooks, Vernon C. (2014). "This is a detailed list of the changes I made in PC DOS 7.0". PC DOS Retro. Archived from the original on 2020-02-18. Retrieved 2020-02-18. […] DOSKEY.COM […] Move code to HMA if available. […] SHARE.EXE […] Move code to HMA if available and added /NOHMA option force loading low. […]
  14. Sweger, Kristofer (2007) [2002-07-15]. "ANSIPLUS and Windows". Archived from the original on 2021-11-28. Retrieved 2021-11-28. […] ANSIPLUS's code cannot be loaded to the HMA under MS-DOS 7 (Windows 9x only) because there apparently is not enough unused HMA memory available. […]
  15. Paul, Matthias R. (2002-08-13). "Suche freien Speicherbereich unterhalb von 1 MB, der nicht von OS überschrieben wird" (in German). Newsgroup:  de.comp.lang.assembler.x86. Archived from the original on 2017-09-04. Retrieved 2017-09-03.
  16. Paul, Matthias R. (1997-10-02) [1997-09-29]. "Caldera OpenDOS 7.01/7.02 Update Alpha 3 IBMBIO.COM - README.TXT and BOOT.TXT - A short description of how OpenDOS is booted". Archived from the original on 2003-10-04. Retrieved 2009-03-29.

Further reading