Code page 949 (IBM)

Last updated
IBM code page 949
IBM Korea KS PC-Data encoding (IBM-949).svg
Layout of the IBM-949 code page
Alias(es)
  • IBM-949, x-IBM949
  • ASCII-based: IBM-949C, x-IBM949C, cp949c
  • Ambiguous with UHC: 949, cp949
Language(s) Korean
Created by IBM
Classification Extended ISO 646, variable-width encoding, CJK encoding
Extends EUC-KR
Preceded by Code page 944

IBM code page 949 (IBM-949) is a character encoding which has been used by IBM to represent Korean language text on computers. It is a variable-width encoding which represents the characters from the Wansung code defined by the South Korean standard KS X 1001 in a format compatible with EUC-KR, but adds IBM extensions for additional hanja, additional precomposed Hangul syllables, and user-defined characters.

Contents

Giving values in hexadecimal, bytes 0x00 through 0x7F are used for single byte KS X 1003 (ISO 646:KR) characters, a similar set to ASCII but with a won sign rather than a backslash. Bytes 0x80 through 0x84 are used for IBM single byte extension characters. Lead bytes 0x8F through 0xA0 are used for IBM double byte extension characters. Lead bytes 0xA1 through 0xFE are used for Wansung code (KS X 1001 characters in EUC-KR form, double byte), but with some unused space opened up for user-defined use.

Although both are sometimes named "cp949", IBM-949 is different from Windows code page 949 (IBM-1363), which is Microsoft's Unified Hangul Code, a different extension of EUC-KR. It should also not be confused with IBM's implementation of plain EUC-KR (IBM-970). Code page 949 in OS/2 is the IBM code page; however, a third-party patch exists to change this. [1]

Terminology and encoding labelling

Both IBM-949 and Unified Hangul Code (Windows-949) are known as "code page 949" (or "cp949") although they share only the EUC-KR subset in common. Neither has a standardised IANA-registered label to identify it. Although UHC is included in the WHATWG Encoding Standard, [2] with labels including "windows-949", [3] IBM-949 is not. IBM-949 therefore is not permitted in HTML5.

Although the meaning of the label "ibm-949" (and conversely "windows-949" and "ms949") is unambiguous where these labels are supported, the interpretation of the encoding labels "949" and "cp949" consequently varies between implementations. For example, International Components for Unicode uses "cp949", "949", "ibm-949" and "x-IBM949" to refer to IBM-949, [4] and additionally the labels "cp949c", "ibm-949c" and "x-IBM949C" to refer to an variant which uses unmodified ASCII mappings for 0x20–7E (resulting in duplicate mappings for the backslash), [5] while (of the labels incorporating the code page number 949) only "ms949" and "windows-949" are assigned to UHC. [6] This is in contrast to Python, which recognises both "cp949" and "949" (in addition to the more explicit "ms949" and "uhc", but not "windows-949") as labels for UHC, and does not include an IBM-949 codec. [7] The code page 949 used by Korean language versions of OS/2 is the IBM code page; to add support for the entire Unicode set of Korean syllables, a third-party patch exists to replace it with the Microsoft code page. [1]

IBM-949 is a variable width encoding defined as the combination of two fixed-width code pages, the single-byte Code page 1088 and the double-byte Code page 951. [8] [9] [10]

History

A version of Code page 951 (a DBCS-PC, i.e. double-byte non-EUC non-EBCDIC, code), the double-byte component for IBM-949, is defined in the September 1992 revision of IBM Corporate Specification C-H 3-3220-125, along with Code page 834 (a DBCS-Host, i.e. double-byte EBCDIC, code), which is the double byte component of Code page 933. [11] This version of Code page 949/951 considered the entire lead byte range 0x8F–A0 to be a user-defined region, and included only standard Wansung assignments and user-defined areas, thus not including some characters which Code page 933/834 included. [11] Some later versions, such as that implemented by International Components for Unicode (ICU), shrink the user-defined region to include these characters as extensions. [12]

IBM code pages 934 and 944
Language(s) Korean
Extends N-byte Hangul Code
Transforms / Encodes Code page 933
Succeeded byIBM code page 949

The earlier October 1989 revision of C-H 3-3220-125 had instead defined Code page 926 as its DBCS-PC code, which encoded the same characters as IBM-834 in a layout differing from both IBM-951 and IBM-834, which had a different lead byte range and was not an EUC-KR extension. [11] IBM-926 was combined with Code page 891 or Code page 1040 (respectively 8-bit N-byte Hangul Code and an extension thereof; compare how Shift JIS extends 8-bit JIS X 0201) to form IBM-934 or IBM-944 respectively. [13] [14]

Code page 944/926 are now deprecated in favour of IBM-949. The 1992 revision designates code page 926 as "restricted" ("limited to the particular environment for which [it is] registered") and does not give its chart or mappings from the other code pages, [11] and CCSID 944 is categorised as "coexistence and migration" [14] (contrast "interoperable" for CCSID 949). [8] International Components for Unicode includes Unicode mappings for IBM-949 [4] [12] and IBM-933, but its IBM-944 mapping was removed in 2001. [15]

Single byte codes

IBM code page 949 (single byte component: 1088) [16] [17] [4] [5] [12]
0123456789ABCDEF
0x NUL
1x
2x  SP   ! " # $ % & ' ( ) * + , - . /
3x 0 1 2 3 4 5 6 7 8 9 : ; < = > ?
4x @ A B C D E F G H I J K L M N O
5x P Q R S T U V W X Y Z [ ] ^ _
6x ` a b c d e f g h i j k l m n o
7x p q r s t u v w x y z { | } ~
8x ¢ ¬ \ ¦ 8F
9x 90 91 92 93 94 95 96 97 98 99 9A 9B 9C 9D 9E 9F
Ax A0 1-_ 2-_ 3-_ 4-_ 5-_ 6-_ 7-_ 8-_ 9-_ 10-_ 11-_ 12-_ 13-_14-_15-_
Bx 16-_ 17-_ 18-_ 19-_ 20-_ 21-_ 22-_ 23-_ 24-_ 25-_ 26-_ 27-_ 28-_ 29-_ 30-_ 31-_
Cx 32-_ 33-_ 34-_ 35-_ 36-_ 37-_ 38-_ 39-_ 40-_ 41-_ 42-_ 43-_ 44-_ 45-_ 46-_ 47-_
Dx 48-_ 49-_ 50-_ 51-_ 52-_ 53-_ 54-_ 55-_ 56-_ 57-_ 58-_ 59-_ 60-_ 61-_ 62-_ 63-_
Ex 64-_ 65-_ 66-_ 67-_ 68-_ 69-_ 70-_ 71-_ 72-_ 73-_ 74-_ 75-_ 76-_ 77-_ 78-_ 79-_
Fx 80-_ 81-_ 82-_ 83-_ 84-_ 85-_ 86-_ 87-_ 88-_ 89-_ 90-_ 91-_ 92-_ 93-_ 94-_
  Differences from code page 437 (for 0x00–7F) or EUC-KR (for 0x80–FF)

Double byte codes

Lead bytes 0x8F–99, 0xC9, 0xFE (user-defined ranges)

IBM-949 is designed to support a maximum of 1880 UDC (user-defined characters), [8] including the user-defined rows (lead bytes 0xC9 and 0xFE) of the Wansung plane, and ranges outside the Wansung plane. In this version, the lead bytes 0x8F–A0 contain a maximum of 1692 UDC, and lead bytes 0xC9 and 0xFE contain a maximum of 94 each (i.e. with trail bytes 0xA1–FE). [11] However, when the extensions to support the entire double-byte repertoire of IBM-933 are implemented, they use lead bytes 0x9A–A0, resulting in a smaller maximum number of characters left for user definition. [4] [12]

When mapped to Unicode, 0xC9A1–C9FE (between the syllable and hanja ranges) are mapped to the Unicode Private Use Area code points U+E000–E05D, while 0xFEA1–FEFE (between the end of the hanja range and the end of the plane) are mapped to U+E05E–E0BB. Outside the Wansung plane, 0x8FA0–9AA5 (where the second byte is in the range 0xA1–FE) are mapped to the Private Use Area code points U+E0BC–E4CA. [4] The last of these ranges cuts into the start of the 0x9A row (shown below).

Collectively these private use ranges cover the code points U+E000–E4CA, allowing 1227 UDC to be mapped from IBM-949 to Unicode. [12] The separate private use area range U+F843–F86E is used by IBM to map some characters within the extended hanja range. [12] This follows early recommendations from the Unicode Consortium that corporate characters be allocated from U+F8FF downward and user-defined characters be allocated from U+E000 upward, [18] and is part of a larger corporate private use area scheme which is defined internally by IBM, and uses the range U+F83D–F8FF. [19] [20]

Lead bytes 0x9A–9D (extended symbols and hanja)

According to the 1992 specification, this entire range is user-defined. [11] As implemented in the codec contributed to ICU by IBM, however, 0x9AA1 through 0x9AA5 are the end of the user-defined range. The remainder of this range includes some non-Hangul characters included in Code page 933 but not in Wansung code. 0x9AA6 through 0x9AAB contain miscellaneous technical or mathematical symbols. The remainder contains hanja additional to those included in KS X 1001, although some are mapped by IBM to the Private Use Area. [12]

IBM code page 949 (extended symbols and hanja) [12] [22]
0123456789ABCDEF
9AAx ǂ [lower-alpha 1] ʺ
9ABx
9ACx
9ADx
9AEx
9AFx
9BAx 祿 窿
9BBx
9BCx
9BDx
9BEx 巿
9BFx
9CAx 婿
9CBx 宿
9CCx
9CDx
9CEx
9CFx
9DAx
9DBx
9DCx [lower-alpha 2]
9DDx
9DEx
9DFx

Lead bytes 0x9E–A0 (extended hanja and hangul syllables)

According to the 1992 specification, this entire range is user-defined. [11] As implemented in the codec contributed to ICU by IBM, 0x9EA1 through 0x9EAC contain the remainder of the extended hanja. The rest of the range contains a few additional Hangul syllables which are not available in pre-composed form in pure EUC-KR. Unlike Unified Hangul Code, this is insufficient to support all non-partial Johab syllables absent in Wansung code. [12]

Significant amongst these are 뢔 (0x9EFC), 쌰 (0x9FE6), 쎼 (0x9FED), 쓔 (0x9FF3) and 쬬 (0xA0C1), which correspond to the beginnings of the standard Wansung characters 뢨, 썅, 쏀, 쓩, and 쭁 respectively, when partly entered in an input method editor.

IBM code page 949 (extended hanja and hangul syllables) [12]
0123456789ABCDEF
9EAx
9EBx
9ECx꾿
9EDx
9EEx
9EFx
9FAx
9FBx
9FCx뻿
9FDx
9FEx
9FFx
A0Ax
A0Bx
A0Cx찿
A0Dx
A0Ex
A0Fx

Lead bytes 0xA1–C8, 0xCA–FD (standard Wansung)

See also

Footnotes

  1. This is not included for IPA support. Rather, in Code page 933, SO 0x4160 is a not-equals sign displayed with a slash, while IBM-933 SO 0x418D is one displayed with a backslash (i.e. =⃥). [11] Although it is IBM-933 SO 0x4160 which is mapped to the usual not-equals GCGID SA540080 (fullwidth of SA540000), it is IBM-933 SO 0x418D which is mapped to EUC-KR and IBM-949 0xA1C1, [11] due to the reference glyph for the not-equals sign in KS C 5601-1987 also showing it with a backslash. [23] Hence, U+2260, which is mapped to EUC-KR and therefore IBM-949 0xA1C1, is mapped to IBM-933 SO 0x418D, leaving IBM-933 SO 0x4160 (and therefore IBM-949 0x9AA6) to be mapped to the visually similar character at U+01C2. [24]
  2. The mapping from IBM is U+5231 刱, but the glyph in the IBM document C-H 3-3220-125 1992-09 is closer to U+5259 (host code 62D5). [11]

Related Research Articles

<span class="mw-page-title-main">Character encoding</span> Using numbers to represent text characters

Character encoding is the process of assigning numbers to graphical characters, especially the written characters of human language, allowing them to be stored, transmitted, and transformed using digital computers. The numerical values that make up a character encoding are known as "code points" and collectively comprise a "code space", a "code page", or a "character map".

While Hypertext Markup Language (HTML) has been in use since 1991, HTML 4.0 from December 1997 was the first standardized version where international characters were given reasonably complete treatment. When an HTML document includes special characters outside the range of seven-bit ASCII, two goals are worth considering: the information's integrity, and universal browser display.

Big-5 or Big5 is a Chinese character encoding method used in Taiwan, Hong Kong, and Macau for traditional Chinese characters.

ISO/IEC 8859-7:2003, Information technology — 8-bit single-byte coded graphic character sets — Part 7: Latin/Greek alphabet, is part of the ISO/IEC 8859 series of ASCII-based standard character encodings, first edition published in 1987. It is informally referred to as Latin/Greek. It was designed to cover the modern Greek language. The original 1987 version of the standard had the same character assignments as the Greek national standard ELOT 928, published in 1986. The table in this article shows the updated 2003 version which adds three characters. Microsoft has assigned code page 28597 a.k.a. Windows-28597 to ISO-8859-7 in Windows. IBM has assigned code page 813 to ISO 8859-7. (IBM CCSID 813 is the original encoding. CCSID 4909 adds the euro sign. CCSID 9005 further adds the drachma sign and ypogegrammeni.)

A double-byte character set (DBCS) is a character encoding in which either all characters are encoded in two bytes, or merely every graphic character not representable by an accompanying single-byte character set (SBCS) is encoded in two bytes. A DBCS supports national languages that contain many unique characters or symbols. Examples of such languages include Japanese and Chinese. Korean Hangul does not contain as many characters, but KS X 1001 supports both Hangul and Hanja, and uses two bytes per character.

Shift JIS is a character encoding for the Japanese language, originally developed by the Japanese company ASCII Corporation in conjunction with Microsoft and standardized as JIS X 0208 Appendix 1.

Extended Unix Code (EUC) is a multibyte character encoding system used primarily for Japanese, Korean, and simplified Chinese (characters).

Windows code page 1253, commonly known by its IANA-registered name Windows-1253 or abbreviated as cp1253, is a Microsoft Windows code page used to write modern Greek. It is not capable of supporting the older polytonic Greek.

A CCSID is a 16-bit number that represents a particular encoding of a specific code page. For example, Unicode is a code page that has several character encoding schemes —including UTF-8, UTF-16 and UTF-32—but which may or may not actually be accompanied by a CCSID number to indicate that this encoding is being used.

Windows code page 936, is Microsoft's legacy (pre-Unicode) character encoding for representing simplified Chinese text on computers. It is one of the four Windows DBCSs for East Asian languages, accompanying code pages 932 (Japanese), 949 (Korean) and 950. It is a variant of the Mainland Chinese Guójiā Biāozhǔn Kuòzhǎn (GBK) encoding, and roughly corresponds to IBM code page 1386.

<span class="mw-page-title-main">Code page 950</span> Windows character set for Traditional Chinese

Code page 950 is the code page used on Microsoft Windows for Traditional Chinese. It is Microsoft's implementation of the de facto standard Big5 character encoding. The code page is not registered with IANA, and hence, it is not a standard to communicate information over the internet, although it is usually labelled simply as big5, including by Microsoft library functions.

<span class="mw-page-title-main">Unified Hangul Code</span> Windows character set for Korean

Unified Hangul Code (UHC), or Extended Wansung, also known under Microsoft Windows as Code Page 949, is the Microsoft Windows code page for the Korean language. It is an extension of Wansung Code to include all 11172 non-partial Hangul syllables present in Johab. This corresponds to the pre-composed syllables available in Unicode 2.0 and later.

<span class="mw-page-title-main">JIS X 0201</span> Japanese single byte character encoding

JIS X 0201, a Japanese Industrial Standard developed in 1969, was the first Japanese electronic character set to become widely used. The character set was initially known as JIS C 6220 before the JIS category reform. Its two forms were a 7-bit encoding or an 8-bit encoding, although the 8-bit form was dominant until Unicode replaced it. The full name of this standard is 7-bit and 8-bit coded character sets for information interchange (7ビット及び8ビットの情報交換用符号化文字集合).

JIS X 0212 is a Japanese Industrial Standard defining a coded character set for encoding supplementary characters for use in Japanese. This standard is intended to supplement JIS X 0208. It is numbered 953 or 5049 as an IBM code page.

KPS 9566 is a North Korean standard specifying a character encoding for the Chosŏn'gŭl (Hangul) writing system used for the Korean language. The edition of 1997 specified an ISO 2022-compliant 94×94 two-byte coded character set. Subsequent editions have added additional encoded characters outside of the 94×94 plane, in a manner comparable to UHC or GBK.

KS X 1001, "Code for Information Interchange ", formerly called KS C 5601, is a South Korean coded character set standard to represent Hangul and Hanja characters on a computer.

Microsoft Windows code page 932, also called Windows-31J amongst other names, is the Microsoft Windows code page for the Japanese language, which is an extended variant of the Shift JIS Japanese character encoding. It contains standard 7-bit ASCII codes, and Japanese characters are indicated by the high bit of the first byte being set to 1. Some code points in this page require a second byte, so characters use either 8 or 16 bits for encoding.

Code page 942 is one of IBM's extensions of Shift JIS. The coded character sets are JIS X 0201, JIS X 0208, IBM extensions for IBM 1880 UDC and IBM extensions. It is the combination of the single-byte Code page 1041 and the double-byte Code page 301.

IBM code page 936 is a character encoding for Simplified Chinese including 1880 user-defined characters (UDC), which was superseded in 1993. It is a combination of the single-byte Code page 903 and the double-byte Code page 928. Code page 946 uses the same double-byte component, but an extended single-byte component.

Several mutually incompatible versions of the Extended Binary Coded Decimal Interchange Code (EBCDIC) have been used to represent the Japanese language on computers, including variants defined by Hitachi, Fujitsu, IBM and others. Some are variable-width encodings, employing locking shift codes to switch between single-byte and double-byte modes. Unlike other EBCDIC locales, the lowercase basic Latin letters are often not preserved in their usual locations.

References

  1. 1 2 Borgendale, Ken. "OS/2 Codepage and Keyboard Display Tools".
  2. van Kesteren, Anne, "5. Indexes (§ index EUC-KR)", Encoding Standard, WHATWG, This matches the KS X 1001 standard and the Unified Hangul Code, more commonly known together as Windows Codepage 949.
  3. van Kesteren, Anne. "4.2. Names and labels". Encoding Standard. WHATWG.
  4. 1 2 3 4 5 "Converter Explorer: ibm-949_P110-1999 (alias x-IBM949)", International Components for Unicode , Unicode Consortium
  5. 1 2 "Converter Explorer: ibm-949_P11A-1999 (alias x-IBM949C)", International Components for Unicode , Unicode Consortium . This is the ASCII-based version of IBM-949.
  6. "windows-949-2000", Converter Explorer, International Components for Unicode
  7. "codecs — Codec registry and base classes § Standard Encodings". Python 3.7.2 documentation. Python Software Foundation.
  8. 1 2 3 "Coded character set identifiers: CCSID 949". IBM Globalization. IBM. Archived from the original on 2014-11-29.
  9. "CCSID 1088 information document". Archived from the original on 2016-03-26.
  10. "Code page 951 information document". Archived from the original on 2017-01-16.
  11. 1 2 3 4 5 6 7 8 9 10 11 "IBM Korean Graphic Character Set: DBCS-Host and DBCS-PC" (PDF). IBM. 2001 [1992]. C-H 3-3220-125 1992-09.
  12. 1 2 3 4 5 6 7 8 9 10 International Components for Unicode (ICU), ibm-949_P110-1999.ucm, 2002-12-03
  13. "Coded character set identifiers: CCSID 934". IBM Globalization. IBM. Archived from the original on 2014-12-02.
  14. 1 2 "Coded character set identifiers: CCSID 944". IBM Globalization. IBM. Archived from the original on 2014-12-01.
  15. Viswanadha, Ram (2001-11-01). "ICU-1281 Remove unwanted ucmfiles". International Components for Unicode .
  16. Code Page CPGID 01088 (pdf) (PDF), IBM
  17. Code Page CPGID 01088 (txt), IBM
  18. "2.0: Changes in Unicode 1.0" (PDF). The Unicode Standard, Version 1.1. Unicode Consortium. pp. 3–4. UTR #4.
  19. 1 2 "CPGID 01449: IBM default PUA". IBM Globalization: Code page identifiers. IBM. Archived from the original on 2015-09-16. IBM has designated 195 positions from U+F83D to U+F8FF for use as IBM Corporate-zone and intends to use them consistently within IBM whenever there is a need to maintain the round-trip integrity of IBM characters.
  20. IBM (1997). unicode.nam: Allow the Unicode characters to be specified using either the IBM or PostScript like names. (Included with Borgendale, Ken, OS/2 Codepage and Keyboard Display Tools )
  21. "ibm-933_P110-1995.ucm". International Components for Unicode .
  22. Private Use Area mapped hanja are identified from code charts. The IBM document C-H 3-3220-125 1992-09 gives code charts for the code pages used as the double-byte components for Code page 933 and an older version of Code page 949 without these extensions; however, the hanja in this section correspond to (and are in the same order as) the subset of table 7 for which a "PC Code" is not listed. [11] The Corporate Private Use Area mappings are also co-ordinated with other code pages, [19] including Code page 933, [21] which can be used to obtain the "Host Code" for a given Corporate Private Use Area mapping.
  23. Korea Bureau of Standards (1988-10-01). Korean Graphic Character Set for Information Interchange (PDF). ITSCJ/IPSJ. ISO-IR-149.
  24. "ibm-933_P110-1995 (lead bytes 0E41)". Converter Explorer. International Components for Unicode.