Unicode

Unicode is a text encoding standard that assigns numeric codes to writing systems from all over the world.

This page compiles some facts about Unicode that Toaqists may find useful to know.

Unicode vs. UTF-8

Unicode, the standard, dictates (for example) that the character ꝡ is represented by the number 42849, or A761 in hexadecimal.

These "codepoint numbers" are usually written as U+ followed by the hexadecimal number: U+A761.

UTF-8, an encoding, dictates how to encode that number across bytes in a file: it says U+A761 is ea 9d a1.

There are other encodings of Unicode, but they are not as commonly used. For example, in UTF-16, the encoding of U+A761 is simply a7 61.

Combining characters and normalization

A letter with a diacritic like é can be represented as a precomposed form (é U+00E9 latin small letter e with acute) or as a sequence of a base letter and combining characters (e U+0065 latin small letter e and ◌́ U+0301 combining acute accent).

Unicode text may be normalized to smooth over these differences: either by precomposing everything as much as possible (normalization form C or NFC) or by decomposing everything into combining characters (normalization form D or NFD).

Normalization also pins down the order of combining characters. Underdots come before hats. The string é + underdot has NFC ẹ + acute and NFD e + underdot + acute.

Dotless ı and normalization

The letter í decomposes into i + acute, not ı + acute. Placing diacritics on a dotless ı may produce wrong-looking results. Compare:

Letter	NFC	NFD
ı	ı	ı
î	î	i + circumflex
ı̣	ı + underdot	ı + underdot
ị̂	ị + circumflex	i + underdot + circumflex

Precomposed tone–underdot combos

Not all Toaq tone–underdot combos have precomposed characters. This table shows precomposed characters in green and NFC forms in red:

a	ạ	ạ́	ạ̈	ậ
u	ụ	ụ́	ụ̈	ụ̂
ı	ı̣	ị́	ị̈	ị̂
o	ọ	ọ́	ọ̈	ộ
e	ẹ	ẹ́	ẹ̈	ệ

Paradoxically, depending on the font and operating system, the "abnormal" forms (like é + underdot) may show up more correctly. They are demonstrated in the table below:

a	ạ	ạ́	ạ̈	ậ
u	ụ	ụ́	ụ̈	ụ̂
ı	ı̣	ị́	ị̈	ị̂
o	ọ	ọ́	ọ̈	ộ
e	ẹ	ẹ́	ẹ̈	ệ

(MediaWiki normalizes page contents, meaning that the above table has had to use HTML entities to get the desired effect (e.g. ị́ for ị́). Template:T will do this for you.)

See also

• Latin writing system
• Input methods