ASCII and Unicode: What They Are and What They Are For

What a character encoding is

An encoding is an agreement: "the number 65 means the letter A". When you type a text, the program stores those numbers; when someone else opens it, their program translates them back into letters using the same agreement. If the two sides do not use the same table, the text comes out wrong.

Two ideas need to be kept apart. One is the character set, the list of which numbers correspond to which symbols. The other is how those numbers are stored in bytes. In ASCII the two coincide in a trivial way; in Unicode they do not, and that is where UTF-8 comes in.

ASCII: 128 characters of 7 bits

ASCII (American Standard Code for Information Interchange) was standardized in the 1960s. It uses 7 bits, that is 2 to the power of 7 = 128 codes, from 0 to 127. Of those, 33 are control characters (0 to 31 and 127) and 95 are printable (32 to 126, including the space).

It does not include accented letters, the ñ or the euro sign: it was designed for English. That explains why other tables were needed later.

Relationships worth remembering

The table is not in random order. With a few regularities you can read codes from memory.

• The digits "0" to "9" take 48 to 57 (30 to 39 in hexadecimal). A digit's value is its code minus 48: "7" is 55, and 55 − 48 = 7.
• Uppercase letters run from 65 ("A") to 90 ("Z") and lowercase from 97 ("a") to 122 ("z").
• The difference between an uppercase letter and its lowercase version is always 32: "A" = 65 and "a" = 97; "C" = 67 and "c" = 99. In binary only one bit changes: 01000001 for "A" and 01100001 for "a".
• To find which place of the alphabet an uppercase letter holds, subtract 64: "C" is 67 and 67 − 64 = 3, the third letter.
• Because all uppercase codes (65 to 90) are lower than the lowercase ones (97 to 122), sorting "by code" puts "Zebra" before "apple". Many programs need a separate comparison option to sort properly.

Line endings: LF, CR and CRLF

A line break looks simple, but it is written differently depending on the system. Unix, Linux and macOS use a single character, LF (10). Windows uses two in a row, CR and LF (13 and 10), known as CRLF. Old teletypes needed two separate commands: return the carriage to the start and advance the paper one line.

That difference is the reason for files that look like one long line in a basic editor, or that show a strange symbol at the end of every row. If you want to know the code of each, type LF or CR in the ASCII table's search box.

"Extended ASCII" is not a standard

Since a byte has 8 bits and ASCII uses only 7, there were 128 free codes left (128 to 255) and each manufacturer and region filled them its own way. This is known as "extended ASCII", but there is no single table: there are many, called code pages. Two of the best known are Latin-1 (ISO-8859-1), where é is code 233, and Windows-1252, similar but with differences in codes 128 to 159 (the euro sign is there, for example).

The problem is that code 233 means "é" in one table and something else in another. A text written with one table and read with another showed garbage. Unicode was created to end that mess.

Unicode and UTF-8

Unicode gives a unique number, called a code point, to each character of almost all the writing systems in the world, plus symbols and emojis. Today it has more than 150,000 characters. It is written with the prefix U+ and in hexadecimal: "A" is U+0041, "ñ" is U+00F1, the euro sign is U+20AC and the grinning face emoji is U+1F600. The first 128 code points are exactly those of ASCII.

UTF-8 is the most widely used way to store those numbers as bytes, and it uses 1 to 4 depending on the character. The 128 ASCII characters take a single byte identical to the old one, so an ASCII text is also a valid UTF-8 text. Everything else takes more bytes.

Why "ñ" appears instead of "ñ"

This is the most common error and it has an exact explanation. In UTF-8 the ñ is two bytes: C3 and B1. If a program thinks the text is Latin-1, it reads each byte as a character: C3 is "Ã" and B1 is "±". Result: "ñ". The same happens with "é", which looks like "é", and with "canción", which appears as "canción".

The fix is almost always to tell the program to open the file as UTF-8 (in a text editor or in the import settings of a spreadsheet). When it happens the other way around, Latin-1 text read as UTF-8, diamonds with a question mark usually appear, because those lone bytes do not form valid UTF-8 sequences.

How to use the ASCII table and the converter

The "ASCII table" tool shows the 128 characters with their code in decimal, hexadecimal, octal and binary, and the names of the control ones.

1. Type a character, a code or the name of a control character (for example A, 65, 41 or LF) in the search box to find its row.
2. Use the filter buttons (Control, Digits, Uppercase, Lowercase, Symbols) to see just one group.
3. To convert, type some text in "Text" and pick the format (decimal, hexadecimal, octal or binary): the codes appear below. "Hola" gives 72 111 108 97 in decimal and 48 6F 6C 61 in hexadecimal.
4. You can also go the other way: paste codes separated by spaces or commas in "Codes" and the text is built. In hexadecimal, prefixes such as 0x41 are accepted.

If the text has characters that are not ASCII, such as ñ, the tool says so and shows their Unicode code. To see the real UTF-8 bytes use "Text to binary": ñ gives 11000011 10110001, two 8-bit bytes. The limits: the table shows standard ASCII only, not extended, and everything is calculated in your browser without sending the text to any server.

Frequently asked questions