UTF-8 handling

Learn UTF-8 string handling.

UTF-8 decoding

UTF-8 decoding turns a byte sequence into a Unicode code point. An ASCII byte (0-127) is a single character. A two-byte sequence has the form 110xxxxx 10xxxxxx; the low 5 bits of the first byte and the low 6 bits of the second byte are combined.

Example: the UTF-8 bytes of 'ü' are 0xC3 0xBC. Decoded: ((0xC3 & 0x1F) << 6) | (0xBC & 0x3F) = 0x03 << 6 | 0x3C = 0xFC = 252. So the Unicode code point of 'ü' is U+00FC.

Decoder function

A decoder function signature: unsigned decode_utf8(const char *s). We process bytes cast to unsigned char to avoid sign extension.

If b[0] < 0x80 it is ASCII; return b[0] directly. Otherwise it is a two-byte sequence: ((b[0] & 0x1F) << 6) | (b[1] & 0x3F). Longer sequences decode similarly with three or four bytes; in this lesson we focus on two bytes. We print the result with printf("%u", cp).

Exercise Decode a UTF-8 code point

Write unsigned decode_utf8(const char *s): set const unsigned char *b = (const unsigned char *)s;; if b[0] < 0x80 return b[0]; else return ((b[0] & 0x1F) << 6) | (b[1] & 0x3F). In main compute unsigned cp = decode_utf8("ü"); and printf("codepoint: %u\n", cp);.

Show example solution

#include <stdio.h>

unsigned decode_utf8(const char *s) {
    const unsigned char *b = (const unsigned char *)s;
    if (b[0] < 0x80) return b[0];
    return ((b[0] & 0x1F) << 6) | (b[1] & 0x3F);
}

int main(void) {
    unsigned cp = decode_utf8("ü");
    printf("codepoint: %u\n", cp);
    return 0;
}

Test yourself

UTF-8 masking

0x1F

First byte mask
0x3F

Continuation mask
<< 6

Shift
0x80

ASCII boundary
How many bits of the code point does 0xC3 carry?
- 5 bits
- 6 bits
- 7 bits
- 8 bits
Show answer

5 bits — The first byte yields 5 bits via & 0x1F.

Mask the continuation byte

unsigned cp = ((b[0] & 0x1F) << 6) | (b[1] & ____);

Show answer

unsigned cp = ((b[0] & 0x1F) << 6) | (b[1] & 0x3F);

Next lesson: Summary - Unicode

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies