UTF-8 handling
- C course
- Advanced
- Lesson 65/83
- 5 min
Learn UTF-8 string handling.
UTF-8 decoding
UTF-8 decoding turns a byte sequence into a Unicode code point. An ASCII byte (0-127) is a single character. A two-byte sequence has the form 110xxxxx 10xxxxxx; the low 5 bits of the first byte and the low 6 bits of the second byte are combined.
Example: the UTF-8 bytes of 'ü' are 0xC3 0xBC. Decoded: ((0xC3 & 0x1F) << 6) | (0xBC & 0x3F) = 0x03 << 6 | 0x3C = 0xFC = 252. So the Unicode code point of 'ü' is U+00FC.
Decoder function
A decoder function signature: unsigned decode_utf8(const char *s). We process bytes cast to unsigned char to avoid sign extension.
If b[0] < 0x80 it is ASCII; return b[0] directly. Otherwise it is a two-byte sequence: ((b[0] & 0x1F) << 6) | (b[1] & 0x3F). Longer sequences decode similarly with three or four bytes; in this lesson we focus on two bytes. We print the result with printf("%u", cp).
Practice Decode a UTF-8 code point
Write unsigned decode_utf8(const char *s): set const unsigned char *b = (const unsigned char *)s;; if b[0] < 0x80 return b[0]; else return ((b[0] & 0x1F) << 6) | (b[1] & 0x3F). In main compute unsigned cp = decode_utf8("ü"); and printf("codepoint: %u\n", cp);.
Starter code
#include <stdio.h>
// Write unsigned decode_utf8(const char *s)
// ASCII returns b[0]; else decode two bytes
// main: decode "ü" and print the code point
int main(void) {
return 0;
}Sample solution
#include <stdio.h>
unsigned decode_utf8(const char *s) {
const unsigned char *b = (const unsigned char *)s;
if (b[0] < 0x80) return b[0];
return ((b[0] & 0x1F) << 6) | (b[1] & 0x3F);
}
int main(void) {
unsigned cp = decode_utf8("ü");
printf("codepoint: %u\n", cp);
return 0;
}Expected output
codepoint: 252
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 UTF-8 masking
| Term | Meaning |
|---|---|
0x1F |
First byte mask |
0x3F |
Continuation mask |
<< 6 |
Shift |
0x80 |
ASCII boundary |
2 How many bits of the code point does 0xC3 carry?
- 5 bits Correct answer
- 6 bits
- 7 bits
- 8 bits
Explanation: The first byte yields 5 bits via & 0x1F.
3 Mask the continuation byte
unsigned cp = ((b[0] & 0x1F) << 6) | (b[1] & ____);unsigned cp = ((b[0] & 0x1F) << 6) | (b[1] & 0x3F);Explanation: 0x3F takes the low 6 bits of the continuation byte.