The Protobuf Wire Format — Decoded Byte by Byte

The Whole Format in One Idea

A message is a concatenation of fields. Each field is a tag followed by a value, encoded per the tag's wire type. That's it — the format is recursive (a nested message is just a length-delimited value that happens to be more fields).

flowchart LR subgraph Field["one field = Tag-Length-Value"] T["TAG
(field# << 3) | wire_type
varint"] --> L["LEN
(only for wire type 2)
varint"] L --> V["VALUE
varint / fixed / bytes"] end Field --> Field2["next field..."]

No schema needed to skip. The tag alone tells a decoder the field number and the wire type — and the wire type tells it exactly how many bytes the value spans. So an old parser can walk past a field it's never heard of. That single property is the whole reason schema evolution works.

Rule 1 — The Tag Byte

Every field starts with a tag, itself a varint. The low 3 bits are the wire type; the rest is the field number:

tag         = (field_number << 3) | wire_type
field_number = tag >> 3
wire_type    = tag & 0b111        # low 3 bits

Wire type	Name	Used by
`0`	Varint	int32/64, uint32/64, sint32/64, bool, enum
`1`	64-bit	fixed64, sfixed64, double
`2`	Length-delimited	string, bytes, embedded messages, packed repeated
`3` / `4`	Start / End group	Deprecated (legacy groups)
`5`	32-bit	fixed32, sfixed32, float

Worked example: field 1, wire type 2 (a string) → (1 << 3) | 2 = 8 | 2 = 10 = 0x0A. Field 2, wire type 0 (a varint) → (2 << 3) | 0 = 16 = 0x10. Those two tag bytes show up in the decode below.

Rule 2 — Varints (the workhorse)

A varint stores an integer in as few bytes as its magnitude needs. Each byte gives 7 bits of payload; the high bit (MSB) is a continuation flag — 1 means "more bytes follow", 0 means "last byte". Groups are stored least-significant first.

Decoding `150` → bytes `96 01`

Byte	Binary	Continuation?	7-bit payload
`0x96`	`1001 0110`	1 → more	`001 0110`
`0x01`	`0000 0001`	0 → last	`000 0001`

Strip the MSBs, then reassemble least-significant group first: 0000001 ++ 0010110 = 0000001 0010110 = 10010110 = 150. (128 + 16 + 4 + 2 = 150.) ✅

The negative-int trap: a plain int32 holding -1 is sign-extended to a full 64-bit value (0xFFFFFFFFFFFFFFFF) before varint encoding — that's 10 bytes for a single small number. This is exactly why sint32/sint64 exist.

Rule 3 — ZigZag (for signed values)

Varints are efficient for small non-negative numbers. sint32/sint64 first map signed integers so that small magnitudes — positive or negative — become small unsigned values, then varint-encode the result:

# 32-bit zigzag
encoded = (n << 1) ^ (n >> 31)     # n >> 31 is arithmetic: all-1s if negative, else 0
# decode
n = (encoded >> 1) ^ -(encoded & 1)

Original `n`	ZigZag encoded
`0`	`0`
`-1`	`1`
`1`	`2`
`-2`	`3`
`2`	`4`

Check -1: (-1 << 1) ^ (-1 >> 31) = (-2) ^ (-1) = 1. It "zig-zags" between positive and negative so both stay near zero and encode to a single byte.

Rule 4 — Length-Delimited (wire type 2)

Strings, bytes, embedded messages, and packed repeated fields all use the same shape: tag → length (varint) → that many raw bytes. Because the length is explicit, a decoder can copy or skip the whole blob without understanding its contents.

0A 03 42 6F 62
│  │  └──────── "Bob"  (0x42='B' 0x6F='o' 0x62='b')
│  └─────────── length = 3
└────────────── tag 0x0A → field 1, wire type 2

Recursion for free: an embedded message is encoded exactly like a string — a length followed by its raw bytes. So the entire format is "fields, one of which may be a bag of fields." No special container syntax anywhere.

Rule 5 — Put It Together: Full Decode

Message: User { string name = 1; int32 id = 2; bool admin = 3; } with name="Bob", id=150, admin=true. On the wire:

0A 03 42 6F 62 10 96 01 18 01

Bytes	Role	Decoded
`0A`	tag → field 1, type 2	`name` (string)
`03`	length	3 bytes follow
`42 6F 62`	value	`"Bob"`
`10`	tag → field 2, type 0	`id` (varint)
`96 01`	value (varint)	`150`
`18`	tag → field 3, type 0	`admin` (varint)
`01`	value (varint)	`true`

Tag check for field 3: (3 << 3) | 0 = 24 = 0x18. ✅ Ten bytes total — the same data as {"name":"Bob","id":150,"admin":true} (35 bytes of JSON).

Try it yourself: protoc --decode_raw < message.bin walks any protobuf blob using only the wire types — no .proto needed. It's the fastest way to sanity-check what a service is actually putting on the wire. Now that skipping unknown fields makes sense, read Schema Evolution.