Protobuf Basics — .proto, Fields, and protoc

The Schema Is the Source of Truth

A .proto file describes messages in a language-neutral IDL. The compiler (protoc) reads it and emits native classes with getters, setters, and built-in serialize/parse methods.

One schema, every language. Because all generated code targets the identical wire format, a Python producer and a Go consumer interoperate with zero shared runtime — they only share the .proto.

A First Message

syntax = "proto3";

package example;

message User {
  string name    = 1;
  int32  id      = 2;
  bool   admin   = 3;
  repeated string emails = 4;
  Role   role    = 5;

  enum Role {
    ROLE_UNSPECIFIED = 0;   // proto3 enums MUST have a 0 default
    ROLE_MEMBER      = 1;
    ROLE_ADMIN       = 2;
  }
}

Read the anatomy:

syntax = "proto3" — the dialect. proto3 is the modern default; it dropped required and made scalars default-valued.
string name = 1 — type, name, and the field number. That = 1 is not a value; it's the field's permanent identity on the wire.
repeated — a list. Zero or more of the field.
enum — a named integer set. In proto3 the first entry must be 0 and acts as the default.

Field Numbers Are the Contract

The name is for humans; the number is for the wire. Protobuf encodes each field with its number, never its name. Rename name → full_name and old data still parses. Change = 1 to = 7 and you've silently broken every existing message.

Numbers 1–15 use a single tag byte — spend them on your hottest, most-repeated fields.
Numbers 16–2047 take two tag bytes; valid range runs up to 2^29 − 1.
19000–19999 are reserved for the implementation — you can't use them.

The why and the safe-change rules live in Schema Evolution; the how they're encoded lives in The Wire Format.

Scalar Types (the ones you'll actually use)

.proto type	Notes	Python type
`int32` / `int64`	Varint. Inefficient for negatives	`int`
`sint32` / `sint64`	Varint + zigzag. Use for values that go negative	`int`
`uint32` / `uint64`	Varint, unsigned	`int`
`fixed64` / `fixed32`	Always 8 / 4 bytes. Cheaper than varint for large numbers	`int`
`bool`	Varint, one byte	`bool`
`string`	UTF-8, length-delimited	`str`
`bytes`	Raw, length-delimited	`bytes`
`double` / `float`	IEEE-754, 8 / 4 bytes	`float`

Pick the right integer. int32 encodes -1 as a bloated ten-byte varint; sint32 zigzags it down to one byte. If a field is ever negative, reach for sint*. See The Wire Format.

Nesting, Presence, and maps

message Order {
  message LineItem {            // nested message type
    string sku      = 1;
    int32  quantity = 2;
  }
  repeated LineItem items = 1;
  map<string, string> labels = 2;   // sugar for repeated key/value pairs

  optional string coupon = 3;   // 'optional' brings back explicit presence
}

Presence in proto3: by default a proto3 scalar can't tell "0 / empty string" from "never set" — both read as the default and both are omitted from the wire. Marking a field optional restores explicit presence (a has_coupon() check), at the cost of a little tracking overhead. Message fields and repeated fields always track presence.

Compile and Use It

# 1. Compile the schema to Python
#    protoc --python_out=. user.proto   ->   user_pb2.py

import user_pb2

# 2. Build a message
u = user_pb2.User(name="Bob", id=150, admin=True)
u.emails.append("bob@example.com")
u.role = user_pb2.User.ROLE_ADMIN

# 3. Serialize to bytes (this is the wire format)
data = u.SerializeToString()
print(data)          # b'\n\x03Bob\x10\x96\x01\x18\x01...'
print(len(data))     # a couple dozen bytes

# 4. Parse it back on the other side
u2 = user_pb2.User()
u2.ParseFromString(data)
print(u2.name, u2.id, u2.admin)   # Bob 150 True

Real-world: teams check the .proto files into a shared repo (often a dedicated schema repo), generate code in CI, and publish the generated packages. The schema — not any one service — is the interface. This same schema also defines gRPC services; see the gRPC collection.