Object identifiers

I recently had the chance to dig in to object identifiers. The basic idea is to pick an identifier that uniquely identifies a given object for the duration of its lifetime.

The natural instinct is to derive that identity from some immutable property of the object. For instance, for a person you might use their name or their email address. The trouble is that oftentimes those immutable properties turn out to be mutable. People change their names and their email addresses, not often but often enough to violate our invariant that an identifier has to be valid in perpetuity. The other issue is that some objects, like product listings, do not have an obvious property to anchor identity to. Ideally we want an identifier that can be used for any kind of object.

Another option is to use a serial number. The first object is assigned ID 1, the second is assigned ID 2, and so forth. The serial number is compact, and it sorts automatically by creation time, which is a nice property. Databases offer this approach natively, under different names (e.g., AUTO_INCREMENT, SERIAL, IDENTITY), and will assign a serial number to each new record. This approach works for any kind of object, but the serial number is only unique within a single table or database, which means you can end up with collisions when you merge across those boundaries. The serial numbers are guessable, which presents some security risk. You also need a central authority to issue serial numbers, which becomes a bottleneck for distributed systems. It would be better if we could generate identifiers anywhere and ensure that they are unique across systems.

The universally unique identifier (UUID) was introduced to solve this coordination problem. UUIDv4 uses a 128-bit identifier, and 122 of those bits are random. That means any machine can generate one without a material risk of collision. It is also prohibitively expensive to compute every possible UUID, so the security risk of discovering identifiers evaporates. This has been the standard for a long time, but it is not without its shortcomings. These are pure random numbers so there is no storage locality. They are also not sorted by creation time, so you have to store a separate created_at field.

The UUIDv7 addresses these issues by allocating 48 bits of that 128-bit space for a timestamp. The timestamp makes the identifier sortable and also improves storage locality. It also preserves enough random bits to avoid collisions, so you can continue to generate identifiers on any machine. This is the approach that modern services are moving toward.

We can refine this approach further by representing these identifiers using a more ergonomic encoding. UUIDs are canonically encoded as a sequence of characters separated by hyphens: 45abac3f-150b-42ce-975d-b83c9eb7dd9e. This representation is opaque in that it does not tell you what type of object it represents. It is also a bit harder to work with because double clicking often selects only a part of the identifier and stops at a hyphen boundary. We can fix this by appending a type prefix and by encoding the id using a constrained set of characters.

The TypeID is one formalization of this approach. A TypeID looks like this: user_2x4y6z8a0b1c2d3e4f5g6h7j8k. The prefix makes it clear that it is a user id. The suffix is a compact Crockford base32 encoding of the UUIDv7 which avoids hyphens and can be selected with a double click. In practice, I would store the UUID in the database and translate back and forth from this representation at the API boundary.