Why Cache Capacity Is Decided by Bytes You Never Counted: Five Rust Layout Changes That Cut 1.1.1.1's DNS Cache 56 Percent

Caching looks like a solved problem until you multiply one design decision by 250 billion. Cloudflare's Big Pineapple, the Rust platform behind 1.1.1.1, holds more than 250 billion DNS cache entries fleet-wide, which means every single wasted byte per entry costs more than 250 GB of memory. In August 2026 they published how five changes to the way one cache entry is laid out in memory cut it from 953 to 420 bytes per entry, a 56 percent reduction that freed roughly 100 TB of RAM, the working memory of about 130 of their Gen 13 servers. Insert throughput went up 43 percent and lookup latency down 19 percent as side effects.

None of the five changes is about storing less data. They are about how much room the container around the data takes, and that distinction is the whole lesson. This post walks each change and the arithmetic behind it.

First, a way to see per-entry memory

The team instrumented a custom Rust GlobalAlloc wrapper to record the count and size of every allocation per cache entry, then measured a benchmark workload matching real traffic: 56 percent A records, 25 percent AAAA, 19 percent TXT, one to four records per entry. That measurement discipline is worth stealing on its own, because the entire 56 percent was invisible until someone asked "how many bytes and how many allocations does one entry cost?" instead of watching aggregate memory graphs.

The starting point: 953 bytes and about 1.1 KB of allocation bookkeeping per entry, for DNS answers that are usually a few dozen bytes of payload.

One: Vec and String become Box slices

A Rust Vec<T> is three words: a pointer, a length, and a capacity. Capacity exists so the vector can grow, but a cache entry's record list never grows - it is built once when the upstream answer arrives and then read forever. Two String fields on a DNS name carry the same dead weight.

The change was to store fixed content as Box<[T]> and Box<str> instead: same pointer and length, no capacity word, no over-allocated heap slack.

// before: 8 Vec/String fields, each with a capacity word it never uses
struct Entry {
    question: String,
    answers: Vec<Record>,
    // ... six more
}

// after: fixed-size slices, capacity word gone
struct Entry {
    question: Box<str>,
    answers: Box<[Record]>,
}

Eight bytes saved per field, eight such fields, so 64 bytes per entry from the struct alone, plus whatever heap over-allocation the vectors had reserved. Cloudflare puts the fleet-wide total from this single change at over 15 TB. The general rule: if a collection is written once and never mutated, it does not need a growable representation.

Two: three section lists become one list with offsets

A DNS answer has three sections: answer, authority, additional. The cache stored each section as its own list, and even after the first change each Box<[T]> is still two words (pointer plus length). Three sections, two words each: six words to say "here are the records".

Since a DNS message can carry at most 65535 records per section, section counts fit in a u16. The new layout keeps one list of records plus two 2-byte offsets marking where each section starts inside it. Six words collapse to one pointer, one length, and four bytes.

// before: three lists, six words total
answer: Box<[Record]>,
authority: Box<[Record]>,
additional: Box<[Record]>,

// after: one list, two u16 offsets marking section starts
records: Box<[Record]>,
authority_start: u16,
additional_start: u16,

28 bytes saved per entry. The same change noted a subtler win from packing boolean flags into a single bitfield, which shrank the struct by more than the size of the booleans themselves, because eight separate one-byte booleans each carry seven bytes of struct padding.

Three: stop storing the owner name you already know

Every DNS record nominally carries its owner name, the domain it belongs to. For most records that name is exactly the question the client asked, which the cache key already stores. Storing it again inside each record was 100 percent redundancy for the common case.

The fix is Option<Box<Name>>: None means "infer from the cache key at read time", Some appears only when the owner actually differs, as with the A records behind a CNAME. The interesting part is what they rejected: keeping DNS wire-format compression pointers in the cached bytes, because following compression pointers on every cache lookup is hot-path work you pay forever. Deferring a cost into a read path that runs billions of times is not a saving.

Four: stop letting one rare variant size the whole enum

This is the change with the widest application beyond DNS. Rust enums are sized by their largest variant. The record-type enum held a Naptr variant at 136 bytes, so every record in the cache, including the A (4 bytes) and AAAA (16 bytes) records that make up over 80 percent of traffic, occupied 144 bytes of enum storage.

// before: every variant pays for NAPTR's 136 bytes
enum RecordData {
    A(Ipv4Addr),          // 4 bytes of data, 144 bytes of enum
    Aaaa(Ipv6Addr),
    Txt(Txt),
    Naptr(Naptr),
}

// after: rare, large variants boxed; enum drops to 24 bytes
enum RecordData {
    A(Ipv4Addr),
    Aaaa(Ipv6Addr),
    Txt(Box<Txt>),
    Naptr(Box<Naptr>),
}

Two honest costs came with it. A boxed Naptr gets its own heap allocation, and jemalloc rounds allocations to size classes, so a 40-byte MX record rounds up to 48; and scattered heap allocations hurt CPU cache locality. Both were accepted because the 120 bytes saved on the dominant record types dwarf both.

flowchart LR
  Q["one cache entry"] --> E["RecordData enum<br/>sized by largest variant"]
  E -->|"inline: every variant<br/>pays 144 bytes"| BIG["A record = 144 bytes"]
  E -->|"boxed rare variants"| SMALL["enum = 24 bytes<br/>A record = 4 bytes"]
  BIG --> W["+120 wasted bytes per A record"]
  SMALL --> S["80% of traffic stops overpaying"]

Five: store record data as wire bytes

The last change went furthest: instead of parsed enum variants per record, the record data is serialized into a reusable scratch buffer and stored as one Box<[u8]> per entry, with a 2-byte length prefix per record. To read it back, the parser walks the buffer sequentially.

This step alone raised insert throughput 13 percent, because building a cache entry became a copy instead of a tree of parsed allocations. The team spelled out why they did not go all the way to caching the entire upstream wire message: DNSSEC handling would force caching two variants of every answer, and a per-lookup parse would replace cheap memory with hot-path CPU. The wire format also gives up random indexing, so round-robin record rotation has to walk linearly, which costs nothing at one to four records per entry.

What it added up to

Per-entry net footprint fell from 953 to 420 bytes (56 percent), allocation bookkeeping from about 1.1 KB to 461 bytes, insert throughput rose from 625,000 to 893,000 entries per second, and lookup latency fell from 828 to 670 ns. In production, p99 resident memory per node fell 43 percent and p90 by 42 percent during a stepped rollout from May 18 to July 6, 2026. The freed 100 TB is being reinvested in larger caches, which means higher hit rates: the memory saved does not just sit there, it makes the cache better.

The transferable lesson is not "box your enums". It is that a cache serving billions of small values is a per-entry layout problem, and the byte you never counted is the one that decides your capacity bill. Measure per-entry cost with the allocator, not with aggregate graphs, then question every field that exists for a mutation that never happens, every pointer to data you can derive, and every variant that pays rent for a neighbor's size.

Grounding for every number in this post: Cloudflare's own write-up, "How we saved 100 terabytes of memory by optimizing 1.1.1.1's DNS cache", blog.cloudflare.com, August 27, 2026.