09 Oct 2026 · infrastructure, operating systems, go · 5 min

Swap is not free RAM: Go GC pauses under memory pressure

The original case

In September, Fernando Simões published an uncomfortable measurement. A Go service inside a cgroup, with swap enabled to absorb memory spikes. The median garbage collector pause was 51 µs. The worst one hit 40 ms. Of those 40 ms, 39 ms went into 228 page faults inside a stop-the-world pause. Over 30 minutes, 312 pauses of that kind.

The cause is structural. The Go GC keeps metadata outside the heap, in a region the runtime never returns to the system. Those pages are read on every collection cycle. The kernel picks what to send to swap by age, and metadata pages, rarely touched compared with the rest, end up leaving memory. When the collector reads them, the whole process is stopped waiting for the device.

The question

The author measured with swap on NVMe. I wanted to measure with swap on zram, which is what runs on this VM: pages compressed in RAM itself. If a page fault is served by decompressing in memory, the per-page cost should drop a lot, and a 40 ms pause should not show up. Allocation latency, on the other hand, still gets paid.

How I tested

• KVM VM with 4 vCPU, 5.8 GB of RAM, Ubuntu 24.04, kernel 6.8. Swap only on zram, zstd, 5.8 GB virtual device, priority 100.
• The allocator published with the article, from the go-gc-swap-cost repository, built with Go 1.27.1.
• Two identical runs: 2 spikes of 299 MiB on a 59 MiB baseline. The tight run had a 380 MB memory cap and 3 GB of swap; the loose run had a 2 GB cap and 512 MB of swap. Both via systemd-run –user –scope.
• The allocator writes a CSV every 10 ms: stop-the-world pause histogram from /gc/pauses:seconds, process major faults, global swap counters and the latency of each allocation step.

Results

Pause numbers are the histogram bucket upper bound, not continuous measurements: no pause exceeded 196 µs. The swap counters are machine wide, not per process.

What changed and what did not

The stop-the-world pause did not show up. Not close: 196 µs in both conditions against 40 ms in the NVMe case. What did show up was latency local to the goroutine doing the allocation: p99 went from 3.4 ms to 9.7 ms and the worst case from 11 ms to 22 ms, with 69,000 major faults against zero.

That cost is paid only by the allocator, not by the whole process as a GC pause is. But it is the cost a user feels as a slow response. The other number from the original article is worth repeating: building one 511 KiB message, which took 3 to 5 ms, went to 105 ms with swap on NVMe and 903 ms on a network volume.

Method caveats

Both runs happened once, and this VM is not idle. Before the test it already held 3.4 GB of data in zram, compressed into 779 MB (a 4.6 to 1 ratio), and 17.5 million major faults since boot, three days earlier. The global swap moves without me, so the pswpin and pswpout deltas are in the table as an order of magnitude, not as an isolated measurement of my process.

What is clean: both runs used the same allocator, the same Go, the same hardware and the same load size. Only the cgroup memory cap changed.

Why the difference comes from the device

A major page fault costs the time to find the page and bring it back. With swap on NVMe, that means a disk read: queueing, bio submission, waiting. With zram, it means decompressing in memory. The order of magnitude changes, and the pause follows it. The cause sits with the device behind swap, not with the Go collector.

The author also measured the Green Tea GC, the collector that landed in Go 1.26, and saw no relevant impact on metadata reads. I ran Go 1.27.1.

What to do

1. Measure your own case. GC pause depends on heap size, allocation rate and swap device. Someone else conclusion does not transfer.
2. Isolate the service in a cgroup. Set memory.high before memory.max: you get a warning before the kernel starts evicting aggressively.
3. Choose swap on purpose. zram solves the slow disk case, not all of them: when it fills up there is nowhere for cold pages to go and the kernel loses margin. zswap compresses the same way, but in front of a disk swap, with automatic eviction of the oldest pages.
4. Leave headroom. This VM has 5.8 GB and lives with swap in use. If your service needs low p99, spare RAM is cheaper than any swap tuning.