rushort: 16.8M redirects/s on M4 Pro loopback

rushort peaked at 16.8 million redirects a second on my M4 Pro, with 32 connections sending 128 requests per batch. Without pipelining, the median was about 165,000. The server was identical. How much came from the client, and what would hold up on a $4 VPS?

The post that started this concerned 100 million requests a day. Spread evenly, that is about 1,157 requests a second. It says little about what those requests do. A redirect reads a URL already in memory; creating a link has to make a write durable before returning it. I chose a small URL shortener so I could test those paths separately. The daily average leaves out bursts.

what one redirect does

Each HTTP/1.1 connection runs in one tokio task. The parser reads header spans and routes on (method, path). It handles up to 256 buffered requests before yielding, collecting responses for one write_all call. That call may take several socket writes, but avoids a separate send path for each response. The connection loop shows the batch boundary.

Pipelining keeps that loop busy. In the peak M4 Pro setup, 32 connections each sent 128 requests per batch: 32 × 128 = 4,096 requests could be outstanding. The unpipelined setup used 64 connections and allowed 64 outstanding requests. Batch size and concurrency both changed, so this does not isolate socket-call cost.

Resolving a code takes less work. The base62 code decodes to a sequential ID; the low bits choose a shard and integer division chooses a slot in its vector. As an illustration, with the default 64 shards, ID 130 belongs to shard 2, slot 2. There's no hash lookup or per-read allocation here. The stored value is an Arc<str>, so returning it still changes a shared reference count. This is the lookup implementation:

pub fn resolve(&self, code: &str) -> Option<Arc<str>> {
    if code.len() > 11 || (code.len() > 1 && code.starts_with('0')) {
        return None;
    }
    let id = usize::try_from(base62_decode(code)?).ok()?;
    self.shards[id & (self.shards.len() - 1)]
        .read()
        .get(id / self.shards.len())
        .cloned()
}

This layout trades away properties a public shortener might need. Sequential IDs make links guessable. Creating the same URL twice uses two slots. I'm assuming trusted link creators; secret links and abuse controls would need additional design work.

writes take a different path

Durable mode uses SQLite WAL with synchronous=FULL and fullfsync enabled. A single writer commits the record before publishing it to the 64-way sharded RAM cache. Only then does the request receive an acknowledgment. Reversing that order would let a reader resolve a link that the database hadn't committed yet. Blocking database work runs off the tokio workers; redirects keep using the read cache. Surviving a power loss also requires the OS and storage to honor the sync operations.

A lock file rejects a second database owner. Startup checks contiguous IDs, valid URLs, and configured capacity. src/store.rs owns storage; src/lib.rs parses and routes; src/bin/shortener.rs configures the service and bearer-token authentication. SIGINT/SIGTERM allows ten seconds to drain connections. Serving takes roughly 1,000 lines; load generation is separate.

Defaults are 1,024 connections, 32 concurrent writes, five-second header/body/socket deadlines, and one million URLs. Request limits are 8 KiB for the body and 16 KiB for headers. The corresponding options include --max-connections, --max-writes, --timeout, and --max-urls. Write-admission and capacity failures return 503. Duplicate Content-Length, Transfer-Encoding, conflicting expectations, and malformed or oversized requests are rejected with the appropriate 400/413/417/431 response.

The dashboard reads public, CORS-enabled /api/metrics (uptime, URL count, capacity, atomic request/redirect/write/error counters) and /api/host (CPU and memory). Shared counter updates create cache-coherence traffic worth measuring. The site creates links here too. Its whiteye.in/s/… redirect fetches upstream before returning a 302, adding a hop.

what the client counts

I used four workloads. Closed-loop connections send a batch and wait for responses, offering less work when the server slows down. Fixed-rate arrivals keep their schedule independently of responses, allowing a queue to grow. Unpipelined clients wait between requests on each connection. The durable mix adds authenticated link creation and SQLite commits. These tests don't isolate framework overhead or rank rushort against other servers.

The runner, ./bench.sh → scripts/simulate.py, builds release binaries and starts isolated servers with fresh credentials. It saves raw logs and a machine-stamped CLAIMS.md under target/benchmarks/<stamp>/. Fixed-rate arrivals use absolute deadlines and bounded queues. Overflow counts as dropped work and fails the run. Transport errors, wrong statuses, and 302s pointing at the wrong URL fail it too.

Its pipelined latency timer covers the full batch, including scheduling delay in fixed-rate mode. For example, the 17.62ms batch p99 reported below cannot be divided by 128 to produce a 0.138ms request p99. The timer ended when the batch finished; there are no 128 equal request intervals to recover from it. With a queue, a fixed-rate run can have much longer measured latency than a closed-loop run, even when its throughput is lower.

Earlier benchmark versions got this wrong. They divided batch latency by pipeline depth, counted an extra half-second of traffic outside the peak measurement window, and could award PASS to all-500 responses. Those were failures in my measurement code. Regression fixtures now reject those cases, wrong redirect URLs, and duplicate codes. Blackbox tests also check that acknowledged writes remain available after a SIGKILL restart. These checks implement the load-generator contract quoted with the results.

the measurements

M4 Pro, client and server on loopback

Client and server shared a 14-core M4 Pro running macOS, built with lto=thin and codegen-units=1. The full invocation was --repeats 3 --seconds 30, but scenario durations differ. The published results include a 30-second durable mix, three five-second saturation runs, and a ten-second fixed-arrival test.

• Fixed-rate pipelined redirects. At a 10M RPS target for ten seconds, 32 connections and pipeline depth 128 completed 100,000,000 correct requests in 10.002s: 9,997,937 achieved RPS. Batch p99 was 17.62ms, with zero drops. The 16.8M peak came from the separate saturation test.
• Closed-loop saturation. With the same pipelining, the best single run reached 16,847,534 RPS. The median of three was 16,281,158 RPS, with batch p99 around 0.5ms.
• Requests without pipelining. With 64 connections, the median was 164,907 RPS and p99 around 0.5ms. This is a macOS loopback/kqueue baseline for clients waiting between requests, excluding external networking and TLS.
• Durable mixed traffic. For 30 seconds at 1,158 RPS, the mix was 94% GET, 5% authenticated POST, and 1% misses. It passed with zero drops and p99 6.39ms. Acknowledged writes were checked afterwards, including across a SIGKILL restart. Five percent POST means roughly 58 writes a second in this mix. The run gives no basis for claiming 1,158 durable writes a second. A 2× burst at 2,315 total RPS also passed, with p99 8.78ms.
• A larger working set. With 100,000 seed URLs and a 10% miss rate, the saturation result was 13,170,223 RPS.

September 14: the $4 Linux VPS

The VPS had four KVM vCPUs and cost $4 according to its owner. For the September 14 comparison, it ran Linux 5.4.0-208 and Rust 1.98.1. An isolated server listened on 127.0.0.1:18080 with 10,000 seeded links and 32 connections. I pinned the server to CPUs 0–1 and the client to 2–3, then alternated baseline and candidate runs. Across three eight-second runs, the baseline medians were 951,012 RPS at pipeline depth 128 with 48.2–48.6ms p99 batch latency, and 32,835 RPS without pipelining with 1.5–3.1ms p99. Every controlled run passed the drop, transport-error, and redirect checks.

Batching atomic-counter updates was the first experiment. In the unpinned pipeline-128 comparison, the baseline reached 1,034,168 RPS and the candidate 1,013,335 RPS, about 2.0% lower. The initial pipeline-1 samples overlapped a build, so I excluded them from the comparison.

The second candidate aligned shard locks to cache lines and borrowed the URL while copying the redirect response, avoiding an Arc clone and drop. In the pinned tests it reached 986,427 versus 951,012 RPS at depth 128, a 3.72% gain. At depth 1 it lost 1.11%. That candidate's depth-1 p99 also ranged from 2.3 to 21.8ms, compared with 1.5–3.1ms for the baseline. Eight-second samples on a shared host gave me no reason to prefer a change that won one workload and lost the other. I restored the production binary and configuration; two regression tests from the work remained.

Public redirect latency was a separate problem. Three curl requests from the laptop measured time to first byte without following the 302. Against the VPS's HTTPS endpoint, https://45.196.196.251/rushort/0, they took 302.5, 279.7, and 292.9ms. Against https://whiteye.in/s/0, they took 5,451, 699, and 685ms. Even the two faster website samples were much slower than the direct VPS samples. These are six individual observations, not a latency distribution, and they don't isolate cold starts, TLS, geographic routing, or server time. The extra upstream fetch makes caching immutable mappings or serving the redirect domain through Caddy worth investigating. Neither change was measured here.

September 26: a different VPS configuration

The September 26 rerun used the same four-vCPU host on Linux 5.15, with an isolated RAM-only server and 1,000 preloaded URLs. These runs were unpinned. The synthetic traffic generator was paused during each suite and restored afterwards; production kept running. At 32 connections and pipeline depth 512, three eight-second saturation runs reached 1,383,376–1,428,007 RPS, giving a 1,418,510 RPS median. They had zero drops, errors, or mismatches and 58.1–59.4ms p99 batch latency. A configuration sweep also recorded a best five-second result of 1,465,138 RPS at 16 connections and depth 256.

Scheduling arrivals at a fixed rate exposed a tighter limit. At a 1.3M RPS target, each of three runs verified all 13,000,000 redirects in 10.03–10.07 seconds. Achieved throughput was 1.291–1.296M RPS and p99 batch latency was 116–264ms. A 1.2M target also passed: 12,000,000 correct redirects at 1,199,289 achieved RPS, with 46.9ms batch p99. At a 1.4M target, all 14M responses were correct, but completion took 10.62 seconds. The achieved 1,318,548 RPS fell below the runner's 99% requirement, so it failed; batch p99 had grown to 904ms.

The method and raw results record source baseline ec01ce7, also used on September 14. The earlier 951k test had a different kernel and CPU pinning, with more seed URLs and a shallower pipeline. Its difference from 1.42M doesn't establish a code speedup. Both are VPS loopback results; the 16.8M result belongs to the M4 Pro.

September 30: trying the borrowed lookup again

Results note added October 5, 2026. In the September 30 comparison, I tested borrowing the redirect URL under the shard read guard again. This candidate also included monitoring changes. On Linux 5.15 with four shared VPS CPUs, the server was pinned to 0–1 and the verifying client to 2–3. Each workload used three alternating baseline/candidate repetitions of eight seconds, against isolated ephemeral state on port 18080. The synthetic traffic service was paused and restored; the live service kept running.

With 64 connections, no pipelining, and 10,000 URLs, median throughput moved from 34,423 to 34,566 RPS (+0.42%). With 32 connections and depth 128 over 10,000 URLs, it moved from 1,045,055 to 1,028,081 RPS (−1.62%). At depth 512 over 1,000 URLs, again with 32 connections, it moved from 1,341,131 to 1,333,296 RPS (−0.58%). All 18 completed runs passed with zero drops, transport errors, unexpected statuses, or redirect mismatches. A fourth workload with 100,000 URLs exceeded the setup timeout and was excluded.

I removed the borrowed lookup: a 0.42% round-trip gain alongside losses in both pipelined workloads didn't justify it. What stayed was a one-second cache for /api/host, with system inspection moved off tokio workers, and a monotonic uptime_ms counter for sampling metrics. Those changes address monitoring work and timing; this comparison establishes no redirect-throughput improvement.

what these tests leave out

The generator shared the server's CPU in every throughput test. None included a physical NIC or TLS. The M4 Pro's 10M and 16.8M rates also required 128-deep HTTP pipelining.

The ten-second M4 Pro fixed-rate result came from a fresh server; repeated runs could throttle thermally and drop requests, which the runner treated as failures. Multiplying that rate into roughly 864 billion requests per day adds no evidence about sustained operation. The durable mix ran for 30 seconds and checked process recovery; it tested neither power loss nor replication. I still need a longer run with a specified workload to answer the original daily-capacity question.

The public dashboard includes a continuous synthetic load generator. Its charts show that traffic as well as public requests; they aren't visitor counts.

what I kept

The batched socket path stays. The smaller lookup and counter changes didn't improve the tested workloads consistently enough to keep. Given the gap between pipelined and unpipelined throughput, I'll record the client configuration alongside every result, even when the server code is identical.

For the public demo, I'd measure the website redirect path next. The hundreds of milliseconds in those curl samples are a different scale of problem from a few percent in the in-memory lookup. Timing the upstream fetch separately would help establish how much delay removing that hop could actually save.

reproducing the VPS workload

The September 26 notes record baseline ec01ce7, binary hashes, and setup. Use that baseline to reproduce the measurements; the source links above use later revision ff72c9c, including retained monitoring changes. Start the isolated RAM-only server in one terminal. Run the eight-second saturation test, then the ten-second fixed-rate test, in another. Avoid unrelated load and record any production traffic that remains.

target/release/shortener --bind 127.0.0.1:18080 --ephemeral --preload 1000

# Separate terminal: closed-loop saturation, then fixed arrivals.
target/release/loadgen --target http://127.0.0.1:18080 --preloaded \
  --mode saturate --duration 8 --connections 32 --pipeline 512 --seed 1000

target/release/loadgen --target http://127.0.0.1:18080 --preloaded \
  --rps 1300000 --duration 10 --connections 32 --pipeline 512 \
  --queue 128 --seed 1000

Run the full suite with ./bench.sh --repeats 3 --seconds 30. Keep its generated CLAIMS.md and raw logs with the result, including any failed runs. Report the actual elapsed time and the response checks alongside throughput so someone else can decide whether the workload resembles theirs.