We’re working on bringing Naji — the horse behind the name — home to Jacksonville.Bringing Naji home to Jacksonville.Meet Naji

BlogengineeringChris Lane Jonesupdated

We moved the engine off the main thread. The pixels stayed put.

How Image Horse moved its Rust engine into a back room without ever sending a frame across the wall, and why the obvious way to do it is impossible.

Shipped as the default in v8.32 on September 18th, 2026. Every measurement below comes from the repository's own records: ADR-024, the feasibility note that opened the question, and the transferred-canvas finding that closed the last objection.

Listen narrated 15 min

0:0015:00

Picture a Lowe's with one employee. He runs the register, answers the phone, restocks the shelves, and walks you to aisle 14 for the rubber mallet, and he does all of it in order, one thing at a time, so when somebody at the Pro desk asks him to cut forty pieces of lumber, everyone in line at the register just stands there and waits. That's a browser tab. The one employee is called the main thread, and until August he was also the one doing the heavy photo work in Image Horse.

The heavy photo work is done by a program called stamp_tool, written in Rust and compiled to run in the browser. Every blur, every export, every batch job ran on the same thread that draws your brush strokes. A big operation froze the page for 129–137 ms, and your stroke stuttered while it did.

The obvious fix is a second employee. Browsers have one, it's called a Web Worker, and Image Horse already used one for squeezing photos into WebP and JPEG. That one works because what it's handed is small, or already shaped like a plain block of bytes. The engine is a different animal. The reason it's fast is that the screen reads straight out of the engine's memory, no copy in between. This post is about what happens to that trick when you put a wall between the engine and the screen, and what I did instead.

FIG 1Where things live now. The engine, its memory and the drawing surface all sit in the back room. Requests go in with a ticket number and get an answer back; the per-frame blit, the "put the picture on screen" call, skips the line entirely.

Three doors, one wall

There are exactly three ways to get something from one thread to another, and the browser picks one based on what you hand it. Think of it as moving a filing cabinet to the back room.

How each door behaves. The cost lands on the sending side.
DoorMechanismCostSender keeps it?
CopyStructured clone: photocopy every pageGrows with the sizeYes, two separate cabinets
MoveTransfer list: hand over the keysO(1), the same cost at any sizeNo, the old handle is dead and reading it throws
ShareSharedArrayBuffer: a joint accountFree after the first timeYes, the same bytes, seen from both rooms

Move is the door you want for anything big: ownership changes hands and nothing gets copied. It works by detaching the memory, unhooking it from the sender and rehooking it in the receiver. And that's exactly where the engine hits a wall.

The engine's memory can't be detached. The spec stamps it with an internal mark that postMessage can't carry, so the hand-off fails (whatwg/html #4601). It's not a filing cabinet you can wheel down the hall. It's the floor the engine is standing on.

FIG 2The three doors, and the wall. Copy duplicates and pays by the byte. Move is instant and leaves the sender empty-handed. Share is one block visible from both rooms. The engine's memory is chained to its room: it strains and snaps back.

Why the pixels can't make the trip

Fine, copy them then. Copying is the default door and it isn't free. Surma measured it across five devices and browsers and found the time tracks the size of what you send almost exactly (Surma, Is postMessage slow?). Two budgets fall out of that.

How much you can copy per message and still feel smooth, even on a slow phone.
If you are…Time budgetMax size
Responding to a click100 ms100 KiB
Drawing one frame of an animation16 ms10 KiB

Now put one frame of an Image Horse photo against that.

Bytes per postMessage, log scale1 KiB → 64 MiB

Pointer event{ tool, x, y, pressure }~60 B
Frame budget16 ms10 KiB
Interaction budget100 ms100 KiB
1920×1080 RGBA frame7.9 MiB810× over
4096×4096 RGBA layer64 MiB6,554× over

The line sits at the 10 KiB per-frame budget. Every bar past it is a frame you drop.

FIG 3Copying the pixels on every stroke is off the table before we start. One HD frame is 810 times the per-frame budget, and handing them over is forbidden by the wall above. The pixels can't make that trip at all.

So the canvas moved instead

If the pixels can't come to the screen, the screen goes to the pixels. The browser lets you take the drawing surface out of a canvas element (transferControlToOffscreen()) and hand that to the back room. One message, once per canvas, and from then on the worker draws onto it directly. The element stays on the page, where the layout still sizes it and the overlays still line up with it. The front counter just can't draw on it anymore.

That sounds like more work and it's less. The worst piece of code on the old path, the one that read from the engine and wrote to the screen on every frame, doesn't need a careful rewrite. It just disappears. And every argument against moving the canvas got measured before anyone believed it: zoom and pan survived the move 11 out of 11 times across four browsers, the overlays stayed pinned 9 out of 9, and the low-latency drawing mode kept working 4 out of 4.

Three things had to move together for that to hold.

What lives where, after the move.
StepBeforeIn the worker
Building the engineBuilt at the front counter, then handed overBuilt in the back room, where it runs. One engine, one history.
Combining the layersfront counterback room
Resizing the canvascanvas.width = wthe worker resizes the surface it owns
Putting the picture on screenread engine memory → copy → drawsame code, in the back room, against its own memory
Crosses the wallnothingone "go" message per frame, and nobody waits for a reply
FIG 4Yours to drive: play, scrub, or jump to any of the four beats. The surface leaves the page element and lands next to the engine; from then on pixels flow memory → screen without leaving the back room, and the only per-frame traffic is a "go" that nobody waits on.

The back-room draw is the same dozen lines the front counter used to run, with the same two warnings attached. It has to look up where the pixels are on every call, because when the engine asks for more memory it gets a new block and every old pointer into the old one goes dead. And it copies the pixels into a private buffer before drawing, because drawing straight out of live memory while the engine is still writing to it is the "garbage after five strokes" bug Firefox taught me about.

// engine.worker.ts: fire-and-forget, not routed through the queue
function blit() {
  if (!tool || !surface) return;
  tool.recomposite();
  const view = new Uint8ClampedArray(wasmMemory.buffer, tool.data_ptr(), tool.data_len());
  backbuffer.data.set(view);          // copy once, worker-side
  surface.getContext("2d").putImageData(backbuffer, 0, 0);
}

Notice what's not in there: no ticket number, no reply, no line to wait in. Drawing the picture changes nothing about the photo, so making it wait behind pending edits would delay what you see without protecting anything, and the round trip it would add is the exact lag this whole project exists to remove.

One line, in order

Everything that does change the photo goes through a single door and waits its turn in a strict line. That's not caution for its own sake. Image Horse's undo history is a ledger of edits, and the ledger records them in the order they arrive; no edit carries its own number. So the order messages arrive in has to be the order in the ledger, and that's only true while every edit goes through one door, one at a time. Picture a deli counter that hands out tickets but has three windows calling numbers at once: everyone gets served, nobody gets served in order, and now the ledger says you blurred the plate before you cropped the photo when you did it the other way round, and undo quietly takes back the wrong thing.

Each request carries a ticket number that only ever counts up, so two requests in flight can't grab each other's answers; a request that's been overtaken can be canceled before it runs; and if the engine crashes, the request fails loudly instead of hanging forever. The first rough draft of this had none of those.

Main thread16 calls, no await between them
Worker queuedrain() · FIFO · one at a time
OpLog::appendarrival order · no sequence numbers
FIG 5The gate that can say no. Edits sent without waiting land in the worker's line in the order they were sent and drain one at a time into the ledger; a canceled ticket is refused, not skipped. Run for real with 16 edits in flight at once, the back room's ledger came out byte-for-byte identical to the old front-counter one.
Gate 3, the pile-on: 16 edits fired at once without waiting, then 6 more aimed at their ticket numbers. Waiting politely for each one would have proven nothing.
MeasurementFront counterBack room
ticket numbers returned1..16 in order1..16 in order
edits · bytes7 · 9107 · 910
ledger fingerprintea77112d…ea77112d… identical

A rule like "all edits go through the one door" is the wrong kind of thing to remember and the right kind of thing to test, so there's a test that fails if anyone opens a second door, builds an engine somewhere else, or sneaks around the seam. All three were checked by breaking the code on purpose and watching the test go red.

The rule also needed a correction five days in, and the correction is the interesting half. Two parts of the app legitimately build their own engine, batch export and the settings panel behind it, for a photo you are not currently editing. Sending those through the main door would put their edits in the open photo's ledger, so undo would replay changes to a photo you never opened, and a forty-photo batch would stand in line behind your one open image. The first version of the rule said "every edit," a test enforced exactly that, and the easiest way to make the test pass was to introduce the bug. The rule had to get more specific, not the code more obedient.

One more thing the door had to get right. The front counter talks to the engine through a stand-in that forwards every call to the back room. A stand-in that says "sure, I have that" to every question would make every "does this build support X?" check in the app answer yes, for every X, forever. So at startup the worker reads the real engine's list of functions and sends it over, and the stand-in forwards only those names. A list the engine wrote about itself can't drift from the engine, and the test that asks for a function that doesn't exist and gets a "no" back is the row that proves the gate is live.

The IOU problem

Here's the part of this project that cost me the most sleep, and it never once produced an error message. When a call moves to the back room, you no longer get the answer back immediately. You get a promise, which is the JavaScript word for an IOU: "the answer is coming." If you've ever looked at derivatives, you already know the trap. A futures contract for a thousand barrels of oil is not a thousand barrels of oil, and if you check your warehouse by asking "do I have a piece of paper about oil?" the answer is always yes. The Dutch found that one out with tulip contracts in 1637 and it's been a reliable way to lose money ever since.

In code, a guard like if (!tool.has_source()) reads fine after the move. But if the await is missing, the thing being checked is now an IOU, and an IOU is never "nothing," so the guard always passes. Nothing throws, nothing logs, and the tool quietly does the exact thing the guard existed to prevent.

The variants got worse the deeper I went. Turn an IOU into a list and you get an empty list, which also counts as "something," so a check written specifically to catch a missing value could never fire, and the math downstream turned a whole batch into NaN. One engine call reports whether a layer had text to bake in; un-awaited, it reads as "yes" after the first layer, so every export would have told you your redo history had been thrown away.

TypeScript doesn't save you here, and the reason is worth writing down. A function that returns an IOU-for-a-number is allowed to sit in a slot that says "returns nothing," because an IOU you're going to ignore is, technically, nothing. One of the pen tool's handlers sat in exactly that kind of slot, so the move would have type-checked clean while a typeof id === "number" check silently went false and pen paths stopped staying selected.

A void return is not proof that nothing is consumed.

So I ratcheted it, the same way I ratcheted the file sizes in the entropy post. A test pins the number of calls that still read the answer the old way, and that number may only go down; a script, not a hand-written list, is the authority on what's left; and every batch was checked by putting the bug back in and confirming a test turned red. Twice the tests caught something the script structurally couldn't see, because the script reads the line where the call is made, and a guard one line below it is invisible from there.

The numbers

Speed was never the obstacle. I measured the wall between the rooms before writing a line of the move, and the real obstacle turned out to be a counting problem.

Latency is not the obstacle. The obstacle is 117 synchronous reads.

docs/engine-worker-feasibility.md
Measured, not estimated. Medians unless stated.
MeasurementResult
Round trip to the back room and back, empty message, 50 tries0.100 ms median · 0.300 ms worst-of-20 · 0.6% of a frame
Cold sharpen on a 48 MB photo, front counter vs back room468.6 ms → 483.6 ms (1.03×)
Drawing one frame, back room vs front counter22.1 ms vs 23.9 ms
Back-room warm-up, cold vs warmed up ahead of time715.4 ms vs 392.1 ms
Places in the app that call the engine206 total: 77 don't need an answer · 117 read the answer · 12 on the hot path
Shared memory, special headers, threaded WebAssemblynone needed
Page freeze per heavy operation129–137 ms → 0

The third door, shared memory, never got opened. It was tested and would have worked, but a back room that owns its own engine and swaps ownership of a few big things needs none of it, and this project had already tried multi-threaded WebAssembly once and dropped it at 8 to 31 times slower than the plain version. The simplest setup that could work is the one that shipped.

Six stages, each one shippable

The plan was staged on purpose. Stages one and two added no worker and changed nothing you could see: they put every engine call behind a single door and removed the places that read and wrote engine memory in the same breath. Those were worth shipping even if everything after them had been abandoned, which is the point of doing them first. Stage three built the back room and left it switched off, and the build didn't even include it, because nothing used it yet. That got written down as the honest status rather than smoothed over.

Every stage could be undone on its own, and the whole thing sat behind a switch that stayed off for five weeks. It became the default on September 18th, 2026, in v8.32.

See it yourself

The wall is a claim from a 2019 spec discussion, and browsers move. Paste this into the console of any page, in each browser you care about, and see which door it takes.

const m = new WebAssembly.Memory({ initial: 1 });
try {
  // structuredClone uses the same transfer machinery as postMessage
  structuredClone(m.buffer, { transfer: [m.buffer] });
  console.log("copied silently, byteLength still", m.buffer.byteLength);
} catch (e) {
  console.log("threw:", e.constructor.name, e.message);
}

And in the editor itself, the back room is the default for every photo. Add ?ih_engine_worker=0 to the URL to put a tab back on the front counter, run a blur on a big photo in each, and watch the pointer. It takes effect on the next load, like every kill switch in this app. It's there because a change this big should be something you can undo from your own browser without waiting on me for a release.

Open the betaRead engine.worker.ts

Sources: Phase 0 feasibility · a12 design and gates · ADR-024 · Surma, Is postMessage slow? · whatwg/html #4601 · MDN, Transferable objects

The engine works in the back room now. The tab just paints.