Running LaMa Inpainting in the Browser — WebGPU First, WASM Where It Counts

Most "AI image editors" are a file input in front of a GPU cluster. Your photo leaves your machine, gets processed somewhere you have no visibility into, and comes back. For background removal and object cleanup — operations people run on personal, sometimes sensitive images — that has always felt like the wrong trade.

Inori Studio is my attempt at the alternative: a browser image editor where background removal and inpainting run entirely on-device. Zero bytes leave the machine. No API keys, no per-request cost, no upload spinner on rural connections. This post is about the harder half of that problem — getting the LaMa inpainting model running smoothly across real-world browsers and GPUs.

Picking the model and the runtime

LaMa (Large Mask Inpainting) remains one of the best open choices for object removal. Its Fourier Convolutions (FFC) give it unusually good results on large masks and repetitive structures — fences, hair, text — which is exactly what people delete.

For execution, ONNX Runtime Web is the pragmatic bridge: export the model once, then choose between the WebGPU execution provider (fast, GPU-resident) and the WASM/CPU path (slow but universal). The plan was simple: WebGPU when available, WASM as fallback.

const session = await ort.InferenceSession.create(modelUrl, {
  executionProviders: hasWebGPU ? ['webgpu'] : ['wasm'],
});

Where WebGPU fell short

The first surprise was that LaMa is not a friendly citizen of the WebGPU execution provider. The FFC blocks rely on FFT-style operations that compile to shader pipelines some browser/GPU combinations simply fail to build — the same session creation that works on one Chrome/GPU pair crashes with shader compilation errors on another. Debugging across driver versions is not how I want to spend an afternoon, and users do not care whose bug it is.

The fix was to stop treating WebGPU as all-or-nothing. Background removal (rembg via WebGPU) stays fully on the GPU path — it compiles cleanly everywhere we tested. LaMa runs inside a Web Worker on the WASM execution provider instead. Yes, CPU inference of an inpainting network sounds slow; in practice, constrained correctly, it is fast enough, and it never white-screens someone's browser.

The memory trap: never resize the whole frame

The naive integration resizes the entire image to the model's input size, runs inference, and scales the result back. On a 4K photo that means allocating giant tensors, losing detail, and multi-second stalls — sometimes OOM tab crashes on mobile.

Instead, crop only the masked region. Take the mask's bounding box, pad it for context, letterbox that crop to 512×512, run the model on just those pixels, then composite the result back into the original resolution frame. Memory usage drops by orders of magnitude because tensor size tracks the edit, not the photograph.

// Pseudocode: the worker hot path
const box = maskBoundingBox(mask, { padding: 32 });
const crop = cropWithLetterbox(image, box, { size: 512 });
const output = await lamaSession.run(crop.tensor);
compositeBack(image, output, box); // un-letterbox + blend edges

Keeping the UI alive

All inference lives in workers — decode, masking, both model sessions. The main thread only handles Konva canvas interactions and receives progress messages. Even during a long inpaint, zoom/pan stays at full framerate, which matters more for perceived quality than shaving 200ms off the model itself.

What this buys

The broader lesson: "browser AI" is not one runtime decision but a routing problem. Per-model, ask whether WebGPU actually compiles for it, and keep a WASM path that is merely okay rather than perfect. A tool that always works slowly beats one that usually works fast.

• Zero uploads — privacy by architecture, not policy page
• No server costs, no rate limits, no cold starts
• Works offline once assets are cached
• ~2s background removal on consumer GPUs; inpainting bounded to 512×512 crops regardless of image size