Anatomy of an Automated Manga Translation Pipeline

Scanlation has a well-worn manual workflow: clean the bubbles, transcribe the source text, translate, re-typeset. Each step is tedious and each chapter repeats the last. My Comic Translator project compresses that loop into a single CLI run — drop in a folder of pages, get back typeset pages — processing roughly 20 pages in about 3 minutes.

This post walks through the parts that actually took iteration: erasing old text without leaving scars, reading it reliably, and putting new text back so the page still looks like manga.

Erasing text: dilation beats rectangles

The obvious approach — detect text regions, fill their bounding boxes — fails immediately on real pages. OCR bounding boxes hug the detected glyphs, and glyphs like descenders, tails, and punctuation edges routinely escape them. Fill the rectangle and you clip half a character into the bubble wall.

The fix is embarrassingly simple: dilate. Expanding each detected region with cv2.dilate adds a safety margin around every box before inpainting, which pushed erase quality to effectively 100% — no more ghost strokes peeking out after cleaning.

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9, 3))
for box in ocr_boxes:
    x1, y1, x2, y2 = box
    x1, y1 = max(0, x1 - PAD), max(0, y1 - PAD)
    x2, y2 = min(w, x2 + PAD), min(h, y2 + PAD)
    region = bubble[y1:y2, x1:x2]
    # dilate the text mask so glyph edges are fully covered
    mask = cv2.dilate(text_mask(region), kernel, iterations=1)

Fill color: sample, never assume white

The second scar-generator was assuming bubbles are white. Real pages use off-white paper tones, aged scans, yellowish screentones, and outright colored bubbles. Filling with pure #FFFFFF produces ugly rectangular patches that scream "processed".

Instead the cleaner samples pixels along the outer perimeter of each text region and fills with the dominant color. The patch inherits whatever the artist actually used — which also survives JPEG noise far better than a flat flood fill.

Reading order and OCR

PaddleOCR does the recognition, but detection alone doesn't give you translation-ready text — you need reading order. Manga panels flow right-to-left, top-to-bottom, and dialogue within a bubble needs line grouping. Sorting detections by position and grouping lines into logical blocks turned out to matter more for translation quality than any OCR parameter tuning.

Translation is delegated to an LLM call per block, which conveniently preserves context across a conversation-heavy page better than sentence-by-sentence APIs.

Typesetting with Pillow

Typesetting is where most homegrown pipelines look obviously fake. Three rules got mine past that: auto-fit font size by measuring rendered width against the bubble's inscribed ellipse, wrap conservatively (manga bubbles read best with short ragged lines), and center vertically on the cleaned region, not the erased box.

for size in range(max_size, min_size, -1):
    font = ImageFont.truetype(FONT_PATH, size)
    lines = wrap_text(translation, font, max_width)
    if total_height(lines, font) <= bubble_height:
        break
draw_centered(draw, lines, bubble_center)

Results and what I would do differently

If I rebuilt it today I would try a dedicated comic speech-bubble detector instead of leaning on OCR boxes for segmentation, and move inpainting from OpenCV's basic fill to a small diffusion-free inpainting net for textured backgrounds. But the core insight holds: the wins came from boring geometry — margins, sampling, fitting — not from bigger models.

• ~20 pages per 3-minute run, fully hands-off including typesetting
• Batch mode processes complete chapters without per-page tuning
• Dilation margins fixed ~all residual text scarring
• Perimeter color sampling eliminated the white-patch look entirely