fix(extractor): keep sparse columns from sharing a baseline by mangeshraut712 · Pull Request #602 · firecrawl/pdf-inspector

mangeshraut712 · 2026-09-27T13:49:27Z

Fixes #482.

Sparse two-column lines were fused because the gutter is too narrow for column detection, and the same-baseline split only fired for a lowercase continuation or a bold/regular mismatch. Same-style clauses, a one-word wrap sitting a column-width void away, and a right-aligned tag beside a title all stayed on one line.

Checked against the sample in that issue (en-22-mixed-layout.pdf):

Left column: record source type, page count, and at (42, 584) and Right column: inspect tables, images, links, and reading at (318, 584), both 10pt. The gap is about 36pt, under a column split and over a word space. Both runs are full clauses, so they now become two lines. A short same-style label row (two words, ~50pt apart) stays one line.
checksum. at (42, 569) and order. at (318, 569) are one word each, about 230pt apart. A void wider than 8em and 100pt is no longer treated as a word space. Table header cells in the same file (Status / Metric / Value, about 70pt apart) stay one row.
Mixed layout report at (42, 755), 16pt, and EN-22 at (544, 756), 9pt, are 1pt apart in y and hundreds of points apart in x. The tag is its own line. A digit page number on a title line stays joined.

cargo test --offline --lib layout::tests — 44 passed. cargo clippy --offline -- -D warnings is clean.

Summary by cubic

Fixes #482: sparse two-column lines sharing a baseline were fused when the gutter was too narrow for column detection. The same-baseline split now fires in more cases, works for right-to-left streams, and no longer breaks a body line when its middle run arrives later.

Gutter distance is measured symmetrically, and only runs continuing in reading order split, so a symmetric gap cannot break a body line before the missing run arrives.
Gutter width comes from the measured layout, so a zero or negative width no longer invents a gap.
Same-style clauses split only when both runs are 6+ words and the gap is real (50pt or 5× font size); short label rows and small holes in justified lines stay joined.
Gaps wider than 8em and 100pt split one-word leftover lines and two short column-end runs, but three or more same-style header cells (1–3 words per cell) stay on one row, even across a column-width gap with one narrow gap or an interrupting lower line.
The header lookahead stops at the next baseline and counts a later cell only where grouping would append it: fonts may differ within the void threshold, but a later cell across a large void must match the font of every cell already on the row, so a rejected cell cannot glue the row together.
A front-matter roman folio (i/v/x numerals, 1–39) stays on its contents entry below 300pt, including punctuated titles like Why? and numbered titles like 1. Introduction.; a wrapped fragment beside iii and roman-shaped words like mix still split.
Right-aligned tags split from a title; digit page numbers stay on the title line.
Adds regression tests covering each split and non-split case.

^{Written for commit 8215f90. Summary will update on new commits.}

Same-style column clauses and a far right-aligned tag were joined when the gutter was too narrow for column detection.

A right-to-left stream still splits on the same gutter, a roman folio stays on its entry, and a small hole in one justified line stays one line.

A zero or negative measured width was pushing the run's right edge the wrong way and splitting one line. Front-matter folios stay on their entry; a roman-shaped word such as "mix" still splits.

cubic-dev-ai

All reported issues were addressed across 1 file (changes from recent commits).

_{Shadow auto-approve: would not auto-approve. Auto-approval blocked by 1 unresolved issue from previous reviews.
Tip: cubic used a learning from your PR history. Let your coding agent read cubic learnings directly with the cubic MCP.

Re-trigger cubic}

A roman token stays on the entry only across a contents-leader gap. The sample column gutter beside "iii" still splits.

cubic-dev-ai

All reported issues were addressed across 1 file (changes from recent commits).

_{Shadow auto-approve: would not auto-approve. Auto-approval blocked by 1 unresolved issue from previous reviews.

Re-trigger cubic}

A contents entry keeps its roman folio below 300pt. A wrapped fragment such as "checksum." still splits from "iii".

A title such as "Why?" stays with its roman page number. A short sentence fragment such as "checksum." still splits from "iii".

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would not auto-approve. Auto-approval blocked by 1 unresolved issue from previous reviews.

Re-trigger cubic}

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would not auto-approve. Auto-approval blocked by 1 unresolved P0–P2 issue from previous reviews.

View guided diff | Re-trigger cubic}

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would not auto-approve. Auto-approval blocked by 1 unresolved P0–P2 issue from previous reviews.

View guided diff | Re-trigger cubic}

Same-style header cells stay together across a column-width gap. Wrapped leftovers and a different-size tag still split. Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

devin-ai-integration

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

Devin Review found 1 new potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

devin-ai-integration · 2026-10-12T06:12:19Z

group_single_column

+                    if gap > (item.font_size.max(last_item.font_size) * 8.0).max(100.0)
+                        && continues_line
+                        && !folio_on_entry
+                        && !same_style_header_row(last_line, &item)


🟡 Short column fragments fuse across wide gaps

When two sparse columns end with short unpunctuated runs, same_style_header_row treats them as header cells. It suppresses the wide-gap split, joining unrelated column text into one extracted line.

Learn more

The single-column grouping path also handles pages where column detection found no reliable gutter, so one input stream can contain text from both columns. The wide-gap rule in group_single_column previously separated alphabetic runs over 100pt apart when they continued in reading order. The new header exception uses only word counts and approximate bold/font-size agreement, which also describes independent column leftovers with no sentence-ending period. Those leftovers now share a TextLine and are rendered in one textual line.

Example: On a page with no detected columns, Summary at x=42 and Results at x=318 share a baseline, 10pt regular font, and each has a narrow width. The gap exceeds 100pt; the old rule made two lines, but same_style_header_row now makes Summary Results one line.

Recommended fix: Restrict the exception using evidence that these are cells of the same table or label row, rather than treating all short same-style runs at arbitrary distance as headers. Keep the wide-gap split for ambiguous single-line sparse-column fragments; add a regression case for two short non-period-terminated column leftovers alongside the wide-header case.

Was this helpful? React with 👍 or 👎 to provide feedback.

…e row Two short unpunctuated runs a column apart are column leftovers, so the void split still separates them. Three same-style header cells stay one row. Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

A cell boundary is the clause gutter, not the 100pt void, so a header whose first gap is under that void still counts as three cells. The lookahead stops at the next baseline. Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would require human review. The diff is truncated before the core same-baseline split logic and its regression tests, so the claim that these heuristics are bounded to sparse two-column cases cannot be verified.

View guided diff | Re-trigger cubic}

A third same-baseline header cell still confirms the row when a few later baselines sit between it and the cells already seen. The scan stops after four other baselines, so a distant short label does not join two column leftovers. Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would require human review. Fixes sparse two-column lines sharing a baseline with new gutter/void/folio heuristics and regression tests, but the core split logic and most tests are truncated in the diff, so the boundedness of the behavior change is unverified.

View guided diff | Re-trigger cubic}

A later cell on the same baseline cannot join the line once a lower row is current, so it is not evidence for keeping the cells already seen together. Two short column leftovers still split. Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would require human review. Fixes same-baseline sparse two-column fusion with new gutter/void/folio heuristics and regression tests; the core split logic and most tests are still truncated, so the behavior change's boundedness is unverified.

View guided diff | Re-trigger cubic}

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would require human review. Extends same-baseline column splitting with gutter, void, and roman-folio heuristics plus header-row lookahead; most core split logic and regression tests are truncated, so the behavior change's boundedness remains unverified.

View guided diff | Re-trigger cubic}

… out-of-order gap from splitting one The header lookahead counts a later cell only when every cell already on the row matches that cell's font. The prose gutter split runs only for a run that continues in reading order, so a symmetric gap cannot break a body line before the missing run arrives. Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

devin-ai-integration

Devin Review found 2 new potential issues.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

devin-ai-integration · 2026-10-14T08:35:04Z

+                    let continues_line = (item.x > last_item.x) != page_rtl;
+                    if line_wordy
+                        && incoming_wordy
+                        && continues_line


🟡 Reversed prose line splits at justified gap

When an LTR stream emits the right fragment first, column_boundary splits a lowercase prose fragment across an unfilled 31–50pt gap. A justified single-column line becomes two lines.

Learn more

The prose split applies once the gap exceeds 30pt, even though the separate same-style clause threshold permits gaps up to 50pt in one justified line. column_boundary now treats a backwards step without a third run in its gap as a column transition. In stream order, that same step can simply be the two fragments of one line emitted right-first. The previous direction guard did not split such backwards steps.

Example: On an LTR page, a right run at x=275 reads “before the close of the year”; a left run at x=40 ends at x=240 and reads “the report continues with these figures”. Emit the right run first on the same baseline. The 35pt gap has no filler, so the left run starts a second line instead of joining the sentence.

Recommended fix: Keep the backwards-step column detection for clear column gutters, but distinguish those from the smaller justified spaces accepted by clause_gutter. Add a reversed, same-baseline prose-line test with a 31–50pt gap.

Was this helpful? React with 👍 or 👎 to provide feedback.

devin-ai-integration · 2026-10-14T08:35:06Z

+        let accepts_row = cells.iter().all(|cell| {
+            let scale = cell.font_size.max(next.font_size).max(1.0);
+            (cell.font_size - next.font_size).abs() <= scale * 0.2
+        });
+        if !accepts_row {
+            break;
+        }


🟡 Unjoinable header label joins earlier cells

When a differently sized third label starts lowercase prose, same_style_header_row counts it within the void limit. The prose split rejects that label later, leaving two distant header cells falsely joined.

Learn more

The lookahead decides whether a third run can preserve an earlier pair across a wide void. A font mismatch within the next run's 100pt void limit no longer stops that lookahead. But the actual grouping step also has a prose split at gaps over 30pt; a three-word lowercase label can trigger it without reaching the void rule. Counting that run as a third cell keeps the earlier pair together even though it cannot join them.

Example: On one baseline, use Status at x=47 with width 30 and 8pt font; Description at x=200 with width 50 and 10pt font; and the current owner at x=330 with 12.5pt font. The 80pt gap to the last label passes this font guard. The first two labels join across 123pt, then the third starts another line under the prose rule.

Recommended fix: Before counting an upcoming header cell, apply the actual merge conditions from group_single_column, including the prose split against the growing line. Share the predicate or test prospective joining in order so the lookahead cannot count a cell that grouping rejects.

Was this helpful? React with 👍 or 👎 to provide feedback.

…eader A backwards stream step is a column boundary unless a same-baseline run fills the gap, so two complete clauses still split when the right one arrives first. Y-sorting walks a baseline left to right, including on an RTL page. A header lookahead cell whose font differs from an earlier cell still counts when grouping would append it. Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would require human review. Revises same-baseline line/column splitting heuristics in layout.rs with gutter, void, and header-row lookahead changes; core logic and regression tests are truncated, so the boundedness of the behavior change remains unverified.

View guided diff | Re-trigger cubic}

…rom review - Require the column-gutter threshold before splitting right-first prose across a backward stream step (31/35/50pt gap regressions) - Header lookahead now reuses the prose-split predicate used by grouping - Drop the lookahead-only font-size rejection for cells within the void threshold; keep the mismatched-font check across large voids

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Shadow auto-approve: would require human review. Revises same-baseline line-splitting heuristics in layout.rs with gap, void, folio, and header-row changes; core logic and regression tests are truncated, so the boundedness of the behavior change remains unverified.

View guided diff | Re-trigger cubic}

fix(extractor): keep sparse columns from sharing a baseline

186950f

Same-style column clauses and a far right-aligned tag were joined when the gutter was too narrow for column detection.

This comment was marked as resolved.

Sign in to view

fix(extractor): measure column gutters in either direction

1525583

A right-to-left stream still splits on the same gutter, a roman folio stays on its entry, and a small hole in one justified line stays one line.

This comment was marked as resolved.

Sign in to view

fix(extractor): size column gutters from the layout width

dafd242

A zero or negative measured width was pushing the run's right edge the wrong way and splitting one line. Front-matter folios stay on their entry; a roman-shaped word such as "mix" still splits.

cubic-dev-ai Bot reviewed Sep 27, 2026

View reviewed changes

fix(extractor): keep column gutters from counting as folios

7951bf2

A roman token stays on the entry only across a contents-leader gap. The sample column gutter beside "iii" still splits.

cubic-dev-ai Bot reviewed Sep 27, 2026

View reviewed changes

fix(extractor): keep closer front-matter folios on the entry

28d24be

A contents entry keeps its roman folio below 300pt. A wrapped fragment such as "checksum." still splits from "iii".

This comment was marked as resolved.

Sign in to view

fix(extractor): keep punctuated contents titles with their folios

748b236

A title such as "Why?" stays with its roman page number. A short sentence fragment such as "checksum." still splits from "iii".

cubic-dev-ai Bot reviewed Sep 27, 2026

View reviewed changes

mangeshraut712 and others added 2 commits October 6, 2026 10:40

Merge branch 'main' into cursor/sparse-column-gap-4a36

cfa5292

Apply the column-void split only to runs in reading order

8ecc52f

cubic-dev-ai Bot reviewed Oct 11, 2026

View reviewed changes

This comment was marked as resolved.

Sign in to view

fix(extractor): keep numbered contents titles with their roman folios

ebc4213

cubic-dev-ai Bot reviewed Oct 12, 2026

View reviewed changes

cursoragent and others added 2 commits October 12, 2026 06:09

fix(extractor): keep wide table headers on one row

0ba155a

Same-style header cells stay together across a column-width gap. Wrapped leftovers and a different-size tag still split. Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

Merge branch 'main' into cursor/sparse-column-gap-4a36

1bd6f3e

Co-authored-by: Mangesh Raut <mangeshraut712@users.noreply.github.com>

devin-ai-integration Bot reviewed Oct 12, 2026

View reviewed changes

This comment was marked as resolved.

Sign in to view

cubic-dev-ai Bot reviewed Oct 12, 2026

View reviewed changes

This comment was marked as resolved.

Sign in to view

fix(extractor): measure header lookahead against the line anchor

18b9912

This comment was marked as resolved.

Sign in to view

cubic-dev-ai Bot reviewed Oct 12, 2026

View reviewed changes

fix(extractor): count only header cells grouping can add to the line

688beb9

cubic-dev-ai Bot reviewed Oct 14, 2026

View reviewed changes

devin-ai-integration Bot reviewed Oct 14, 2026

View reviewed changes

This comment was marked as resolved.

Sign in to view

cubic-dev-ai Bot reviewed Oct 14, 2026

View reviewed changes

cubic-dev-ai Bot reviewed Oct 16, 2026

View reviewed changes

Conversation

mangeshraut712 commented Sep 27, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by cubic

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

mangeshraut712 commented Sep 27, 2026 •

edited by cubic-dev-ai Bot

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

devin-ai-integration Bot left a comment •

edited

Loading

devin-ai-integration Bot Oct 12, 2026 •

edited

Loading

devin-ai-integration Bot Oct 14, 2026 •

edited

Loading

devin-ai-integration Bot Oct 14, 2026 •

edited

Loading