Skip to content

Feat: add LoadedPdf handle and PdfDocument handle (wasm) for repeated page processing - #642

Open
ruihe774 wants to merge 3 commits into
firecrawl:mainfrom
ruihe774:feat/loaded-pdf-handle
Open

ruihe774 wants to merge 3 commits into
firecrawl:mainfrom
ruihe774:feat/loaded-pdf-handle

Conversation

@ruihe774

@ruihe774 ruihe774 commented Oct 23, 2026 •

Copy link
Copy Markdown

Currently the whole PDF doc is re-processed for extracting a new page range. This PR adds an API to let the API caller hold a loaded PDF doc so that later extraction does not require re-processing. I did a benchmark with NASA/TM-2014-217388 (release build, NASA PDF, one page per call, markdown asserted identical):

Pages One-shot LoadedPdf
20 752 ms (37.6 ms/page) 52 ms (2.4 ms/page)
197 7.14 s (36.2 ms/page) 249 ms (1.2 ms/page)

LoadedPdf (Rust) and PdfDocument (WebAssembly) open a PDF once and process it any number of times: LoadedPdf::process_pages(options) and new PdfDocument(data, password?).processPages(options) give the result process_pdf_mem_with_options and processPdf give for the same options, without parsing the file again. Detection, the font CMaps and the structure tree are computed by the first call that needs them and kept.

This PR is implemented with the assistance from AI


Summary by cubic

Adds LoadedPdf (Rust) and PdfDocument (WebAssembly) handles that parse a PDF once and process any pages of it any number of times. process_pdf_mem_with_options and processPdf re-parse the whole file on every call even for one page; opening once drops per-page processing from ~36 ms to ~1.2 ms on a 197-page benchmark PDF.

Notes

  • LoadedPdf::process_pages(options) and new PdfDocument(data, password?).processPages(options) return the same result as the one-shot functions for the same options.
  • Detection is cached for the most recent DetectionConfig only; the font CMaps and the structure tree are computed by the first call that needs them and kept.
  • process_pages ignores options.password since the document is already open; page_count, document type, and the OCR page list still describe the whole document.
  • The one-shot functions now delegate to the same pipeline, so their results are unchanged.

Written for commit 118cf04. Summary will update on new commits.

View guided diff

process_pdf_mem_with_options parses the file and recomputes detection, font
CMaps and the structure tree on every call, even for one page. LoadedPdf::open
loads once; process_pages runs the existing pipeline (now a LoadedPdf method
that borrows the document, which the one-shot functions also call) and caches
those document-wide results. Adds PartialEq to DetectionConfig/ScanStrategy and
Clone to PdfTypeResult for the detection cache.
new PdfDocument(data, password?) parses once; pageCount and processPages(options)
reuse the parsed document. Documented in the wasm README and CHANGELOG.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files

Shadow auto-approve: would auto-approve with 3 open P3 issues. Adds optional Rust/WebAssembly PDF handles that parse once and cache document-derived data across repeated page processing, while one-shot APIs delegate to the same pipeline; integration tests pin equivalence with one-shot results.
Reply to a comment to ask cubic a question or push back. It learns from your replies.
Click Fix with cubic on a comment, or reply @cubic-dev-ai fix this, and cubic writes the fix.

View guided diff | Re-trigger cubic

Comment thread wasm/README.md Outdated
Comment thread CHANGELOG.md Outdated
Comment thread CHANGELOG.md Outdated
Fix the undeclared `password` in the PdfDocument README example, reword
the changelog sentence on API equivalence, and clarify that detection is
cached only for the most recent DetectionConfig.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qPK5UkzZdiiQ8q97sdRSq

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Adds LoadedPdf/PdfDocument handles that parse a PDF once and process it repeatedly; one-shot entry points now share the same pipeline. Equivalence tests and docs keep this bounded and clearly beneficial.

View guided diff | Re-trigger cubic

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant