[MartinThoma]

FlateDecode._decode_png_prediction() undoes the PNG filter of every row of predicted image data in Python, one byte at a time. The Up filter, the most common one, is now undone for the whole row with a few integer operations, and the Paeth filter does less work per byte. Decoding PNG predicted images is up to 2.3x faster on CPython.

Where the speed comes from

• Up: each byte is the sum of the filtered byte and the byte above it, modulo 256, independent of the other bytes of the row. Both rows are now converted into one integer each and added at once: the low 7 bits of all bytes are added together, which cannot carry into the next byte, and the top bit of each byte is restored with XOR. A row of 4,464 bytes takes 0.011 ms instead of 0.44 ms.
• Paeth: each byte depends on the decoded byte to its left, so this remains a loop. The three distances are now computed from two differences instead of from the predictor p = left + up - up_left, and made positive by comparisons instead of three abs() calls: 1.15 ms instead of 1.35 ms per row of 4,464 bytes.
• The decoded row is no longer copied to bytes to serve as the previous row.

The Sub, Average and None filters are unchanged. The 9.4 MB image of pdf_font_garbled.pdf, the file of the image benchmarks in tests/bench.py, has 1,569 Up, 424 Paeth, 103 Sub and 8 Average rows. Decoding it takes 0.64 s instead of 1.47 s.

Benchmark

Image extraction benchmark of py-pdf/benchmarks (name and data of all images of all pages), best of 3 runs, AMD Ryzen 7 PRO 4750U:

PDF	PNG rows	main	this change	Speedup
4 (1707.09725)	7,321	9.75 s	8.37 s	1.2x
pdf_font_garbled.pdf	21,040	26.51 s	17.37 s	1.5x

CPython 3.15.0. PDFs 1 to 3 of the benchmark contain no PNG predicted data and are not affected.

On PyPy 3.11 (PyPy 8.0.1), whose JIT compiles the byte loop well and whose big integers are slower, the Up filter takes 0.027 ms instead of 0.011 ms per row: pdf_font_garbled.pdf takes 14.70 s instead of 14.04 s (5% slower), PDF 4 3.49 s as before.

Behavior

Unchanged. This includes rows with fewer than 8 bits per pixel, where the number of bytes per pixel used for Sub, Average and Paeth is 0 instead of 1 as in the PNG specification; this is kept as is.

Verification

• The full test suite passes; ruff and mypy report no issues.
• For the 445 PDFs the tests use (tests/pdf_cache, resources,
  sample-files), the decoded data of all 1,010 streams using a PNG
  predictor, and the content stream operations, extracted text,
  warnings and exceptions of all 13,070 pages are the same as on main.
• 301 inputs, the rows of the image above and random rows using all
  filter types with 1 to 6 bytes per pixel, including 127 inputs with
  fewer than 8 bits per pixel and non-rectangular data, decode to the
  same bytes as on main, on CPython and on PyPy.
• test_flatedecode__decode_png_prediction__matches_png_decoder compares
  the decoded rows with the PNG decoder of Pillow, for all filter types
  and 1 to 4 bytes per pixel; it passes on main as well. Changing the
  new code so that the Up filter carries into the next byte or ignores
  the previous row, or the Paeth filter computes a wrong distance,
  drops an absolute value or breaks ties between up and up_left
  differently makes it fail. Breaking ties between left and up
  differently cannot change the result: if their distances are equal
  but their values differ, up_left is closer.
• test_flatedecode__decode_png_prediction__matches_image_without_predictor
  uses the new sample files 031-png-predictors and
  032-png-predictors-low-bit-depth (Add tagged PDF and PNG predictor samples, document 017 sample-files#45). Each page
  shows the same image without a predictor and with /Predictor 15, using
  the PNG filters None, Sub, Up, Average and Paeth in bands of 50 rows,
  so that a decoding error is visible in the band of the broken filter.
  The 8-bit RGB image decodes to the same data as its reference. The
  1, 2 and 4-bit grayscale images do not, as on main: their bytes per
  pixel are rounded down to 0 (see Behavior). These cases are marked as
  strict xfail.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.65%. Comparing base (7f0fb27) to head (b285ef7).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4312   +/-   ##
=======================================
  Coverage   98.65%   98.65%           
=======================================
  Files          59       59           
  Lines       11957    11967   +10     
  Branches     2267     2269    +2     
=======================================
+ Hits        11796    11806   +10     
  Misses         84       84           
  Partials       77       77           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[MartinThoma]

[MartinThoma]

@stefan6419846 How do you like the visual explanation of the PR? :-) If it's helpful, I'll keep adding those in similar cases :-)