Add EncodedByteAlign Support for CCITTFaxDecode

[shartzog]

Problem

The tiff header trick pypdf uses to 'decode' data compressed with the /CCITTFaxDecode filter back into a tiff image does not support the /EncodedByteAlign parameter shown below:

Example

Using python 3.10.6 on Windows with pypdf v6.5.0 and pillow v11.3.0

Inputs

byte_aligned_tiff.pdf
normal_tiff.pdf

Tests

from pathlib import Path

from pypdf import PdfReader
from PIL.Image import Image

dir_str = "./"

# Test /CCITTFaxDecode Filter with /EncodedByteAlign true
byte_aligned_rdr = PdfReader(f"{dir_str}byte_aligned_tiff.pdf")
byte_aligned_img: Image = byte_aligned_rdr.pages[0].images[0].image
byte_aligned_data: bytes = byte_aligned_rdr.pages[0].images[0].data
# Saving via the PIL interface results in an 'empty' image
byte_aligned_img.save(f"{dir_str}byte_aligned_image.tiff")
# Saving the 'decoded' (aka wrapped with a tiff header) bytes directly results in a few black streaks.
Path(f"{dir_str}byte_aligned_data.tiff").write_bytes(byte_aligned_data)

# Test Standard /CCITTFaxDecode Filter
normal_rdr = PdfReader(f"{dir_str}normal_tiff.pdf")
normal_img: Image = normal_rdr.pages[0].images[0].image
normal_data: bytes = normal_rdr.pages[0].images[0].data
# The 'normal' tiff works either way and produces equivalent output
normal_img.save(f"{dir_str}normal_image.tiff")
Path(f"{dir_str}normal_data.tiff").write_bytes(normal_data)

Outputs

Byte Aligned ❌

byte_aligned_image.tiff
byte_aligned_data.tiff

Standard ✅

normal_image.tiff
normal_data.tiff

---

Proposal

Implement a native python /CCITTFaxDecode decompression algorithm and use it if and only if the /EncodedByteAlign parameter is true and the /K parameter is -1 indicating CCITT Group 4 compression.

Proof of Concept

The prototype module below is a crude proof of concept written mostly by Claude.

If there's sufficient interest, I'll work on cleaning it up, integrating it into the existing decoding / image workflow, and submitting a PR.

Prototype Module

ccitt_g4_decoder.py

Usage

from pathlib import Path

from pypdf import PdfReader

from ccitt_g4_decoder import CCITTGroup4Decoder

dir_str = "./"

# Test with /EncodedByteAlign true
byte_aligned_rdr = PdfReader(f"{dir_str}byte_aligned_tiff.pdf")
byte_aligned_object = byte_aligned_rdr.pages[0].images[0].indirect_reference
CCITTGroup4Decoder(byte_aligned_object).image.save(f"{dir_str}byte_aligned_decoded.tiff")

# Test Standard
normal_rdr = PdfReader(f"{dir_str}normal_tiff.pdf")
normal_object = normal_rdr.pages[0].images[0].indirect_reference
CCITTGroup4Decoder(normal_object).image.save(f"{dir_str}normal_decoded.tiff")

Outputs

Byte Aligned ✅

byte_aligned_decoded.tiff

Standard ✅

normal_decoded.tiff

---

Next Steps

Confirm the prototype's behavior on a broader sample set. If other community members have access to PDFs that use /EncodedByteAlign with the /CCITTFaxDecode filter or know a means to produce such PDFs from image inputs, please reach out.

[stefan6419846]

Thanks for opening this discussion. It seems like the title is incorrect, as we are talking about the /EncodedByteAlign parameter, which is defined as follows:

A flag indicating whether the filter shall expect extra 0 bits before each encoded line so that the line begins on a byte boundary. If true, the filter shall skip over encoded bits to begin decoding each line at a byte boundary. If false, the filter shall not expect extra bits in the encoded representation. Default value: false.

Regarding your actual remarks:

Implement a native python /CCITTFaxDecode decompression algorithm and use it if and only if the /EncodedBytesAlign parameter is true and the /K parameter is -1 indicating CCITT Group 4 compression.
If there's sufficient interest, I'll work on cleaning it up, integrating it into the existing decoding / image workflow, and submitting a PR.

In general, having missing features added to pypdf itself is desirable, especially when there are relevant decoding parameters to consider.

My main issue with the currently proposed code is that we would have 23 KiB and/or 1000 lines of own code to maintain afterwards "just" to support one "simple" parameter, while having to understand the internals of CCITT Group 4/TIFF files as well - with all its possible quirks. I am a bit unsure about the benefits versus the maintenance costs for this, due to as far as I understand, there is no easy way to have some existing library do this.

If other community members have access to PDFs that use /EncodedByteAlign with the /CCITTFaxDecode filter or know a means to produce such PDFs from image inputs, please reach out.

From our test files, it seems like the PDF3200_2008.pdf one (the PDF 1.7 specification itself) has some matches after some basic grepping.

[shartzog]

Thanks for catching the typo! Fixed in title and throughout.

[shartzog]

I agree: the implementation as it stands is pretty dodgy and nowhere close to understandable or supportable. If it can't be made so, it's definitely not worth it, but seeing as you've indicated interest in the functionality itself, I think I'll take a bit of time to understand just what in the hell Claude has done here to see how much fluff could be eliminated from reuse of existing pypdf features and/or simplification of the implementation itself. More to come.