Skip to content

TIKA-4829 -- handle byte passing more cleanly - #3043

Open
tballison wants to merge 2 commits into
mainfrom
TIKA-4829-carry-inline-bytes
Open

TIKA-4829 -- handle byte passing more cleanly#3043
tballison wants to merge 2 commits into
mainfrom
TIKA-4829-carry-inline-bytes

Conversation

@tballison

Copy link
Copy Markdown
Contributor

Thanks for your contribution to Apache Tika! Your help is appreciated!

Before opening the pull request, please verify that

  • there is an open issue on the Tika issue tracker which describes the problem or the improvement. We cannot accept pull requests without an issue because the change wouldn't be listed in the release notes.
  • the issue ID (TIKA-XXXX)
    • is referenced in the title of the pull request
    • and placed in front of your commit messages surrounded by square brackets ([TIKA-XXXX] Issue or pull request title)
  • commits are squashed into a single one (or few commits for larger changes)
  • Tika is successfully built and unit tests pass by running ./mvnw clean test
  • there should be no conflicts when merging the pull request branch into the recent main branch. If there are conflicts, please try to rebase the pull request branch on top of a freshly pulled main branch
  • if you add new module that downstream users will depend upon add it to relevant group in tika-bom/pom.xml.

We will be able to faster integrate your pull request if these conditions are met. If you have any questions how to fix your problem or about using Tika in general, please sign up for the Tika mailing list. Thanks!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Tika Pipes IPC (Smile) serialization path so inline document bytes travel efficiently as raw binary at the tuple top level (instead of being forced through ParseContext’s JSON-config round trip), and reduces wire overhead by disabling Smile’s default 7-bit binary encoding.

Changes:

  • Add a dedicated inlineBytes top-level binary field to FetchEmitTuple IPC serialization/deserialization, keeping the payload out of ParseContext JSON config handling.
  • Configure the IPC SmileFactory to disable ENCODE_BINARY_AS_7BIT to avoid size expansion and transcoding overhead for binary payloads.
  • Add targeted unit tests and a CHANGES.txt entry covering the new wire behavior and request-body rejection of inlineBytes.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/InlineBytesWireTest.java Adds tests validating raw-binary carriage over IPC, ParseContext isolation, and request-body rejection.
tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/JsonPipesIpc.java Disables Smile 7-bit binary encoding for IPC serialization to reduce overhead.
tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/FetchEmitTupleSerializer.java Writes inline payload as a top-level binary field and removes it from the serialized ParseContext.
tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/serialization/FetchEmitTupleDeserializer.java Reads inline payload from the new top-level field into typed ParseContext and rejects it for restricted (request) inputs.
tika-pipes/tika-pipes-core/src/main/java/org/apache/tika/pipes/core/fetcher/InlineBytes.java Updates class-level documentation to describe the new IPC wire behavior.
CHANGES.txt Documents the IPC wire-format change and Smile encoding adjustment under 4.1.0.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants