Repository navigation
Give each bundled Keynote version its own descriptor pool - #81
Merged
Merged
Conversation
The versions/ layout could only ever hold one version. protoc emits code that
registers into descriptor_pool.Default(), which is global to the process and
keyed by .proto filename, and every Keynote version compiles the same
filenames. Importing a second version's mapping raised:
TypeError: Couldn't build proto file into descriptor pool:
duplicate file name TSDArchives.proto
so the machinery was not merely unused, it could not work. dumper/run.py now
writes a per-version pool module and rewrites that version's generated modules
to register into it, which lets several versions coexist.
Bundles Keynote 14.5 as a second version. Its schemas are in fact identical to
14.4's - all 33 shared .proto files match once trailing whitespace is ignored,
the type registry is identical at 631 entries, and 14.5 merely drops
TSKArchives_sos.proto - so this buys no new parsing ability. It is here to keep
the multi-version path exercised by the test suite rather than decorative,
which is what let it rot unnoticed.
Also fixes a regression this would otherwise have introduced: the extension
lookup added for charts in #77 consulted descriptor_pool.Default(), which no
longer holds any of these descriptors. It now uses the pool the message class
was actually built in, so it stays correct per version. Without this change the
chart fix silently degrades back to opaque archives.
Note that __supported_keynote_version__ becomes 14.5, so the derived package
version moves from 1.14.4.x to 1.14.5.x.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0179M4xvAKPGrgKpy4AeCsM7
This was referenced Aug 8, 2026
psobot
added a commit
that referenced
this pull request
Aug 9, 2026
Bundling eight Keynote versions took the wheel to 1.45 MB, 93% of it generated
_pb2.py. Two things inflate those: protoc embeds each schema's serialized
FileDescriptorProto as a Python bytes literal, where every non-printable byte
becomes a four-character \xNN escape - 6.48 MB of source around 3.10 MB of data
- and only 119 of the 258 version-file pairs are distinct, because Apple often
changes nothing in a given schema between releases.
Ask protoc for the FileDescriptorSet instead and store the descriptors
themselves: deduplicated, concatenated in filename order so near-identical
schemas compress against each other, and LZMA'd. Ordering is worth more than it
sounds - pickling the blobs separately gave 0.41 MB, concatenating them 0.09 MB.
wheel 1.45 MB -> 0.15 MB
archive 0.13 MB for all eight versions
which is smaller than the 0.21 MB single-version wheel this all started from,
and makes each further version nearly free.
Message classes come from message_factory against a per-version DescriptorPool,
so the pools that #81 introduced now fall out of the design rather than needing
generated code rewritten to use them. dumper/rewrite_imports.py and
dumper/rewrite_descriptor_pool.py are no longer needed.
Loading is also slightly faster: 12 ms to decompress the archive once, plus
~5 ms per version, against 33 ms to import one version's generated modules.
Two things found while building this:
- pool_for() held the module lock while calling _read_archive(), which takes
it too. A plain Lock deadlocked on the first pool built - it hung rather
than failed. Now an RLock, asserted directly by a test, because a test that
hangs is worse than one that fails.
- The type registry lived in the generated mapping modules, which this
replaces, and it cannot be derived from .proto files - it is extracted from
Keynote itself. It now lives in protos/versions/*/registry.json, checked in
beside the schemas it describes.
Also stops shipping the top-level dumper package, which had been landing in
site-packages since well before this change.
Verified byte-identical output against the generated-module implementation for
ls, cat, and unpack/pack under 10.2, 12.1 and 14.5. 227 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0179M4xvAKPGrgKpy4AeCsM7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change allows us to handle multiple Keynote versions simultaneously, opening the door to backwards-compatibility. It also adds support for Keynote 14.5.