reset encoder state after a failed encode - #348
Merged
agronholm merged 3 commits intoOct 1, 2026
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
While checking how a reused
CBOREncoderbehaves once anencode()call has raised, I noticed that nothing reachesfpafterwards.encode()bumpsencode_depthbefore dispatching and only decrements it on the success path, so after aCBOREncodeError(an unsupported type, a naive datetime with no default timezone, adefaulthook raising) the counter is stuck at 1 on that instance and every later top-level encode finishes at depth 1 instead of 0, skipping the flush and the reset of the shared container and string reference tables.enc.encode(object())followed byenc.encode("hello")leavesfpempty, and the partial output of the failed item stays at the front of the buffer, so once a later message pushes it past the 4 KiB threshold the stream drains with the truncated item fused onto the next one: a failed[shared, object()], thenencode(shared), then a larger message decodes on the receiving side as[['hello'], ['hello']], an item that was never encoded, with the trailing reference cut off. The stale tables are the other half of the risk, since a later message can emit tag 29 or tag 25 references against a registry the receiving decoder has never seen.The depth decrement and the depth-zero reset now run whether
encode_valuesucceeded or not, and on failure the buffered partial item is discarded rather than flushed, so the stream only ever carries complete items and the next encode starts from the same clean state a successful one leaves behind.encode()is the one place the depth is tracked, so this also coversencode_to_bytes,encode_semanticand re-entry from hooks;dump()anddumps()build a fresh encoder per call and are unaffected. The other state savers in the encoder (disable_value_sharing,disable_string_referencing, the namespace restore inencode_semantic) already restore on both paths, so this was the only one left. The regression test fails on master with an emptyfp.Checklist
tests/) which would fail without your patchdocs/), in case of behavior changes or new featuresdocs/versionhistory.rst).