Skip to content

fix: Support decoding N-dimensional arrays - #1

Merged
nebkat merged 1 commit into
mainfrom
nd-array-decoding
Aug 26, 2026
Merged

nebkat merged 1 commit into
mainfrom
nd-array-decoding

Conversation

@nebkat

@nebkat nebkat commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Fixes a decoding gap: N-dimensional arrays are a draft 3 construct that this library
rejected as invalid, so valid draft 3 data could not be read at all.

bjdataDecode([0x5B, 0x24, 0x55, 0x23, 0x5B, 0x55, 2, 0x55, 3, 0x5D, 1, 2, 3, 4, 5, 6]);
// before: FormatException: Invalid BJData marker: 0x5b
// after:  [[1, 2, 3], [4, 5, 6]]

What it does

  • [$type#[Nx Ny ...] decodes to nested lists, with the innermost axis kept as the typed
    list, so the spec's 2×3×4 example comes back as a List<List<Uint8List>> matching the
    JSON in the specification verbatim.
  • The nesting is built from views, not copies. An N-dimensional array is still one
    contiguous allocation once decoded — a test pins the slices at offsets 0 and 20 into a
    single 24-byte buffer.
  • The column-major form is handled too. [$type#[[Nx Ny ...]] is how MATLAB and
    FORTRAN write it, and given JSONLab shares an author with the spec it is likely common
    in real data. Elements are permuted whole, at whatever width, so it reads identically to
    a row-major array of the same shape. Verified against the specification's own
    column-major byte sequence.
  • Dimension arrays are accepted in both optimized and non-optimized form; an object
    counted by one is rejected.

Also throws a FormatException rather than a RangeError when a string or buffer runs
past the end of the input, which a payload shorter than its dimensions makes easy to hit.

Not included

Encoding N-dimensional arrays. A decoded array is written back as nested arrays — the
values are unchanged but the format is not. That is added in the follow-up PR, which also
makes the round-trip byte-exact.

Verification

Both worked examples from the specification decode byte-for-byte. 203 tests on the VM and
183 on Chrome, plus dart format, dart analyze --fatal-infos and the examples.

🤖 Generated with Claude Code

N-dimensional arrays are a draft 3 construct that this library rejected as
invalid, so valid draft 3 data could not be read.

`[$type#[Nx Ny ...]` now decodes to nested lists, keeping the innermost axis as
the typed list. The nesting is built from views rather than copies, so an
N-dimensional array is still one contiguous allocation once decoded. Dimension
arrays are accepted in both optimized and non-optimized form.

The column-major form, `[$type#[[Nx Ny ...]]` as MATLAB and FORTRAN write it,
is reordered into row-major order so that it reads the same way as an array of
the same shape written row-major. Elements are permuted whole, whatever their
width, so this works for every strong type.

Encoding N-dimensional arrays is not supported, so a decoded array is written
back as nested arrays; the values are unchanged.

Also throws a FormatException rather than a RangeError when a string or buffer
runs past the end of the input, which a payload shorter than its dimensions
makes easy to hit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@nebkat

nebkat commented Aug 26, 2026

Copy link
Copy Markdown
Owner Author

Base of a two-PR stack:

Merge this one first; #2 retargets to main automatically afterwards.

🤖 Generated with Claude Code

@nebkat
nebkat merged commit a82a0bc into main Aug 26, 2026
8 checks passed
@nebkat
nebkat deleted the nd-array-decoding branch September 6, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant