Skip to content

Add Onimusha Way of the Sword (oniws) support - #4

Open
Namsku wants to merge 3 commits into
masterfrom
namsku-OniWoTS-Support
Open

Add Onimusha Way of the Sword (oniws) support#4
Namsku wants to merge 3 commits into
masterfrom
namsku-OniWoTS-Support

Conversation

@Namsku

@Namsku Namsku commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Add Onimusha: Way of the Sword (oniws) support

Vendors the two oniws data files and wires the new game in as first-class support: .scn.21, .user.3, .pfb.18 (RSZ v16) and .msg.23 (version-parametric MSG) all parse/read/edit/write.

Vendored data (pinned upstreams)

Vendored file Upstream source SHA-256
src/reeutils/data/rszoniws.json.gz seifhassine/REasy rszoniwots.json @ 2401f18bbe38c9db7eae1b8101af0e8905d4accd 75A04960E37CB44158D94E55C2B58FD75A51575F8134C8F258FBD7CF1AE39B2E
src/reeutils/data/paklist.oniws.txt.gz kagenocookie/REE-Lib-Resources oniws/oniws_files.list @ 6776919a1a03b1f980761004b8b303f37ce60d9b 29555421B4C132CE72E8217B71DBCC3DF8E4F5173CFC36D7A96A8EE5EA83B5B0

Changes

  • Data: rszoniws.json.gz (RSZ type repo) + paklist.oniws.txt.gz (133,012 paths) embedded under data/; game registered automatically via the GetSupportedGamesCore() resource-convention intersection.
  • Empty templates for the import/write path: empty.scn.21 (128 B), empty.user.3 (112 B), empty.pfb.18 (128 B) — generated from real retail files, all RSZ v16; CLI export→import round-trips byte-identical.
  • Registration: ree-engine-details.md supported-games list, GameNames.ONIWS = "oniws", OriginalPakHelper oniws arms (OnimushaWotS install dir, rszoniws repo).
  • Tests: TestOniws.cs (6 tests — type-repo smoke + corpus round-trips against the retail install), TestOniwsRegistration.cs (4 tests — supported-game registration, embedded data, empty templates, RSZ v16 parse).
  • Docs/CLI polish: README Supported-formats row + .fsmv2.42 read-only note; Program.cs find/tree examples.

Verification results

  • dotnet build src/reeutils/reeutils.csproj -c Release — 0 warnings
  • dotnet build test/IntelOrca.Biohazard.REE.Tests -c Release — 0 warnings
  • REE.Tests full suite: 64 passed, 68 skipped, 0 failed (skips = paks not installed, expected)
  • TestOniws: 6/6 passed (with STEAM_DIR=E:\Steam\steamapps\common)
  • TestOniwsRegistration: 4/4 passed
  • find --pak <retail pak> -g oniws "natives/stm/**/*.scn.21" → 14,169 matches, 0 "No file name found" (100% hash resolution)
  • inspect ACE_SAVE_MSG.msg.23File type: MSG / Version: 23 / Entries: 28
  • tree --json Area.scn.21 → valid JSON with resolved app.* type names (e.g. via.GameObject)

Known caveats (documented, not fixed)

  • .fsmv2.42 files (1,539 in pak) can be read but .fsmv2 import is unsupported for this version — documented in README.
  • app.EventLightConstraintSetter tree dump incomplete — upstream REasy day-one dump incompleteness.
  • Pre-existing, unrelated: 3 REEUtils.Tests failures (TestBhvt ×2, TestFixModifier ×1) reproduce identically on base commit e97eceb, before this branch.

namsku added 2 commits September 4, 2026 10:44
Vendor oniws data files and add full parse/read/edit/write support for the new Onimusha: Way of the Sword file formats.

Pinned upstream sources:
- REasy rszoniwots.json (commit 2401f18bbe38c9db7eae1b8101af0e8905d4accd) -> src/reeutils/data/rszoniws.json.gz (SHA-256 75A04960E37CB44158D94E55C2B58FD75A51575F8134C8F258FBD7CF1AE39B2E)
- REE-Lib-Resources oniws_files.list (commit 6776919a1a03b1f980761004b8b303f37ce60d9b) -> src/reeutils/data/paklist.oniws.txt.gz (SHA-256 29555421B4C132CE72E8217B71DBCC3DF8E4F5173CFC36D7A96A8EE5EA83B5B0)

New formats (all parse/read/edit/write):
- .scn.21 / .user.3 / .pfb.18 via RSZ v16 with empty-template import support (empty.scn.21 128B, empty.user.3 112B, empty.pfb.18 128B), CLI export/import round-trips byte-identical
- .msg.23 via version-parametric MSG support (File type MSG / Version 23)

Verification results:
- TestOniws: 6/6 passed (TypeRepository smoke + corpus round-trips: AchievementCountData.user.3, Area.scn.21, FadeCreator.pfb.18, ACE_SAVE_MSG.msg.23)
- TestOniwsRegistration: 4/4 passed (supported game, embedded data, empty templates, RSZ v16 parse)
- Full REE.Tests: 64 passed, 68 skipped, 0 failed
- find .scn.21: 14,169 matches, 0 unresolved
- inspect ACE_SAVE_MSG.msg.23: MSG / Version 23 / Entries 28
- tree --json Area.scn.21: valid JSON with resolved via.GameObject types

Note: .fsmv2.42 files can be read but import is unsupported (documented in README and ree-engine-details.md).
Known caveat: app.EventLightConstraintSetter tree dump incomplete (documented, not fixed).
RszFieldType.Data fields (raw byte blobs, e.g. via.Folder/value data on
Onimusha scenes) were read as base64 in the binary reader and written as
base64 by the JSON exporter, but the JSON importer had no mapping for the
type and threw 'Unsupported RSZ value type Data' inside GetValueClrType.

Add Data to s_rawValueTypes so ReadValue decodes the base64 payload, and
map it to byte[] in GetValueClrType as a fallback for legacy dumps that
do not use base64. Verified end-to-end on the oniws Area.scn.21 scene:
export -> edit -> import -> re-export is byte-identical at the JSON level
and the imported scene tree matches the native binary->builder->binary
rebuild exactly. Regression test added to TestOniws.
@Namsku

Namsku commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

Fix: JSON import of .scn.21\ scene files (follow-up commit 03a6b98)

The export->import JSON round-trip on oniws .scn.21\ failed with \Unsupported RSZ value type 'Data'. Root cause: \RszFieldType.Data\ — a raw byte-blob field type used by \�ia.Folder/value data — was read as base64 in the binary reader and written as base64 by the JSON exporter, but the importer had no mapping for it.

Fix: register \RszFieldType.Data\ in \s_rawValueTypes\ so \ReadValue\ decodes the base64 payload, and map it to \�yte[]\ in \GetValueClrType\ for legacy dumps. This is additive and only unblocks imports.

Verified:

  • \TestOniws\ 7/7 green including new \Rebuild_Oniws_Scene_Json_RoundTrip\ regression test.
  • \TestRszSerialization\ + scene/user/pfb/msg/file-handler suites green (65 tests).
  • CLI: \�xport\ → edit JSON → \import\ → re-\�xport\ on \Area.scn.21\ is JSON-identical; imported tree matches the native binary rebuild byte-for-byte.

So all four oniws formats are now editable end-to-end: .user.3\ ✅ .msg.23\ ✅ .pfb.18\ ✅ .scn.21\ ✅ (was the last blocker).

…ests

SpanReader.ReadString previously only treated exactly 0 as null, but oniws files
contain strings with negative encoded lengths (sentinel for null), which drifted
the RSZ instance walk. Treat length <= 0 as null.

Add TestOniwsCorpus: a 48-file JSON round-trip idempotency corpus across
.scn.21 / .pfb.18 / .user.3 / .msg.23 (skips when STEAM_DIR is not set).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant