craft: S5 infrastructure — RAFT entry types, CraftPeerFetcher, write_counter by sbinmalek · Pull Request #172 · eBay/HomeBlocks

sbinmalek · 2026-08-10T19:08:49Z

Add the wire format definitions and scaffolding needed by S5 (SyncRSCommitLSN + InternalLogin RAFT state machine entries). Implementations land in follow-up PRs; this PR makes the base branch compile cleanly.

craft_raft_entries.hpp: CraftEntryType enum, CraftEntryHeader, SyncRSCommitLSNPayload (variable-length with trailing int64_t array), InternalLoginPayload, and helpers (sync_rs_commit_lsn_key_size, serialize_sync_rs_commit_lsn, parse_empty_slots)
craft_repl_dev.hpp: add CraftPeerFetcher interface (server-to-server fetch abstraction; production wired in S9), set_peer_fetcher() setter, write_counter_ atomic (periodic SyncRSCommitLSN auto-fire trigger), update apply_sync_rs_commit_lsn signature to carry empty_slots
craft_repl_dev.cpp: match stub signature to updated header
home_blks_config.fbs: add sync_rs_commit_lsn_interval setting (128)

Copilot

Pull request overview

Scaffolds CRAFT S5 RAFT metadata and peer-fetch infrastructure for later implementation.

Changes:

Defines RAFT entry wire formats and serialization helpers.
Adds peer-fetch and write-trigger state to CraftReplDev.
Adds configurable SyncRSCommitLSN interval.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
`src/lib/craft/craft_raft_entries.hpp`	Adds RAFT entry types, payloads, and helpers.
`src/lib/craft/craft_repl_dev.hpp`	Adds peer fetcher and S5 state scaffolding.
`src/lib/craft/craft_repl_dev.cpp`	Updates the apply-helper stub signature.
`src/lib/home_blks_config.fbs`	Adds the synchronization interval setting.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

sbinmalek · 2026-08-10T19:48:26Z

+inline std::vector< int64_t > parse_empty_slots(const SyncRSCommitLSNPayload* p) {
+    const auto* src = reinterpret_cast< const int64_t* >(p + 1);
+    return std::vector< int64_t >(src, src + p->num_empty_slots);


Added the check for corrupt entry

sbinmalek · 2026-08-10T19:49:04Z

+    auto* p            = reinterpret_cast< SyncRSCommitLSNPayload* >(buf);
+    p->rs_commit_lsn   = rs_commit_lsn;
+    p->client_token    = client_token;
+    p->num_empty_slots = static_cast< uint32_t >(empty_slots.size());
+    std::memcpy(p + 1, empty_slots.data(), empty_slots.size() * sizeof(int64_t));


Added the check for empty empty_slots

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

codecov-commenter · 2026-08-19T23:07:02Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (dev/v6.x@1181da7). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
src/lib/craft/craft_repl_dev.cpp	0.00%	1 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##             dev/v6.x     #172   +/-   ##
===========================================
  Coverage            ?   45.60%           
===========================================
  Files               ?       18           
  Lines               ?     1035           
  Branches            ?      451           
===========================================
  Hits                ?      472           
  Misses              ?      264           
  Partials            ?      299

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

shosseinimotlagh · 2026-08-28T20:23:58Z

+
+class CraftPeerFetcher {
+public:
+    virtual async_result< std::vector< JournalSlot > > fetch_from_peer(std::vector< int64_t > lsns) = 0;


const std::vector<int64_t>&

Is this for return or param?

this is for param

shosseinimotlagh · 2026-08-28T20:28:22Z

+
+// ─── header blob ─────────────────────────────────────────────────────────────
+
+struct CraftEntryHeader {


carries no version/magic field

Why do we need that?

…counter Add the wire format definitions and scaffolding needed by S5 (SyncRSCommitLSN + InternalLogin RAFT state machine entries). Implementations land in follow-up PRs; this PR makes the base branch compile cleanly. - craft_raft_entries.hpp: CraftEntryType enum, CraftEntryHeader, SyncRSCommitLSNPayload (variable-length with trailing int64_t array), InternalLoginPayload, and helpers (sync_rs_commit_lsn_key_size, serialize_sync_rs_commit_lsn, parse_empty_slots) - craft_repl_dev.hpp: add CraftPeerFetcher interface (server-to-server fetch abstraction; production wired in S9), set_peer_fetcher() setter, write_counter_ atomic (periodic SyncRSCommitLSN auto-fire trigger), update apply_sync_rs_commit_lsn signature to carry empty_slots - craft_repl_dev.cpp: match stub signature to updated header - home_blks_config.fbs: add sync_rs_commit_lsn_interval setting (128)

- parse_empty_slots now takes the raw key blob instead of a trusted struct pointer, and returns std::optional instead of an unconditional vector. Validates the fixed prefix is present and that the persisted num_empty_slots exactly accounts for the blob's actual size before ever constructing the vector -- a corrupt/truncated RAFT entry no longer risks an out-of-bounds read or a bogus multi-GB allocation during replay. - serialize_sync_rs_commit_lsn now zeroes the fixed prefix before writing fields, so SyncRSCommitLSNPayload's compiler-inserted trailing padding (20 real bytes, sizeof rounds to 24) can't leak prior buffer contents into a persisted/replicated entry or make equivalent entries byte-different. - Guard the empty_slots memcpy: vector::data() may return null when empty, and memcpy(dest, nullptr, 0) is UB even at zero length.

Bump the craft_client pin to 0.4.0 and reconcile CraftReplDev's peer-plane surface with craft_client's include/craft/peer.hpp: - CraftPeerFetcher now mirrors craft_peer's two methods 1:1 -- get_rs_commit_lsn(term, is_login) and fetch_data(lsns), replacing the single fetch_from_peer(lsns). No call sites existed yet. - CraftReplDev::get_rs_commit_lsn() (the responder side) gains the same (term, is_login) params craft_peer declares, unused for now, matching craft_client's own reference implementation. - Add test_craft_journal_slot_wire: round-trips homeblocks::JournalSlot through craft_client's real peer_codec.cpp instead of assuming layout compatibility. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Also bumped conan version

shosseinimotlagh · 2026-08-31T22:43:34Z

    hs_data_chunk_size_mb: uint32 = 2048;
+
+    // how often (in appended LSNs) the leader auto-proposes a SyncRSCommitLSN entry;
+    sync_rs_commit_lsn_interval: uint32 = 128;


nit: add unit to the identifier sync_rs_commit_lsn_interval_sec, sync_rs_commit_lsn_interval_ms ...

This is not a timer. sync will be proposed every X number of log entries.

shosseinimotlagh

LGTM!

sbinmalek requested a balanced review from Copilot August 10, 2026 19:08

Copilot started reviewing on behalf of sbinmalek August 10, 2026 19:09 View session

Copilot AI reviewed Aug 10, 2026

View reviewed changes

sbinmalek marked this pull request as ready for review August 10, 2026 19:56

sbinmalek requested a balanced review from Copilot August 10, 2026 19:56

Copilot started reviewing on behalf of sbinmalek August 10, 2026 19:57 View session

Copilot AI reviewed Aug 10, 2026

View reviewed changes

sbinmalek added the craft label Aug 10, 2026

sbinmalek requested review from shosseinimotlagh and szmyd August 10, 2026 20:17

szmyd requested a review from raakella1 August 18, 2026 16:34

shosseinimotlagh reviewed Aug 28, 2026

View reviewed changes

Comment thread src/lib/craft/craft_raft_entries.hpp Outdated

shosseinimotlagh reviewed Aug 28, 2026

View reviewed changes

Comment thread src/lib/craft/craft_raft_entries.hpp

shosseinimotlagh reviewed Aug 28, 2026

View reviewed changes

Comment thread src/lib/craft/craft_raft_entries.hpp

shosseinimotlagh previously approved these changes Aug 28, 2026

View reviewed changes

sbinmalek and others added 5 commits August 31, 2026 12:40

Applied clang code style

131a65a

craft: Addressing PR comments

f900867

sbinmalek dismissed shosseinimotlagh’s stale review via 47393a5 August 31, 2026 19:56

sbinmalek force-pushed the craft_S5_base branch 2 times, most recently from 47393a5 to f322ca7 Compare August 31, 2026 19:57

craft: Fixed variable names after rebasing

7253e77

* Also bumped conan version

sbinmalek force-pushed the craft_S5_base branch from f322ca7 to 7253e77 Compare August 31, 2026 22:19

shosseinimotlagh reviewed Aug 31, 2026

View reviewed changes

shosseinimotlagh approved these changes Aug 31, 2026

View reviewed changes

sbinmalek merged commit 8aed11c into eBay:dev/v6.x Aug 31, 2026
22 checks passed


		// ─── header blob ─────────────────────────────────────────────────────────────

		struct CraftEntryHeader {

Uh oh!

Conversation

sbinmalek commented Aug 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

codecov-commenter commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shosseinimotlagh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov-commenter commented Aug 19, 2026 •

edited

Loading