Repository navigation
feat!: add cleanIncoming and change incoming packets key - #127
Merged
Merged
Conversation
aedes-persistence v11 makes cleanIncoming a required method: aedes calls it on a clean-session CONNECT so the QoS 2 dedup table does not survive the session ([MQTT-3.1.2-6]). Incoming packets moved from one string key per message (incoming:<clientId>:<messageId>) to one hash per client (incoming:<clientId>), keyed by messageId. cleanIncoming is then a single DEL of that key: no KEYS, no SCAN, and single-slot on a cluster. incomingStorePacket/GetPacket/DelPacket issue one command each, as before. Adds migrations.js from11to12 to drop the now unreachable legacy keys.
Review follow-ups on the v12 incoming key schema change.
migrations.js:
- Scan every master node when handed a Redis.Cluster. SCAN is a keyless
command, so ioredis routed it to one arbitrary node: a cluster-wide scan
saw 102 of 300 legacy keys, then reported success. The migration now
fans out over `db.nodes('master')`, matching clusterMatchRetained.
- Honour the connection's keyPrefix. ioredis does not prefix a SCAN MATCH
pattern but does prefix key arguments, so a prefixed connection matched
nothing and would have re-prefixed the keys it got back.
- Match `incoming:*:*` instead of `incoming:*`. clientIds are URI-encoded,
so only the old keys carry a second colon; live v12 hashes are no longer
read at all. TYPE stays as a guard on what does match, and TYPE and DEL
are now pipelined per scan page instead of two round trips per key.
- Call back with the number of keys removed, so an operator can tell a
real run from one pointed at the wrong database.
- Call back outside the try, so a throwing callback cannot fire it twice.
- Use DEL, not UNLINK: UNLINK needs Redis 4.0 and the cluster fixture in
docker/ is 3.2.8, where the migration failed outright.
test/migrations.js: new. Covers legacy keys removed, v12 hashes and
unrelated keys left alone, idempotency, an empty database, keyPrefix, and
full cluster coverage. All four cases fail against the previous version.
UPGRADE.md: state the preconditions. Running the migration while a v11
broker still serves destroys live dedup state, which causes the duplicate
delivery the schema change exists to prevent. Also documents the rolling
upgrade, how to invoke it, and the downgrade leak.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Adds
cleanIncoming(client)to the redis persistence.aedes-persistencev11 makes it a required interface method: aedes calls it on a CONNECT withcleanSessionset, alongsidecleanSubscriptions, so the QoS 2 dedup table does not survive a discarded session.Without it (GHSA-p8r9-qf8w-p73r), the first colliding
messageIdpublished by a reconnected client is matched against the stale entry, acknowledged, and never delivered — a silent message loss, and a violation of [MQTT-3.1.2-6].persistence.jswrapsAsyncRedisPersistenceinCallBackPersistence, which already delegates both the callback formcleanIncoming(client, cb)→cb(err, client)and the promise form. So the only thing needed here is the async method.Redis commands, and why this is safe at scale
Incoming packets used to be one string key per message,
incoming:<clientId>:<messageId>. There was no per-client index, so deleting a client's packets in one shot was not possible — onlyKEYS/SCANover the keyspace, which blocks the server or walks every key on every clean-session CONNECT.They now live in one hash per client,
incoming:<clientId>, withmessageIdas the field:incomingStorePacketSET incoming:<c>:<mid>HSET incoming:<c> <mid>incomingGetPacketGET incoming:<c>:<mid>HGET incoming:<c> <mid>incomingDelPacketDEL incoming:<c>:<mid>HDEL incoming:<c> <mid>cleanIncomingDEL incoming:<c>cleanIncomingis a singleDELof a single key. NoKEYS, noSCAN, no loop overincomingDelPacket. Cost is proportional to that one client's in-flight packets, not to the keyspace.DEL(or oneDELper messageId) to clean by client.The alternatives considered and rejected:
SCAN MATCH incoming:<c>:*(walks the whole keyspace per clean-session CONNECT, and fans out per node on a cluster), and keeping string keys plus a per-clientSETindex of messageIds (doubles the writes on the hot path, adds an index that can drift, and still needs a cross-slot delete).Schema change
This is a breaking key-schema change, so it lands as v12. Documented in
UPGRADE.mdalongside the existing v9/v10 notes, and noted in the README next to the "Changes in v4.x" entry.Incoming packets are in-flight state only, so nothing durable is lost across the upgrade. The old string keys are unreachable afterwards: start from a clean database, or run the new
migrations.jsfrom11to12, whichSCANs forincoming:*, deletes only the string-typed (legacy) ones, and leaves the new hashes alone. It follows the same(db, cb)shape as the existingfrom9to10.Requires aedes-persistence 11.0.0
Depends on moscajs/aedes-persistence#111 (
fix/clean-incoming-on-session-end), which is not published yet.package.jsonis set to the correct final state,"aedes-persistence": "^11.0.0".CI on this PR will be red until aedes-persistence 11.0.0 is on npm —
npm installcannot resolve^11.0.0. That is expected; no action needed here beyond re-running CI after the release.How this was verified locally
Installed the unreleased branch without saving it:
and ran the full suite against Redis 6.2.5 and the cluster compose service, on Node 22:
test/abs.js(abstract conformance, standalone)test/cluster.js(abstract conformance, cluster)test/own.jsnpm run lintThe three
cleanIncoming-relevant conformance cases all pass in both the standalone and cluster runs:clean incoming packetscovers the "other clients must not be touched" assertion, andclean incoming packets with no stored packetscovers the no-op case (DELon a missing key returns 0, it does not error).The
from11to12migration was exercised against a live Redis: legacy string keys removed, new per-client hashes and unrelated keys (retained:*) untouched, idempotent on a second run and a no-op on an empty database.