[core] Delete the output of a discarded compaction instead of leaking it

[Stephen0421]

Purpose

The files written by a compaction whose result is never committed are left behind as orphan files. There are two ways to lose track of them:

1. CompactFutureManager#cancelCompaction cancels the future with taskFuture.cancel(true). When the cancellation wins the race against the completion of the task, FutureTask silently drops the value returned by call(). The files of that result are already on disk, but the reference to them is gone, so no caller can account for them.
2. A task failing halfway lost the output of the steps which had already finished, because only the returned result was ever visible to the caller.

This change makes the ownership of the produced files explicit:

• CompactTask accumulates its output into a single result, so a partial result of a failed task is still known.
• Publishing a finished result and declaring it cancelled are mutually exclusive. Either the task publishes first and the manager can still salvage the result after FutureTask dropped it, or the cancellation wins and the task is the one which deletes the files. There is no state in which both sides believe the other one takes care of them.
• A cancelled or failed task deletes its own output. Output files which are the very same physical file as an input (a file only upgraded to another level) are kept, since previous snapshots still require them.
• The cleanup clears the cancellation interrupt while deleting and restores it afterwards. File systems backed by RPC fail their calls immediately while the interrupt flag is set, and deleteQuietly only warns, so the cleanup would silently leave exactly the orphan files it is supposed to remove.

The deletion hooks (CompactRewriter#deleteProduced and the delete of the append rewriters) are abstract rather than defaulted to a no-op, so a rewriter which produces files and forgets to implement them is a compile error instead of a runtime leak.

Note that only file-side cleanup is performed. In-memory side effects applied during compaction (deletion-vector removals, clustering key-index updates) are not rolled back; the invariant that a cancelled result must not be consumed by the same writer's prepareCommit is documented on CompactTask#cancel. Today cancelCompaction is only reachable from writer close(), where the maintainer is thrown away together with the writer.

Tests

New CompactCancellationTest covers the discarded-output paths:

• a cancelled task and a task failing in a later step delete their output;
• a file which is only upgraded is not deleted;
• the deletion file of a discarded result is cleaned up;
• a finished result is not lost when the cancellation wins the FutureTask race;
• a cancellation landing exactly at the point where the task is about to publish its result, after the caller has already given up on it;
• the cleanup runs with the interrupt flag cleared and restores it afterwards;
• a 500 iteration loop racing a real cancellation against the completion of the task, asserting the invariant that the output is either reported to the caller or deleted by the task.

[JingsongLi]

Deleting files is a high-risk operation; simply avoiding their deletion poses no problem at all.

[Stephen0421]

Deleting files is a high-risk operation; simply avoiding their deletion poses no problem at all.

Got it. I will close the pr.