diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 23c522b..a40ca33 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -41,6 +41,9 @@ jobs: - name: Build run: pnpm build + - name: Real isolated local-run-as service install + run: sudo env GITHUB_ACTIONS=true PATH="$PATH" python3 scripts/run-as-smoke.py --isolated-ci --cli "$PWD/dist/index.js" --node "$(command -v node)" + - name: Real CLI fleet install, update, and recovery run: python3 scripts/cli-dogfood.py --evidence-dir "${{ runner.temp }}/cli-dogfood" diff --git a/docs/design/artifact-harness-compatibility.md b/docs/design/artifact-harness-compatibility.md index 9ce1edd..6f79761 100644 --- a/docs/design/artifact-harness-compatibility.md +++ b/docs/design/artifact-harness-compatibility.md @@ -58,48 +58,73 @@ path, not from the inverse Codex/Copilot path. ## Built-In Matrix -Agentwheel also accepts `transport: "local-run-as"` on a named Linux agent with a -required `runAs` account. It resolves `root: "home"` to that account's passwd home -and reads runtime files and install manifests through a path-checked, service-user -worker. In the first release this transport is **read-only**: plans, status, and -ownership recovery reports and legacy adoption previews work; installs, uninstalls, journal recovery, and -other target mutations fail before writes. An absolute target root must be owned -by the account and equal its home for home-rooted mappings. SSH and ordinary -local transports retain their existing layouts. +Agentwheel accepts `transport: "local-run-as"` on a named Linux agent with a +required `runAs` account. Home-rooted mappings use that account's passwd home; +the absolute install root must equal it and have the pinned account's UID/GID. +Plans, status, install, uninstall and durable journal recovery use a service-user +worker. Ownership adoption previews remain available; adoption apply and Fleet +normalization require the separate migration support. + +Requirements are Linux 5.6+ with `openat2`, a supported 64-bit architecture +(x86-64, AArch64 or RISC-V), Python 3 and noninteractive sudo. The immutable Node +worker delegates filesystem operations to its packaged Python helper using +`RESOLVE_BENEATH|RESOLVE_NO_SYMLINKS|RESOLVE_NO_MAGICLINKS` and directory +file descriptors. Unsupported kernels, identities, paths or helpers fail closed +before the apply lock. SSH and ordinary local transports keep their layouts. +This confines worker path resolution, including concurrent symlink replacement; +it does not prevent an already running process of the same UID from relocating +an opened directory inode or changing installed content after a write. + +Caller-owned source staging supplies bounded bytes and planned hashes, never +source paths or credentials. Sources must have trusted, stable ancestors. +Files default to `0600` and directories to `0700`; backup/rollback preserves +existing modes. Symlinks, hardlinks, special files, malformed entry trees and +snapshots over 8 MiB, 4096 entries or 64 path components are rejected. Target +manifests, locks, backups and journals belong to the service user; source graph +locks and governed receipts remain in the caller workspace. Recovery revalidates +the receipt's pinned user, UID, GID, home and root before target access. + +Programmatic adapters, semantic plugin operations, arbitrary commands and runtime +reloads are rejected before install writes. The sole subprocess capability is +OpenClaw `config validate --json`, through a dedicated operation with fixed argv +and a scrubbed environment. It invokes Node with `openclaw.mjs` from a trusted +package root (default `/usr/local/lib/node_modules/openclaw`, caller override +`AGENTWHEEL_RUN_AS_OPENCLAW_ROOT`). The full package tree, contained symlink +dependencies, executable and ancestors must be immutable to the service UID and +supplementary groups. Group/world-writable ancestor masks are rejected even +when the owning group differs, covering named POSIX ACL grants. Service-owned installations are rejected. Validation reads a confined candidate +and uses a temporary caller-owned immutable config snapshot, with Linux POSIX +ACLs granting read access only to the caller and intended service UID (other +users cannot traverse its directory or read the file), preventing a target +symlink swap during the native validator read. Any `$include` key is rejected recursively; use self-contained rendered config +for this transport. Python runs in +isolated mode (`-I`), excluding service user startup files. Operator-managed runtime +code is a prerequisite, and validation does not grant general command execution. +See [OpenClaw's validation contract](https://docs.openclaw.ai/cli/config). + The worker uses a checked absolute Node executable (`/usr/bin/node` by default; -`AGENTWHEEL_RUN_AS_NODE` may select another absolute path). The executable and -worker must be regular files that the service UID cannot own or write. One -worker copy is staged per process and removed on exit, SIGINT, or SIGTERM. -Each `sudo` child has a 120-second timeout, and worker `readFile` accepts only -regular files. -The source-staging module prepares versioned, bounded byte snapshots for future -install support. It rejects links and special files, validates the complete -entry hierarchy and content hash, and carries fixed private modes rather than -source ownership or source paths. Source ancestors must be trusted and stable; -this module does not provide target confinement or enable writes. Target write, -command, recovery and ownership-migration support remain unavailable. - -For a dedicated operator and service account, grant only the UID check and the -read worker invocation. Replace the example account, home, UID/GID, and Node -path with the resolved values on the target host; the Node path must be its -canonical absolute path. Validate the resulting file with `visudo -cf` before -installing it in sudoers. The wildcard covers the random, caller-owned worker -staging directory and should be granted only to a trusted operator: +`AGENTWHEEL_RUN_AS_NODE` may choose another absolute path) and `/usr/bin/python3`. +Linux POSIX ACL support on the caller temporary filesystem is required for +OpenClaw validation; unsupported ACLs fail closed. Node, Python, worker and +helper must be regular files the service UID cannot own +or write. Worker/helper copies are staged per caller process and removed on exit, +SIGINT or SIGTERM. Sudo children have a 120-second timeout. An operator grants +sudo explicitly; Agentwheel never edits sudoers. Replace the synthetic account, +home, UID/GID and canonical Node path before validating an operator's grant with +`visudo -cf`. The caller-owned staging wildcard is suitable only for a trusted +operator: ```sudoers Cmnd_Alias AGENTWHEEL_RUN_AS_ID = /usr/bin/id -u -Cmnd_Alias AGENTWHEEL_RUN_AS_READ = /usr/bin/env -i HOME=/home/example PATH=/usr/local/bin\:/usr/bin\:/bin /usr/bin/node /tmp/agentwheel-run-as-*/run-as-worker.mjs --root /home/example --uid 998 --gid 998 -operator ALL=(service) NOPASSWD: AGENTWHEEL_RUN_AS_ID, AGENTWHEEL_RUN_AS_READ +Cmnd_Alias AGENTWHEEL_RUN_AS_WORKER = /usr/bin/env -i HOME=/home/example PATH=/usr/local/bin\:/usr/bin\:/bin /usr/bin/node /tmp/agentwheel-run-as-*/run-as-worker.mjs --root /home/example --uid 998 --gid 998 +operator ALL=(service) NOPASSWD: AGENTWHEEL_RUN_AS_ID, AGENTWHEEL_RUN_AS_WORKER ``` `ownership recovery-plan` requires `--fleet` for registered Fleet targets; -it may omit `--fleet` for a nested workspace outside any Fleet. - -Legacy adoption previews read runtime claims and full ownership snapshots through -the worker. Their digest includes the run-as endpoint identity. Workspace source -locks remain caller-side. Adoption apply remains disabled, and previews do not -advertise an executable apply command for this transport. This does not enable -fleet normalization or ownership migration. +it may omit it for a nested workspace outside any Fleet. Legacy previews read +runtime manifests and full ownership snapshots in the worker, with the endpoint +identity in their digest. They do not advertise an executable adoption command +until migration apply support is enabled. | Harness | Artifact | `local` | `user` | Notes | |---|---|---|---|---| diff --git a/scripts/run-as-smoke.py b/scripts/run-as-smoke.py new file mode 100644 index 0000000..a6a112b --- /dev/null +++ b/scripts/run-as-smoke.py @@ -0,0 +1,365 @@ +#!/usr/bin/env python3 +"""Privileged local-run-as smoke test for a disposable GitHub Actions runner only. + +Creates a synthetic non-login account, exercises the built CLI and default sudo +transport, and deletes that account and its temporary fixture in a finally block. +Never run this script against a persistent host or a real runtime installation. +""" +import argparse +import grp +import json +import os +from pathlib import Path +import pwd +import secrets +import shutil +import stat +import subprocess +import sys +import tempfile +import threading +import time + + +def require(condition, message): + if not condition: + raise RuntimeError(message) + + +def run(argv, *, cwd, env, expected=0): + print('RUN ' + ' '.join(map(str, argv)), flush=True) + result = subprocess.run(list(map(str, argv)), cwd=cwd, env=env, + capture_output=True, text=True, timeout=180) + if expected is not None and result.returncode != expected: + raise RuntimeError(f'Command failed ({result.returncode}): {argv}\n' + f'stdout:\n{result.stdout}\nstderr:\n{result.stderr}') + return result + + +def tree_snapshot(root): + """Include bytes, ownership, modes and mtimes; reads may update atime.""" + result = {} + for path in [root, *sorted(root.rglob('*'))]: + info = path.lstat() + require(not stat.S_ISLNK(info.st_mode), f'Unexpected fixture symlink: {path}') + result[str(path.relative_to(root))] = ( + info.st_mode, info.st_uid, info.st_gid, info.st_mtime_ns, + path.read_bytes() if path.is_file() else None) + return result + + +def write_json(path, value): + path.write_text(json.dumps(value, indent=2) + '\n') + path.chmod(0o600) + + +def smoke(args): + # Refuse before creating files or invoking account-management commands. + require(args.isolated_ci and os.environ.get('GITHUB_ACTIONS') == 'true' + and sys.platform == 'linux' and os.geteuid() == 0, + 'Refusing: requires --isolated-ci, GITHUB_ACTIONS=true, Linux and euid 0 ' + 'on a disposable runner; do not run on a persistent host.') + cli, node = args.cli.resolve(strict=True), args.node.resolve(strict=True) + require(cli.is_file() and node.is_file(), 'Built CLI and Node executable must be files') + require(Path('/usr/bin/python3').is_file(), 'Default helper /usr/bin/python3 is required') + for executable in ('/usr/sbin/useradd', '/usr/sbin/userdel', '/usr/sbin/groupdel', + '/usr/bin/sudo', '/usr/bin/test', '/usr/sbin/nologin'): + require(Path(executable).is_file(), f'Required runner tool missing: {executable}') + old_umask = os.umask(0o077) + base = Path(tempfile.mkdtemp(prefix='aw-runas-ci-')) + base.chmod(0o711) + user = 'awsmoke_' + secrets.token_hex(5) + created = False + service_gid = None + failure = None + confidentiality_stop = threading.Event() + confidentiality_thread = None + confidentiality = {} + try: + caller = base / 'caller' + caller.mkdir(mode=0o700) + caller_home, workspace, pack = (caller / name for name in ('home', 'workspace', 'pack')) + for path in (caller_home, workspace, pack): + path.mkdir(mode=0o700) + # Hosted toolcache/checkouts may use group-writable modes. Provision + # immutable root-owned test tooling instead of weakening code trust. + tools = base / 'trusted-tools' + tools.mkdir(mode=0o755) + tools.chmod(0o755) + isolated_node = tools / 'node' + shutil.copyfile(node, isolated_node) + isolated_node.chmod(0o755) + node = isolated_node + distribution = caller / 'dist' + distribution.mkdir(mode=0o700) + for name in ('index.js', 'run-as-worker.js', 'run-as-fs.py'): + shutil.copyfile(cli.parent / name, distribution / name) + (distribution / name).chmod(0o644) + # The caller CLI keeps its external dependencies; the service worker + # is standalone and never traverses this private caller directory. + (caller / 'node_modules').symlink_to(cli.parent.parent / 'node_modules', target_is_directory=True) + cli = distribution / 'index.js' + env = {'AGENTWHEEL_RUN_AS_NODE': str(node), 'HOME': str(caller_home), 'XDG_CONFIG_HOME': str(caller_home / '.config'), + 'XDG_CACHE_HOME': str(caller_home / '.cache'), + 'XDG_STATE_HOME': str(caller_home / '.local/state'), + 'PATH': str(node.parent) + ':/usr/local/bin:/usr/bin:/bin'} + service_home = base / 'service-home' + sibling = base / 'untouched-sibling' + sibling.mkdir(mode=0o700) + (sibling / 'sentinel').write_text('unchanged\n') + sibling_before = tree_snapshot(sibling) + run(['/usr/sbin/useradd', '--user-group', '--no-create-home', '--home-dir', service_home, + '--shell', '/usr/sbin/nologin', user], cwd=base, env=env) + created = True + identity = pwd.getpwnam(user) + service_gid = identity.pw_gid + require(identity.pw_uid > 0 and identity.pw_gid > 0, 'Expected separate non-root service UID/GID') + require(identity.pw_shell == '/usr/sbin/nologin', 'Expected non-login account') + service_home.mkdir(mode=0o700) + os.chown(service_home, identity.pw_uid, identity.pw_gid) + + skill = pack / 'skills/fixture/SKILL.md' + skill.parent.mkdir(parents=True, mode=0o700) + skill.write_text('---\nname: fixture\ndescription: Isolated run-as smoke fixture.\n---\n\nSynthetic fixture.\n') + settings = pack / 'settings/openclaw.json' + settings.parent.mkdir(mode=0o700) + reference = '${AW_SMOKE_EXTERNAL_VALUE}' + write_json(settings, {'env': {'vars': {'AW_SMOKE_REFERENCE': '${secret:AW_SMOKE_EXTERNAL_VALUE}'}}}) + write_json(pack / 'openpack.json', { + 'schemaVersion': 2, 'name': 'smoke/run-as', 'version': '1.0.0', + 'provides': [{'type': 'skills', 'path': 'skills'}, + {'type': 'settings', 'path': 'settings/openclaw.json'}]}) + (workspace / '.agentwheel').mkdir(mode=0o700) + write_json(workspace / '.agentwheel/config.json', { + 'schemaVersion': 4, + 'mutationPolicy': {'reason': 'required', 'journal': 'required', 'revisioning': {'mode': 'off'}}, + 'packages': [{'name': 'fixture', 'source': str(pack), 'driver': 'local', 'mode': 'pinned', + 'adapter': 'openclaw', 'installationType': 'user'}], + 'agents': {'service': {'adapter': 'openclaw', 'root': str(service_home), + 'transport': 'local-run-as', 'runAs': user, 'installationType': 'user'}}, + 'profiles': {'service': {'runtimes': [{'agent': 'service'}]}}}) + + # Caller-owned immutable package, separate from the service's home. + validator_root = base / 'validator' + validator_root.mkdir(mode=0o755) + validator_root.chmod(0o755) + env['AGENTWHEEL_RUN_AS_OPENCLAW_ROOT'] = str(validator_root) + validator = validator_root / 'openclaw.mjs' + validator.write_text("""import fs from 'node:fs'; +import path from 'node:path'; +import assert from 'node:assert/strict'; +const home = process.env.HOME, config = process.env.OPENCLAW_CONFIG_PATH; +assert.deepEqual(process.argv.slice(2), ['config', 'validate', '--json']); +assert.equal(process.getuid(), fs.statSync(home).uid); assert.notEqual(process.getuid(), 0); +assert.ok(path.dirname(config).startsWith('/tmp/agentwheel-validate-')); +assert.equal(path.basename(config), 'openclaw.json'); +assert.notEqual(fs.statSync(config).uid, process.getuid()); +assert.equal(fs.statSync(config).mode & 0o777, 0o440); +assert.equal(fs.statSync(path.dirname(config)).mode & 0o777, 0o710); +assert.equal(JSON.parse(fs.readFileSync(config)).env.vars.AW_SMOKE_REFERENCE, '${AW_SMOKE_EXTERNAL_VALUE}'); +for (const key of ['AW_SMOKE_EXTERNAL_VALUE', 'GITHUB_TOKEN', 'GH_TOKEN', 'AWS_SECRET_ACCESS_KEY']) assert.equal(process.env[key], undefined); +fs.writeFileSync(path.join(home, 'validator-invocation.json'), JSON.stringify({argv: process.argv.slice(2), uid: process.getuid(), gid: process.getgid(), config}), {mode: 0o600}); +const ack = path.join(home, 'validation-confidentiality-ok'); +const deadline = Date.now() + 60_000; +while (!fs.existsSync(ack) && Date.now() < deadline) await new Promise(resolve => setTimeout(resolve, 25)); +assert.ok(fs.existsSync(ack), 'confidentiality observer timed out'); +assert.equal(fs.statSync(ack).uid, process.getuid()); +assert.equal(fs.statSync(ack).mode & 0o777, 0o600); +assert.deepEqual(JSON.parse(fs.readFileSync(ack)), {fileReadDenied: true, directoryListDenied: true}); +console.log(JSON.stringify({valid: true})); +""") + validator.chmod(0o644) + inaccessible = run(['/usr/bin/sudo', '-n', '-u', user, '--', '/usr/bin/test', '-r', skill], + cwd=base, env=env, expected=None) + require(inaccessible.returncode != 0, 'Service user unexpectedly read caller-private source') + initial_target = tree_snapshot(service_home) + aw = [node, cli, '--no-update-check', '--reason', 'Isolated CI run-as smoke'] + scope = ['--local', '--profile', 'service'] + # scan reads and validates the source through the real CLI; plan resolves + # its graph and probes the real service transport without target writes. + run(aw + ['scan', str(pack), '--local'], cwd=workspace, env=env) + planned = json.loads(run(aw + ['plan', *scope, '--format', 'json'], cwd=workspace, env=env).stdout) + require(not planned['applied'] and len(planned['targets']) == 1, 'Unexpected initial plan targets') + target = planned['targets'][0] + require(target['targetRoot'] == str(service_home) and target['installationType'] == 'user', + 'Plan did not select the service home/user installation') + require(not target['hasBlockingChanges'] and target['summary']['create'] == 2, + f'Expected skill and settings creates: {target}') + require(tree_snapshot(service_home) == initial_target, 'Initial graph plan wrote to target') + unrelated = pwd.getpwnam('nobody') + require(unrelated.pw_uid not in (0, identity.pw_uid) + and unrelated.pw_gid != identity.pw_gid, 'Expected a distinct unrelated UID/GID') + invocation = service_home / 'validator-invocation.json' + ack = service_home / 'validation-confidentiality-ok' + + def acknowledge(value): + # Root records the result as a private service-owned fixture file. + pending = ack.with_name(ack.name + '.pending') + fd = os.open(pending, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + try: + os.write(fd, json.dumps(value).encode()) + os.fchown(fd, identity.pw_uid, identity.pw_gid) + os.fsync(fd) + finally: + os.close(fd) + os.rename(pending, ack) + + def observe_confidentiality(): + try: + deadline = time.monotonic() + 70 + while not confidentiality_stop.is_set() and time.monotonic() < deadline: + try: + call = json.loads(invocation.read_text()) + except (FileNotFoundError, json.JSONDecodeError): + confidentiality_stop.wait(0.025) + continue + config = Path(call['config']) + require(config.parent.parent == Path('/tmp') + and config.parent.name.startswith('agentwheel-validate-') + and config.name == 'openclaw.json', 'Unexpected validation snapshot path') + require(config.is_file(), 'Snapshot disappeared before confidentiality checks') + # This real third-UID process receives the known filename; + # unreadability must come from permissions, not obscurity. + probe = r"""import json, os, pathlib, sys +snapshot=pathlib.Path(sys.argv[1]) +result={} +for key, action in [('fileReadDenied', snapshot.read_bytes), + ('directoryListDenied', lambda: os.listdir(snapshot.parent))]: + try: + action() + except PermissionError: + result[key]=True + else: + raise RuntimeError('Unrelated user unexpectedly accessed '+key) +print(json.dumps(result)) +""" + command = ['/usr/bin/sudo', '-n', '-u', unrelated.pw_name, '--', + '/usr/bin/python3', '-I', '-c', probe, str(config)] + print('CHECK unrelated UID snapshot read and directory-list denial', flush=True) + result = subprocess.run(command, cwd='/', env=env, capture_output=True, + text=True, timeout=10) + require(result.returncode == 0, + f'Confidentiality probe failed: {result.stdout} {result.stderr}') + checked = json.loads(result.stdout) + require(checked == {'fileReadDenied': True, 'directoryListDenied': True}, + f'Incomplete confidentiality checks: {checked}') + require(config.is_file(), 'Snapshot disappeared during confidentiality checks') + confidentiality.update(checked) + acknowledge(checked) + return + raise RuntimeError('Confidentiality observer did not see the live validation snapshot') + except Exception as error: + confidentiality['error'] = str(error) + if not ack.exists(): + try: + acknowledge({'error': str(error)}) + except OSError: + pass + + confidentiality_thread = threading.Thread(target=observe_confidentiality, + name='validation-confidentiality', daemon=True) + confidentiality_thread.start() + installed = json.loads(run(aw + ['install', *scope, '--format', 'json'], cwd=workspace, env=env).stdout) + confidentiality_thread.join(timeout=15) + require(not confidentiality_thread.is_alive(), 'Confidentiality observer did not finish') + require(confidentiality == {'fileReadDenied': True, 'directoryListDenied': True}, + f'Validation confidentiality checks failed: {confidentiality}') + require(installed['applied'], 'CLI install did not report apply') + installed_skill = service_home / '.openclaw/skills/fixture/SKILL.md' + require(installed_skill.read_bytes() == skill.read_bytes(), 'Installed skill bytes differ') + installed_settings = service_home / '.openclaw/openclaw.json' + require(json.loads(installed_settings.read_text())['env']['vars']['AW_SMOKE_REFERENCE'] == reference, + 'Literal environment reference was not preserved') + call = json.loads((service_home / 'validator-invocation.json').read_text()) + require(call['argv'] == ['config', 'validate', '--json'] + and (call['uid'], call['gid']) == (identity.pw_uid, identity.pw_gid), + 'Settings validation did not use the allowlisted service operation') + manifests = list((service_home / '.agentwheel').glob('*.install-manifest.json')) + require(len(manifests) == 1, 'Expected one partitioned target install manifest') + manifest = json.loads(manifests[0].read_text()) + key = manifest['stateKey'] + require(key.startswith('openclaw.user.') and len(key) > len('openclaw.user.'), + f'Expected fingerprint-partitioned user state key: {key}') + require(manifests[0].name == key + '.install-manifest.json', 'Manifest state-key/path mismatch') + require(len(manifest['entries']) == 2, 'Expected skill and settings manifest entries') + for path in [service_home, *service_home.rglob('*')]: + info = path.lstat() + require((info.st_uid, info.st_gid) == (identity.pw_uid, identity.pw_gid), + f'Wrong service ownership: {path}') + require(not stat.S_ISLNK(info.st_mode), f'Unexpected target symlink: {path}') + expected_mode = 0o700 if path.is_dir() else 0o600 + require(stat.S_IMODE(info.st_mode) == expected_mode, + f'Wrong target mode: {path}: {stat.S_IMODE(info.st_mode):04o}') + status = json.loads(run(aw + ['status', *scope, '--installation-type', 'user', '--refresh', '--json'], + cwd=workspace, env=env).stdout) + require(len(status['targets']) == 1 and status['targets'][0]['health'] == 'PASS', + f'Installed target status failed: {status}') + before = tree_snapshot(service_home) + source_before = tree_snapshot(pack) + replanned = json.loads(run(aw + ['install', *scope, '--dry-run', '--format', 'json'], + cwd=workspace, env=env).stdout) + require(not replanned['applied'] and len(replanned['targets']) == 1, 'Dry-run unexpectedly applied') + summary = replanned['targets'][0]['summary'] + require(summary['skip'] == 2 and all(summary[name] == 0 for name in + ('create', 'update', 'remove', 'drift', 'conflict', 'plugin')), f'Replan was not a no-op: {summary}') + require(tree_snapshot(service_home) == before, 'Second dry-run wrote to service target') + require(tree_snapshot(pack) == source_before, 'Second dry-run wrote to private source') + require(tree_snapshot(sibling) == sibling_before, 'Sibling sentinel changed') + require(not (caller_home / '.openclaw').exists() and not (workspace / '.openclaw').exists(), + 'Unexpected caller-side runtime output') + require({path.name for path in base.iterdir()} == {'caller', 'service-home', 'untouched-sibling', 'validator', 'trusted-tools'}, + 'Unexpected sibling runtime output') + print('PASS: sudo/explicit Node/Python transport, private source, service validation, ' + 'unrelated UID snapshot read/list denial, ' + 'ownership/modes, manifest partitioning, status and read-only no-op replan', flush=True) + except Exception as error: + failure = error + finally: + cleanup_errors = [] + confidentiality_stop.set() + if confidentiality_thread is not None: + confidentiality_thread.join(timeout=15) + if confidentiality_thread.is_alive(): + cleanup_errors.append('Confidentiality observer did not stop before fixture cleanup') + cleanup_env = {'PATH': '/usr/sbin:/usr/bin:/sbin:/bin'} + if created: + try: + run(['/usr/sbin/userdel', user], cwd='/', env=cleanup_env) + except Exception as error: + cleanup_errors.append(str(error)) + try: + # userdel may already delete its private primary group. + group = grp.getgrnam(user) + require(group.gr_gid == service_gid, 'Refusing to delete a changed fixture group') + run(['/usr/sbin/groupdel', user], cwd='/', env=cleanup_env) + except KeyError: + pass + except Exception as error: + cleanup_errors.append(str(error)) + try: + shutil.rmtree(base) + except Exception as error: + cleanup_errors.append(str(error)) + os.umask(old_umask) + if cleanup_errors: + raise RuntimeError('Fixture cleanup failed: ' + '; '.join(cleanup_errors)) from failure + if failure: + raise failure + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--isolated-ci', action='store_true') + parser.add_argument('--cli', type=Path, required=True) + parser.add_argument('--node', type=Path, required=True) + args = parser.parse_args() + try: + smoke(args) + except Exception as error: + print(f'FAIL: {error}', file=sys.stderr, flush=True) + return 1 + return 0 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/src/cli/index.ts b/src/cli/index.ts index 24955dc..e0c6655 100644 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -1836,10 +1836,11 @@ async function runInstallCommand( } const targets = await resolveCliTargets(normalizedOptions); - if (behavior.apply && targets.some((target) => target.transport === "local-run-as")) { - throw new Error("local-run-as is read-only in PR1; apply and runtime mutations are not supported."); - } + for (const target of targets) { + if (target.transport === "local-run-as" && (target.reloadRuntimes || shouldReloadRuntimes(normalizedOptions))) { + throw new Error("local-run-as refuses runtime reload before writes."); + } const targetOptions = optionsForResolvedTarget(normalizedOptions, target); const config = await readMergedWorkspaceConfig(target.workspaceRoot); const configured = nameOrSource ? findConfiguredPackageForTarget(packagesForCliTarget(config.packages, target, targetOptions), nameOrSource, targetOptions, target) : undefined; @@ -2351,8 +2352,10 @@ function optionsForResolvedTarget void }) { +async function resolveAdapterForTarget(target: RuntimeTarget, options: { adapterConfig?: string; adapterModule?: string; allowAdapterCode?: boolean; reloadRuntimes?: boolean; restartRuntimes?: boolean; warn?: (message: string) => void }) { const adapterOptions = adapterOptionsForTarget(target, options); + if (target.transport === "local-run-as" && (target.reloadRuntimes || shouldReloadRuntimes(options))) throw new Error("local-run-as refuses runtime reload before writes."); + if (target.transport === "local-run-as" && adapterOptions.adapterModule) throw new Error("local-run-as refuses programmatic adapters before loading code."); return resolveAdapter({ adapter: target.adapter, adapterConfig: adapterOptions.adapterConfig, diff --git a/src/install/apply.ts b/src/install/apply.ts index 5e6e3ef..1b93208 100644 --- a/src/install/apply.ts +++ b/src/install/apply.ts @@ -10,6 +10,7 @@ import type { GraphLock } from "../model/graph-lock.js"; import { canonicalGraphLockJson, readGraphLock, writeGraphLock } from "../model/graph-lock.js"; import { localTransport } from "../transport/index.js"; import type { TargetTransport } from "../transport/index.js"; +import { assertRunAsPlanSupported } from "../transport/run-as-policy.js"; import { mergeJsonFile } from "./json-merge.js"; import { mergeOpenClawJsonFile } from "./openclaw-json-merge.js"; import { canonicalInstallManifestJson, computeInstallManifestInventoryRevision, computeSourceLockRevision, listInstallManifests, readInstallManifest, readSourceLock, removeStateFiles, withManifestRevision, writeInstallManifest, writeSourceLock } from "./manifest.js"; @@ -21,6 +22,7 @@ import { applyJournalPath, assertGovernedRuntimeTransportSupported, assertApplyJournalRecoveryAllowed, + assertLinkedApplyJournal, listApplyJournals, localPathExists, readApplyJournal, @@ -97,22 +99,52 @@ export async function applyCombinedInstallPlan(plan: InstallPlan, options: Apply return applyPlanTransactionally(plan, options); } +export interface ApplyRecoveryLink { + path: string; + operationId: string; + journalDigest: string; + transportDescription: string; + reason: string; + noCommit: boolean; + resolveOnly?: boolean; +} + export async function recoverPendingApply( targetRoot: string, adapter: string, transport: TargetTransport = localTransport, scope: { installationType?: string; stateKey?: string } = {}, + expectedLink?: ApplyRecoveryLink, ): Promise { assertGovernedRuntimeTransportSupported(transport); + if (transport.kind === "local-run-as" && !expectedLink) throw new Error("Run-as recovery requires a caller-owned durable receipt link."); + const validateJournal = (journal: ApplyJournal): void => { + if (expectedLink) { + if (expectedLink.path !== applyJournalPath(targetRoot, adapter, scope)) throw new Error("Recovery receipt journal path mismatch."); + assertLinkedApplyJournal(journal, expectedLink.path, expectedLink.operationId, expectedLink.journalDigest, expectedLink.transportDescription, transport); + if (journal.mutation?.reason !== expectedLink.reason || journal.mutation.noCommit !== expectedLink.noCommit) { + throw new Error("Runtime apply journal mutation metadata does not match its durable receipt."); + } + } + assertApplyJournalRecoveryAllowed(journal); + assertRunAsPlanSupported({ operations: journal.operations, adapterCode: journal.manifest.adapterCode } as InstallPlan, transport); + if (journal.operations.some((operation) => operation.action === "plugin" || operation.action === "program" || operation.semanticPlugin)) { + throw new Error("Cannot automatically recover a journal containing semantic plugin or programmatic operations"); + } + }; + await assertRuntimeJournalGate(targetRoot, adapter, transport, scope, true); + const preflightJournal = await readApplyJournal(targetRoot, adapter, transport, scope); + if (!preflightJournal) return undefined; + validateJournal(preflightJournal); const lock = await acquireApplyLock(targetRoot, adapter, transport, {}, scope); try { await assertRuntimeJournalGate(targetRoot, adapter, transport, scope, true); const journal = await readApplyJournal(targetRoot, adapter, transport, scope); if (!journal) return undefined; - assertApplyJournalRecoveryAllowed(journal); - - if (journal.operations.some((operation) => operation.action === "plugin" || operation.action === "program" || operation.semanticPlugin)) { - throw new Error("Cannot automatically recover a journal containing semantic plugin or programmatic operations"); + validateJournal(journal); + if (expectedLink?.resolveOnly) { + await removeApplyJournal(targetRoot, adapter, transport, scope, journal); + return journal.manifest; } const now = new Date().toISOString(); @@ -121,7 +153,7 @@ export async function recoverPendingApply( for (const [index, operation] of journal.operations.entries()) { assertOperationContained(operation, journal.targetRoot); const started = startedByIndex.get(index); - if (started?.completed || (started && await operationLanded(operation, transport))) { + if (started && await operationLanded(operation, transport)) { started.completed = true; await writeApplyJournal(journal, transport); const entry = await entryForCompletedOperation(operation, transport, now, journal.graphLockDigest); @@ -131,7 +163,7 @@ export async function recoverPendingApply( if (operationNeedsSource(operation) && operation.sourcePath && !(await localPathExists(operation.sourcePath))) { await rollbackStartedOperations(journal, transport); - await removeApplyJournal(targetRoot, adapter, transport, scope); + await removeApplyJournal(targetRoot, adapter, transport, scope, journal); return undefined; } @@ -196,6 +228,7 @@ async function applyPlanTransactionally( } assertGovernedRuntimeTransportSupported(transport); + assertRunAsPlanSupported(plan, transport); const lock = await acquireApplyLock(plan.targetRoot, plan.adapter, transport, options.lock, scope); try { @@ -297,6 +330,7 @@ export async function uninstall(plan: InstallPlan, options: UninstallOptions | b if (resolvedOptions.dryRun) return { removed: resolvedOptions.keepFiles ? 0 : removable.length, kept: kept.length, removedDrifted }; assertGovernedRuntimeTransportSupported(transport); + assertRunAsPlanSupported(plan, transport); const preservedKept = resolvedOptions.keepFiles ? kept.filter((operation) => shouldPreserveKeptOperationWhenKeepingFiles(operation)) @@ -475,7 +509,7 @@ async function commitJournalState( await removeApplyJournal(journal.targetRoot, journal.adapter, transport, { installationType: journal.installationType, stateKey: journal.stateKey, - }); + }, journal); if (await readApplyJournal(journal.targetRoot, journal.adapter, transport, { installationType: journal.installationType, stateKey: journal.stateKey, @@ -732,7 +766,7 @@ async function applyOperation( } else if (operation.mergeStrategy) { throw new Error(`Merge strategy '${operation.mergeStrategy}' is not implemented by apply.`); } else { - await transport.atomicCopy(operation.sourcePath, operation.destPath, operation.kind); + await transport.atomicCopy(operation.sourcePath, operation.destPath, operation.kind, operation.desiredHash); } const hash = await transport.hashPath(operation.destPath); if (!operation.mergeStrategy && hash !== operation.desiredHash) { @@ -1383,6 +1417,11 @@ async function validateOpenClawConfig( destPath: string, transport: TargetTransport, ): Promise { + if (transport.kind === "local-run-as") { + if (!transport.validateOpenClawConfig) throw new Error("Run-as transport lacks the allowlisted OpenClaw validator."); + await transport.validateOpenClawConfig(configPath); + return; + } if (!transport.execFile) { throw new Error(`Cannot validate OpenClaw config over ${transport.description}: transport does not support command execution.`); } diff --git a/src/install/transaction.ts b/src/install/transaction.ts index e64b9d2..cea62e1 100644 --- a/src/install/transaction.ts +++ b/src/install/transaction.ts @@ -6,6 +6,7 @@ import type { GraphLock } from "../model/graph-lock.js"; import type { ExactManagedInstructionBlockSnapshot } from "./instructions-block.js"; import { localTransport } from "../transport/index.js"; import type { TargetTransport } from "../transport/index.js"; +import type { RuntimeJournalReceipt } from "../mutation/receipts.js"; import type { InstallOperation } from "./plan.js"; import { metadataDir, stateKeyFor, type InstallStateScope } from "./paths.js"; import { @@ -157,6 +158,8 @@ export async function acquireApplyLock( options: ApplyLockOptions = {}, scope: InstallStateScope = {}, ): Promise { + assertGovernedRuntimeTransportSupported(transport); + if (transport.kind === "local-run-as") await transport.prepareApply!(); const lockPath = applyLockPath(targetRoot, adapter, scope); const ownerPath = join(lockPath, "owner.json"); const metadata = { @@ -185,17 +188,12 @@ export async function acquireApplyLock( } export async function writeApplyJournal(journal: ApplyJournal, transport: TargetTransport = localTransport): Promise { - if (transport.kind === "local-run-as") throw new Error("local-run-as is read-only in PR1; apply and runtime mutations are not supported."); + assertGovernedRuntimeTransportSupported(transport); const path = applyJournalPath(journal.targetRoot, journal.adapter, { installationType: journal.installationType, stateKey: journal.stateKey, }); - let link: { - path: string; - transport: Exclude; - transportDescription: string; - journalDigest: string; - } | undefined; + let link: Omit | undefined; if (journal.mutation) { const digest = applyJournalLinkDigest(journal, transport); if (journal.mutation.journalDigest && journal.mutation.journalDigest !== digest) { @@ -214,6 +212,7 @@ export async function writeApplyJournal(journal: ApplyJournal, transport: Target transport: transport.kind, transportDescription: transport.description, journalDigest: digest, + ...(transport.kind === "local-run-as" ? { runAsIdentity: transport.runAsIdentity! } : {}), }; await reserveActiveRuntimeJournal(link); } @@ -281,9 +280,26 @@ export async function readLinkedLocalApplyJournal( operationId: string, expectedDigest: string, expectedTransportDescription: string, + transport: TargetTransport = localTransport, ): Promise { + assertGovernedRuntimeTransportSupported(transport); + if (transport.description !== expectedTransportDescription) throw new Error("Runtime apply journal transport does not match its durable receipt."); const absolutePath = resolve(path); - const journal = parseApplyJournal(JSON.parse(await readFile(absolutePath, "utf8"))); + const journal = parseApplyJournal(JSON.parse(await transport.readFile(absolutePath))); + assertLinkedApplyJournal(journal, absolutePath, operationId, expectedDigest, expectedTransportDescription, transport); + return journal; +} + +/** Validate the exact journal object that recovery will consume while holding the runtime lock. */ +export function assertLinkedApplyJournal( + journal: ApplyJournal, + absolutePath: string, + operationId: string, + expectedDigest: string, + expectedTransportDescription: string, + transport: TargetTransport, +): void { + if (transport.description !== expectedTransportDescription) throw new Error("Runtime apply journal transport does not match its durable receipt."); if (journal.version !== 2 || !journal.mutation) { throw new Error(`Runtime apply journal ${absolutePath} is not linked to a governed mutation.`); } @@ -292,13 +308,13 @@ export async function readLinkedLocalApplyJournal( `Runtime apply journal ${absolutePath} belongs to mutation ${journal.mutation.operationId}, not ${operationId}.`, ); } - if (journal.mutation.transport?.kind !== "local" + if (journal.mutation.transport?.kind !== transport.kind || journal.mutation.transport.description !== expectedTransportDescription) { throw new Error(`Runtime apply journal ${absolutePath} transport metadata does not match its durable receipt.`); } if (journal.mutation.journalDigest !== expectedDigest || applyJournalLinkDigest(journal, { - kind: "local", + kind: transport.kind, description: expectedTransportDescription, }) !== expectedDigest) { throw new Error(`Runtime apply journal ${absolutePath} digest does not match its durable receipt.`); @@ -310,11 +326,13 @@ export async function readLinkedLocalApplyJournal( if (absolutePath !== expectedPath) { throw new Error(`Runtime apply journal path mismatch: expected ${expectedPath}, found ${absolutePath}.`); } - return journal; + assertApplyJournalCompletedOperations(journal); } -export async function removeApplyJournal(targetRoot: string, adapter: string, transport: TargetTransport = localTransport, scope: InstallStateScope = {}): Promise { - const existing = await readApplyJournal(targetRoot, adapter, transport, scope); +export const readLinkedApplyJournal = readLinkedLocalApplyJournal; + +export async function removeApplyJournal(targetRoot: string, adapter: string, transport: TargetTransport = localTransport, scope: InstallStateScope = {}, consumedJournal?: ApplyJournal): Promise { + const existing = consumedJournal ?? await readApplyJournal(targetRoot, adapter, transport, scope); const path = applyJournalPath(targetRoot, adapter, scope); if (existing?.mutation) await beginResolveMutationRuntimeJournal(existing.mutation.operationId, path); await transport.rm(applyJournalPath(targetRoot, adapter, scope)); @@ -364,11 +382,11 @@ export function mutationMetadataForApplyJournal(): ApplyJournal["mutation"] { } export function assertGovernedRuntimeTransportSupported(transport: TargetTransport): void { - if (transport.kind === "local-run-as") { - throw new Error("local-run-as is read-only in PR1; apply and runtime mutations are not supported."); + if (transport.kind === "local-run-as" && (!transport.runAsIdentity || !transport.prepareApply)) { + throw new Error("Local run-as apply requires a pinned identity and apply preparation."); } const active = activeMutationMetadata(); - if (active && transport.kind !== "local") { + if (active && transport.kind === "ssh") { throw new Error( `Governed runtime apply refuses ${transport.description} before writes; durable remote journal recovery is not implemented.`, ); @@ -387,7 +405,7 @@ export function applyJournalLinkDigest( reason: journal.mutation.reason, noCommit: journal.mutation.noCommit, }, - transport, + transport: { kind: transport.kind, description: transport.description }, mode: journal.mode ?? "apply", adapter: journal.adapter, installationType: journal.installationType ?? null, @@ -400,6 +418,14 @@ export function applyJournalLinkDigest( stateRemoveKey: journal.stateRemoveKey ?? null, workspaceConfigPath: journal.workspaceConfigPath ?? null, operations: journal.operations, + // Preserve existing local v2 receipt digests across upgrades. Run-as is a + // new contract: every caller-side write payload must be pinned. + ...(transport.kind === "local-run-as" ? { + manifest: journal.manifest, + sourceLock: journal.sourceLock ?? null, + graphLock: journal.graphLock ?? null, + workspaceConfig: journal.workspaceConfig ?? null, + } : {}), ...(journal.metadataAudit ? { metadataAudit: journal.metadataAudit, manifest: journal.manifest } : {}), ...(journal.metadataGuards ? { metadataGuards: journal.metadataGuards, manifest: journal.manifest } : {}), })).digest("hex"); @@ -434,6 +460,7 @@ function parseApplyJournal(value: unknown): ApplyJournal { journalDigest: parseJournalDigest(journal.mutation.journalDigest), }; } + assertApplyJournalCompletedOperations(journal); return journal; } @@ -442,7 +469,7 @@ function parseMutationTransport(value: unknown): NonNullable 1024) { @@ -478,7 +505,7 @@ export async function recordBackup( scope: InstallStateScope = {}, ): Promise { const hadExisting = await transport.pathExists(operation.destPath); - if (!hadExisting || transport.kind !== "local" || (operation.action !== "update" && operation.action !== "remove" && operation.action !== "create")) { + if (!hadExisting || transport.kind === "ssh" || (operation.action !== "update" && operation.action !== "remove" && operation.action !== "create")) { return { index, destPath: operation.destPath, @@ -488,9 +515,16 @@ export async function recordBackup( } const backupPath = join(applyBackupDir(targetRoot, adapter, scope), String(index)); - await rm(backupPath, { recursive: true, force: true }); - await mkdir(dirname(backupPath), { recursive: true }); - await cp(operation.destPath, backupPath, { recursive: operation.kind === "dir", dereference: true }); + if (transport.kind === "local-run-as") { + assertGovernedRuntimeTransportSupported(transport); + await transport.rm(backupPath); + if (!transport.copyWithin) throw new Error("Run-as backups require confined target copies."); + await transport.copyWithin(operation.destPath, backupPath, operation.kind); + } else { + await rm(backupPath, { recursive: true, force: true }); + await mkdir(dirname(backupPath), { recursive: true }); + await cp(operation.destPath, backupPath, { recursive: operation.kind === "dir", dereference: true }); + } return { index, destPath: operation.destPath, @@ -506,7 +540,10 @@ export async function rollbackCompletedOperations( ): Promise { for (const item of [...completed].sort((a, b) => b.index - a.index)) { if (item.hadExisting && item.backupPath) { - await transport.atomicCopy(item.backupPath, item.destPath, item.kind); + if (transport.kind === "local-run-as") { + if (!transport.copyWithin) throw new Error("Run-as rollback requires confined target copies."); + await transport.copyWithin(item.backupPath, item.destPath, item.kind); + } else await transport.atomicCopy(item.backupPath, item.destPath, item.kind); } else if (!item.hadExisting) { await transport.rm(item.destPath); } else { @@ -558,3 +595,29 @@ export async function localPathExists(path: string): Promise { return false; } } + +/** Progress is mutable, but may only reference the immutable operation and its deterministic backup. */ +export function assertApplyJournalCompletedOperations(journal: ApplyJournal): void { + if (!Array.isArray(journal.operations) || !Array.isArray(journal.completed)) throw new Error("Invalid apply journal operations or progress."); + const indices = new Set(); + for (const item of journal.completed) { + if (!item || !Number.isSafeInteger(item.index) || item.index < 0 || indices.has(item.index)) { + throw new Error("Invalid or duplicate apply journal completion index."); + } + indices.add(item.index); + const operation = journal.operations[item.index]; + if (!operation || !["create", "update", "remove"].includes(operation.action) + || item.destPath !== operation.destPath || item.kind !== operation.kind + || typeof item.hadExisting !== "boolean" + || (item.completed !== undefined && typeof item.completed !== "boolean")) { + throw new Error("Apply journal completion does not match its immutable operation."); + } + const expectedBackup = join(applyBackupDir(journal.targetRoot, journal.adapter, { + installationType: journal.installationType, stateKey: journal.stateKey, + }), String(item.index)); + if ((item.backupPath !== undefined && (item.backupPath !== expectedBackup || !item.hadExisting)) + || (journal.mutation?.transport?.kind === "local-run-as" && item.hadExisting && item.backupPath !== expectedBackup)) { + throw new Error("Apply journal completion has an invalid deterministic backup path."); + } + } +} diff --git a/src/lifecycle/profile.ts b/src/lifecycle/profile.ts index db6d128..df601c1 100644 --- a/src/lifecycle/profile.ts +++ b/src/lifecycle/profile.ts @@ -68,10 +68,7 @@ export async function syncProfile(options: ProfileSyncOptions): Promise - resolveProfileRuntimeTarget(runtime, config, options.workspaceRoot, options.installationType, options.fleetId).transport === "local-run-as")) { - throw new Error("local-run-as is read-only in PR1; apply and runtime mutations are not supported."); - } + const packages = options.source ? [await packageFromSource(options.source, options)] @@ -95,6 +92,9 @@ export async function syncProfile(options: ProfileSyncOptions): Promise a.path.localeCompare(b.path)); } + +function sameRunAsIdentity( + left: RuntimeJournalReceipt["runAsIdentity"], + right: RuntimeJournalReceipt["runAsIdentity"], +): boolean { + if (!left || !right) return left === right; + return left.user === right.user && left.uid === right.uid && left.gid === right.gid + && left.home === right.home && left.root === right.root; +} diff --git a/src/mutation/receipts.ts b/src/mutation/receipts.ts index dce2455..da05428 100644 --- a/src/mutation/receipts.ts +++ b/src/mutation/receipts.ts @@ -1,7 +1,7 @@ import { createHash, randomUUID } from "node:crypto"; import { chmod, link, mkdir, open, readFile, readdir, rename, rm } from "node:fs/promises"; import { homedir } from "node:os"; -import { dirname, join, resolve } from "node:path"; +import { dirname, isAbsolute, join, parse, resolve } from "node:path"; import { z } from "zod"; import { revisionProviderConfigSchema } from "../model/mutation.js"; import { revisionPathSchema, revisionProviderResultSchema, mutationOperationIdSchema } from "./protocol.js"; @@ -30,13 +30,27 @@ const receiptStatusSchema = z.enum([ "failed", ]); +const runAsPathSchema = z.string().min(1).max(4096).refine((path) => isAbsolute(path) && resolve(path) === path); +const runAsIdentitySchema = z.object({ + user: z.string().regex(/^[a-z_][a-z0-9_-]*[$]?$/i).refine((user) => user !== "root"), + uid: z.number().int().positive().max(Number.MAX_SAFE_INTEGER), + gid: z.number().int().nonnegative().max(Number.MAX_SAFE_INTEGER), + home: runAsPathSchema, + root: runAsPathSchema.refine((path) => path !== parse(path).root), +}).strict(); + export const runtimeJournalSchema = z.object({ path: z.string().min(1).max(4096), status: z.enum(["reserved", "pending", "resolving", "resolved"]), - transport: z.enum(["local", "ssh"]), + transport: z.enum(["local", "ssh", "local-run-as"]), + runAsIdentity: runAsIdentitySchema.optional(), transportDescription: z.string().min(1).max(1024), journalDigest: z.string().regex(/^[a-f0-9]{64}$/), -}).strict(); +}).strict().superRefine((journal, ctx) => { + if ((journal.transport === "local-run-as") !== (journal.runAsIdentity !== undefined)) { + ctx.addIssue({ code: "custom", path: ["runAsIdentity"], message: "Only local-run-as journals require a pinned run-as identity." }); + } +}); export type RuntimeJournalReceipt = z.infer; const mutationReceiptBaseSchema = z.object({ diff --git a/src/transport/run-as-fs.py b/src/transport/run-as-fs.py new file mode 100644 index 0000000..4da5c50 --- /dev/null +++ b/src/transport/run-as-fs.py @@ -0,0 +1,381 @@ +#!/usr/bin/python3 +"""Bounded, fail-closed Linux filesystem protocol. No command execution. + +All target resolution uses openat2 below one verified directory descriptor. +Only single-component mkdir/unlink/rename calls use parent directory descriptors. +""" +import argparse +import base64 +import ctypes +import errno +import json +import os +import platform +import secrets +import stat +import sys +from contextlib import contextmanager + +MAX_BYTES = 8 * 1024 * 1024 +MAX_ENTRIES = 4096 +MAX_DEPTH = 64 +MAX_REQUEST = 20 * 1024 * 1024 +RESOLVE = 0x08 | 0x04 | 0x02 # BENEATH | NO_SYMLINKS | NO_MAGICLINKS +IGNORED = {'.git', 'node_modules', '__pycache__', '.DS_Store'} + + +class How(ctypes.Structure): + _fields_ = [('flags', ctypes.c_uint64), ('mode', ctypes.c_uint64), + ('resolve', ctypes.c_uint64)] + + +def reject(message): + raise OSError(errno.EINVAL, message) + + +def relative(path, allow_root=True): + if not isinstance(path, str) or '\x00' in path or '\\' in path or path.startswith('/'): + reject('Expected a contained relative path') + if path in ('', '.') and allow_root: + return '.' + parts = path.split('/') + if len(parts) > MAX_DEPTH or any(p in ('', '.', '..') for p in parts): + reject('Invalid contained path or depth limit exceeded') + path.encode('utf-8', 'strict') + return path + + +def decode(content): + if not isinstance(content, str) or len(content) > ((MAX_BYTES + 2) // 3) * 4: + reject('File content limit exceeded') + try: + value = base64.b64decode(content, validate=True) + except (ValueError, TypeError): + reject('Invalid base64 file content') + if len(value) > MAX_BYTES: + reject('File content limit exceeded') + return value + + +class Filesystem: + def __init__(self, root, uid, gid): + if sys.platform != 'linux' or platform.machine() not in ('x86_64', 'aarch64', 'riscv64'): + raise OSError(errno.ENOSYS, 'Supported Linux openat2 architecture required') + if uid <= 0 or gid < 0 or (os.getuid(), os.geteuid(), os.getgid(), os.getegid()) != (uid, uid, gid, gid): + raise OSError(errno.EPERM, 'Pinned non-root UID and GID must match the process') + if not isinstance(root, str) or not root.startswith('/') or root == '/': + reject('Expected an absolute non-root directory') + name = relative(root[1:], False) + self.libc = ctypes.CDLL(None, use_errno=True) + self.libc.syscall.restype = ctypes.c_long + anchor = os.open('/', os.O_RDONLY | os.O_DIRECTORY | os.O_CLOEXEC) + try: + self.root = self.open_at(anchor, name, os.O_RDONLY | os.O_DIRECTORY) + finally: + os.close(anchor) + owner = os.fstat(self.root) + if (owner.st_uid, owner.st_gid) != (uid, gid): + os.close(self.root) + raise OSError(errno.EPERM, 'Root directory must have the pinned owner and group') + + def open_at(self, anchor, path, flags, mode=0): + how = How(flags | os.O_CLOEXEC, mode, RESOLVE) + fd = self.libc.syscall(ctypes.c_long(437), ctypes.c_int(anchor), + ctypes.c_char_p(os.fsencode(path)), ctypes.byref(how), ctypes.sizeof(how)) + if fd < 0: + code = ctypes.get_errno() + raise OSError(code, 'Confined openat2 failed: ' + os.strerror(code)) + return fd + + @contextmanager + def opened(self, path, flags=os.O_PATH, mode=0): + fd = self.open_at(self.root, relative(path), flags, mode) + try: + self.safe(os.fstat(fd)) + yield fd + finally: + os.close(fd) + + @staticmethod + def safe(info): + if not (stat.S_ISREG(info.st_mode) or stat.S_ISDIR(info.st_mode)): + reject('Unsupported special file or symlink') + if stat.S_ISREG(info.st_mode) and info.st_nlink != 1: + reject('Hardlinked regular files are unsupported') + return info + + def info(self, path): + with self.opened(path) as fd: + return os.fstat(fd) + + def names(self, path): + with self.opened(path, os.O_RDONLY | os.O_DIRECTORY) as fd: + names = [] + with os.scandir(fd) as entries: + for entry in entries: + relative(entry.name, False) + names.append(entry.name) + if len(names) > MAX_ENTRIES: + reject('Tree entry limit exceeded') + return sorted(names) + + def read(self, path): + with self.opened(path, os.O_RDONLY | os.O_NONBLOCK) as fd: + if not stat.S_ISREG(os.fstat(fd).st_mode): + reject('Expected a regular file') + chunks, size = [], 0 + while True: + chunk = os.read(fd, min(65536, MAX_BYTES + 1 - size)) + if not chunk: + break + size += len(chunk) + if size > MAX_BYTES: + reject('File content limit exceeded') + chunks.append(chunk) + self.safe(os.fstat(fd)) + return b''.join(chunks) + + @staticmethod + def child(parent, name): + return name if parent == '.' else parent + '/' + name + + def snapshot(self, path, include_ignored): + output, size = [], 0 + + def visit(target, suffix): + nonlocal size + if len(output) >= MAX_ENTRIES or (suffix and len(suffix.split('/')) > MAX_DEPTH): + reject('Tree entry or depth limit exceeded') + info = self.info(target) + entry = {'path': suffix, 'kind': 'dir' if stat.S_ISDIR(info.st_mode) else 'file', + 'mode': stat.S_IMODE(info.st_mode) & 0o777} + output.append(entry) + if entry['kind'] == 'dir': + for name in self.names(target): + if not include_ignored and (name in IGNORED or name.endswith(('.pyc', '.pyo'))): + continue + visit(self.child(target, name), name if not suffix else suffix + '/' + name) + else: + data = self.read(target) + size += len(data) + if size > MAX_BYTES: + reject('Tree content limit exceeded') + entry['content'] = base64.b64encode(data).decode('ascii') + visit(relative(path), '') + return output + + @contextmanager + def parent(self, path, create=False): + path = relative(path, False) + parts = path.split('/') + prefix = '.' + if create: + for part in parts[:-1]: + with self.opened(prefix, os.O_RDONLY | os.O_DIRECTORY) as fd: + try: + os.mkdir(part, 0o700, dir_fd=fd) + except FileExistsError: + pass + else: + # Persist the new inode before publishing its parent entry. + child = self.child(prefix, part) + with self.opened(child, os.O_RDONLY | os.O_DIRECTORY) as child_fd: + os.fsync(child_fd) + os.fsync(fd) + prefix = self.child(prefix, part) + with self.opened(prefix, os.O_RDONLY | os.O_DIRECTORY): + pass + parent = '/'.join(parts[:-1]) or '.' + with self.opened(parent, os.O_RDONLY | os.O_DIRECTORY) as fd: + yield fd, parts[-1] + + def mkdir(self, path): + with self.parent(path, True) as (fd, name): + os.mkdir(name, 0o700, dir_fd=fd) + with self.opened(path, os.O_RDONLY | os.O_DIRECTORY) as child_fd: + os.fsync(child_fd) + os.fsync(fd) + + def rm(self, path, budget=None): + path = relative(path, False) # The pinned root cannot be removed. + if budget is None: + budget = [0] + try: + info = self.info(path) + except FileNotFoundError: + return + budget[0] += 1 + if budget[0] > MAX_ENTRIES: + reject('Tree entry limit exceeded') + if stat.S_ISDIR(info.st_mode): + for name in self.names(path): + self.rm(self.child(path, name), budget) + with self.parent(path) as (fd, name): + os.rmdir(name, dir_fd=fd) + os.fsync(fd) + else: + with self.parent(path) as (fd, name): + os.unlink(name, dir_fd=fd) + os.fsync(fd) + + def write(self, path, data, mode=None): + path = relative(path, False) + try: + info = self.info(path) + if not stat.S_ISREG(info.st_mode): + reject('Expected a regular file destination') + existing_mode = stat.S_IMODE(info.st_mode) & 0o777 + except FileNotFoundError: + existing_mode = 0o600 + with self.parent(path, True) as (parent, name): + temp = '.agentwheel-file-' + secrets.token_hex(16) + fd = self.open_at(parent, temp, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + try: + try: + offset = 0 + while offset < len(data): + count = os.write(fd, data[offset:]) + if not count: + raise OSError(errno.EIO, 'Incomplete file write') + offset += count + os.fchmod(fd, existing_mode if mode is None else mode) + os.fsync(fd) + finally: + os.close(fd) + # A swapped symlink is replaced as a directory entry, never followed. + os.rename(temp, name, src_dir_fd=parent, dst_dir_fd=parent) + os.fsync(parent) + finally: + try: + os.unlink(temp, dir_fd=parent) + os.fsync(parent) + except FileNotFoundError: + pass + + @staticmethod + def validate_snapshot(entries): + if not isinstance(entries, list) or not entries or len(entries) > MAX_ENTRIES: + reject('Invalid snapshot entry count') + parsed, size = {}, 0 + for entry in entries: + if not isinstance(entry, dict): + reject('Invalid snapshot entry') + path = relative(entry.get('path')) + if path in parsed or entry.get('kind') not in ('file', 'dir'): + reject('Duplicate or invalid snapshot entry') + mode = entry.get('mode') + if type(mode) is not int or mode < 0 or mode > 0o777: + reject('Invalid snapshot mode') + data = decode(entry.get('content')) if entry['kind'] == 'file' else None + size += len(data) if data is not None else 0 + if size > MAX_BYTES: + reject('Tree content limit exceeded') + parsed[path] = (entry['kind'], mode, data) + if '.' not in parsed: + reject('Snapshot root is missing') + for path in parsed: + if path == '.': + continue + parent = path.rsplit('/', 1)[0] if '/' in path else '.' + if parent not in parsed or parsed[parent][0] != 'dir': + reject('Snapshot hierarchy is incomplete') + return parsed + + def install(self, path, entries): + path = relative(path, False) + parsed = self.validate_snapshot(entries) # Validate every byte before mutation. + with self.parent(path, True) as (parent, name): + temp = '.agentwheel-tree-' + secrets.token_hex(16) + prefix = path.rsplit('/', 1)[0] if '/' in path else '.' + staged = self.child(prefix, temp) + try: + if parsed['.'][0] == 'file': + self.write(staged, parsed['.'][2], parsed['.'][1]) + else: + self.mkdir(staged) + for suffix in sorted((p for p in parsed if p != '.'), key=lambda p: (p.count('/'), p)): + kind, mode, data = parsed[suffix] + target = self.child(staged, suffix) + if kind == 'dir': + self.mkdir(target) + else: + self.write(target, data, mode) + for suffix in sorted(parsed, key=lambda p: -1 if p == '.' else p.count('/'), reverse=True): + kind, mode, _ = parsed[suffix] + if kind == 'dir': + target = staged if suffix == '.' else self.child(staged, suffix) + with self.opened(target, os.O_RDONLY | os.O_DIRECTORY) as fd: + os.fchmod(fd, mode) + os.fsync(fd) + self.rm(path) + os.rename(temp, name, src_dir_fd=parent, dst_dir_fd=parent) + os.fsync(parent) + finally: + # Successful rename leaves no staging entry. Permission errors fail closed. + self.rm(staged) + + def execute(self, request): + if not isinstance(request, dict) or type(request.get('version')) is not int or request['version'] != 1: + reject('Unsupported protocol version') + operation = request.get('operation') + if operation == 'probe': + with self.opened('.', os.O_RDONLY | os.O_DIRECTORY): + return True + path = relative(request.get('path')) + if operation == 'pathExists': + try: + self.info(path) + return True + except FileNotFoundError: + return False + if operation == 'listDir': + try: + return self.names(path) + except FileNotFoundError: + return [] + if operation == 'readFile': + return base64.b64encode(self.read(path)).decode('ascii') + if operation == 'snapshot': + if type(request.get('includeIgnored')) is not bool: + reject('includeIgnored must be boolean') + return self.snapshot(path, request['includeIgnored']) + if operation == 'mkdirExclusive': + self.mkdir(path) + elif operation == 'writeFileAtomic': + self.write(path, decode(request.get('content'))) + elif operation == 'installSnapshot': + self.install(path, request.get('entries')) + elif operation == 'copyWithin': + self.install(path, self.snapshot(relative(request.get('source')), True)) + elif operation == 'rm': + self.rm(path) + else: + reject('Unsupported filesystem operation') + return None + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--root', required=True) + parser.add_argument('--uid', required=True, type=int) + parser.add_argument('--gid', required=True, type=int) + args = parser.parse_args() + filesystem = None + try: + raw = sys.stdin.buffer.read(MAX_REQUEST + 1) + if len(raw) > MAX_REQUEST: + reject('Request size limit exceeded') + request = json.loads(raw) + filesystem = Filesystem(args.root, args.uid, args.gid) + response = {'version': 1, 'result': filesystem.execute(request)} + except Exception as error: + code = errno.errorcode.get(error.errno, 'EINVAL') if isinstance(error, OSError) else 'EINVAL' + response = {'version': 1, 'error': {'message': str(error), 'code': code}} + finally: + if filesystem is not None: + os.close(filesystem.root) + print(json.dumps(response, separators=(',', ':'))) + + +if __name__ == '__main__': + main() diff --git a/src/transport/run-as-policy.ts b/src/transport/run-as-policy.ts new file mode 100644 index 0000000..090c692 --- /dev/null +++ b/src/transport/run-as-policy.ts @@ -0,0 +1,11 @@ +import type { InstallPlan } from "../install/plan.js"; +import type { TargetTransport } from "./types.js"; + +/** Reject executable adapter/plugin plans before acquiring any target lock. */ +export function assertRunAsPlanSupported(plan: InstallPlan, transport: TargetTransport): void { + if (transport.kind !== "local-run-as") return; + if (plan.adapterCode || plan.operations.some((operation) => operation.action === "program" || operation.action === "plugin" + || operation.programmaticOperation || operation.programmaticApply || operation.semanticPlugin || operation.semanticCommand)) { + throw new Error("local-run-as refuses programmatic adapters and semantic plugin operations before writes."); + } +} diff --git a/src/transport/run-as-staging.ts b/src/transport/run-as-staging.ts index a0b5bcd..fe7b931 100644 --- a/src/transport/run-as-staging.ts +++ b/src/transport/run-as-staging.ts @@ -27,7 +27,7 @@ function assertEntryPath(path: string): void { } } -function payloadHash(payload: RunAsCopyPayload): string { +export function hashRunAsCopyPayload(payload: RunAsCopyPayload): string { const fileHash = (content: string) => createHash("sha256").update("file\0").update(Buffer.from(content, "base64")).digest("hex"); if (payload.kind === "file") { const root = payload.entries[0]!; @@ -76,7 +76,7 @@ export function validateRunAsCopyPayload(value: unknown): RunAsCopyPayload { throw new Error("Run-as staged entry has no parent directory."); } } - if (payloadHash(payload) !== payload.hash) throw new Error("Run-as staged content hash mismatch."); + if (hashRunAsCopyPayload(payload) !== payload.hash) throw new Error("Run-as staged content hash mismatch."); return payload; } diff --git a/src/transport/run-as-trust.ts b/src/transport/run-as-trust.ts new file mode 100644 index 0000000..0e8ea27 --- /dev/null +++ b/src/transport/run-as-trust.ts @@ -0,0 +1,45 @@ +import { lstatSync, readdirSync, realpathSync, statSync } from "node:fs"; +import { dirname, isAbsolute, join, relative, sep } from "node:path"; + +/** Reject code the service can replace, including through writable ancestors. */ +export function trustedRunAsCode(path: string, uid: number, groups: readonly number[], label: string): string { + if (!isAbsolute(path)) throw new Error(`Run-as ${label} path must be absolute.`); + const canonical = realpathSync(path); + let child = canonical; + let info = statSync(child); + if (!info.isFile() || info.uid === uid || (info.mode & 0o022) !== 0) throw new Error(`Run-as ${label} must be an immutable regular file.`); + while (child !== dirname(child)) { + const parent = dirname(child), parentInfo = statSync(parent); + // POSIX ACL write grants require a writable group-class mask. Reject it + // regardless of owning group, covering named-user/group ACL entries too. + const writable = (parentInfo.mode & 0o002) !== 0 || (parentInfo.mode & 0o020) !== 0; + // Sticky directories protect entries owned by another user, unless the + // service itself owns the directory (and can remove any entry). + if (parentInfo.uid === uid || (writable && !((parentInfo.mode & 0o1000) && info.uid !== uid))) { + throw new Error(`Run-as ${label} has a service-writable ancestor.`); + } + child = parent; info = parentInfo; + } + return canonical; +} + +/** The validator and its package dependencies must come from a trusted install. */ +export function trustedOpenClawEntry(root: string, uid: number, groups: readonly number[]): string { + if (!isAbsolute(root) || realpathSync(root) !== root) throw new Error("OpenClaw validator root must be canonical and absolute."); + const entry = trustedRunAsCode(join(root, "openclaw.mjs"), uid, groups, "OpenClaw validator"); + let count = 0; + const visit = (path: string): void => { + if (++count > 100_000) throw new Error("OpenClaw validator installation exceeds the trust-check limit."); + const info = lstatSync(path); + if (info.uid === uid || (info.mode & 0o022) !== 0 && !info.isSymbolicLink()) throw new Error("OpenClaw validator installation is service-writable."); + if (info.isSymbolicLink()) { + const resolved = realpathSync(path), rel = relative(root, resolved); + if (rel === ".." || rel.startsWith(`..${sep}`) || isAbsolute(rel)) throw new Error("OpenClaw validator dependency escapes its trusted installation."); + return; // The resolved destination is visited through the canonical tree. + } + if (info.isDirectory()) for (const name of readdirSync(path)) visit(join(path, name)); + else if (!info.isFile()) throw new Error("OpenClaw validator installation contains a special file."); + }; + visit(root); + return entry; +} diff --git a/src/transport/run-as-worker.ts b/src/transport/run-as-worker.ts index a2c31bf..a5eeb7a 100644 --- a/src/transport/run-as-worker.ts +++ b/src/transport/run-as-worker.ts @@ -1,98 +1,145 @@ -import { lstat, readdir, readFile, realpath } from "node:fs/promises"; -import { isAbsolute, join, parse, relative, resolve, sep } from "node:path"; -import { ownershipRuntimeSnapshot } from "./ownership-snapshot.js"; -import { hashPath } from "../utils/fs.js"; +import { execFile } from "node:child_process"; +import { createHash } from "node:crypto"; +import { trustedOpenClawEntry, trustedRunAsCode } from "./run-as-trust.js"; +import { isAbsolute, join, posix, relative, resolve, sep } from "node:path"; +import { fileURLToPath } from "node:url"; +import { z } from "zod"; +import { parse as parseJsonc, type ParseError } from "jsonc-parser"; +import { readFileSync } from "node:fs"; +import { hashRunAsCopyPayload, validateRunAsCopyPayload } from "./run-as-staging.js"; export type ReadOperation = "pathExists" | "hashPath" | "readFile" | "listDir" | "ownershipSnapshot"; export interface ReadRequest { version: 1; operation: ReadOperation; path: string } +const relativePath = z.string().min(1).max(4096).refine((path) => path === "." || (!isAbsolute(path) + && !path.includes("\\") && !path.includes("\0") && path.split("/").every((part) => part !== "" && part !== "." && part !== "..")), { message: "Run-as path is not a contained relative path." }); +const requestSchema = z.object({ + version: z.literal(1), + operation: z.enum(["pathExists", "hashPath", "readFile", "listDir", "ownershipSnapshot", "probe", "mkdirExclusive", "writeFileAtomic", "copy", "copyWithin", "rm", "validateOpenClawConfig"]), + path: relativePath, + validatorRoot: z.string().optional(), validationSnapshot: z.string().optional(), content: z.string().optional(), source: relativePath.optional(), payload: z.unknown().optional(), +}).strict(); +export type RunAsRequest = z.infer; +const snapshotEntrySchema = z.object({ path: z.string(), kind: z.enum(["file", "dir"]), mode: z.number().int().min(0).max(0o7777), content: z.string().optional() }).strict(); +const helperPath = fileURLToPath(new URL("./run-as-fs.py", import.meta.url)); +const pythonPath = "/usr/bin/python3"; +const maxBytes = 20 * 1024 * 1024; -/** Resolve a relative target path without following symlink components. */ +/** Lexical gate; actual filesystem resolution is always delegated to openat2. */ export async function containedReadPath(root: string, requested: string): Promise { - if (!isAbsolute(root) || resolve(root) !== root || !requested || requested.includes("\0") || requested.includes("\\") || isAbsolute(requested) - || (requested !== "." && requested.split(sep).some((part) => part === ".." || part === "." || part === ""))) { - throw new Error("Run-as read path is not a contained relative path."); - } - let current = parse(root).root; - for (const part of root.slice(current.length).split(sep).filter(Boolean)) { - current = join(current, part); - if ((await lstat(current)).isSymbolicLink()) throw new Error("Run-as root contains a symlink."); - } - if (await realpath(root) !== root) throw new Error("Run-as root changed."); - if (requested === ".") return root; - current = root; - for (const part of requested.split(sep)) { - current = join(current, part); - try { - if ((await lstat(current)).isSymbolicLink()) throw new Error("Run-as path contains a symlink."); - } catch (error) { - if ((error as NodeJS.ErrnoException).code === "ENOENT") break; - throw error; - } - } - const rel = relative(root, current); - if (!rel || rel === ".." || rel.startsWith(`..${sep}`) || isAbsolute(rel)) throw new Error("Run-as path escapes target root."); - return current; + relativePath.parse(requested); + if (!isAbsolute(root) || resolve(root) !== root) throw new Error("Run-as root must be absolute and normalized."); + return resolve(root, requested); +} + +async function filesystem(root: string, uid: number, gid: number, request: Record): Promise { + return new Promise((resolveResult, reject) => { + const child = execFile(pythonPath, ["-I", helperPath, "--root", root, "--uid", String(uid), "--gid", String(gid)], + { timeout: 120_000, maxBuffer: maxBytes, env: { HOME: root, PATH: "/usr/local/bin:/usr/bin:/bin" } }, (error, stdout) => { + if (error) { reject(error); return; } + try { + const response = JSON.parse(stdout) as { version?: number; result?: unknown; error?: { message: string; code?: string } }; + if (response.version !== 1) throw new Error("Invalid confined filesystem response."); + if (response.error) throw Object.assign(new Error(response.error.message), { code: response.error.code }); + if (!("result" in response)) throw new Error("Missing confined filesystem result."); + resolveResult(response.result); + } catch (failure) { reject(failure); } + }); + child.stdin?.on("error", reject); + child.stdin?.end(JSON.stringify({ version: 1, ...request })); + }); } -async function rejectNestedLinks(path: string): Promise { - const stat = await lstat(path); - if (stat.isSymbolicLink()) throw new Error("Run-as read path contains a symlink."); - if (!stat.isDirectory()) return; - for (const entry of await readdir(path)) await rejectNestedLinks(join(path, entry)); +function sortedChildren(entries: T[], directory: string, locale = false): T[] { + return entries.filter((entry) => entry.path !== "" && posix.dirname(entry.path) === (directory || ".")) + .sort((left, right) => locale ? posix.basename(left.path).localeCompare(posix.basename(right.path)) + : posix.basename(left.path) < posix.basename(right.path) ? -1 : posix.basename(left.path) > posix.basename(right.path) ? 1 : 0); } -export async function executeRead(root: string, request: ReadRequest): Promise { - if (request.version !== 1 || !["pathExists", "hashPath", "readFile", "listDir", "ownershipSnapshot"].includes(request.operation)) { - throw new Error("Unsupported run-as read protocol request."); +export async function executeRequest(root: string, uid: number, gid: number, input: unknown): Promise { + const request = requestSchema.parse(input); + await containedReadPath(root, request.path); + const fs = (operation: string, fields: Record = {}) => filesystem(root, uid, gid, { operation, path: request.path, ...fields }); + if (["pathExists", "listDir", "mkdirExclusive", "rm", "probe"].includes(request.operation)) return fs(request.operation); + if (request.operation === "readFile") return Buffer.from(String(await fs("readFile")), "base64").toString("utf8"); + if (request.operation === "hashPath" || request.operation === "ownershipSnapshot") { + const entries = z.array(snapshotEntrySchema).min(1).max(4096).parse(await fs("snapshot", { includeIgnored: request.operation === "ownershipSnapshot" })); + if (request.operation === "hashPath") { + const staged = entries.map((entry) => entry.kind === "dir" + ? { path: entry.path, kind: "dir" as const, mode: 0o700 as const } + : { path: entry.path, kind: "file" as const, mode: 0o600 as const, content: entry.content! }); + const payload = { version: 1 as const, kind: entries[0]!.kind, hash: "0".repeat(64), entries: staged }; + payload.hash = hashRunAsCopyPayload(payload); + return validateRunAsCopyPayload(payload).hash; + } + const hash = createHash("sha256"); + const visit = (entry: z.infer): void => { + hash.update(JSON.stringify([entry.path, entry.kind, entry.mode])); + if (entry.kind === "dir") for (const child of sortedChildren(entries, entry.path)) visit(child); + else { const bytes = Buffer.from(entry.content!, "base64"); hash.update(String(bytes.length)).update("\0").update(bytes); } + }; + if (entries[0]!.path !== "") throw new Error("Invalid ownership snapshot root."); + visit(entries[0]!); + return hash.digest("hex"); } - const path = await containedReadPath(root, request.path); - let exists = true; - try { await lstat(path); } - catch (error) { - if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; - exists = false; + if (request.operation === "copy") { + const payload = validateRunAsCopyPayload(request.payload); + return fs("installSnapshot", { entries: payload.entries }); } - if (request.operation === "pathExists") return exists; - if (request.operation === "listDir" && !exists) return []; - if (!exists) throw new Error("Run-as target path does not exist."); - if (request.operation === "ownershipSnapshot") return ownershipRuntimeSnapshot(path); - if (request.operation === "hashPath") { - await rejectNestedLinks(path); - return hashPath(path); + if (request.operation === "copyWithin") { + if (!request.source) throw new Error("Run-as copyWithin requires a contained source."); + return fs("copyWithin", { source: request.source }); } - if (request.operation === "readFile") { - if (!(await lstat(path)).isFile()) throw new Error("Run-as readFile requires a regular file."); - return readFile(path, "utf8"); + if (request.operation === "writeFileAtomic") { + if (request.content === undefined) throw new Error("Run-as write requires content."); + return fs("writeFileAtomic", { content: Buffer.from(request.content).toString("base64") }); } - return readdir(path); + // This is the sole subprocess operation; no caller-supplied command or argv exists. + await fs("readFile"); // Confined regular-file/path verification before validation. + const groups = [...new Set([gid, ...(process.getgroups?.() ?? [])])]; + const entry = trustedOpenClawEntry(request.validatorRoot ?? "/usr/local/lib/node_modules/openclaw", uid, groups); + const binary = trustedRunAsCode(process.execPath, uid, groups, "Node executable"); + if (!request.validationSnapshot) throw new Error("OpenClaw validation requires a caller-owned config snapshot."); + const configPath = trustedRunAsCode(request.validationSnapshot, uid, groups, "validation snapshot"); + assertSelfContainedOpenClawConfig(readFileSync(configPath, "utf8")); + const result = await new Promise((resolveResult, reject) => { + execFile(binary, [entry, "config", "validate", "--json"], { + timeout: 120_000, maxBuffer: maxBytes, cwd: root, + env: { HOME: root, PATH: "/usr/local/bin:/usr/bin:/bin", OPENCLAW_CONFIG_PATH: configPath }, + }, (error, stdout) => error ? reject(new Error("OpenClaw config validator failed.")) : resolveResult(stdout)); + }); + if (JSON.parse(result)?.valid !== true) throw new Error("OpenClaw config validation failed."); + return null; +} + +export function assertSelfContainedOpenClawConfig(content: string): void { + const errors: ParseError[] = []; + const config = parseJsonc(content, errors, { allowTrailingComma: true, disallowComments: false }) as unknown; + if (errors.length) throw new Error("local-run-as OpenClaw validation requires valid JSON/JSONC syntax; render a self-contained config."); + const rejectIncludes = (value: unknown): void => { + if (!value || typeof value !== "object") return; + if (!Array.isArray(value) && Object.hasOwn(value, "$include")) throw new Error("local-run-as OpenClaw validation does not support $include; render a self-contained config."); + for (const child of Object.values(value)) rejectIncludes(child); + }; + rejectIncludes(config); +} + +export async function executeRead(root: string, request: ReadRequest): Promise { + return executeRequest(root, process.getuid!(), process.getgid!(), request); } async function main(): Promise { const [flagRoot, root, flagUid, rawUid, flagGid, rawGid] = process.argv.slice(2); - if (flagRoot !== "--root" || flagUid !== "--uid" || flagGid !== "--gid" || !root || !rawUid || !rawGid || process.argv.length !== 8) { - throw new Error("Invalid run-as worker arguments."); + if (flagRoot !== "--root" || flagUid !== "--uid" || flagGid !== "--gid" || !root || !rawUid || !rawGid || process.argv.length !== 8) throw new Error("Invalid run-as worker arguments."); + const uid = Number(rawUid), gid = Number(rawGid); + if (!Number.isSafeInteger(uid) || uid <= 0 || !Number.isSafeInteger(gid) || gid < 0 || process.getuid?.() !== uid || process.getgid?.() !== gid) throw new Error("Run-as worker UID/GID mismatch."); + for (const code of [helperPath, pythonPath]) trustedRunAsCode(code, uid, [...new Set([gid, ...(process.getgroups?.() ?? [])])], "helper/interpreter"); + const chunks: Buffer[] = []; let size = 0; + for await (const chunk of process.stdin) { size += chunk.length; if (size > maxBytes) throw new Error("Run-as request exceeds 20 MiB."); chunks.push(chunk); } + try { + const result = await executeRequest(root, uid, gid, JSON.parse(Buffer.concat(chunks).toString("utf8"))); + process.stdout.write(`${JSON.stringify({ version: 1, result })}\n`); + } catch (error) { + process.stdout.write(`${JSON.stringify({ version: 1, error: { message: error instanceof Error ? error.message : String(error), code: (error as NodeJS.ErrnoException).code } })}\n`); } - const uid = Number(rawUid); - const gid = Number(rawGid); - if (!Number.isSafeInteger(uid) || uid <= 0 || !Number.isSafeInteger(gid) || gid < 0 - || process.getuid?.() !== uid || process.getgid?.() !== gid || (await lstat(root)).uid !== uid) { - throw new Error("Run-as worker UID or root ownership mismatch."); - } - const chunks: Buffer[] = []; - let size = 0; - for await (const chunk of process.stdin) { - size += chunk.length; - if (size > 65536) throw new Error("Run-as request exceeds 64 KiB."); - chunks.push(chunk); - } - const request = JSON.parse(Buffer.concat(chunks).toString("utf8")) as ReadRequest; - const result = await executeRead(root, request); - process.stdout.write(`${JSON.stringify({ version: 1, result })}\n`); -} - -if (process.argv[1] && /run-as-worker\.(?:js|mjs|ts)$/.test(process.argv[1])) { - main().catch((error) => { - process.stderr.write(`run-as worker: ${error instanceof Error ? error.message : String(error)}\n`); - process.exitCode = 1; - }); } +if (process.argv[1] && /run-as-worker\.(?:js|mjs|ts)$/.test(process.argv[1])) main().catch((error) => { process.stderr.write(`run-as worker: ${error instanceof Error ? error.message : String(error)}\n`); process.exitCode = 1; }); diff --git a/src/transport/run-as.ts b/src/transport/run-as.ts index 8d5a764..7ffa5dd 100644 --- a/src/transport/run-as.ts +++ b/src/transport/run-as.ts @@ -1,42 +1,61 @@ -import { execFile } from "node:child_process"; -import { accessSync, chmodSync, constants, copyFileSync, mkdtempSync, realpathSync, rmSync, statSync } from "node:fs"; +import { execFile, execFileSync } from "node:child_process"; +import { accessSync, chmodSync, constants, copyFileSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { isAbsolute, join, relative, resolve, sep } from "node:path"; import { fileURLToPath } from "node:url"; +import { z } from "zod"; import { promisify } from "node:util"; import type { RunAsIdentity } from "./run-as-identity.js"; -import type { ReadOperation, ReadRequest } from "./run-as-worker.js"; +import type { RunAsRequest } from "./run-as-worker.js"; +import { trustedRunAsCode } from "./run-as-trust.js"; +import { resolveRunAsIdentity } from "./run-as-identity.js"; +import { stageRunAsCopy } from "./run-as-staging.js"; +import { hashPath } from "../utils/fs.js"; import type { TargetTransport } from "./types.js"; const execFileAsync = promisify(execFile); const workerPath = fileURLToPath(new URL("./run-as-worker.js", import.meta.url)); const nodePath = process.env.AGENTWHEEL_RUN_AS_NODE ?? "/usr/bin/node"; const runAsTimeoutMs = 120_000; -const readonlyError = "local-run-as is read-only in PR1; apply and runtime mutations are not supported."; +const commandError = "local-run-as permits only the dedicated OpenClaw config validation operation; arbitrary commands are forbidden."; +// Caller-side sharing grants only the intended service UID access. Linux ACL +// masks are encoded directly through Python's stdlib; no shell or ACL utility. +const validationAclScript = `import os,stat,struct,sys +uid=int(sys.argv[2]) +if uid <= 0 or uid == os.geteuid(): raise RuntimeError("distinct non-root service UID required") +def acl(owner,service,mask): + entries=[(1,owner,0xffffffff),(2,service,uid),(4,0,0xffffffff),(16,mask,0xffffffff),(32,0,0xffffffff)] + return struct.pack("(); let cleanupInstalled = false; -function assertTrustedFile(path: string, uid: number, label: string): string { - if (!isAbsolute(path)) throw new Error(`Run-as ${label} path must be absolute: ${path}`); - const canonical = realpathSync(path); - const stat = statSync(canonical); - if (!stat.isFile() || stat.uid === uid || (stat.mode & 0o022) !== 0) { - throw new Error(`Run-as ${label} is writable by the service user or is not a regular file: ${path}`); - } - return canonical; -} - function cleanupStagedWorkers(): void { for (const { directory } of stagedWorkers.values()) rmSync(directory, { recursive: true, force: true }); stagedWorkers.clear(); } -function stageWorkerOnce(worker: string): string { +function stageWorkerOnce(worker: string, helperSource: string): string { const existing = stagedWorkers.get(worker); if (existing) return existing.worker; const directory = mkdtempSync(join("/tmp", "agentwheel-run-as-")); try { const staged = join(directory, "run-as-worker.mjs"); copyFileSync(worker, staged); + const helper = join(directory, "run-as-fs.py"); + copyFileSync(helperSource, helper); + chmodSync(helper, 0o444); chmodSync(staged, 0o444); chmodSync(directory, 0o711); stagedWorkers.set(worker, { directory, worker: staged }); @@ -72,15 +91,22 @@ export async function defaultRunAsRunner(command: string, args: string[], input? } export function createRunAsTransport(identity: RunAsIdentity, runner: RunAsRunner = defaultRunAsRunner, worker = workerPath, nodeExecutable = nodePath): TargetTransport { + identity = Object.freeze({ ...identity }); if (!isAbsolute(identity.root) || resolve(identity.root) !== identity.root || identity.uid <= 0) throw new Error("Invalid run-as target identity."); - const executable = assertTrustedFile(nodeExecutable, identity.uid, "Node executable"); + const groups = runner === defaultRunAsRunner + ? execFileSync("/usr/bin/id", ["-G", identity.user], { encoding: "utf8", timeout: 10_000 }).trim().split(/\s+/).map(Number) + : [identity.gid]; + if (!groups.length || groups.some((group) => !Number.isSafeInteger(group) || group < 0) || !groups.includes(identity.gid)) throw new Error("Invalid run-as supplementary groups."); + const trust = (path: string, label: string) => trustedRunAsCode(path, identity.uid, groups, label); + const executable = trust(nodeExecutable, "Node executable"); let stagedWorker = worker; if (runner === defaultRunAsRunner) { accessSync(worker, constants.R_OK); - const trustedWorker = assertTrustedFile(worker, identity.uid, "worker"); + const trustedWorker = trust(worker, "worker"); + const trustedHelper = trust(fileURLToPath(new URL("./run-as-fs.py", import.meta.url)), "filesystem helper"); // Recovery planning may set TMPDIR to a private 0700 directory. The worker // needs a caller-owned staging directory traversable by the service user. - stagedWorker = stageWorkerOnce(trustedWorker); + stagedWorker = stageWorkerOnce(trustedWorker, trustedHelper); } const prefix = ["-n", "-u", identity.user, "-H", "--"]; let preflight: Promise | undefined; @@ -93,37 +119,87 @@ export function createRunAsTransport(identity: RunAsIdentity, runner: RunAsRunne })(); return preflight; } - async function run(operation: ReadOperation, path: string): Promise { + async function run(operation: RunAsRequest["operation"], path: string, fields: Partial = {}): Promise { const rel = relative(identity.root, path) || "."; if (rel === ".." || rel.startsWith(`..${sep}`) || isAbsolute(rel)) throw new Error(`Run-as read path escapes ${identity.root}: ${path}`); await verifyUid(); - const request: ReadRequest = { version: 1, operation, path: rel }; + const request: RunAsRequest = { ...fields, version: 1, operation, path: rel }; + const input = JSON.stringify(request); + if (Buffer.byteLength(input) > 20 * 1024 * 1024) throw new Error("Run-as request exceeds 20 MiB."); const args = [...prefix, "/usr/bin/env", "-i", `HOME=${identity.home}`, "PATH=/usr/local/bin:/usr/bin:/bin", executable, stagedWorker, "--root", identity.root, "--uid", String(identity.uid), "--gid", String(identity.gid)]; - const { stdout } = await runner("/usr/bin/sudo", args, JSON.stringify(request)); + const { stdout } = await runner("/usr/bin/sudo", args, input); if (stdout.length > 20 * 1024 * 1024) throw new Error("Run-as worker result exceeds 20 MiB."); - const response = JSON.parse(stdout) as { version?: number; result?: unknown }; + const response = JSON.parse(stdout) as { version?: number; result?: unknown; error?: { message: string; code?: string } }; + if (response.error) throw Object.assign(new Error(response.error.message), { code: response.error.code }); if (response.version !== 1 || !("result" in response)) throw new Error("Invalid run-as worker response."); - return response.result; + const schema = operation === "pathExists" || operation === "probe" ? z.boolean() + : operation === "hashPath" || operation === "ownershipSnapshot" ? z.string().regex(/^[a-f0-9]{64}$/) + : operation === "readFile" ? z.string() + : operation === "listDir" ? z.array(z.string().min(1).refine((name) => !/[\\/\0]/.test(name) && name !== "." && name !== "..")) + : z.null(); + return schema.parse(response.result); + } + let ready: Promise | undefined; + async function prepareApply(): Promise { + ready ??= (async () => { + if (runner === defaultRunAsRunner) { + const current = resolveRunAsIdentity(identity.user, identity.root); + if (["user", "uid", "gid", "home", "root"].some((key) => current[key as keyof RunAsIdentity] !== identity[key as keyof RunAsIdentity])) { + throw new Error("Run-as account identity changed before apply."); + } + } + await run("probe", identity.root); + })(); + return ready; } return { kind: "local-run-as", description: `local-run-as://${identity.user}:${identity.uid}@${identity.root}`, homeRoot: identity.home, + runAsIdentity: Object.freeze({ ...identity }), + prepareApply, pathExists: async (path) => Boolean(await run("pathExists", path)), hashPath: async (path) => String(await run("hashPath", path)), readFile: async (path) => String(await run("readFile", path)), ownershipSnapshot: async (path) => String(await run("ownershipSnapshot", path)), listDir: async (path) => (await run("listDir", path)) as string[], - mkdirExclusive: async () => { throw new Error(readonlyError); }, - writeFileAtomic: async () => { throw new Error(readonlyError); }, - writeJsonAtomic: async () => { throw new Error(readonlyError); }, - atomicCopy: async () => { throw new Error(readonlyError); }, - rm: async () => { throw new Error(readonlyError); }, - execFile: async () => { throw new Error(readonlyError); }, + mkdirExclusive: async (path) => { await prepareApply(); await run("mkdirExclusive", path); }, + writeFileAtomic: async (path, content) => { await prepareApply(); await run("writeFileAtomic", path, { content }); }, + writeJsonAtomic: async (path, data) => { await prepareApply(); await run("writeFileAtomic", path, { content: `${JSON.stringify(data, null, 2)}\n` }); }, + atomicCopy: async (source, dest, kind, expectedHash) => { + await prepareApply(); + const payload = await stageRunAsCopy(source, kind, expectedHash ?? await hashPath(source)); + await run("copy", dest, { payload }); + }, + copyWithin: async (source, dest) => { + await prepareApply(); + const rel = relative(identity.root, source); + if (rel === ".." || rel.startsWith(`..${sep}`) || isAbsolute(rel)) throw new Error("Run-as backup source escapes target root."); + await run("copyWithin", dest, { source: rel || "." }); + }, + rm: async (path) => { await prepareApply(); await run("rm", path); }, + validateOpenClawConfig: async (path) => { + await prepareApply(); + const content = String(await run("readFile", path)); + const directory = mkdtempSync(join("/tmp", "agentwheel-validate-")); + try { + chmodSync(directory, 0o700); + const snapshot = join(directory, "openclaw.json"); + writeFileSync(snapshot, content, { mode: 0o600, flag: "wx" }); + chmodSync(snapshot, 0o600); + const python = trust("/usr/bin/python3", "Python executable"); + try { + await execFileAsync(python, ["-I", "-c", validationAclScript, directory, String(identity.uid)], { + timeout: 10_000, maxBuffer: 4096, env: { PATH: "/usr/bin:/bin" }, + }); + } catch { throw new Error("OpenClaw validation snapshot requires Linux POSIX ACL support for the service UID."); } + await run("validateOpenClawConfig", path, { + validatorRoot: process.env.AGENTWHEEL_RUN_AS_OPENCLAW_ROOT ?? "/usr/local/lib/node_modules/openclaw", + validationSnapshot: snapshot, + }); + } finally { rmSync(directory, { recursive: true, force: true }); } + }, + execFile: async () => { throw new Error(commandError); }, }; } - -export function assertRunAsReadOnly(transport: Pick): void { - if (transport.kind === "local-run-as") throw new Error(readonlyError); -} diff --git a/src/transport/types.ts b/src/transport/types.ts index 9e1501b..9fbbd91 100644 --- a/src/transport/types.ts +++ b/src/transport/types.ts @@ -1,9 +1,13 @@ +import type { RunAsIdentity } from "./run-as-identity.js"; export type TransportKind = "local" | "ssh" | "local-run-as"; export interface TargetTransport { kind: TransportKind; description: string; homeRoot?: string; + runAsIdentity?: RunAsIdentity; + prepareApply?(): Promise; + validateOpenClawConfig?(path: string): Promise; pathExists(path: string): Promise; mkdirExclusive(path: string): Promise; hashPath(path: string): Promise; @@ -12,7 +16,8 @@ export interface TargetTransport { listDir(path: string): Promise; writeFileAtomic(path: string, content: string): Promise; writeJsonAtomic(path: string, data: unknown): Promise; - atomicCopy(source: string, dest: string, kind: "file" | "dir"): Promise; + atomicCopy(source: string, dest: string, kind: "file" | "dir", expectedHash?: string): Promise; + copyWithin?(source: string, dest: string, kind: "file" | "dir"): Promise; rm(path: string): Promise; execFile?(command: string, args: string[], options?: { cwd?: string }): Promise; } diff --git a/test/run-as-apply.test.ts b/test/run-as-apply.test.ts new file mode 100644 index 0000000..911a70e --- /dev/null +++ b/test/run-as-apply.test.ts @@ -0,0 +1,158 @@ +import { chmod, mkdtemp, mkdir, readFile, readdir, rm, stat, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { GovernedMutation } from "../src/mutation/coordinator.js"; +import { createMutationReceipt, readMutationReceipt } from "../src/mutation/receipts.js"; +import { applyCombinedInstallPlan, recoverPendingApply } from "../src/install/apply.js"; +import { readInstallManifest } from "../src/install/manifest.js"; +import { applyJournalPath } from "../src/install/transaction.js"; +import type { InstallOperation, InstallPlan } from "../src/install/plan.js"; +import { createRunAsTransport, type RunAsRunner } from "../src/transport/run-as.js"; +import { assertSelfContainedOpenClawConfig, executeRequest, type RunAsRequest } from "../src/transport/run-as-worker.js"; +import { hashPath } from "../src/utils/fs.js"; + +const roots: string[] = []; +afterEach(async () => { for (const root of roots) await rm(root, { recursive: true, force: true }); roots.length = 0; }); +async function fixture() { + const root = await mkdtemp(join(tmpdir(), "aw-run-as-apply-")); roots.push(root); + const target = join(root, "target"), source = join(root, "private-source"); + await mkdir(target, { mode: 0o700 }); await mkdir(source, { mode: 0o700 }); + const requests: RunAsRequest[] = []; + let failPath: string | undefined; + const runner: RunAsRunner = async (_command, _args, input) => { + if (!input) return { stdout: `${process.getuid!()}\n` }; + const request = JSON.parse(input) as RunAsRequest; requests.push(request); + try { + if (request.operation === "copy" && request.path === failPath) throw new Error("Injected pre-copy failure"); + return { stdout: JSON.stringify({ version: 1, result: await executeRequest(target, process.getuid!(), process.getgid!(), request) }) }; + } catch (error) { return { stdout: JSON.stringify({ version: 1, error: { message: String(error), code: (error as NodeJS.ErrnoException).code } }) }; } + }; + // The injected runner executes the helper directly, never the Node argv. + // Python is a required system-owned binary; /usr/bin/node is optional. + const transport = createRunAsTransport({ user: "fixture", uid: process.getuid!(), gid: process.getgid!(), home: target, root: target }, runner, "/fixture/run-as-worker.mjs", "/usr/bin/python3"); + return { target, source, transport, requests, fail: (path: string) => { failPath = path; } }; +} +const stateKey = `openclaw.user.${"a".repeat(64)}`; +const scope = { installationType: "user", stateKey }; +async function operation(source: string, target: string, name: string, action: "create" | "update" = "create"): Promise { + return { action, artifactType: "skills", artifactName: name, kind: "file", sourcePath: source, + destPath: join(target, ".openclaw", name), relativeDestPath: `.openclaw/${name}`, + desiredHash: await hashPath(source), reason: "fixture", channel: "managed", owners: ["fixture"], workspaceOwner: "workspace-root:/fixture" }; +} +function plan(targetRoot: string, operations: InstallOperation[], baseRevision: string | null = null): InstallPlan { + return { adapter: "openclaw", targetRoot, ...scope, baseRevision, operations, hasBlockingChanges: false }; +} + +describe.skipIf(process.getuid?.() === 0)("confined run-as transactional apply", () => { + it("installs staged bytes with private modes and commits a service-side manifest", async () => { + const f = await fixture(); const source = join(f.source, "skill"); await writeFile(source, "private source bytes"); + const op = await operation(source, f.target, "skill"); + const manifest = await applyCombinedInstallPlan(plan(f.target, [op]), { transport: f.transport }); + expect(await readFile(op.destPath, "utf8")).toBe("private source bytes"); + expect((await stat(op.destPath)).mode & 0o777).toBe(0o600); + expect((await stat(join(f.target, ".openclaw"))).mode & 0o777).toBe(0o700); + expect((await readInstallManifest(f.target, "openclaw", f.transport, scope))?.revision).toBe(manifest.revision); + const copy = f.requests.find((request) => request.operation === "copy")!; + expect(JSON.stringify(copy)).not.toContain(f.source); + expect(copy).toMatchObject({ path: ".openclaw/skill", payload: { hash: op.desiredHash } }); + expect(f.requests.some((request) => request.operation === "writeFileAtomic" && request.path.startsWith(".agentwheel/"))).toBe(true); + await expect(stat(applyJournalPath(f.target, "openclaw", scope))).rejects.toMatchObject({ code: "ENOENT" }); + const noop = { ...op, action: "skip" as const, currentHash: op.desiredHash, manifestHash: op.desiredHash }; + const before = f.requests.filter((request) => request.operation === "copy").length; + await applyCombinedInstallPlan(plan(f.target, [noop], manifest.revision), { transport: f.transport }); + expect(f.requests.filter((request) => request.operation === "copy")).toHaveLength(before); + }, 60_000); + + it("recovers a failed update through service-owned backups with exact prior modes", async () => { + const f = await fixture(); const source = join(f.source, "first"); await writeFile(source, "old"); + const first = await operation(source, f.target, "first"); + const manifest = await applyCombinedInstallPlan(plan(f.target, [first]), { transport: f.transport }); + await chmod(first.destPath, 0o640); + await writeFile(source, "new"); const update = await operation(source, f.target, "first", "update"); + const secondSource = join(f.source, "second"); await writeFile(secondSource, "second"); + const second = await operation(secondSource, f.target, "second"); f.fail(second.relativeDestPath); + const previousStateRoot = process.env.AGENTWHEEL_MUTATION_STATE_ROOT; + process.env.AGENTWHEEL_MUTATION_STATE_ROOT = join(f.source, "mutation-state"); + const receipt = await createMutationReceipt({ + operationId: "backup-recovery", commandName: "agentwheel install", reason: "Recover service-owned backup", + noCommit: false, workspaceRoot: f.source, repositoryRoot: null, expectedHead: null, expectedManifestDigest: null, + revisionMode: "off", provider: null, preexistingPaths: [], paths: [], runtimeJournals: [], status: "prepared", + }); + const mutation = GovernedMutation.activateExisting(receipt, undefined, { path: "test", release: async () => {} }); + try { + await expect(applyCombinedInstallPlan(plan(f.target, [update, second], manifest.revision), { transport: f.transport })).rejects.toThrow(/Injected/); + } finally { + await mutation.fail(new Error("Injected update interruption")); + } + expect(await readFile(first.destPath, "utf8")).toBe("new"); + await rm(secondSource); + const callerReceipt = await readMutationReceipt(receipt.operationId); + const pinned = callerReceipt.runtimeJournals[0]!; + try { + expect(await recoverPendingApply(f.target, "openclaw", f.transport, scope, { + ...pinned, operationId: callerReceipt.operationId, reason: callerReceipt.reason, noCommit: callerReceipt.noCommit, + })).toBeUndefined(); + } finally { + if (previousStateRoot === undefined) delete process.env.AGENTWHEEL_MUTATION_STATE_ROOT; + else process.env.AGENTWHEEL_MUTATION_STATE_ROOT = previousStateRoot; + } + expect(await readFile(first.destPath, "utf8")).toBe("old"); + expect((await stat(first.destPath)).mode & 0o777).toBe(0o640); + expect((await readInstallManifest(f.target, "openclaw", f.transport, scope))?.revision).toBe(manifest.revision); + expect(f.requests.filter((request) => request.operation === "copyWithin").length).toBeGreaterThanOrEqual(2); + }, 60_000); + + it("rejects nested OpenClaw includes before invoking native code", () => { + for (const content of ['{"$include":"/etc/passwd"}', '{"nested":[{"$include":"relative.json"}]}', '{ // JSONC\n "$include": "outside.json", }']) { + expect(() => assertSelfContainedOpenClawConfig(content)).toThrow(/does not support \$include/); + } + for (const content of ["{$include:'/tmp/external.json'}", "{'$include':'/tmp/external.json'}", '{"safe":true, $include:"/tmp/external.json"}']) { + expect(() => assertSelfContainedOpenClawConfig(content)).toThrow(/valid JSON\/JSONC syntax/); + } + expect(() => assertSelfContainedOpenClawConfig('{ // literal reference\n "env": { "value": "${NAME}" }, }')).not.toThrow(); + }); + + it("rejects service-owned validators and arbitrary validation commands", async () => { + const f = await fixture(); + const packageRoot = join(f.target, "validator"); await mkdir(packageRoot); + const binary = join(packageRoot, "openclaw.mjs"); + await writeFile(binary, 'throw new Error("must not execute")'); + const config = join(f.target, "config.json"); await writeFile(config, "{}"); + await expect(executeRequest(f.target, process.getuid!(), process.getgid!(), { + version: 1, operation: "validateOpenClawConfig", path: "config.json", validatorRoot: packageRoot, + })).rejects.toThrow(/immutable/); + await expect(executeRequest(f.target, process.getuid!(), process.getgid!(), { version: 1, operation: "execFile", path: ".", command: binary })).rejects.toThrow(); + }); + + it("isolates Python startup from service-owned user site packages", async () => { + const f = await fixture(); + const { execFileSync } = await import("node:child_process"); + const version = execFileSync("/usr/bin/python3", ["-I", "-c", "import sys;print(f'{sys.version_info.major}.{sys.version_info.minor}')"], { encoding: "utf8" }).trim(); + const site = join(f.target, ".local", "lib", `python${version}`, "site-packages"); + await mkdir(site, { recursive: true }); const marker = join(f.target, "injected"); + await writeFile(join(site, "injection.pth"), `import pathlib; pathlib.Path(${JSON.stringify(marker)}).write_text("executed")\n`); + expect(await executeRequest(f.target, process.getuid!(), process.getgid!(), { version: 1, operation: "probe", path: "." })).toBe(true); + await expect(stat(marker)).rejects.toMatchObject({ code: "ENOENT" }); + }); + + it("stages artifact sources even when they reside beneath the target root", async () => { + const f = await fixture(); const source = join(f.target, "source"); await writeFile(source, "bytes", { mode: 0o644 }); + const dest = join(f.target, "dest"); + await expect(f.transport.atomicCopy(source, dest, "file", "0".repeat(64))).rejects.toThrow(/hash/); + expect(f.requests.some((request) => request.operation === "copyWithin")).toBe(false); + await f.transport.atomicCopy(source, dest, "file", await hashPath(source)); + expect((await stat(dest)).mode & 0o777).toBe(0o600); + expect(f.requests.at(-1)?.operation).toBe("copy"); + }); + + it("rejects executable plans and arbitrary commands before any target write", async () => { + const f = await fixture(); const source = join(f.source, "skill"); await writeFile(source, "bytes"); + const op = await operation(source, f.target, "skill"); + for (const patch of [{ action: "plugin" }, { semanticCommand: ["sh", "-c", "false"] }, { programmaticOperation: {} }]) { + await expect(applyCombinedInstallPlan(plan(f.target, [{ ...op, ...patch } as InstallOperation]), { transport: f.transport })).rejects.toThrow(/programmatic adapters and semantic/); + } + await expect(f.transport.execFile!("bash", ["-lc", "false"])).rejects.toThrow(/arbitrary commands/); + expect(f.requests).toEqual([]); expect(await readdir(f.target)).toEqual([]); + }); +}); diff --git a/test/run-as-cli.test.ts b/test/run-as-cli.test.ts index 295c038..d7415a8 100644 --- a/test/run-as-cli.test.ts +++ b/test/run-as-cli.test.ts @@ -77,13 +77,13 @@ describe("run-as CLI safeguards", () => { expect(JSON.parse(await readFile(report, "utf8"))).toMatchObject({ targetRoot: target }); }); - it.skipIf(process.getuid?.() === 0)("refuses profile apply before touching a local-run-as target", async () => { + it.skipIf(process.getuid?.() === 0)("refuses profile reload before touching a local-run-as target", async () => { const { workspace, target, home, config } = await fixture(); config.agents = { service: { adapter: "openclaw", root: target, transport: "local-run-as", runAs: userInfo().username, installationType: "user" } }; await writeConfig(workspace, config); - const result = await runCli(workspace, home, ["install", "--profile", "service"]); + const result = await runCli(workspace, home, ["install", "--profile", "service", "--reload-runtimes"]); expect(result.code).not.toBe(0); - expect(result.stderr).toMatch(/local-run-as is read-only in PR1/); + expect(result.stderr).toMatch(/local-run-as refuses.*reload/); expect(await readdir(target)).toEqual([]); }); }); diff --git a/test/run-as-fs.test.ts b/test/run-as-fs.test.ts new file mode 100644 index 0000000..c9c7844 --- /dev/null +++ b/test/run-as-fs.test.ts @@ -0,0 +1,266 @@ +import { spawn, spawnSync } from "node:child_process"; +import { chmod, link, mkdir, mkdtemp, readFile, rm, symlink, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { afterEach, describe, expect, it } from "vitest"; + +const helper = fileURLToPath(new URL("../src/transport/run-as-fs.py", import.meta.url)); +const uid = process.getuid?.() ?? 0; +const gid = process.getgid?.() ?? 0; +const roots: string[] = []; +const linuxUser = process.platform === "linux" && uid > 0 && gid > 0; +function request(root: string, operation: string, fields: Record = {}, identity = { uid, gid }) { + const result = spawnSync("/usr/bin/python3", ["-I", helper, "--root", root, "--uid", String(identity.uid), "--gid", String(identity.gid)], { + input: JSON.stringify({ version: 1, operation, ...fields }), encoding: "utf8", maxBuffer: 24 * 1024 * 1024, + }); + if (result.error) throw result.error; + expect(result.status, result.stderr).toBe(0); + return JSON.parse(result.stdout) as { version: number; result?: unknown; error?: { code: string; message: string } }; +} +async function fixture() { + const root = await mkdtemp(join(tmpdir(), "aw-fs-test-")); + roots.push(root); + return root; +} +const content = (text: string | Buffer) => Buffer.from(text).toString("base64"); +// Instrument real syscalls in a disposable subprocess; production has no test hooks. +function tracedRequest(root: string, operations: Record[], failAfter?: string) { + const source = ` +import errno,json,os,runpy,sys +module=runpy.run_path(sys.argv[1],run_name='filesystem_test') +root=sys.argv[2] +config=json.load(sys.stdin) +events=[] +def label(fd): + path=os.readlink('/proc/self/fd/'+str(fd)) + return '.' if path==root else path.removeprefix(root+'/') +def child(fd,name): + parent=label(fd) + return str(name) if parent=='.' else parent+'/'+str(name) +def wrap(name): + original=getattr(os,name) + def call(*args,**kwargs): + if name=='fsync': + event=[name,label(args[0])] + if events and events[-1][0]==config.get('failAfter'): + events.append(['fsync-failed',event[1]]) + raise OSError(errno.EIO,'Injected durability failure') + elif name=='fchmod': event=[name,label(args[0])] + elif name=='rename': event=[name,child(kwargs['src_dir_fd'],args[0]),child(kwargs['dst_dir_fd'],args[1]),label(kwargs['dst_dir_fd'])] + else: event=[name,child(kwargs['dir_fd'],args[0]),label(kwargs['dir_fd'])] + result=original(*args,**kwargs) + events.append(event) + return result + setattr(os,name,call) +for name in ('mkdir','unlink','rmdir','rename','fchmod','fsync'): wrap(name) +filesystem=module['Filesystem'](root,os.getuid(),os.getgid()) +try: + for operation in config['operations']: + filesystem.execute({'version':1,**operation}) + response={'result':True} +except OSError as error: + response={'error':{'code':errno.errorcode.get(error.errno),'message':str(error)}} +finally: + os.close(filesystem.root) +print(json.dumps({**response,'events':events})) +`; + const result = spawnSync("/usr/bin/python3", ["-I", "-c", source, helper, root], { + input: JSON.stringify({ operations, failAfter }), encoding: "utf8", maxBuffer: 24 * 1024 * 1024, + }); + if (result.error) throw result.error; + expect(result.status, result.stderr).toBe(0); + return JSON.parse(result.stdout) as { result?: boolean; error?: { code: string }; events: string[][] }; +} +afterEach(async () => { await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true }))); }); + +describe("confined filesystem identity", () => { + it("rejects root and mismatched pinned identities without writing", async () => { + const root = await fixture(); + expect(request(root, "probe", {}, { uid: 0, gid: 0 }).error?.code).toBe("EPERM"); + expect(request(root, "probe", {}, { uid: uid + 1, gid: gid || 1 }).error?.code).toBe("EPERM"); + }); +}); + +describe.skipIf(!linuxUser)("Linux openat2 confined filesystem", () => { + it("probes without writes and preserves structured missing/existing errors", async () => { + const root = await fixture(); + expect(request(root, "probe").result).toBe(true); + expect(request(root, "listDir", { path: "." }).result).toEqual([]); + expect(request(root, "pathExists", { path: "missing" }).result).toBe(false); + expect(request(root, "listDir", { path: "missing" }).result).toEqual([]); + expect(request(root, "readFile", { path: "missing" }).error?.code).toBe("ENOENT"); + expect(request(root, "mkdirExclusive", { path: "parents/lock" }).error).toBeUndefined(); + expect(request(root, "mkdirExclusive", { path: "parents/lock" }).error?.code).toBe("EEXIST"); + }); + + it("copies byte-exact trees and restores modes, ignored content and empty directories", async () => { + const root = await fixture(); + await mkdir(join(root, "tree", "empty"), { recursive: true }); + await mkdir(join(root, "tree", ".git")); + await writeFile(join(root, "tree", ".git", "config"), "ignored"); + await writeFile(join(root, "tree", "binary"), Buffer.from([0, 255, 128, 10]), { mode: 0o640 }); + await chmod(join(root, "tree"), 0o750); + const before = request(root, "snapshot", { path: "tree", includeIgnored: true }).result; + const filtered = request(root, "snapshot", { path: "tree", includeIgnored: false }).result as Array<{ path: string }>; + expect(filtered.map(entry => entry.path)).toEqual(["", "binary", "empty"]); + expect(request(root, "copyWithin", { source: "tree", path: "backup" }).error).toBeUndefined(); + expect(request(root, "snapshot", { path: "backup", includeIgnored: true }).result).toEqual(before); + expect(request(root, "writeFileAtomic", { path: "tree/binary", content: content("changed") }).error).toBeUndefined(); + expect((request(root, "snapshot", { path: "tree/binary", includeIgnored: true }).result as Array<{ mode: number }>)[0]?.mode).toBe(0o640); + expect(request(root, "copyWithin", { source: "backup", path: "tree" }).error).toBeUndefined(); + expect(request(root, "snapshot", { path: "tree", includeIgnored: true }).result).toEqual(before); + expect(request(root, "readFile", { path: "tree/binary" }).result).toBe(content(Buffer.from([0, 255, 128, 10]))); + expect(request(root, "rm", { path: "tree" }).error).toBeUndefined(); + expect(request(root, "rm", { path: "tree" }).error).toBeUndefined(); + }); + + it("validates a complete hierarchy, bytes and permission bits before writes", async () => { + const root = await fixture(); + await writeFile(join(root, "target"), "original"); + const invalid = [ + [{ path: "child", kind: "file", mode: 0o600, content: content("x") }], + [{ path: "", kind: "dir", mode: 0o755 }, { path: "missing/child", kind: "dir", mode: 0o700 }], + [{ path: "", kind: "file", mode: 0o4600, content: content("x") }], + [{ path: "", kind: "file", mode: 0o600, content: "!" }], + [{ path: "", kind: "dir", mode: 0o700 }, { path: "", kind: "dir", mode: 0o700 }], + ]; + for (const entries of invalid) { + expect(request(root, "installSnapshot", { path: "target", entries }).error?.code).toBe("EINVAL"); + expect(await readFile(join(root, "target"), "utf8")).toBe("original"); + } + expect(request(root, "installSnapshot", { path: "target", entries: [ + { path: "", kind: "dir", mode: 0o750 }, + { path: "empty", kind: "dir", mode: 0o710 }, + { path: "file", kind: "file", mode: 0o604, content: content("hello") }, + ] }).error).toBeUndefined(); + expect(request(root, "snapshot", { path: "target", includeIgnored: true }).result).toEqual([ + { path: "", kind: "dir", mode: 0o750 }, + { path: "empty", kind: "dir", mode: 0o710 }, + { path: "file", kind: "file", mode: 0o604, content: content("hello") }, + ]); + }); + + it("rejects traversal, root mutations, symlink roots and symlinks in every operation", async () => { + const root = await fixture(); + const outside = await fixture(); + await writeFile(join(outside, "sentinel"), "unchanged"); + await symlink(outside, join(root, "escape")); + for (const path of ["../sentinel", "/sentinel", "a/../sentinel", "x\\sentinel", "bad\0name", "a//b"]) { + expect(request(root, "pathExists", { path }).error?.code).toBe("EINVAL"); + } + for (const operation of ["rm", "mkdirExclusive", "writeFileAtomic", "installSnapshot", "copyWithin"]) { + expect(request(root, operation, { path: ".", source: "escape", content: content("x"), entries: [] }).error).toBeDefined(); + } + for (const operation of ["pathExists", "listDir", "readFile", "snapshot", "rm", "writeFileAtomic"]) { + expect(request(root, operation, { path: "escape/sentinel", includeIgnored: true, content: content("x") }).error).toBeDefined(); + } + expect(request(join(root, "escape"), "probe").error?.code).toBe("ELOOP"); + expect(await readFile(join(outside, "sentinel"), "utf8")).toBe("unchanged"); + }); + + it("rejects hardlinks and special files without blocking", async () => { + const root = await fixture(); + await writeFile(join(root, "file"), "original"); + await link(join(root, "file"), join(root, "hardlink")); + expect(spawnSync("/usr/bin/mkfifo", [join(root, "fifo")]).status).toBe(0); + for (const path of ["file", "hardlink", "fifo"]) { + for (const operation of ["pathExists", "readFile", "snapshot", "rm", "writeFileAtomic"]) { + expect(request(root, operation, { path, content: content("x"), includeIgnored: true }).error?.code).toBe("EINVAL"); + } + } + }); + + it("bounds decoded file bytes, aggregate bytes, entry counts and depth", async () => { + const root = await fixture(); + const huge = content(Buffer.alloc(8 * 1024 * 1024 + 1)); + expect(request(root, "writeFileAtomic", { path: "huge", content: huge }).error?.code).toBe("EINVAL"); + expect(request(root, "pathExists", { path: "huge" }).result).toBe(false); + expect(request(root, "mkdirExclusive", { path: Array(65).fill("d").join("/") }).error?.code).toBe("EINVAL"); + const entries = [{ path: "", kind: "dir", mode: 0o700 }, ...Array.from({ length: 4096 }, (_, i) => ({ path: `f${i}`, kind: "dir", mode: 0o700 }))]; + expect(request(root, "installSnapshot", { path: "huge", entries }).error?.code).toBe("EINVAL"); + const data = content(Buffer.alloc(4 * 1024 * 1024 + 1)); + expect(request(root, "installSnapshot", { path: "huge", entries: [ + { path: "", kind: "dir", mode: 0o700 }, + { path: "a", kind: "file", mode: 0o600, content: data }, + { path: "b", kind: "file", mode: 0o600, content: data }, + ] }).error?.code).toBe("EINVAL"); + }); + + it("durably orders new ancestors, empty backup directories, chmod, rename and recursive removal", async () => { + const root = await fixture(); + const traced = tracedRequest(root, [ + { operation: "mkdirExclusive", path: "ancestors/nested/empty" }, + { operation: "writeFileAtomic", path: "ancestors/nested/file", content: content("durable") }, + { operation: "copyWithin", source: "ancestors", path: "backup" }, + { operation: "rm", path: "backup" }, + { operation: "rm", path: "ancestors" }, + ]); + expect(traced.error).toBeUndefined(); + expect(traced.result).toBe(true); + const { events } = traced; + expect(events.some(event => event[0] === "mkdir" && event[1]?.endsWith("/empty"))).toBe(true); + expect(events.some(event => event[0] === "fchmod" && event[1]?.endsWith("/empty"))).toBe(true); + expect(events.some(event => event[0] === "rename")).toBe(true); + expect(events.some(event => event[0] === "unlink")).toBe(true); + expect(events.some(event => event[0] === "rmdir")).toBe(true); + for (let i = 0; i < events.length; i++) { + const event = events[i]!; + if (event[0] === "mkdir") { + // The new directory inode and its parent's entry must both be durable + // before the next mutation, including every newly created ancestor. + expect(events[i + 1]).toEqual(["fsync", event[1]]); + expect(events[i + 2]).toEqual(["fsync", event[2]]); + } else if (event[0] === "fchmod") { + expect(events[i + 1]).toEqual(["fsync", event[1]]); + } else if (event[0] === "rename") { + expect(events[i + 1]).toEqual(["fsync", event[3]]); + } else if (event[0] === "unlink" || event[0] === "rmdir") { + expect(events[i + 1]).toEqual(["fsync", event[2]]); + } + } + expect(request(root, "listDir", { path: "." }).result).toEqual([]); + }); + + it("never acknowledges a mutation when directory durability fails", async () => { + const cases: Array<{ failAfter: string; prepare?: (root: string) => Promise; operation: Record }> = [ + { failAfter: "mkdir", operation: { operation: "mkdirExclusive", path: "ancestor/child" } }, + { failAfter: "fsync", operation: { operation: "mkdirExclusive", path: "ancestor/child" } }, + { failAfter: "fchmod", operation: { operation: "installSnapshot", path: "empty-backup", entries: [{ path: "", kind: "dir", mode: 0o750 }] } }, + { failAfter: "rename", operation: { operation: "writeFileAtomic", path: "file", content: content("written") } }, + { failAfter: "unlink", prepare: root => writeFile(join(root, "file"), "remove"), operation: { operation: "rm", path: "file" } }, + { failAfter: "rmdir", prepare: root => mkdir(join(root, "empty")), operation: { operation: "rm", path: "empty" } }, + ]; + for (const testCase of cases) { + const root = await fixture(); + await testCase.prepare?.(root); + const traced = tracedRequest(root, [testCase.operation], testCase.failAfter); + expect(traced.result).toBeUndefined(); + expect(traced.error?.code).toBe("EIO"); + expect(traced.events.some(event => event[0] === "fsync-failed")).toBe(true); + } + }); + + it("never follows an adversarial concurrently replaced directory symlink", async () => { + const root = await fixture(); + const outside = await fixture(); + await writeFile(join(outside, "sentinel"), "unchanged"); + await mkdir(join(root, "switch")); + // The attacker alternates a real directory and a link while helpers write below it. + const attacker = spawn("/usr/bin/python3", ["-I", "-c", [ + "import os,sys,time", "root,outside=sys.argv[1:]", "p=root+'/switch'", "park=root+'/park'", + "end=time.monotonic()+5", "while time.monotonic()(resolve => attacker.once("close", () => resolve())); + } + expect(await readFile(join(outside, "sentinel"), "utf8")).toBe("unchanged"); + }, 15_000); +}); diff --git a/test/run-as-recovery.test.ts b/test/run-as-recovery.test.ts new file mode 100644 index 0000000..c2905a9 --- /dev/null +++ b/test/run-as-recovery.test.ts @@ -0,0 +1,228 @@ +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { recoverPendingApply } from "../src/install/apply.js"; +import { + acquireApplyLock, applyBackupDir, applyJournalLinkDigest, applyJournalPath, + assertApplyJournalCompletedOperations, assertGovernedRuntimeTransportSupported, readLinkedApplyJournal, recordBackup, + type ApplyJournal, +} from "../src/install/transaction.js"; +import { beginMutationPathDeclarations, endMutationPathDeclarations } from "../src/mutation/declarations.js"; +import { GovernedMutation, recoverMutationRuntime, reserveActiveRuntimeJournal } from "../src/mutation/coordinator.js"; +import { createMutationReceipt, runtimeJournalSchema } from "../src/mutation/receipts.js"; +import { localTransport, type TargetTransport } from "../src/transport/index.js"; +import type { RunAsIdentity } from "../src/transport/run-as-identity.js"; + +const seams = vi.hoisted(() => ({ resolve: vi.fn(), create: vi.fn() })); +vi.mock("../src/transport/run-as-identity.js", async (original) => ({ + ...await original(), + resolveRunAsIdentity: seams.resolve, +})); +vi.mock("../src/transport/index.js", async (original) => ({ + ...await original(), + createRunAsTransport: seams.create, +})); + +const identity: RunAsIdentity = { user: "service", uid: 1234, gid: 1234, home: "/srv/service", root: "/srv/service" }; +const description = "local-run-as service (1234:1234) /srv/service"; +const originalStateRoot = process.env.AGENTWHEEL_MUTATION_STATE_ROOT; +const roots: string[] = []; +afterEach(async () => { + vi.clearAllMocks(); + if (originalStateRoot === undefined) delete process.env.AGENTWHEEL_MUTATION_STATE_ROOT; + else process.env.AGENTWHEEL_MUTATION_STATE_ROOT = originalStateRoot; + await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true }))); +}); + +function fakeTransport(): TargetTransport { + return { + ...localTransport, kind: "local-run-as", description, + runAsIdentity: identity, prepareApply: vi.fn(async () => {}), + pathExists: vi.fn(async () => true), rm: vi.fn(async () => {}), + copyWithin: vi.fn(async () => {}), atomicCopy: vi.fn(async () => {}), mkdirExclusive: vi.fn(async () => {}), + writeJsonAtomic: vi.fn(async () => {}), + }; +} +function journal(): ApplyJournal { + return { + version: 2, mutation: { operationId: "run-as-recovery", reason: "Recover interrupted install", noCommit: false }, + adapter: "codex", targetRoot: identity.root, baseRevision: null, + createdAt: "2026-01-01T00:00:00.000Z", updatedAt: "2026-01-01T00:00:00.000Z", + operations: [], completed: [], manifest: {} as ApplyJournal["manifest"], + }; +} +function link() { + return { path: applyJournalPath(identity.root, "codex"), status: "pending" as const, + transport: "local-run-as" as const, transportDescription: description, + journalDigest: "a".repeat(64), runAsIdentity: identity }; +} + +describe("durable local run-as recovery", () => { + it("retains the established local v2 digest across the run-as upgrade", () => { + // Golden digest from the preceding release's local v2 contract. + expect(applyJournalLinkDigest(journal(), { kind: "local", description: "local" })) + .toBe("9af0bbb66aff10ebbb56bf4b51b7c9af5da44fb762c00bd8789f89a00f0b7b49"); + }); + it("requires exact structured identity only for run-as receipts", () => { + expect(runtimeJournalSchema.parse(link()).runAsIdentity).toEqual(identity); + expect(runtimeJournalSchema.safeParse({ ...link(), runAsIdentity: undefined }).success).toBe(false); + expect(runtimeJournalSchema.safeParse({ ...link(), transport: "local" }).success).toBe(false); + expect(runtimeJournalSchema.safeParse({ ...link(), runAsIdentity: { ...identity, extra: true } }).success).toBe(false); + for (const patch of [{ user: "root" }, { uid: 0 }, { gid: -1 }, { root: "/" }, { root: "/srv/../service" }, { home: "relative" }]) { + expect(runtimeJournalSchema.safeParse({ ...link(), runAsIdentity: { ...identity, ...patch } }).success).toBe(false); + } + }); + + it("prepares the service transport before any lock write and fails closed without its capabilities", async () => { + const transport = fakeTransport(); + const calls: string[] = []; + transport.prepareApply = async () => { calls.push("prepare"); }; + transport.mkdirExclusive = async () => { calls.push("mkdir"); }; + await acquireApplyLock(identity.root, "codex", transport); + expect(calls).toEqual(["prepare", "mkdir"]); + expect(() => assertGovernedRuntimeTransportSupported({ ...transport, runAsIdentity: undefined })).toThrow(/pinned identity/); + expect(() => assertGovernedRuntimeTransportSupported({ ...transport, prepareApply: undefined })).toThrow(/preparation/); + }); + + it("backs up inaccessible target files entirely through the service transport", async () => { + const transport = fakeTransport(); + const destPath = join(identity.root, "skills", "example.md"); + const operation = { action: "update", kind: "file", destPath } as ApplyJournal["operations"][number]; + const backup = await recordBackup(operation, 3, identity.root, "codex", transport); + const backupPath = join(applyBackupDir(identity.root, "codex"), "3"); + expect(transport.rm).toHaveBeenCalledWith(backupPath); + expect(transport.copyWithin).toHaveBeenCalledWith(destPath, backupPath, "file"); + expect(transport.atomicCopy).not.toHaveBeenCalled(); + expect(backup.backupPath).toBe(backupPath); + }); + + it("reads linked journals through transport and enforces descriptor, digest and path", async () => { + const transport = fakeTransport(); + const value = journal(); + const digest = applyJournalLinkDigest(value, transport); + expect(digest).toBe(applyJournalLinkDigest(value, { kind: transport.kind, description })); + value.mutation!.transport = { kind: transport.kind, description }; + value.mutation!.journalDigest = digest; + transport.readFile = vi.fn(async () => JSON.stringify(value)); + const path = applyJournalPath(value.targetRoot, value.adapter); + await expect(readLinkedApplyJournal(path, "run-as-recovery", digest, description, transport)).resolves.toMatchObject({ targetRoot: identity.root }); + expect(transport.readFile).toHaveBeenCalledWith(path); + await expect(readLinkedApplyJournal(path, "run-as-recovery", digest, "wrong", transport)).rejects.toThrow(/transport/); + await expect(readLinkedApplyJournal(path, "run-as-recovery", "b".repeat(64), description, transport)).rejects.toThrow(/digest/); + await expect(readLinkedApplyJournal(`${path}.wrong`, "run-as-recovery", digest, description, transport)).rejects.toThrow(/path mismatch/); + value.mutation!.transport.kind = "local"; + await expect(readLinkedApplyJournal(path, "run-as-recovery", digest, description, transport)).rejects.toThrow(/transport metadata/); + }); + + it("pins every caller-side write payload and the ordinary manifest", () => { + const value = journal(); + const transport = fakeTransport(); + const digest = applyJournalLinkDigest(value, transport); + for (const patch of [ + { workspaceConfig: { untrusted: true } }, { graphLock: { untrusted: true } }, + { manifest: { untrusted: true } }, { sourceLock: { untrusted: true } }, + ]) { + expect(applyJournalLinkDigest({ ...value, ...patch } as ApplyJournal, transport)).not.toBe(digest); + } + }); + + it("rejects forged completion destinations, indices and backup paths", () => { + const value = journal(); + const destPath = join(identity.root, "example.md"); + value.operations = [{ action: "update", kind: "file", destPath } as ApplyJournal["operations"][number]]; + value.mutation!.transport = { kind: "local-run-as", description }; + const completion = { index: 0, kind: "file" as const, destPath, hadExisting: true, + backupPath: join(applyBackupDir(identity.root, "codex"), "0") }; + for (const patch of [{ index: 1 }, { destPath: "/outside" }, { backupPath: "/outside" }, { hadExisting: false }]) { + value.completed = [{ ...completion, ...patch }]; + expect(() => assertApplyJournalCompletedOperations(value)).toThrow(/completion/); + } + value.completed = [completion, completion]; + expect(() => assertApplyJournalCompletedOperations(value)).toThrow(/duplicate/); + }); + + it("refuses executable recovery before preparing or creating the target lock", async () => { + const transport = fakeTransport(); + const value = journal(); + value.operations = [{ action: "program" } as ApplyJournal["operations"][number]]; + value.mutation!.transport = { kind: "local-run-as", description }; + const digest = applyJournalLinkDigest(value, transport); + value.mutation!.journalDigest = digest; + transport.readFile = vi.fn(async () => JSON.stringify(value)); + await expect(recoverPendingApply(identity.root, "codex", transport, {}, { + ...link(), operationId: "run-as-recovery", journalDigest: digest, + reason: value.mutation!.reason, noCommit: false, + })).rejects.toThrow(/programmatic/); + expect(transport.prepareApply).not.toHaveBeenCalled(); + expect(transport.mkdirExclusive).not.toHaveBeenCalled(); + }); + + it("revalidates the consumed journal under the runtime lock after a service-side swap", async () => { + const transport = fakeTransport(); + const value = journal(); + value.mutation!.transport = { kind: "local-run-as", description }; + const digest = applyJournalLinkDigest(value, transport); + value.mutation!.journalDigest = digest; + const swapped = { ...value, workspaceConfigPath: "/outside/config.json", workspaceConfig: { unsafe: true } }; + let locked = false; + transport.mkdirExclusive = vi.fn(async () => { locked = true; }); + transport.listDir = vi.fn(async () => []); + transport.readFile = vi.fn(async () => JSON.stringify(locked ? swapped : value)); + await expect(recoverPendingApply(identity.root, "codex", transport, {}, { + ...link(), operationId: "run-as-recovery", journalDigest: digest, + reason: value.mutation!.reason, noCommit: false, + })).rejects.toThrow(/digest/); + expect(transport.copyWithin).not.toHaveBeenCalled(); + expect(transport.atomicCopy).not.toHaveBeenCalled(); + expect(transport.writeJsonAtomic).toHaveBeenCalledTimes(1); // lock owner only + }); + + it("rejects receipt link collisions when any pinned account field changes", async () => { + const workspaceRoot = await mkdtemp(join(tmpdir(), "agentwheel-runas-link-")); + roots.push(workspaceRoot); + process.env.AGENTWHEEL_MUTATION_STATE_ROOT = join(workspaceRoot, "state"); + const receipt = await createMutationReceipt({ + operationId: "run-as-link", commandName: "agentwheel install", reason: "Reject identity collision", + noCommit: false, workspaceRoot, repositoryRoot: null, expectedHead: null, expectedManifestDigest: null, + revisionMode: "off", provider: null, preexistingPaths: [], paths: [], status: "partial", + runtimeJournals: [link()], + }); + const mutation = GovernedMutation.activateExisting(receipt, undefined, { path: "test", release: async () => {} }); + try { + for (const patch of [{ user: "another" }, { uid: 2345 }, { gid: 2345 }, { home: "/srv/another" }, { root: "/srv/another" }]) { + await expect(reserveActiveRuntimeJournal({ ...link(), runAsIdentity: { ...identity, ...patch } })).rejects.toThrow(/link collision/); + } + } finally { + await mutation.fail(new Error("test completed")); + } + }); + + it.each([false, true])("revalidates pinned identity before any target access (changed=%s)", async (changed) => { + const workspaceRoot = await mkdtemp(join(tmpdir(), "agentwheel-runas-recovery-")); + roots.push(workspaceRoot); + process.env.AGENTWHEEL_MUTATION_STATE_ROOT = join(workspaceRoot, "state"); + const transport = fakeTransport(); + transport.pathExists = vi.fn(async () => false); + seams.resolve.mockReturnValue({ ...identity, uid: changed ? 5678 : identity.uid }); + seams.create.mockReturnValue(transport); + await createMutationReceipt({ + operationId: "run-as-recovery", commandName: "agentwheel install", reason: "Recover interrupted install", + noCommit: false, workspaceRoot, repositoryRoot: null, expectedHead: null, expectedManifestDigest: null, + revisionMode: "off", provider: null, preexistingPaths: [], paths: [], status: "partial", + runtimeJournals: [{ ...link(), status: "resolving" }], + }); + beginMutationPathDeclarations(workspaceRoot, "run-as-recovery"); + endMutationPathDeclarations(); + if (changed) { + await expect(recoverMutationRuntime("run-as-recovery")).rejects.toThrow(/identity changed/); + expect(seams.create).not.toHaveBeenCalled(); + expect(transport.pathExists).not.toHaveBeenCalled(); + } else { + const receipt = await recoverMutationRuntime("run-as-recovery"); + expect(receipt.runtimeJournals[0]?.status).toBe("resolved"); + expect(transport.pathExists).toHaveBeenCalledWith(link().path); + } + expect(seams.resolve).toHaveBeenCalledWith(identity.user, identity.root); + }); +}); diff --git a/test/run-as-trust.test.ts b/test/run-as-trust.test.ts new file mode 100644 index 0000000..8f9a99a --- /dev/null +++ b/test/run-as-trust.test.ts @@ -0,0 +1,46 @@ +import { chmod, mkdtemp, mkdir, rm, symlink, writeFile } from "node:fs/promises"; +import { execFileSync } from "node:child_process"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; +import { trustedOpenClawEntry, trustedRunAsCode } from "../src/transport/run-as-trust.js"; +const roots: string[] = []; +const uid = process.getuid!() === 65534 ? 65533 : 65534; +afterEach(async () => { for (const root of roots) await rm(root, { recursive: true, force: true }); roots.length = 0; }); +async function fixture() { + const root = await mkdtemp(join(tmpdir(), "aw-code-trust-")); roots.push(root); + const entry = join(root, "openclaw.mjs"); await writeFile(entry, "export {};", { mode: 0o644 }); + return { root, entry }; +} +describe("run-as executable trust", () => { + it("allows protected entries under sticky /tmp and rejects replaceable ancestors", async () => { + const { root, entry } = await fixture(); + expect(trustedRunAsCode(entry, uid, [], "fixture")).toBe(entry); + await chmod(root, 0o777); + expect(() => trustedRunAsCode(entry, uid, [], "fixture")).toThrow(/ancestor/); + await chmod(root, 0o1777); + expect(trustedRunAsCode(entry, uid, [], "fixture")).toBe(entry); + expect(() => trustedRunAsCode(entry, process.getuid!(), [], "fixture")).toThrow(/immutable/); + }); + it("rejects service supplementary-group writes to ancestor entries", async () => { + const { root, entry } = await fixture(); await chmod(root, 0o770); + expect(() => trustedRunAsCode(entry, uid, [process.getgid!()], "fixture")).toThrow(/ancestor/); + expect(() => trustedRunAsCode(entry, uid, [], "fixture")).toThrow(/ancestor/); + }); + it("rejects a named-user POSIX ACL grant despite an unrelated owning group", async () => { + const { root, entry } = await fixture(); + // Linux ACL access xattr: owner rwx, named service rwx, group r-x, + // mask rwx, other --- (the group-class mode reflects the ACL mask). + execFileSync("/usr/bin/python3", ["-I", "-c", "import os,struct,sys; entries=[(1,7,4294967295),(2,7,int(sys.argv[2])),(4,5,4294967295),(16,7,4294967295),(32,0,4294967295)]; os.setxattr(sys.argv[1], 'system.posix_acl_access', struct.pack(' trustedRunAsCode(entry, uid, [], "fixture")).toThrow(/ancestor/); + }); + it("checks every validator dependency and rejects symlinks outside the pinned package", async () => { + const { root, entry } = await fixture(); + expect(trustedOpenClawEntry(root, uid, [])).toBe(entry); + const dependencies = join(root, "node_modules"); await mkdir(dependencies, { mode: 0o755 }); + const dep = join(dependencies, "dep.mjs"); await writeFile(dep, "export {};", { mode: 0o666 }); await chmod(dep, 0o666); + expect(() => trustedOpenClawEntry(root, uid, [])).toThrow(/service-writable/); + await chmod(dep, 0o644); await symlink("/etc/passwd", join(dependencies, "escape")); + expect(() => trustedOpenClawEntry(root, uid, [])).toThrow(/escapes/); + }); +}); diff --git a/test/run-as.test.ts b/test/run-as.test.ts index 02bbc31..ff4af35 100644 --- a/test/run-as.test.ts +++ b/test/run-as.test.ts @@ -79,7 +79,7 @@ describe("local run-as read transport", () => { const calls: Array<{ command: string; args: string[]; input?: string }> = []; const runner: RunAsRunner = async (command, args, input) => { calls.push({ command, args, input }); - return { stdout: input ? JSON.stringify({ version: 1, result: true }) : `${service.uid}\n` }; + return { stdout: input ? JSON.stringify({ version: 1, result: JSON.parse(input).operation === "readFile" ? "content" : true }) : `${service.uid}\n` }; }; const transport = createRunAsTransport(service, runner, "/fixed/run-as-worker.mjs", node); expect(await transport.pathExists(join(root, "managed"))).toBe(true); @@ -94,7 +94,7 @@ describe("local run-as read transport", () => { expect(calls.filter(({ args }) => args.at(-2) === "/usr/bin/id" && args.at(-1) === "-u")).toHaveLength(1); }); - it("fails closed on sudo denial, a wrong UID, escaping paths and all writes", async () => { + it("fails closed on sudo denial, a wrong UID, escaping paths and mutations before a valid preflight", async () => { const root = await fixture(); const node = await trustedNodeFixture(root); const service = distinctServiceIdentity(root); @@ -103,9 +103,9 @@ describe("local run-as read transport", () => { const wrong = createRunAsTransport(service, async () => ({ stdout: "0\n" }), "/fixed/worker", node); await expect(wrong.pathExists(join(root, "x"))).rejects.toThrow(new RegExp(`expected ${service.uid}`)); await expect(wrong.pathExists("/etc/passwd")).rejects.toThrow(/escapes/); - await expect(wrong.writeFileAtomic(join(root, "x"), "data")).rejects.toThrow(/read-only in PR1/); - const plan = { hasBlockingChanges: false, installationType: "user", stateKey: "openclaw.user.test" } as InstallPlan; - await expect(applyCombinedInstallPlan(plan, { transport: wrong })).rejects.toThrow(/read-only in PR1/); + await expect(wrong.writeFileAtomic(join(root, "x"), "data")).rejects.toThrow(new RegExp(`expected ${service.uid}`)); + const plan = { hasBlockingChanges: false, adapter: "openclaw", targetRoot: root, operations: [], baseRevision: null, installationType: "user", stateKey: "openclaw.user.test" } as InstallPlan; + await expect(applyCombinedInstallPlan(plan, { transport: wrong })).rejects.toThrow(new RegExp(`expected ${service.uid}`)); }); it("rejects a service-owned or writable Node executable before sudo", async () => { @@ -114,13 +114,36 @@ describe("local run-as read transport", () => { await writeFile(executable, "fixture"); const currentUid = process.getuid!(); if (currentUid > 0) { - expect(() => createRunAsTransport({ ...identity(root), uid: currentUid }, async () => ({ stdout: "" }), "/fixed/worker", executable)).toThrow(/Node executable is writable/); + expect(() => createRunAsTransport({ ...identity(root), uid: currentUid }, async () => ({ stdout: "" }), "/fixed/worker", executable)).toThrow(/Node executable must be an immutable/); } await chmod(executable, 0o666); - expect(() => createRunAsTransport(identity(root), async () => ({ stdout: "" }), "/fixed/worker", executable)).toThrow(/Node executable is writable/); + expect(() => createRunAsTransport(identity(root), async () => ({ stdout: "" }), "/fixed/worker", executable)).toThrow(/Node executable must be an immutable/); expect(() => createRunAsTransport(identity(root), async () => ({ stdout: "" }), "/fixed/worker", "node")).toThrow(/must be absolute/); }); + it("shares validation snapshots only with the named UID despite restrictive caller umask", async () => { + const root = await fixture(), node = await trustedNodeFixture(root), service = distinctServiceIdentity(root); + let snapshot: string | undefined; + const runner: RunAsRunner = async (_command, _args, input) => { + if (!input) return { stdout: String(service.uid) }; + const request = JSON.parse(input); + if (request.operation === "validateOpenClawConfig") { + snapshot = request.validationSnapshot; + expect(statSync(snapshot!).mode & 0o777).toBe(0o440); + const { execFileSync } = await import("node:child_process"); + const acl = JSON.parse(execFileSync("/usr/bin/python3", ["-I", "-c", "import os,struct,json,sys; a=os.getxattr(sys.argv[1],'system.posix_acl_access'); print(json.dumps([struct.unpack(' statSync(snapshot!)).toThrow(/ENOENT/); + }); + it("times out stalled child processes with and without request input", async () => { const args = ["-e", "setInterval(() => {}, 1000)"]; for (const input of [undefined, "request"]) { @@ -141,7 +164,7 @@ describe("local run-as read transport", () => { await chmod(file, 0o640); expect(await executeRead(root, request)).not.toBe(changed); await symlink("/etc/passwd", join(root, "__pycache__", "escape")); - await expect(executeRead(root, request)).rejects.toThrow(/Unsupported ownership runtime path/); + await expect(executeRead(root, request)).rejects.toThrow(/symbolic links|symlink/); }); it("rejects directory reads through the worker readFile operation", async () => { @@ -160,6 +183,6 @@ describe("local run-as read transport", () => { await expect(containedReadPath(root, "/etc/passwd")).rejects.toThrow(/contained/); await expect(containedReadPath(root, "x\\y")).rejects.toThrow(/contained/); await symlink("/etc/passwd", join(root, "escape")); - await expect(executeRead(root, { version: 1, operation: "readFile", path: "escape" })).rejects.toThrow(/symlink/); + await expect(executeRead(root, { version: 1, operation: "readFile", path: "escape" })).rejects.toThrow(/symbolic links|symlink/); }); }); diff --git a/tsup.config.ts b/tsup.config.ts index 119429a..0443116 100644 --- a/tsup.config.ts +++ b/tsup.config.ts @@ -1,13 +1,16 @@ +import { chmod, copyFile } from "node:fs/promises"; import { defineConfig } from "tsup"; export default defineConfig({ entry: { index: "src/cli/index.ts", "run-as-worker": "src/transport/run-as-worker.ts" }, format: ["esm"], splitting: false, + noExternal: ["zod", "jsonc-parser"], dts: false, clean: true, sourcemap: false, target: "node18", + async onSuccess() { await copyFile("src/transport/run-as-fs.py", "dist/run-as-fs.py"); await chmod("dist/run-as-fs.py", 0o644); }, banner: { js: "#!/usr/bin/env node", },