Typed operations, engines, workspace, query & MCP (#689) by tony · Pull Request #690 · tmux-python/libtmux

tony · 2026-06-21T14:02:37Z

Summary

Adds the typed experimental stack under libtmux.experimental: operations,
engines, eager/lazy/async objects, plans, declarative workspaces, live pane
queries, and an optional MCP server.

The work is additive and remains outside libtmux's stable-versioning policy. It
implements the architecture from #688 and #689 without changing the existing
public ORM API.

What ships

Typed operations and results

58 frozen, keyword-only operation classes across Server, Session, Window,
Pane, and Client scopes.
Constructor signatures expose only target parameters accepted by the
underlying tmux command.
Typed result classes preserve argv, status, output, created identifiers, and
incomplete composed-operation failures.
Lazy plans resolve typed forward references and support sequential,
semicolon-folded, and {marked} dispatch with identical result semantics.
Serialization, MCP schemas, generated API signatures, and workspace rebasing
share the same constructor-field contract.

Engines and command boundaries

Transport	Sync	Async
Subprocess	`SubprocessEngine`	`AsyncSubprocessEngine`
In-memory testing	`MockEngine`	`AsyncMockEngine`
Control mode (`tmux -C`)	`ControlModeEngine`	`AsyncControlModeEngine`
Native imsg	`ImsgEngine`	—

Every engine returns the same CommandResult shape. Direct argv, control-mode
lines, and imsg frames preserve literal semicolons, newlines, carriage returns,
and leading-dash positional values. Typed separator provenance prevents user
data from becoming a second tmux command.

Objects, queries, and workspaces

Eager, lazy, and async Server/Session/Window/Pane/Client objects use the same
operation spine.
panes() provides lazy filtering, ordering, limiting, mapping, and folded
per-pane commands against either a live engine or snapshots.
Declarative Workspace values compile into inspectable plans, support YAML,
environment and option propagation, host-step fold boundaries, workspace
sets, and tmux 3.7 floating panes.
Create navigation fails with a typed diagnostic when a successful engine
result omits the identifier required to construct the next object.

MCP server

The optional libtmux[mcp] extra provides curated tools, one tool per
operation, plan tools, workspace tools, resources, prompts, and output
monitoring. Safety is evaluated from the concrete payload across direct,
per-operation, plan, workspace, and event paths. Executable shell,
configuration, environment, option, and active-format payloads require the
destructive tier; target-aware tools also protect the caller's own tmux
hierarchy.

Documentation and developer experience

A dedicated operations reference is grouped by Server, Session, Window,
Pane, and Client, with one generated API page per operation.
Querying has separate hierarchy, QueryList, tmux-format, Neo, and snapshot
guides with API references, examples, happy paths, and failure paths.
Every engine has a reference page and relevant tutorial.
User examples run against isolated real tmux servers; MockEngine is
reserved for explicitly offline engine/testing documentation.
Generated constructor prose comes from checked NumPy Attributes sections,
and operation badges/cross-links use the normal gp-sphinx API presentation.
Documentation examples execute in pytest/MyST sandboxes and the Sphinx build
verifies the operation registry cannot drift from its pages.

Verification

uv run ruff check . --fix --show-fixes
uv run ruff format .
uv run ty check
uv run mypy
uv run py.test --reruns 0 -vvv: 3,104 passed, 1 skipped
just build-docs: 141 pages built successfully
Live adversarial probes passed across subprocess, async subprocess,
control-mode, async control-mode, and imsg transports.

Refs #688, #689.

codecov · 2026-06-21T14:03:52Z

Codecov Report

❌ Patch coverage is 75.30414% with 812 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.76%. Comparing base (db2f386) to head (54e6892).

Files with missing lines	Patch %	Lines
scripts/mcp_swap.py	73.40%	144 Missing and 44 partials ⚠️
src/libtmux/experimental/engines/imsg/base.py	54.96%	149 Missing and 28 partials ⚠️
src/libtmux/experimental/engines/control_mode.py	68.55%	94 Missing and 34 partials ⚠️
...libtmux/experimental/engines/async_control_mode.py	75.39%	84 Missing and 40 partials ⚠️
src/libtmux/experimental/engines/imsg/v8.py	76.47%	39 Missing and 17 partials ⚠️
src/libtmux/experimental/mcp/__init__.py	47.61%	28 Missing and 5 partials ⚠️
docs/_ext/tmuxop/render.py	84.86%	15 Missing and 8 partials ⚠️
docs/_ext/tmuxop/domain.py	88.18%	10 Missing and 5 partials ⚠️
src/libtmux/experimental/engines/base.py	88.49%	8 Missing and 5 partials ⚠️
src/libtmux/experimental/mcp/_policy.py	90.65%	5 Missing and 5 partials ⚠️
... and 9 more

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #690       +/-   ##
===========================================
+ Coverage   52.44%   79.76%   +27.32%     
===========================================
  Files          26      234      +208     
  Lines        3726    15902    +12176     
  Branches      747     1981     +1234     
===========================================
+ Hits         1954    12685    +10731     
- Misses       1468     2505     +1037     
- Partials      304      712      +408

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

why: Record the experimental operations/engines layer for the upcoming release so the unreleased section tracks what landed. what: - Add a "What's new" deliverable under the unreleased 0.59.x section for the experimental operations and engines layer (#690) - Defer the release lead paragraph until the version is cut

tony · 2026-06-21T18:54:52Z

Code review

Found 2 issues:

LazyPlan resolves a forward SlotRef only for target, never for src_target, so a dual-target op (JoinPane, SwapPane, MovePane, BreakPane, SwapWindow, MoveWindow, LinkWindow) whose src_target comes from an earlier plan.add(...) reaches render() with the slot unresolved and raises TypeError: cannot render an unresolved SlotRef. (bug: _resolve() substitutes operation.target but not operation.src_target, even though serialize.py already handles both)

libtmux/src/libtmux/experimental/ops/plan.py

Lines 81 to 97 in e115eaf

    
           def _resolve( 
        
               operation: Operation[t.Any], 
        
               bindings: dict[int, str], 
        
           ) -> Operation[t.Any]: 
        
               """Substitute a :class:`SlotRef` target with a captured concrete id.""" 
        
               target = operation.target 
        
               if not isinstance(target, SlotRef): 
        
                   return operation 
        
               try: 
        
                   concrete = bindings[target.slot] + target.suffix 
        
               except KeyError as error: 
        
                   msg = ( 
        
                       f"slot {target.slot} has no captured id yet; a plan step can only " 
        
                       f"target an earlier step that creates an object" 
        
                   ) 
        
                   raise OperationError(msg) from error 
        
               return dataclasses.replace(operation, target=_target_from_id(concrete))

SaveBuffer declares contradictory metadata: safety = "mutating" together with effects = Effects(read_only=True), where read_only is documented as "does not change tmux state". Its read peer ShowBuffer uses safety = "readonly", and a consumer filtering registry.select(lambda s: s.safety == "readonly") would omit save_buffer despite effects.read_only=True. (bug: inconsistent safety/effects declarations)

libtmux/src/libtmux/experimental/ops/_ops/save_buffer.py

Lines 38 to 41 in e115eaf

    
           result_cls = AckResult 
        
           safety = "mutating" 
        
           effects = Effects(read_only=True)

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

tony · 2026-06-21T20:06:28Z

Code review

Found 1 issue:

LazyPlan resolves forward SlotRefs on every dispatch path except the {marked} fold's decorates. _drive resolves the create op but builds decorates raw, so a chainable dual-target decorate (SwapPane/JoinPane/MovePane) whose src_target is a forward slot reaches render_marked unresolved and raises TypeError: cannot render an unresolved SlotRef. The single-op and chain paths both call _resolve; this one does not. (bug: decorates = [self._operations[i] for i in decorate_idx] skips _resolve, so src_target SlotRefs survive into render under MarkedPlanner)

libtmux/src/libtmux/experimental/ops/plan.py

Lines 214 to 219 in 2e0b112

    
           create_idx, *decorate_idx = step.indices 
        
           create = _resolve(self._operations[create_idx], bindings) 
        
           decorates = [self._operations[i] for i in decorate_idx] 
        
           merged: CommandResult = yield _Chain( 
        
               render_marked(create, decorates, version), 
        
           )

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

tony · 2026-06-22T01:30:30Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

tony · 2026-06-22T01:56:26Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

why: Record the experimental operations/engines layer for the upcoming release so the unreleased section tracks what landed. what: - Add a "What's new" deliverable under the unreleased 0.59.x section for the experimental operations and engines layer (#690) - Defer the release lead paragraph until the version is cut

tony · 2026-07-05T12:15:03Z

Code review

No issues found. Fresh review of the current HEAD (4393b02a) — the ops / engines / facade / workspace / query / MCP surface was checked for bugs and AGENTS.md compliance and is clean. (The prior review comments predate ~12 days of commits.)

🤖 Generated with Claude Code

why: Record the experimental operations/engines layer for the upcoming release so the unreleased section tracks what landed. what: - Add a "What's new" deliverable under the unreleased 0.59.x section for the experimental operations and engines layer (#690) - Defer the release lead paragraph until the version is cut

why: A create rendered with `-P -F` tells tmux exactly how many ids to print, but nothing checked that they arrived. A short or empty line left the missing ids None and logged nothing, so the loss only surfaced later and elsewhere -- an unresolvable plan reference, or an AttributeError wrapping a None id. The id-parse path had no logging at all. what: - Check the capture invariant in build_result: warn when a create did not complete, and error when a complete create captured fewer ids than its format requested (which is what catches the partial case). - Warn in status_for when tmux exits 0 but writes to stderr, since the result is downgraded to failed on stderr text alone. - Use the documented tmux_* extra keys; heavy stdout/stderr stay DEBUG.

why: The ops engines had almost no logging, so a command that returned nothing looked identical to one that worked. Control mode was worse: a dropped or misattributed reply block is silent, and correlation desync could only be inferred after the fact. what: - Log each subprocess dispatch at DEBUG (argv, exit code, stdout, stderr), matching what libtmux.common already provides. - Add BlockSequenceMonitor: tmux's guard echoes a strictly increasing command number, so a solicited block whose number does not advance is provable desync -- one integer compare per block. - Warn on the silent control-mode paths: an unparseable %begin guard, a surplus solicited block, a drained solicited reply, and an async block arriving with no pending command. - Heavy stdout/stderr stay DEBUG behind isEnabledFor guards.

why: tmux reports a lost server on the client's stderr, not in an %error block, so _read_stderr logged that text and dropped it. A caller saw only how libtmux noticed ("tmux -C closed stdout") and never what tmux said ("server exited unexpectedly"), leaving control-mode failures strictly less diagnosable than subprocess ones. what: - Keep a bounded tail of the tmux process's stderr as it is read. - Add _died() to build connection-death errors with that tail appended, and raise through it from the exit and closed-stdout sites. - Leave the message untouched when stderr had nothing to add.

why: Each cell killed its session between builds, dropping the server to zero sessions. Under tmux's exit-empty default the server then began exiting, and the next build's new-session could reach the still-bound socket mid-shutdown and fail with "server exited unexpectedly" -- an intermittent create failure whose rate rose with machine load. Control mode never hit it: its tmux -C phantom already pinned the server, which is why only subprocess cells failed. what: - Give every bench server a keepalive session, so a cell's teardown never drops it to zero. Setting exit-empty alone cannot work: an empty server exits before the option can land. - Offload the async cells' kill-session to a thread; a blocking subprocess issued from inside a running event loop measurably raised the failure rate.

why: _cleanup only knows this process's socket dir, so a run killed before its atexit hook leaves its scratch dir -- and any tmux still bound to it -- behind for good. Those survivors keep consuming CPU and descriptors, and machine load is exactly what makes the server-teardown race fire, so an unreaped leak feeds the failure it came from. what: - On import, remove any ltbench-* scratch dir with no live tmux bound to it; report how many were reclaimed. - Leave dirs with a live server alone -- they may belong to a concurrent run, and stealing another run's servers is worse than leaking a hung client until its server exits.

why: Driving the MCP server across agent CLIs surfaced footguns the swapper couldn't see -- the repo registered under a name other than the derived default, un-reverted swaps and orphaned backups piling up, a state entry whose backup had vanished, and API-key env vars silently overriding subscription auth. Isolated testing also needed a per-entry env (LIBTMUX_SOCKET) that use-local couldn't write. what: - Add read-only `doctor`: cross-CLI entry inventory, server-name mismatch hint, outstanding-swap and orphaned-backup health, missing backup detection, and auth-env override warnings - Add repeatable `use-local --env KEY=VALUE`, layered over preserved env - Add a naming hint on use-local when the repo is registered under another server name - Keep the new logic server-agnostic (no hardcoded slug); the derived default here is `libtmux-engine`, from the console-script entry - Tests for env injection, the naming hint, and doctor; CHANGES under Development

why: Unit tests prove the MCP server's internals but not that a real agent CLI can discover a tool, clear its approval gate, call it, and survive cancelling it mid-flight. Capture the end-to-end harness so it is reproducible across Claude / Codex / Cursor / Gemini / Grok / agy. what: - Add SKILL.md and references/cli-matrix.md under .agents/skills - Concretize the generalized template for libtmux: server slug `libtmux`, backend-isolation lever an isolated tmux socket (LIBTMUX_SOCKET -> `tmux -L <scratch>`) - Document the three fidelity layers, approval/cancellation traps, and the mcp_swap doctor/use-local --env wiring

why: Operation pages need semantic links and styled facts backed by the registry. what: - Add operation and catalog directives with lifecycle handling - Render gp-sphinx cards, badges, source links, and catalogs - Test serial, parallel, incremental, invalid, and inventory builds

why: Rerun-free documentation tests exposed waits that matched echoed input or captured before output reached the pane. what: - Synchronize controlled commands through tmux wait channels - Match exact runtime-produced lines for bounded polling - Keep MCP guidance aligned with event-backed waiting

why: Published examples need reproducible shell, home, and tmux state with reruns disabled. what: - Add per-example home and short tmux socket isolation - Make pytest the canonical documentation example runner - Test block isolation and author-facing failure locations

why: Give every experimental operation a stable, navigable API page with an executable example and explicit failure semantics. what: - Split experimental guidance into operations, engines, and plans - Add registry-backed Server, Session, Window, Pane, and Client pages - Test page coverage, navigation, rendering, and doctest execution

why: Activate gp-sphinx card styling and responsive layout rules for custom operation entries. what: - Wrap operation entries in the shared API card shell - Assert the rendered shell contract in the Sphinx integration test

why: Let users choose a query API by freshness, filtering boundary, and failure behavior instead of navigating one mixed topic. what: - Add hierarchy, QueryList, tmux-format, Neo, and snapshot guides - Test independent happy and sad examples plus styled API sections - Preserve traversal and filtering URLs with redirects

why: Keep the verified documentation extension and tests in the repository canonical format. what: - Apply ruff line wrapping to the operation domain and query tests

why: Operation cards flattened Python signatures into literal text, bypassing Sphinx Python-domain semantics and gp-sphinx API styling. what: - Render operation classes and constructors as native Python descriptions - Register public class and constructor inventory targets - Test RST and MyST rendering against semantic gp-sphinx markup

why: Wall-clock corrections can move backward and invalidate timeout measurements in retry_until and its tests. what: - Measure retry deadlines with the monotonic clock - Measure elapsed-time assertions with the same clock

why: Operation pages need real tmux proofs and API references that match the normal gp-sphinx presentation. what: - Replace mock operation examples with live typed outcomes - Add engine, tutorial, and result API references - Align operation cards, source links, parameters, and badges - Test documentation inventory, rendering, and tmux behavior

why: CI checks the documentation extension under a different package root and against Sphinx's installed domain annotations. what: - Use package-relative imports for both Sphinx and mypy discovery - Follow inherited Sphinx domain attribute typing - Inspect public constructor signatures without unsafe __init__ access

why: tmux releases disagree on whether list-clients includes a suspended client, making the documented output version-dependent. what: - Verify the real AckResult and empty command output - Preserve the portable session-survival assertion - Document client-list visibility as version-dependent

why: Deleting a control-owned session can race tmux's deferred global notifications and terminate supported tmux servers. what: - Attach control clients to safe existing sessions without updating the environment - Bootstrap empty or unsafe servers through tracked subprocess execution - Serialize lifecycle transitions without blocking dependent fallback commands - Cover live hook, concurrency, cancellation, and cross-version paths

why: CI type-checks against Python 3.10, where Task.cancelling is not available in the asyncio stubs. what: - Replace the version-specific cancellation probe with a portable task wait - Make the shared startup future's nullable type explicit - Align lifecycle test overrides and terminal paths with their contracts

why: Whole-file backups could be overwritten or unwound out of order, and malformed state could abort configuration recovery. what: - Preserve first backups and enforce per-config LIFO ordering - Checkpoint state before config writes and each successful revert - Fail closed on missing backups, corrupt state, and malformed configs - Cover repeat swaps, partial failures, and recovery diagnostics

why: Experimental operations, engines, and agent tools could diverge from their documented constructor, result, and command-boundary contracts. what: - Make Attributes the checked source for generated constructor API prose - Expose only tmux-supported targets across Python, MCP, and payloads - Preserve create results and tmux 3.7 composed-operation invariants - Gate executable MCP payloads across every executable tool path - Preserve literal arguments across every experimental tmux transport - Add the direct YAML dependency and tested failure guidance

why: Show how typed Python chains become folded tmux command sequences while one asynchronous control-mode client carries the work. what: - Add a tabbed live tutorial for forward references and planner folding - Replace simulated plan examples with real tmux execution - Test visible compiled commands against actual control-mode dispatches - Add API destinations and navigation for plan concepts

why: First-party record fields must render with complete semantic prose, while gp-sphinx a36 now owns documented-member deduplication. what: - Expand the runtime contract across supported record declarations - Define inheritance, ordering, and exemption rules in AGENTS.md - Remove the obsolete no-undoc-members workaround and source test

why: CI must type-check the untyped doctest dependency boundary and the heterogeneous results produced by a compiled operation plan. what: - Describe the consumed doctest finder interface with a protocol - Narrow the terminal plan result before reading message text

why: Operation's ten class variables were documented as a hand-written definition list under Notes, because an Attributes entry for a class variable used to be dropped from the build. gp-sphinx renders one now, with the annotation and value alongside the prose, so the workaround costs a reader the type and default it cannot state. what: - Move kind, command, scope, result_cls, chainable, primitive, safety, effects, min_version, and flag_version_map into the Attributes section, dropping the Notes list Requires a gp-sphinx release carrying the class-variable rendering; under the pinned 0.0.1a36 these descriptions do not render.

why: Engine users need transport-specific examples that expose real output and boundaries without treating mock results as server evidence. what: - Add one executable first-success example per concrete engine - Map every engine to a focused live or offline tutorial - Test tutorial ownership and transport boundaries

why: gp-sphinx removes tabs.js after rendering, leaving pages with a missing asset reference even though inline tabs operate through CSS. what: - Filter only tabs.js from Sphinx page contexts at late priority - Verify final tab markup and assets with a one-page Sphinx build

tony added the enhancement label Jun 21, 2026

tony changed the title ~~Typed operations and engines: inert op spine (#689)~~ Typed operations and engines: spine + 4 engines + facades (#689) Jun 21, 2026

tony changed the title ~~Typed operations and engines: spine + 4 engines + facades (#689)~~ Typed operations and engines Jun 21, 2026

tony changed the title ~~Typed operations and engines~~ Typed operations & engines: spine, 6 engines, plans, models, facades (#689) Jun 21, 2026

tony force-pushed the engine-ops branch from 49768fd to 2904003 Compare June 23, 2026 02:48

tony mentioned this pull request Jun 27, 2026

Agent-state monitor over the experimental control-mode engine #692

Open

6 tasks

tony force-pushed the engine-ops branch from 13b6965 to 3965d94 Compare June 27, 2026 17:51

tony force-pushed the engine-ops branch from 1c98ccb to 8f2c6a1 Compare June 28, 2026 23:48

tony force-pushed the engine-ops branch from 8f2c6a1 to 23f9bbf Compare July 4, 2026 13:40

tony force-pushed the engine-ops branch from 23f9bbf to f542fb2 Compare July 4, 2026 14:39

tony changed the title ~~Typed operations & engines: spine, 6 engines, plans, models, facades (#689)~~ Typed operations, engines, workspace, query & MCP (#689) Jul 4, 2026

tony mentioned this pull request Jul 4, 2026

Fluent forward-ref build tier: split-type pane handles + plan() #700

Merged

tony force-pushed the engine-ops branch from ff34c94 to 32a0337 Compare July 5, 2026 13:49

tony force-pushed the engine-ops branch from 1fe8fc6 to e88e87b Compare July 5, 2026 23:16

tony force-pushed the engine-ops branch from e88e87b to cd20516 Compare July 11, 2026 10:33

tony force-pushed the engine-ops branch from 22557d9 to e6b44ac Compare July 12, 2026 11:25

tony added 26 commits July 26, 2026 06:25

Docs(fix[domain]): Wrap operation cards

8b7b2a8

why: Activate gp-sphinx card styling and responsive layout rules for custom operation entries. what: - Wrap operation entries in the shared API card shell - Assert the rendered shell contract in the Sphinx integration test

Docs(style[format]): Apply ruff output

751ac03

why: Keep the verified documentation extension and tests in the repository canonical format. what: - Apply ruff line wrapping to the operation domain and query tests

Retry(fix[clock]): Use monotonic time

3d7010d

why: Wall-clock corrections can move backward and invalidate timeout measurements in retry_until and its tests. what: - Measure retry deadlines with the monotonic clock - Measure elapsed-time assertions with the same clock

tony force-pushed the engine-ops branch from d0e37e0 to 4305b78 Compare July 26, 2026 11:47

tony added 3 commits July 26, 2026 09:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Typed operations, engines, workspace, query & MCP (#689)#690

Typed operations, engines, workspace, query & MCP (#689)#690
tony wants to merge 194 commits into
masterfrom
engine-ops

tony commented Jun 21, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jun 21, 2026 •

edited

Loading

Uh oh!

tony commented Jun 21, 2026

Uh oh!

tony commented Jun 21, 2026

Uh oh!

tony commented Jun 22, 2026

Uh oh!

tony commented Jun 22, 2026

Uh oh!

tony commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

tony commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What ships

Typed operations and results

Engines and command boundaries

Objects, queries, and workspaces

MCP server

Documentation and developer experience

Verification

Uh oh!

codecov Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tony commented Jun 21, 2026

Code review

Uh oh!

tony commented Jun 21, 2026

Code review

Uh oh!

tony commented Jun 22, 2026

Code review

Uh oh!

tony commented Jun 22, 2026

Code review

Uh oh!

tony commented Jul 5, 2026

Code review

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

tony commented Jun 21, 2026 •

edited

Loading

codecov Bot commented Jun 21, 2026 •

edited

Loading