Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
e1527b1
feat(browser): attach host-owned pages with revocable agent control a…
Qiyuanqiii Nov 12, 2026
5b4436f
feat(desktop): host shared browser tabs in native WebView2 controllers
Qiyuanqiii Nov 12, 2026
1354018
fix(desktop): keep human input available during agent browser control
Qiyuanqiii Nov 12, 2026
6a4d460
feat(web): add native shared browser workspace controls
Qiyuanqiii Nov 12, 2026
6a2402a
test(desktop): verify native worker gates and concurrent human input
Qiyuanqiii Nov 12, 2026
e058951
fix(desktop): track native cursor with host window lifecycle
Qiyuanqiii Nov 12, 2026
ae32628
fix(desktop): pass native bounds with Windows ARM64 aggregate ABI
Qiyuanqiii Nov 12, 2026
d38f134
fix(desktop): preserve native login popup opener and navigation results
Qiyuanqiii Nov 12, 2026
6f27d59
feat(desktop): connect native shared tabs to chat and agent sessions
Qiyuanqiii Nov 12, 2026
a16f0b5
fix(browser): preserve background tab viewport and exact lease cleanup
Qiyuanqiii Nov 12, 2026
8b89a0c
feat(browser): add same-page coordinate input with durable authority
Qiyuanqiii Nov 12, 2026
c7cd646
fix(browser): fence native request sends and position DOM clicks
Qiyuanqiii Nov 12, 2026
a5f4dfe
fix(browser): fence native HTTP dispatch at actual deferral completion
Qiyuanqiii Nov 12, 2026
faacde6
fix(desktop): forward native dispatch cancellation and page resize ev…
Qiyuanqiii Nov 12, 2026
34b34b8
feat(web): unify shared browser tabs and new tab workspace
Qiyuanqiii Nov 12, 2026
8d15dd4
test(desktop): exercise authenticated native browser and agent compos…
Qiyuanqiii Nov 12, 2026
0ad5ff2
docs(browser): record shared tab behavior and register native protocols
Qiyuanqiii Nov 12, 2026
1cc2334
fix(web): preserve model harness after unchanged provider save
Qiyuanqiii Nov 12, 2026
284c220
fix(ui): keep next-message action inside the composer
Qiyuanqiii Nov 12, 2026
bd60587
fix(chat): preserve literal file examples and anchor resource feedback
Qiyuanqiii Nov 12, 2026
7b954b1
fix(browser): reserve a native-free workspace resize strip
Qiyuanqiii Nov 12, 2026
d15b4ba
fix(ui): clarify model settings and conversation input counts
Qiyuanqiii Nov 12, 2026
143a64d
feat(chat): group work and final replies by durable user task
Qiyuanqiii Nov 12, 2026
93dd626
fix(chat): reveal located task work once without pinning it open
Qiyuanqiii Nov 12, 2026
f189ce2
Project transcript tasks from durable input execution boundaries
Qiyuanqiii Nov 12, 2026
a9cc113
test(chat): align conversation statistics wording
Qiyuanqiii Nov 12, 2026
2b57651
Validate task metadata in strict transcript responses
Qiyuanqiii Nov 12, 2026
dc3e688
test(chat): verify task metadata through API and narrative projection
Qiyuanqiii Nov 12, 2026
cdb97b5
Add authorized workspace image reads and model image projection
Qiyuanqiii Nov 12, 2026
f762f65
Check Full readiness and ground root tool availability in request sch…
Qiyuanqiii Nov 12, 2026
ec9b3a5
docs: compare native tool calls and programmatic execution
Qiyuanqiii Nov 12, 2026
ffacdff
test(desktop): exercise agent tools against shared native browser
Qiyuanqiii Nov 12, 2026
8fa2088
feat(images): wire workspace image activity and API contracts
Qiyuanqiii Nov 12, 2026
8a459b2
Verify image results using durable host call identities
Qiyuanqiii Nov 12, 2026
f4bb768
Preserve actionable Full permission failures through chat
Qiyuanqiii Nov 12, 2026
faa8870
fix(desktop): render background agent tabs without exposing native input
Qiyuanqiii Nov 12, 2026
dd63424
fix(images): bind model pixels to the exact root tool call
Qiyuanqiii Nov 12, 2026
1bcc6df
Document observed tool configuration and delivered runtime fixes
Qiyuanqiii Nov 12, 2026
1efd834
fix(images): accept recorded root call attribution
Qiyuanqiii Nov 12, 2026
232d1df
fix: reactivate Full access for idle running chats
Qiyuanqiii Nov 12, 2026
7a8575b
fix(web): link plain file paths beside prose punctuation
Qiyuanqiii Nov 12, 2026
0c88913
fix(chat): keep live user messages before the work they trigger
Qiyuanqiii Nov 12, 2026
3794241
fix: retain confirmed Full access for the next chat run
Qiyuanqiii Nov 12, 2026
d00c0e1
docs(protocols): register root tool exposure and workspace image evid…
Qiyuanqiii Nov 12, 2026
e1c91b0
Simplify daily permissions and move environment controls to advanced …
Qiyuanqiii Nov 12, 2026
1d208a4
Distinguish pre-creation permission choices from active grants
Qiyuanqiii Nov 12, 2026
527dd87
feat: default Windows chats to the local command runtime
Qiyuanqiii Nov 12, 2026
b1b5096
fix: reject unready command environments before model requests
Qiyuanqiii Nov 12, 2026
fe45617
feat: select chat permissions before the first message
Qiyuanqiii Nov 12, 2026
07b28e8
fix: enable ordinary Windows desktop browser access by default
Qiyuanqiii Nov 12, 2026
cdb7bbb
Keep the active task input ahead of live work until final commit
Qiyuanqiii Nov 12, 2026
cfbabc2
fix: bind pending inputs from recorded tool boundaries
Qiyuanqiii Nov 12, 2026
e9f9079
fix: retain consumed input binding after cancellation
Qiyuanqiii Nov 12, 2026
b3c01e7
fix: offer host-selected shell commands through durable runtime
Qiyuanqiii Nov 12, 2026
8d36d9a
fix: share command schema fields across input branches
Qiyuanqiii Nov 12, 2026
17b5347
test: exercise model-facing shell and browser tool continuation
Qiyuanqiii Nov 12, 2026
4575910
test: align migration history and settings coverage with current beha…
Qiyuanqiii Nov 13, 2026
f779963
fix: preserve pending approvals when Full access needs reactivation
Qiyuanqiii Nov 13, 2026
9a7c746
fix: preserve native pointer types across WebView2 callbacks
Qiyuanqiii Nov 13, 2026
a233daa
test: avoid unrelated Git dependency in command replay conflict coverage
Qiyuanqiii Nov 13, 2026
0aadd06
fix: fit tool guidance to the current request and align desktop integ…
Qiyuanqiii Nov 14, 2026
fa0caf3
fix(web): preserve message order, first turns, and browser recovery
Qiyuanqiii Nov 15, 2026
3bc87aa
fix(agent): bound image continuations and keep edits tied to their task
Qiyuanqiii Nov 15, 2026
6892e06
fix(browser): bind explicit handoffs to a tab and fence native input …
Qiyuanqiii Nov 15, 2026
cd28aa9
test(web): verify advanced CDP stays disabled without active Full access
Qiyuanqiii Nov 15, 2026
70d40b3
test(changes): exercise legacy replies in separate runs without a tas…
Qiyuanqiii Nov 16, 2026
61f25c4
fix(web): preserve task-wide reply actions and lock edits during prov…
Qiyuanqiii Nov 16, 2026
5226fc1
fix(browser): retain the attached tab when input is cancelled before …
Qiyuanqiii Nov 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions cmd/cyberagent-desktop/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ import (
"path/filepath"
goruntime "runtime"
"strings"
"sync/atomic"
"time"

"cyberagent-workbench/internal/app"
"cyberagent-workbench/internal/apperror"
Expand Down Expand Up @@ -723,6 +725,11 @@ func runDesktop(config desktopOptions) error {
}
}

sharedBrowserOptions, closeSharedBrowser, err := prepareSharedBrowser(homePath)
if err != nil {
return err
}
defer closeSharedBrowser()
databasePath := filepath.Join(homePath, "cyberagent.db")
var computerUseExecutable string
if goruntime.GOOS == "windows" && config.runExecution {
Expand All @@ -732,6 +739,7 @@ func runDesktop(config desktopOptions) error {
}
}
controlPlane, err := desktop.OpenControlPlane(desktop.ControlPlaneConfig{
SharedBrowser: sharedBrowserOptions,
TerminalSettingsStore: terminalSettingsStore,
DatabasePath: databasePath, HomePath: homePath, ReadToken: readToken,
ControlToken: controlToken,
Expand Down Expand Up @@ -827,6 +835,12 @@ func runDesktop(config desktopOptions) error {
commandRuntimeAdapterInstalled, commandRuntimeAdapterReady :=
controlPlane.CommandRuntimeProcessStatus()
bridge, err := desktop.NewDesktopBridge(desktop.DesktopBridgeConfig{
SharedBrowser: func() desktop.SharedBrowserController {
if controlPlane.SharedBrowser() == nil {
return nil
}
return controlPlane.SharedBrowser()
}(),
TerminalSettingsStore: terminalSettingsStore,
TerminalSettingsConfirmer: func() desktop.DesktopTerminalSettingsConfirmer {
if terminalSettingsStore == nil {
Expand Down Expand Up @@ -907,6 +921,7 @@ func runDesktop(config desktopOptions) error {
return err
}

var browserClosing, browserClosed atomic.Bool
appOptions := &options.App{
Title: app.Name, Width: defaultDesktopWidth, Height: defaultDesktopHeight,
MinWidth: 480, MinHeight: 600,
Expand All @@ -926,6 +941,33 @@ func runDesktop(config desktopOptions) error {
OnShutdown: func(context.Context) {
lifecycle.Stop()
},
OnBeforeClose: func(ctx context.Context) bool {
browser := controlPlane.SharedBrowser()
if browser == nil || browserClosed.Load() {
return false
}
if browserClosing.CompareAndSwap(false, true) {
// Keep the native message pump alive while outstanding CDP calls
// detach. Blocking its UI thread here would deadlock COM cleanup.
go func() {
closeCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
result := make(chan error, 1)
go func() { result <- browser.Close(closeCtx) }()
select {
case err := <-result:
if err != nil {
fmt.Fprintln(os.Stderr, "browser shutdown:", err)
}
case <-closeCtx.Done():
fmt.Fprintln(os.Stderr, "browser shutdown:", closeCtx.Err())
}
browserClosed.Store(true)
runtime.Quit(ctx)
}()
}
return true
},
Bind: []interface{}{bridge},
// Keep Wails' production context-menu filter: editable controls and
// selected text get native edit actions; ordinary page surfaces do not.
Expand Down
9 changes: 9 additions & 0 deletions cmd/cyberagent-desktop/shared_browser_other.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
//go:build !windows && desktop

package main

import "cyberagent-workbench/internal/desktop"

func prepareSharedBrowser(string) (*desktop.SharedBrowserOptions, func(), error) {
return nil, func() {}, nil
}
65 changes: 65 additions & 0 deletions cmd/cyberagent-desktop/shared_browser_windows.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
//go:build windows && desktop

package main

import (
"context"
"errors"
"os"
"runtime"
"unsafe"

"cyberagent-workbench/internal/desktop"
"cyberagent-workbench/internal/desktopbrowser"
wailsruntime "github.com/wailsapp/wails/v2/pkg/runtime"
"golang.org/x/sys/windows"
)

func prepareSharedBrowser(home string) (*desktop.SharedBrowserOptions, func(), error) {
runtime.LockOSThread()
dispatcher, err := desktopbrowser.NewDispatcher()
if err != nil {
runtime.UnlockOSThread()
return nil, nil, err
}
cleanup := func() { _ = dispatcher.Close(); runtime.UnlockOSThread() }
return &desktop.SharedBrowserOptions{HomePath: home, CreateHost: func(ctx context.Context, profile string, onEvent func(desktopbrowser.Event)) (*desktopbrowser.Host, error) {
hwnd, err := sharedBrowserParent(dispatcher.ThreadID())
if err != nil {
return nil, err
}
return desktopbrowser.NewHost(ctx, desktopbrowser.HostOptions{ParentHWND: hwnd, Dispatcher: dispatcher, UserDataDir: profile, OnEvent: onEvent})
}, SetFullscreen: func(ctx context.Context, enabled bool) error {
if enabled {
wailsruntime.WindowFullscreen(ctx)
} else {
wailsruntime.WindowUnfullscreen(ctx)
}
return nil
}}, cleanup, nil
}

// Select the shell's configured class on its exact UI thread and process.
// Foreground windows, captions, and browser child windows are not candidates.
func sharedBrowserParent(threadID uint32) (uintptr, error) {
user32 := windows.NewLazySystemDLL("user32.dll")
var matches []uintptr
callback := windows.NewCallback(func(hwnd, _ uintptr) uintptr {
var pid uint32
owner, _, _ := user32.NewProc("GetWindowThreadProcessId").Call(hwnd, uintptr(unsafe.Pointer(&pid)))
if uint32(owner) != threadID || pid != uint32(os.Getpid()) {
return 1
}
var class [256]uint16
n, _, _ := user32.NewProc("GetClassNameW").Call(hwnd, uintptr(unsafe.Pointer(&class[0])), uintptr(len(class)))
if n > 0 && windows.UTF16ToString(class[:]) == "CyberAgentWorkbench" {
matches = append(matches, hwnd)
}
return 1
})
user32.NewProc("EnumThreadWindows").Call(uintptr(threadID), callback, 0)
if len(matches) != 1 {
return 0, errors.New("shared browser requires the running desktop window")
}
return matches[0], nil
}
103 changes: 103 additions & 0 deletions docs/adr/0212-native-shared-browser-tabs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# ADR 0212: Native shared browser tabs

Date: 2026-11-12

## Decision

The Windows desktop embeds independent WebView2 controllers inside its workspace.
The workspace has one tab strip, a navigation/address row, and a content region.
The content region shows either the application's new-tab page or the actual
native web page. Split view, full workspace view, and operating-system fullscreen
use the same container. A hidden native surface keeps its document and login.

External pages have neither the Wails bridge nor the application's API tokens.
Each chat owns a browser profile, separate from the application's renderer and
the user's regular browser. Tabs in a chat share cookies. A new chat attempts
to adopt its draft browser before sending the first turn. If adoption fails,
the message is still submitted and the user can reopen the browser in the chat.
The profile index migration is atomic and retries do not consume a later draft.
Recent pages on the new-tab screen are a bounded, session-local list of successful
native navigations; no sample visits or recommendations are manufactured.

The Agent attaches to the exact host-owned page. The attachment creates no new
browser context or substitute page. Both DOM/reference actions and coordinate
actions use the existing authorized browser tool path and that same page.
Coordinate actions validate the document epoch and current viewport. The native
Agent pointer is separate from the operator's system pointer and cannot receive
mouse input or keyboard focus.

Operator input stays enabled during Agent work. Clicking, typing, or changing
the displayed tab does not cancel the Agent. An attachment retains its stable
target tab when the operator views another tab. The tab strip identifies the
Agent's tab. The toolbar can explicitly hand the displayed tab to the Agent;
the request names that tab so a concurrent switch cannot substitute another.
Navigation and resizing invalidate
old references/coordinates so a stale action must observe the current document.
Closing the target page or revoking Run authority invalidates its Agent lease.
Releasing that lease preserves the user-owned page and profile; its cleanup
receipt records detachment, not destruction of a browser process or profile.

Native login popups use the same WebView2 environment and the NewWindowRequested
deferral/NewWindow protocol. This preserves window.opener and the browser's own
navigation, POST, and window-closing behavior. The desktop registers the page
that the native host created instead of opening a second copy of the URL.

## Permissions and platform boundaries

The existing Run, tool-call, document, and network authorities remain mandatory
for Agent operations. Every WebView installs all-source WebResourceRequested
filters before its first navigation, including shared and service workers.
Only one Agent network lease can govern a profile at a time. Revocation aborts
pending requests; a paused Agent request cannot become an operator request.

WebView2 cannot reliably attribute an asynchronous HTTP request to a human input
versus an Agent input. During an Agent attachment, the profile's HTTP requests
therefore share the Run's origin and HTTP-write policy, including operator-triggered
requests. Existing WebSocket/WebRTC messages are not covered by the HTTP request
filter. This implementation does not provide per-message approval for those
transports. These limits must not be reported as complete network containment.

Windows runtimes without the required request-source filter interface can host
manual pages but reject Agent attachment. Non-Windows desktop builds retain
their existing browser behavior. Windows amd64 has native runtime verification;
Windows ARM64 has compilation/ABI coverage and still needs device verification.

## Lifecycle and compatibility

The desktop binding is `desktop_shared_browser.v1`, advertised by
`shared_browser_enabled`. Old renderers and unsupported hosts retain their
existing behavior. Surface updates carry monotonically increasing revisions,
CSS bounds, and the renderer's device scale; stale geometry cannot re-show a
hidden or replaced surface. HTML menus and modal overlays temporarily hide the
native child to respect Windows child-window stacking.

COM work is marshalled onto the Wails STA through an owned dispatcher window.
Shutdown first detaches Agent work and closes controllers while the message loop
is still running, then exits the application. Native pages and cursor overlays
also close when their parent window is destroyed.

Rollback retains browser profile directories and the profile index. It does not
delete login data or reinterpret existing Agent tool records. The browser_move
tool's durable registration and new migration preserve the existing browser
authority checks and historical migration checksums.

## Verification

The Windows integration test uses a hidden native parent window, the actual
WebView2 runtime, SQLite-backed workspace/chat ownership, and the desktop bridge.
It checks a redirect login with an HttpOnly cookie, attachment to that exact
authenticated page, DOM-reference clicks before and after another tab is selected,
login popups and opener callbacks, visit history, and draft profile adoption and
isolation. It runs without model calls or operating-system mouse input:

```powershell
$env:UC_NATIVE_BROWSER_PROBE='1'
go test -race ./internal/desktopbrowser ./internal/desktop -run 'TestNative|TestSharedBrowser' -count=1 -timeout 120s
```

React tests cover the shared tab strip, keyboard focus across tool and website
tabs, surface geometry, fullscreen restoration, and adoption before the first
turn. Headless layout screenshots exercise the application styles with a test
bridge. These screenshots do not establish native-window positioning. Interactive
foreground acceptance of the complete desktop app remains unperformed after the
operator stopped desktop input automation.
Loading
Loading