diff --git a/AGENTS.md b/AGENTS.md index 6caa6619..41f55f49 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,6 +6,7 @@ Start here: - Repo overview and commands: [README.md](README.md) - Working instructions and project conventions: [CLAUDE.md](CLAUDE.md) - Canonical feature architecture standard: [docs/FEATURE_ARCHITECTURE_STANDARD.md](docs/FEATURE_ARCHITECTURE_STANDARD.md) +- Agent team launch/runtime debugging runbook: [docs/team-management/debugging-agent-teams.md](docs/team-management/debugging-agent-teams.md) For new features: - Default home for medium and large features: `src/features//` @@ -15,6 +16,7 @@ For new features: ## Review guidelines - Treat regressions in agent team messaging, task lifecycle, session parsing, code review UI, and provider/runtime detection as high priority. +- For team launch hangs, OpenCode `registered`/`bootstrap unconfirmed`, missing teammate replies, or suspicious task logs, follow [docs/team-management/debugging-agent-teams.md](docs/team-management/debugging-agent-teams.md) before changing code. - Verify new medium and large features follow `docs/FEATURE_ARCHITECTURE_STANDARD.md`, especially cross-process boundaries and public feature entrypoints. - Check that Electron main, preload, renderer, and shared code keep their responsibilities separate and use the documented path aliases. - Flag changes that manually concatenate agent block markers instead of using `wrapAgentBlock(text)`. diff --git a/CLAUDE.md b/CLAUDE.md index ed080edb..bd649074 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -110,6 +110,12 @@ Keep orphaned Task calls (no matching subagent) for visibility. Claude Code's "Orchestrate Teams" feature: multiple sessions coordinate as a team. Official docs: https://code.claude.com/docs/en/agent-teams +#### Debugging Team Launches And Teammates +- Use [`docs/team-management/debugging-agent-teams.md`](docs/team-management/debugging-agent-teams.md) when a team launch hangs, a teammate remains `registered`, OpenCode shows `bootstrap unconfirmed`, messages are missing, or Task Log Stream looks wrong. +- Always correlate UI diagnostics with persisted files under `~/.claude/teams//`, live process state, and runtime-specific evidence before changing code. +- For OpenCode secondary lanes, do not confuse primary filesystem readiness with lane bootstrap readiness. A missing OpenCode inbox during primary launch is not automatically a bug. +- Do not treat `member_briefing` as runtime evidence. OpenCode deliverability requires lane-scoped committed runtime evidence such as `opencode-sessions.json` plus its manifest entry. + #### Message Delivery Architecture - **Lead** reads ONLY stdin (stream-json). Messages to lead must go through `relayLeadInboxMessages()` which converts inbox entries to stdin. - **Teammates** are independent CLI processes. Claude Code runtime monitors each teammate's inbox file and delivers messages between turns. No relay through lead needed. diff --git a/docs/team-management/debugging-agent-teams.md b/docs/team-management/debugging-agent-teams.md new file mode 100644 index 00000000..a6baac42 --- /dev/null +++ b/docs/team-management/debugging-agent-teams.md @@ -0,0 +1,176 @@ +# Debugging Agent Teams + +Use this runbook when a team launch hangs, a teammate is marked `registered` or `failed_to_start`, messages do not appear, or OpenCode participants look online but do not answer. + +## First Rule + +Do not guess from the UI alone. Always correlate: +- UI diagnostics copied from the launch/member detail panel +- persisted team files under `~/.claude/teams//` +- live process table +- runtime-specific evidence, especially OpenCode lane manifests + +## Key Files + +Team root: + +```bash +TEAM="" +TEAM_DIR="$HOME/.claude/teams/$TEAM" +``` + +Important files and folders: +- `config.json` - configured members, provider/model selection, project path +- `members-meta.json` - member metadata, removed members, worktree settings if present +- `launch-state.json` - current app-side truth for member launch/liveness +- `bootstrap-state.json` - bootstrap phase summary when present +- `bootstrap-journal.jsonl` - ordered bootstrap events from the CLI/runtime +- `inboxes/*.json` - durable inbox messages for user, lead, and native teammates +- `sentMessages.json` - app-side sent-message records +- `tasks/*.json` - task board state +- `.opencode-runtime/lanes.json` - OpenCode lane index +- `.opencode-runtime/lanes//manifest.json` - lane-scoped runtime store manifest +- `.opencode-runtime/lanes//opencode-sessions.json` - committed OpenCode session evidence + +Quick inspection: + +```bash +jq '.teamLaunchState, .summary, .members' "$TEAM_DIR/launch-state.json" +jq '.lanes' "$TEAM_DIR/.opencode-runtime/lanes.json" 2>/dev/null +find "$TEAM_DIR/.opencode-runtime" -maxdepth 3 -type f | sort +tail -80 "$TEAM_DIR/bootstrap-journal.jsonl" 2>/dev/null +``` + +## Launch Phases + +Primary launch and OpenCode secondary lanes are different paths. + +- Primary CLI members are created by the main provisioning process. +- OpenCode secondary members are launched as side lanes after primary filesystem readiness. +- Missing `inboxes/.json` is not automatically a launch bug. OpenCode side lanes do not have to be primary inbox-created before they start. +- The UI can show the team still launching while primary members are already usable, because "all teammates joined" waits for secondary lanes too. + +When a launch hangs at `Prepared communication channels for X/Y members`, check whether `Y` incorrectly includes secondary OpenCode members. The filesystem monitor should wait for `effectiveMembers`, not every requested member. + +## Member State Meanings + +Common `launch-state.json` cases: + +- `confirmed_alive` with `bootstrapConfirmed: true` - member is usable. +- `registered` / `runtime_pending_bootstrap` - process or lane exists, but bootstrap proof is not committed yet. +- `registered_only` - app has persisted metadata, but no live runtime proof. +- `runtime_process_candidate` - process/session was observed, but committed runtime evidence is incomplete or pending. +- `failed_to_start` with `runtime_process` - a process exists, but the launch gate still failed. Inspect diagnostics and runtime evidence. +- `failed_to_start` with `stale_metadata` - persisted pid/session is old or dead. + +Do not treat `member_briefing` alone as runtime evidence. For OpenCode, the authoritative proof is committed bootstrap/session evidence in the lane runtime store. + +## OpenCode Debug Flow + +For an OpenCode teammate: + +```bash +MEMBER="" +jq --arg member "$MEMBER" '.members[$member]' "$TEAM_DIR/launch-state.json" +jq '.lanes' "$TEAM_DIR/.opencode-runtime/lanes.json" 2>/dev/null +find "$TEAM_DIR/.opencode-runtime/lanes" -maxdepth 3 -type f | sort +``` + +Expected healthy OpenCode lane: +- `lanes.json` has the lane state `active` +- lane `manifest.json` has `activeRunId` +- lane manifest has at least one runtime evidence entry, usually `opencode.sessionStore` +- lane directory has `opencode-sessions.json` +- `launch-state.json` member has `runtimeRunId`, `runtimeSessionId`, and `bootstrapConfirmed: true` + +If the bridge says bootstrap succeeded but the manifest has `entries: []`, the issue is evidence commit, not model behavior. The member must not be considered deliverable until `opencode-sessions.json` and its manifest entry exist. + +OpenCode bridge ledger, if needed: + +```bash +LEDGER="$HOME/Library/Application Support/claude-agent-teams-ui/opencode-bridge/command-ledger.json" +jq --arg team "$TEAM" '.data[] | select(.teamName == $team)' "$LEDGER" 2>/dev/null +``` + +Live process checks: + +```bash +pgrep -af "opencode serve" +ps -p -o pid,ppid,etime,command +``` + +Do not kill all OpenCode processes as a debugging shortcut. First identify whether the pid belongs to the current team/lane. Some OpenCode temp `libopentui.dylib` files are held by live `opencode serve` processes and should only be cleaned after those processes are stopped. + +## Messaging Debug Flow + +Lead and teammates use different delivery paths: + +- Lead reads stdin. Messages to lead go through `relayLeadInboxMessages()`. +- Native teammates read their inbox files directly. +- OpenCode teammates receive prompts through runtime delivery and must reply via `agent-teams_message_send`. +- Teammate-to-user replies should appear in `inboxes/user.json` or app sent-message projections. + +If a notification appears but the Messages UI does not show it: + +```bash +jq '.' "$TEAM_DIR/inboxes/user.json" 2>/dev/null +jq '.' "$TEAM_DIR/sentMessages.json" 2>/dev/null +``` + +Check `from`, `to`, `messageId`, `relayOfMessageId`, and `taskRefs`. Unknown authors should be rejected or normalized at the write boundary, not silently rendered as fake teammates. + +For OpenCode "message saved but not delivered" cases, inspect the OpenCode prompt-delivery ledger and response proof. Do not synthesize visible replies in the frontend. + +## Task And Work-Stall Debug Flow + +For task stalls: + +```bash +TASK="" +rg -n "$TASK" "$TEAM_DIR/tasks" "$TEAM_DIR/inboxes" "$TEAM_DIR/bootstrap-journal.jsonl" 2>/dev/null +``` + +Important distinctions: +- Delivery proof means the agent received the message. +- Task progress proof means the agent made meaningful task progress. +- A weak comment like "starting work" is not strong progress. +- `task_add_comment` should be evaluated from the actual persisted comment text, not only from the tool call. + +Task-stall monitor defaults: +- General task-stall monitor is for all agents. +- OpenCode direct remediation is provider-specific and should nudge the OpenCode owner first. +- If OpenCode remediation is not accepted, fallback to lead alert. +- Watchdog/remediation must not auto-start new OpenCode processes. + +## Task Log Stream Debug Flow + +Task Log Stream is a projection, not a separate source of truth. + +For OpenCode tasks, a healthy stream should show native tool rows such as `read`, `bash`, `edit`, `write`, plus Agent Teams MCP rows. If it only shows `agent-teams_*` calls: +- confirm the task has OpenCode attribution for the member/session +- confirm the OpenCode transcript contains native tools inside the bounded task window +- check whether the task was assigned after the native work happened +- do not widen attribution so far that unrelated session work is pulled into the task + +If Changes says "No file changes recorded" while native `write`/`edit` rows exist, inspect the ledger/backfill path. Task logs can show runtime tools even when `.board-task-changes/**` was not created. + +## Safe Fix Checklist + +Before changing launch or runtime logic: +- Preserve stale-run, tombstone, stopped-team, and removed-member guards. +- Do not make `member_briefing` runtime evidence. +- Do not make delivery/watchdog auto-launch a fresh OpenCode lane. +- Keep primary launch readiness separate from secondary OpenCode lane readiness. +- Keep runtime evidence lane-scoped. Never let one OpenCode lane satisfy another lane. +- Add a regression test for the exact state shape you found in `launch-state.json`. + +Recommended verification: + +```bash +pnpm vitest run test/main/services/team/TeamProvisioningService.test.ts +pnpm vitest run test/main/services/team/TeamAgentLaunchMatrix.safe-e2e.test.ts +pnpm typecheck --pretty false +git diff --check +``` + +Use narrower test commands first when editing a focused path, then run the broader suite that covers launch, delivery, and liveness. diff --git a/src/main/services/team/TeamProvisioningService.ts b/src/main/services/team/TeamProvisioningService.ts index d3eac7b0..020f6921 100644 --- a/src/main/services/team/TeamProvisioningService.ts +++ b/src/main/services/team/TeamProvisioningService.ts @@ -155,22 +155,34 @@ import { import { clearOpenCodeRuntimeLaneStorage, getOpenCodeLaneScopedRuntimeFilePath, + getOpenCodeRuntimeManifestPath, getOpenCodeRuntimeRunTombstonesPath, getOpenCodeTeamRuntimeDirectory, inspectOpenCodeRuntimeLaneStorage, migrateLegacyOpenCodeRuntimeState, OpenCodeRuntimeManifestEvidenceReader, + readCommittedOpenCodeBootstrapSessionEvidence, readOpenCodeRuntimeLaneIndex, recoverStaleOpenCodeRuntimeLaneIndexEntry, removeOpenCodeRuntimeLaneIndexEntry, setOpenCodeRuntimeActiveRunManifest, upsertOpenCodeRuntimeLaneIndexEntry, } from './opencode/store/OpenCodeRuntimeManifestEvidenceReader'; +import type { + OpenCodeCommittedBootstrapSessionRecord, + OpenCodeRuntimeLaneIndexEntry, +} from './opencode/store/OpenCodeRuntimeManifestEvidenceReader'; import { createRuntimeRunTombstoneStore, type RuntimeEvidenceKind, RuntimeStaleEvidenceError, } from './opencode/store/RuntimeRunTombstoneStore'; +import { + createRuntimeStoreManifestStore, + createRuntimeStoreReceiptStore, + OPENCODE_RUNTIME_STORE_DESCRIPTORS, + RuntimeStoreBatchWriter, +} from './opencode/store/RuntimeStoreManifest'; import { OpenCodeTaskLogAttributionStore } from './taskLogs/stream/OpenCodeTaskLogAttributionStore'; import { buildActionModeProtocol } from './actionModeInstructions'; import { isAgentTeamsToolUse } from './agentTeamsToolNames'; @@ -1890,6 +1902,142 @@ function isPersistedOpenCodeSecondaryLaneMember( ); } +function namesMatchCaseInsensitive(left: string, right: string): boolean { + return left.trim().toLowerCase() === right.trim().toLowerCase(); +} + +function isOpenCodeOverlayMemberRemoved( + metaMembers: readonly { name?: string; removedAt?: unknown }[], + memberName: string +): boolean { + return metaMembers.some( + (member) => + typeof member.name === 'string' && + namesMatchCaseInsensitive(member.name, memberName) && + member.removedAt != null + ); +} + +function hasStaleOpenCodeSecondaryLaunchDiagnostic( + member: PersistedTeamLaunchMemberState +): boolean { + return hasStaleOpenCodeDiagnostics(getOpenCodeLaunchDiagnosticValues(member)); +} + +function hasRealOpenCodeLaunchDiagnostic(member: PersistedTeamLaunchMemberState): boolean { + const text = getOpenCodeLaunchDiagnosticValues(member) + .filter((value): value is string => typeof value === 'string') + .join('\n') + .toLowerCase(); + return text.length > 0 && hasRealOpenCodeFailureDiagnostic(text); +} + +function getOpenCodeLaunchDiagnosticValues( + member: PersistedTeamLaunchMemberState +): readonly unknown[] { + return [member.hardFailureReason, member.runtimeDiagnostic, ...(member.diagnostics ?? [])]; +} + +function hasStaleOpenCodeDiagnostics(values: readonly unknown[] | undefined): boolean { + const text = (values ?? []) + .filter((value): value is string => typeof value === 'string') + .join('\n') + .toLowerCase(); + if (!text) { + return false; + } + if (hasRealOpenCodeFailureDiagnostic(text)) { + return false; + } + return ( + text.includes('no lane runtime evidence') || + text.includes('no runtime evidence') || + text.includes('runtime evidence was not committed') || + text.includes('no lane runtime evidence was committed') || + text.includes('registered runtime metadata without live process') || + text.includes('member has persisted runtime metadata only') || + text.includes('opencode bridge reported member launch failure') || + text.includes(OPENCODE_UNCOMMITTED_BOOTSTRAP_DIAGNOSTIC.toLowerCase()) + ); +} + +function hasRealOpenCodeFailureDiagnostic(text: string): boolean { + return ( + /\bauth(?:entication|orization)?\b/.test(text) || + text.includes('api key') || + text.includes('unauthorized') || + text.includes('forbidden') || + text.includes('invalid_request') || + text.includes('model not found') || + text.includes('not found in live opencode catalog') || + text.includes('provider unavailable') || + text.includes('quota') || + text.includes('credits') || + text.includes('max_tokens') || + text.includes('rate limit') || + text.includes('member removed') || + text.includes('session conflict') || + text.includes('run tombstoned') || + text.includes('stop requested') || + text.includes('relaunch started') + ); +} + +function promoteOpenCodeSecondaryMemberFromCommittedBootstrapEvidence(input: { + current: PersistedTeamLaunchMemberState; + previous: PersistedTeamLaunchMemberState | null; + session: OpenCodeCommittedBootstrapSessionRecord; + now: string; +}): PersistedTeamLaunchMemberState { + const observedAt = input.session.observedAt ?? input.now; + const diagnostics = [ + ...new Set([ + ...filterStaleOpenCodeOverlayDiagnostics(input.current.diagnostics), + 'opencode_bootstrap_evidence_committed', + ]), + ]; + const runtimeAlive = input.current.runtimeAlive === true; + return { + ...input.previous, + ...input.current, + launchState: 'confirmed_alive', + agentToolAccepted: true, + bootstrapConfirmed: true, + runtimeAlive, + hardFailure: false, + hardFailureReason: undefined, + runtimeRunId: input.session.runId ?? input.current.runtimeRunId, + runtimeSessionId: input.session.id, + livenessKind: runtimeAlive + ? input.current.livenessKind + : input.current.livenessKind === 'runtime_process' || + input.current.livenessKind === 'runtime_process_candidate' + ? input.current.livenessKind + : 'confirmed_bootstrap', + runtimeDiagnostic: 'OpenCode bootstrap evidence committed.', + runtimeDiagnosticSeverity: 'info', + firstSpawnAcceptedAt: + input.current.firstSpawnAcceptedAt ?? input.previous?.firstSpawnAcceptedAt ?? observedAt, + lastHeartbeatAt: input.current.lastHeartbeatAt ?? input.previous?.lastHeartbeatAt ?? observedAt, + runtimeLastSeenAt: runtimeAlive ? (input.current.runtimeLastSeenAt ?? observedAt) : undefined, + lastRuntimeAliveAt: runtimeAlive + ? (input.current.lastRuntimeAliveAt ?? input.previous?.lastRuntimeAliveAt ?? observedAt) + : input.current.lastRuntimeAliveAt, + lastEvaluatedAt: input.now, + sources: { + ...(input.previous?.sources ?? {}), + ...(input.current.sources ?? {}), + nativeHeartbeat: true, + processAlive: runtimeAlive || undefined, + }, + diagnostics, + }; +} + +function filterStaleOpenCodeOverlayDiagnostics(values: readonly string[] | undefined): string[] { + return (values ?? []).filter((value) => !hasStaleOpenCodeDiagnostics([value])); +} + function isRecoverablePersistedOpenCodeTerminalRuntimeCandidate( member: PersistedTeamLaunchMemberState | undefined | null ): boolean { @@ -8324,6 +8472,14 @@ export class TeamProvisioningService { previousMember: idempotent.previousMember, }); if (idempotent.state === 'duplicate') { + await this.commitOpenCodeRuntimeBootstrapSessionEvidence({ + teamName, + runId, + laneId, + memberName, + runtimeSessionId, + observedAt, + }); return { ok: true, providerId: 'opencode', @@ -8344,6 +8500,14 @@ export class TeamProvisioningService { runId ); } + await this.commitOpenCodeRuntimeBootstrapSessionEvidence({ + teamName, + runId, + laneId, + memberName, + runtimeSessionId, + observedAt, + }); await this.updateOpenCodeRuntimeMemberLiveness({ teamName, runId, @@ -8368,6 +8532,126 @@ export class TeamProvisioningService { }; } + private async commitOpenCodeRuntimeBootstrapSessionEvidence(input: { + teamName: string; + runId: string; + laneId: string; + memberName: string; + runtimeSessionId: string; + observedAt: string; + }): Promise { + const descriptor = OPENCODE_RUNTIME_STORE_DESCRIPTORS.find( + (candidate) => candidate.schemaName === 'opencode.sessionStore' + ); + if (!descriptor) { + throw new Error('OpenCode runtime session store descriptor is not registered'); + } + + const manifestPath = getOpenCodeRuntimeManifestPath( + getTeamsBasePath(), + input.teamName, + input.laneId + ); + const runtimeDirectory = path.dirname(manifestPath); + await fs.promises.mkdir(runtimeDirectory, { recursive: true }); + const sessionStorePath = path.join(runtimeDirectory, descriptor.relativePath); + const existingSessions = await this.readOpenCodeRuntimeSessionStore(sessionStorePath); + const session = { + id: input.runtimeSessionId, + teamName: input.teamName, + memberName: input.memberName, + runId: input.runId, + laneId: input.laneId, + providerId: 'opencode', + observedAt: input.observedAt, + source: 'runtime_bootstrap_checkin', + }; + const sessions = this.mergeOpenCodeRuntimeSessionRecords(existingSessions, session); + const manifestStore = createRuntimeStoreManifestStore({ + filePath: manifestPath, + teamName: input.teamName, + }); + const receiptStore = createRuntimeStoreReceiptStore({ + filePath: path.join(runtimeDirectory, 'opencode-runtime-receipts.json'), + }); + const writer = new RuntimeStoreBatchWriter(runtimeDirectory, manifestStore, receiptStore); + + await writer.writeBatch({ + teamName: input.teamName, + runId: input.runId, + capabilitySnapshotId: null, + behaviorFingerprint: null, + reason: 'launch_checkpoint', + writes: [ + { + descriptor, + data: { sessions }, + }, + ], + }); + } + + private async readOpenCodeRuntimeSessionStore( + filePath: string + ): Promise[]> { + let raw: string; + try { + raw = await fs.promises.readFile(filePath, 'utf8'); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') { + return []; + } + throw error; + } + + try { + const parsed = JSON.parse(raw) as unknown; + const record = + parsed && typeof parsed === 'object' && !Array.isArray(parsed) + ? (parsed as Record) + : null; + const data = + record && Object.prototype.hasOwnProperty.call(record, 'data') ? record.data : record; + const sessions = + data && typeof data === 'object' && !Array.isArray(data) + ? (data as Record).sessions + : null; + return Array.isArray(sessions) + ? sessions.filter( + (session): session is Record => + Boolean(session) && typeof session === 'object' && !Array.isArray(session) + ) + : []; + } catch { + return []; + } + } + + private mergeOpenCodeRuntimeSessionRecords( + existingSessions: Record[], + session: Record + ): Record[] { + const sessionId = typeof session.id === 'string' ? session.id.trim() : ''; + const memberName = typeof session.memberName === 'string' ? session.memberName.trim() : ''; + const runId = typeof session.runId === 'string' ? session.runId.trim() : ''; + const laneId = typeof session.laneId === 'string' ? session.laneId.trim() : ''; + const filtered = existingSessions.filter((candidate) => { + const candidateId = typeof candidate.id === 'string' ? candidate.id.trim() : ''; + if (sessionId && candidateId === sessionId) { + return false; + } + const sameMember = + memberName && + runId && + laneId && + candidate.memberName === memberName && + candidate.runId === runId && + candidate.laneId === laneId; + return !sameMember; + }); + return [...filtered, session]; + } + private async resolveOpenCodeRuntimeBootstrapCheckinIdempotency(input: { teamName: string; runId: string; @@ -13881,6 +14165,11 @@ export class TeamProvisioningService { result: TeamRuntimeLaunchResult, input: TeamRuntimeLaunchInput ): Promise { + await this.commitOpenCodeRuntimeAdapterLaunchSessionEvidence({ + teamName: input.teamName, + laneId: input.laneId?.trim() || 'primary', + result, + }); const members: Record = {}; for (const member of input.expectedMembers) { const evidence = result.members[member.name]; @@ -13894,8 +14183,32 @@ export class TeamProvisioningService { launchPhase: result.launchPhase, members, }); - await this.writeLaunchStateSnapshot(input.teamName, snapshot); - return snapshot; + return this.writeLaunchStateSnapshot(input.teamName, snapshot); + } + + private async commitOpenCodeRuntimeAdapterLaunchSessionEvidence(params: { + teamName: string; + laneId: string; + result: TeamRuntimeLaunchResult; + }): Promise { + for (const [memberName, evidence] of Object.entries(params.result.members)) { + const runtimeSessionId = evidence.sessionId?.trim(); + const confirmed = + evidence.launchState === 'confirmed_alive' || + evidence.bootstrapConfirmed === true || + evidence.livenessKind === 'confirmed_bootstrap'; + if (!confirmed || !runtimeSessionId) { + continue; + } + await this.commitOpenCodeRuntimeBootstrapSessionEvidence({ + teamName: params.teamName, + runId: params.result.runId, + laneId: params.laneId, + memberName, + runtimeSessionId, + observedAt: nowIso(), + }); + } } private toOpenCodePersistedLaunchMember( @@ -16367,7 +16680,7 @@ export class TeamProvisioningService { }) ) { run.lastMemberSpawnAuditConfigReadWarningAt = now; - logger.warn(`[${run.teamName}] auditMemberSpawnStatuses: config.json not readable`); + logger.debug(`[${run.teamName}] auditMemberSpawnStatuses: config.json not readable`); } return; } @@ -17251,11 +17564,252 @@ export class TeamProvisioningService { await clearBootstrapState(teamName); } + private async applyOpenCodeSecondaryEvidenceOverlay(params: { + teamName: string; + snapshot: PersistedTeamLaunchSnapshot; + previousSnapshot?: PersistedTeamLaunchSnapshot | null; + metaMembers?: Awaited>; + }): Promise { + const candidates = this.collectOpenCodeSecondaryOverlayCandidates( + params.snapshot, + params.previousSnapshot ?? null + ); + if (candidates.length === 0) { + return params.snapshot; + } + + const laneIndex = await readOpenCodeRuntimeLaneIndex(getTeamsBasePath(), params.teamName).catch( + () => null + ); + let changed = false; + const nextMembers: Record = { + ...params.snapshot.members, + }; + const metaMembers = params.metaMembers ?? []; + + for (const memberName of candidates) { + const current = nextMembers[memberName]; + const previous = params.previousSnapshot?.members[memberName] ?? null; + const baseMember = current ?? previous; + if (!baseMember || !isPersistedOpenCodeSecondaryLaneMember(baseMember)) { + continue; + } + const laneId = baseMember.laneId?.trim(); + if (!laneId) { + continue; + } + const laneEntry = laneIndex?.lanes[laneId] ?? null; + const evidence = await readCommittedOpenCodeBootstrapSessionEvidence({ + teamsBasePath: getTeamsBasePath(), + teamName: params.teamName, + laneId, + }).catch((error: unknown) => ({ + state: 'invalid_store' as const, + committed: false, + sessions: [], + diagnostics: [ + `OpenCode committed bootstrap evidence read failed: ${getErrorMessage(error)}`, + ], + })); + const decision = await this.classifyOpenCodeSecondaryEvidenceOverlay({ + teamName: params.teamName, + memberName, + current: baseMember, + previous, + laneEntry, + metaMembers, + sessions: evidence.committed ? evidence.sessions : [], + diagnostics: evidence.diagnostics, + }); + if (decision.kind !== 'confirmed_bootstrap') { + continue; + } + const promoted = promoteOpenCodeSecondaryMemberFromCommittedBootstrapEvidence({ + current: baseMember, + previous, + session: decision.session, + now: nowIso(), + }); + if (!current || JSON.stringify(promoted) !== JSON.stringify(current)) { + nextMembers[memberName] = promoted; + changed = true; + } + } + + if (!changed) { + return params.snapshot; + } + + return createPersistedLaunchSnapshot({ + teamName: params.snapshot.teamName, + expectedMembers: params.snapshot.expectedMembers, + bootstrapExpectedMembers: params.snapshot.bootstrapExpectedMembers, + leadSessionId: params.snapshot.leadSessionId, + launchPhase: params.snapshot.launchPhase, + members: nextMembers, + updatedAt: nowIso(), + }); + } + + private hasCommittedOpenCodeSecondaryEvidenceOverlayDelta( + snapshot: PersistedTeamLaunchSnapshot | null, + previousSnapshot: PersistedTeamLaunchSnapshot | null + ): boolean { + if (!snapshot) { + return false; + } + return Object.entries(snapshot.members).some(([memberName, member]) => { + if (!member.diagnostics?.includes('opencode_bootstrap_evidence_committed')) { + return false; + } + const previous = previousSnapshot?.members[memberName]; + return ( + previous?.launchState !== member.launchState || + previous?.bootstrapConfirmed !== member.bootstrapConfirmed || + previous?.runtimeSessionId !== member.runtimeSessionId + ); + }); + } + + private collectOpenCodeSecondaryOverlayCandidates( + snapshot: PersistedTeamLaunchSnapshot, + previousSnapshot: PersistedTeamLaunchSnapshot | null + ): string[] { + const names = new Set(); + const allNames = new Set([ + ...Object.keys(snapshot.members), + ...Object.keys(previousSnapshot?.members ?? {}), + ]); + for (const memberName of allNames) { + const current = snapshot.members[memberName]; + const previous = previousSnapshot?.members[memberName]; + const candidate = current ?? previous; + if (!isPersistedOpenCodeSecondaryLaneMember(candidate)) { + continue; + } + if (!current || this.needsOpenCodeSecondaryEvidenceOverlay(current, previous ?? null)) { + names.add(memberName); + } + } + return [...names].sort((left, right) => left.localeCompare(right)); + } + + private needsOpenCodeSecondaryEvidenceOverlay( + current: PersistedTeamLaunchMemberState, + previous: PersistedTeamLaunchMemberState | null + ): boolean { + if (current.launchState === 'confirmed_alive' && current.bootstrapConfirmed) { + return false; + } + if ( + previous?.launchState === 'confirmed_alive' && + previous.bootstrapConfirmed && + current.launchState !== 'confirmed_alive' + ) { + return true; + } + if ( + current.launchState === 'starting' || + current.launchState === 'runtime_pending_bootstrap' || + current.launchState === 'runtime_pending_permission' + ) { + return true; + } + return ( + current.launchState === 'failed_to_start' && + hasStaleOpenCodeSecondaryLaunchDiagnostic(current) + ); + } + + private async classifyOpenCodeSecondaryEvidenceOverlay(params: { + teamName: string; + memberName: string; + current: PersistedTeamLaunchMemberState; + previous: PersistedTeamLaunchMemberState | null; + laneEntry: OpenCodeRuntimeLaneIndexEntry | null; + metaMembers: Awaited>; + sessions: OpenCodeCommittedBootstrapSessionRecord[]; + diagnostics: readonly string[]; + }): Promise< + | { kind: 'blocked' | 'none' | 'ambiguous' | 'conflict'; diagnostics: string[] } + | { kind: 'confirmed_bootstrap'; session: OpenCodeCommittedBootstrapSessionRecord } + > { + if (isOpenCodeOverlayMemberRemoved(params.metaMembers, params.memberName)) { + return { kind: 'blocked', diagnostics: ['opencode_overlay_member_removed'] }; + } + if (params.laneEntry?.state === 'stopped') { + return { kind: 'blocked', diagnostics: ['opencode_overlay_lane_stopped'] }; + } + if (hasRealOpenCodeLaunchDiagnostic(params.current)) { + return { kind: 'blocked', diagnostics: ['opencode_overlay_real_failure_preserved'] }; + } + if ( + params.current.launchState === 'failed_to_start' && + !hasStaleOpenCodeSecondaryLaunchDiagnostic(params.current) + ) { + return { kind: 'blocked', diagnostics: ['opencode_overlay_real_failure_preserved'] }; + } + if ( + params.laneEntry?.state === 'degraded' && + !hasStaleOpenCodeSecondaryLaunchDiagnostic(params.current) && + !hasStaleOpenCodeDiagnostics(params.laneEntry.diagnostics) + ) { + return { kind: 'blocked', diagnostics: ['opencode_overlay_degraded_lane_preserved'] }; + } + + const memberSessions = params.sessions.filter((session) => + namesMatchCaseInsensitive(session.memberName, params.memberName) + ); + if (memberSessions.length === 0) { + return { kind: 'none', diagnostics: [...params.diagnostics, 'opencode_overlay_no_session'] }; + } + + const expectedSessionId = + params.current.runtimeSessionId?.trim() || params.previous?.runtimeSessionId?.trim() || ''; + const selected = expectedSessionId + ? memberSessions.find((session) => session.id === expectedSessionId) + : memberSessions.length === 1 + ? memberSessions[0] + : null; + if (!selected) { + return { + kind: expectedSessionId ? 'conflict' : 'ambiguous', + diagnostics: [ + expectedSessionId + ? 'opencode_overlay_session_conflict' + : 'opencode_overlay_ambiguous_sessions', + ], + }; + } + + if (selected.runId) { + const tombstoneStore = createRuntimeRunTombstoneStore({ + filePath: getOpenCodeRuntimeRunTombstonesPath( + getTeamsBasePath(), + params.teamName, + params.current.laneId + ), + }); + const tombstone = await tombstoneStore + .find({ + teamName: params.teamName, + runId: selected.runId, + evidenceKind: 'bootstrap_checkin', + }) + .catch(() => null); + if (tombstone) { + return { kind: 'blocked', diagnostics: ['opencode_overlay_run_tombstoned'] }; + } + } + + return { kind: 'confirmed_bootstrap', session: selected }; + } + private async writeLaunchStateSnapshot( teamName: string, snapshot: PersistedTeamLaunchSnapshot - ): Promise { - await this.enqueueLaunchStateStoreOperation(teamName, () => + ): Promise { + return this.enqueueLaunchStateStoreOperation(teamName, () => this.writeLaunchStateSnapshotNow(teamName, snapshot) ); } @@ -17263,8 +17817,17 @@ export class TeamProvisioningService { private async writeLaunchStateSnapshotNow( teamName: string, snapshot: PersistedTeamLaunchSnapshot - ): Promise { - await this.launchStateStore.write(teamName, snapshot); + ): Promise { + const previousSnapshot = await this.launchStateStore.read(teamName).catch(() => null); + const metaMembers = await this.membersMetaStore.getMembers(teamName).catch(() => []); + const overlaidSnapshot = await this.applyOpenCodeSecondaryEvidenceOverlay({ + teamName, + snapshot, + previousSnapshot, + metaMembers, + }); + await this.launchStateStore.write(teamName, overlaidSnapshot); + return overlaidSnapshot; } private async enqueueLaunchStateStoreOperation( @@ -17613,6 +18176,9 @@ export class TeamProvisioningService { run: ProvisioningRun, lane: MixedSecondaryRuntimeLaneState ): Promise { + if (!this.isCurrentTrackedRun(run)) { + return; + } let snapshot: PersistedTeamLaunchSnapshot | null = null; if (run.isLaunch) { snapshot = await this.persistLaunchStateSnapshot(run, this.getMixedSecondaryLaunchPhase(run)); @@ -17620,9 +18186,6 @@ export class TeamProvisioningService { if (snapshot) { this.syncRunMemberSpawnStatusesFromSnapshot(run, snapshot); } - if (!this.isCurrentTrackedRun(run)) { - return; - } this.emitMemberSpawnChange(run, lane.member.name); } @@ -17644,6 +18207,11 @@ export class TeamProvisioningService { if (!claimsBootstrapConfirmed) { return params.result; } + await this.commitOpenCodeRuntimeAdapterLaunchSessionEvidence({ + teamName: params.teamName, + laneId: params.laneId, + result: params.result, + }); const storage = await inspectOpenCodeRuntimeLaneStorage({ teamsBasePath: getTeamsBasePath(), @@ -17948,10 +18516,10 @@ export class TeamProvisioningService { return null; } - await this.writeLaunchStateSnapshotNow(run.teamName, filteredSnapshot); + const writtenSnapshot = await this.writeLaunchStateSnapshotNow(run.teamName, filteredSnapshot); this.agentRuntimeSnapshotCache.delete(run.teamName); this.liveTeamAgentRuntimeMetadataCache.delete(run.teamName); - return filteredSnapshot; + return writtenSnapshot; } private async launchSingleMixedSecondaryLane( @@ -17959,6 +18527,23 @@ export class TeamProvisioningService { lane: MixedSecondaryRuntimeLaneState ): Promise { const requestedDiagnostics = [...lane.diagnostics]; + const shouldAbortLaunch = (): boolean => + run.cancelRequested || + run.processKilled || + this.stoppingSecondaryRuntimeTeams.has(run.teamName); + const finishCancelledLane = async (): Promise => { + await clearOpenCodeRuntimeLaneStorage({ + teamsBasePath: getTeamsBasePath(), + teamName: run.teamName, + laneId: lane.laneId, + }).catch(() => undefined); + this.deleteSecondaryRuntimeRun(run.teamName, lane.laneId); + lane.state = 'finished'; + }; + if (shouldAbortLaunch()) { + await finishCancelledLane(); + return; + } const adapter = this.getOpenCodeRuntimeAdapter(); if (!adapter) { const message = 'OpenCode runtime adapter is not registered for mixed team launch.'; @@ -17996,6 +18581,10 @@ export class TeamProvisioningService { teamName: run.teamName, laneId: lane.laneId, }); + if (shouldAbortLaunch()) { + await finishCancelledLane(); + return; + } await upsertOpenCodeRuntimeLaneIndexEntry({ teamsBasePath: getTeamsBasePath(), teamName: run.teamName, @@ -18003,6 +18592,10 @@ export class TeamProvisioningService { state: migration.degraded ? 'degraded' : 'active', diagnostics: migration.diagnostics, }); + if (shouldAbortLaunch()) { + await finishCancelledLane(); + return; + } lane.state = 'launching'; lane.runId = lane.runId ?? randomUUID(); @@ -18021,12 +18614,20 @@ export class TeamProvisioningService { const previousLaunchState = await this.launchStateStore.read(run.teamName); try { + if (shouldAbortLaunch()) { + await finishCancelledLane(); + return; + } await setOpenCodeRuntimeActiveRunManifest({ teamsBasePath: getTeamsBasePath(), teamName: run.teamName, laneId: lane.laneId, runId: lane.runId, }); + if (shouldAbortLaunch()) { + await finishCancelledLane(); + return; + } const rawResult = await adapter.launch({ runId: lane.runId, laneId: lane.laneId, @@ -18051,14 +18652,18 @@ export class TeamProvisioningService { ], previousLaunchState, }); + if (shouldAbortLaunch()) { + await finishCancelledLane(); + return; + } const result = await this.guardCommittedOpenCodeSecondaryLaneEvidence({ teamName: run.teamName, laneId: lane.laneId, memberName: lane.member.name, result: rawResult, }); - if (run.cancelRequested || run.processKilled) { - this.deleteSecondaryRuntimeRun(run.teamName, lane.laneId); + if (shouldAbortLaunch()) { + await finishCancelledLane(); return; } lane.result = result; @@ -18083,8 +18688,8 @@ export class TeamProvisioningService { this.deleteSecondaryRuntimeRun(run.teamName, lane.laneId); } } catch (error) { - if (run.cancelRequested || run.processKilled) { - this.deleteSecondaryRuntimeRun(run.teamName, lane.laneId); + if (shouldAbortLaunch()) { + await finishCancelledLane(); return; } const message = error instanceof Error ? error.message : String(error); @@ -18132,6 +18737,13 @@ export class TeamProvisioningService { ): Promise { const adapter = this.getOpenCodeRuntimeAdapter(); const previousLaunchState = await this.launchStateStore.read(run.teamName); + await upsertOpenCodeRuntimeLaneIndexEntry({ + teamsBasePath: getTeamsBasePath(), + teamName: run.teamName, + laneId: lane.laneId, + state: 'stopped', + diagnostics: [`OpenCode lane stop requested: ${reason}`], + }).catch(() => undefined); try { if (adapter && lane.runId) { @@ -18181,6 +18793,11 @@ export class TeamProvisioningService { const launch = async () => { try { if (run.cancelRequested || run.processKilled) { + await clearOpenCodeRuntimeLaneStorage({ + teamsBasePath: getTeamsBasePath(), + teamName: run.teamName, + laneId: lane.laneId, + }).catch(() => undefined); this.deleteSecondaryRuntimeRun(run.teamName, lane.laneId); lane.state = 'finished'; return; @@ -18188,6 +18805,11 @@ export class TeamProvisioningService { await this.launchSingleMixedSecondaryLane(run, lane); } catch (error) { if (run.cancelRequested || run.processKilled) { + await clearOpenCodeRuntimeLaneStorage({ + teamsBasePath: getTeamsBasePath(), + teamName: run.teamName, + laneId: lane.laneId, + }).catch(() => undefined); this.deleteSecondaryRuntimeRun(run.teamName, lane.laneId); return; } @@ -18526,8 +19148,7 @@ export class TeamProvisioningService { primaryStatuses, secondaryMembers, }); - await this.writeLaunchStateSnapshot(teamName, recoveredSnapshot); - return recoveredSnapshot; + return this.writeLaunchStateSnapshot(teamName, recoveredSnapshot); } private async tryRecoverActiveOpenCodeSecondaryLaneFromRuntime(params: { @@ -18752,28 +19373,58 @@ export class TeamProvisioningService { const filteredRecoveredMixedSnapshot = recoveredMixedSnapshot ? this.filterRemovedMembersFromLaunchSnapshot(recoveredMixedSnapshot, metaMembers) : null; + const overlaidRecoveredMixedSnapshot = filteredRecoveredMixedSnapshot + ? await this.applyOpenCodeSecondaryEvidenceOverlay({ + teamName, + snapshot: filteredRecoveredMixedSnapshot, + previousSnapshot: persisted, + metaMembers, + }) + : null; + const stableRecoveredMixedSnapshot = + overlaidRecoveredMixedSnapshot && + this.hasCommittedOpenCodeSecondaryEvidenceOverlayDelta( + overlaidRecoveredMixedSnapshot, + persisted + ) + ? await this.writeLaunchStateSnapshot(teamName, overlaidRecoveredMixedSnapshot) + : overlaidRecoveredMixedSnapshot; const filteredBootstrapSnapshot = bootstrapSnapshot ? this.filterRemovedMembersFromLaunchSnapshot(bootstrapSnapshot, metaMembers) : null; if ( - filteredRecoveredMixedSnapshot && + stableRecoveredMixedSnapshot && !this.needsBootstrapAcceptanceReconcile( - filteredRecoveredMixedSnapshot, + stableRecoveredMixedSnapshot, filteredBootstrapSnapshot ) && - !(await this.hasBootstrapTranscriptLaunchReconcileOutcome(filteredRecoveredMixedSnapshot)) + !(await this.hasBootstrapTranscriptLaunchReconcileOutcome(stableRecoveredMixedSnapshot)) ) { return { - snapshot: filteredRecoveredMixedSnapshot, - statuses: snapshotToMemberSpawnStatuses(filteredRecoveredMixedSnapshot), + snapshot: stableRecoveredMixedSnapshot, + statuses: snapshotToMemberSpawnStatuses(stableRecoveredMixedSnapshot), }; } - const filteredPersisted = - filteredRecoveredMixedSnapshot ?? + const filteredPersistedBase = + stableRecoveredMixedSnapshot ?? (persisted ? this.filterRemovedMembersFromLaunchSnapshot(persisted, metaMembers) : null); + const filteredPersisted = filteredPersistedBase + ? await this.applyOpenCodeSecondaryEvidenceOverlay({ + teamName, + snapshot: filteredPersistedBase, + previousSnapshot: persisted, + metaMembers, + }) + : null; + const shouldPersistCommittedEvidenceOverlay = + this.hasCommittedOpenCodeSecondaryEvidenceOverlayDelta(filteredPersisted, persisted); + const persistedWithCommittedEvidence = + filteredPersisted && shouldPersistCommittedEvidenceOverlay + ? await this.writeLaunchStateSnapshot(teamName, filteredPersisted) + : filteredPersisted; const preferredSnapshot = choosePreferredLaunchSnapshot( filteredBootstrapSnapshot, - filteredPersisted + persistedWithCommittedEvidence ); if (preferredSnapshot && preferredSnapshot === filteredBootstrapSnapshot) { return { @@ -18781,7 +19432,7 @@ export class TeamProvisioningService { statuses: snapshotToMemberSpawnStatuses(preferredSnapshot), }; } - if (!filteredPersisted) { + if (!persistedWithCommittedEvidence) { return { snapshot: null, statuses: {} }; } @@ -18814,20 +19465,26 @@ export class TeamProvisioningService { ); if ( - this.hasPrimaryOnlyLaneAwareLaunchMetadata(filteredPersisted) && - !this.hasLeadInboxLaunchReconcileHeartbeat(filteredPersisted, leadInboxMessages) && - !this.needsBootstrapAcceptanceReconcile(filteredPersisted, filteredBootstrapSnapshot) && - !(await this.hasBootstrapTranscriptLaunchReconcileOutcome(filteredPersisted)) + this.hasMixedLaunchMetadata(persistedWithCommittedEvidence) && + !this.hasLeadInboxLaunchReconcileHeartbeat( + persistedWithCommittedEvidence, + leadInboxMessages + ) && + !this.needsBootstrapAcceptanceReconcile( + persistedWithCommittedEvidence, + filteredBootstrapSnapshot + ) && + !(await this.hasBootstrapTranscriptLaunchReconcileOutcome(persistedWithCommittedEvidence)) ) { return { - snapshot: filteredPersisted, - statuses: snapshotToMemberSpawnStatuses(filteredPersisted), + snapshot: persistedWithCommittedEvidence, + statuses: snapshotToMemberSpawnStatuses(persistedWithCommittedEvidence), }; } const liveRuntimeByMember = await this.getLiveTeamAgentRuntimeMetadata(teamName); - const nextMembers = { ...filteredPersisted.members }; - const persistedMemberNames = this.getPersistedLaunchMemberNames(filteredPersisted); + const nextMembers = { ...persistedWithCommittedEvidence.members }; + const persistedMemberNames = this.getPersistedLaunchMemberNames(persistedWithCommittedEvidence); const now = nowIso(); for (const expected of persistedMemberNames) { const bootstrapMember = bootstrapSnapshot?.members[expected]; @@ -18961,8 +19618,8 @@ export class TeamProvisioningService { const reconciled = createPersistedLaunchSnapshot({ teamName, expectedMembers: persistedMemberNames, - leadSessionId: filteredPersisted.leadSessionId, - launchPhase: filteredPersisted.launchPhase, + leadSessionId: persistedWithCommittedEvidence.leadSessionId, + launchPhase: persistedWithCommittedEvidence.launchPhase, members: nextMembers, updatedAt: now, }); @@ -18975,10 +19632,10 @@ export class TeamProvisioningService { return { snapshot: null, statuses: {} }; } - await this.writeLaunchStateSnapshot(teamName, reconciled); + const writtenSnapshot = await this.writeLaunchStateSnapshot(teamName, reconciled); return { - snapshot: reconciled, - statuses: snapshotToMemberSpawnStatuses(reconciled), + snapshot: writtenSnapshot, + statuses: snapshotToMemberSpawnStatuses(writtenSnapshot), }; } @@ -22761,6 +23418,10 @@ export class TeamProvisioningService { const configuredTeamDir = path.join(getTeamsBasePath(), run.teamName); const defaultTeamDir = path.join(getAutoDetectedClaudeBasePath(), 'teams', run.teamName); const tasksDir = path.join(getTasksBasePath(), run.teamName); + const primaryProvisioningMembers = Array.isArray(run.effectiveMembers) + ? run.effectiveMembers + : request.members; + const primaryProvisioningMemberCount = primaryProvisioningMembers.length; const resolveTeamDir = async (): Promise => { const configPath = path.join(configuredTeamDir, 'config.json'); @@ -22815,10 +23476,11 @@ export class TeamProvisioningService { if (run.deterministicBootstrap) { const registeredNames = await this.getRegisteredTeamMemberNames(run.teamName); const registeredMembers = registeredNames - ? request.members.filter((member) => registeredNames.has(member.name)).length + ? primaryProvisioningMembers.filter((member) => registeredNames.has(member.name)) + .length : 0; - if (registeredMembers >= request.members.length) { + if (registeredMembers >= primaryProvisioningMemberCount) { run.fsPhase = 'all_files_found'; if (!run.provisioningComplete) { void this.handleProvisioningTurnComplete(run); @@ -22827,7 +23489,7 @@ export class TeamProvisioningService { } } - if (request.members.length === 0) { + if (primaryProvisioningMemberCount === 0) { if (run.deterministicBootstrap) { run.fsPhase = 'all_files_found'; if (!run.provisioningComplete) { @@ -22842,7 +23504,7 @@ export class TeamProvisioningService { const teamDir = (await resolveTeamDir()) ?? configuredTeamDir; const inboxDir = path.join(teamDir, 'inboxes'); const inboxCount = await countFiles(inboxDir, '.json'); - if (inboxCount >= request.members.length) { + if (inboxCount >= primaryProvisioningMemberCount) { run.fsPhase = 'waiting_tasks'; const progress = updateProgress( run, @@ -22854,7 +23516,7 @@ export class TeamProvisioningService { const progress = updateProgress( run, 'assembling', - `Prepared communication channels for ${inboxCount}/${request.members.length} members` + `Prepared communication channels for ${inboxCount}/${primaryProvisioningMemberCount} members` ); run.onProgress(progress); } diff --git a/src/main/services/team/opencode/store/OpenCodeRuntimeManifestEvidenceReader.ts b/src/main/services/team/opencode/store/OpenCodeRuntimeManifestEvidenceReader.ts index a0f5a6ea..4e0599e3 100644 --- a/src/main/services/team/opencode/store/OpenCodeRuntimeManifestEvidenceReader.ts +++ b/src/main/services/team/opencode/store/OpenCodeRuntimeManifestEvidenceReader.ts @@ -11,11 +11,13 @@ import { createRuntimeStoreManifestStore, OPENCODE_RUNTIME_STORE_DESCRIPTORS, OPENCODE_RUNTIME_STORE_MANIFEST_SCHEMA_VERSION, + RuntimeStoreFileInspector, validateRuntimeStoreManifest, } from './RuntimeStoreManifest'; import type { RuntimeStoreManifestEvidence } from '../bridge/OpenCodeBridgeCommandContract'; import type { RuntimeStoreManifestReader } from '../bridge/OpenCodeStateChangingBridgeCommandService'; +import type { RuntimeStoreManifestEntryState } from './RuntimeStoreManifest'; const logger = createLogger('OpenCodeRuntimeManifestEvidenceReader'); @@ -56,6 +58,23 @@ export interface OpenCodeRuntimeLaneIndex { lanes: Record; } +export interface OpenCodeCommittedBootstrapSessionRecord { + id: string; + teamName: string; + memberName: string; + laneId: string; + runId: string | null; + observedAt: string | null; + source: 'runtime_bootstrap_checkin'; +} + +export interface OpenCodeCommittedBootstrapSessionEvidence { + state: RuntimeStoreManifestEntryState | 'invalid_store' | 'descriptor_missing'; + committed: boolean; + sessions: OpenCodeCommittedBootstrapSessionRecord[]; + diagnostics: string[]; +} + function createEmptyOpenCodeRuntimeLaneIndex( updatedAt = new Date().toISOString() ): OpenCodeRuntimeLaneIndex { @@ -233,6 +252,82 @@ async function fileExists(filePath: string): Promise { } } +async function readOpenCodeBootstrapSessionStore( + filePath: string, + expected: { + teamName: string; + laneId: string; + } +): Promise { + const raw = await readFile(filePath, 'utf8'); + const parsed = JSON.parse(raw) as unknown; + const record = + parsed && typeof parsed === 'object' && !Array.isArray(parsed) + ? (parsed as Record) + : null; + const data = + record && Object.prototype.hasOwnProperty.call(record, 'data') ? record.data : record; + const sessions = + data && typeof data === 'object' && !Array.isArray(data) + ? (data as Record).sessions + : null; + + if (!Array.isArray(sessions)) { + return []; + } + + return sessions.flatMap((session): OpenCodeCommittedBootstrapSessionRecord[] => { + const normalized = normalizeOpenCodeBootstrapSessionRecord(session); + if (!normalized) { + return []; + } + if (normalized.teamName !== expected.teamName || normalized.laneId !== expected.laneId) { + return []; + } + return [normalized]; + }); +} + +function normalizeOpenCodeBootstrapSessionRecord( + value: unknown +): OpenCodeCommittedBootstrapSessionRecord | null { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + return null; + } + const record = value as Record; + const id = normalizeNonEmptyStoreString(record.id); + const teamName = normalizeNonEmptyStoreString(record.teamName); + const memberName = normalizeNonEmptyStoreString(record.memberName); + const laneId = normalizeNonEmptyStoreString(record.laneId); + const source = normalizeNonEmptyStoreString(record.source); + if (!id || !teamName || !memberName || !laneId || source !== 'runtime_bootstrap_checkin') { + return null; + } + const observedAt = normalizeOptionalStoreIso(record.observedAt); + return { + id, + teamName, + memberName, + laneId, + runId: normalizeNonEmptyStoreString(record.runId), + observedAt, + source: 'runtime_bootstrap_checkin', + }; +} + +function normalizeNonEmptyStoreString(value: unknown): string | null { + return typeof value === 'string' && value.trim().length > 0 ? value.trim() : null; +} + +function normalizeOptionalStoreIso(value: unknown): string | null { + const text = normalizeNonEmptyStoreString(value); + if (!text) { + return null; + } + const parsed = Date.parse(text); + return Number.isFinite(parsed) ? new Date(parsed).toISOString() : text; +} + async function resolveOpenCodeRuntimeManifestReadPath( teamsBasePath: string, teamName: string, @@ -390,6 +485,87 @@ export function getOpenCodeLaneScopedRuntimeFilePath(params: { ); } +export async function readCommittedOpenCodeBootstrapSessionEvidence(params: { + teamsBasePath: string; + teamName: string; + laneId: string; +}): Promise { + const descriptor = OPENCODE_RUNTIME_STORE_DESCRIPTORS.find( + (candidate) => candidate.schemaName === 'opencode.sessionStore' + ); + if (!descriptor) { + return { + state: 'descriptor_missing', + committed: false, + sessions: [], + diagnostics: ['OpenCode session store descriptor is not registered.'], + }; + } + + const runtimeDirectory = getOpenCodeTeamRuntimeLaneDirectory( + params.teamsBasePath, + params.teamName, + params.laneId + ); + const manifestPath = getOpenCodeRuntimeManifestPath( + params.teamsBasePath, + params.teamName, + params.laneId + ); + const manifestStore = createRuntimeStoreManifestStore({ + filePath: manifestPath, + teamName: params.teamName, + }); + const manifest = await manifestStore.read().catch(() => null); + if (!manifest) { + return { + state: 'invalid_store', + committed: false, + sessions: [], + diagnostics: ['OpenCode runtime manifest could not be read.'], + }; + } + + const inspection = await new RuntimeStoreFileInspector(runtimeDirectory) + .inspect({ descriptor, manifest }) + .catch((error: unknown) => ({ + state: 'invalid_store' as const, + message: `OpenCode session store inspection failed: ${ + error instanceof Error ? error.message : String(error) + }`, + })); + const diagnostics = inspection.message ? [inspection.message] : []; + if (inspection.state !== 'healthy') { + return { + state: inspection.state, + committed: false, + sessions: [], + diagnostics, + }; + } + + const sessionStorePath = path.join(runtimeDirectory, descriptor.relativePath); + const sessions = await readOpenCodeBootstrapSessionStore(sessionStorePath, params).catch( + (error: unknown) => { + diagnostics.push( + `OpenCode session store could not be parsed: ${ + error instanceof Error ? error.message : String(error) + }` + ); + return []; + } + ); + if (sessions.length === 0) { + diagnostics.push('OpenCode session store has no committed bootstrap check-in sessions.'); + } + return { + state: 'healthy', + committed: true, + sessions, + diagnostics, + }; +} + export async function readOpenCodeRuntimeLaneIndex( teamsBasePath: string, teamName: string diff --git a/src/renderer/components/chat/viewers/MarkdownViewer.tsx b/src/renderer/components/chat/viewers/MarkdownViewer.tsx index 8c2a7082..a91caa9e 100644 --- a/src/renderer/components/chat/viewers/MarkdownViewer.tsx +++ b/src/renderer/components/chat/viewers/MarkdownViewer.tsx @@ -85,11 +85,80 @@ const EMPTY_TEAM_COLOR_MAP = new Map(); const NOOP_TEAM_CLICK = (): void => undefined; type ViewerMarkdownMode = 'default' | 'compact-preview'; +type HastElementLike = { + tagName?: string; + value?: string; + children?: unknown[]; +}; // ============================================================================= // Helpers // ============================================================================= +function isHastElementLike(value: unknown): value is HastElementLike { + return typeof value === 'object' && value !== null; +} + +function getHastChildren(value: unknown): unknown[] { + return isHastElementLike(value) && Array.isArray(value.children) ? value.children : []; +} + +function getHastText(value: unknown): string { + if (typeof value === 'string' || typeof value === 'number') { + return String(value); + } + if (!isHastElementLike(value)) { + return ''; + } + if (typeof value.value === 'string') { + return value.value; + } + return getHastChildren(value).map(getHastText).join(' ').replace(/\s+/g, ' ').trim(); +} + +function collectHastElementsByTag(value: unknown, tagName: string): HastElementLike[] { + const result: HastElementLike[] = []; + const visit = (node: unknown): void => { + if (!isHastElementLike(node)) return; + if (node.tagName === tagName) { + result.push(node); + } + for (const child of getHastChildren(node)) { + visit(child); + } + }; + visit(value); + return result; +} + +function getDirectCellElements(row: HastElementLike): HastElementLike[] { + return getHastChildren(row).filter( + (child): child is HastElementLike => + isHastElementLike(child) && (child.tagName === 'th' || child.tagName === 'td') + ); +} + +function buildCompactTableSummary(node: unknown, fallbackChildren: React.ReactNode): string { + const rows = collectHastElementsByTag(node, 'tr'); + const headerRow = + rows.find((row) => getDirectCellElements(row).some((cell) => cell.tagName === 'th')) ?? + rows[0] ?? + null; + const headerCells = headerRow ? getDirectCellElements(headerRow) : []; + const headers = headerCells.map(getHastText).filter(Boolean); + const bodyRowCount = rows.filter((row) => row !== headerRow).length; + const fallbackText = extractTextFromReactNode(fallbackChildren).replace(/\s+/g, ' ').trim(); + const previewSource = headers.length > 0 ? headers.join(' | ') : fallbackText; + const preview = + previewSource.length > 72 ? `${previewSource.slice(0, 69).trimEnd()}...` : previewSource; + const rowLabel = bodyRowCount === 1 ? '1 row' : `${bodyRowCount} rows`; + + if (preview) { + return `Table: ${preview} - ${rowLabel}`; + } + return `Table - ${rowLabel}`; +} + /** * Custom URL transform that preserves task://, mention://, and team:// protocols. * react-markdown v10 strips non-standard protocols by default. @@ -354,6 +423,18 @@ function createViewerMarkdownComponents( ); + const renderCompactTableSummary = ( + node: unknown, + children: React.ReactNode + ): React.ReactElement => { + const summary = buildCompactTableSummary(node, children); + return ( + + {summary}{' '} + + ); + }; + return { // Headings h1: ({ children }) => @@ -705,9 +786,9 @@ function createViewerMarkdownComponents( ), // Tables - table: ({ children }) => + table: ({ children, node }) => isCompactPreview ? ( - {children} + renderCompactTableSummary(node, children) ) : (
isCompactPreview ? ( - {children} + {children} ) : ( {children} ), th: ({ children }) => isCompactPreview ? ( - renderCompactInline(children, 'font-semibold', { color: PROSE_HEADING }) + ) : ( ) : (
+ {hl(children)} + isCompactPreview ? ( - renderCompactInline(children, '', { color: PROSE_BODY }) + + {hl(children)} + 0 ? `, +${remainingCount} more` : ''}`; +} + function getMemberNamesFromSpawnSources(params: { memberSpawnStatuses: MemberSpawnStatusCollection; memberSpawnSnapshotStatuses?: MemberSpawnStatusesSnapshot['statuses']; @@ -223,13 +233,71 @@ function getPendingDiagnosticNameGroups(params: { return groups; } +function getPendingSpawnNames(params: { + memberSpawnStatuses: MemberSpawnStatusCollection; + memberSpawnSnapshotStatuses?: MemberSpawnStatusesSnapshot['statuses']; + memberSpawnSnapshotUpdatedAt?: string; +}): string[] { + return getMemberNamesFromSpawnSources(params).filter((name) => { + const liveEntry = + params.memberSpawnStatuses instanceof Map + ? params.memberSpawnStatuses.get(name) + : params.memberSpawnStatuses?.[name]; + const snapshotEntry = params.memberSpawnSnapshotStatuses?.[name]; + const entry = getPreferredSpawnEntry({ + liveEntry, + snapshotEntry, + snapshotUpdatedAt: params.memberSpawnSnapshotUpdatedAt, + }); + return ( + entry != null && + entry.launchState !== 'confirmed_alive' && + !isFailedSpawnEntry(entry) && + !isSkippedSpawnEntry(entry) + ); + }); +} + +function isOpenCodeSecondaryMember(member: ProvisioningMemberLike | undefined): boolean { + if (!member || member.removedAt != null || member.providerId !== 'opencode') { + return false; + } + return ( + member.laneKind === 'secondary' || + member.laneOwnerProviderId === 'opencode' || + member.laneId?.startsWith('secondary:opencode:') === true + ); +} + +function buildOpenCodeSecondaryWaitPhrase(params: { + members: readonly ProvisioningMemberLike[]; + memberSpawnStatuses: MemberSpawnStatusCollection; + memberSpawnSnapshotStatuses?: MemberSpawnStatusesSnapshot['statuses']; + memberSpawnSnapshotUpdatedAt?: string; +}): string | null { + const pendingNames = getPendingSpawnNames({ + memberSpawnStatuses: params.memberSpawnStatuses, + memberSpawnSnapshotStatuses: params.memberSpawnSnapshotStatuses, + memberSpawnSnapshotUpdatedAt: params.memberSpawnSnapshotUpdatedAt, + }); + if (pendingNames.length === 0) { + return null; + } + + const memberByName = new Map(params.members.map((member) => [member.name, member])); + const pendingOnlyOpenCodeSecondary = pendingNames.every((name) => + isOpenCodeSecondaryMember(memberByName.get(name)) + ); + return pendingOnlyOpenCodeSecondary + ? `Waiting for OpenCode: ${formatMemberNameList(pendingNames)}` + : null; +} + function formatNamedPendingDiagnostic(label: string, names: readonly string[]): string | null { if (names.length === 0) { return null; } - const listedNames = names.slice(0, MAX_PENDING_DIAGNOSTIC_NAMES).join(', '); - const remainingCount = names.length - Math.min(names.length, MAX_PENDING_DIAGNOSTIC_NAMES); - return `${label}: ${listedNames}${remainingCount > 0 ? `, +${remainingCount} more` : ''}`; + return `${label}: ${formatMemberNameList(names)}`; } function formatCountPendingDiagnostic(count: number | undefined, label: string): string | null { @@ -578,6 +646,12 @@ export function buildTeamProvisioningPresentation({ memberSpawnSnapshotStatuses: memberSpawnSnapshot?.statuses, memberSpawnSnapshotUpdatedAt: memberSpawnSnapshot?.updatedAt, }); + const openCodeSecondaryWaitPhrase = buildOpenCodeSecondaryWaitPhrase({ + members, + memberSpawnStatuses, + memberSpawnSnapshotStatuses: memberSpawnSnapshot?.statuses, + memberSpawnSnapshotUpdatedAt: memberSpawnSnapshot?.updatedAt, + }); const { allTeammatesConfirmedAlive, hasMembersStillJoining, remainingJoinCount } = getLaunchJoinState({ @@ -637,13 +711,14 @@ export function buildTeamProvisioningPresentation({ permissionBlockedCount === remainingJoinCount; const pendingDetailPhrase = pendingMembersAwaitApproval ? buildAwaitingPermissionPhrase(permissionBlockedCount) - : buildPendingDiagnosticPhrase({ + : (openCodeSecondaryWaitPhrase ?? + buildPendingDiagnosticPhrase({ summary: memberSpawnSnapshot?.summary, memberSpawnStatuses, memberSpawnSnapshotStatuses: memberSpawnSnapshot?.statuses, memberSpawnSnapshotUpdatedAt: memberSpawnSnapshot?.updatedAt, fallbackJoiningPhrase: joiningPhrase, - }); + })); const readyCompactDetail = failedSpawnCount > 0 ? (failedSpawnCompactDetail ?? @@ -684,7 +759,9 @@ export function buildTeamProvisioningPresentation({ ? 'Team launched - lead online' : allTeammatesConfirmedAlive ? `Team launched - all ${expectedTeammateCount} teammates joined` - : 'Finishing launch'; + : openCodeSecondaryWaitPhrase + ? 'Core team ready' + : 'Finishing launch'; return { progress, @@ -721,7 +798,9 @@ export function buildTeamProvisioningPresentation({ : skippedSpawnCount > 0 ? 'Launch continued with skipped teammates' : hasMembersStillJoining - ? 'Finishing launch' + ? openCodeSecondaryWaitPhrase + ? 'Core team ready' + : 'Finishing launch' : 'Team launched', compactDetail: readyCompactDetail, compactTone: @@ -750,13 +829,14 @@ export function buildTeamProvisioningPresentation({ permissionBlockedCount > 0 && permissionBlockedCount === remainingJoinCount ? buildAwaitingPermissionPhrase(permissionBlockedCount) - : buildPendingDiagnosticPhrase({ + : (openCodeSecondaryWaitPhrase ?? + buildPendingDiagnosticPhrase({ summary: memberSpawnSnapshot?.summary, memberSpawnStatuses, memberSpawnSnapshotStatuses: memberSpawnSnapshot?.statuses, memberSpawnSnapshotUpdatedAt: memberSpawnSnapshot?.updatedAt, fallbackJoiningPhrase: activeJoiningPhrase, - }); + })); return { progress, isActive: true, @@ -773,22 +853,24 @@ export function buildTeamProvisioningPresentation({ allTeammatesConfirmedAlive, hasMembersStillJoining, remainingJoinCount, - panelTitle: 'Launching team', + panelTitle: openCodeSecondaryWaitPhrase ? 'Core team ready' : 'Launching team', panelMessage: failedSpawnCount > 0 ? (failedSpawnPanelMessage ?? genericFailedSpawnPanelMessage ?? progress.message) : skippedSpawnCount > 0 ? (skippedSpawnPanelMessage ?? `${skippedSpawnCount}/${Math.max(expectedTeammateCount, skippedSpawnCount)} teammates skipped for this launch`) - : hasMembersStillJoining && - permissionBlockedCount > 0 && - permissionBlockedCount === remainingJoinCount - ? activePendingDetailPhrase - : progress.message, + : openCodeSecondaryWaitPhrase + ? openCodeSecondaryWaitPhrase + : hasMembersStillJoining && + permissionBlockedCount > 0 && + permissionBlockedCount === remainingJoinCount + ? activePendingDetailPhrase + : progress.message, panelMessageSeverity: failedSpawnCount > 0 || skippedSpawnCount > 0 ? 'warning' : progress.messageSeverity, defaultLiveOutputOpen: false, - compactTitle: 'Launching team', + compactTitle: openCodeSecondaryWaitPhrase ? 'Core team ready' : 'Launching team', compactDetail: failedSpawnCount > 0 ? (failedSpawnCompactDetail ?? @@ -796,13 +878,15 @@ export function buildTeamProvisioningPresentation({ : skippedSpawnCount > 0 ? (skippedSpawnCompactDetail ?? `${skippedSpawnCount} teammate${skippedSpawnCount === 1 ? '' : 's'} skipped`) - : hasMembersStillJoining && failedSpawnCount === 0 && permissionBlockedCount > 0 - ? permissionBlockedCount === remainingJoinCount - ? buildAwaitingPermissionPhrase(permissionBlockedCount) - : `${heartbeatConfirmedCount}/${expectedTeammateCount} teammates confirmed` - : expectedTeammateCount > 0 && progressStepIndex >= 2 - ? `${heartbeatConfirmedCount}/${expectedTeammateCount} teammates confirmed` - : progress.message, + : openCodeSecondaryWaitPhrase + ? openCodeSecondaryWaitPhrase + : hasMembersStillJoining && failedSpawnCount === 0 && permissionBlockedCount > 0 + ? permissionBlockedCount === remainingJoinCount + ? buildAwaitingPermissionPhrase(permissionBlockedCount) + : `${heartbeatConfirmedCount}/${expectedTeammateCount} teammates confirmed` + : expectedTeammateCount > 0 && progressStepIndex >= 2 + ? `${heartbeatConfirmedCount}/${expectedTeammateCount} teammates confirmed` + : progress.message, compactTone: failedSpawnCount > 0 || skippedSpawnCount > 0 ? 'warning' : 'default', }; } diff --git a/test/main/services/team/OpenCodeRuntimeManifestEvidenceReader.test.ts b/test/main/services/team/OpenCodeRuntimeManifestEvidenceReader.test.ts index c8473d60..6c48631f 100644 --- a/test/main/services/team/OpenCodeRuntimeManifestEvidenceReader.test.ts +++ b/test/main/services/team/OpenCodeRuntimeManifestEvidenceReader.test.ts @@ -12,12 +12,19 @@ import { getOpenCodeTeamRuntimeDirectory, inspectOpenCodeRuntimeLaneStorage, migrateLegacyOpenCodeRuntimeState, + readCommittedOpenCodeBootstrapSessionEvidence, readOpenCodeRuntimeLaneIndex, recoverStaleOpenCodeRuntimeLaneIndexEntry, setOpenCodeRuntimeActiveRunManifest, upsertOpenCodeRuntimeLaneIndexEntry, } from '../../../../src/main/services/team/opencode/store/OpenCodeRuntimeManifestEvidenceReader'; -import { createDefaultRuntimeStoreManifest } from '../../../../src/main/services/team/opencode/store/RuntimeStoreManifest'; +import { + createRuntimeStoreManifestStore, + createRuntimeStoreReceiptStore, + OPENCODE_RUNTIME_STORE_DESCRIPTORS, + RuntimeStoreBatchWriter, + createDefaultRuntimeStoreManifest, +} from '../../../../src/main/services/team/opencode/store/RuntimeStoreManifest'; describe('OpenCodeRuntimeManifestEvidenceReader migration', () => { let tempDir: string; @@ -32,6 +39,127 @@ describe('OpenCodeRuntimeManifestEvidenceReader migration', () => { await fs.rm(tempDir, { recursive: true, force: true }); }); + async function writeCommittedSessionStore(input: { + teamName: string; + laneId: string; + sessions: unknown[]; + }) { + const descriptor = OPENCODE_RUNTIME_STORE_DESCRIPTORS.find( + (candidate) => candidate.schemaName === 'opencode.sessionStore' + ); + if (!descriptor) throw new Error('session descriptor missing'); + const manifestPath = getOpenCodeRuntimeManifestPath(tempDir, input.teamName, input.laneId); + const runtimeDirectory = path.dirname(manifestPath); + await fs.mkdir(runtimeDirectory, { recursive: true }); + const writer = new RuntimeStoreBatchWriter( + runtimeDirectory, + createRuntimeStoreManifestStore({ filePath: manifestPath, teamName: input.teamName }), + createRuntimeStoreReceiptStore({ + filePath: path.join(runtimeDirectory, 'opencode-runtime-receipts.json'), + }), + { + clock: () => now, + batchIdFactory: () => 'batch-1', + receiptIdFactory: () => 'receipt-1', + } + ); + await writer.writeBatch({ + teamName: input.teamName, + runId: 'runtime-run-1', + capabilitySnapshotId: null, + behaviorFingerprint: null, + reason: 'launch_checkpoint', + writes: [{ descriptor, data: { sessions: input.sessions } }], + }); + } + + it('reads only committed OpenCode bootstrap check-in session evidence', async () => { + const teamName = 'team-committed-session'; + const laneId = 'secondary:opencode:tom'; + await writeCommittedSessionStore({ + teamName, + laneId, + sessions: [ + { + id: 'ses-tom', + teamName, + memberName: 'tom', + runId: 'runtime-run-1', + laneId, + providerId: 'opencode', + observedAt: '2026-04-22T10:00:00.000Z', + source: 'runtime_bootstrap_checkin', + }, + { + id: 'ses-ignored', + teamName, + memberName: 'tom', + runId: 'runtime-run-1', + laneId, + source: 'member_briefing', + }, + ], + }); + + await expect( + readCommittedOpenCodeBootstrapSessionEvidence({ teamsBasePath: tempDir, teamName, laneId }) + ).resolves.toMatchObject({ + state: 'healthy', + committed: true, + sessions: [ + { + id: 'ses-tom', + teamName, + memberName: 'tom', + laneId, + runId: 'runtime-run-1', + source: 'runtime_bootstrap_checkin', + }, + ], + }); + }); + + it('does not treat an uncommitted session file as OpenCode bootstrap evidence', async () => { + const teamName = 'team-uncommitted-session'; + const laneId = 'secondary:opencode:tom'; + const sessionPath = getOpenCodeLaneScopedRuntimeFilePath({ + teamsBasePath: tempDir, + teamName, + laneId, + fileName: 'opencode-sessions.json', + }); + await fs.mkdir(path.dirname(sessionPath), { recursive: true }); + await fs.writeFile( + sessionPath, + JSON.stringify({ + schemaVersion: 1, + updatedAt: '2026-04-22T10:00:00.000Z', + data: { + sessions: [ + { + id: 'ses-tom', + teamName, + memberName: 'tom', + laneId, + source: 'runtime_bootstrap_checkin', + }, + ], + }, + }), + 'utf8' + ); + + const evidence = await readCommittedOpenCodeBootstrapSessionEvidence({ + teamsBasePath: tempDir, + teamName, + laneId, + }); + + expect(evidence.committed).toBe(false); + expect(evidence.state).toBe('uncommitted_write'); + expect(evidence.sessions).toEqual([]); + }); + it('migrates legacy team-scoped OpenCode runtime files into the addressed lane', async () => { const teamName = 'team-alpha'; const laneId = 'secondary:opencode:alice'; diff --git a/test/main/services/team/TeamAgentLaunchMatrix.safe-e2e.test.ts b/test/main/services/team/TeamAgentLaunchMatrix.safe-e2e.test.ts index 150f4b54..c3b16348 100644 --- a/test/main/services/team/TeamAgentLaunchMatrix.safe-e2e.test.ts +++ b/test/main/services/team/TeamAgentLaunchMatrix.safe-e2e.test.ts @@ -16958,6 +16958,7 @@ class FakeOpenCodeRuntimeAdapter implements TeamLaunchRuntimeAdapter { hardFailure: failed, hardFailureReason: failed ? 'fake_open_code_launch_failure' : undefined, pendingPermissionRequestIds: permissionPending ? [`perm-${member.name}`] : undefined, + sessionId: failed ? undefined : `session-${member.name}`, runtimePid: failed ? undefined : 10_000 + index, livenessKind, pidSource: failed ? undefined : 'opencode_bridge', diff --git a/test/main/services/team/TeamProvisioningService.test.ts b/test/main/services/team/TeamProvisioningService.test.ts index b9597222..b5587acf 100644 --- a/test/main/services/team/TeamProvisioningService.test.ts +++ b/test/main/services/team/TeamProvisioningService.test.ts @@ -139,7 +139,13 @@ import { setOpenCodeRuntimeActiveRunManifest, upsertOpenCodeRuntimeLaneIndexEntry, } from '@main/services/team/opencode/store/OpenCodeRuntimeManifestEvidenceReader'; -import { createDefaultRuntimeStoreManifest } from '@main/services/team/opencode/store/RuntimeStoreManifest'; +import { + createDefaultRuntimeStoreManifest, + createRuntimeStoreManifestStore, + createRuntimeStoreReceiptStore, + OPENCODE_RUNTIME_STORE_DESCRIPTORS, + RuntimeStoreBatchWriter, +} from '@main/services/team/opencode/store/RuntimeStoreManifest'; import { ClaudeBinaryResolver } from '@main/services/team/ClaudeBinaryResolver'; import { TeamRuntimeAdapterRegistry } from '@main/services/team/runtime/TeamRuntimeAdapter'; import { spawnCli } from '@main/utils/childProcess'; @@ -344,6 +350,41 @@ function writeMembersMeta( ); } +async function writeCommittedOpenCodeSessionStore(input: { + teamName: string; + laneId: string; + runId: string; + sessions: unknown[]; +}): Promise { + const descriptor = OPENCODE_RUNTIME_STORE_DESCRIPTORS.find( + (candidate) => candidate.schemaName === 'opencode.sessionStore' + ); + if (!descriptor) throw new Error('session descriptor missing'); + const manifestPath = getOpenCodeRuntimeManifestPath(tempTeamsBase, input.teamName, input.laneId); + const runtimeDirectory = path.dirname(manifestPath); + await fsPromises.mkdir(runtimeDirectory, { recursive: true }); + const writer = new RuntimeStoreBatchWriter( + runtimeDirectory, + createRuntimeStoreManifestStore({ filePath: manifestPath, teamName: input.teamName }), + createRuntimeStoreReceiptStore({ + filePath: path.join(runtimeDirectory, 'opencode-runtime-receipts.json'), + }), + { + clock: () => new Date('2026-04-22T12:00:00.000Z'), + batchIdFactory: () => `batch-${input.runId}`, + receiptIdFactory: () => `receipt-${input.runId}`, + } + ); + await writer.writeBatch({ + teamName: input.teamName, + runId: input.runId, + capabilitySnapshotId: null, + behaviorFingerprint: null, + reason: 'launch_checkpoint', + writes: [{ descriptor, data: { sessions: input.sessions } }], + }); +} + function createMemberSpawnStatusEntry( overrides: Record = {} ): Record { @@ -3116,26 +3157,50 @@ describe('TeamProvisioningService', () => { it('launches the OpenCode secondary lane with side-lane provider and member runtime identity', async () => { const svc = new TeamProvisioningService(); - const adapterLaunch = vi.fn(async (input: Record) => ({ - runId: String(input.runId), - teamName: String(input.teamName), - launchPhase: 'finished', - teamLaunchState: 'clean_success', - members: { - bob: { - memberName: 'bob', - providerId: 'opencode', - launchState: 'confirmed_alive', - agentToolAccepted: true, - runtimeAlive: true, - bootstrapConfirmed: true, - hardFailure: false, - diagnostics: [], + const adapterLaunch = vi.fn(async (input: Record) => { + const teamName = String(input.teamName); + const laneId = String(input.laneId); + const runId = String(input.runId); + const manifestPath = getOpenCodeRuntimeManifestPath(tempTeamsBase, teamName, laneId); + await fsPromises.mkdir(path.dirname(manifestPath), { recursive: true }); + await fsPromises.writeFile( + manifestPath, + `${JSON.stringify( + { + ...createDefaultRuntimeStoreManifest(teamName, '2026-04-22T12:00:00.000Z'), + activeRunId: runId, + }, + null, + 2 + )}\n`, + 'utf8' + ); + await fsPromises.writeFile( + path.join(path.dirname(manifestPath), 'opencode-sessions.json'), + `${JSON.stringify({ sessions: [{ id: 'oc-session-bob' }] })}\n`, + 'utf8' + ); + return { + runId, + teamName, + launchPhase: 'finished', + teamLaunchState: 'clean_success', + members: { + bob: { + memberName: 'bob', + providerId: 'opencode', + launchState: 'confirmed_alive', + agentToolAccepted: true, + runtimeAlive: true, + bootstrapConfirmed: true, + hardFailure: false, + diagnostics: [], + }, }, - }, - warnings: [], - diagnostics: [], - })); + warnings: [], + diagnostics: [], + }; + }); const registry = new TeamRuntimeAdapterRegistry([ { @@ -6589,6 +6654,90 @@ describe('TeamProvisioningService', () => { ).resolves.toBeUndefined(); }); + it('commits lane-scoped OpenCode session evidence when bootstrap check-in is accepted', async () => { + const svc = new TeamProvisioningService(); + const teamName = 'mixed-team'; + const laneId = 'secondary:opencode:bob'; + const runId = 'opencode-run-1'; + const teamDir = path.join(tempTeamsBase, teamName); + await fsPromises.mkdir(teamDir, { recursive: true }); + await fsPromises.writeFile( + path.join(teamDir, 'config.json'), + `${JSON.stringify({ + name: teamName, + projectPath: '/tmp/mixed-team', + members: [ + { name: 'team-lead', providerId: 'codex' }, + { name: 'bob', providerId: 'opencode' }, + ], + })}\n`, + 'utf8' + ); + + (svc as any).aliveRunByTeam.set(teamName, 'lead-run'); + (svc as any).runs.set('lead-run', { + runId: 'lead-run', + teamName, + request: { + providerId: 'codex', + }, + }); + (svc as any).setSecondaryRuntimeRun({ + teamName, + runId, + providerId: 'opencode', + laneId, + memberName: 'bob', + cwd: '/tmp/mixed-team', + }); + + await expect( + svc.recordOpenCodeRuntimeBootstrapCheckin({ + teamName, + runId, + memberName: 'bob', + runtimeSessionId: 'session-bob', + observedAt: '2026-04-22T12:05:00.000Z', + }) + ).resolves.toMatchObject({ + ok: true, + state: 'accepted', + runtimeSessionId: 'session-bob', + }); + + const manifestPath = getOpenCodeRuntimeManifestPath(tempTeamsBase, teamName, laneId); + const manifest = JSON.parse(await fsPromises.readFile(manifestPath, 'utf8')) as { + data: { entries: Array<{ schemaName: string; relativePath: string; runId: string }> }; + }; + expect(manifest.data.entries).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + schemaName: 'opencode.sessionStore', + relativePath: 'opencode-sessions.json', + runId, + }), + ]) + ); + const sessionStore = JSON.parse( + await fsPromises.readFile( + path.join(path.dirname(manifestPath), 'opencode-sessions.json'), + 'utf8' + ) + ) as { + data: { + sessions: Array<{ id: string; memberName: string; runId: string; laneId: string }>; + }; + }; + expect(sessionStore.data.sessions).toEqual([ + expect.objectContaining({ + id: 'session-bob', + memberName: 'bob', + runId, + laneId, + }), + ]); + }); + it('uses the secondary lane run id for OpenCode runtime delivery journal acceptance', async () => { const svc = new TeamProvisioningService(); const delivered = new Map< @@ -8177,6 +8326,67 @@ describe('TeamProvisioningService', () => { }); describe('safe app launch matrix', () => { + it('does not wait for OpenCode secondary inboxes before completing primary filesystem readiness', async () => { + const teamName = 'mixed-secondary-fs-readiness'; + const teamDir = path.join(tempTeamsBase, teamName); + fs.mkdirSync(path.join(teamDir, 'inboxes'), { recursive: true }); + fs.writeFileSync( + path.join(teamDir, 'config.json'), + `${JSON.stringify( + { + name: teamName, + members: [ + { name: 'team-lead', agentType: 'team-lead', providerId: 'codex' }, + { name: 'alice', providerId: 'codex' }, + ], + }, + null, + 2 + )}\n`, + 'utf8' + ); + fs.writeFileSync(path.join(teamDir, 'inboxes', 'alice.json'), '[]\n', 'utf8'); + + const svc = new TeamProvisioningService(); + const complete = vi + .spyOn(svc as any, 'handleProvisioningTurnComplete') + .mockResolvedValue(undefined); + const run = { + runId: 'run-mixed-secondary-fs-readiness', + teamName, + cancelRequested: false, + processKilled: false, + provisioningComplete: false, + deterministicBootstrap: true, + fsPhase: 'waiting_members', + effectiveMembers: [{ name: 'alice', providerId: 'codex' }], + progress: { state: 'assembling' }, + onProgress: vi.fn(), + fsMonitorHandle: null, + } as any; + + (svc as any).startFilesystemMonitor(run, { + teamName, + cwd: tempClaudeRoot, + providerId: 'codex', + model: 'gpt-5.4', + members: [ + { name: 'alice', providerId: 'codex' }, + { name: 'tom', providerId: 'opencode' }, + ], + }); + + await vi.waitFor(() => expect(complete).toHaveBeenCalledTimes(1)); + expect(run.fsPhase).toBe('all_files_found'); + expect(run.onProgress).not.toHaveBeenCalledWith( + expect.objectContaining({ + message: 'Prepared communication channels for 1/2 members', + }) + ); + + (svc as any).stopFilesystemMonitor(run); + }); + function createSafeLaunchService(options?: { memberWorktreeManager?: { ensureMemberWorktree: ReturnType }; }) { @@ -10402,6 +10612,193 @@ describe('TeamProvisioningService', () => { }); }); + it('promotes OpenCode secondary pending launch state from committed bootstrap session evidence', async () => { + const teamName = 'zz-opencode-committed-overlay-promotes'; + const leadSessionId = 'lead-session'; + const laneId = 'secondary:opencode:tom'; + const runId = 'opencode-run-tom'; + + writeTeamMeta(teamName, { + providerId: 'codex', + providerBackendId: 'codex-native', + model: 'gpt-5.4', + }); + writeMembersMeta(teamName, [ + { + name: 'tom', + providerId: 'opencode', + model: 'openrouter/minimax/minimax-m2.5', + }, + ]); + writeLaunchConfig(teamName, '/Users/test/proj', leadSessionId, ['tom']); + await upsertOpenCodeRuntimeLaneIndexEntry({ + teamsBasePath: tempTeamsBase, + teamName, + laneId, + state: 'active', + diagnostics: ['OpenCode bridge reported bootstrap confirmation, but no lane runtime evidence was committed.'], + }); + await writeCommittedOpenCodeSessionStore({ + teamName, + laneId, + runId, + sessions: [ + { + id: 'ses-tom', + teamName, + memberName: 'tom', + laneId, + runId, + providerId: 'opencode', + observedAt: '2026-04-22T12:00:00.000Z', + source: 'runtime_bootstrap_checkin', + }, + ], + }); + writeLaunchState(teamName, leadSessionId, { + tom: { + providerId: 'opencode', + laneId, + laneKind: 'secondary', + laneOwnerProviderId: 'opencode', + launchState: 'runtime_pending_bootstrap', + agentToolAccepted: true, + runtimeAlive: false, + bootstrapConfirmed: false, + hardFailure: false, + hardFailureReason: undefined, + diagnostics: [ + 'OpenCode bridge reported bootstrap confirmation, but no lane runtime evidence was committed.', + ], + lastEvaluatedAt: '2026-04-22T12:00:01.000Z', + }, + }); + + const svc = new TeamProvisioningService(); + const result = await svc.getMemberSpawnStatuses(teamName); + + expect(result.statuses.tom).toMatchObject({ + launchState: 'confirmed_alive', + agentToolAccepted: true, + bootstrapConfirmed: true, + runtimeAlive: false, + }); + const persisted = JSON.parse( + await fsPromises.readFile(getTeamLaunchStatePath(teamName), 'utf8') + ); + expect(persisted.members.tom).toMatchObject({ + launchState: 'confirmed_alive', + bootstrapConfirmed: true, + runtimeAlive: false, + runtimeSessionId: 'ses-tom', + }); + }); + + it('prevents stale OpenCode secondary pending or missing writes from downgrading committed bootstrap evidence', async () => { + const teamName = 'zz-opencode-committed-overlay-write-boundary'; + const leadSessionId = 'lead-session'; + const laneId = 'secondary:opencode:tom'; + const runId = 'opencode-run-tom'; + + writeMembersMeta(teamName, [{ name: 'tom', providerId: 'opencode' }]); + await upsertOpenCodeRuntimeLaneIndexEntry({ + teamsBasePath: tempTeamsBase, + teamName, + laneId, + state: 'active', + }); + await writeCommittedOpenCodeSessionStore({ + teamName, + laneId, + runId, + sessions: [ + { + id: 'ses-tom', + teamName, + memberName: 'tom', + laneId, + runId, + observedAt: '2026-04-22T12:00:00.000Z', + source: 'runtime_bootstrap_checkin', + }, + ], + }); + writeLaunchState(teamName, leadSessionId, { + tom: { + providerId: 'opencode', + laneId, + laneKind: 'secondary', + laneOwnerProviderId: 'opencode', + launchState: 'confirmed_alive', + agentToolAccepted: true, + runtimeAlive: false, + bootstrapConfirmed: true, + hardFailure: false, + runtimeSessionId: 'ses-tom', + }, + }); + const staleSnapshot = createPersistedLaunchSnapshot({ + teamName, + leadSessionId, + launchPhase: 'active', + expectedMembers: ['tom'], + members: { + tom: { + name: 'tom', + providerId: 'opencode', + laneId, + laneKind: 'secondary', + laneOwnerProviderId: 'opencode', + launchState: 'runtime_pending_bootstrap', + agentToolAccepted: true, + runtimeAlive: false, + bootstrapConfirmed: false, + hardFailure: false, + lastEvaluatedAt: '2026-04-22T12:00:02.000Z', + diagnostics: [ + 'OpenCode bridge reported bootstrap confirmation, but no lane runtime evidence was committed.', + ], + }, + }, + updatedAt: '2026-04-22T12:00:02.000Z', + }); + + const svc = new TeamProvisioningService(); + await (svc as any).writeLaunchStateSnapshot(teamName, staleSnapshot); + + const persisted = JSON.parse( + await fsPromises.readFile(getTeamLaunchStatePath(teamName), 'utf8') + ); + expect(persisted.members.tom).toMatchObject({ + launchState: 'confirmed_alive', + bootstrapConfirmed: true, + runtimeAlive: false, + runtimeSessionId: 'ses-tom', + }); + expect(persisted.teamLaunchState).toBe('clean_success'); + + const missingMemberSnapshot = createPersistedLaunchSnapshot({ + teamName, + leadSessionId, + launchPhase: 'active', + expectedMembers: [], + members: {}, + updatedAt: '2026-04-22T12:00:03.000Z', + }); + await (svc as any).writeLaunchStateSnapshot(teamName, missingMemberSnapshot); + + const persistedAfterMissingWrite = JSON.parse( + await fsPromises.readFile(getTeamLaunchStatePath(teamName), 'utf8') + ); + expect(persistedAfterMissingWrite.members.tom).toMatchObject({ + launchState: 'confirmed_alive', + bootstrapConfirmed: true, + runtimeAlive: false, + runtimeSessionId: 'ses-tom', + }); + expect(persistedAfterMissingWrite.teamLaunchState).toBe('clean_success'); + }); + it('marks a live teammate bootstrap as confirmed from transcript even when runtime discovery is stale', async () => { allowConsoleLogs(); const teamName = 'zz-live-bootstrap-transcript-success-without-runtime'; diff --git a/test/renderer/utils/teamProvisioningPresentation.test.ts b/test/renderer/utils/teamProvisioningPresentation.test.ts index 01b292b3..a07043b8 100644 --- a/test/renderer/utils/teamProvisioningPresentation.test.ts +++ b/test/renderer/utils/teamProvisioningPresentation.test.ts @@ -556,6 +556,204 @@ describe('buildTeamProvisioningPresentation', () => { expect(presentation?.failedSpawnCount).toBe(0); }); + it('shows core team ready when only OpenCode secondary lanes are still joining', () => { + const presentation = buildTeamProvisioningPresentation({ + progress: { + runId: 'run-opencode-secondary-ready', + teamName: 'mixed-team', + state: 'ready', + startedAt: '2026-04-13T10:00:00.000Z', + updatedAt: '2026-04-13T10:00:08.000Z', + message: 'Team provisioned - waiting for secondary runtime lane: tom', + messageSeverity: undefined, + pid: 4321, + cliLogsTail: '', + assistantOutput: '', + }, + members: [ + { + name: 'team-lead', + agentType: 'team-lead', + providerId: 'codex', + status: 'active', + currentTaskId: null, + taskCount: 0, + lastActiveAt: null, + messageCount: 0, + }, + { + name: 'alice', + providerId: 'codex', + laneKind: 'primary', + status: 'active', + currentTaskId: null, + taskCount: 0, + lastActiveAt: null, + messageCount: 0, + }, + { + name: 'tom', + providerId: 'opencode', + laneId: 'secondary:opencode:tom', + laneKind: 'secondary', + laneOwnerProviderId: 'opencode', + status: 'unknown', + currentTaskId: null, + taskCount: 0, + lastActiveAt: null, + messageCount: 0, + }, + ], + memberSpawnStatuses: { + alice: { + status: 'online', + launchState: 'confirmed_alive', + updatedAt: '2026-04-13T10:00:05.000Z', + runtimeAlive: true, + bootstrapConfirmed: true, + hardFailure: false, + agentToolAccepted: true, + }, + tom: { + status: 'online', + launchState: 'runtime_pending_bootstrap', + updatedAt: '2026-04-13T10:00:07.000Z', + runtimeAlive: true, + livenessSource: 'process', + livenessKind: 'runtime_process_candidate', + bootstrapConfirmed: false, + hardFailure: false, + agentToolAccepted: true, + }, + }, + memberSpawnSnapshot: { + expectedMembers: ['alice', 'tom'], + statuses: { + alice: { + status: 'online', + launchState: 'confirmed_alive', + updatedAt: '2026-04-13T10:00:05.000Z', + runtimeAlive: true, + bootstrapConfirmed: true, + hardFailure: false, + agentToolAccepted: true, + }, + tom: { + status: 'online', + launchState: 'runtime_pending_bootstrap', + updatedAt: '2026-04-13T10:00:07.000Z', + runtimeAlive: true, + livenessSource: 'process', + livenessKind: 'runtime_process_candidate', + bootstrapConfirmed: false, + hardFailure: false, + agentToolAccepted: true, + }, + }, + summary: { + confirmedCount: 1, + pendingCount: 1, + failedCount: 0, + runtimeAlivePendingCount: 0, + runtimeCandidatePendingCount: 1, + }, + }, + }); + + expect(presentation?.successMessage).toBe('Core team ready'); + expect(presentation?.panelMessage).toBe('Waiting for OpenCode: tom'); + expect(presentation?.compactTitle).toBe('Core team ready'); + expect(presentation?.compactDetail).toBe('Waiting for OpenCode: tom'); + expect(presentation?.currentStepIndex).toBe(2); + }); + + it('does not show core team ready while a primary member is still joining', () => { + const presentation = buildTeamProvisioningPresentation({ + progress: { + runId: 'run-primary-still-starting', + teamName: 'mixed-team', + state: 'ready', + startedAt: '2026-04-13T10:00:00.000Z', + updatedAt: '2026-04-13T10:00:08.000Z', + message: 'Team provisioned - waiting for members', + messageSeverity: undefined, + pid: 4321, + cliLogsTail: '', + assistantOutput: '', + }, + members: [ + { + name: 'team-lead', + agentType: 'team-lead', + providerId: 'codex', + status: 'active', + currentTaskId: null, + taskCount: 0, + lastActiveAt: null, + messageCount: 0, + }, + { + name: 'alice', + providerId: 'codex', + laneKind: 'primary', + status: 'unknown', + currentTaskId: null, + taskCount: 0, + lastActiveAt: null, + messageCount: 0, + }, + { + name: 'tom', + providerId: 'opencode', + laneId: 'secondary:opencode:tom', + laneKind: 'secondary', + laneOwnerProviderId: 'opencode', + status: 'unknown', + currentTaskId: null, + taskCount: 0, + lastActiveAt: null, + messageCount: 0, + }, + ], + memberSpawnStatuses: { + alice: { + status: 'waiting', + launchState: 'starting', + updatedAt: '2026-04-13T10:00:05.000Z', + runtimeAlive: false, + bootstrapConfirmed: false, + hardFailure: false, + agentToolAccepted: true, + }, + tom: { + status: 'online', + launchState: 'runtime_pending_bootstrap', + updatedAt: '2026-04-13T10:00:07.000Z', + runtimeAlive: true, + livenessSource: 'process', + livenessKind: 'runtime_process_candidate', + bootstrapConfirmed: false, + hardFailure: false, + agentToolAccepted: true, + }, + }, + memberSpawnSnapshot: { + expectedMembers: ['alice', 'tom'], + summary: { + confirmedCount: 0, + pendingCount: 2, + failedCount: 0, + runtimeAlivePendingCount: 0, + runtimeCandidatePendingCount: 1, + }, + }, + }); + + expect(presentation?.successMessage).toBe('Finishing launch'); + expect(presentation?.panelMessage).not.toBe('Waiting for OpenCode: tom'); + expect(presentation?.compactTitle).toBe('Finishing launch'); + }); + it('surfaces permission-blocked teammates as awaiting approval while launch is finishing', () => { const presentation = buildTeamProvisioningPresentation({ progress: {