test(runtime): cover codex-native phase 1 rollout truth
This commit is contained in:
parent
92a3124e3f
commit
e83e3cbcc9
4 changed files with 389 additions and 3 deletions
|
|
@ -3590,6 +3590,11 @@ Assessment:
|
|||
- `🎯 9 🛡️ 9 🧠 5`
|
||||
- Rough surface: `900-1800` lines
|
||||
|
||||
Status as of 2026-04-19:
|
||||
|
||||
- implementation-complete
|
||||
- sign-off evidence captured in [codex-native-runtime-phase-1-signoff-evidence.md](/Users/belief/dev/projects/claude/claude_team_codex_native_runtime_plan/docs/research/codex-native-runtime-phase-1-signoff-evidence.md)
|
||||
|
||||
Goal:
|
||||
|
||||
- prepare `codex-native` for safe internal unlock without changing default provider behavior
|
||||
|
|
|
|||
204
docs/research/codex-native-runtime-phase-1-signoff-evidence.md
Normal file
204
docs/research/codex-native-runtime-phase-1-signoff-evidence.md
Normal file
|
|
@ -0,0 +1,204 @@
|
|||
# Codex Native Runtime - Phase 1 Sign-off Evidence
|
||||
|
||||
Captured on 2026-04-19.
|
||||
|
||||
This file records the repo-visible evidence package for the Phase 1 exit gate described in:
|
||||
|
||||
- [codex-native-runtime-integration-decision.md](./codex-native-runtime-integration-decision.md)
|
||||
|
||||
## Verdict
|
||||
|
||||
Phase 1 internal unlock preparation is now complete.
|
||||
|
||||
What this proves:
|
||||
|
||||
- `codex-native` can be enabled intentionally through the internal unlock policy
|
||||
- old Codex lanes remain the default and `auto` still resolves to the old adapter/API world
|
||||
- lane-specific rollout states are explicit and honest:
|
||||
- `locked`
|
||||
- `ready`
|
||||
- `authentication-required`
|
||||
- `runtime-missing`
|
||||
- those states now survive all the way through:
|
||||
- orchestrator runtime status
|
||||
- bridge parsing
|
||||
- dashboard/runtime copy
|
||||
- settings/runtime copy
|
||||
- provisioning summaries
|
||||
|
||||
What this does **not** mean:
|
||||
|
||||
- `codex-native` should become the default Codex lane
|
||||
- `auto` should start resolving to `codex-native`
|
||||
- broader approval, plugin, or interactive parity claims are now safe
|
||||
- limited internal unlock has already started
|
||||
|
||||
That is Phase 2 territory.
|
||||
|
||||
## Command Package
|
||||
|
||||
### `agent_teams_orchestrator`
|
||||
|
||||
Executed:
|
||||
|
||||
```bash
|
||||
bun test src/services/runtimeBackends/codexBackendResolver.test.ts \
|
||||
src/services/runtimeBackends/registry.agentTeams.test.ts \
|
||||
src/services/runtimeBackends/registry.codexNativeStates.test.ts
|
||||
```
|
||||
|
||||
Observed result:
|
||||
|
||||
- `14 pass`
|
||||
- `0 fail`
|
||||
|
||||
Executed:
|
||||
|
||||
```bash
|
||||
bun run signoff:codex-native-phase1
|
||||
```
|
||||
|
||||
Observed result:
|
||||
|
||||
- exit code `0`
|
||||
- five live CLI rollout scenarios verified:
|
||||
- `locked`
|
||||
- `internal-unlock-ready`
|
||||
- `authentication-required`
|
||||
- `runtime-missing`
|
||||
- `auto-fallback-stays-old-lane`
|
||||
|
||||
### `claude_team`
|
||||
|
||||
Executed:
|
||||
|
||||
```bash
|
||||
pnpm exec vitest run \
|
||||
test/main/services/runtime/ClaudeMultimodelBridgeService.test.ts \
|
||||
test/main/services/runtime/providerAwareCliEnv.test.ts \
|
||||
test/main/services/runtime/ProviderConnectionService.test.ts \
|
||||
test/renderer/components/runtime/providerConnectionUi.test.ts \
|
||||
test/renderer/components/runtime/ProviderRuntimeBackendSelector.test.ts \
|
||||
test/renderer/components/runtime/ProviderRuntimeSettingsDialog.test.ts \
|
||||
test/renderer/components/team/dialogs/ProvisioningProviderStatusList.test.ts \
|
||||
test/renderer/components/cli/CliStatusVisibility.test.ts \
|
||||
test/main/services/parsing/CodexNativePhase0Smoke.test.ts
|
||||
```
|
||||
|
||||
Observed result:
|
||||
|
||||
- `9` files passed
|
||||
- `83` tests passed
|
||||
- `0` failures
|
||||
|
||||
## Live CLI Rollout Evidence
|
||||
|
||||
Runner:
|
||||
|
||||
```bash
|
||||
runtime status --provider codex --json
|
||||
```
|
||||
|
||||
Observed live scenarios:
|
||||
|
||||
### Locked
|
||||
|
||||
- selected backend: `codex-native`
|
||||
- resolved backend: `codex-native`
|
||||
- provider status: `Codex native runtime ready`
|
||||
- native option:
|
||||
- `selectable=false`
|
||||
- `available=true`
|
||||
- `state=locked`
|
||||
- `audience=internal`
|
||||
- `statusMessage=Ready but locked`
|
||||
|
||||
### Internal unlock ready
|
||||
|
||||
- selected backend: `codex-native`
|
||||
- resolved backend: `codex-native`
|
||||
- provider status: `Codex native runtime ready`
|
||||
- native option:
|
||||
- `selectable=true`
|
||||
- `available=true`
|
||||
- `state=ready`
|
||||
- `audience=internal`
|
||||
- `statusMessage=Ready for internal use`
|
||||
|
||||
### Authentication required
|
||||
|
||||
- selected backend: `codex-native`
|
||||
- resolved backend: `null`
|
||||
- provider status: `Codex native runtime not ready`
|
||||
- native option:
|
||||
- `selectable=false`
|
||||
- `available=false`
|
||||
- `state=authentication-required`
|
||||
- `audience=internal`
|
||||
- `statusMessage=Authentication required`
|
||||
|
||||
### Runtime missing
|
||||
|
||||
- selected backend: `codex-native`
|
||||
- resolved backend: `null`
|
||||
- provider status: `Codex native runtime not ready`
|
||||
- native option:
|
||||
- `selectable=false`
|
||||
- `available=false`
|
||||
- `state=runtime-missing`
|
||||
- `audience=internal`
|
||||
- `statusMessage=Codex CLI not found`
|
||||
|
||||
### Auto fallback stays on the old lane
|
||||
|
||||
- selected backend: `auto`
|
||||
- resolved backend: `api`
|
||||
- provider status: `Resolved to OpenAI API`
|
||||
- native option remains visible for internal rollout:
|
||||
- `selectable=true`
|
||||
- `available=true`
|
||||
- `state=ready`
|
||||
- `audience=internal`
|
||||
- `statusMessage=Ready for internal use`
|
||||
|
||||
This is the explicit proof that internal unlock availability does **not** mutate `auto` resolution.
|
||||
|
||||
## App-facing Truth Proof
|
||||
|
||||
Covered by green targeted tests:
|
||||
|
||||
- `test/main/services/runtime/ClaudeMultimodelBridgeService.test.ts`
|
||||
- `test/main/services/runtime/providerAwareCliEnv.test.ts`
|
||||
- `test/main/services/runtime/ProviderConnectionService.test.ts`
|
||||
- `test/renderer/components/runtime/providerConnectionUi.test.ts`
|
||||
- `test/renderer/components/runtime/ProviderRuntimeBackendSelector.test.ts`
|
||||
- `test/renderer/components/runtime/ProviderRuntimeSettingsDialog.test.ts`
|
||||
- `test/renderer/components/team/dialogs/ProvisioningProviderStatusList.test.ts`
|
||||
- `test/renderer/components/cli/CliStatusVisibility.test.ts`
|
||||
|
||||
These tests prove:
|
||||
|
||||
- internal unlock state survives bridge parsing
|
||||
- internal unlock env survives provider-aware child env building
|
||||
- dashboard and settings do not flatten native rollout states into generic `Connected via API key`
|
||||
- locked/runtime-missing/auth-required states stay visible in user-facing copy
|
||||
- provisioning summaries keep native rollout state visible
|
||||
|
||||
## Phase 1 Exit Gate Conclusion
|
||||
|
||||
✅ The Phase 1 exit gate is satisfied.
|
||||
|
||||
The lane can now be enabled intentionally by internal users, while:
|
||||
|
||||
- old Codex lanes remain the safe default
|
||||
- `auto` still avoids `codex-native`
|
||||
- degraded or blocked native states remain explicit and honest
|
||||
|
||||
⚠️ The lane should still remain:
|
||||
|
||||
- non-default
|
||||
- explicitly internal
|
||||
- rollout-gated
|
||||
- conservative in capability claims
|
||||
|
||||
The next step is **Phase 2 - limited internal unlock**, not broad rollout.
|
||||
|
|
@ -189,4 +189,27 @@ describe('buildProviderAwareCliEnv', () => {
|
|||
expect(result.env.CLAUDE_CODE_GEMINI_BACKEND).toBe('api');
|
||||
expect(result.env.CLAUDE_CODE_CODEX_BACKEND).toBe('adapter');
|
||||
});
|
||||
|
||||
it('preserves codex-native internal unlock env across provider-aware child env building', async () => {
|
||||
buildEnrichedEnvMock.mockReturnValue({
|
||||
PATH: '/usr/bin',
|
||||
CLAUDE_CODE_CODEX_NATIVE_INTERNAL_UNLOCK: '1',
|
||||
});
|
||||
|
||||
const { buildProviderAwareCliEnv } = await import(
|
||||
'../../../../src/main/services/runtime/providerAwareCliEnv'
|
||||
);
|
||||
const result = await buildProviderAwareCliEnv({
|
||||
providerId: 'codex',
|
||||
});
|
||||
|
||||
expect(applyConfiguredConnectionEnvMock).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
CLAUDE_CODE_CODEX_BACKEND: 'adapter',
|
||||
CLAUDE_CODE_CODEX_NATIVE_INTERNAL_UNLOCK: '1',
|
||||
}),
|
||||
'codex'
|
||||
);
|
||||
expect(result.env.CLAUDE_CODE_CODEX_NATIVE_INTERNAL_UNLOCK).toBe('1');
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -76,9 +76,15 @@ vi.mock('@renderer/components/runtime/ProviderRuntimeSettingsDialog', () => ({
|
|||
},
|
||||
}));
|
||||
|
||||
vi.mock('@renderer/components/runtime/ProviderRuntimeBackendSelector', () => ({
|
||||
getProviderRuntimeBackendSummary: () => null,
|
||||
}));
|
||||
vi.mock('@renderer/components/runtime/ProviderRuntimeBackendSelector', async () => {
|
||||
const actual =
|
||||
await vi.importActual<typeof import('@renderer/components/runtime/ProviderRuntimeBackendSelector')>(
|
||||
'@renderer/components/runtime/ProviderRuntimeBackendSelector'
|
||||
);
|
||||
return {
|
||||
getProviderRuntimeBackendSummary: actual.getProviderRuntimeBackendSummary,
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock('@renderer/components/settings/components', async () => {
|
||||
const actual = await vi.importActual<object>('@renderer/components/settings/components');
|
||||
|
|
@ -200,6 +206,70 @@ function createApiKeyModeProviderIssue(providerId: 'anthropic' | 'codex'): Recor
|
|||
};
|
||||
}
|
||||
|
||||
function createCodexNativeRolloutProvider(
|
||||
overrides?: Partial<Record<string, unknown>> & {
|
||||
state?: 'ready' | 'locked' | 'authentication-required' | 'runtime-missing' | 'degraded';
|
||||
audience?: 'general' | 'internal';
|
||||
selectable?: boolean;
|
||||
available?: boolean;
|
||||
statusMessage?: string | null;
|
||||
detailMessage?: string | null;
|
||||
}
|
||||
): Record<string, unknown> {
|
||||
return {
|
||||
providerId: 'codex',
|
||||
displayName: 'Codex',
|
||||
supported: true,
|
||||
authenticated:
|
||||
overrides?.state === 'ready' || overrides?.state === 'locked' || overrides?.available === true,
|
||||
authMethod:
|
||||
overrides?.state === 'ready' || overrides?.state === 'locked' || overrides?.available === true
|
||||
? 'api_key'
|
||||
: null,
|
||||
verificationState:
|
||||
overrides?.state === 'ready' || overrides?.state === 'locked' || overrides?.available === true
|
||||
? 'verified'
|
||||
: 'unknown',
|
||||
statusMessage: overrides?.statusMessage ?? 'Ready but locked',
|
||||
detailMessage: overrides?.detailMessage ?? 'Internal rollout only.',
|
||||
selectedBackendId: 'codex-native',
|
||||
resolvedBackendId:
|
||||
overrides?.state === 'ready' || overrides?.state === 'locked' || overrides?.available === true
|
||||
? 'codex-native'
|
||||
: null,
|
||||
models: ['gpt-5-codex'],
|
||||
canLoginFromUi: false,
|
||||
capabilities: {
|
||||
teamLaunch: true,
|
||||
oneShot: true,
|
||||
},
|
||||
availableBackends: [
|
||||
{
|
||||
id: 'codex-native',
|
||||
label: 'Codex native',
|
||||
description: 'Use codex exec JSON mode.',
|
||||
selectable: overrides?.selectable ?? false,
|
||||
recommended: false,
|
||||
available: overrides?.available ?? true,
|
||||
state: overrides?.state ?? 'locked',
|
||||
audience: overrides?.audience ?? 'internal',
|
||||
statusMessage: overrides?.statusMessage ?? 'Ready but locked',
|
||||
detailMessage: overrides?.detailMessage ?? 'Internal rollout only.',
|
||||
},
|
||||
],
|
||||
backend:
|
||||
overrides?.state === 'ready' || overrides?.state === 'locked' || overrides?.available === true
|
||||
? {
|
||||
kind: 'codex-native',
|
||||
label: 'Codex native',
|
||||
endpointLabel: 'codex exec --json',
|
||||
authMethodDetail: 'api_key',
|
||||
}
|
||||
: null,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe('CLI status visibility during completed install state', () => {
|
||||
afterEach(() => {
|
||||
document.body.innerHTML = '';
|
||||
|
|
@ -769,4 +839,88 @@ describe('CLI status visibility during completed install state', () => {
|
|||
await Promise.resolve();
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps dashboard codex-native rollout truth explicit for locked internal lanes', async () => {
|
||||
vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true);
|
||||
storeState.cliInstallerState = 'idle';
|
||||
storeState.cliStatus = createInstalledCliStatus({
|
||||
flavor: 'agent_teams_orchestrator',
|
||||
displayName: 'agent_teams_orchestrator',
|
||||
supportsSelfUpdate: false,
|
||||
showVersionDetails: false,
|
||||
showBinaryPath: false,
|
||||
authLoggedIn: true,
|
||||
providers: [
|
||||
createCodexNativeRolloutProvider({
|
||||
state: 'locked',
|
||||
available: true,
|
||||
selectable: false,
|
||||
statusMessage: 'Ready but locked',
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
const host = document.createElement('div');
|
||||
document.body.appendChild(host);
|
||||
const root = createRoot(host);
|
||||
|
||||
await act(async () => {
|
||||
root.render(React.createElement(CliStatusBanner));
|
||||
await Promise.resolve();
|
||||
});
|
||||
|
||||
expect(host.textContent).toContain('Ready but locked');
|
||||
expect(host.textContent).toContain('Runtime: Codex native - internal - locked');
|
||||
expect(host.textContent).not.toContain('Connected via API key');
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
await Promise.resolve();
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps settings codex-native rollout truth explicit for runtime-missing lanes', async () => {
|
||||
vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true);
|
||||
storeState.cliInstallerState = 'idle';
|
||||
storeState.cliStatus = createInstalledCliStatus({
|
||||
flavor: 'agent_teams_orchestrator',
|
||||
displayName: 'agent_teams_orchestrator',
|
||||
supportsSelfUpdate: false,
|
||||
showVersionDetails: false,
|
||||
showBinaryPath: false,
|
||||
authLoggedIn: false,
|
||||
providers: [
|
||||
createCodexNativeRolloutProvider({
|
||||
authenticated: false,
|
||||
authMethod: null,
|
||||
verificationState: 'unknown',
|
||||
state: 'runtime-missing',
|
||||
available: false,
|
||||
selectable: false,
|
||||
statusMessage: 'Codex CLI not found',
|
||||
detailMessage: 'Install the codex CLI before enabling the lane.',
|
||||
backend: null,
|
||||
resolvedBackendId: null,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
const host = document.createElement('div');
|
||||
document.body.appendChild(host);
|
||||
const root = createRoot(host);
|
||||
|
||||
await act(async () => {
|
||||
root.render(React.createElement(CliStatusSection));
|
||||
await Promise.resolve();
|
||||
});
|
||||
|
||||
expect(host.textContent).toContain('Codex CLI not found');
|
||||
expect(host.textContent).toContain('Runtime: Codex native - internal - runtime missing');
|
||||
expect(host.textContent).not.toContain('Connected via API key');
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
await Promise.resolve();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
Loading…
Reference in a new issue