Update CONTRIBUTING.md with new guidelines for PRs and AI-assisted contributions; remove outdated session analysis report and cost calculation design documents.
This commit is contained in:
matt
parent
1233a6f155
commit
8f85b48863
5 changed files with 18 additions and 1489 deletions
|
|
@ -23,10 +23,27 @@ pnpm build
|
|||
```
|
||||
|
||||
## Pull Request Guidelines
|
||||
- Keep changes focused and small.
|
||||
- Keep changes focused and small — one purpose per PR.
|
||||
- Add/adjust tests for behavior changes.
|
||||
- Update docs when changing public behavior or setup.
|
||||
- Use clear PR titles and include a short validation checklist.
|
||||
- **Large changes (new features, new dependencies, large data additions) must have a discussion in an Issue first.** Do not open a large PR without prior agreement on the approach.
|
||||
- Avoid committing large hardcoded data blobs. If data can be fetched at runtime or generated at build time, prefer that approach.
|
||||
|
||||
## AI-Assisted Contributions
|
||||
|
||||
AI coding tools are welcome, but **you are responsible for what you submit**:
|
||||
|
||||
- **Review before submitting.** Read every line of AI-generated code and understand what it does. Do not submit raw, unreviewed AI output.
|
||||
- **Do not commit AI workflow artifacts.** Planning documents, session logs, step-by-step plans, or other outputs from AI tools (e.g. `docs/plans/`, `.speckit/`, etc.) do not belong in the repository.
|
||||
- **Test it yourself.** AI-generated code must be manually verified — run the app, confirm the feature works, check edge cases.
|
||||
- **Keep it intentional.** Every line in your PR should exist for a reason you can explain. If you can't explain why a piece of code is there, remove it.
|
||||
|
||||
## What Does NOT Belong in the Repo
|
||||
- Personal planning/workflow artifacts (AI session plans, task lists, etc.)
|
||||
- Large static data that could be fetched at runtime
|
||||
- Generated files that aren't part of the build output
|
||||
- Experimental features without prior discussion
|
||||
|
||||
## Commit Style
|
||||
- Prefer conventional commits (`feat:`, `fix:`, `chore:`, `docs:`).
|
||||
|
|
|
|||
|
|
@ -1,107 +0,0 @@
|
|||
# Session Analysis Report — Design Document
|
||||
|
||||
**Date:** 2026-02-21
|
||||
**Status:** Approved
|
||||
|
||||
## Overview
|
||||
|
||||
Port the `scripts/analyze-session.py` analysis logic to TypeScript and display results as a beautifully formatted report in a new tab. An Activity icon button in the session toolbar triggers the analysis.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **TypeScript port** — no Python dependency; runs in-process in the renderer
|
||||
- **New tab** — opens a dedicated report tab (consistent with Settings/Notifications pattern)
|
||||
- **Activity icon** — in the toolbar next to the Export dropdown
|
||||
- **Full port** — all ~30 analysis sections from the Python script
|
||||
- **Renderer-only** — no new IPC; `SessionDetail.messages` already has all raw data
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
TabBar.tsx (Activity button click)
|
||||
→ store action: openSessionReport(sourceTabId)
|
||||
→ creates tab { type: 'report', projectId, sessionId, sourceTabId }
|
||||
→ SessionReportTab mounts
|
||||
→ analyzeSession(sessionDetail) from sessionAnalyzer.ts
|
||||
→ renders report sections
|
||||
```
|
||||
|
||||
## New Files
|
||||
|
||||
### Types
|
||||
- `src/renderer/types/sessionReport.ts` — `SessionReport` interface with all section types
|
||||
|
||||
### Analysis Engine
|
||||
- `src/renderer/utils/sessionAnalyzer.ts` — `analyzeSession(detail: SessionDetail): SessionReport`
|
||||
- Single-pass over `detail.messages` (mirrors the Python script's accumulator pattern)
|
||||
- Post-pass aggregation for derived metrics
|
||||
- Uses `detail.session` for metadata, `detail.processes` for subagent data
|
||||
|
||||
### Report UI
|
||||
- `src/renderer/components/report/SessionReportTab.tsx` — Main report tab
|
||||
- `src/renderer/components/report/sections/` — Section components:
|
||||
- `OverviewSection` — Session ID, project, duration, message count, context assessment
|
||||
- `CostSection` — Cost by model, total, per-commit, per-line
|
||||
- `TokenSection` — Token usage by model, cache economics, density timeline
|
||||
- `ToolSection` — Tool counts, success rates
|
||||
- `SubagentSection` — Subagent metrics, cost, token usage
|
||||
- `ErrorSection` — Tool errors, permission denials
|
||||
- `GitSection` — Commits, pushes, branches, lines changed
|
||||
- `FrictionSection` — User corrections, thrashing signals
|
||||
- `TimelineSection` — Idle gaps, model switches, key events
|
||||
- `ConversationTreeSection` — Tree depth, branching, sidechains
|
||||
- `QualitySection` — Prompt quality, startup overhead, test progression
|
||||
|
||||
### Integration Points (modified files)
|
||||
- `src/renderer/types/tabs.ts` — Add `'report'` tab type
|
||||
- `src/renderer/store/slices/tabSlice.ts` — Add `openSessionReport` action
|
||||
- `src/renderer/components/layout/TabBar.tsx` — Add Activity icon button
|
||||
- `src/renderer/App.tsx` (or routing equivalent) — Route `report` tabs to `SessionReportTab`
|
||||
|
||||
## Report Visual Design
|
||||
|
||||
- Each section is a card with `bg-surface-raised` background and `border-border` border
|
||||
- Section headers with lucide-react icons and bold titles
|
||||
- Data in tables and stat grids using theme-aware CSS variables
|
||||
- Color-coded assessments: green (healthy), amber (warning), red (critical)
|
||||
- Collapsible detail sections for verbose data (thinking blocks, error details, idle gaps)
|
||||
- Scrollable report body with sticky section navigation
|
||||
|
||||
## Data Flow
|
||||
|
||||
The `SessionDetail` already contains:
|
||||
- `messages: ParsedMessage[]` — raw messages with toolCalls, toolResults, usage, model, timestamp, uuid/parentUuid, isMeta, cwd, gitBranch, agentId, isSidechain
|
||||
- `session: Session` — metadata (contextConsumption, compactionCount, phaseBreakdown, etc.)
|
||||
- `processes: Process[]` — subagent executions with nested messages and metrics
|
||||
- `metrics: SessionMetrics` — pre-computed aggregates
|
||||
|
||||
The analyzer works directly with these types — no JSON serialization or IPC needed.
|
||||
|
||||
## Analysis Sections (ported from Python)
|
||||
|
||||
| Section | Key Metrics |
|
||||
|---------|------------|
|
||||
| Overview | Duration, message count, context consumption, compaction count |
|
||||
| Cost Analysis | Parent + subagent cost, cost by model, per-commit, per-line |
|
||||
| Token Usage | By model (input/output/cache), totals, cache read % |
|
||||
| Cache Economics | Creation 5m/1h, read/write ratio, cold start, efficiency % |
|
||||
| Tool Usage | Counts, success rates per tool |
|
||||
| Subagent Metrics | Count, tokens, duration, cost per agent |
|
||||
| Errors | Tool errors, permission denials, affected tools |
|
||||
| Git Activity | Commits, pushes, branch creations, lines changed |
|
||||
| Friction Signals | Correction count, friction rate, keyword matches |
|
||||
| Thrashing | Bash near-duplicates, file edit rework |
|
||||
| Conversation Tree | Max depth, sidechain count, branch points |
|
||||
| Idle Analysis | Gap count, total idle time, active working time |
|
||||
| Model Switches | Switch count, models used |
|
||||
| Working Directories | Unique dirs, change count |
|
||||
| Test Progression | Snapshots, trajectory (improving/regressing/stable) |
|
||||
| Startup Overhead | Messages/tokens before first work tool |
|
||||
| Token Density Timeline | Quartile averages |
|
||||
| Prompt Quality | First message length, friction rate, assessment |
|
||||
| Thinking Blocks | Count, signal analysis (alternatives, uncertainty, planning) |
|
||||
| Key Events | Timestamped skill invocations, task launches |
|
||||
| Service Tiers | API tier usage distribution |
|
||||
| File Read Redundancy | Reads per unique file, redundant files |
|
||||
| Compact Summaries | Count of context compaction events |
|
||||
| Out-of-scope Findings | Keywords detected in assistant responses |
|
||||
|
|
@ -1,875 +0,0 @@
|
|||
# Session Analysis Report Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Add a toolbar button that runs a full session analysis and displays results in a beautifully formatted report tab.
|
||||
|
||||
**Architecture:** Pure renderer-side analysis engine (`sessionAnalyzer.ts`) processes `SessionDetail.messages` in a single pass. Report opens in a new tab type (`'report'`) rendered by `SessionReportTab`. No new IPC needed — all data is already available from `tabSessionData`.
|
||||
|
||||
**Tech Stack:** React 18, TypeScript, Zustand, Tailwind CSS, lucide-react icons
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add report types
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] File exists at `src/renderer/types/sessionReport.ts`
|
||||
- [ ] `SessionReport` interface is exported from `src/renderer/types/sessionReport.ts`
|
||||
- [ ] `pnpm typecheck` passes with no new errors
|
||||
|
||||
**Files:**
|
||||
- Create: `src/renderer/types/sessionReport.ts`
|
||||
|
||||
**Step 1: Create the SessionReport type file**
|
||||
|
||||
This file defines all the report section types. The analyzer will return a `SessionReport` object.
|
||||
|
||||
```typescript
|
||||
/**
|
||||
* Session analysis report types.
|
||||
* Output of analyzeSession() — one interface per report section.
|
||||
*/
|
||||
|
||||
// =============================================================================
|
||||
// Pricing
|
||||
// =============================================================================
|
||||
|
||||
export interface ModelPricing {
|
||||
input: number;
|
||||
output: number;
|
||||
cache_read: number;
|
||||
cache_creation: number;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Report Sections
|
||||
// =============================================================================
|
||||
|
||||
export interface ReportOverview {
|
||||
sessionId: string;
|
||||
projectId: string;
|
||||
projectPath: string;
|
||||
firstMessage: string;
|
||||
messageCount: number;
|
||||
hasSubagents: boolean;
|
||||
contextConsumption: number;
|
||||
contextConsumptionPct: number | null;
|
||||
contextAssessment: 'critical' | 'high' | 'moderate' | 'healthy' | null;
|
||||
compactionCount: number;
|
||||
gitBranch: string;
|
||||
startTime: Date | null;
|
||||
endTime: Date | null;
|
||||
durationSeconds: number;
|
||||
durationHuman: string;
|
||||
totalMessages: number;
|
||||
}
|
||||
|
||||
export interface ModelTokenStats {
|
||||
apiCalls: number;
|
||||
inputTokens: number;
|
||||
outputTokens: number;
|
||||
cacheCreation: number;
|
||||
cacheRead: number;
|
||||
costUsd: number;
|
||||
}
|
||||
|
||||
export interface TokenTotals {
|
||||
inputTokens: number;
|
||||
outputTokens: number;
|
||||
cacheCreation: number;
|
||||
cacheRead: number;
|
||||
grandTotal: number;
|
||||
cacheReadPct: number;
|
||||
}
|
||||
|
||||
export interface ReportTokenUsage {
|
||||
byModel: Record<string, ModelTokenStats>;
|
||||
totals: TokenTotals;
|
||||
}
|
||||
|
||||
export interface ReportCostAnalysis {
|
||||
parentCostUsd: number;
|
||||
subagentCostUsd: number;
|
||||
totalSessionCostUsd: number;
|
||||
costByModel: Record<string, number>;
|
||||
costPerCommit: number | null;
|
||||
costPerLineChanged: number | null;
|
||||
}
|
||||
|
||||
export interface ReportCacheEconomics {
|
||||
cacheCreation5m: number;
|
||||
cacheCreation1h: number;
|
||||
cacheRead: number;
|
||||
cacheEfficiencyPct: number;
|
||||
coldStartDetected: boolean;
|
||||
cacheReadToWriteRatio: number;
|
||||
}
|
||||
|
||||
export interface ToolSuccessRate {
|
||||
totalCalls: number;
|
||||
errors: number;
|
||||
successRatePct: number;
|
||||
}
|
||||
|
||||
export interface ReportToolUsage {
|
||||
counts: Record<string, number>;
|
||||
totalCalls: number;
|
||||
successRates: Record<string, ToolSuccessRate>;
|
||||
}
|
||||
|
||||
export interface SubagentEntry {
|
||||
description: string;
|
||||
subagentType: string;
|
||||
model: string;
|
||||
totalTokens: number;
|
||||
totalDurationMs: number;
|
||||
totalToolUseCount: number;
|
||||
costUsd: number;
|
||||
costNote?: string;
|
||||
}
|
||||
|
||||
export interface ReportSubagentMetrics {
|
||||
count: number;
|
||||
totalTokens: number;
|
||||
totalDurationMs: number;
|
||||
totalToolUseCount: number;
|
||||
totalCostUsd: number;
|
||||
byAgent: SubagentEntry[];
|
||||
}
|
||||
|
||||
export interface ToolError {
|
||||
tool: string;
|
||||
inputPreview: string;
|
||||
error: string;
|
||||
messageIndex: number;
|
||||
isPermissionDenial: boolean;
|
||||
}
|
||||
|
||||
export interface ReportErrors {
|
||||
errors: ToolError[];
|
||||
permissionDenials: {
|
||||
count: number;
|
||||
denials: ToolError[];
|
||||
affectedTools: string[];
|
||||
};
|
||||
}
|
||||
|
||||
export interface GitCommit {
|
||||
messagePreview: string;
|
||||
messageIndex: number;
|
||||
}
|
||||
|
||||
export interface ReportGitActivity {
|
||||
commitCount: number;
|
||||
commits: GitCommit[];
|
||||
pushCount: number;
|
||||
branchCreations: string[];
|
||||
linesAdded: number;
|
||||
linesRemoved: number;
|
||||
linesChanged: number;
|
||||
}
|
||||
|
||||
export interface FrictionCorrection {
|
||||
messageIndex: number;
|
||||
keyword: string;
|
||||
preview: string;
|
||||
}
|
||||
|
||||
export interface ReportFrictionSignals {
|
||||
correctionCount: number;
|
||||
corrections: FrictionCorrection[];
|
||||
frictionRate: number;
|
||||
}
|
||||
|
||||
export interface ReportThrashingSignals {
|
||||
bashNearDuplicates: { prefix: string; count: number }[];
|
||||
editReworkFiles: { filePath: string; editIndices: number[] }[];
|
||||
}
|
||||
|
||||
export interface ReportConversationTree {
|
||||
totalNodes: number;
|
||||
maxDepth: number;
|
||||
sidechainCount: number;
|
||||
branchPoints: number;
|
||||
branchDetails: {
|
||||
parentUuid: string;
|
||||
childCount: number;
|
||||
parentMessageIndex: number | undefined;
|
||||
}[];
|
||||
}
|
||||
|
||||
export interface IdleGap {
|
||||
gapSeconds: number;
|
||||
gapHuman: string;
|
||||
afterMessageIndex: number;
|
||||
}
|
||||
|
||||
export interface ReportIdleAnalysis {
|
||||
idleThresholdSeconds: number;
|
||||
idleGapCount: number;
|
||||
totalIdleSeconds: number;
|
||||
totalIdleHuman: string;
|
||||
wallClockSeconds: number;
|
||||
activeWorkingSeconds: number;
|
||||
activeWorkingHuman: string;
|
||||
idlePct: number;
|
||||
longestGaps: IdleGap[];
|
||||
}
|
||||
|
||||
export interface ModelSwitch {
|
||||
from: string;
|
||||
to: string;
|
||||
messageIndex: number;
|
||||
timestamp: Date | null;
|
||||
}
|
||||
|
||||
export interface ReportModelSwitches {
|
||||
count: number;
|
||||
switches: ModelSwitch[];
|
||||
modelsUsed: string[];
|
||||
}
|
||||
|
||||
export interface ReportWorkingDirectories {
|
||||
uniqueDirectories: string[];
|
||||
directoryCount: number;
|
||||
changes: { from: string; to: string; messageIndex: number }[];
|
||||
changeCount: number;
|
||||
isMultiDirectory: boolean;
|
||||
}
|
||||
|
||||
export interface TestSnapshot {
|
||||
messageIndex: number;
|
||||
passed: number;
|
||||
failed: number;
|
||||
total: number;
|
||||
raw: string;
|
||||
}
|
||||
|
||||
export interface ReportTestProgression {
|
||||
snapshotCount: number;
|
||||
snapshots: TestSnapshot[];
|
||||
trajectory: 'improving' | 'regressing' | 'stable' | 'insufficient_data';
|
||||
firstSnapshot: TestSnapshot | null;
|
||||
lastSnapshot: TestSnapshot | null;
|
||||
}
|
||||
|
||||
export interface ReportStartupOverhead {
|
||||
messagesBeforeFirstWork: number;
|
||||
tokensBeforeFirstWork: number;
|
||||
pctOfTotal: number;
|
||||
}
|
||||
|
||||
export interface ReportTokenDensityTimeline {
|
||||
quartiles: { q: number; avgTokens: number; messageCount: number }[];
|
||||
}
|
||||
|
||||
export interface ReportPromptQuality {
|
||||
firstMessageLengthChars: number;
|
||||
userMessageCount: number;
|
||||
correctionCount: number;
|
||||
frictionRate: number;
|
||||
assessment: 'underspecified' | 'verbose_but_unclear' | 'well_specified' | 'moderate_friction';
|
||||
note: string;
|
||||
}
|
||||
|
||||
export interface ThinkingBlockAnalysis {
|
||||
messageIndex: number;
|
||||
preview: string;
|
||||
charLength: number;
|
||||
signals: Record<string, boolean>;
|
||||
}
|
||||
|
||||
export interface ReportThinkingBlocks {
|
||||
count: number;
|
||||
analyzedCount: number;
|
||||
signalSummary: Record<string, number>;
|
||||
notableBlocks: ThinkingBlockAnalysis[];
|
||||
}
|
||||
|
||||
export interface KeyEvent {
|
||||
timestamp: Date;
|
||||
label: string;
|
||||
deltaSeconds?: number;
|
||||
deltaHuman?: string;
|
||||
}
|
||||
|
||||
export interface ReportFileReadRedundancy {
|
||||
totalReads: number;
|
||||
uniqueFiles: number;
|
||||
readsPerUniqueFile: number;
|
||||
redundantFiles: Record<string, number>;
|
||||
}
|
||||
|
||||
// =============================================================================
|
||||
// Combined Report
|
||||
// =============================================================================
|
||||
|
||||
export interface SessionReport {
|
||||
overview: ReportOverview;
|
||||
tokenUsage: ReportTokenUsage;
|
||||
costAnalysis: ReportCostAnalysis;
|
||||
cacheEconomics: ReportCacheEconomics;
|
||||
toolUsage: ReportToolUsage;
|
||||
subagentMetrics: ReportSubagentMetrics;
|
||||
errors: ReportErrors;
|
||||
gitActivity: ReportGitActivity;
|
||||
frictionSignals: ReportFrictionSignals;
|
||||
thrashingSignals: ReportThrashingSignals;
|
||||
conversationTree: ReportConversationTree;
|
||||
idleAnalysis: ReportIdleAnalysis;
|
||||
modelSwitches: ReportModelSwitches;
|
||||
workingDirectories: ReportWorkingDirectories;
|
||||
testProgression: ReportTestProgression;
|
||||
startupOverhead: ReportStartupOverhead;
|
||||
tokenDensityTimeline: ReportTokenDensityTimeline;
|
||||
promptQuality: ReportPromptQuality;
|
||||
thinkingBlocks: ReportThinkingBlocks;
|
||||
keyEvents: KeyEvent[];
|
||||
messageTypes: Record<string, number>;
|
||||
serviceTiers: Record<string, number>;
|
||||
fileReadRedundancy: ReportFileReadRedundancy;
|
||||
compactionCount: number;
|
||||
gitBranches: string[];
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2: Verify types compile**
|
||||
|
||||
Run: `pnpm typecheck`
|
||||
Expected: No errors related to sessionReport.ts (file is only types, no imports yet)
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add src/renderer/types/sessionReport.ts
|
||||
git commit -m "feat(report): add session analysis report type definitions"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Build the session analyzer
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] File exists at `src/renderer/utils/sessionAnalyzer.ts`
|
||||
- [ ] `analyzeSession` function is exported from `src/renderer/utils/sessionAnalyzer.ts`
|
||||
- [ ] `pnpm typecheck` passes with no new errors
|
||||
|
||||
**Files:**
|
||||
- Create: `src/renderer/utils/sessionAnalyzer.ts`
|
||||
|
||||
**Docs to reference:**
|
||||
- `scripts/analyze-session.py` — the Python script being ported (all logic)
|
||||
- `src/main/types/messages.ts` — `ParsedMessage`, `ToolCall`, `ToolResult`
|
||||
- `src/main/types/domain.ts` — `Session`, `SessionMetrics`, `TokenUsage` (= `UsageMetadata`)
|
||||
- `src/main/types/chunks.ts` — `SessionDetail`, `Process`
|
||||
- `src/main/types/jsonl.ts` — `UsageMetadata` (input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens)
|
||||
|
||||
**Step 1: Create the analyzer**
|
||||
|
||||
Port all logic from `scripts/analyze-session.py` to TypeScript. The analyzer takes a `SessionDetail` (which has `session`, `messages`, `processes`, `metrics`) and returns a `SessionReport`.
|
||||
|
||||
Key mapping from Python to TS:
|
||||
- Python `data["messages"]` → `detail.messages: ParsedMessage[]`
|
||||
- Python `data["session"]` → `detail.session: Session`
|
||||
- Python `m.get("toolCalls", [])` → `msg.toolCalls: ToolCall[]`
|
||||
- Python `m.get("toolResults", [])` → `msg.toolResults: ToolResult[]`
|
||||
- Python `m.get("usage")` → `msg.usage?: TokenUsage` (fields: `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`)
|
||||
- Python `m.get("model")` → `msg.model?: string`
|
||||
- Python `m.get("timestamp")` → `msg.timestamp: Date` (already parsed)
|
||||
- Python `m.get("isMeta")` → `msg.isMeta: boolean`
|
||||
- Python `m.get("uuid")` → `msg.uuid: string`
|
||||
- Python `m.get("parentUuid")` → `msg.parentUuid: string | null`
|
||||
- Python `m.get("cwd")` → `msg.cwd?: string`
|
||||
- Python `m.get("gitBranch")` → `msg.gitBranch?: string`
|
||||
- Python `m.get("isSidechain")` → `msg.isSidechain: boolean`
|
||||
- Python `m.get("isCompactSummary")` → `msg.isCompactSummary?: boolean`
|
||||
- Python `m.get("agentId")` → `msg.agentId?: string`
|
||||
- For subagent data, use `detail.processes: Process[]` (already resolved with metrics, duration, description, subagentType)
|
||||
|
||||
The function signature:
|
||||
|
||||
```typescript
|
||||
import type { SessionDetail } from '@renderer/types/data';
|
||||
import type { SessionReport } from '@renderer/types/sessionReport';
|
||||
|
||||
export function analyzeSession(detail: SessionDetail): SessionReport { ... }
|
||||
```
|
||||
|
||||
Follow the Python script's single-pass pattern:
|
||||
1. Initialize accumulators
|
||||
2. Loop over `detail.messages` once, extracting all data
|
||||
3. Post-pass aggregation
|
||||
4. Return typed `SessionReport`
|
||||
|
||||
For content text extraction, use this helper (mirrors Python's `extract_text_content`):
|
||||
|
||||
```typescript
|
||||
function extractTextContent(msg: ParsedMessage): string {
|
||||
const { content } = msg;
|
||||
if (typeof content === 'string') return content;
|
||||
if (Array.isArray(content)) {
|
||||
return content
|
||||
.filter((block) => block.type === 'text')
|
||||
.map((block) => block.text)
|
||||
.join(' ');
|
||||
}
|
||||
return '';
|
||||
}
|
||||
```
|
||||
|
||||
For pricing, port the `MODEL_PRICING` table and `costUsd()` function directly.
|
||||
|
||||
For subagent metrics, use `detail.processes` instead of parsing `<usage>` tags — the data is already resolved:
|
||||
```typescript
|
||||
const subagentEntries: SubagentEntry[] = detail.processes.map((proc) => ({
|
||||
description: proc.description ?? 'unknown',
|
||||
subagentType: proc.subagentType ?? 'unknown',
|
||||
model: 'default (inherits parent)',
|
||||
totalTokens: proc.metrics.totalTokens,
|
||||
totalDurationMs: proc.durationMs,
|
||||
totalToolUseCount: proc.messages.reduce((sum, m) => sum + m.toolCalls.length, 0),
|
||||
costUsd: proc.metrics.costUsd ?? 0,
|
||||
}));
|
||||
```
|
||||
|
||||
Port ALL regex patterns from Python:
|
||||
- `FRICTION_PATTERNS` — friction keyword detection
|
||||
- `PERMISSION_PATTERNS` — permission denial detection
|
||||
- `TEST_PASS_PATTERNS`, `TEST_FAIL_PATTERNS`, `TEST_SUMMARY_PATTERN` — test output parsing
|
||||
- `THINKING_SIGNALS` — thinking block content analysis
|
||||
|
||||
**Step 2: Verify it compiles**
|
||||
|
||||
Run: `pnpm typecheck`
|
||||
Expected: PASS
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add src/renderer/utils/sessionAnalyzer.ts
|
||||
git commit -m "feat(report): add session analyzer engine (TS port of analyze-session.py)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Write analyzer tests
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] File exists at `test/renderer/utils/sessionAnalyzer.test.ts`
|
||||
- [ ] `pnpm test test/renderer/utils/sessionAnalyzer.test.ts` passes with all tests green
|
||||
|
||||
**Files:**
|
||||
- Create: `test/renderer/utils/sessionAnalyzer.test.ts`
|
||||
|
||||
**Step 1: Write tests**
|
||||
|
||||
Test the analyzer with mock `SessionDetail` objects. At minimum:
|
||||
|
||||
1. **Empty session** — no messages, returns zeroed report
|
||||
2. **Basic session** — a few user + assistant messages with usage data, verify overview, token counts, cost
|
||||
3. **Tool usage** — messages with toolCalls and toolResults, verify tool counts and success rates
|
||||
4. **Error detection** — toolResults with `isError: true`, verify error list and permission denial detection
|
||||
5. **Friction detection** — user messages with "no,", "wrong", "actually" keywords
|
||||
6. **Git activity** — Bash toolCalls containing "git commit", "git push"
|
||||
7. **Idle gaps** — messages with timestamps >60s apart
|
||||
8. **Model switches** — assistant messages with different model fields
|
||||
9. **Conversation tree** — messages with uuid/parentUuid, verify depth and branching
|
||||
|
||||
Create a `createMockMessage()` helper for building `ParsedMessage` objects easily:
|
||||
|
||||
```typescript
|
||||
function createMockMessage(overrides: Partial<ParsedMessage> = {}): ParsedMessage {
|
||||
return {
|
||||
uuid: crypto.randomUUID(),
|
||||
parentUuid: null,
|
||||
type: 'assistant',
|
||||
timestamp: new Date(),
|
||||
content: '',
|
||||
isSidechain: false,
|
||||
isMeta: false,
|
||||
toolCalls: [],
|
||||
toolResults: [],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
And a `createMockDetail()` helper:
|
||||
|
||||
```typescript
|
||||
function createMockDetail(overrides: Partial<SessionDetail> = {}): SessionDetail {
|
||||
return {
|
||||
session: { id: 'test', projectId: 'test', projectPath: '/test', createdAt: Date.now(), hasSubagents: false, messageCount: 0 } as Session,
|
||||
messages: [],
|
||||
chunks: [],
|
||||
processes: [],
|
||||
metrics: { durationMs: 0, totalTokens: 0, inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheCreationTokens: 0, messageCount: 0 },
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2: Run tests**
|
||||
|
||||
Run: `pnpm test test/renderer/utils/sessionAnalyzer.test.ts`
|
||||
Expected: All tests pass
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add test/renderer/utils/sessionAnalyzer.test.ts
|
||||
git commit -m "test(report): add session analyzer tests"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add 'report' tab type and store action
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] `src/renderer/types/tabs.ts` contains `'report'` in the Tab type union
|
||||
- [ ] `src/renderer/components/layout/SortableTab.tsx` contains `report: Activity` in TAB_ICONS
|
||||
- [ ] `openSessionReport` is declared in TabSlice interface in `src/renderer/store/slices/tabSlice.ts`
|
||||
- [ ] `pnpm typecheck` passes with no new errors
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/renderer/types/tabs.ts:79` — add `'report'` to Tab type union
|
||||
- Modify: `src/renderer/components/layout/SortableTab.tsx:28-33` — add report icon to TAB_ICONS
|
||||
- Modify: `src/renderer/store/slices/tabSlice.ts` — add `openSessionReport` action
|
||||
- Modify: `src/renderer/store/types.ts` (if needed for new slice, but likely just extend tabSlice)
|
||||
|
||||
**Step 1: Add 'report' to Tab type**
|
||||
|
||||
In `src/renderer/types/tabs.ts`, line 79, change:
|
||||
```typescript
|
||||
type: 'session' | 'dashboard' | 'notifications' | 'settings';
|
||||
```
|
||||
to:
|
||||
```typescript
|
||||
type: 'session' | 'dashboard' | 'notifications' | 'settings' | 'report';
|
||||
```
|
||||
|
||||
**Step 2: Add report icon to SortableTab**
|
||||
|
||||
In `src/renderer/components/layout/SortableTab.tsx`, add `Activity` to the lucide-react import and to `TAB_ICONS`:
|
||||
|
||||
```typescript
|
||||
import { Activity, Bell, FileText, LayoutDashboard, Pin, Search, Settings, X } from 'lucide-react';
|
||||
|
||||
const TAB_ICONS = {
|
||||
dashboard: LayoutDashboard,
|
||||
notifications: Bell,
|
||||
settings: Settings,
|
||||
session: FileText,
|
||||
report: Activity,
|
||||
} as const;
|
||||
```
|
||||
|
||||
**Step 3: Add openSessionReport action to tabSlice**
|
||||
|
||||
In `src/renderer/store/slices/tabSlice.ts`, add to the `TabSlice` interface:
|
||||
|
||||
```typescript
|
||||
openSessionReport: (sourceTabId: string) => void;
|
||||
```
|
||||
|
||||
Implement it following the `openNotificationsTab` pattern. It needs to:
|
||||
1. Get `tabSessionData[sourceTabId]` to find the sessionDetail
|
||||
2. Extract the session's firstMessage for the tab label
|
||||
3. Open a new tab with `type: 'report'`, the same `projectId` and `sessionId` as the source tab
|
||||
|
||||
```typescript
|
||||
openSessionReport: (sourceTabId: string) => {
|
||||
const state = get();
|
||||
const sourceTab = getAllTabs(state.paneLayout).find((t) => t.id === sourceTabId);
|
||||
if (!sourceTab || sourceTab.type !== 'session') return;
|
||||
|
||||
const tabData = state.tabSessionData[sourceTabId];
|
||||
const sessionDetail = tabData?.sessionDetail;
|
||||
const label = sessionDetail?.session.firstMessage
|
||||
? `Report: ${truncateLabel(sessionDetail.session.firstMessage, 30)}`
|
||||
: 'Session Report';
|
||||
|
||||
state.openTab({
|
||||
type: 'report',
|
||||
label,
|
||||
projectId: sourceTab.projectId,
|
||||
sessionId: sourceTab.sessionId,
|
||||
});
|
||||
},
|
||||
```
|
||||
|
||||
**Step 4: Verify types compile**
|
||||
|
||||
Run: `pnpm typecheck`
|
||||
Expected: PASS (PaneContent.tsx will have a gap for the `report` type — we'll add it in Task 6)
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/renderer/types/tabs.ts src/renderer/components/layout/SortableTab.tsx src/renderer/store/slices/tabSlice.ts
|
||||
git commit -m "feat(report): add 'report' tab type and openSessionReport store action"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Build the report UI components
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] File exists at `src/renderer/components/report/SessionReportTab.tsx`
|
||||
- [ ] File exists at `src/renderer/components/report/ReportSection.tsx`
|
||||
- [ ] Files exist at `src/renderer/components/report/sections/OverviewSection.tsx`, `CostSection.tsx`, `TokenSection.tsx`, `ToolSection.tsx`, `SubagentSection.tsx`, `ErrorSection.tsx`, `GitSection.tsx`, `FrictionSection.tsx`, `TimelineSection.tsx`, `QualitySection.tsx`
|
||||
- [ ] `pnpm typecheck` passes with no new errors
|
||||
|
||||
**Files:**
|
||||
- Create: `src/renderer/components/report/SessionReportTab.tsx`
|
||||
- Create: `src/renderer/components/report/ReportSection.tsx` — reusable section card wrapper
|
||||
- Create: `src/renderer/components/report/sections/OverviewSection.tsx`
|
||||
- Create: `src/renderer/components/report/sections/CostSection.tsx`
|
||||
- Create: `src/renderer/components/report/sections/TokenSection.tsx`
|
||||
- Create: `src/renderer/components/report/sections/ToolSection.tsx`
|
||||
- Create: `src/renderer/components/report/sections/SubagentSection.tsx`
|
||||
- Create: `src/renderer/components/report/sections/ErrorSection.tsx`
|
||||
- Create: `src/renderer/components/report/sections/GitSection.tsx`
|
||||
- Create: `src/renderer/components/report/sections/FrictionSection.tsx`
|
||||
- Create: `src/renderer/components/report/sections/TimelineSection.tsx`
|
||||
- Create: `src/renderer/components/report/sections/QualitySection.tsx`
|
||||
|
||||
**Docs to reference:**
|
||||
- `src/renderer/index.css` — CSS variables for theming
|
||||
- `.claude/rules/tailwind.md` — Theme architecture (use `bg-surface-raised`, `text-text`, `border-border`, etc.)
|
||||
- `src/renderer/components/common/TokenUsageDisplay.tsx` — Example of formatted token display
|
||||
- `src/renderer/utils/formatters.ts` — Existing formatting utilities
|
||||
|
||||
**Step 1: Create ReportSection wrapper**
|
||||
|
||||
A reusable card component for each report section:
|
||||
|
||||
```tsx
|
||||
interface ReportSectionProps {
|
||||
title: string;
|
||||
icon: React.ComponentType<{ className?: string }>;
|
||||
children: React.ReactNode;
|
||||
defaultCollapsed?: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
Uses `bg-surface-raised`, `border-border`, collapsible with ChevronDown/ChevronRight toggle.
|
||||
|
||||
**Step 2: Create section components**
|
||||
|
||||
Each section receives its typed data from the `SessionReport` and renders it. Design guidelines:
|
||||
|
||||
- **Stat grids**: 2-4 columns of key metrics with label + value
|
||||
- **Tables**: For lists of items (tools, errors, subagents) using `<table>` with `text-xs`
|
||||
- **Color coding**: Use inline styles with CSS variables — green for good, amber for warning, red for critical
|
||||
- **Collapsible details**: For verbose lists (errors, thinking blocks), show count in header and expand for details
|
||||
|
||||
Key section designs:
|
||||
|
||||
**OverviewSection**: Grid of 6-8 stat cards (duration, messages, context %, compaction, branch, cost)
|
||||
|
||||
**CostSection**: Cost by model table + stat cards for total, per-commit, per-line
|
||||
|
||||
**TokenSection**: By-model table (input/output/cache-read/cache-create/cost per model) + totals row + cache economics stats
|
||||
|
||||
**ToolSection**: Sorted table (tool name, calls, errors, success %) — highlight tools with <90% success rate
|
||||
|
||||
**SubagentSection**: Table of subagents (description, type, tokens, duration, cost) + summary stats
|
||||
|
||||
**ErrorSection**: Grouped by tool, expandable error details with input preview
|
||||
|
||||
**GitSection**: Commits list + stat cards (pushes, branches, lines added/removed)
|
||||
|
||||
**FrictionSection**: Friction rate badge + corrections list with message previews + thrashing signals
|
||||
|
||||
**TimelineSection**: Idle gaps table + model switches list + key events timeline
|
||||
|
||||
**QualitySection**: Prompt quality assessment badge + startup overhead stats + test progression
|
||||
|
||||
**Step 3: Create SessionReportTab**
|
||||
|
||||
Main component that:
|
||||
1. Gets `sessionDetail` from `tabSessionData` using the tab's `sessionId` (find the source session tab's data)
|
||||
2. Calls `analyzeSession(sessionDetail)` with `useMemo`
|
||||
3. Renders a scrollable container with all section components
|
||||
4. Shows loading/error states if session data isn't loaded
|
||||
|
||||
```tsx
|
||||
import { useMemo } from 'react';
|
||||
import { useStore } from '@renderer/store';
|
||||
import { analyzeSession } from '@renderer/utils/sessionAnalyzer';
|
||||
import type { Tab } from '@renderer/types/tabs';
|
||||
|
||||
interface SessionReportTabProps {
|
||||
tab: Tab;
|
||||
}
|
||||
|
||||
export const SessionReportTab = ({ tab }: SessionReportTabProps) => {
|
||||
// Find session data from any session tab with matching sessionId
|
||||
const sessionDetail = useStore((s) => {
|
||||
const allTabs = s.paneLayout.panes.flatMap((p) => p.tabs);
|
||||
const sourceTab = allTabs.find(
|
||||
(t) => t.type === 'session' && t.sessionId === tab.sessionId
|
||||
);
|
||||
return sourceTab ? s.tabSessionData[sourceTab.id]?.sessionDetail : null;
|
||||
});
|
||||
|
||||
const report = useMemo(
|
||||
() => (sessionDetail ? analyzeSession(sessionDetail) : null),
|
||||
[sessionDetail]
|
||||
);
|
||||
|
||||
if (!report) {
|
||||
return <div>No session data available. Open the session first.</div>;
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="h-full overflow-y-auto p-6" style={{ backgroundColor: 'var(--color-surface)' }}>
|
||||
<h1 className="mb-6 text-lg font-semibold" style={{ color: 'var(--color-text)' }}>
|
||||
Session Analysis Report
|
||||
</h1>
|
||||
<div className="flex flex-col gap-4">
|
||||
<OverviewSection data={report.overview} />
|
||||
<CostSection data={report.costAnalysis} />
|
||||
<TokenSection data={report.tokenUsage} cacheEconomics={report.cacheEconomics} />
|
||||
<ToolSection data={report.toolUsage} />
|
||||
<SubagentSection data={report.subagentMetrics} />
|
||||
<ErrorSection data={report.errors} />
|
||||
<GitSection data={report.gitActivity} />
|
||||
<FrictionSection data={report.frictionSignals} thrashing={report.thrashingSignals} />
|
||||
<TimelineSection idle={report.idleAnalysis} modelSwitches={report.modelSwitches} keyEvents={report.keyEvents} />
|
||||
<QualitySection prompt={report.promptQuality} startup={report.startupOverhead} testProgression={report.testProgression} />
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
**Step 4: Verify it compiles**
|
||||
|
||||
Run: `pnpm typecheck`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/renderer/components/report/
|
||||
git commit -m "feat(report): add session report tab and all section components"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Wire up routing and toolbar button
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] `src/renderer/components/layout/PaneContent.tsx` imports and renders `SessionReportTab` for `tab.type === 'report'`
|
||||
- [ ] `src/renderer/components/layout/TabBar.tsx` contains an Activity button with `onClick` calling `openSessionReport`
|
||||
- [ ] `pnpm typecheck` passes with no new errors
|
||||
- [ ] `pnpm test` passes with all existing tests green
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/renderer/components/layout/PaneContent.tsx:42-49` — add report tab routing
|
||||
- Modify: `src/renderer/components/layout/TabBar.tsx:17,56,102-107,384-387` — add analyze button
|
||||
|
||||
**Step 1: Add report routing in PaneContent**
|
||||
|
||||
In `src/renderer/components/layout/PaneContent.tsx`, import `SessionReportTab` and add the route:
|
||||
|
||||
```tsx
|
||||
import { SessionReportTab } from '../report/SessionReportTab';
|
||||
```
|
||||
|
||||
In the tab rendering map (around line 42), add before or after the session case:
|
||||
|
||||
```tsx
|
||||
{tab.type === 'report' && <SessionReportTab tab={tab} />}
|
||||
```
|
||||
|
||||
**Step 2: Add analyze button in TabBar**
|
||||
|
||||
In `src/renderer/components/layout/TabBar.tsx`:
|
||||
|
||||
1. Add `Activity` to the lucide-react import (line 17)
|
||||
2. Add `openSessionReport` to the store destructure (line 56 area)
|
||||
3. Add a hover state: `const [analyzeHover, setAnalyzeHover] = useState(false);`
|
||||
4. Add the button next to ExportDropdown (after line 387):
|
||||
|
||||
```tsx
|
||||
{/* Analyze button - show only for session tabs with loaded data */}
|
||||
{activeTab?.type === 'session' && activeTabSessionDetail && activeTabId && (
|
||||
<button
|
||||
onClick={() => openSessionReport(activeTabId)}
|
||||
onMouseEnter={() => setAnalyzeHover(true)}
|
||||
onMouseLeave={() => setAnalyzeHover(false)}
|
||||
className="rounded-md p-2 transition-colors"
|
||||
style={{
|
||||
color: analyzeHover ? 'var(--color-text)' : 'var(--color-text-muted)',
|
||||
backgroundColor: analyzeHover ? 'var(--color-surface-raised)' : 'transparent',
|
||||
}}
|
||||
title="Analyze Session"
|
||||
>
|
||||
<Activity className="size-4" />
|
||||
</button>
|
||||
)}
|
||||
```
|
||||
|
||||
**Step 3: Verify it compiles**
|
||||
|
||||
Run: `pnpm typecheck`
|
||||
Expected: PASS
|
||||
|
||||
**Step 4: Run existing tests to verify nothing broke**
|
||||
|
||||
Run: `pnpm test`
|
||||
Expected: All existing tests still pass
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/renderer/components/layout/PaneContent.tsx src/renderer/components/layout/TabBar.tsx
|
||||
git commit -m "feat(report): wire up toolbar button and report tab routing"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Manual verification and polish
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- [ ] [MANUAL] App launches with `pnpm dev` and report tab opens when Activity button is clicked
|
||||
- [ ] [MANUAL] All report sections render with data from the active session
|
||||
- [ ] `pnpm test` passes
|
||||
- [ ] `pnpm typecheck` passes
|
||||
- [ ] `pnpm lint:fix && pnpm format` passes with no remaining issues
|
||||
|
||||
**Step 1: Run the app**
|
||||
|
||||
Run: `pnpm dev`
|
||||
|
||||
1. Open a session tab
|
||||
2. Click the Activity (analyze) icon in the toolbar
|
||||
3. Verify a new "Report: ..." tab opens
|
||||
4. Verify all sections render with data
|
||||
5. Check that section cards use correct theme colors
|
||||
6. Verify collapsible sections work
|
||||
7. Verify the tab icon shows the Activity icon in the tab bar
|
||||
|
||||
**Step 2: Run full test suite**
|
||||
|
||||
Run: `pnpm test`
|
||||
Expected: All tests pass
|
||||
|
||||
**Step 3: Run typecheck**
|
||||
|
||||
Run: `pnpm typecheck`
|
||||
Expected: No errors
|
||||
|
||||
**Step 4: Run lint and format**
|
||||
|
||||
Run: `pnpm lint:fix && pnpm format`
|
||||
|
||||
**Step 5: Final commit if any polish changes**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "feat(report): polish and fix lint issues"
|
||||
```
|
||||
|
|
@ -1,71 +0,0 @@
|
|||
# Unify Cost Calculation — Design Document
|
||||
|
||||
**Date:** 2026-02-23
|
||||
**Branch:** `feat/unify-cost-calculation`
|
||||
**Related:** PR #60 (Session Analysis Report), PR #65 (Cost Calculation Metric), Issue #72 (Plan Usage Tracking)
|
||||
|
||||
## Problem
|
||||
|
||||
Cost calculation exists in two places with different pricing data and logic:
|
||||
|
||||
1. **Main process** (`src/main/utils/jsonl.ts`): Uses LiteLLM-sourced `pricing.json` (206 models, tiered 200k-token pricing). Populates `SessionMetrics.costUsd` for the chat UI.
|
||||
2. **Renderer** (`src/renderer/utils/sessionAnalyzer.ts`): Uses a hardcoded 6-model pricing table with no tiered pricing. Generates per-model cost breakdown for the Session Report.
|
||||
|
||||
The two systems can produce different cost numbers for the same session and will drift further as models change.
|
||||
|
||||
## Solution
|
||||
|
||||
Create a single shared pricing module that both processes import.
|
||||
|
||||
### New Module: `src/shared/utils/pricing.ts`
|
||||
|
||||
**Exports:**
|
||||
|
||||
| Export | Description |
|
||||
|--------|-------------|
|
||||
| `ModelPricing` | Interface for per-model rates (input, output, cache read, cache creation, plus tiered variants) |
|
||||
| `getPricing(modelName: string): ModelPricing \| null` | Model lookup: exact match, lowercase, case-insensitive scan |
|
||||
| `calculateTieredCost(tokens, baseRate, tieredRate?): number` | Applies 200k-token tier threshold |
|
||||
| `calculateMessageCost(model, inputTokens, outputTokens, cacheReadTokens, cacheCreationTokens): number` | Computes cost for a single API call |
|
||||
|
||||
**Pricing data:** Static `import pricingData from '../../../resources/pricing.json'` with `resolveJsonModule: true`. Replaces `fs.readFileSync` runtime loading.
|
||||
|
||||
### Consumer Changes
|
||||
|
||||
**`src/main/utils/jsonl.ts`:**
|
||||
- Remove: `ModelPricing` interface, `loadPricingData()`, `calculateTieredCost()`, `getPricing()`, `fs`/`path` imports
|
||||
- Keep: `calculateMetrics()` function
|
||||
- Change: inline cost loop body → call `calculateMessageCost()` from `@shared/utils/pricing`
|
||||
|
||||
**`src/renderer/utils/sessionAnalyzer.ts`:**
|
||||
- Remove: `MODEL_PRICING` table (~40 lines), `DEFAULT_PRICING`, local `getPricing()`, local `costUsd()`
|
||||
- Change: calls at lines 476 and 900 → `calculateMessageCost()` from `@shared/utils/pricing`
|
||||
|
||||
**Tests:**
|
||||
- `test/main/utils/costCalculation.test.ts` → update to test shared module functions
|
||||
- `test/renderer/utils/sessionAnalyzer.test.ts` → mock `@shared/utils/pricing` instead of local functions
|
||||
- New `test/shared/utils/pricing.test.ts` for the shared module
|
||||
|
||||
### Pricing JSON Import Strategy
|
||||
|
||||
- `pricing.json` stays in `resources/` for Electron's `extraResources` packaging
|
||||
- Both Vite (renderer) and electron-vite (main) resolve the JSON import at compile time
|
||||
- Remove the `fs.readFileSync` dev/prod path logic from `jsonl.ts`
|
||||
|
||||
### Fallback Behavior
|
||||
|
||||
- `getPricing()` returns `null` for unknown models
|
||||
- `calculateMessageCost()` returns `0` for unknown models (matches current `jsonl.ts` behavior)
|
||||
- Session analyzer callers can apply a default if needed
|
||||
|
||||
### What Changes for Users
|
||||
|
||||
- Report costs become more accurate (tiered pricing, 206 models instead of 6)
|
||||
- Cost numbers between chat view and Session Report now agree exactly
|
||||
- Small UI change: Visible Context header adds a "parent only · view full cost" action when available
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Plan usage tracking (see Issue #72 — pending community feedback)
|
||||
- New UI surfaces for cost display
|
||||
- Changes to the `costFormatting.ts` shared utility
|
||||
|
|
@ -1,435 +0,0 @@
|
|||
# Unify Cost Calculation Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Replace dual cost calculation systems with a single shared pricing module used by both main and renderer processes.
|
||||
|
||||
**Architecture:** Create `src/shared/utils/pricing.ts` that statically imports `resources/pricing.json` and exports all pricing functions. Both `jsonl.ts` (main) and `sessionAnalyzer.ts` (renderer) consume this module instead of maintaining their own pricing logic.
|
||||
|
||||
**Tech Stack:** TypeScript, Vitest, electron-vite (resolveJsonModule)
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
### Task 1: Create the shared pricing module with tests
|
||||
|
||||
**Files:**
|
||||
- Create: `src/shared/utils/pricing.ts`
|
||||
- Test: `test/shared/utils/pricing.test.ts`
|
||||
|
||||
**Step 1: Write the failing tests**
|
||||
|
||||
Create `test/shared/utils/pricing.test.ts`:
|
||||
|
||||
```typescript
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import {
|
||||
getPricing,
|
||||
calculateTieredCost,
|
||||
calculateMessageCost,
|
||||
getDisplayPricing,
|
||||
} from '@shared/utils/pricing';
|
||||
|
||||
describe('Shared Pricing Module', () => {
|
||||
describe('getPricing', () => {
|
||||
it('should find pricing by exact model name', () => {
|
||||
// Use a model known to exist in pricing.json
|
||||
const pricing = getPricing('claude-3-5-sonnet-20241022');
|
||||
expect(pricing).not.toBeNull();
|
||||
expect(pricing!.input_cost_per_token).toBeGreaterThan(0);
|
||||
expect(pricing!.output_cost_per_token).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('should find pricing case-insensitively', () => {
|
||||
const pricing = getPricing('Claude-3-5-Sonnet-20241022');
|
||||
expect(pricing).not.toBeNull();
|
||||
});
|
||||
|
||||
it('should return null for unknown models', () => {
|
||||
const pricing = getPricing('totally-fake-model-xyz');
|
||||
expect(pricing).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('calculateTieredCost', () => {
|
||||
it('should use base rate for tokens below 200k', () => {
|
||||
const cost = calculateTieredCost(100_000, 0.000003);
|
||||
expect(cost).toBeCloseTo(0.3, 6);
|
||||
});
|
||||
|
||||
it('should apply tiered rate above 200k', () => {
|
||||
const cost = calculateTieredCost(250_000, 0.000003, 0.000006);
|
||||
// (200000 * 0.000003) + (50000 * 0.000006) = 0.6 + 0.3 = 0.9
|
||||
expect(cost).toBeCloseTo(0.9, 6);
|
||||
});
|
||||
|
||||
it('should use base rate when no tiered rate provided', () => {
|
||||
const cost = calculateTieredCost(250_000, 0.000015);
|
||||
expect(cost).toBeCloseTo(3.75, 6);
|
||||
});
|
||||
|
||||
it('should return 0 for zero or negative tokens', () => {
|
||||
expect(calculateTieredCost(0, 0.000003)).toBe(0);
|
||||
expect(calculateTieredCost(-100, 0.000003)).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('calculateMessageCost', () => {
|
||||
it('should compute cost for a known model', () => {
|
||||
// claude-3-5-sonnet-20241022: input=0.000003, output=0.000015
|
||||
const cost = calculateMessageCost('claude-3-5-sonnet-20241022', 1000, 500, 0, 0);
|
||||
// (1000 * 0.000003) + (500 * 0.000015) = 0.003 + 0.0075 = 0.0105
|
||||
expect(cost).toBeCloseTo(0.0105, 6);
|
||||
});
|
||||
|
||||
it('should return 0 for unknown models', () => {
|
||||
const cost = calculateMessageCost('unknown-model', 1000, 500, 0, 0);
|
||||
expect(cost).toBe(0);
|
||||
});
|
||||
|
||||
it('should include cache token costs', () => {
|
||||
const cost = calculateMessageCost('claude-3-5-sonnet-20241022', 1000, 500, 300, 200);
|
||||
expect(cost).toBeGreaterThan(0.0105); // more than just input+output
|
||||
});
|
||||
});
|
||||
|
||||
describe('getDisplayPricing', () => {
|
||||
it('should return per-million rates for a known model', () => {
|
||||
const dp = getDisplayPricing('claude-3-5-sonnet-20241022');
|
||||
expect(dp).not.toBeNull();
|
||||
expect(dp!.input).toBeCloseTo(3.0, 1); // $3/M input
|
||||
expect(dp!.output).toBeCloseTo(15.0, 1); // $15/M output
|
||||
});
|
||||
|
||||
it('should return null for unknown models', () => {
|
||||
expect(getDisplayPricing('unknown-model')).toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Step 2: Run tests to verify they fail**
|
||||
|
||||
Run: `pnpm vitest run test/shared/utils/pricing.test.ts`
|
||||
Expected: FAIL — module does not exist
|
||||
|
||||
**Step 3: Create the shared pricing module**
|
||||
|
||||
Create `src/shared/utils/pricing.ts`:
|
||||
|
||||
```typescript
|
||||
import pricingData from '../../../resources/pricing.json';
|
||||
|
||||
export interface LiteLLMPricing {
|
||||
input_cost_per_token: number;
|
||||
output_cost_per_token: number;
|
||||
cache_creation_input_token_cost?: number;
|
||||
cache_read_input_token_cost?: number;
|
||||
input_cost_per_token_above_200k_tokens?: number;
|
||||
output_cost_per_token_above_200k_tokens?: number;
|
||||
cache_creation_input_token_cost_above_200k_tokens?: number;
|
||||
cache_read_input_token_cost_above_200k_tokens?: number;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export interface DisplayPricing {
|
||||
input: number;
|
||||
output: number;
|
||||
cache_read: number;
|
||||
cache_creation: number;
|
||||
}
|
||||
|
||||
const TIER_THRESHOLD = 200_000;
|
||||
|
||||
const pricingMap = pricingData as Record<string, unknown>;
|
||||
|
||||
function tryGetPricing(key: string): LiteLLMPricing | null {
|
||||
const entry = pricingMap[key];
|
||||
if (
|
||||
entry &&
|
||||
typeof entry === 'object' &&
|
||||
'input_cost_per_token' in entry &&
|
||||
'output_cost_per_token' in entry
|
||||
) {
|
||||
return entry as LiteLLMPricing;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function getPricing(modelName: string): LiteLLMPricing | null {
|
||||
const exact = tryGetPricing(modelName);
|
||||
if (exact) return exact;
|
||||
|
||||
const lowerName = modelName.toLowerCase();
|
||||
const lower = tryGetPricing(lowerName);
|
||||
if (lower) return lower;
|
||||
|
||||
for (const key of Object.keys(pricingMap)) {
|
||||
if (key.toLowerCase() === lowerName) {
|
||||
const match = tryGetPricing(key);
|
||||
if (match) return match;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
export function calculateTieredCost(
|
||||
tokens: number,
|
||||
baseRate: number,
|
||||
tieredRate?: number
|
||||
): number {
|
||||
if (tokens <= 0) return 0;
|
||||
if (!tieredRate || tokens <= TIER_THRESHOLD) {
|
||||
return tokens * baseRate;
|
||||
}
|
||||
const costBelow = TIER_THRESHOLD * baseRate;
|
||||
const costAbove = (tokens - TIER_THRESHOLD) * tieredRate;
|
||||
return costBelow + costAbove;
|
||||
}
|
||||
|
||||
export function calculateMessageCost(
|
||||
modelName: string,
|
||||
inputTokens: number,
|
||||
outputTokens: number,
|
||||
cacheReadTokens: number,
|
||||
cacheCreationTokens: number
|
||||
): number {
|
||||
const pricing = getPricing(modelName);
|
||||
if (!pricing) return 0;
|
||||
|
||||
const inputCost = calculateTieredCost(
|
||||
inputTokens,
|
||||
pricing.input_cost_per_token,
|
||||
pricing.input_cost_per_token_above_200k_tokens
|
||||
);
|
||||
const outputCost = calculateTieredCost(
|
||||
outputTokens,
|
||||
pricing.output_cost_per_token,
|
||||
pricing.output_cost_per_token_above_200k_tokens
|
||||
);
|
||||
const cacheCreationCost = calculateTieredCost(
|
||||
cacheCreationTokens,
|
||||
pricing.cache_creation_input_token_cost ?? 0,
|
||||
pricing.cache_creation_input_token_cost_above_200k_tokens
|
||||
);
|
||||
const cacheReadCost = calculateTieredCost(
|
||||
cacheReadTokens,
|
||||
pricing.cache_read_input_token_cost ?? 0,
|
||||
pricing.cache_read_input_token_cost_above_200k_tokens
|
||||
);
|
||||
|
||||
return inputCost + outputCost + cacheCreationCost + cacheReadCost;
|
||||
}
|
||||
|
||||
export function getDisplayPricing(modelName: string): DisplayPricing | null {
|
||||
const pricing = getPricing(modelName);
|
||||
if (!pricing) return null;
|
||||
|
||||
return {
|
||||
input: pricing.input_cost_per_token * 1_000_000,
|
||||
output: pricing.output_cost_per_token * 1_000_000,
|
||||
cache_read: (pricing.cache_read_input_token_cost ?? 0) * 1_000_000,
|
||||
cache_creation: (pricing.cache_creation_input_token_cost ?? 0) * 1_000_000,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
**Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pnpm vitest run test/shared/utils/pricing.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/shared/utils/pricing.ts test/shared/utils/pricing.test.ts
|
||||
git commit -m "feat: add shared pricing module with LiteLLM data"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Wire jsonl.ts to use the shared pricing module
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/main/utils/jsonl.ts:219-400` (remove pricing functions, update calculateMetrics)
|
||||
- Test: `test/main/utils/costCalculation.test.ts` (update to remove fs mocking)
|
||||
|
||||
**Step 1: Update the cost calculation test to use static imports**
|
||||
|
||||
The existing tests mock `fs.readFileSync` to provide pricing data. Since the shared module uses a static JSON import, the tests should instead test against real pricing data or mock the shared module.
|
||||
|
||||
Update `test/main/utils/costCalculation.test.ts`:
|
||||
- Remove `import * as fs from 'fs'` and `vi.mock('fs')`
|
||||
- Remove `mockPricingData` and the `beforeEach` that mocks `fs.readFileSync`
|
||||
- Update model names in tests to match models that exist in `resources/pricing.json` (the existing `claude-3-5-sonnet-20241022` and `claude-3-opus-20240229` should already be there)
|
||||
- Update expected cost values to match the actual rates from `pricing.json` (verify they match the existing mock data — they should be identical since the mock was based on real rates)
|
||||
- Remove the "pricing data load failure" test (line 409-449) — there's no runtime file loading to fail anymore
|
||||
- Keep all other test cases and assertions as-is
|
||||
|
||||
**Step 2: Run updated tests to verify they fail**
|
||||
|
||||
Run: `pnpm vitest run test/main/utils/costCalculation.test.ts`
|
||||
Expected: FAIL — jsonl.ts still has old imports
|
||||
|
||||
**Step 3: Update jsonl.ts**
|
||||
|
||||
In `src/main/utils/jsonl.ts`:
|
||||
- Remove lines 219-320: the `fs`/`path` imports, `ModelPricing` interface, `TIER_THRESHOLD`, `pricingCache`, `loadPricingData()`, `calculateTieredCost()`, `getPricing()`
|
||||
- Add import at top of file: `import { calculateMessageCost } from '@shared/utils/pricing';`
|
||||
- In `calculateMetrics()` (lines 354-400), replace the inline cost calculation block (lines 374-398) with:
|
||||
|
||||
```typescript
|
||||
if (msg.model) {
|
||||
costUsd += calculateMessageCost(
|
||||
msg.model,
|
||||
msgInputTokens,
|
||||
msgOutputTokens,
|
||||
msgCacheReadTokens,
|
||||
msgCacheCreationTokens
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
- Remove the unused `modelName` variable (line 338) and the block that sets it (lines 370-372)
|
||||
|
||||
**Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `pnpm vitest run test/main/utils/costCalculation.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Run full test suite to check for regressions**
|
||||
|
||||
Run: `pnpm test`
|
||||
Expected: All tests pass
|
||||
|
||||
**Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src/main/utils/jsonl.ts test/main/utils/costCalculation.test.ts
|
||||
git commit -m "refactor: wire jsonl.ts to shared pricing module"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Wire sessionAnalyzer.ts and CostSection.tsx to use the shared pricing module
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/renderer/utils/sessionAnalyzer.ts:32,60-130` (remove local pricing)
|
||||
- Modify: `src/renderer/types/sessionReport.ts:23-28` (update ModelPricing type)
|
||||
- Modify: `src/renderer/components/report/sections/CostSection.tsx:3,10,39-46,204` (update imports and usage)
|
||||
|
||||
**Step 1: Run existing session analyzer tests as baseline**
|
||||
|
||||
Run: `pnpm vitest run test/renderer/utils/sessionAnalyzer.test.ts`
|
||||
Expected: PASS (baseline)
|
||||
|
||||
**Step 2: Update sessionAnalyzer.ts**
|
||||
|
||||
In `src/renderer/utils/sessionAnalyzer.ts`:
|
||||
- Remove the `ModelPricing` import from `@renderer/types/sessionReport` (line 32)
|
||||
- Remove lines 60-130: `MODEL_PRICING` table, `DEFAULT_PRICING`, `getPricing()`, `costUsd()`
|
||||
- Add import: `import { calculateMessageCost, getDisplayPricing } from '@shared/utils/pricing';`
|
||||
- Export `getDisplayPricing` as `getPricing` for backward compat with CostSection: `export { getDisplayPricing as getPricing } from '@shared/utils/pricing';`
|
||||
- Replace `costUsd(model, inpTok, outTok, cr, cc)` at line 476 with `calculateMessageCost(model, inpTok, outTok, cr, cc)`
|
||||
- Replace `costUsd(subagentModel, ...)` at line 900 with `calculateMessageCost(subagentModel, proc.metrics.inputTokens, proc.metrics.outputTokens, proc.metrics.cacheReadTokens, proc.metrics.cacheCreationTokens)`
|
||||
|
||||
**Step 3: Update sessionReport.ts ModelPricing type**
|
||||
|
||||
In `src/renderer/types/sessionReport.ts`:
|
||||
- Replace the existing `ModelPricing` interface (lines 23-28) with a re-export from the shared module:
|
||||
|
||||
```typescript
|
||||
export type { DisplayPricing as ModelPricing } from '@shared/utils/pricing';
|
||||
```
|
||||
|
||||
This keeps backward compatibility — `CostSection.tsx` imports `ModelPricing` from here and expects `{ input, output, cache_read, cache_creation }` which matches `DisplayPricing`.
|
||||
|
||||
**Step 4: Update CostSection.tsx**
|
||||
|
||||
In `src/renderer/components/report/sections/CostSection.tsx`:
|
||||
- Line 3: Change `import { getPricing } from '@renderer/utils/sessionAnalyzer'` to `import { getPricing } from '@renderer/utils/sessionAnalyzer'` — no change needed if we re-export from sessionAnalyzer. Verify the import still resolves.
|
||||
- The `ModelPricing` import from `@renderer/types/sessionReport` (line 10) continues to work via the re-export.
|
||||
- The `CostBreakdownCard` (lines 34-46) uses `pricing.input`, `pricing.output`, etc. as per-million rates — this matches `DisplayPricing` from `getDisplayPricing()`.
|
||||
|
||||
**Step 5: Run session analyzer tests**
|
||||
|
||||
Run: `pnpm vitest run test/renderer/utils/sessionAnalyzer.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
**Step 6: Run full test suite**
|
||||
|
||||
Run: `pnpm test`
|
||||
Expected: All tests pass
|
||||
|
||||
**Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add src/renderer/utils/sessionAnalyzer.ts src/renderer/types/sessionReport.ts src/renderer/components/report/sections/CostSection.tsx
|
||||
git commit -m "refactor: wire session analyzer and CostSection to shared pricing"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Typecheck, lint, and verify the app runs
|
||||
|
||||
**Files:**
|
||||
- No new files — verification only
|
||||
|
||||
**Step 1: Run typecheck**
|
||||
|
||||
Run: `pnpm typecheck`
|
||||
Expected: No errors
|
||||
|
||||
**Step 2: Run linter**
|
||||
|
||||
Run: `pnpm lint:fix`
|
||||
Expected: Clean or auto-fixed
|
||||
|
||||
**Step 3: Run full test suite**
|
||||
|
||||
Run: `pnpm test`
|
||||
Expected: All tests pass
|
||||
|
||||
**Step 4: Run the app and verify cost display**
|
||||
|
||||
Run: `pnpm dev`
|
||||
- Open a session with known token usage
|
||||
- Verify `TokenUsageDisplay` shows cost in the chat view
|
||||
- Open the Session Report tab and verify cost-by-model breakdown renders
|
||||
- Verify CostBreakdownCard expands with per-token-type rates
|
||||
- Confirm chat view cost and report cost show the same total
|
||||
|
||||
**Step 5: Commit any fixes from verification**
|
||||
|
||||
If any fixes were needed, commit them:
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix: address typecheck/lint issues from cost unification"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Clean up dead code from package.json extraResources
|
||||
|
||||
**Files:**
|
||||
- Modify: `package.json` (optional — evaluate if `extraResources` for pricing.json is still needed)
|
||||
|
||||
**Step 1: Check if pricing.json is still loaded at runtime anywhere**
|
||||
|
||||
Search for any remaining `fs.readFileSync` or runtime references to `pricing.json`:
|
||||
|
||||
Run: `grep -r "pricing.json" src/`
|
||||
Expected: Only the static import in `src/shared/utils/pricing.ts`
|
||||
|
||||
**Step 2: Evaluate extraResources**
|
||||
|
||||
If no runtime file loading remains, the `extraResources` entry for `pricing.json` in `package.json` is dead config. However, removing it is low-risk and low-priority — it just means the file gets copied to the app bundle uselessly. Leave it for now unless it causes issues. Document the decision.
|
||||
|
||||
**Step 3: Final commit**
|
||||
|
||||
```bash
|
||||
git add docs/plans/2026-02-23-unify-cost-calculation.md
|
||||
git commit -m "docs: finalize implementation plan for cost unification"
|
||||
```
|
||||
Loading…
Reference in a new issue