71b8d13b24
Create a hierarchical CLAUDE.md documentation system for the entire Open Notebook codebase with focus on concise, pattern-driven reference cards rather than comprehensive tutorials. ## Changes ### Core Documentation System - Updated `.claude/commands/build-claude-md.md` to distinguish between leaf and parent modules, with special handling for prompt/template modules - Established clear patterns: * Leaf modules (40-70 lines): Components, hooks, API clients * Parent modules (50-150 lines): Architecture, cross-layer patterns, data flows * Template modules: Pattern focus, not catalog listings ### Generated Documentation Created 15 CLAUDE.md reference files across the project: **Frontend (React/Next.js)** - frontend/src/CLAUDE.md: Architecture overview, data flow, three-tier design - frontend/src/lib/hooks/CLAUDE.md: React Query patterns, state management - frontend/src/lib/api/CLAUDE.md: Axios client, FormData handling, interceptors - frontend/src/lib/stores/CLAUDE.md: Zustand state persistence, auth patterns - frontend/src/components/ui/CLAUDE.md: Radix UI primitives, CVA styling **Backend (Python/FastAPI)** - open_notebook/CLAUDE.md: System architecture, layer interactions - open_notebook/ai/CLAUDE.md: Model provisioning, Esperanto integration - open_notebook/domain/CLAUDE.md: Data models, ObjectModel/RecordModel patterns - open_notebook/database/CLAUDE.md: Repository pattern, async migrations - open_notebook/graphs/CLAUDE.md: LangGraph workflows, async orchestration - open_notebook/utils/CLAUDE.md: Cross-cutting utilities, context building - open_notebook/podcasts/CLAUDE.md: Episode/speaker profiles, job tracking **API & Other** - api/CLAUDE.md: REST layer, service architecture - commands/CLAUDE.md: Async command handlers, job queue patterns - prompts/CLAUDE.md: Jinja2 templates, prompt engineering patterns (refactored) **Project Root** - CLAUDE.md: Project overview, three-tier architecture, tech stack, getting started ### Key Features - Zero duplication: Parent modules reference child CLAUDE.md files, don't repeat them - Pattern-focused: Emphasizes how components work together, not component catalogs - Scannable: Short bullets, code examples only when necessary (1-2 per file) - Practical: "How to extend" guides, quirks/gotchas for each module - Navigation: Root CLAUDE.md acts as hub pointing to specialized documentation ### Cleanup - Removed unused `batch_fix_services.py` - Removed deprecated `open_notebook/plugins/podcasts.py` - Updated .gitignore for documentation consistency ## Impact New contributors can now: 1. Read root CLAUDE.md for system architecture (5 min) 2. Jump to specific layer documentation (frontend, api, open_notebook) 3. Dive into module-specific patterns in child CLAUDE.md files (1 min per module) All documentation is lean, reference-focused, and avoids duplication.
49 lines
2.9 KiB
Markdown
49 lines
2.9 KiB
Markdown
# Commands Module
|
||
|
||
**Purpose**: Defines async command handlers for long-running operations via `surreal-commands` job queue system.
|
||
|
||
## Key Components
|
||
|
||
- **`process_source_command`**: Ingests content through `source_graph`, creates embeddings (optional), and generates insights. Retries on transaction conflicts (exp. jitter, max 5×).
|
||
- **`embed_single_item_command`**: Embeds individual sources/notes/insights; splits content into chunks for vector storage.
|
||
- **`rebuild_embeddings_command`**: Bulk re-embed all/existing items with selective type filtering.
|
||
- **`generate_podcast_command`**: Creates podcasts via `podcast-creator` library using stored episode/speaker profiles.
|
||
- **`process_text_command`** (example): Test fixture for text operations (uppercase, lowercase, reverse, word_count).
|
||
- **`analyze_data_command`** (example): Test fixture for numeric aggregations.
|
||
|
||
## Important Patterns
|
||
|
||
- **Pydantic I/O**: All commands use `CommandInput`/`CommandOutput` subclasses for type safety and serialization.
|
||
- **Error handling**: Permanent errors return failure output; `RuntimeError` exceptions auto-retry via surreal-commands.
|
||
- **Model dumping**: Recursive `full_model_dump()` utility converts Pydantic models → dicts for DB/API responses.
|
||
- **Logging**: Uses `loguru.logger` throughout; logs execution start/end and key metrics (processing time, counts).
|
||
- **Time tracking**: All commands measure `start_time` → `processing_time` for monitoring.
|
||
|
||
## Dependencies
|
||
|
||
**External**: `surreal_commands` (command decorator, job queue), `loguru`, `pydantic`, `podcast_creator`
|
||
**Internal**: `open_notebook.domain.*` (Source, Note, Transformation), `open_notebook.graphs.source`, `open_notebook.ai.models`
|
||
|
||
## Quirks & Edge Cases
|
||
|
||
- **source_commands**: `ensure_record_id()` wraps command IDs for DB storage; transaction conflicts trigger exponential backoff retry (1-30s). Non-`RuntimeError` exceptions are permanent.
|
||
- **embedding_commands**: Queries DB directly for item state; chunk index must match source's chunk list. Model availability checked at command start.
|
||
- **podcast_commands**: Profiles loaded from SurrealDB by name (must exist); briefing can be extended with suffix. Episode records created mid-execution.
|
||
- **Example commands**: Accept optional `delay_seconds` for testing async behavior; not for production.
|
||
|
||
## Code Example
|
||
|
||
```python
|
||
@command("process_source", app="open_notebook", retry={...})
|
||
async def process_source_command(input_data: SourceProcessingInput) -> SourceProcessingOutput:
|
||
start_time = time.time()
|
||
try:
|
||
transformations = [await Transformation.get(id) for id in input_data.transformations]
|
||
source = await Source.get(input_data.source_id)
|
||
result = await source_graph.ainvoke({...})
|
||
return SourceProcessingOutput(success=True, ...)
|
||
except RuntimeError as e:
|
||
raise # Retry this
|
||
except Exception as e:
|
||
return SourceProcessingOutput(success=False, error_message=str(e))
|
||
```
|