Commit graph

9 commits

Author SHA1 Message Date
Luis Novo
301dd4e20a
fix: async insight creation to prevent transaction conflicts (#512)
Migrate insight creation to the command system with automatic retry logic
to prevent SurrealDB transaction conflicts during batch imports.

Changes:
- Add create_insight_command with retry logic for transaction conflicts
- Add run_transformation_command for async transformation execution
- Make Source.add_insight() fire-and-forget (returns command_id)
- Update POST /sources/{id}/insights to return 202 Accepted immediately
- Frontend polls command status until complete, then refreshes
- Auto-update notebook page icon when source gains insights
- Add i18n keys for insight generation feedback

Related to #489
2026-01-31 15:51:27 -03:00
Luis Novo
4e411e0488
feat: add cascade deletion for notebooks with delete preview (#471)
* feat: decrease chunking size for maximum ollama compatibility

* docs: improve i18n info on Claude.md

* feat: add cascade deletion for notebooks with delete preview

- Add Notebook.get_delete_preview() to show counts of affected items
- Add Notebook.delete(delete_exclusive_sources) for cascade deletion
- Always delete notes when notebook is deleted
- Allow user to choose: delete or keep exclusive sources
- Shared sources are always unlinked but never deleted
- Add NotebookDeleteDialog component with radio button options
- Add delete-preview API endpoint
- Update delete endpoint with delete_exclusive_sources param
- Add i18n support for all 5 locales

Closes #77

* docs: remove harcoded config settings
2026-01-25 14:56:14 -03:00
LUIS NOVO
71b8d13b24 docs: generate comprehensive CLAUDE.md reference documentation across codebase
Create a hierarchical CLAUDE.md documentation system for the entire Open Notebook
codebase with focus on concise, pattern-driven reference cards rather than
comprehensive tutorials.

## Changes

### Core Documentation System
- Updated `.claude/commands/build-claude-md.md` to distinguish between leaf and
  parent modules, with special handling for prompt/template modules
- Established clear patterns:
  * Leaf modules (40-70 lines): Components, hooks, API clients
  * Parent modules (50-150 lines): Architecture, cross-layer patterns, data flows
  * Template modules: Pattern focus, not catalog listings

### Generated Documentation
Created 15 CLAUDE.md reference files across the project:

**Frontend (React/Next.js)**
- frontend/src/CLAUDE.md: Architecture overview, data flow, three-tier design
- frontend/src/lib/hooks/CLAUDE.md: React Query patterns, state management
- frontend/src/lib/api/CLAUDE.md: Axios client, FormData handling, interceptors
- frontend/src/lib/stores/CLAUDE.md: Zustand state persistence, auth patterns
- frontend/src/components/ui/CLAUDE.md: Radix UI primitives, CVA styling

**Backend (Python/FastAPI)**
- open_notebook/CLAUDE.md: System architecture, layer interactions
- open_notebook/ai/CLAUDE.md: Model provisioning, Esperanto integration
- open_notebook/domain/CLAUDE.md: Data models, ObjectModel/RecordModel patterns
- open_notebook/database/CLAUDE.md: Repository pattern, async migrations
- open_notebook/graphs/CLAUDE.md: LangGraph workflows, async orchestration
- open_notebook/utils/CLAUDE.md: Cross-cutting utilities, context building
- open_notebook/podcasts/CLAUDE.md: Episode/speaker profiles, job tracking

**API & Other**
- api/CLAUDE.md: REST layer, service architecture
- commands/CLAUDE.md: Async command handlers, job queue patterns
- prompts/CLAUDE.md: Jinja2 templates, prompt engineering patterns (refactored)

**Project Root**
- CLAUDE.md: Project overview, three-tier architecture, tech stack, getting started

### Key Features
- Zero duplication: Parent modules reference child CLAUDE.md files, don't repeat them
- Pattern-focused: Emphasizes how components work together, not component catalogs
- Scannable: Short bullets, code examples only when necessary (1-2 per file)
- Practical: "How to extend" guides, quirks/gotchas for each module
- Navigation: Root CLAUDE.md acts as hub pointing to specialized documentation

### Cleanup
- Removed unused `batch_fix_services.py`
- Removed deprecated `open_notebook/plugins/podcasts.py`
- Updated .gitignore for documentation consistency

## Impact
New contributors can now:
1. Read root CLAUDE.md for system architecture (5 min)
2. Jump to specific layer documentation (frontend, api, open_notebook)
3. Dive into module-specific patterns in child CLAUDE.md files (1 min per module)
All documentation is lean, reference-focused, and avoids duplication.
2026-01-03 16:27:52 -03:00
Justin Florentine
8644f1923c
feat: add delete insight functionality
Add ability to delete insights from sources with a confirmation dialog.
Uses AlertDialog component for a native React experience instead of
browser confirm().

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-18 18:04:41 -05:00
LUIS NOVO
26e989fdbb fix: query key consistency and timeout configuration
- Use QUERY_KEYS.sourcesInfinite for infinite scroll query key
  Starting with ['sources', ...] ensures mutations that invalidate
  ['sources'] will also invalidate the infinite scroll cache

- Use httpx.Timeout for chat service with short connect (10s) and
  long read (600s) timeouts. Prevents 10 min wait on connection errors
2025-12-14 21:55:38 -03:00
LUIS NOVO
3001537aa7 fix: increase timeout for Ollama and local LLM operations
Users with Ollama reported timeout errors on notebook chat while the
backend was still processing. The answer would appear after refresh.

- Frontend axios timeout: 5 min → 10 min
- Backend chat service timeout: 2 min → 10 min

Local LLMs can take several minutes for complex questions with large
contexts, especially on slower hardware.
2025-12-14 11:31:03 -03:00
Luis Novo
992442150e
feat: add ability to link existing sources to notebooks (OSS-311) (#177)
* fix: small issue where users cant change podcast segments

* chore: remove playwright mcp from gut

* feat: add ability to link existing sources to notebooks (OSS-311)

Implemented bidirectional source-notebook linking functionality:

Backend changes:
- Add POST endpoint to link sources to notebooks
- Include notebook associations in source detail response
- Implement idempotent linking with proper RecordID handling

Frontend changes:
- Add AddExistingSourceDialog with search and multi-select
- Add NotebookAssociations component for source detail view
- Add dropdown menu to "Add Source" button (new/existing)
- Implement useAddSourcesToNotebook hook with graceful error handling
- Fix dialog pointer-events during close animation
- Add loading states and disable checkboxes for linked sources
- Optimize dialog width with proper responsive breakpoints

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: address PR review feedback

- Fix sources.py query to use correct reference direction (OUT where IN)
- Remove debug console.log statements
- Add truncation warning for 100+ source lists

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-19 17:52:14 -03:00
Luis Novo
b5666c4d68
Fix/increase fix: increase API client timeouts for transformation operations timeouts (#170)
* fix: increase API client timeouts for transformation operations

- Increase frontend timeout from 30s to 300s (5 minutes)
- Increase Streamlit API client timeout from 30s to 300s
- Add API_CLIENT_TIMEOUT environment variable for configurability
- Add ESPERANTO_LLM_TIMEOUT environment variable documentation
- Update .env.example with comprehensive timeout documentation

Fixes #131 - API timeout errors during transformation generation
Transformations now have sufficient time to complete on slower
hardware (Ollama, LM Studio) without frontend timeout errors.

Users can now configure timeouts for both the API client layer
(API_CLIENT_TIMEOUT) and the LLM provider layer (ESPERANTO_LLM_TIMEOUT)
to accommodate their specific hardware and network conditions.

* docs: add timeout configuration documentation

- Add comprehensive timeout troubleshooting section to common-issues.md
- Add FAQ entry about timeout errors during transformations
- Document API_CLIENT_TIMEOUT and ESPERANTO_LLM_TIMEOUT usage
- Provide specific timeout recommendations for different hardware/network scenarios
- Link to GitHub issue #131 for reference

* chore: bump

* refactor: improve timeout configuration with validation and consistency

Based on PR review feedback, this commit addresses several improvements:

**Timeout Validation:**
- Add validation to ensure timeout values are between 30s and 3600s
- Invalid values fall back to default 300s with warning logs
- Handles edge cases (negative, zero, invalid strings)

**Fix Hard-coded Timeouts:**
- Replace all hard-coded timeout values in api/client.py
- ask_simple: 300s → self.timeout
- execute_transformation: 120s → self.timeout
- embed_content: 120s → self.timeout
- create_source: 300s → self.timeout
- rebuild_embeddings: Uses smart logic (2x timeout, max 3600s)

**Improved Documentation:**
- Add clarifying comments about ms vs seconds (frontend vs backend)
- Document that frontend uses 300000ms = backend 300s
- Add inline documentation for rebuild_embeddings timeout logic

**Development Dependencies:**
- Add pytest>=8.0.0 to dev dependencies for future test coverage

This makes timeout configuration more robust, consistent, and user-friendly
while maintaining backward compatibility.
2025-10-19 11:37:24 -03:00
Luis Novo
b7e656a319
Version 1 (#160)
New front-end
Launch Chat API
Manage Sources
Enable re-embedding of all contents
Sources can be added without a notebook now
Improved settings
Enable model selector on all chats
Background processing for better experience
Dark mode
Improved Notes

Improved Docs: 
- Remove all Streamlit references from documentation
- Update deployment guides with React frontend setup
- Fix Docker environment variables format (SURREAL_URL, SURREAL_PASSWORD)
- Update docker image tag from :latest to :v1-latest
- Change navigation references (Settings → Models to just Models)
- Update development setup to include frontend npm commands
- Add MIGRATION.md guide for users upgrading from Streamlit
- Update quick-start guide with correct environment variables
- Add port 5055 documentation for API access
- Update project structure to reflect frontend/ directory
- Remove outdated source-chat documentation files
2025-10-18 12:46:22 -03:00