Commit graph

92 commits

Author SHA1 Message Date
Luis Novo
b7bba2461c fix: bump esperanto to 2.19.7 to fix base_url/api_key config in multiple embedding providers
Fixes the same kwargs vs self.* issue found in Azure, OpenAI, Voyage,
Google, and Jina embedding providers.
2026-03-11 15:51:52 -05:00
Luis Novo
06f8be8409 fix: bump esperanto to 2.19.6 to fix Ollama embedding base_url
The OllamaEmbeddingModel was ignoring the base_url from credentials/config,
always falling back to env vars or localhost. This caused embedding failures
for users with custom Ollama endpoints.

Fixes #655
2026-03-11 14:46:16 -05:00
Luis Novo
803d9710c5 chore: bump version to 1.8.1 2026-03-10 20:20:16 -05:00
Luis Novo
d6b76f63a8
fix(deps): bump esperanto to 2.19.5 (#657) 2026-03-10 18:35:09 -03:00
Luis Novo
7910f683f6
fix(podcasts): enable language support by bumping podcast-creator to 0.12.0 (#645)
The language field on EpisodeProfile was being saved to the database but
had no effect during generation because podcast-creator 0.11.x didn't
support the language parameter. Version 0.12.0 adds language support to
the generation pipeline (outline + transcript templates), and since
open-notebook already passes the full episode profile config to
podcast-creator, the language field is picked up automatically.

Closes #640
2026-03-03 11:50:16 -03:00
Luis Novo
eac837d555
feat(podcasts): model registry integration, credential passthrough & new features (#632)
* feat(podcasts): integrate model registry for profiles and credential passthrough

Replace loose provider/model string fields with record<model> references
in podcast profiles, enabling credential passthrough to podcast-creator.

Backend:
- EpisodeProfile: outline_llm, transcript_llm (record<model>) replace
  outline_provider/outline_model strings. New language field (BCP 47).
- SpeakerProfile: voice_model (record<model>) replaces tts_provider/
  tts_model strings. Per-speaker voice_model override support.
- Migration 14: schema changes making legacy fields optional, adding new
  record<model> fields.
- Data migration (migration.py): auto-converts legacy profiles to model
  registry references on startup. Idempotent.
- podcast_commands.py: resolves credentials for ALL profiles before
  calling podcast-creator.
- New /api/languages endpoint (pycountry + babel) with BCP 47 locale
  codes (pt-BR, en-US, etc.).

Frontend:
- Episode/speaker profile forms use ModelSelector instead of manual
  provider/model dropdowns.
- Language dropdown with BCP 47 codes in episode profile form.
- Per-speaker TTS voice model override in speaker profile form.
- "Templates" tab renamed to "Profiles".
- Setup required badge on unconfigured profiles.
- i18n updated across all 8 locales.

Closes #486, closes #552

* fix(i18n): remove unused legacy podcast provider/model keys

Remove 10 orphaned i18n keys across all 8 locales that were left behind
after replacing manual provider/model dropdowns with ModelSelector.

* fix: address review violations in podcast model registry

- P1: Remove profiles with failed model resolution from dicts to prevent
  podcast-creator validation errors on unrelated profiles
- P2: Use centralized QUERY_KEYS.languages instead of inline key
- P3: Fix ISO 639-1 → BCP 47 in model field description and CLAUDE.md
- P3: Update "templates" → "profiles" in locale string values (all 8)

* chore: bump version to 1.8.0
2026-02-27 11:06:47 -03:00
Luis Novo
5d84ab0768 fix: embedding batch sizing and 413 error classification (1.7.4)
- Add batching to generate_embeddings() (50 texts per batch with per-batch retry)
  to prevent 413 Payload Too Large errors on large documents
- Add 413 error classification rule for user-friendly error messages
- Fix misleading "Created 0 embedded chunks" log in process_source_command
  by removing premature get_embedded_chunks() call (embedding is fire-and-forget)

Closes #594
2026-02-18 11:39:47 -03:00
Luis Novo
c666966b8c
fix: podcast failure recovery and retry (1.7.3) (#595)
* fix: surface podcast errors and enable retry for failed episodes

Fixes #335, #300

Re-raise exceptions in podcast command so surreal-commands marks jobs as
failed instead of completed. Surface error_message in API responses and
add a retry endpoint that deletes the failed episode and re-submits the
generation job. Frontend shows error details on failed episodes with a
retry button. Translations added for all 8 locales.

* fix: bump podcast-creator to >= 0.10

Fixes #302

* chore: release 1.7.3 - podcast failure recovery and retry

Bump podcast-creator to >= 0.11.2, disable automatic retries for
podcast generation to prevent duplicate episodes, and bump version
to 1.7.3.

Fixes #211, #218, #185, #355, #300, #302

* fix: resolve TypeScript error in handleRetry return type
2026-02-17 21:24:57 -03:00
Luis Novo
189a30c570 fix: bump podcast-creator to >= 0.9.4
Fixes #211
2026-02-17 17:32:34 -03:00
Luis Novo
20e18fdd0d feat: improve error clarity for LLM provider failures (#506)
Replace generic "An unexpected error occurred" messages with descriptive,
user-friendly error messages when LLM operations fail. Errors like invalid
API keys, wrong model names, and rate limits now surface clearly in the UI.

Adds error classification utility, global FastAPI exception handlers, and
frontend getApiErrorMessage() helper. Bumps version to 1.7.2.
2026-02-16 16:15:46 -03:00
Luis Novo
115e1cc3e8 chore: bump podcast-creator to 0.9.1 2026-02-16 15:27:29 -03:00
Luis Novo
e66111b0de fix: bump esperanto to 2.19.3 to fix openai_compatible provider name
Esperanto 2.19.3 normalizes provider names by converting underscores
to hyphens, fixing the ValueError when using openai_compatible.

Closes #570
2026-02-15 08:32:22 -03:00
Luis Novo
78ae2096e6 chore: bump version to 1.7.1 2026-02-14 21:06:00 -03:00
Luis Novo
9b507f111c
fix: update esperanto to fix ElevenLabs TTS credential passthrough (#578)
Esperanto's AIFactory.create_text_to_speech() did not accept a config
dict like the other factory methods, so credentials configured via the
UI were not passed through. Fixed upstream in esperanto 2.9.2.

Refs #571
2026-02-14 19:12:49 -03:00
Luis Novo
877c303b02
fix: update esperanto dep and increase transformation max_tokens (#568)
* fix: increase transformation max_tokens from 5055 to 8192

Closes #565

* chore: update esperanto dep to fix api keys passing via config - fixes: #567
2026-02-12 07:33:27 -03:00
Luis Novo
3cb8c73cf1
chore: bump version to 1.7.0 (#554) 2026-02-10 08:36:32 -03:00
Luis Novo
3f352cfcce
feat: credential-based API key management (#477) (#540)
* feat: replace provider config with credential-based system (#477)

Introduce a new credential management system replacing the old
ProviderConfig singleton and standalone Models page. Each credential
stores encrypted API keys and provider-specific configuration with
full CRUD support via a unified settings UI.

Backend:
- Add Credential domain model with encrypted API key storage
- Add credentials API router (CRUD, discovery, registration, testing)
- Add encryption utilities for secure key storage
- Add key_provider for DB-first env-var fallback provisioning
- Add connection tester and model discovery services
- Integrate ModelManager with credential-based config
- Add provider name normalization for Esperanto compatibility
- Add database migrations 11-12 for credential schema

Frontend:
- Rewrite settings/api-keys page with credential management UI
- Add model discovery dialog with search and custom model support
- Add compact default model assignments (primary/advanced layout)
- Add inline model testing and credential connection testing
- Add env-var migration banner
- Update navigation to unified settings page
- Remove standalone models page and old settings components

i18n:
- Update all 7 locale files with credential and model management keys

Closes #477

Co-Authored-By: JFMD <git@jfmd.us>
Co-Authored-By: OraCatQAQ <570768706@qq.com>

* fix: address PR #540 review comments

- Fix docs referencing removed Models page
- Fix error-handler returning raw messages instead of i18n keys
- Fix auth.py misleading docstring and missing no-password guard
- Fix connection_tester using wrong env var for openai_compatible
- Add provision_provider_keys before model discovery/sync
- Update CLAUDE.md to reflect credential-based system
- Fix missing closing brace in api-keys page useEffect

* fix: add logging to credential migration and surface errors in UI

- Add comprehensive logging to migrate-from-env and
  migrate-from-provider-config endpoints (start, per-provider
  progress, success/failure with stack traces, final summary)
- Fix frontend migration hooks ignoring errors array from response
- Show error toast when migration fails instead of "nothing to migrate"
- Invalidate status/envStatus queries after migration so banner updates

* docs: update CLAUDE.md files for credential system

Replace stale ProviderConfig and /api-keys/ references across 8 CLAUDE.md
files to reflect the new Credential-based system from PR #540.

* docs: update user documentation for credential-based system

Replace env var API key instructions with Settings UI credential
workflow across all user-facing documentation. The new flow is:
set OPEN_NOTEBOOK_ENCRYPTION_KEY → start services → add credential
in Settings UI → test → discover models → register.

- Rewrite ai-providers.md, api-configuration.md, environment-reference.md
- Update all quick-start guides and installation docs
- Update ollama.md, openai-compatible.md, local-tts/stt networking sections
- Update reverse-proxy.md, development-setup.md, security.md
- Fix broken links to non-existent docs/deployment/ paths
- Add credentials endpoints to api-reference.md
- Move all API key env vars to deprecated/legacy sections

* chore: bump version to 1.7.0-rc1

Release candidate for credential-based provider management system.

* fix: initialize provider before try block in test_credential

Prevents UnboundLocalError when Credential.get() throws (e.g.,
invalid credential_id) before provider is assigned.

* fix: reorder down migration to drop index before table

Removes duplicate REMOVE FIELD statement and reorders so the index
is dropped before the table, preventing rollback failures.

* refactor: simplify encryption key to always derive via SHA-256

Remove the dual code path in _ensure_fernet_key() that detected native
Fernet keys. Since the credential system is new, always deriving via
SHA-256 removes unnecessary complexity. Also removes the generate_key()
function and Fernet.generate_key() references from docs.

* fix: correct mock patch targets in embedding tests and URL validation

Fix embedding tests patching wrong module path for model_manager
(was targeting open_notebook.utils.embedding.model_manager but it's
imported locally from open_notebook.ai.models). Also fix URL validation
to allow unresolvable hostnames since they may be valid in the
deployment environment (e.g., Azure endpoints, internal DNS).

* feat: add global setup banner for encryption and migration status

Show a persistent banner in AppShell when encryption key is missing
(red) or env var API keys can be migrated (amber), so users see
these prompts on every page instead of only on Settings > API Keys.

Includes a docs link for the encryption banner and i18n support
across all 7 locales.

* docs: several improvements to docker-compose e env examples

* Update README.md

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* docs: fix env var format in README and update model setup instructions

Align the encryption key snippet in README Step 2 with the list
format used in the compose file. Replace deprecated "Settings →
Models" instructions with credential-based Discover Models flow.

* fix: address credential system review issues

- Fix SSRF bypass via IPv4-mapped IPv6 addresses (::ffff:169.254.x.x)
- Fix TTS connection test missing config parameter
- Add Azure-specific model discovery using api-key auth header
- Add Vertex static model list for credential-based discovery
- Fix PROVIDER_DISCOVERY_FUNCTIONS incorrect azure/vertex mapping
- Extract business logic to api/credentials_service.py (service layer)
- Move credential Pydantic schemas to api/models.py
- Update tests to use new service imports and ValueError assertions

* fix: sanitize error responses and migrate key_provider to Credential

- Replace raw exception messages in all credential router 500 responses
  with generic error strings (internal details logged server-side only)
- Refactor key_provider.py to use Credential.get_by_provider() instead
  of deprecated ProviderConfig.get_instance()
- Remove unused functions (get_provider_configs, get_default_api_key,
  get_provider_config) that were dead code

---------

Co-authored-by: JFMD <git@jfmd.us>
Co-authored-by: OraCatQAQ <570768706@qq.com>
2026-02-10 08:30:22 -03:00
Luis Novo
c4ed1b18ec
chore: bump surreal-commands to 1.3.1 (#517)
* chore: bump surreal-commands to 1.3.1

Required for improved retry logging with command_id in error messages.

Ref #513

* chore: update uv.lock for surreal-commands 1.3.1
2026-01-31 19:00:15 -03:00
Luis Novo
03f9edfec2
feat: use standard HTTP_PROXY/HTTPS_PROXY environment variables (#499)
Update proxy configuration to use industry-standard environment variables
(HTTP_PROXY, HTTPS_PROXY, NO_PROXY) instead of custom variables.

The underlying libraries (esperanto, content-core, podcast-creator)
now automatically detect proxy settings from these standard variables.

- Bump content-core>=1.14.1 (fixes #494)
- Bump esperanto>=2.18
- Bump podcast-creator>=0.9
- Update documentation with new proxy configuration
2026-01-29 23:31:02 -03:00
Luis Novo
6dc9a3db50
feat: detect HTML content in clipboard for text sources (#475)
* chore: bump content-core to support html to markdown

* feat: detect HTML content in clipboard for text sources

- Add paste handler to detect text/html format in clipboard
- Use HTML content instead of plain text when available
- Display info message when HTML is detected
- Add translations for all supported languages (en-US, pt-BR, ja-JP, zh-CN, zh-TW)

* fix: reset HTML detection banner on plain text paste

Clear the hasHtmlContent flag when pasting plain text (no HTML in
clipboard) so the banner doesn't persist incorrectly after replacing
HTML content with plain text.
2026-01-25 21:36:58 -03:00
Luis Novo
28936d3944
fix: connection error with llama.cpp and OpenAI-compatible providers (#466)
* docs: update CHANGELOG for v1.6.0 release

* fix: connection error with llama.cpp and OpenAI-compatible providers

Bump Esperanto to 2.17.2 which fixes LangChain connection errors
caused by garbage collection closing shared HTTP clients.

Closes #465
2026-01-24 09:39:38 -03:00
Luis Novo
47c513edfd
fix: improve error logging for chat model configuration issues (#458)
* docs: update CHANGELOG for v1.6.0 release

* fix: improve error logging for chat model configuration issues (#358)

- Add detailed error logging in provision.py when model lookup fails
- Add warning logging in models.py when default model is not configured
- Add traceback logging in chat router exception handler
- Update Ollama docs with model name configuration guidance
- Update troubleshooting docs with "Failed to send message" solutions
- Bump version to 1.6.1

* chore: uvlock
2026-01-23 16:45:13 -03:00
Luis Novo
d8006ff5cb
feat: content-type aware chunking and unified embedding (#444)
* feat: content-type aware chunking and unified embedding

- Add chunking.py with HTML, Markdown, and plain text detection
- Add embedding.py with mean pooling for large content
- Create dedicated commands: embed_note, embed_insight, embed_source
- Use fire-and-forget pattern for embedding via submit_command()
- Refactor rebuild_embeddings_command to delegate to individual commands
- Remove legacy commands and needs_embedding() methods
- Reduce chunk size to 1500 chars for Ollama compatibility
- Update CLAUDE.md documentation for new architecture

Fixes #350, #142

* fix: address code review issues

- Note.save() now returns command_id for tracking embedding jobs
- Add length check after generate_embeddings() to fail fast on mismatch
- Add numpy as explicit dependency (was transitive)
- Remove hardcoded chunk sizes from docstrings

* docs: address code review comments

- Rename "SYNC PATH" to "DOMAIN MODEL PATH" in embedding router
- Add test_chunking.py and test_embedding.py to Testing Strategy
- Clarify auto-embedding behavior for each domain model

* fix: clean thinking tags from prompt graph output

Adds clean_thinking_content() to prompt.py to handle extended thinking
models that return <think>...</think> tags. This fixes empty titles
when saving notes from chat.

* chore: remove local docker-compose from git

* fix(frontend): handle null parent_id in search results

Add defensive check for null parent_id in search results to prevent
"Cannot read properties of null (reading 'split')" error. This can
happen with orphaned records in the database.

* fix: cascade delete embeddings and insights when source is deleted

When deleting a Source, now also deletes associated:
- source_embedding records
- source_insight records

This prevents orphaned records that cause null parent_id errors
in vector search results.

* fix: add cleanup for orphan embedding/insight records in migration 10

Deletes source_embedding and source_insight records where the
linked source no longer exists (source.id = NONE).

* chore: bump esperanto to 2.16

Increases ctx_num for Ollama models to accommodate larger notebook
context windows. See: https://github.com/lfnovo/esperanto/pull/69
2026-01-21 23:49:08 -03:00
Luis Novo
da8c98b178
chore: bump version to 1.5.2 and update CHANGELOG (#437) 2026-01-15 22:35:29 -03:00
Luis Novo
c6ec1fcddf
fix(i18n): resolve podcast dialog translation infinite loop and profile issues (#435)
* fix(i18n): resolve podcast dialog translation infinite loop and profile issues

- Remove incorrect translation keys for user-defined episode profiles
- Cache translation strings in ContentSelectionPanel to avoid repeated
  Proxy accesses that triggered infinite loop detection
- Stabilize useEffect dependencies with dataKey pattern to prevent
  re-initialization on every keystroke
- Replace unstable sourcesQueries prop with stable fetchingNotebookIds set
- Clean up unused getSourceModes function and TranslationKeys import

* chore: bump lock

* chore: bump version to 1.5.1 and update CHANGELOG

* fix: guard .join() call in dataKey when query data is undefined
2026-01-15 21:50:27 -03:00
Luis Novo
b7ff0ccfe9
chore: post-i18n cleanup and version bump to 1.5.0 (#433)
* chore: post-i18n cleanup and version bump to 1.5.0

- Restore missing .dockerignore entries (notebook_data, surreal_data, docs, etc.)
- Fix lint command for Next.js 16 (use eslint directly instead of next lint)
- Remove aria-describedby={undefined} causing Radix UI warnings
- Bump version to 1.5.0
- Update CHANGELOG with i18n features
- Add multi-language UI mention to README
- Add i18n contribution guide to README.dev
- Document i18n system in CLAUDE.md files

Closes #344, #349, #360

* docs: fix provider order in CLAUDE.md to match layout.tsx
2026-01-15 14:20:13 -03:00
LUIS NOVO
e7c3ef0520 chore: bump to 1.4 2026-01-14 13:08:06 -03:00
LUIS NOVO
52177f7546 fix: add CORS headers to error responses and document file upload limits
- Added custom exception handler to ensure CORS headers are included in
  all HTTP error responses from the API
- Added documentation for 413 (Payload Too Large) errors when behind
  reverse proxies (nginx, traefik, kubernetes ingress)
- Added client_max_body_size to nginx configuration examples
- Documented how to configure CORS headers for proxy-level error responses

Fixes #401
2026-01-09 20:08:13 -03:00
LUIS NOVO
48e2800211 fix: reduce retry log noise during concurrent chunk processing
Addresses issue #362 - users were seeing hundreds of ERROR/WARNING logs
when processing large documents due to SurrealDB v2 transaction conflicts
during concurrent chunk embedding operations.

Changes:
- Upgraded to surreal-commands v1.3.0 which includes retry_log_level feature
- Increased retry attempts from 5 to 15 with max wait time 120s (from 30s)
  to handle deep queues during concurrent processing
- Set retry_log_level to "debug" in embed_chunk and process_source commands
- Changed repository.py RuntimeError logging from ERROR to DEBUG level
- Updated command exception handlers to log retries at DEBUG level
- Updated documentation to reflect retry strategy

This is a temporary workaround for SurrealDB v2.x transaction conflict
issues with SEARCH indexes. Settings can be reduced after migrating to
SurrealDB v3 which fixes the underlying concurrency issue.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-05 11:30:55 -03:00
LUIS NOVO
b1bd522c5c feat: improve dev commands, update all langchain dependencies to their latest major versions 2026-01-05 08:22:41 -03:00
LUIS NOVO
77feff344f chore: bump 2026-01-04 11:44:58 -03:00
LUIS NOVO
1be8ef1116 chore: bump to 1.2.4 2025-12-14 21:34:49 -03:00
Luis Novo
5d5b6bd035
feat(ui): add command palette for quick navigation and search (#288)
* feat(ui): add command palette for quick navigation and search

Replace top bar search with a command palette (⌘K / Ctrl+K) that provides:
- Quick navigation to all app sections
- Create shortcuts for sources, notebooks, and podcasts
- Theme switching (light/dark/system)
- Search and Ask functionality for non-matching queries

This approach saves screen real estate while providing faster access
to common actions through keyboard shortcuts.

Co-authored-by: EmbroiderSnow <1497411439@qq.com>

* chore: bump to 1.2.3

* feat(command-palette): add notebook quick navigation

Users can now type a notebook name in the command palette (⌘K) to
navigate directly to that notebook. Shows up to 8 most recent
notebooks, with cmdk filtering all notebooks when typing.

* fix(command-palette): address code review issues

- Skip ⌘K/Ctrl+K shortcut when focus is inside input, textarea, select,
  or contentEditable elements to preserve native keyboard handling
- Remove 8-item limit on notebooks so all notebooks are searchable
  via cmdk filtering

* perf(command-palette): memoize command matching and add platform shortcuts

- Memoize hasCommandMatch computation with useMemo to avoid
  recalculating on every render
- Show platform-specific keyboard shortcut in sidebar hint:
  ⌘K on macOS, Ctrl+K on Windows/Linux

* fix(command-palette): add spinner to notebooks loading state

Show a spinning Loader2 icon alongside the "Loading notebooks..." text
for clearer visual feedback when the command palette is fetching data.

---------

Co-authored-by: EmbroiderSnow <1497411439@qq.com>
2025-12-01 14:59:17 -03:00
Luis Novo
45a99831a9
Hide sources notes (#273)
* fix: add missing overflow wrapper to notebooks list page

Adds flex-1 overflow-y-auto wrapper to enable proper scrolling
when notebook list exceeds viewport height. Matches the layout
pattern used by all other dashboard pages.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: reorder transformation routes to prevent dynamic route interception

Moved static routes (/transformations/execute and /transformations/default-prompt)
before dynamic routes (/transformations/{transformation_id}) to ensure FastAPI
matches them correctly. Previously, requests to static routes were incorrectly
captured by the dynamic route handler.

Fixes #250

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: bump to 1.2.1

* hide source and notes panel - fixes #193

* feat: improve layout for mobile views

* bump version to 1.2.2

* fix: address PR review feedback for collapsible columns

- Remove unused CollapseButton component from CollapsibleColumn.tsx
- Rename useCollapseButton to createCollapseButton (not a React hook)
- Move dialogs outside Card in SourcesColumn.tsx for consistency
- Add useMemo for collapseButton in both columns to prevent re-renders

* feat: support multiple sources

* fix: prevent ChatColumn double mounting on desktop

Add useIsDesktop hook to conditionally render mobile view only on
mobile screens. Previously, the mobile ChatColumn was hidden via CSS
on desktop but still mounted, causing duplicate hooks initialization
and redundant network requests.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-25 16:59:26 -03:00
Luis Novo
b42cc06e65
fix: UI scrolling and API route ordering issues (#253)
* fix: add missing overflow wrapper to notebooks list page

Adds flex-1 overflow-y-auto wrapper to enable proper scrolling
when notebook list exceeds viewport height. Matches the layout
pattern used by all other dashboard pages.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: reorder transformation routes to prevent dynamic route interception

Moved static routes (/transformations/execute and /transformations/default-prompt)
before dynamic routes (/transformations/{transformation_id}) to ensure FastAPI
matches them correctly. Previously, requests to static routes were incorrectly
captured by the dynamic route handler.

Fixes #250

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: bump to 1.2.1

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-04 21:15:00 -03:00
Luis Novo
6fe78a64f7
chore: bump esperanto for anthropic on langchain (#244) 2025-11-01 15:32:52 -03:00
Luis Novo
f79a9040ae
Release 1.2 (#242)
* chore: improve podcast transcripts

* fix: remove date from insight - fixes #241

* fix: improve scrolling on source and insights - fixes #237

* chore: update esperanto to fix: #234

* chore: update esperanto to fix #226

* fix: process vectorization as subcommands to handle larger documents more gracefully - fix: #229

* feat: enable background job retry capabilities

* feat: reenable content types that were disabled during alpha version

* fix: remove unnecessary model caching causing many issues.

* feat: support multiple azure endpoints and keys just like openai compatible. Fixes #215

* docs: update azure variables

* chore: bump and update dependencies
2025-11-01 14:40:00 -03:00
Luis Novo
a287d3b248
refactor: optimize duplicate model validation and improve error handling (#219)
* feat: prevent duplicate model names under same provider

Implement case-insensitive validation to prevent users from creating
duplicate model names under the same provider. This validation is
implemented both in the backend API and the frontend UI.

Changes:
- Backend: Add duplicate check in create_model endpoint (case-insensitive)
- Frontend: Add client-side validation in AddModelForm
- Frontend: Improve error message display from backend
- Tests: Add unit tests for duplicate model validation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: optimize duplicate model validation and improve error handling

- Replace O(n) model iteration with efficient SurrealDB query for duplicate check
- Improve error message to include model name and provider for better UX
- Remove frontend duplicate validation (backend-only enforcement)
- Fix test authentication by setting OPEN_NOTEBOOK_PASSWORD before imports
- Update test mocking to use repo_query instead of Model.get_all()
- Add pytest fixture for TestClient to ensure proper test isolation

All 11 tests passing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* remove unnecessary package

* fix: replace any with unknown type in error handler

- Change error type from 'any' to 'unknown' to satisfy ESLint
- Add proper type assertion for error object structure
- Maintains same runtime behavior with better type safety

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-25 08:48:18 -03:00
Luis Novo
a0a2282bfa
Delete note functionality (#216)
* feat: Enable note deletion.

* enable dismissing the upgrade button

* delete notes

* fix chat session dialog error

* chore: bump

* chore: bump correctly
2025-10-24 18:27:02 -03:00
Luis Novo
9bdfd99f1b
feat: simplify reverse proxy configuration with Next.js rewrites (#213)
* feat: simplify reverse proxy configuration with Next.js rewrites

Add Next.js API rewrites to proxy /api/* requests internally from port 8502
to the FastAPI backend on port 5055. This eliminates the need for complex
reverse proxy configurations with multiple upstreams and location blocks.

Changes:
- Add rewrites to next.config.ts proxying /api/* to INTERNAL_API_URL
- Introduce INTERNAL_API_URL env var (defaults to http://localhost:5055)
- Update supervisord configs to pass INTERNAL_API_URL to Next.js
- Document INTERNAL_API_URL in .env.example with usage examples
- Add simplified reverse proxy examples for nginx, Traefik, Caddy, Coolify
- Update README architecture diagram to show internal proxying
- Add explanatory comments to _config route handler

Benefits:
- Reduces reverse proxy config from 12 lines to 3 (75% reduction)
- Single-port deployment (8502 only) for 95% of use cases
- Zero breaking changes - backward compatible with existing setups
- Zero performance overhead (validated through testing)
- Preserves proxy headers (X-Forwarded-*) for rate limiting/SSL

Resolves: #179
Related: OSS-321

* fix: rename _config to config to fix production routing

CRITICAL BUG FIX: The /_config endpoint has never worked in production builds
because Next.js treats folders starting with underscore as "private folders"
and excludes them from routing entirely.

This endpoint is critical for:
- Providing API_URL to the browser at runtime
- Enabling zero-config deployments with auto-detection
- Supporting reverse proxy scenarios where API URL differs from frontend URL

Changes:
- Rename frontend/src/app/_config/ → frontend/src/app/config/
- Update client code references (/_config → /config)
- Update documentation with correct endpoint path
- Bump version to 1.1.0 (minor version for new rewrites feature + bug fix)

Impact:
- Runtime configuration now works in production builds
- /config returns {"apiUrl":"http://localhost:5055"} correctly
- Auto-detection for reverse proxy deployments now functional

Related: #179, OSS-321

* fix: resolve React hook exhaustive-deps warning in AddExistingSourceDialog

Wrap performSearch function in useCallback to properly memoize it and satisfy
React Hook exhaustive-deps rule. This prevents unnecessary re-renders and
ensures the useEffect dependency array is correctly specified.

Changes:
- Import useCallback from React
- Wrap performSearch with useCallback([debouncedSearchQuery, allSources])
- Add performSearch to useEffect dependency array

* final fixes
2025-10-24 11:24:14 -03:00
Luis Novo
18b4dfdb77
Claude/add initial tests 011 cukte9g4 qwj hjw7 g3ny rf (#190)
* test: add comprehensive unit tests for domain module

Add 24 comprehensive unit tests covering the open_notebook.domain module:

**ObjectModel Base (5 tests)**
- Create and update operations with timestamps
- Get by ID with class resolution
- Delete validation
- Relationship creation

**RecordModel Singleton (3 tests)**
- Singleton pattern behavior
- Async database loading
- Update persistence

**ModelManager (3 tests)**
- Singleton pattern
- Model instance caching
- Default model retrieval

**Notebook Domain (3 tests)**
- Name validation (empty/whitespace)
- Source relationship queries
- Archived flag defaults

**Source Domain (3 tests)**
- Text vectorization and chunking
- Insight validation and creation
- RecordID command field parsing

**Note Domain (2 tests)**
- Content validation
- Embedding configuration

**Podcast Domain (2 tests)**
- Speaker profile validation
- Episode profile segment validation

**Additional Tests (3 tests)**
- ChatSession relationships
- Transformation creation
- ContentSettings defaults

All tests use proper mocking to avoid database dependencies and validate
both business logic and error handling. Tests follow pytest best practices
with async support, fixtures, and comprehensive assertions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* test: add comprehensive tests for utils and graphs modules

Add 56 new unit tests covering utils and graphs modules:

**Utils Module Tests (36 tests)**

Text Utilities (13 tests):
- Text splitting with various chunk sizes
- ASCII and non-printable character removal
- Thinking tag parsing and cleaning (single/multiple tags)
- Edge cases (empty strings, invalid input, large content)

Token Utilities (4 tests):
- Token counting with tiktoken
- Cost calculation
- Fallback behavior when tiktoken unavailable

Version Utilities (7 tests):
- Semantic version comparison (equal, less, greater, prerelease)
- Installed package version retrieval
- GitHub version fetching with URL validation

Context Builder (12 tests):
- ContextItem and ContextConfig creation
- Builder initialization with various parameters
- Priority sorting and deduplication
- Token-based truncation
- Response formatting
- Source and notebook context building
- Convenience functions

**Graphs Module Tests (20 tests)**

Model Provisioning (4 tests):
- Default model selection
- Large context model triggering (>105k tokens)
- Specific model ID selection
- Kwargs pass-through

Tools (3 tests):
- Current timestamp format validation
- Timestamp validity checking
- Tool decoration verification

Prompt Graph (5 tests):
- PatternChainState structure
- Model calling with/without parser
- Graph compilation and execution

Transformation Graph (8 tests):
- TransformationState structure
- Transformation with source objects
- Transformation with direct input text
- Thinking content cleaning
- Content validation
- Graph compilation and execution
- Default prompt integration

All tests use proper mocking to avoid external dependencies (network,
database) and validate both success paths and error handling.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* improve tests

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 16:54:59 -03:00
Luis Novo
fc8a4a0c64
fix: resolve API_URL config routing conflict with reverse proxies (#191)
Move runtime configuration endpoint from /api/runtime-config to /_config to avoid
conflicts with reverse proxies that route all /api/* requests to the FastAPI backend.

This fixes an issue where users with reverse proxies would see port 5055 incorrectly
appended to their API_URL even when explicitly set via environment variable.

Changes:
- Move frontend/src/app/api/runtime-config/route.ts to frontend/src/app/_config/route.ts
- Update config.ts to fetch from /_config instead of /api/runtime-config
- Add troubleshooting documentation for reverse proxy users
- Update all reverse proxy examples to show correct routing (catch-all handles /_config)
- Bump version to 1.0.11

The new /_config endpoint is automatically handled by standard reverse proxy catch-all
rules (location / { proxy_pass http://frontend; }), requiring no additional configuration
for most users.

Fixes issue where API_URL environment variable was being ignored in reverse proxy setups,
causing CORS errors with "Status code: (null)" and incorrect port 5055 being added.
2025-10-21 12:06:24 -03:00
Luis Novo
305c26fe92
fix: fix supervisor env variables not being set (#183) 2025-10-20 15:46:37 -03:00
LUIS NOVO
a9af195485 fix: set version cache to 24hrs 2025-10-19 18:05:04 -03:00
LUIS NOVO
aa91523a09 chore: bump 2025-10-19 17:52:56 -03:00
Luis Novo
aa593c60bd
feat: add persistent tiktoken cache to reduce re-downloads (#171)
Configure tiktoken to cache tokenizer encodings in ./data/tiktoken-cache
instead of using system temp directory. This prevents re-downloading
encoding files on every container restart and improves startup time.

Changes:
- Add TIKTOKEN_CACHE_DIR configuration in config.py
- Set TIKTOKEN_CACHE_DIR environment variable in token_utils.py
- Bump version to 1.0.7
2025-10-19 14:50:52 -03:00
Luis Novo
b5666c4d68
Fix/increase fix: increase API client timeouts for transformation operations timeouts (#170)
* fix: increase API client timeouts for transformation operations

- Increase frontend timeout from 30s to 300s (5 minutes)
- Increase Streamlit API client timeout from 30s to 300s
- Add API_CLIENT_TIMEOUT environment variable for configurability
- Add ESPERANTO_LLM_TIMEOUT environment variable documentation
- Update .env.example with comprehensive timeout documentation

Fixes #131 - API timeout errors during transformation generation
Transformations now have sufficient time to complete on slower
hardware (Ollama, LM Studio) without frontend timeout errors.

Users can now configure timeouts for both the API client layer
(API_CLIENT_TIMEOUT) and the LLM provider layer (ESPERANTO_LLM_TIMEOUT)
to accommodate their specific hardware and network conditions.

* docs: add timeout configuration documentation

- Add comprehensive timeout troubleshooting section to common-issues.md
- Add FAQ entry about timeout errors during transformations
- Document API_CLIENT_TIMEOUT and ESPERANTO_LLM_TIMEOUT usage
- Provide specific timeout recommendations for different hardware/network scenarios
- Link to GitHub issue #131 for reference

* chore: bump

* refactor: improve timeout configuration with validation and consistency

Based on PR review feedback, this commit addresses several improvements:

**Timeout Validation:**
- Add validation to ensure timeout values are between 30s and 3600s
- Invalid values fall back to default 300s with warning logs
- Handles edge cases (negative, zero, invalid strings)

**Fix Hard-coded Timeouts:**
- Replace all hard-coded timeout values in api/client.py
- ask_simple: 300s → self.timeout
- execute_transformation: 120s → self.timeout
- embed_content: 120s → self.timeout
- create_source: 300s → self.timeout
- rebuild_embeddings: Uses smart logic (2x timeout, max 3600s)

**Improved Documentation:**
- Add clarifying comments about ms vs seconds (frontend vs backend)
- Document that frontend uses 300000ms = backend 300s
- Add inline documentation for rebuild_embeddings timeout logic

**Development Dependencies:**
- Add pytest>=8.0.0 to dev dependencies for future test coverage

This makes timeout configuration more robust, consistent, and user-friendly
while maintaining backward compatibility.
2025-10-19 11:37:24 -03:00
LUIS NOVO
e601ff3a6e chore: bump to 1.0.5 2025-10-19 10:46:42 -03:00
LUIS NOVO
9670e3553d remove libmagic references (deprecated) 2025-10-19 09:00:40 -03:00
Luis Novo
04b5a9c96a
Implement a serverside fix for reverse proxy users (#169) 2025-10-19 08:02:21 -03:00