In air-gapped / offline Docker deployments, tiktoken.get_encoding() tries
to download the encoding file from openaipublic.blob.core.windows.net.
When that request fails it raises a URLError / OSError — not an ImportError
— so the previous except clause silently missed it and the crash surfaced in
the UI.
Widened `except ImportError` to `except Exception` so all failures —
"not installed" and "network unreachable" — fall through to the word-count
fallback (words × 1.3). Added a loguru WARNING so operators can see when
the fallback is active.
TIKTOKEN_CACHE_DIR now reads from the environment with a blank-safe
fallback (`or` guard prevents os.makedirs("") on empty env var). This lets
Docker images redirect the cache to a path outside /app/data/ so user-data
volume mounts cannot shadow the pre-baked encoding.
Both images now pre-download the o200k_base encoding during the builder
stage (internet is available at build time) and copy it into the runtime
image at /app/tiktoken-cache. ENV TIKTOKEN_CACHE_DIR=/app/tiktoken-cache
is set in the runtime stage so no network call is ever needed at runtime.
Added test_token_count_network_error_fallback in tests/test_utils.py:
patches tiktoken.get_encoding with a URLError and asserts token_count()
returns a positive int instead of raising.
Fixes#264
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: content-type aware chunking and unified embedding
- Add chunking.py with HTML, Markdown, and plain text detection
- Add embedding.py with mean pooling for large content
- Create dedicated commands: embed_note, embed_insight, embed_source
- Use fire-and-forget pattern for embedding via submit_command()
- Refactor rebuild_embeddings_command to delegate to individual commands
- Remove legacy commands and needs_embedding() methods
- Reduce chunk size to 1500 chars for Ollama compatibility
- Update CLAUDE.md documentation for new architecture
Fixes#350, #142
* fix: address code review issues
- Note.save() now returns command_id for tracking embedding jobs
- Add length check after generate_embeddings() to fail fast on mismatch
- Add numpy as explicit dependency (was transitive)
- Remove hardcoded chunk sizes from docstrings
* docs: address code review comments
- Rename "SYNC PATH" to "DOMAIN MODEL PATH" in embedding router
- Add test_chunking.py and test_embedding.py to Testing Strategy
- Clarify auto-embedding behavior for each domain model
* fix: clean thinking tags from prompt graph output
Adds clean_thinking_content() to prompt.py to handle extended thinking
models that return <think>...</think> tags. This fixes empty titles
when saving notes from chat.
* chore: remove local docker-compose from git
* fix(frontend): handle null parent_id in search results
Add defensive check for null parent_id in search results to prevent
"Cannot read properties of null (reading 'split')" error. This can
happen with orphaned records in the database.
* fix: cascade delete embeddings and insights when source is deleted
When deleting a Source, now also deletes associated:
- source_embedding records
- source_insight records
This prevents orphaned records that cause null parent_id errors
in vector search results.
* fix: add cleanup for orphan embedding/insight records in migration 10
Deletes source_embedding and source_insight records where the
linked source no longer exists (source.id = NONE).
* chore: bump esperanto to 2.16
Increases ctx_num for Ollama models to accommodate larger notebook
context windows. See: https://github.com/lfnovo/esperanto/pull/69
Use model_copy() instead of creating new AIMessage to preserve
response_metadata, id, usage_metadata, etc. Also adds test coverage
for malformed thinking tags pattern.
Addresses PR #333 feedback from lfnovo and cubic-dev-ai.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: simplify reverse proxy configuration with Next.js rewrites
Add Next.js API rewrites to proxy /api/* requests internally from port 8502
to the FastAPI backend on port 5055. This eliminates the need for complex
reverse proxy configurations with multiple upstreams and location blocks.
Changes:
- Add rewrites to next.config.ts proxying /api/* to INTERNAL_API_URL
- Introduce INTERNAL_API_URL env var (defaults to http://localhost:5055)
- Update supervisord configs to pass INTERNAL_API_URL to Next.js
- Document INTERNAL_API_URL in .env.example with usage examples
- Add simplified reverse proxy examples for nginx, Traefik, Caddy, Coolify
- Update README architecture diagram to show internal proxying
- Add explanatory comments to _config route handler
Benefits:
- Reduces reverse proxy config from 12 lines to 3 (75% reduction)
- Single-port deployment (8502 only) for 95% of use cases
- Zero breaking changes - backward compatible with existing setups
- Zero performance overhead (validated through testing)
- Preserves proxy headers (X-Forwarded-*) for rate limiting/SSL
Resolves: #179
Related: OSS-321
* fix: rename _config to config to fix production routing
CRITICAL BUG FIX: The /_config endpoint has never worked in production builds
because Next.js treats folders starting with underscore as "private folders"
and excludes them from routing entirely.
This endpoint is critical for:
- Providing API_URL to the browser at runtime
- Enabling zero-config deployments with auto-detection
- Supporting reverse proxy scenarios where API URL differs from frontend URL
Changes:
- Rename frontend/src/app/_config/ → frontend/src/app/config/
- Update client code references (/_config → /config)
- Update documentation with correct endpoint path
- Bump version to 1.1.0 (minor version for new rewrites feature + bug fix)
Impact:
- Runtime configuration now works in production builds
- /config returns {"apiUrl":"http://localhost:5055"} correctly
- Auto-detection for reverse proxy deployments now functional
Related: #179, OSS-321
* fix: resolve React hook exhaustive-deps warning in AddExistingSourceDialog
Wrap performSearch function in useCallback to properly memoize it and satisfy
React Hook exhaustive-deps rule. This prevents unnecessary re-renders and
ensures the useEffect dependency array is correctly specified.
Changes:
- Import useCallback from React
- Wrap performSearch with useCallback([debouncedSearchQuery, allSources])
- Add performSearch to useEffect dependency array
* final fixes
orihatav