open-notebook/open_notebook
Luis Novo 4f33b854dd
feat: add environment variables for chunk size configuration (#520)
Adds OPEN_NOTEBOOK_CHUNK_SIZE and OPEN_NOTEBOOK_CHUNK_OVERLAP environment
variables to allow users to configure chunking behavior for different
embedding models with varying context window limits.

Key changes:
- CHUNK_SIZE is now configurable via OPEN_NOTEBOOK_CHUNK_SIZE (default: 1200)
- CHUNK_OVERLAP is configurable via OPEN_NOTEBOOK_CHUNK_OVERLAP (default: 15%)
- Validation with warnings for invalid or out-of-range values
- Updated documentation with configuration examples

This enables users of models like mxbai-embed-large with limited context
windows to reduce chunk size accordingly.

Closes #510
2026-01-31 19:30:56 -03:00
..
ai fix: improve error logging for chat model configuration issues (#458) 2026-01-23 16:45:13 -03:00
database Fix re-embedding issues and improve retry strategy (#515) 2026-01-31 18:55:01 -03:00
domain fix: async insight creation to prevent transaction conflicts (#512) 2026-01-31 15:51:27 -03:00
graphs feat: content-type aware chunking and unified embedding (#444) 2026-01-21 23:49:08 -03:00
podcasts Feat/localization tests docker (#371) 2026-01-15 13:51:05 -03:00
utils feat: add environment variables for chunk size configuration (#520) 2026-01-31 19:30:56 -03:00
__init__.py refactor: move environment variables loading to application entry point (#283) 2025-12-01 14:59:50 -03:00
CLAUDE.md feat: content-type aware chunking and unified embedding (#444) 2026-01-21 23:49:08 -03:00
config.py feat: add persistent tiktoken cache to reduce re-downloads (#171) 2025-10-19 14:50:52 -03:00
exceptions.py refactor database module and migrations 2024-10-30 16:33:07 -03:00