Commit graph

47 commits

Author SHA1 Message Date
Luis Novo
aa593c60bd
feat: add persistent tiktoken cache to reduce re-downloads (#171)
Configure tiktoken to cache tokenizer encodings in ./data/tiktoken-cache
instead of using system temp directory. This prevents re-downloading
encoding files on every container restart and improves startup time.

Changes:
- Add TIKTOKEN_CACHE_DIR configuration in config.py
- Set TIKTOKEN_CACHE_DIR environment variable in token_utils.py
- Bump version to 1.0.7
2025-10-19 14:50:52 -03:00
Luis Novo
b5666c4d68
Fix/increase fix: increase API client timeouts for transformation operations timeouts (#170)
* fix: increase API client timeouts for transformation operations

- Increase frontend timeout from 30s to 300s (5 minutes)
- Increase Streamlit API client timeout from 30s to 300s
- Add API_CLIENT_TIMEOUT environment variable for configurability
- Add ESPERANTO_LLM_TIMEOUT environment variable documentation
- Update .env.example with comprehensive timeout documentation

Fixes #131 - API timeout errors during transformation generation
Transformations now have sufficient time to complete on slower
hardware (Ollama, LM Studio) without frontend timeout errors.

Users can now configure timeouts for both the API client layer
(API_CLIENT_TIMEOUT) and the LLM provider layer (ESPERANTO_LLM_TIMEOUT)
to accommodate their specific hardware and network conditions.

* docs: add timeout configuration documentation

- Add comprehensive timeout troubleshooting section to common-issues.md
- Add FAQ entry about timeout errors during transformations
- Document API_CLIENT_TIMEOUT and ESPERANTO_LLM_TIMEOUT usage
- Provide specific timeout recommendations for different hardware/network scenarios
- Link to GitHub issue #131 for reference

* chore: bump

* refactor: improve timeout configuration with validation and consistency

Based on PR review feedback, this commit addresses several improvements:

**Timeout Validation:**
- Add validation to ensure timeout values are between 30s and 3600s
- Invalid values fall back to default 300s with warning logs
- Handles edge cases (negative, zero, invalid strings)

**Fix Hard-coded Timeouts:**
- Replace all hard-coded timeout values in api/client.py
- ask_simple: 300s → self.timeout
- execute_transformation: 120s → self.timeout
- embed_content: 120s → self.timeout
- create_source: 300s → self.timeout
- rebuild_embeddings: Uses smart logic (2x timeout, max 3600s)

**Improved Documentation:**
- Add clarifying comments about ms vs seconds (frontend vs backend)
- Document that frontend uses 300000ms = backend 300s
- Add inline documentation for rebuild_embeddings timeout logic

**Development Dependencies:**
- Add pytest>=8.0.0 to dev dependencies for future test coverage

This makes timeout configuration more robust, consistent, and user-friendly
while maintaining backward compatibility.
2025-10-19 11:37:24 -03:00
LUIS NOVO
e601ff3a6e chore: bump to 1.0.5 2025-10-19 10:46:42 -03:00
LUIS NOVO
9670e3553d remove libmagic references (deprecated) 2025-10-19 09:00:40 -03:00
Luis Novo
04b5a9c96a
Implement a serverside fix for reverse proxy users (#169) 2025-10-19 08:02:21 -03:00
Luis Novo
8829eb40c5
Retire streamlit (#166)
* fix text

* remove lint from docker publish workflow

* remove streamlit app
2025-10-18 22:56:46 -03:00
LUIS NOVO
059ee29e18 chore: relax ruff a bit 2025-10-18 13:14:55 -03:00
Luis Novo
b7e656a319
Version 1 (#160)
New front-end
Launch Chat API
Manage Sources
Enable re-embedding of all contents
Sources can be added without a notebook now
Improved settings
Enable model selector on all chats
Background processing for better experience
Dark mode
Improved Notes

Improved Docs: 
- Remove all Streamlit references from documentation
- Update deployment guides with React frontend setup
- Fix Docker environment variables format (SURREAL_URL, SURREAL_PASSWORD)
- Update docker image tag from :latest to :v1-latest
- Change navigation references (Settings → Models to just Models)
- Update development setup to include frontend npm commands
- Add MIGRATION.md guide for users upgrading from Streamlit
- Update quick-start guide with correct environment variables
- Add port 5055 documentation for API access
- Update project structure to reflect frontend/ directory
- Remove outdated source-chat documentation files
2025-10-18 12:46:22 -03:00
Luis Novo
fa27fe561a
Several hotfixes (#130)
* fix: prevent project failing to start when cannot talk to github - fixes #128

* improve ollama documentation - see #127

* chore: update esperanto library to enable gpt-5 - see #107; update podcast-creator library to enable TTS_BATCH_SIZE - fixes #125

* add info on ollama env variables

* chore: ignore dev logs

* chore: bump
2025-09-14 10:58:16 -03:00
LUIS NOVO
929cd262a6 fix: fix open router, google and vertex ai provider selection 2025-07-27 22:30:15 -03:00
LUIS NOVO
fbc3f3ad42 chore: bump 2025-07-17 09:55:30 -03:00
Luis Novo
d7b0fff954
Api podcast migration (#93)
Creates the API layer for Open Notebook
Creates a services API gateway for the Streamlit front-end
Migrates the SurrealDB SDK to the official one
Change all database calls to async
New podcast framework supporting multiple speaker configurations
Implement the surreal-commands library for async processing
Improve docker image and docker-compose configurations
2025-07-17 08:36:11 -03:00
LUIS NOVO
f92b41e510 chore: bump version 2025-06-26 11:56:27 -03:00
LUIS NOVO
c77c84ffd2 chore: update deps 2025-06-14 09:31:22 -03:00
LUIS NOVO
219c752aca chore: add new langchain packages for new providers 2025-06-10 11:56:51 -03:00
LUIS NOVO
7517dd6dab chore: bump version 2025-06-09 20:47:31 -03:00
LUIS NOVO
1333cad3f3 chore: python magic on windows 2025-06-01 09:44:48 -03:00
LUIS NOVO
2afbd36cb4 refactor: implement ai_prompter library 2025-06-01 08:09:33 -03:00
LUIS NOVO
bf7d6c04a2 fix: use forked podcastfy because of youtube-transcript version conflict. 2025-05-30 15:40:19 -03:00
LUIS NOVO
de03fc63a2 chore: bump version 2025-05-30 15:26:11 -03:00
LUIS NOVO
36e928eb75 feat: replace content processing engine with content-core 2025-05-30 13:35:46 -03:00
LUIS NOVO
656ab520e7 forcing 3.12 as maximum python version to fix pydub issue 2025-04-26 06:23:31 -03:00
熊鑫伟 Xinwei Xiong
edf0cca1f3
Update pyproject.toml 2025-04-08 17:43:40 +08:00
LUIS NOVO
92c2ac331b bump version 2025-03-11 21:05:42 -03:00
LUIS NOVO
b1e0e8ad99 migrate to uv 2025-03-11 20:49:10 -03:00
LUIS NOVO
917debeea8 changed to forked surreal repo until migration bug is fixed 2024-11-20 11:53:25 -03:00
LUIS NOVO
add786d112 version bump and migration 2024-11-19 00:02:01 -03:00
LUIS NOVO
dd99531b00 final tweaks to podcast 2024-11-13 21:46:15 -03:00
LUIS NOVO
70358a1be7 poetry update 2024-11-13 17:03:19 -03:00
LUIS NOVO
066c7a06e2 improve search functions 2024-11-13 15:52:44 -03:00
LUIS NOVO
321234e485 add support for GROQ models 2024-11-13 15:09:56 -03:00
LUIS NOVO
7978833473 enable gemini text to speech model for podcast generation 2024-11-07 19:45:56 -03:00
LUIS NOVO
64c123c130 version bump 2024-11-06 13:32:39 -03:00
LUIS NOVO
b89250d3ca temporary fix to config cache 2024-11-01 17:06:10 -03:00
LUIS NOVO
2f2cdabd2d Fix issue with model defaults and bump version 2024-10-30 18:28:29 -03:00
LUIS NOVO
f64897fbf8 support ollama, vertex and gemini embedding models 2024-10-30 15:19:16 -03:00
LUIS NOVO
64f41bb162 bump version 2024-10-29 07:30:17 -03:00
LUIS NOVO
9ca9ac9de1 office packages 2024-10-28 16:33:31 -03:00
LUIS NOVO
d5e3766df8 bump version to 0.0.5 2024-10-28 10:32:56 -03:00
LUIS NOVO
424cb8b16a rename repo and add upgrade version check 2024-10-28 09:21:39 -03:00
LUIS NOVO
aaa7831ab1 add gemini support 2024-10-26 05:41:06 -03:00
LUIS NOVO
01f8eab10e add podcast support 2024-10-26 05:17:58 -03:00
LUIS NOVO
c3f3c9cb93 update docs to new release info 2024-10-23 15:40:28 -03:00
LUIS NOVO
e020511876 replace surreal sdk with surreal-lite until they support v2 2024-10-23 10:58:21 -03:00
LUIS NOVO
093c5dabbb build 0.0.2 2024-10-22 18:27:04 -03:00
LUIS NOVO
93f766f40d add support for vertex, anthropic, litellm, ollama and open router 2024-10-22 16:41:20 -03:00
LUIS NOVO
bcd260a28b Initial commit with all features 2024-10-21 14:56:10 -03:00