open-notebook/README.md
Luis Novo d7b0fff954
Api podcast migration (#93)
Creates the API layer for Open Notebook
Creates a services API gateway for the Streamlit front-end
Migrates the SurrealDB SDK to the official one
Change all database calls to async
New podcast framework supporting multiple speaker configurations
Implement the surreal-commands library for async processing
Improve docker image and docker-compose configurations
2025-07-17 08:36:11 -03:00

19 KiB
Raw Blame History

Forks Stargazers Issues MIT License


Logo

Open Notebook

An open source, privacy-focused alternative to Google's Notebook LM!
Join our Discord server for help, to share workflow ideas, and suggest features!
Checkout our website »

Get Started (Setup) · Features

📢 Open Notebook is under very active development

Open Notebook is under active development! We're moving fast and making improvements every week. Your feedback is incredibly valuable to me during this exciting phase and it gives me motivation to keep improving and building this amazing tool. Please feel free to star the project if you find it useful, and don't hesitate to reach out with any questions or suggestions. I'm excited to see how you'll use it and what ideas you'll bring to the project! Let's build something amazing together! 🚀

Installation Issues?

We have a CustomGPT built to help you install Open Notebook. Check it out here. It will help you through each step of the process.

There are also some basic docker/openai installation guide available if you prefer to install it manually.

Table of Contents
  1. About The Project
  2. Setting Up
  3. Usage
  4. Features
  5. Roadmap
  6. Contributing
  7. License
  8. Contact
  9. Acknowledgments

About The Project

New Notebook

An open source, privacy-focused alternative to Google's Notebook LM. Why give Google more of our data when we can take control of our own research workflows?

In a world dominated by Artificial Intelligence, having the ability to think 🧠 and acquire new knowledge 💡, is a skill that should not be a privilege for a few, nor restricted to a single provider.

Open Notebook empowers you to manage your research, generate AI-assisted notes, and interact with your content—on your terms.

Learn more about our project at https://www.open-notebook.ai

(back to top)

Built With

Python SurrealDB LangChain Streamlit

(back to top)

⚙️ Setting Up

Go to the Setup Guide to learn how to set up the tool in details.

📚 Need help choosing AI models? Check out our Model Selection Guide for recommended combinations and provider comparisons.

You don't need to clone this repo if you just want to use the app without building from source! Take a look at the Open Notebook Boilerplate repo with a sample of how to set it up for maximum feature usability.

Running from source

Start by cloning this repo and cd into it.

git clone https://github.com/lfnovo/open-notebook
cd open-notebook

Rename .env.example into .env and set up your API keys. Also, repeat the process for docker.env if you plan to run this using docker.

cp .env.example .env
cp .env.example docker.env

Edit .env for your API keys.

🔐 Password Protection (Optional)

For users hosting Open Notebook publicly (e.g., on PikaPods, cloud services), you can protect your instance with a password:

# Add this to your .env file
OPEN_NOTEBOOK_PASSWORD=your_secure_password_here

When this environment variable is set:

  • Streamlit UI: Users must enter the password on first access
  • REST API: All API calls require the password in the Authorization header (Authorization: Bearer your_password)
  • Local Usage: If not set, no authentication is required (default behavior)

API Usage with Password:

# Example API call with password
curl -H "Authorization: Bearer your_password" http://localhost:5055/api/notebooks

This provides basic protection for public deployments while keeping local usage simple and password-free.

📚 For detailed security information, see the Security Guide.

🚀 Quick Start

After setting up your environment, simply run:

make start-all

This single command will start all required services (database, API, worker, and UI) for you!

System Dependencies

This project requires some system dependencies:

# macOS
brew install libmagic

# Ubuntu/Debian
sudo apt-get install libmagic-dev

# Fedora/RHEL/CentOS
sudo dnf install file-devel

Installing Python Dependencies

Install all required Python packages:

uv sync
uv pip install python-magic

Running the Application

Open Notebook now requires four services to run: the database, API backend, worker, and Streamlit interface.

Easiest Way: Use make start-all

After completing the setup above, the recommended way to run Open Notebook is:

make start-all

This single command will:

  • Start SurrealDB database on port 8000
  • Start FastAPI backend on port 5055
  • Start Background Worker for podcast generation and transformations
  • Start Streamlit UI on port 8502

Once running, access Open Notebook at http://localhost:8502 🎉

Manual Setup (Development)

If you prefer to start services individually:

# 1. Start SurrealDB database
make database
# or: docker compose up -d surrealdb

# 2. Start the FastAPI backend (in terminal 1)
make api
# or: uv run --env-file .env uvicorn api.main:app --host 0.0.0.0 --port 5055

# 3. Start the background worker (in terminal 2)
make worker
# or: uv run --env-file .env surreal-commands-worker --import-modules commands

# 4. Start Streamlit UI (in terminal 3)
make run
# or: uv run --env-file .env streamlit run app_home.py

Service Endpoints

  • Streamlit UI: http://localhost:8502
  • REST API: http://localhost:5055
  • API Documentation: http://localhost:5055/docs (Interactive Swagger UI)
  • SurrealDB: http://localhost:8000

Service Management

# Check if all services are running
make status

# Stop all services
make stop-all

# Restart worker only
make worker-restart

Note: The worker is required for podcast generation and content transformations. Without it, these features will queue jobs but not process them.

Provider Support Matrix

Thanks to the Esperanto library, we support this providers out of the box!

Provider LLM Support Embedding Support Speech-to-Text Text-to-Speech
OpenAI
Anthropic
Groq
Google (GenAI)
Vertex AI
Ollama
Perplexity
ElevenLabs
Azure OpenAI
Mistral
DeepSeek
Voyage
xAI
OpenRouter

Common Issues and Solutions

If you encounter a port already in use error:

Port 8502 is already in use

Find and stop the running process:

# Find the process using port 8502
lsof -i :8502

# Kill the process (replace PID with the actual process ID)
kill -9 PID

Or specify a different port:

uv run --env-file .env streamlit run app_home.py --server.port=8503

Running with Docker

Open Notebook offers two Docker deployment options:

Option 1: Multi-Container (Default)

If you prefer separate containers for each service:

# Run the full stack (SurrealDB + Streamlit + API)
docker compose --profile multi up

For platforms like PikaPods or if you prefer an all-in-one solution:

# Run everything in a single container
docker compose -f docker-compose.single.yml up -d

Or directly:

docker run -d \
  --name open-notebook \
  -p 8502:8502 -p 5055:5055 \
  -v ./notebook_data:/app/data \
  -v ./surreal_single_data:/mydata \
  -e OPENAI_API_KEY=your_key \
  lfnovo/open_notebook:latest-single

Both setups provide:

  • Streamlit UI: http://localhost:8502
  • REST API: http://localhost:5055
  • API Documentation: http://localhost:5055/docs (Interactive Swagger UI)

📚 For detailed single-container deployment instructions, see the Single-Container Deployment Guide.

Docker with Password Protection: To enable password protection in Docker, add OPEN_NOTEBOOK_PASSWORD=your_password to your environment variables.

API Documentation

Open Notebook now includes a comprehensive REST API that provides programmatic access to all functionality. The API includes endpoints for:

  • Notebooks: Create, read, update, delete notebooks
  • Sources: Manage research sources (links, files, text)
  • Notes: Create and manage notes
  • Search: Full-text and vector search capabilities
  • Models: Manage AI models and providers
  • Transformations: Execute content transformations
  • Settings: Application configuration
  • Context: Generate context for AI interactions
  • Embedding: Vectorize content for search

Visit http://localhost:5055/docs when the API is running to explore the interactive API documentation.

(back to top)

Usage

Go to the Usage page to learn how to use all features.

(back to top)

Features

  • Multi-Notebook Support: Organize your research across multiple notebooks effortlessly.
  • Multi-model support: Open AI, Anthropic, Gemini, Vertex AI, Open Router, X.AI, Groq, Ollama. (Model Selection Guide)
  • Reasoning Model Support: Full support for thinking models like DeepSeek-R1, Qwen3, and Magistral with collapsible reasoning sections.
  • Comprehensive REST API: Full programmatic access to all functionality for building custom integrations.
  • Optional Password Protection: Secure your public deployments with simple password authentication for both UI and API.
  • Advanced Podcast Generator: Create professional podcasts with 1-4 speakers using Episode Profiles. Superior flexibility compared to Google Notebook LM's 2-speaker limitation.
  • Broad Content Integration: Works with links, PDFs, EPUB, Office, TXT, Markdown files, YouTube videos, Audio files, Video files and pasted text.
  • Content Transformation: Powerful customizable actions to summarize, extract insights, and more.
  • AI-Powered Notes: Write notes yourself or let the AI assist you in generating insights.
  • Integrated Search Engines: Built-in full-text and vector search for faster information retrieval.
  • Fine-Grained Context Management: Choose exactly what to share with the AI to maintain control.
  • Citations: Ask questions about your documents and get answers with citations.

Check out our podcast sample

(back to top)

📝 Notebook Page

Three intuitive columns to streamline your work:

  1. Sources: Manage all research materials.
  2. Notes: Create or AI-generated notes.
  3. Chat: Chat with the AI, leveraging your content.

⚙️ Context Configuration

Take control of your data. Decide what gets sent to the AI with three context options:

  • No context
  • Summary only
  • Full content

Plus, you can add your project description to help the AI provide more accurate and helpful responses.

🔍 Integrated Search for Your Items

Locate anything across your research with ease using full-text and vector-based search.

💬 Powerful open prompts

Jinja based prompts that are easy to customize to your own preferences.

(back to top)

Roadmap

  • React Frontend: Modern React-based frontend to replace Streamlit.
  • Live Front-End Updates: Real-time UI updates for a smoother experience.
  • Async Processing: Faster UI through asynchronous content processing.
  • Cross-Notebook Sources and Notes: Reuse research notes across projects.
  • Bookmark Integration: Integrate with your favorite bookmarking app.
  • Comprehensive REST API: Full API coverage for all functionality.
  • Multi-model support: Open AI, Anthropic, Vertex AI, Open Router, Ollama, etc.
  • Insight Generation: New tools for creating insights - transformations
  • Advanced Podcast Generator: Professional multi-speaker podcasts with Episode Profiles and background processing.
  • Multiple Chat Sessions: Juggle different discussions within the same notebook.
  • Enhanced Citations: Improved layout and finer control for citations.
  • Better Embeddings & Summarization: Smarter ways to distill information.

See the open issues for a full list of proposed features (and known issues).

(back to top)

Contributing

We would love your contributions! Specifically, we're looking for help with:

  • Front-End Development: Improve the UI/UX by moving beyond Streamlit.
  • Testing & Bug Fixes: Help make Open Notebook more robust.
  • Feature Development: Lets make the coolest note-taking tool together!

See more at CONTRIBUTING

(back to top)

License

Open Notebook is MIT licensed. See the LICENSE file for details.

(back to top)

Contact

Luis Novo - @lfnovo Join our Discord server for help, share workflow ideas, and suggest features.

(back to top)

Acknowledgments

This project uses some amazing third-party libraries

(back to top)