diff --git a/claude-goal.md b/claude-goal.md new file mode 100644 index 0000000..3eb6270 --- /dev/null +++ b/claude-goal.md @@ -0,0 +1,47 @@ +# Goal: Fix Source List Auto-Refresh After Adding New Source + +## Issue Reference +GitHub Issue: #526 + +## Problem +When a user adds a new source to a notebook (via URL, file upload, or text), the source list in the notebook page does not auto-refresh. The user sees a toast notification saying "Source Queued" but must manually refresh the browser to see the new source appear in the list. + +## Root Cause +The notebook page uses `useNotebookSources()` which queries with key `QUERY_KEYS.sourcesInfinite(notebookId)` = `['sources', 'infinite', notebookId]`. + +However, the mutation hooks that create sources only invalidate different query keys: +- `useCreateSource` invalidates `QUERY_KEYS.sources(notebookId)` = `['sources', notebookId]` +- `useFileUpload` invalidates `QUERY_KEYS.sources(variables.notebookId)` = `['sources', notebookId]` + +React Query's prefix matching does NOT cascade from `['sources', notebookId]` to `['sources', 'infinite', notebookId]` because the second element differs (`notebookId` vs `'infinite'`). + +Note: Other hooks like `useDeleteSource`, `useUpdateSource`, etc. correctly use `queryClient.invalidateQueries({ queryKey: ['sources'] })` which IS a prefix of all source queries and does cascade properly. + +## Fix +Add `sourcesInfinite` query key invalidation in both `useCreateSource` and `useFileUpload` hooks in `frontend/src/lib/hooks/use-sources.ts`. + +### Key Files +- `frontend/src/lib/hooks/use-sources.ts` - Contains all source mutation hooks (main fix location) +- `frontend/src/lib/api/query-client.ts` - Defines `QUERY_KEYS` including `sourcesInfinite` +- `frontend/src/app/(dashboard)/notebooks/[id]/page.tsx` - Notebook page that uses `useNotebookSources` +- `frontend/src/app/(dashboard)/notebooks/components/SourcesColumn.tsx` - Source list UI component + +### Implementation Details + +**In `useCreateSource` (line ~89-139)**: +After each `QUERY_KEYS.sources(notebookId)` invalidation, also invalidate `QUERY_KEYS.sourcesInfinite(notebookId)`. + +**In `useFileUpload` (line ~195-220)**: +After `QUERY_KEYS.sources(variables.notebookId)` invalidation, also invalidate `QUERY_KEYS.sourcesInfinite(variables.notebookId)`. + +### Why This Works +- The backend creates the source record immediately (even for async processing, with title "Processing...") +- The source is linked to the notebook immediately +- So invalidating the query will refetch and show the new source +- `SourceCard` already has `useSourceStatus()` that polls every 2 seconds during processing, so the status will auto-update + +### Gotchas +- No need for polling/delays - the source record exists in DB when the API returns +- No need for WebSocket/SSE - simple cache invalidation is sufficient +- The async path creates a minimal source with "Processing..." title that gets updated when processing completes +- `SourceCard` already handles status polling via `useSourceStatus()` hook diff --git a/dev-init.sh b/dev-init.sh new file mode 100755 index 0000000..7530f02 --- /dev/null +++ b/dev-init.sh @@ -0,0 +1,33 @@ +#!/bin/bash +# Development environment setup for Open Notebook +# This script installs dependencies and starts all required services + +set -e + +echo "=== Open Notebook Development Setup ===" + +# 1. Install Python dependencies +echo "Installing Python dependencies..." +uv sync + +# 2. Install frontend dependencies +echo "Installing frontend dependencies..." +cd frontend && npm install && cd .. + +# 3. Start services using Makefile +echo "" +echo "=== Starting Services ===" +echo "Use 'make start-all' to start all services (DB + API + Worker + Frontend)" +echo "Or start individually:" +echo " make database - Start SurrealDB" +echo " make api - Start FastAPI backend (port 5055)" +echo " make worker-start - Start background worker" +echo " make frontend - Start Next.js frontend (port 3000)" +echo "" +echo "Access points:" +echo " Frontend: http://localhost:3000" +echo " API: http://localhost:5055" +echo " API Docs: http://localhost:5055/docs" +echo "" +echo "To check status: make status" +echo "To stop all: make stop-all" diff --git a/feature_list.json b/feature_list.json new file mode 100644 index 0000000..caa9dd3 --- /dev/null +++ b/feature_list.json @@ -0,0 +1,93 @@ +[ + { + "category": "functional", + "description": "Source list auto-refreshes after creating a source via URL - useCreateSource invalidates sourcesInfinite query key so the notebook source list updates without manual refresh", + "steps": [ + "Step 1: Navigate to a notebook page", + "Step 2: Note the current source list contents", + "Step 3: Add a new source via URL (which uses useCreateSource with async_processing)", + "Step 4: Verify the toast notification appears", + "Step 5: Verify the source list automatically updates to include the new source without manual page refresh" + ], + "passes": false + }, + { + "category": "functional", + "description": "Source list auto-refreshes after file upload - useFileUpload invalidates sourcesInfinite query key so the notebook source list updates without manual refresh", + "steps": [ + "Step 1: Navigate to a notebook page", + "Step 2: Note the current source list contents", + "Step 3: Upload a file as a new source (which uses useFileUpload)", + "Step 4: Verify the toast notification appears", + "Step 5: Verify the source list automatically updates to include the new uploaded source without manual page refresh" + ], + "passes": false + }, + { + "category": "functional", + "description": "Source list auto-refreshes after creating a text source - useCreateSource handles text type sources and invalidates sourcesInfinite", + "steps": [ + "Step 1: Navigate to a notebook page", + "Step 2: Add a new text source via the add source dialog", + "Step 3: Verify the source list automatically updates to include the new text source" + ], + "passes": false + }, + { + "category": "functional", + "description": "Batch source creation refreshes list - when multiple sources are added in batch mode, the source list updates after each", + "steps": [ + "Step 1: Navigate to a notebook page", + "Step 2: Use the batch mode to add multiple URLs at once", + "Step 3: Verify the source list updates to show the newly added sources" + ], + "passes": false + }, + { + "category": "functional", + "description": "Source list shows processing status for async sources - new sources appear with processing indicator while being processed in the background", + "steps": [ + "Step 1: Navigate to a notebook page", + "Step 2: Add a new source with async_processing enabled", + "Step 3: Verify the source appears in the list with a processing/queued status indicator", + "Step 4: Wait for processing to complete", + "Step 5: Verify the source status updates to completed without manual refresh (via useSourceStatus polling)" + ], + "passes": false + }, + { + "category": "functional", + "description": "Existing source operations still work - useDeleteSource, useUpdateSource, useAddSourcesToNotebook, useRemoveSourceFromNotebook continue to properly refresh the source list", + "steps": [ + "Step 1: Navigate to a notebook with existing sources", + "Step 2: Delete a source and verify it disappears from the list", + "Step 3: Add an existing source from another notebook and verify it appears", + "Step 4: Remove a source from the notebook and verify it disappears" + ], + "passes": false + }, + { + "category": "functional", + "description": "Multi-notebook source creation refreshes all affected notebooks - when a source is created targeting multiple notebooks, sourcesInfinite is invalidated for each", + "steps": [ + "Step 1: Open a notebook page", + "Step 2: Create a source targeting multiple notebooks via the notebooks field", + "Step 3: Verify the current notebook's source list updates", + "Step 4: Navigate to another targeted notebook", + "Step 5: Verify that notebook's source list also shows the new source" + ], + "passes": false + }, + { + "category": "style", + "description": "No visual regressions in source list after fix - the source list layout, infinite scroll, and loading states remain unchanged", + "steps": [ + "Step 1: Navigate to a notebook with many sources (enough for infinite scroll)", + "Step 2: Verify sources display correctly with proper cards", + "Step 3: Scroll down to trigger infinite scroll loading", + "Step 4: Verify additional sources load correctly", + "Step 5: Add a new source and verify the list updates without layout jumps or visual glitches" + ], + "passes": false + } +]