chore: initial setup for source list auto-refresh fix (#526)
This commit is contained in:
Luis Novo
parent
a42e2a347e
commit
992c23d524
3 changed files with 173 additions and 0 deletions
47
claude-goal.md
Normal file
47
claude-goal.md
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
# Goal: Fix Source List Auto-Refresh After Adding New Source
|
||||
|
||||
## Issue Reference
|
||||
GitHub Issue: #526
|
||||
|
||||
## Problem
|
||||
When a user adds a new source to a notebook (via URL, file upload, or text), the source list in the notebook page does not auto-refresh. The user sees a toast notification saying "Source Queued" but must manually refresh the browser to see the new source appear in the list.
|
||||
|
||||
## Root Cause
|
||||
The notebook page uses `useNotebookSources()` which queries with key `QUERY_KEYS.sourcesInfinite(notebookId)` = `['sources', 'infinite', notebookId]`.
|
||||
|
||||
However, the mutation hooks that create sources only invalidate different query keys:
|
||||
- `useCreateSource` invalidates `QUERY_KEYS.sources(notebookId)` = `['sources', notebookId]`
|
||||
- `useFileUpload` invalidates `QUERY_KEYS.sources(variables.notebookId)` = `['sources', notebookId]`
|
||||
|
||||
React Query's prefix matching does NOT cascade from `['sources', notebookId]` to `['sources', 'infinite', notebookId]` because the second element differs (`notebookId` vs `'infinite'`).
|
||||
|
||||
Note: Other hooks like `useDeleteSource`, `useUpdateSource`, etc. correctly use `queryClient.invalidateQueries({ queryKey: ['sources'] })` which IS a prefix of all source queries and does cascade properly.
|
||||
|
||||
## Fix
|
||||
Add `sourcesInfinite` query key invalidation in both `useCreateSource` and `useFileUpload` hooks in `frontend/src/lib/hooks/use-sources.ts`.
|
||||
|
||||
### Key Files
|
||||
- `frontend/src/lib/hooks/use-sources.ts` - Contains all source mutation hooks (main fix location)
|
||||
- `frontend/src/lib/api/query-client.ts` - Defines `QUERY_KEYS` including `sourcesInfinite`
|
||||
- `frontend/src/app/(dashboard)/notebooks/[id]/page.tsx` - Notebook page that uses `useNotebookSources`
|
||||
- `frontend/src/app/(dashboard)/notebooks/components/SourcesColumn.tsx` - Source list UI component
|
||||
|
||||
### Implementation Details
|
||||
|
||||
**In `useCreateSource` (line ~89-139)**:
|
||||
After each `QUERY_KEYS.sources(notebookId)` invalidation, also invalidate `QUERY_KEYS.sourcesInfinite(notebookId)`.
|
||||
|
||||
**In `useFileUpload` (line ~195-220)**:
|
||||
After `QUERY_KEYS.sources(variables.notebookId)` invalidation, also invalidate `QUERY_KEYS.sourcesInfinite(variables.notebookId)`.
|
||||
|
||||
### Why This Works
|
||||
- The backend creates the source record immediately (even for async processing, with title "Processing...")
|
||||
- The source is linked to the notebook immediately
|
||||
- So invalidating the query will refetch and show the new source
|
||||
- `SourceCard` already has `useSourceStatus()` that polls every 2 seconds during processing, so the status will auto-update
|
||||
|
||||
### Gotchas
|
||||
- No need for polling/delays - the source record exists in DB when the API returns
|
||||
- No need for WebSocket/SSE - simple cache invalidation is sufficient
|
||||
- The async path creates a minimal source with "Processing..." title that gets updated when processing completes
|
||||
- `SourceCard` already handles status polling via `useSourceStatus()` hook
|
||||
33
dev-init.sh
Executable file
33
dev-init.sh
Executable file
|
|
@ -0,0 +1,33 @@
|
|||
#!/bin/bash
|
||||
# Development environment setup for Open Notebook
|
||||
# This script installs dependencies and starts all required services
|
||||
|
||||
set -e
|
||||
|
||||
echo "=== Open Notebook Development Setup ==="
|
||||
|
||||
# 1. Install Python dependencies
|
||||
echo "Installing Python dependencies..."
|
||||
uv sync
|
||||
|
||||
# 2. Install frontend dependencies
|
||||
echo "Installing frontend dependencies..."
|
||||
cd frontend && npm install && cd ..
|
||||
|
||||
# 3. Start services using Makefile
|
||||
echo ""
|
||||
echo "=== Starting Services ==="
|
||||
echo "Use 'make start-all' to start all services (DB + API + Worker + Frontend)"
|
||||
echo "Or start individually:"
|
||||
echo " make database - Start SurrealDB"
|
||||
echo " make api - Start FastAPI backend (port 5055)"
|
||||
echo " make worker-start - Start background worker"
|
||||
echo " make frontend - Start Next.js frontend (port 3000)"
|
||||
echo ""
|
||||
echo "Access points:"
|
||||
echo " Frontend: http://localhost:3000"
|
||||
echo " API: http://localhost:5055"
|
||||
echo " API Docs: http://localhost:5055/docs"
|
||||
echo ""
|
||||
echo "To check status: make status"
|
||||
echo "To stop all: make stop-all"
|
||||
93
feature_list.json
Normal file
93
feature_list.json
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
[
|
||||
{
|
||||
"category": "functional",
|
||||
"description": "Source list auto-refreshes after creating a source via URL - useCreateSource invalidates sourcesInfinite query key so the notebook source list updates without manual refresh",
|
||||
"steps": [
|
||||
"Step 1: Navigate to a notebook page",
|
||||
"Step 2: Note the current source list contents",
|
||||
"Step 3: Add a new source via URL (which uses useCreateSource with async_processing)",
|
||||
"Step 4: Verify the toast notification appears",
|
||||
"Step 5: Verify the source list automatically updates to include the new source without manual page refresh"
|
||||
],
|
||||
"passes": false
|
||||
},
|
||||
{
|
||||
"category": "functional",
|
||||
"description": "Source list auto-refreshes after file upload - useFileUpload invalidates sourcesInfinite query key so the notebook source list updates without manual refresh",
|
||||
"steps": [
|
||||
"Step 1: Navigate to a notebook page",
|
||||
"Step 2: Note the current source list contents",
|
||||
"Step 3: Upload a file as a new source (which uses useFileUpload)",
|
||||
"Step 4: Verify the toast notification appears",
|
||||
"Step 5: Verify the source list automatically updates to include the new uploaded source without manual page refresh"
|
||||
],
|
||||
"passes": false
|
||||
},
|
||||
{
|
||||
"category": "functional",
|
||||
"description": "Source list auto-refreshes after creating a text source - useCreateSource handles text type sources and invalidates sourcesInfinite",
|
||||
"steps": [
|
||||
"Step 1: Navigate to a notebook page",
|
||||
"Step 2: Add a new text source via the add source dialog",
|
||||
"Step 3: Verify the source list automatically updates to include the new text source"
|
||||
],
|
||||
"passes": false
|
||||
},
|
||||
{
|
||||
"category": "functional",
|
||||
"description": "Batch source creation refreshes list - when multiple sources are added in batch mode, the source list updates after each",
|
||||
"steps": [
|
||||
"Step 1: Navigate to a notebook page",
|
||||
"Step 2: Use the batch mode to add multiple URLs at once",
|
||||
"Step 3: Verify the source list updates to show the newly added sources"
|
||||
],
|
||||
"passes": false
|
||||
},
|
||||
{
|
||||
"category": "functional",
|
||||
"description": "Source list shows processing status for async sources - new sources appear with processing indicator while being processed in the background",
|
||||
"steps": [
|
||||
"Step 1: Navigate to a notebook page",
|
||||
"Step 2: Add a new source with async_processing enabled",
|
||||
"Step 3: Verify the source appears in the list with a processing/queued status indicator",
|
||||
"Step 4: Wait for processing to complete",
|
||||
"Step 5: Verify the source status updates to completed without manual refresh (via useSourceStatus polling)"
|
||||
],
|
||||
"passes": false
|
||||
},
|
||||
{
|
||||
"category": "functional",
|
||||
"description": "Existing source operations still work - useDeleteSource, useUpdateSource, useAddSourcesToNotebook, useRemoveSourceFromNotebook continue to properly refresh the source list",
|
||||
"steps": [
|
||||
"Step 1: Navigate to a notebook with existing sources",
|
||||
"Step 2: Delete a source and verify it disappears from the list",
|
||||
"Step 3: Add an existing source from another notebook and verify it appears",
|
||||
"Step 4: Remove a source from the notebook and verify it disappears"
|
||||
],
|
||||
"passes": false
|
||||
},
|
||||
{
|
||||
"category": "functional",
|
||||
"description": "Multi-notebook source creation refreshes all affected notebooks - when a source is created targeting multiple notebooks, sourcesInfinite is invalidated for each",
|
||||
"steps": [
|
||||
"Step 1: Open a notebook page",
|
||||
"Step 2: Create a source targeting multiple notebooks via the notebooks field",
|
||||
"Step 3: Verify the current notebook's source list updates",
|
||||
"Step 4: Navigate to another targeted notebook",
|
||||
"Step 5: Verify that notebook's source list also shows the new source"
|
||||
],
|
||||
"passes": false
|
||||
},
|
||||
{
|
||||
"category": "style",
|
||||
"description": "No visual regressions in source list after fix - the source list layout, infinite scroll, and loading states remain unchanged",
|
||||
"steps": [
|
||||
"Step 1: Navigate to a notebook with many sources (enough for infinite scroll)",
|
||||
"Step 2: Verify sources display correctly with proper cards",
|
||||
"Step 3: Scroll down to trigger infinite scroll loading",
|
||||
"Step 4: Verify additional sources load correctly",
|
||||
"Step 5: Add a new source and verify the list updates without layout jumps or visual glitches"
|
||||
],
|
||||
"passes": false
|
||||
}
|
||||
]
|
||||
Loading…
Reference in a new issue