readline.createInterface runs an expensive Unicode line-break regex + extra
stream/string-decoder machinery per chunk. The main transcript parser (parseJsonlStream)
already uses a buffer + manual newline split; these per-team readers still used readline.
Add readJsonlLines(): an async generator that yields a JSONL file's lines via a chunked
utf8 stream read + a plain '\n' split (drop-in for 'for await (const line of rl)'), so the
consumers' loop bodies are unchanged. Stream is utf8-decoded before splitting, so multi-byte
chars across chunk boundaries are safe; trailing CR (CRLF) is stripped; empty lines and a
final newline-less line are yielded, matching readline; breaking out of the loop destroys
the stream via the generator's finally.
Adopt it in MemberStatsComputer, TaskBoundaryParser, and FileContentResolver (file-history
scan). Behavior-identical (their existing tests pass: 18 + 6 + 12) plus 6 new tests for the
generator (CRLF, empty lines, no-trailing-newline, early break, multi-byte chunk boundary).
Note: session-browser readline paths (jsonl metadata extractor, metadataExtraction,
SessionContentFilter) are off the launch path and left as-is for now.
listTeams() deep-cloned ALL team summaries via structuredClone on every call -- even
cache hits and concurrent in-flight awaiters. A heap allocation sample of a launch
put this (listTeams -> cloneTeamSummaries -> structuredClone) as the single largest
memory allocator, driving heap churn + GC pressure during launch (this stand has ~158
teams, and listTeams is called constantly: startup, notification init, task projection,
IPC polls, provisioning).
Build ONE deep-frozen, independent snapshot per uncached load and hand the same
reference to the cache entry, in-flight awaiters, and every later reader. The single
cloneTeamSummaries keeps it independent of any cached config the loader returns;
freezing lets all readers share it safely. Audited every listTeams consumer -- all
iterate / map / filter / serialize, none mutate -- and the freeze turns any stray
future mutation into a loud error rather than silent cross-caller corruption.
TeamConfigReader 26/26 (added a frozen + same-reference regression test), and the
listTeams consumers (TeamDataService 116, CrossTeamService 26) all pass under frozen
summaries.
fileBelongsToTeam streamed the head window via createReadStream + readline. readline's
line iterator runs an expensive Unicode line-break regex and stream/string-decoder
machinery per chunk, which showed up as a top main-thread cost during launch (the line-
split regex alone was ~5.7% in the warm launch profile).
Replace it with a bounded chunked fs.read + a plain '\n' split. JSONL is strictly
newline-delimited and each line is trim()'d (so a trailing CR from CRLF is dropped),
so a '\n' split is cheaper and more correct (it will not split on a bare CR or a
Unicode line/paragraph separator inside a JSON string value, which readline would). A
StringDecoder preserves multi-byte UTF-8 sequences that straddle a chunk boundary.
Byte-identical semantics to the old loop: inspect up to TEAM_AFFINITY_SCAN_LINES
non-empty lines, first match wins via early break, and a final line is honored even
without a trailing newline. Reads in 64KB chunks so a team decided in its first lines
is not penalized by a huge file. Adds tests for CRLF endings + no-trailing-newline,
a multi-byte char straddling the 64KB boundary, and the 40-line window bound (21 pass).
On the live resolution path collectRootJsonlSessionIds already stat()s each root
jsonl for its mtime-window filter, then fileBelongsToTeam stat()ed the very same
file again for its cache validation -- two fs.stat syscalls (plus two Stats
allocations) per file, every poll. fileBelongsToTeam now takes an optional
precomputed stat and the mtime-filter caller passes the stat it already has, so the
file is statted once. Measured 20 files -> 20 stat calls on the mtime path (was ~40).
Using a single stat snapshot is also slightly more consistent than two reads that
could straddle a concurrent write. The other call site (subagent scan) passes no
stat and is unchanged (fileBelongsToTeam stats it itself). Adds a regression test
that a caller-supplied stat is the one recorded in the affinity cache.
fileBelongsToTeam only cached POSITIVE affinity durably; a negative verdict was
re-decided on any change, so during a launch every non-matching transcript in the
project dir that grew (mtime+size change from an active session) was re-streamed
(createReadStream+readline) and re-parsed (up to 40 head lines) on every bootstrap
poll. A live atlas-hq-5 launch profile put this whole subsystem (readline streaming
+ fileBelongsToTeam + line/team matching) at ~31% of main-thread JS, the single
largest launch cost.
A team's first 40 head lines are immutable for an append-only transcript, so a
`false` decided from a FULL inspected window (>= TEAM_AFFINITY_SCAN_LINES) stays
valid while the file only grows. Track headWindowFull on the cache entry and short-
circuit such negatives the same way positives are short-circuited (size >= cached).
Short files (partial window) are still re-scanned on growth, so a team mention that
later lands inside the head window is still detected. A shrink/rewrite (size <
cached) forces a re-scan, identical to the positive path.
Behavior-preserving for affinity correctness (no new false negatives); only removes
redundant re-streams. Adds regression tests for both the durable-negative and the
short-file-flips-to-true cases.
Checkpoint of in-progress work:
- renderer: team messages panel/composer, messagesPanelLogic, teamSlice,
AnimatedHeightReveal plus their tests
- main: runtime process usage-stats caching (ignoreCachedMisses, bounded
eviction), alive-run-id helpers, team watch-scope notify wiring
Note: the getTeamAgentRuntimeSnapshot rssBytes expectation in
TeamAgentLaunchMatrix.safe-e2e is environment-dependent and still red.
TeamProvisioningService imports notifyTeamWatchScopeChanged (added with the
setAliveRunId/deleteAliveRunId helpers) but the export was missing, so a clean
checkout of the branch failed to typecheck. Add the export plus a test; the
call-site wiring stays as in-progress work.
A team launch repeatedly changes the watched target set (new dirs appear), and each
change tore down the chokidar watcher and recreated it over the full target set.
On macOS chokidar uses kqueue with one fd per watched file, so every rebuild
re-opened an fd for EVERY watched file (the large always-watched inbox set plus
scoped dirs). Profiling a 6-member mixed launch showed ~54k open() syscalls dominated
by these rebuilds.
Keep one persistent watcher and apply target-set changes with add()/unwatch() on the
delta only, so a reconcile opens fds for just the newly added dirs. The initial
watcher still uses ignoreInitial for a silent startup baseline, and
emitExistingFilesForNewTargets still backfills files already present in newly added
dirs, so the emitted event surface is unchanged. Because the watcher is no longer
recreated per reconcile, the stale-old-generation and close-throws-during-rebuild
failure modes are gone; their tests are replaced with incremental add/unwatch and
persistent-watcher coverage. All 69 watcher tests pass.
During launch the live-status loop resumes every alive member every audit cycle.
resumeActiveIntervalsForMember runs a synchronous file-lock + full read of every
task file, so for an N-member team with M task files it did N locked passes x M
readFileSync per cycle (e.g. 6 members x 20 task files), blocking the main event
loop. Profiling a 6-member mixed launch showed mutateTeamTasks/withFileLockSync as
a top main-thread cost (~14%).
Add resumeActiveIntervalsForMembers that applies the identical per-member resume
logic against a member set in a single locked pass, and use it in the live-status
loop. Same mutations, but one lock + task read per cycle instead of one per member.
Adds a test covering multi-member resume in one pass.
A team launch creates many directories/files in quick succession (worktrees,
inboxes, session logs), and each addDir/unlinkDir event triggered a full
TeamTaskWatchRegistry reconcile that tore down and recreated the entire chokidar
watcher (re-opening a kqueue fd per watched file on macOS). Profiling a 6-member
mixed-team launch showed kqueue churn (kevent) as a top native cost and watcher
rebuild as the top remaining main-thread JS cost after the transcript fix.
Debounce the event-driven reconcile (250ms) so a burst collapses into one rebuild.
collectTargets re-reads the current directory state and emitExistingFilesForNewTargets
backfills files created before the rebuild, so no change is missed; requestReconcile,
startup, and the periodic 30s reconcile stay immediate. Adds a test asserting a
burst of addDir events yields a single rebuild.
During launch, the bootstrap-wait loop polls each member and, per member, re-read
and re-JSON.parsed the same growing transcript tail (readRecentBootstrapTranscriptOutcome
was the top main-thread JS hotspot at ~21% during bootstrap, ~40% with its helpers).
The same file was parsed once per member per poll.
Memoize the parsed tail by (filePath, mtime, size) in a shared cache so the file is
read + parsed once per change and reused across all members. The per-member filter
and failure/success scan is byte-for-byte the same logic; only the redundant read +
JSON.parse is removed. Cache is bounded (LRU, same cap as the outcome cache) and
invalidated on mtime/size change, matching the existing outcome cache semantics.
Adds a test asserting the tail is parsed once and shared while per-member outcome
detection is unchanged.
The main process watched every team directory under ~/.claude/teams (one shallow
chokidar target per team root, per team inboxes, and per task dir). On macOS this
falls back to kqueue, which needs one fd per watched file, so a workspace with
many teams kept ~1600 descriptors open and made startup and reconcile work scale
with the number of teams on disk.
Scope the team-root and task watching to teams that are running or currently
engaged in the UI. The teams root and every team's inboxes are still watched for
all teams, so cross-team message delivery, the lead inbox->stdin relay, and
notifications are unchanged. Idle teams are static, so dropping their team-root/
task watches is safe; opening a team (getData) or launching it re-adds it via an
immediate watch-scope refresh. The provider falls back to watching every team
when unset, and the EMFILE polling fallback is intentionally left unscoped so a
scope change can never look like a deletion.
Measured on a 162-team workspace: open team fds 1600 -> 730, with team-root
watching restored the moment a team is opened or goes live.
