readline.createInterface runs an expensive Unicode line-break regex + extra
stream/string-decoder machinery per chunk. The main transcript parser (parseJsonlStream)
already uses a buffer + manual newline split; these per-team readers still used readline.
Add readJsonlLines(): an async generator that yields a JSONL file's lines via a chunked
utf8 stream read + a plain '\n' split (drop-in for 'for await (const line of rl)'), so the
consumers' loop bodies are unchanged. Stream is utf8-decoded before splitting, so multi-byte
chars across chunk boundaries are safe; trailing CR (CRLF) is stripped; empty lines and a
final newline-less line are yielded, matching readline; breaking out of the loop destroys
the stream via the generator's finally.
Adopt it in MemberStatsComputer, TaskBoundaryParser, and FileContentResolver (file-history
scan). Behavior-identical (their existing tests pass: 18 + 6 + 12) plus 6 new tests for the
generator (CRLF, empty lines, no-trailing-newline, early break, multi-byte chunk boundary).
Note: session-browser readline paths (jsonl metadata extractor, metadataExtraction,
SessionContentFilter) are off the launch path and left as-is for now.
listTeams() deep-cloned ALL team summaries via structuredClone on every call -- even
cache hits and concurrent in-flight awaiters. A heap allocation sample of a launch
put this (listTeams -> cloneTeamSummaries -> structuredClone) as the single largest
memory allocator, driving heap churn + GC pressure during launch (this stand has ~158
teams, and listTeams is called constantly: startup, notification init, task projection,
IPC polls, provisioning).
Build ONE deep-frozen, independent snapshot per uncached load and hand the same
reference to the cache entry, in-flight awaiters, and every later reader. The single
cloneTeamSummaries keeps it independent of any cached config the loader returns;
freezing lets all readers share it safely. Audited every listTeams consumer -- all
iterate / map / filter / serialize, none mutate -- and the freeze turns any stray
future mutation into a loud error rather than silent cross-caller corruption.
TeamConfigReader 26/26 (added a frozen + same-reference regression test), and the
listTeams consumers (TeamDataService 116, CrossTeamService 26) all pass under frozen
summaries.
fileBelongsToTeam streamed the head window via createReadStream + readline. readline's
line iterator runs an expensive Unicode line-break regex and stream/string-decoder
machinery per chunk, which showed up as a top main-thread cost during launch (the line-
split regex alone was ~5.7% in the warm launch profile).
Replace it with a bounded chunked fs.read + a plain '\n' split. JSONL is strictly
newline-delimited and each line is trim()'d (so a trailing CR from CRLF is dropped),
so a '\n' split is cheaper and more correct (it will not split on a bare CR or a
Unicode line/paragraph separator inside a JSON string value, which readline would). A
StringDecoder preserves multi-byte UTF-8 sequences that straddle a chunk boundary.
Byte-identical semantics to the old loop: inspect up to TEAM_AFFINITY_SCAN_LINES
non-empty lines, first match wins via early break, and a final line is honored even
without a trailing newline. Reads in 64KB chunks so a team decided in its first lines
is not penalized by a huge file. Adds tests for CRLF endings + no-trailing-newline,
a multi-byte char straddling the 64KB boundary, and the 40-line window bound (21 pass).
On the live resolution path collectRootJsonlSessionIds already stat()s each root
jsonl for its mtime-window filter, then fileBelongsToTeam stat()ed the very same
file again for its cache validation -- two fs.stat syscalls (plus two Stats
allocations) per file, every poll. fileBelongsToTeam now takes an optional
precomputed stat and the mtime-filter caller passes the stat it already has, so the
file is statted once. Measured 20 files -> 20 stat calls on the mtime path (was ~40).
Using a single stat snapshot is also slightly more consistent than two reads that
could straddle a concurrent write. The other call site (subagent scan) passes no
stat and is unchanged (fileBelongsToTeam stats it itself). Adds a regression test
that a caller-supplied stat is the one recorded in the affinity cache.
fileBelongsToTeam only cached POSITIVE affinity durably; a negative verdict was
re-decided on any change, so during a launch every non-matching transcript in the
project dir that grew (mtime+size change from an active session) was re-streamed
(createReadStream+readline) and re-parsed (up to 40 head lines) on every bootstrap
poll. A live atlas-hq-5 launch profile put this whole subsystem (readline streaming
+ fileBelongsToTeam + line/team matching) at ~31% of main-thread JS, the single
largest launch cost.
A team's first 40 head lines are immutable for an append-only transcript, so a
`false` decided from a FULL inspected window (>= TEAM_AFFINITY_SCAN_LINES) stays
valid while the file only grows. Track headWindowFull on the cache entry and short-
circuit such negatives the same way positives are short-circuited (size >= cached).
Short files (partial window) are still re-scanned on growth, so a team mention that
later lands inside the head window is still detected. A shrink/rewrite (size <
cached) forces a re-scan, identical to the positive path.
Behavior-preserving for affinity correctness (no new false negatives); only removes
redundant re-streams. Adds regression tests for both the durable-negative and the
short-file-flips-to-true cases.
Checkpoint of in-progress work:
- renderer: team messages panel/composer, messagesPanelLogic, teamSlice,
AnimatedHeightReveal plus their tests
- main: runtime process usage-stats caching (ignoreCachedMisses, bounded
eviction), alive-run-id helpers, team watch-scope notify wiring
Note: the getTeamAgentRuntimeSnapshot rssBytes expectation in
TeamAgentLaunchMatrix.safe-e2e is environment-dependent and still red.
TeamProvisioningService imports notifyTeamWatchScopeChanged (added with the
setAliveRunId/deleteAliveRunId helpers) but the export was missing, so a clean
checkout of the branch failed to typecheck. Add the export plus a test; the
call-site wiring stays as in-progress work.
A team launch repeatedly changes the watched target set (new dirs appear), and each
change tore down the chokidar watcher and recreated it over the full target set.
On macOS chokidar uses kqueue with one fd per watched file, so every rebuild
re-opened an fd for EVERY watched file (the large always-watched inbox set plus
scoped dirs). Profiling a 6-member mixed launch showed ~54k open() syscalls dominated
by these rebuilds.
Keep one persistent watcher and apply target-set changes with add()/unwatch() on the
delta only, so a reconcile opens fds for just the newly added dirs. The initial
watcher still uses ignoreInitial for a silent startup baseline, and
emitExistingFilesForNewTargets still backfills files already present in newly added
dirs, so the emitted event surface is unchanged. Because the watcher is no longer
recreated per reconcile, the stale-old-generation and close-throws-during-rebuild
failure modes are gone; their tests are replaced with incremental add/unwatch and
persistent-watcher coverage. All 69 watcher tests pass.
