Case study · NPA system · 2026
Making the collateral list 89× faster
The collateral list is a page the NPA team keeps open all day, yet every page change took almost 20 seconds. Here is how I found the cause, fixed it, and proved every row on screen stayed exactly the same.
- Open the list
18 s0.2 s- Full export
35 s4 s- Logical reads
257,4249,510- CPU time
2,188 ms78 ms
01 The problem
The list shows 20 rows at a time out of about 3,500 assets, yet every page took 18–20 seconds, deep pages were just as slow, and Export took 35 seconds — users waited on almost every search or page change.
02 Finding the cause
- Row numbers and totals came from ROW_NUMBER() / COUNT() OVER(), which forced the database to process every filtered row (3,500+) before cutting the result down to 20.
- So the "latest technician status" lookup (OUTER APPLY … TOP 1) ran 3,500 times instead of 20 — against a table with no usable index.
- The paging step joined all 14 tables even though the filters used only a few. One was a cross-database join that needs a collation conversion, so it can't use an index; past page 1 the optimizer switched to a nested loop that re-read that table for every row.
03 The fix
- Page first on narrow columns (id + row number + total) with OFFSET / FETCH, then join back for the full columns and technician status — only for that page's 20 rows.
- In the paging step, join only the tables that change the row count plus the ones the filters actually reference.
- Force a hash join on the cross-database data so it is read once instead of once per row.
- Turn the latest-status lookup into a derived table with ROW_NUMBER, so the table is read once — for the page and for Export.
04 Proving nothing broke
- Checked before changing anything that the joins don't multiply rows: COUNT(*) = COUNT(DISTINCT id) = 3,503.
- Compared the new query with the old one row by row across 9 filter combinations: identical, zero differences.
- Edge cases (an invalid page range) still return what they did before — 0 rows, not every row.
- Wrote down why the collation and the hash hint matter in a GOTCHAS doc, so the next person doesn't remove them by accident.
05 Measured results
| Measured | Before | After | Faster |
|---|---|---|---|
| Page 1 (no filter) | 17,962 ms | 203 ms | 88.5× |
| Page 3 (no filter) | 18,352 ms | 205 ms | 89.5× |
| Deep page | 18,566 ms | 140 ms | 133× |
| Export | 35 s | 4 s | 8.8× |
| Logical reads | 257,424 | 9,510 | 27× |
| CPU | 2,188 ms | 78 ms | 28× |
06 A bug found on the way
The "technician updated status" column showed "-" even after technicians had updated it, because the mobile app and the web app wrote the status to different places (measured: 15 assets had a status, only 3 had history). Both paths now write both places, with a guard against duplicate rows when a technician submits the form more than once.
07 Another slow page, a different cause
The eviction case page was slow for a different reason: parallel query steps waited on each other when the database server was busy. Limiting those steps to a single thread (MAXDOP 1) took the first tab from 33.8 s to 1.9–2.4 s and its Excel export from 13.5 s to 2–3 s, with identical results.
08 Where AI fits in
Every commit in this work was co-authored with Claude Code. The part I own: timing both versions on the same data, checking row by row that the results didn't change, and only then shipping it with a rollback.