Skip to content

feat(7642): bin/compactStalePads — staleness-gated bulk compaction#7708

Merged
SamTV12345 merged 2 commits intoether:developfrom
JohnMcLear:feat/compact-stale-pads-7642
May 10, 2026
Merged

feat(7642): bin/compactStalePads — staleness-gated bulk compaction#7708
SamTV12345 merged 2 commits intoether:developfrom
JohnMcLear:feat/compact-stale-pads-7642

Conversation

@JohnMcLear
Copy link
Copy Markdown
Member

Closes #7642.

Summary

  • Adds bin/compactStalePads.ts with --older-than <days>, --keep N, and --dry-run. Composes listAllPadsgetLastEditedcompactPad, mirroring the per-pad error tolerance / dry-run / tally shape of bin/compactAllPads.
  • Hot pads in active timeslider use are skipped; only pads not edited within the cutoff are compacted. Counts fresh-skipped vs. compacted vs. failed in the final report.
  • A getLastEdited fault on one pad is counted as a failure but doesn't stop the run — same posture as a getRevisionsCount fault in compactAllPads.

Design decisions

  • Staleness stays out of the API surface. Per the issue, targeting which pads to compact is a CLI concern; the compactPad endpoint is unchanged. Keeps both surfaces simple and lets non-time staleness signals (rev count, byte size — issue calls these out as follow-ons) layer on later without further API churn.
  • Daily-cron variant deferred. The issue asks for a decision on whether cleanup.compactOlderThanDays ships in the same PR. I've kept this PR scoped to the on-demand operator tool (the primary acceptance bullet) and split the cron variant out as a follow-up — easier to review, and the cron variant has its own settings/lifecycle questions (timer registration, log shape, interaction with cleanup.enabled) that deserve their own diff. Happy to fold it in here if reviewers prefer.

Test plan

  • pnpm ts-check clean
  • mocha tests/backend/specs/compactPad.ts — 30 passing, including:
    • parseArgs accepts --older-than / --keep / --dry-run, rejects missing/negative/unknown
    • cutoff filters fresh pads from stale ones with injected now
    • --keep N is plumbed through to compactPad
    • --dry-run does not call compactPad
    • one stale pad failing to compact does not abort the run
    • a getLastEdited failure on one pad is counted as failure but doesn't stop iteration
    • listAllPads failure aborts cleanly with failed=1
    • empty instance / all-fresh instance / --older-than 0 edge cases
    • end-to-end through real /api/1.3.1/getLastEdited + compactPad

Semver

patch (CLI addition; no API surface change)

🤖 Generated with Claude Code

Adds bin/compactStalePads with --older-than / --keep / --dry-run.
Composes listAllPads → getLastEdited → compactPad so hot pads in
active timeslider use are left alone and only the cold tail is
compacted. Targeting stays a CLI concern; compactPad's API surface
is unchanged.

Per-pad failures (including a getLastEdited fault) don't stop the
run — same error-tolerance shape as compactAllPads. End-to-end test
plumbs through the real /api/1.3.1/getLastEdited + compactPad
endpoints to lock the adapter contract.

Daily-cron variant (cleanup.compactOlderThanDays setting) deferred
to a follow-up so this PR stays focused on the on-demand operator
tool from the issue's primary acceptance bullet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@qodo-code-review
Copy link
Copy Markdown

ⓘ You've reached your Qodo monthly free-tier limit. Reviews pause until next month — upgrade your plan to continue now, or link your paid account if you already have one.

@qodo-free-for-open-source-projects
Copy link
Copy Markdown

Review Summary by Qodo

Add bin/compactStalePads for staleness-gated bulk compaction

✨ Enhancement

Grey Divider

Walkthroughs

Description
• Adds bin/compactStalePads.ts CLI tool for staleness-gated bulk pad compaction
• Filters pads by edit recency using --older-than <days> before compacting
• Supports --keep N and --dry-run flags matching compactAllPads interface
• Implements per-pad error tolerance; failures don't abort the run
• Comprehensive test coverage with 30 passing tests including end-to-end validation
Diagram
flowchart LR
  listAllPads["listAllPads()"]
  getLastEdited["getLastEdited(padId)"]
  filter["Filter by cutoff<br/>olderThanDays"]
  getRevCount["getRevisionsCount(padId)"]
  compactPad["compactPad(padId,<br/>keepRevisions)"]
  report["CompactStaleReport<br/>ok/failed/skipped"]
  
  listAllPads -->|all pads| getLastEdited
  getLastEdited -->|lastEdited ts| filter
  filter -->|stale pads| getRevCount
  getRevCount -->|before count| compactPad
  compactPad -->|success/failure| report
Loading

Grey Divider

File Changes

1. bin/compactStalePads.ts ✨ Enhancement +307/-0

Staleness-gated bulk pad compaction CLI tool

• New CLI tool for compacting pads not edited within --older-than N days
• Exports runCompactStale() core loop with injectable API and logger for testability
• Implements two-pass algorithm: first identifies stale pads via getLastEdited, then compacts them
• Per-pad failures (including getLastEdited faults) counted but don't stop iteration
• CLI binding uses fetch + APIKEY auth to invoke real HTTP endpoints
• Supports --keep N to retain last N revisions and --dry-run for preview mode

bin/compactStalePads.ts


2. src/tests/backend/specs/compactPad.ts 🧪 Tests +265/-0

Comprehensive test coverage for staleness-gated compaction

• Adds 30 new test cases for runCompactStale() and parseArgs() functions
• Tests argument parsing: validates --older-than, --keep, --dry-run flags
• Tests filtering logic: only stale pads (older than cutoff) are compacted
• Tests error resilience: per-pad failures and getLastEdited faults don't abort
• Tests edge cases: empty instance, all-fresh pads, --older-than 0
• End-to-end test plumbs through real /api/1.3.1/getLastEdited and compactPad endpoints

src/tests/backend/specs/compactPad.ts


3. CHANGELOG.md 📝 Documentation +1/-0

Document staleness-gated compaction CLI tool

• Documents new bin/compactStalePads tool in pad compaction section
• Highlights staleness filtering via --older-than N days parameter
• Notes that targeting is CLI-only; compactPad API surface unchanged
• References issue #7642

CHANGELOG.md


Grey Divider

Qodo Logo

@qodo-free-for-open-source-projects
Copy link
Copy Markdown

qodo-free-for-open-source-projects Bot commented May 9, 2026

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0) 📎 Requirement gaps (2)

Grey Divider


Action required

1. No docs for compactStalePads 📎 Requirement gap ⚙ Maintainability
Description
This PR adds a new operator-facing CLI (bin/compactStalePads) but does not update the admin/CLI
documentation under doc/ to describe its usage and flags. This makes the feature undiscoverable
and violates the documentation update requirement.
Code

bin/compactStalePads.ts[R1-16]

+'use strict';
+
+/*
+ * Compact every pad on the instance that has not been edited recently.
+ *
+ * Usage:
+ *   node bin/compactStalePads.js --older-than 90              # collapse history on pads not edited in 90 days
+ *   node bin/compactStalePads.js --older-than 90 --keep 50    # keep last 50 revisions
+ *   node bin/compactStalePads.js --older-than 90 --dry-run    # list, don't write
+ *
+ * Composes `listAllPads` → `getLastEdited` → `compactPad`. Same shape as
+ * `bin/compactAllPads` (per-pad error tolerance, dry-run, tally), but
+ * filters by edit-recency before touching anything. Targeting which pads
+ * to compact is deliberately a CLI concern and not a `compactPad` API
+ * param — staleness changes from one run to the next, the compaction
+ * primitive does not.
Evidence
PR Compliance ID 4 requires updating admin/CLI documentation to include the new
bin/compactStalePads tool and its flags. The diff shows the new CLI script (including usage and
required flags), but there are no corresponding documentation updates in doc/ in this PR diff.

Update admin/CLI documentation to cover compactStalePads usage (and setting if shipped)
bin/compactStalePads.ts[1-16]
CHANGELOG.md[24-24]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A new CLI tool (`bin/compactStalePads`) was introduced, but the admin/CLI documentation was not updated to describe how to use it (flags `--older-than`, `--keep`, `--dry-run`, and behavior).
## Issue Context
Compliance requires that operator-facing CLI functionality be documented in the appropriate `doc/` admin/CLI documentation so administrators can discover and safely use it.
## Fix Focus Areas
- doc/cli.md[1-200]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. No docs for compactStalePads 📎 Requirement gap ⚙ Maintainability
Description
This PR adds a new operator-facing CLI (bin/compactStalePads) but does not update the admin/CLI
documentation under doc/ to describe its usage and flags. This makes the feature undiscoverable
and violates the documentation update requirement.
Code

bin/compactStalePads.ts[R1-16]

+'use strict';
+
+/*
+ * Compact every pad on the instance that has not been edited recently.
+ *
+ * Usage:
+ *   node bin/compactStalePads.js --older-than 90              # collapse history on pads not edited in 90 days
+ *   node bin/compactStalePads.js --older-than 90 --keep 50    # keep last 50 revisions
+ *   node bin/compactStalePads.js --older-than 90 --dry-run    # list, don't write
+ *
+ * Composes `listAllPads` → `getLastEdited` → `compactPad`. Same shape as
+ * `bin/compactAllPads` (per-pad error tolerance, dry-run, tally), but
+ * filters by edit-recency before touching anything. Targeting which pads
+ * to compact is deliberately a CLI concern and not a `compactPad` API
+ * param — staleness changes from one run to the next, the compaction
+ * primitive does not.
Evidence
PR Compliance ID 4 requires updating admin/CLI documentation to include the new
bin/compactStalePads tool and its flags. The diff shows the new CLI script (including usage and
required flags), but there are no corresponding documentation updates in doc/ in this PR diff.

Update admin/CLI documentation to cover compactStalePads usage (and setting if shipped)
bin/compactStalePads.ts[1-16]
CHANGELOG.md[24-24]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A new CLI tool (`bin/compactStalePads`) was introduced, but the admin/CLI documentation was not updated to describe how to use it (flags `--older-than`, `--keep`, `--dry-run`, and behavior).
## Issue Context
Compliance requires that operator-facing CLI functionality be documented in the appropriate `doc/` admin/CLI documentation so administrators can discover and safely use it.
## Fix Focus Areas
- doc/cli.md[1-200]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

3. Staleness check race window ✓ Resolved 🐞 Bug ☼ Reliability
Description
runCompactStale selects stale pads in a first pass and compacts them in a later pass without
re-checking getLastEdited, so a pad edited between passes can still be compacted despite becoming
“fresh”. This can disrupt active users because compaction can kick sessions from the pad.
Code

bin/compactStalePads.ts[R110-166]

+  // First pass: figure out which pads are actually stale. A getLastEdited
+  // failure on a pad is counted as a failure (we can't decide), but does
+  // not stop the run.
+  const stalePads: string[] = [];
+  for (const padId of padIds) {
+    let lastEdited: number;
+    try {
+      lastEdited = await api.getLastEdited(padId);
+    } catch (e: any) {
+      logger.error(`${padId}: getLastEdited failed: ${e.message ?? e}`);
+      report.failed++;
+      continue;
+    }
+    if (lastEdited > cutoff) {
+      report.skippedFresh++;
+      continue;
+    }
+    stalePads.push(padId);
+  }
+  report.stale = stalePads.length;
+
+  if (stalePads.length === 0) {
+    logger.info(
+        `No stale pads (${report.skippedFresh} fresh, ${report.failed} unreadable).`);
+    return report;
+  }
+
+  logger.info(
+      `${stalePads.length} stale pad(s) to process ` +
+      `(${report.skippedFresh} fresh skipped).`);
+
+  for (let i = 0; i < stalePads.length; i++) {
+    const padId = stalePads[i];
+    const idx = `[${i + 1}/${stalePads.length}]`;
+
+    let before: number;
+    try {
+      before = await api.getRevisionsCount(padId);
+    } catch (e: any) {
+      logger.error(`${idx} ${padId}: getRevisionsCount failed: ${e.message ?? e}`);
+      report.failed++;
+      continue;
+    }
+
+    if (opts.dryRun) {
+      logger.info(`${idx} ${padId}: ${before + 1} revision(s) — would compact`);
+      report.totalRevsBefore += before + 1;
+      continue;
+    }
+
+    try {
+      await api.compactPad(padId, opts.keepRevisions);
+    } catch (e: any) {
+      logger.error(`${idx} ${padId}: compactPad failed: ${e.message ?? e}`);
+      report.failed++;
+      continue;
+    }
Evidence
The script builds a stalePads list based on getLastEdited and later calls compactPad for each ID
without validating that the pad is still stale. The compaction implementation can kick connected
sessions, so compacting a pad that became active during the run is user-impacting.

bin/compactStalePads.ts[110-180]
src/node/db/API.ts[704-720]
src/node/utils/Cleanup.ts[43-60]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`runCompactStale()` determines staleness in a first loop, stores IDs, and later compacts from that stored list. If a pad is edited after selection but before compaction, it can still be compacted, which undermines the “skip hot pads” goal and may kick active sessions.
### Issue Context
Compaction can kick sessions (`Cleanup.deleteRevisions()`), so compacting a pad that becomes active during the run is disruptive.
### Fix Focus Areas
- bin/compactStalePads.ts[110-180]
### Suggested fix
Before compacting each pad (ideally right before calling `compactPad`, after pre-flight `getRevisionsCount`), call `getLastEdited(padId)` again and skip compaction if `lastEdited > cutoff` (increment `skippedFresh` and log). This keeps the staleness guarantee closer to real-time and minimizes the race window.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


4. Staleness check race window ✓ Resolved 🐞 Bug ☼ Reliability
Description
runCompactStale selects stale pads in a first pass and compacts them in a later pass without
re-checking getLastEdited, so a pad edited between passes can still be compacted despite becoming
“fresh”. This can disrupt active users because compaction can kick sessions from the pad.
Code

bin/compactStalePads.ts[R110-166]

+  // First pass: figure out which pads are actually stale. A getLastEdited
+  // failure on a pad is counted as a failure (we can't decide), but does
+  // not stop the run.
+  const stalePads: string[] = [];
+  for (const padId of padIds) {
+    let lastEdited: number;
+    try {
+      lastEdited = await api.getLastEdited(padId);
+    } catch (e: any) {
+      logger.error(`${padId}: getLastEdited failed: ${e.message ?? e}`);
+      report.failed++;
+      continue;
+    }
+    if (lastEdited > cutoff) {
+      report.skippedFresh++;
+      continue;
+    }
+    stalePads.push(padId);
+  }
+  report.stale = stalePads.length;
+
+  if (stalePads.length === 0) {
+    logger.info(
+        `No stale pads (${report.skippedFresh} fresh, ${report.failed} unreadable).`);
+    return report;
+  }
+
+  logger.info(
+      `${stalePads.length} stale pad(s) to process ` +
+      `(${report.skippedFresh} fresh skipped).`);
+
+  for (let i = 0; i < stalePads.length; i++) {
+    const padId = stalePads[i];
+    const idx = `[${i + 1}/${stalePads.length}]`;
+
+    let before: number;
+    try {
+      before = await api.getRevisionsCount(padId);
+    } catch (e: any) {
+      logger.error(`${idx} ${padId}: getRevisionsCount failed: ${e.message ?? e}`);
+      report.failed++;
+      continue;
+    }
+
+    if (opts.dryRun) {
+      logger.info(`${idx} ${padId}: ${before + 1} revision(s) — would compact`);
+      report.totalRevsBefore += before + 1;
+      continue;
+    }
+
+    try {
+      await api.compactPad(padId, opts.keepRevisions);
+    } catch (e: any) {
+      logger.error(`${idx} ${padId}: compactPad failed: ${e.message ?? e}`);
+      report.failed++;
+      continue;
+    }
Evidence
The script builds a stalePads list based on getLastEdited and later calls compactPad for each ID
without validating that the pad is still stale. The compaction implementation can kick connected
sessions, so compacting a pad that became active during the run is user-impacting.

bin/compactStalePads.ts[110-180]
src/node/db/API.ts[704-720]
src/node/utils/Cleanup.ts[43-60]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`runCompactStale()` determines staleness in a first loop, stores IDs, and later compacts from that stored list. If a pad is edited after selection but before compaction, it can still be compacted, which undermines the “skip hot pads” goal and may kick active sessions.
### Issue Context
Compaction can kick sessions (`Cleanup.deleteRevisions()`), so compacting a pad that becomes active during the run is disruptive.
### Fix Focus Areas
- bin/compactStalePads.ts[110-180]
### Suggested fix
Before compacting each pad (ideally right before calling `compactPad`, after pre-flight `getRevisionsCount`), call `getLastEdited(padId)` again and skip compaction if `lastEdited > cutoff` (increment `skippedFresh` and log). This keeps the staleness guarantee closer to real-time and minimizes the race window.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Advisory comments

5. Duplicated CLI wiring logic 🐞 Bug ⚙ Maintainability
Description
compactStalePads duplicates the settings/baseURL/fetch/APIKEY/apiVersion adapter logic used by
compactAllPads, increasing the chance of drift and inconsistent future fixes across the CLIs. This
makes changes to auth, transport, or error-handling more error-prone over time.
Code

bin/compactStalePads.ts[R243-301]

+const isMain = require.main === module;
+if (isMain) {
+  process.on('unhandledRejection', (err) => { throw err; });
+
+  const settings = require('ep_etherpad-lite/tests/container/loadSettings').loadSettings();
+  const baseURL = `${settings.ssl ? 'https' : 'http'}://${settings.ip}:${settings.port}`;
+
+  const apiGet = async (p: string): Promise<any> => {
+    const r = await fetch(baseURL + p);
+    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);
+    return r.json();
+  };
+  const apiPost = async (p: string): Promise<any> => {
+    const r = await fetch(baseURL + p, {method: 'POST'});
+    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);
+    return r.json();
+  };
+
+  const opts = parseArgs(process.argv.slice(2));
+  if (!opts) usage();
+
+  const apikey = fs.readFileSync(
+      path.join(__dirname, '../APIKEY.txt'), {encoding: 'utf-8'}).trim();
+
+  // Bind the abstract API to fetch + APIKEY auth for the CLI shell.
+  const cliApi: CompactStaleApi = {
+    async listAllPads() {
+      const apiInfo = await apiGet('/api/');
+      const apiVersion: string | undefined = apiInfo.currentVersion;
+      if (!apiVersion) throw new Error('No version set in API');
+      (cliApi as any)._apiVersion = apiVersion;
+      const r = await apiGet(`/api/${apiVersion}/listAllPads?apikey=${apikey}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+      return r.data.padIDs ?? [];
+    },
+    async getLastEdited(padId: string) {
+      const v = (cliApi as any)._apiVersion;
+      const r = await apiGet(
+          `/api/${v}/getLastEdited?apikey=${apikey}` +
+          `&padID=${encodeURIComponent(padId)}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+      return r.data.lastEdited;
+    },
+    async getRevisionsCount(padId: string) {
+      const v = (cliApi as any)._apiVersion;
+      const r = await apiGet(
+          `/api/${v}/getRevisionsCount?apikey=${apikey}` +
+          `&padID=${encodeURIComponent(padId)}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+      return r.data.revisions;
+    },
+    async compactPad(padId: string, keepRevisions: number | null) {
+      const v = (cliApi as any)._apiVersion;
+      const params = new URLSearchParams({apikey, padID: padId});
+      if (keepRevisions != null) params.set('keepRevisions', String(keepRevisions));
+      const r = await apiPost(`/api/${v}/compactPad?${params.toString()}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+    },
+  };
Evidence
The new CLI repeats the same “loadSettings → baseURL → apiGet/apiPost → read APIKEY.txt → cache
apiVersion” pattern already present in the existing bulk compaction CLI, but with an additional
endpoint binding. This duplication increases maintenance overhead.

bin/compactStalePads.ts[243-301]
bin/compactAllPads.ts[179-236]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new `bin/compactStalePads.ts` repeats the same CLI plumbing that exists in `bin/compactAllPads.ts` (settings loading, baseURL, fetch wrappers, APIKEY loading, API version caching). Duplicated wiring increases drift risk.
### Issue Context
This is not a functional bug today, but it increases the cost/risk of future changes (auth behavior, error formatting, fetch options).
### Fix Focus Areas
- bin/compactStalePads.ts[243-301]
- bin/compactAllPads.ts[179-236]
### Suggested fix
Create a small shared helper module (e.g., `bin/_compactCliApi.ts`) that:
- loads settings and constructs `baseURL`
- provides `apiGet`/`apiPost`
- reads `APIKEY.txt`
- resolves and caches `apiVersion`
Then have both CLIs build their endpoint-specific adapters on top of that helper.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. Duplicated CLI wiring logic 🐞 Bug ⚙ Maintainability
Description
compactStalePads duplicates the settings/baseURL/fetch/APIKEY/apiVersion adapter logic used by
compactAllPads, increasing the chance of drift and inconsistent future fixes across the CLIs. This
makes changes to auth, transport, or error-handling more error-prone over time.
Code

bin/compactStalePads.ts[R243-301]

+const isMain = require.main === module;
+if (isMain) {
+  process.on('unhandledRejection', (err) => { throw err; });
+
+  const settings = require('ep_etherpad-lite/tests/container/loadSettings').loadSettings();
+  const baseURL = `${settings.ssl ? 'https' : 'http'}://${settings.ip}:${settings.port}`;
+
+  const apiGet = async (p: string): Promise<any> => {
+    const r = await fetch(baseURL + p);
+    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);
+    return r.json();
+  };
+  const apiPost = async (p: string): Promise<any> => {
+    const r = await fetch(baseURL + p, {method: 'POST'});
+    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);
+    return r.json();
+  };
+
+  const opts = parseArgs(process.argv.slice(2));
+  if (!opts) usage();
+
+  const apikey = fs.readFileSync(
+      path.join(__dirname, '../APIKEY.txt'), {encoding: 'utf-8'}).trim();
+
+  // Bind the abstract API to fetch + APIKEY auth for the CLI shell.
+  const cliApi: CompactStaleApi = {
+    async listAllPads() {
+      const apiInfo = await apiGet('/api/');
+      const apiVersion: string | undefined = apiInfo.currentVersion;
+      if (!apiVersion) throw new Error('No version set in API');
+      (cliApi as any)._apiVersion = apiVersion;
+      const r = await apiGet(`/api/${apiVersion}/listAllPads?apikey=${apikey}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+      return r.data.padIDs ?? [];
+    },
+    async getLastEdited(padId: string) {
+      const v = (cliApi as any)._apiVersion;
+      const r = await apiGet(
+          `/api/${v}/getLastEdited?apikey=${apikey}` +
+          `&padID=${encodeURIComponent(padId)}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+      return r.data.lastEdited;
+    },
+    async getRevisionsCount(padId: string) {
+      const v = (cliApi as any)._apiVersion;
+      const r = await apiGet(
+          `/api/${v}/getRevisionsCount?apikey=${apikey}` +
+          `&padID=${encodeURIComponent(padId)}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+      return r.data.revisions;
+    },
+    async compactPad(padId: string, keepRevisions: number | null) {
+      const v = (cliApi as any)._apiVersion;
+      const params = new URLSearchParams({apikey, padID: padId});
+      if (keepRevisions != null) params.set('keepRevisions', String(keepRevisions));
+      const r = await apiPost(`/api/${v}/compactPad?${params.toString()}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+    },
+  };
Evidence
The new CLI repeats the same “loadSettings → baseURL → apiGet/apiPost → read APIKEY.txt → cache
apiVersion” pattern already present in the existing bulk compaction CLI, but with an additional
endpoint binding. This duplication increases maintenance overhead.

bin/compactStalePads.ts[243-301]
bin/compactAllPads.ts[179-236]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new `bin/compactStalePads.ts` repeats the same CLI plumbing that exists in `bin/compactAllPads.ts` (settings loading, baseURL, fetch wrappers, APIKEY loading, API version caching). Duplicated wiring increases drift risk.
### Issue Context
This is not a functional bug today, but it increases the cost/risk of future changes (auth behavior, error formatting, fetch options).
### Fix Focus Areas
- bin/compactStalePads.ts[243-301]
- bin/compactAllPads.ts[179-236]
### Suggested fix
Create a small shared helper module (e.g., `bin/_compactCliApi.ts`) that:
- loads settings and constructs `baseURL`
- provides `apiGet`/`apiPost`
- reads `APIKEY.txt`
- resolves and caches `apiVersion`
Then have both CLIs build their endpoint-specific adapters on top of that helper.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@JohnMcLear
Copy link
Copy Markdown
Member Author

/review

@qodo-code-review
Copy link
Copy Markdown

ⓘ You've reached your Qodo monthly free-tier limit. Reviews pause until next month — upgrade your plan to continue now, or link your paid account if you already have one.

@qodo-free-for-open-source-projects
Copy link
Copy Markdown

qodo-free-for-open-source-projects Bot commented May 9, 2026

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0) 📎 Requirement gaps (1)

Context used

Grey Divider


Action required

1. No docs for compactStalePads 📎 Requirement gap ⚙ Maintainability
Description
This PR adds a new operator-facing CLI (bin/compactStalePads) but does not update the admin/CLI
documentation under doc/ to describe its usage and flags. This makes the feature undiscoverable
and violates the documentation update requirement.
Code

bin/compactStalePads.ts[R1-16]

+'use strict';
+
+/*
+ * Compact every pad on the instance that has not been edited recently.
+ *
+ * Usage:
+ *   node bin/compactStalePads.js --older-than 90              # collapse history on pads not edited in 90 days
+ *   node bin/compactStalePads.js --older-than 90 --keep 50    # keep last 50 revisions
+ *   node bin/compactStalePads.js --older-than 90 --dry-run    # list, don't write
+ *
+ * Composes `listAllPads` → `getLastEdited` → `compactPad`. Same shape as
+ * `bin/compactAllPads` (per-pad error tolerance, dry-run, tally), but
+ * filters by edit-recency before touching anything. Targeting which pads
+ * to compact is deliberately a CLI concern and not a `compactPad` API
+ * param — staleness changes from one run to the next, the compaction
+ * primitive does not.
Evidence
PR Compliance ID 4 requires updating admin/CLI documentation to include the new
bin/compactStalePads tool and its flags. The diff shows the new CLI script (including usage and
required flags), but there are no corresponding documentation updates in doc/ in this PR diff.

Update admin/CLI documentation to cover compactStalePads usage (and setting if shipped)
bin/compactStalePads.ts[1-16]
CHANGELOG.md[24-24]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A new CLI tool (`bin/compactStalePads`) was introduced, but the admin/CLI documentation was not updated to describe how to use it (flags `--older-than`, `--keep`, `--dry-run`, and behavior).

## Issue Context
Compliance requires that operator-facing CLI functionality be documented in the appropriate `doc/` admin/CLI documentation so administrators can discover and safely use it.

## Fix Focus Areas
- doc/cli.md[1-200]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Staleness check race window 🐞 Bug ☼ Reliability
Description
runCompactStale selects stale pads in a first pass and compacts them in a later pass without
re-checking getLastEdited, so a pad edited between passes can still be compacted despite becoming
“fresh”. This can disrupt active users because compaction can kick sessions from the pad.
Code

bin/compactStalePads.ts[R110-166]

+  // First pass: figure out which pads are actually stale. A getLastEdited
+  // failure on a pad is counted as a failure (we can't decide), but does
+  // not stop the run.
+  const stalePads: string[] = [];
+  for (const padId of padIds) {
+    let lastEdited: number;
+    try {
+      lastEdited = await api.getLastEdited(padId);
+    } catch (e: any) {
+      logger.error(`${padId}: getLastEdited failed: ${e.message ?? e}`);
+      report.failed++;
+      continue;
+    }
+    if (lastEdited > cutoff) {
+      report.skippedFresh++;
+      continue;
+    }
+    stalePads.push(padId);
+  }
+  report.stale = stalePads.length;
+
+  if (stalePads.length === 0) {
+    logger.info(
+        `No stale pads (${report.skippedFresh} fresh, ${report.failed} unreadable).`);
+    return report;
+  }
+
+  logger.info(
+      `${stalePads.length} stale pad(s) to process ` +
+      `(${report.skippedFresh} fresh skipped).`);
+
+  for (let i = 0; i < stalePads.length; i++) {
+    const padId = stalePads[i];
+    const idx = `[${i + 1}/${stalePads.length}]`;
+
+    let before: number;
+    try {
+      before = await api.getRevisionsCount(padId);
+    } catch (e: any) {
+      logger.error(`${idx} ${padId}: getRevisionsCount failed: ${e.message ?? e}`);
+      report.failed++;
+      continue;
+    }
+
+    if (opts.dryRun) {
+      logger.info(`${idx} ${padId}: ${before + 1} revision(s) — would compact`);
+      report.totalRevsBefore += before + 1;
+      continue;
+    }
+
+    try {
+      await api.compactPad(padId, opts.keepRevisions);
+    } catch (e: any) {
+      logger.error(`${idx} ${padId}: compactPad failed: ${e.message ?? e}`);
+      report.failed++;
+      continue;
+    }
Evidence
The script builds a stalePads list based on getLastEdited and later calls compactPad for each ID
without validating that the pad is still stale. The compaction implementation can kick connected
sessions, so compacting a pad that became active during the run is user-impacting.

bin/compactStalePads.ts[110-180]
src/node/db/API.ts[704-720]
src/node/utils/Cleanup.ts[43-60]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`runCompactStale()` determines staleness in a first loop, stores IDs, and later compacts from that stored list. If a pad is edited after selection but before compaction, it can still be compacted, which undermines the “skip hot pads” goal and may kick active sessions.

### Issue Context
Compaction can kick sessions (`Cleanup.deleteRevisions()`), so compacting a pad that becomes active during the run is disruptive.

### Fix Focus Areas
- bin/compactStalePads.ts[110-180]

### Suggested fix
Before compacting each pad (ideally right before calling `compactPad`, after pre-flight `getRevisionsCount`), call `getLastEdited(padId)` again and skip compaction if `lastEdited > cutoff` (increment `skippedFresh` and log). This keeps the staleness guarantee closer to real-time and minimizes the race window.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Advisory comments

3. Duplicated CLI wiring logic 🐞 Bug ⚙ Maintainability
Description
compactStalePads duplicates the settings/baseURL/fetch/APIKEY/apiVersion adapter logic used by
compactAllPads, increasing the chance of drift and inconsistent future fixes across the CLIs. This
makes changes to auth, transport, or error-handling more error-prone over time.
Code

bin/compactStalePads.ts[R243-301]

+const isMain = require.main === module;
+if (isMain) {
+  process.on('unhandledRejection', (err) => { throw err; });
+
+  const settings = require('ep_etherpad-lite/tests/container/loadSettings').loadSettings();
+  const baseURL = `${settings.ssl ? 'https' : 'http'}://${settings.ip}:${settings.port}`;
+
+  const apiGet = async (p: string): Promise<any> => {
+    const r = await fetch(baseURL + p);
+    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);
+    return r.json();
+  };
+  const apiPost = async (p: string): Promise<any> => {
+    const r = await fetch(baseURL + p, {method: 'POST'});
+    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);
+    return r.json();
+  };
+
+  const opts = parseArgs(process.argv.slice(2));
+  if (!opts) usage();
+
+  const apikey = fs.readFileSync(
+      path.join(__dirname, '../APIKEY.txt'), {encoding: 'utf-8'}).trim();
+
+  // Bind the abstract API to fetch + APIKEY auth for the CLI shell.
+  const cliApi: CompactStaleApi = {
+    async listAllPads() {
+      const apiInfo = await apiGet('/api/');
+      const apiVersion: string | undefined = apiInfo.currentVersion;
+      if (!apiVersion) throw new Error('No version set in API');
+      (cliApi as any)._apiVersion = apiVersion;
+      const r = await apiGet(`/api/${apiVersion}/listAllPads?apikey=${apikey}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+      return r.data.padIDs ?? [];
+    },
+    async getLastEdited(padId: string) {
+      const v = (cliApi as any)._apiVersion;
+      const r = await apiGet(
+          `/api/${v}/getLastEdited?apikey=${apikey}` +
+          `&padID=${encodeURIComponent(padId)}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+      return r.data.lastEdited;
+    },
+    async getRevisionsCount(padId: string) {
+      const v = (cliApi as any)._apiVersion;
+      const r = await apiGet(
+          `/api/${v}/getRevisionsCount?apikey=${apikey}` +
+          `&padID=${encodeURIComponent(padId)}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+      return r.data.revisions;
+    },
+    async compactPad(padId: string, keepRevisions: number | null) {
+      const v = (cliApi as any)._apiVersion;
+      const params = new URLSearchParams({apikey, padID: padId});
+      if (keepRevisions != null) params.set('keepRevisions', String(keepRevisions));
+      const r = await apiPost(`/api/${v}/compactPad?${params.toString()}`);
+      if (r.code !== 0) throw new Error(JSON.stringify(r));
+    },
+  };
Evidence
The new CLI repeats the same “loadSettings → baseURL → apiGet/apiPost → read APIKEY.txt → cache
apiVersion” pattern already present in the existing bulk compaction CLI, but with an additional
endpoint binding. This duplication increases maintenance overhead.

bin/compactStalePads.ts[243-301]
bin/compactAllPads.ts[179-236]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The new `bin/compactStalePads.ts` repeats the same CLI plumbing that exists in `bin/compactAllPads.ts` (settings loading, baseURL, fetch wrappers, APIKEY loading, API version caching). Duplicated wiring increases drift risk.

### Issue Context
This is not a functional bug today, but it increases the cost/risk of future changes (auth behavior, error formatting, fetch options).

### Fix Focus Areas
- bin/compactStalePads.ts[243-301]
- bin/compactAllPads.ts[179-236]

### Suggested fix
Create a small shared helper module (e.g., `bin/_compactCliApi.ts`) that:
- loads settings and constructs `baseURL`
- provides `apiGet`/`apiPost`
- reads `APIKEY.txt`
- resolves and caches `apiVersion`
Then have both CLIs build their endpoint-specific adapters on top of that helper.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread bin/compactStalePads.ts
Comment on lines +1 to +16
'use strict';

/*
* Compact every pad on the instance that has not been edited recently.
*
* Usage:
* node bin/compactStalePads.js --older-than 90 # collapse history on pads not edited in 90 days
* node bin/compactStalePads.js --older-than 90 --keep 50 # keep last 50 revisions
* node bin/compactStalePads.js --older-than 90 --dry-run # list, don't write
*
* Composes `listAllPads` → `getLastEdited` → `compactPad`. Same shape as
* `bin/compactAllPads` (per-pad error tolerance, dry-run, tally), but
* filters by edit-recency before touching anything. Targeting which pads
* to compact is deliberately a CLI concern and not a `compactPad` API
* param — staleness changes from one run to the next, the compaction
* primitive does not.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. No docs for compactstalepads 📎 Requirement gap ⚙ Maintainability

This PR adds a new operator-facing CLI (bin/compactStalePads) but does not update the admin/CLI
documentation under doc/ to describe its usage and flags. This makes the feature undiscoverable
and violates the documentation update requirement.
Agent Prompt
## Issue description
A new CLI tool (`bin/compactStalePads`) was introduced, but the admin/CLI documentation was not updated to describe how to use it (flags `--older-than`, `--keep`, `--dry-run`, and behavior).

## Issue Context
Compliance requires that operator-facing CLI functionality be documented in the appropriate `doc/` admin/CLI documentation so administrators can discover and safely use it.

## Fix Focus Areas
- doc/cli.md[1-200]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Qodo flagged two real issues:

1. Race window between staleness selection and compaction. On a long
   bulk run a pad could become active between first-pass filtering and
   compactPad, which would then kick those sessions. Added a getLastEdited
   recheck right before each compact call; if the pad is now fresh it's
   reclassified as skippedFresh rather than failed (the user did the
   right thing — edited it — and we bow out).

2. doc/cli.md had nothing on pad compaction at all (gap predates this
   PR; ether#6194 landed without doc updates). Added a Pad compaction section
   covering all three CLIs — compactPad, compactAllPads, compactStalePads
   — so the toolset is discoverable as a unit.

Tests cover both the recheck-skip path and a recheck-failure path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@JohnMcLear
Copy link
Copy Markdown
Member Author

Addressed Qodo's findings in d19dc0e:

  1. No docs for compactStalePads (requirement gap) — fixed. doc/cli.md now has a "Pad compaction" section covering all three CLIs (compactPad, compactAllPads, compactStalePads) with flags, examples, and the gating note. The doc gap actually pre-dates this PR — compactPad/compactAllPads from Limit number of versions of a pad or delete them in Pad Settings #6194 had no entry either — so this closes the whole hole.

  2. Staleness check race window (bug) — fixed. Added a getLastEdited recheck immediately before every compactPad call. If the pad has been edited since first-pass selection, it's reclassified as skippedFresh (not failed) and we move on without kicking sessions. Tests cover both the recheck-skip path and a recheck-failure path.

  3. Duplicated CLI wiring logic (advisory) — deferring to a follow-up. Extracting a shared bin/_compactCliApi.ts helper would also rewrite the already-merged bin/compactPad.ts and bin/compactAllPads.ts to consume it — a refactor that's worth doing, but is broader than this issue's scope and would muddy the diff for reviewers tracking the staleness feature. Happy to open a small follow-up PR right after this lands.

/review

@JohnMcLear JohnMcLear requested a review from SamTV12345 May 9, 2026 12:29
@SamTV12345 SamTV12345 merged commit d6a55c2 into ether:develop May 10, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Staleness gating for compactPad: target pads not edited in N days

2 participants