Skip to content

chore: avoid hammering store at ntx builder startup#2064

Open
SantiagoPittella wants to merge 4 commits intomainfrom
santiagopittella-avoid-ntx-builder-spam-main
Open

chore: avoid hammering store at ntx builder startup#2064
SantiagoPittella wants to merge 4 commits intomainfrom
santiagopittella-avoid-ntx-builder-spam-main

Conversation

@SantiagoPittella
Copy link
Copy Markdown
Collaborator

@SantiagoPittella SantiagoPittella commented May 8, 2026

closes #1952

This PR:

  • Trusts local committed account state on restart and only fetches the unconsumed_notes delta per known account, dropping the per-account GetNetworkAccountDetailsById call.
  • Adds a configurable hydration semaphore in StoreClient (defaulted to 4) that bounds concurrent in-flight startup RPCs.
  • Adds a separate store_sync_checkpoint column on chain_state (required a new SQL migration), distinct from the mempool-driven block_num.

I tried it by:

  1. initializing the main version of the node.
  2. Started multiple network monitors to create network accounts.
  3. Stoped everything.
  4. Installed and initialized the new version with the existing data (i.e. I didn't re-run bootstrap), which ran the migration
  5. Everything continue working as expected.

@SantiagoPittella SantiagoPittella changed the title chore: avoid hamming store at ntx builder startup chore: avoid hammering store at ntx builder startup May 8, 2026
Comment thread crates/ntx-builder/src/clients/store.rs Outdated
Comment thread crates/ntx-builder/src/db/models/queries/mod.rs Outdated
Comment thread crates/ntx-builder/src/db/mod.rs Outdated
Comment thread crates/ntx-builder/src/builder.rs Outdated
/// Bounds the burst of `GetNetworkAccountDetailsById` and `GetUnconsumedNetworkNotes` calls the
/// ntx-builder fires when catching up after a restart.
const DEFAULT_STORE_HYDRATION_CONCURRENCY: NonZeroUsize =
NonZeroUsize::new(4).expect("literal is non-zero");
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know how slow things could be because of this?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, because my local DB isn't big enough to actually measure this. It would be really nice to have a copy of the DB in a server with similar specs to the devnet/testnet ones to compare it. Or at least running it locally with the testnet database.

Do you know if I can get access to it? maybe @Mirko-von-Leipzig ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants