Add experimental block cache bypass via GRAPH_STORE_IGNORE_BLOCK_CACHE#6457
Merged
Add experimental block cache bypass via GRAPH_STORE_IGNORE_BLOCK_CACHE#6457
Conversation
incrypto32
approved these changes
Mar 26, 2026
Member
incrypto32
left a comment
incrypto32
left a comment
There was a problem hiding this comment.
Just few minor nit picks, feel free to ignore and merge as it is.
node/src/store_builder.rs
Outdated
| let (shard, cache_size) = chains | ||
| .get(&name) | ||
| .cloned() | ||
| .unwrap_or_else(|| (PRIMARY_SHARD.clone(), 500)); |
Member
There was a problem hiding this comment.
minor: This should use default_cache_size()
store/postgres/src/block_store.rs
Outdated
| shards: Vec<(String, Shard)>, | ||
| shards: Vec<(String, Shard, BlockNumber)>, | ||
| // Per-chain cache_size settings | ||
| cache_sizes: HashMap<String, BlockNumber>, |
Member
There was a problem hiding this comment.
very minor nit: cache_sizes duplicates data already in shards. Could look it up with .find() or into a struct, but not a big deal, feel free to ignore
store/postgres/src/block_store.rs
Outdated
| ); | ||
| let ident = chain.network_identifier()?; | ||
| let logger = self.logger.new(o!("network" => chain.name.clone())); | ||
| let cache_size = self.cache_sizes.get(&chain.name).copied().unwrap_or(500); |
Member
There was a problem hiding this comment.
This magic number 500 is duplicated in a few places, better to make this a constant somewhere?
Add a boolean env var that, when set, will cause block reads for blocks outside the per-chain cache_size window to behave as if the data field is null. This is the first step toward experimenting with reduced block caching before the full block cache revamp.
lutter
force-pushed
the
lutter/block-sim
branch
from
9bc9d48 to
3aaba10
Compare
Add a per-chain `cache_size` configuration parameter that controls how many blocks from chain head to keep in the block cache. Defaults to 500. Validated to be greater than reorg_threshold.
Pass the per-chain cache_size configuration from TOML config through StoreBuilder and BlockStore to ChainStore, where it will be used to determine which blocks should be treated as uncached.
When GRAPH_STORE_IGNORE_BLOCK_CACHE is set, block reads for blocks that are more than cache_size blocks behind the chain head now behave as if the block doesn't exist in the cache. This allows experimenting with the effects of reduced block caching before the full block cache revamp.
lutter
force-pushed
the
lutter/block-sim
branch
from
3aaba10 to
b438d8d
Compare
Make ChainSection.cache_size the default for chains that don't set cache_size explicitly. Chains deserialized without cache_size get 0 as a sentinel; ChainSection::validate() then fills in the section-level default before the reorg_threshold check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We want to reduce the size of the block cache by not keeping old entries. This PR makes it possible to simulate a pruned block cache without actually deleting the cached data yet. It's an intermediate step to automatically pruning the block cache.
For each chain, there is now a setting
cache_sizethat determines how many blocks behind chain head to cache. It defaults to 500 and must be larger than the reorg threshold. For now, this setting only has an effect whenGRAPH_STORE_IGNORE_BLOCK_CACHE=true. In that case, the system behaves as if the block data had been purged but headers are still present.