perf(app): parallelize per-tx gas classification in checkTotalBlockGas#3399
perf(app): parallelize per-tx gas classification in checkTotalBlockGas#3399amir-deris wants to merge 10 commits into
Conversation
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3399 +/- ##
==========================================
- Coverage 59.24% 59.07% -0.17%
==========================================
Files 2110 2105 -5
Lines 174149 173179 -970
==========================================
- Hits 103175 102311 -864
+ Misses 62041 61978 -63
+ Partials 8933 8890 -43
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
masih
left a comment
There was a problem hiding this comment.
Nice; thank you @amir-deris for picking this work up so quickly. this is impactful!
The current implementation works so technically no hard blocker.
However, I have left a bunch of comments that are worth addressing which should result in a more optimal solution.
🍻
| var ctxMu sync.Mutex | ||
| var wg sync.WaitGroup | ||
|
|
||
| const maxConcurrent = 32 |
There was a problem hiding this comment.
Document the rationale for 32; if there is none please either paremeterise it in config or make it a factor of CPU cores with capped min and max.
| } | ||
| sem <- struct{}{} | ||
| wg.Add(1) | ||
| go func() { |
There was a problem hiding this comment.
What you want here is a worker pool, right?
This implementation works, but: it creates a goroutine per tx. The idea is to use a pool of workers across which work is distributed. Specially since the max degree of concurrency is fixed.
Using the worker pool pattern here means we will have less allocations and the chances are if the work completes fast we might need to instantiate less than 32 concurrent goroutines (ignoring the obvious gain from goroutine reuse at max concurency)
| totalGas, totalGasWanted := uint64(0), uint64(0) | ||
| for _, r := range results { | ||
| if r.malicious { | ||
| return false |
There was a problem hiding this comment.
Why proceed with the remaining concurrent processing if a malicious result is found?
Have a look at err group. that primitive SDK type allows you to then return an error and as a result cancel all remaining concurrent work, which would make a more effective optimisation given that the point of this PR is exactly that.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 016d8f7. Configure here.

Summary
checkTotalBlockGasiterates every tx in a proposed block to compute total gas. Each tx's classification (EVM check, gasless check, gas extraction) is independent, making the loop a natural fan-out target.Changes
checkTotalBlockGas— replaced the serial loop with a two-phase design:classifyTxForGas. Results are written into a pre-allocated[]txGasResultslice indexed by position (no lock needed on the slice).MaxGasWanted/MaxGaslimit enforcement.classifyTxForGas— extracted helper that owns all per-tx logic. EVM txs (the vast majority) run fully unlocked. The non-EVMIsTxGaslesspath takes actxMumutex becausesdk.Contextwraps aCacheMultiStorethat is not goroutine-safe.txGasResult— small struct (gasWanted,gasContrib,skip,malicious) that carries each tx's outcome between phases.nonzeroTxsCnt— it was declared and incremented but never read.Safety
results[i]slot;ctxMuserializes the only shared-state reader (IsTxGasless).ctxMuis uncontested — oracle-vote txs are the only non-EVM path that acquires it.Improvements
TBC
Note
Medium Risk
Touches consensus-critical proposal validation logic and introduces concurrency (goroutines, shared context locking), which could cause subtle race/ordering issues if incorrect despite added guards and tests.
Overview
Parallelizes
checkTotalBlockGasper-tx classification by fanning out tx inspection to a bounded worker pool (max 32) and then performing a serial accumulation/enforcement pass againstMaxGasWanted/MaxGas.Adds
txGasResult/txGasClassifyJoband extractsclassifyTxForGasto encapsulate per-tx gasless/EVM/associate handling, including per-worker panic recovery and an atomic "malicious" short-circuit; uses a mutex to serializeIsTxGaslessdue to non-goroutine-safesdk.Context.Expands tests to cover gas-estimate selection/fallback,
MaxGasWantedenforcement independent ofMaxGas, behavior with many txs (>32), and nil/empty tx slices.Reviewed by Cursor Bugbot for commit edf945e. Bugbot is set up for automated code reviews on this repo. Configure here.