perf(sync): eliminate scheduling delay and increase download buffer#9992
perf(sync): eliminate scheduling delay and increase download buffer#9992qoole wants to merge 1 commit into
Conversation
- Replace 3ms QTimer::singleShot in scheduleNextJob() with QMetaObject::invokeMethod(Qt::QueuedConnection). The _jobScheduled flag already prevents re-entrancy, so the 3ms delay was pure waste. With 10,000 small files this eliminates ~30 seconds of accumulated scheduling latency. - Increase download read buffer from 8KB to 256KB to match the scale of the checksum buffer (500KB). This reduces the number of read/write iterations by 32x for fast downloads while still adapting to actual bytes available via qMin. Signed-off-by: Qoole <2862661+qoole@users.noreply.github.com>
|
Hello there, We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process. Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6 Thank you for contributing to Nextcloud and we hope to hear from you soon! (If you believe you should not receive this message, you can add yourself to the blocklist.) |
1 participant
Summary
Two related sync throughput improvements:
scheduleNextJob3ms timerOwncloudPropagator::scheduleNextJobschedules the next call viaQTimer::singleShot(3, ...). The 3ms delay was originally there to debounce re-entry, but the_jobScheduledflag at the top of the function already prevents that — once set, repeat calls return immediately.Replaced with
QMetaObject::invokeMethod(Qt::QueuedConnection)which posts to the event loop with no artificial delay.With 10,000 small files this delay accumulates to roughly 30 seconds of scheduling latency that is pure idle time on the event loop. On a sync of many tiny files this is the dominant cost of the propagator.
Download read buffer
GETFileJob::newReplyHookandslotMetaDataChangedboth setreply->setReadBufferSize(16 * 1024). The original comment says "keep low so we can easier limit the bandwidth", butBandwidthManagerquantizes byqMin(bytesAvailable, quota)perreadyRead, so the buffer size only changes the maximum granularity of a single read, not the limiting precision.Increased to
256 * 1024to match the order of magnitude of the checksum buffer (500KB). Bandwidth limits still apply correctly via the per-readqMinclamp; they simply operate on slightly larger reads when no limit is set. For unthrottled downloads this reduces read/write iteration count by 16x.Checklist
AI (if applicable)