perf: async .eml checksum, PROPFIND size reuse, E2EE parallel transfers#9995
perf: async .eml checksum, PROPFIND size reuse, E2EE parallel transfers#9995qoole wants to merge 1 commit into
Conversation
- Make .eml checksum computation async during discovery to avoid blocking the event loop on large mailbox syncs. The previous synchronous computeLocalChecksum could pause discovery for several seconds on large .eml files; replaced with a ComputeChecksum job that defers the result via finalize(). - Reuse folder size from the PROPFIND response (RemoteInfo.sizeOfFolder) instead of issuing a redundant PropfindJob for selective sync size checks. The size is already available from the parent PROPFIND that populated serverEntry; an extra round trip per directory is wasteful. - Allow E2EE directories to report FullParallelism so encrypted folders on different paths can transfer concurrently. Within-folder serialization is preserved by PropagateItemJob, so this only relaxes the cross-folder constraint that previously serialized all E2EE transfers globally. Signed-off-by: Qoole <2862661+qoole@users.noreply.github.com>
|
Hello there, We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process. Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6 Thank you for contributing to Nextcloud and we hope to hear from you soon! (If you believe you should not receive this message, you can add yourself to the blocklist.) |
1 participant
Summary
Three independent throughput improvements to discovery and propagation.
Async
.emlchecksum during discovery (discovery.cpp)processFileAnalyzeLocalInfocomputes a local checksum to detect false positives on.emlfile change detection (issues #4754, #4755). The computation was synchronous and could block the discovery thread for seconds on large mailbox files.Replaced with an async
ComputeChecksumjob. The result is captured into the existingfinalize()lambda, so flow is preserved: if the computed checksum matches the database, the item is downgraded toUPDATE_METADATA; otherwise discovery continues normally. While the checksum runs,_pendingAsyncJobskeeps the discovery state machine alive and other work proceeds.PROPFIND folder-size reuse (
discoveryphase.cpp/.h,discovery.cpp)checkSelectiveSyncNewFolderandcheckSelectiveSyncExistingFolderwere issuing a separatePropfindJobfor each newly-encountered remote directory just to learn its size. The parent PROPFIND that producedserverEntryalready has the folder size inRemoteInfo.sizeOfFolder.Both functions now accept the size as a parameter and skip the redundant PROPFIND when it's already known. On a large remote tree this eliminates one HTTP round trip per directory.
E2EE FullParallelism (
owncloudpropagator.cpp)OwncloudPropagator::pushDirectoryToPropagateJobwas forcing E2EE directories toWaitForFinished, which serialized all encrypted folder transfers globally — even folders on completely different paths. Within-folder ordering is already enforced byPropagateItemJob; the cross-folder serialization was a stronger guarantee than needed.Allow E2EE directories to report
FullParallelism. Encrypted folders on different paths can now transfer concurrently. Within-folder serialization is preserved.Checklist
AI (if applicable)