perf: batch E2EE queries, batch dir rename, hash lookup, larger Windows watcher buffer#9993
perf: batch E2EE queries, batch dir rename, hash lookup, larger Windows watcher buffer#9993qoole wants to merge 1 commit into
Conversation
…sh lookup - Batch E2EE path queries: replace N serial getFileRecord calls with single IN query for getRootE2eFolderRecord and findEncryptedAncestorForRecord. With deeply nested encrypted folders this reduces sync startup latency by collapsing per-level DB round trips into a single statement. - Batch directory rename: collect child records first, then process mutations after the streaming query completes. Avoids interleaved read/write on the same statement and lets the journal commit cleanly. Commit the journal once before the batch loop so each child rename starts from a clean transaction boundary instead of accumulating in the previous batch's transaction. - Remove Instantiator destroy/recreate hack on tray open / menu close (eliminates full delegate teardown + rebuild on every interaction). - O(n^2) to O(1) folder adjustment: QHash lookup replaces std::find_if in adjustDeletedFoldersWithNewChildren. Builds a name index once and reuses it instead of rescanning per item. - Windows file watcher buffer: 40KB->256KB start, 64KB->2MB max. Larger initial buffer prevents buffer overflow under heavy file churn, which was triggering expensive full-tree rescans. Signed-off-by: Qoole <2862661+qoole@users.noreply.github.com>
qoole
force-pushed
the
perf/e2ee-batch-and-rename
branch
from
7fe0d3d to
c260fac
Compare
|
Hello there, We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process. Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6 Thank you for contributing to Nextcloud and we hope to hear from you soon! (If you believe you should not receive this message, you can add yourself to the blocklist.) |
1 participant
Summary
A set of related performance fixes touching sync startup, propagation, and the tray.
Batch E2EE path queries (
syncjournaldb.cpp)getRootE2eFolderRecordandfindEncryptedAncestorForRecordpreviously walked up the path one level at a time, issuing a separategetFileRecordper level. For a path 8 levels deep that's 8 round trips to SQLite on every E2EE-related lookup.Replaced with a single
WHERE path IN (...)query that fetches all candidate ancestors at once. The result is then walked locally to find the deepest e2e-encrypted match.Batch directory rename (
propagatorjobs.cpp)PropagateLocalRename::startpreviously processed child journal entries inline while iterating a streamingSELECT. Mixing reads and writes on the same SQLite statement is fragile and prevented batching. Refactored to:SELECTis open.This makes the operation safer and lets the journal batch its writes.
Remove
Instantiatordestroy/recreate hack (MainWindow.qml,CurrentAccountHeaderButton.qml)The tray was destroying and recreating its account
Instantiatoron every menu open/close as a workaround for an old binding glitch. This causes a full delegate teardown and rebuild every time the user opens the tray. The hack is no longer needed; removed it.O(n²) -> O(1) folder adjustment (
owncloudpropagator.cpp)adjustDeletedFoldersWithNewChildrenlooked up each item withstd::find_ifover the full vector, making the function O(n²). Replaced with aQHash<QString, int>index built once and reused for each lookup.Windows file watcher buffer (
folderwatcher_win.cpp)The
ReadDirectoryChangesWbuffer was 40KB initial / 64KB max, which overflows easily under heavy file churn (e.g. large extracts, IDE save cascades). On overflow Windows reports a single "buffer overflow" event and the watcher falls back to a full-tree rescan, which is extremely expensive on large folders.Bumped to 256KB initial / 2MB max. The buffer only grows as needed.
Checklist
AI (if applicable)