Skip to content

fix(upsert): use bounded file match filter to avoid scan planning segfault#11

Open
abnobdoss wants to merge 1 commit into
base/current-mainfrom
fix/upsert-file-match-filter
Open

fix(upsert): use bounded file match filter to avoid scan planning segfault#11
abnobdoss wants to merge 1 commit into
base/current-mainfrom
fix/upsert-file-match-filter

Conversation

@abnobdoss
Copy link
Copy Markdown
Owner

@abnobdoss abnobdoss commented May 26, 2026

Summary

Use a bounded file match filter for the initial upsert target scan. The exact match filter remains in place for overwrite and insert filtering.

Why

Composite-key upserts can build very large exact predicate trees before scanning target files. On affected runtimes that recursive predicate shape can overflow during scan planning. The bounded filter keeps initial scan predicate size tied to the number of join columns instead of the number of source keys.

Testing

  • Reproduced the base-branch failure locally with the large composite-key scan-planning path; the Python 3.10 subprocess exited 139
  • uv run python -m pytest tests/table/test_upsert.py -k "large_composite_key_initial_scan or file_match_filter or create_match_filter"
  • uv run python -m pytest tests/table/test_upsert.py
  • commit hooks

@abnobdoss abnobdoss force-pushed the fix/upsert-file-match-filter branch from 89dfcf6 to 3692011 Compare May 26, 2026 23:59
@abnobdoss abnobdoss changed the title perf(upsert): use bounded file match filter fix(upsert): use bounded file match filter to avoid scan planning segfault May 27, 2026
@abnobdoss abnobdoss force-pushed the fix/upsert-file-match-filter branch from 3692011 to bbfc654 Compare May 27, 2026 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant