Add AVX-512 support#231
Open
Shnatsel wants to merge 40 commits into
Open
Conversation
…edicated AVX-512 implementations for complex int/float vector operations that benefit the most. LLM summary of the changes: Implemented: - Added `X86::Avx512` in the generator with Ice Lake feature set, `native_width = 512`, `max_block_size = 512`. - Generated new `fearless_simd/src/generated/avx512.rs`. - Wired public API: `Avx512`, `x86::Avx512`, `Level::Avx512`, `Level::as_avx512`, dispatch, and `kernel!` support. - Updated runtime/static detection so Ice Lake AVX-512 is selected before AVX2, while `as_avx2()` and `as_sse4_2()` downgrade correctly. - Bumped MSRV/docs/CI/check-target metadata to Rust 1.89. Generator/backend behavior: - 512-bit vectors use native `__m512`, `__m512d`, and `__m512i`. - AVX-512 masks now use raw compact `__mmask8/16/32/64` storage, with no aligned wrapper. - Generic `SimdFrom<__mmask*, S>` / `From<mask*, __mmask*>` now route through `from_bitmask` / `to_bitmask`, so they are correct for non-AVX-512 `S` too. - Added AVX-512 compare/select paths using mask-returning compares and mask blends. - Added direct conversion paths, including `f32 <-> i32/u32` and `u8 <-> u16`. - Added AVX-512 vector slides for vectors only; masks intentionally have no slide support. - Added dedicated AVX-512 zip/unzip/interleave/deinterleave using `permutex2var`, especially for 256/512-bit widths. Tests/coverage: - Extended `#[simd_test]` to include AVX-512. - Added AVX-512 detection/dispatch coverage. - Updated mask bitwise tests for canonical boolean mask lanes. - Added a regression test that AVX-512 mask public types are compact and match `__mmask*` sizes.
…nt the spooky bug I almost introduced
…rage for these ops.
…calar, now we use the dedicated intrinsics.
…ackend, and specialize it for AVX-512. Add test coverage that sets every single bit and verifies it was set correctly.
… test to exercise it. i8/u8 test is still bad because of rust-lang/rust#156891
…rage. Only for 8-bit left shift LLVM autovectorizes the scalar fallback into GFNI instructions on 256-bit halves which emits more instructions but schedules better and ends up being slightly faster according to llvm-mca on sapphire rapids; but the difference isn't huge and I don't want to rely on autovectorization because of its fragility.
…it vectors on AVX-512; expand test coverage
… no cost to throughput
… so they didn't show up earlier when I removed those methods.
Closed
…e get dead code warnings
…ppy --tests` without a reported location, I've failed to isolate it to a specific crate and suppress it there
Collaborator
|
I think it would indeed be great to have a custom PR for 3. |
Contributor
Author
|
It will cause a lot of conflicts if I try to split it, but I have it isolated to its own commit at least: f08f7e6 |
…an't enforce Pod without an external dependency.
Contributor
Author
|
I've run miri on the entire test suite (minus the exhaustive tests that have opt-outs) and filtered out the tests that fail due to unsupported intrinsics. SSE4.2, AVX2 and AVX-512 all pass. |
Merged
# Conflicts: # fearless_simd/src/generated/avx2.rs # fearless_simd/src/generated/neon.rs # fearless_simd/src/generated/sse4_2.rs # fearless_simd/src/generated/wasm.rs # fearless_simd_gen/src/generic.rs # fearless_simd_gen/src/level.rs
Shnatsel
added a commit
to Shnatsel/fearless_simd
that referenced
this pull request
May 25, 2026
"we have `bytemuck` at home" These are the `transmute_copy()` safety improvements from linebender#231 taken one step further: on top of checking the sizes match at compile time and that both types are `Copy`, we also add a marker trait to types that are safe to transmute and require it in `checked_transmute_copy()`, making that function fully safe to call. The design follows the `bytemuck` crate closely. We could just use `bytemuck`, but it would pull in `syn` and other proc macro machinery, and we would still have to write the `impl_aligned_simd_pod!` macro because bytemuck rightfully refuses to derive `Pod` on generic wrappers like `Aligned128<T>` because they may introduce padding depending on the `T`. So the amount of code it saves us is minimal. Rebasing linebender#231 on top of this would be hellish, but a merge commit probably wouldn't be too bad.
…ame name but different semantics from the production code to avoid confusion
…VX-512 has configurable comparison modes that we can use to implement the advertised _precise semantics.
Merged origin/main commit 13dd530. The merge applied cleanly.
Replace AVX512 interleaved load intrinsics emitted by the branch with checked_transmute_copy, then regenerate the generated AVX512 module.
Merged origin/main commit fbc97da. The merge applied cleanly.
Regenerate the branch-added AVX512 module so by-value transmutes use checked_transmute_copy, matching PR linebender#234. Validation: cargo test
Merged origin/main commit 0d13b0a. The merge applied cleanly.
Regenerate the branch-added AVX512 module so reference casts use checked_cast_ref and checked_cast_mut. Also apply the float bit-pattern assertion style from PR linebender#235 to the branch-added f32x16 interleaved-load test. Validation: cargo test
Merged origin/main commit 650815d. The merge applied cleanly.
PR linebender#237 only updates NEON load construction. The AVX512 branch-specific unsafe load sites were already adapted in the PR linebender#233 follow-up, and a search found no remaining load intrinsics needing the linebender#237 pattern.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Yes, really. It's all here. In one humongous PR. Sorry 😅
This is probably best reviewed commit-by-commit. The first commit is still big because the history was getting really messy with changes and rollbacks, and squashing it made it less of a mess.
This also touches other backends in three ways:
set_mask()is now a backend method so it could be specialized per-leveltransmute_copy()is wrapped intochecked_transmute_copy()and the raw version disallowed after I almost had a horrible accident with it. This could be its own PR but I wanted the insurance right away.Everything changed here should be covered by tests. I've expanded test coverage where it was lacking.