[Relax][Frontend][TFLite] Support StableHLO region-based ops and multi-subgraph models#19587
Open
Aharrypotter wants to merge 11 commits into
Open
[Relax][Frontend][TFLite] Support StableHLO region-based ops and multi-subgraph models#19587Aharrypotter wants to merge 11 commits into
Aharrypotter wants to merge 11 commits into
Conversation
… layout np.indices returns (rank, *update_shape) but scatter_nd expects (*update_shape, rank). Add np.moveaxis to transpose the coordinate axis from first to last position.
Contributor
There was a problem hiding this comment.
Code Review
This pull request significantly expands the TFLite frontend's support for StableHLO operators in TVM Relax. It introduces conversions for several complex operations, including REDUCE, SCATTER, SORT, CONVOLUTION, and DOT_GENERAL. Additionally, it adds support for handling multiple subgraphs, which are used as region bodies for these operators. Feedback suggests simplifying the STABLEHLO_CBRT implementation using relax.op.sign and relax.op.abs for better conciseness and potential backend efficiency. Another suggestion points out an opportunity to optimize constant handling during the decomposition of STABLEHLO_COMPOSITE operators by reusing existing expressions instead of re-creating them.
Contributor
Author
|
cc @tlopex |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds Relax TFLite frontend support for 10 additional StableHLO builtin
operators from #19519 item I, building on the 29 ops merged in PR #19536.
The first 5 ops are direct single-subgraph converters:
CBRT,REMAINDER,DYNAMIC_UPDATE_SLICE,DOT_GENERAL, andCONVOLUTION. The remaining 5 opsare region/subgraph-based:
REDUCE,REDUCE_WINDOW,SORT,SCATTER, andCOMPOSITE. To support these, the TFLite frontend is extended to acceptmulti-subgraph models while still converting only
Subgraphs(0)into theRelax main function. Region subgraphs are consumed by their parent op
converters as needed.
Relates to #19519.
Changes
Single-subgraph ops
CBRT— sign-preserving composite expression:where(x < 0, -power(-x, 1/3), power(x, 1/3)). Float dtype only.REMAINDER— truncating remainder viax - y * trunc(x / y), matchingStableHLO semantics (sign follows dividend). Float dtype only.
DYNAMIC_UPDATE_SLICE— static start indices + static shapes only, loweredto
R.scatter_ndwith a coordinate grid generated vianp.indices.Runtime starts and out-of-bounds ranges raise
OpNotImplemented.DOT_GENERAL— canonical 2D matmul subset: no batching dims,lhs_contracting=[1],rhs_contracting=[0], lowered toR.matmul.CONVOLUTION— canonical 2D NHWC/HWIO subset withBatchGroupCount=1,FeatureGroupCount=1, lowered toR.nn.conv2d. Non-canonical dimensionnumbers and grouped/depthwise conv raise
OpNotImplemented.Multi-subgraph infrastructure
from_tflite()assertion frommodel.SubgraphsLength() == 1tomodel.SubgraphsLength() >= 1. OnlySubgraphs(0)is converted into theRelax main function.
_input_type()toSubgraphs(0)inputs, preventing regionparameters from leaking as Relax main function parameters.
_get_stablehlo_simple_body_ophelper for validating and extractingthe single operator from a region body subgraph.
_finish_tflite_modelwithextra_subgraphsparameterfor constructing multi-subgraph TFLite flatbuffers.
Region/subgraph ops
REDUCE— single-op reducer body subgraph. SupportsADD→R.sum,MAXIMUM→R.max,MINIMUM→R.min,MULTIPLY→R.prod.Init value must match the reducer identity element.
SORT— single-op comparator body subgraph.LT→ ascending sort,GT→ descending sort viaR.sort.IsStableis not mapped.REDUCE_WINDOW— NHWC 4D 2D-pooling subset withMAXIMUMreducer andidentity init, lowered to
R.nn.max_pool2d. BaseDilations must be all 1.SCATTER— single-op update computation body subgraph. SupportsADD/MAXIMUM/MINIMUM/MULTIPLY→R.scatter_ndwith thecorresponding reduction mode. Only canonical point-update semantics
(no window dims).
COMPOSITE— inlines a decomposition subgraph through a recursiveOperatorConverterwith an isolatedExprTable, so decomposition tensorbindings cannot overwrite main graph bindings. Only supports composites
without
CompositeAttributes.Not included
STABLEHLO_RESHAPE,STABLEHLO_TRANSPOSE, andSTABLEHLO_SLICEareleft to another contributor.
WHILE,CUSTOM_CALL, andRNG_BIT_GENERATORare deferred to follow-upPRs.
Bug fix
DYNAMIC_UPDATE_SLICEscatter_nd indices layout:np.indicesreturns
(rank, *update_shape)butscatter_ndexpects(*update_shape, rank). Addednp.moveaxisto transpose the coordinateaxis from first to last position.
Testing
All tests use manually-built minimal TFLite flatbuffers with
tvm.ir.assert_structural_equal. Region/subgraph tests construct the smallestvalid body/comparator/update subgraphs. BuiltinOptions2 ops construct their
options via the FlatBuffers schema API.
Result
39 StableHLO operators registered in the Relax TFLite frontend (29 from
PR [Relax][Frontend][TFLite] Add initial StableHLO builtin operator support #19536 + 10 from this PR).
77 StableHLO test cases covering all registered ops, including
structural-equal tests and unsupported/error-path checks:
REMAINDERtruncating semanticsDYNAMIC_UPDATE_SLICEwith dynamic starts and out-of-bounds startsDOT_GENERALwith non-canonical contracting dimensionsCONVOLUTIONwith non-canonical dimension numbers andFeatureGroupCount > 1REDUCEwith unsupported reducer and non-identity init valueSORTwith unsupported comparator and stable sortREDUCE_WINDOWwith unsupported reducer and base dilationSCATTERwith unsupported reducer and update window dimsCOMPOSITEwith composite attributes and scope isolationAll 77 StableHLO tests pass.
References