Skip to content

Make distributed fused lamb test names friendly to keyword filtering#1698

Open
crcrpar wants to merge 3 commits intoNVIDIA:masterfrom
crcrpar:spell-out-combination-of-dist-fused-lamb
Open

Make distributed fused lamb test names friendly to keyword filtering#1698
crcrpar wants to merge 3 commits intoNVIDIA:masterfrom
crcrpar:spell-out-combination-of-dist-fused-lamb

Conversation

@crcrpar
Copy link
Copy Markdown
Collaborator

@crcrpar crcrpar commented Jul 20, 2023

This is merely cosmetic as the current parametrization creates test case names such as NcclDistributedFusedLAMB.test_distributed_fused_lamb_no_copy_True_opt_kwargs_{'overlap_reductions': False, 'dwu_num_blocks': 1, 'dwu_num_chunks': 1, 'fused_norm': True, 'fuse_scale': True, 'clip_after_ar': False} whic is not ideally friendly to keyword filtering.


it might be worth thinking of removing no_copy check here

# Check if collectives have no_copy option
self._reduce_scatter_no_copy = (
'no_copy' in inspect.getfullargspec(torch.distributed.reduce_scatter).args
)
self._all_gather_no_copy = (
'no_copy' in inspect.getfullargspec(torch.distributed.all_gather).args
)

cc @ptrblck @eqy @Aidyn-A

Signed-off-by: Masaki Kozuki <mkozuki@nvidia.com>
crcrpar added 2 commits July 31, 2023 14:00
Signed-off-by: Masaki Kozuki <mkozuki@nvidia.com>
Signed-off-by: Masaki Kozuki <mkozuki@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant