很楠不爱3

这个屌丝很懒，什么也没留下！

热门标签

热门文章

当前位置: article > 正文

torch.distributed

作者：很楠不爱3 | 2024-04-14 05:57:05

赞

踩

torch.distributed

参考 torch.distributed.init_process_group() - 云+社区 - 腾讯云

目录

PyTorch自带的后端

使用哪个后端？

常见的环境变量

torch.distributed.is_available()[source]

torch.distributed.init_process_group(backend, init_method=None, timeout=datetime.timedelta(0, 1800), world_size=-1, rank=-1, store=None, group_name='')[source]

class torch.distributed.Backend

torch.distributed.get_backend(group=)[source]

torch.distributed.get_rank(group=)[source]

torch.distributed.get_world_size(group=)[source]

torch.distributed.is_initialized()[source]

torch.distributed.is_mpi_available()[source]

torch.distributed.is_nccl_available()[source]

共享文件系统初始化

环境变量初始化

torch.distributed.new_group(ranks=None, timeout=datetime.timedelta(0, 1800), backend=None)[source]

点对点通信

torch.distributed.send(tensor, dst, group=, tag=0)[source]

torch.distributed.recv(tensor, src=None, group=, tag=0)[source]

torch.distributed.isend(tensor, dst, group=, tag=0)[source]

torch.distributed.irecv(tensor, src, group=, tag=0)[source]

同步和异步收集操作

torch.distributed.broadcast(tensor, src, group=, async_op=False)[source]

torch.distributed.all_reduce(tensor, op=ReduceOp.SUM, group=, async_op=False)[source]

torch.distributed.reduce(tensor, dst, op=ReduceOp.SUM, group=, async_op=False)[source]

torch.distributed.all_gather(tensor_list, tensor, group=, async_op=False)[source]

torch.distributed.gather(tensor, gather_list=None, dst=0, group=, async_op=False)[source]

torch.distributed.scatter(tensor, scatter_list=None, src=0, group=, async_op=False)[source]

torch.distributed.reduce_scatter(output, input_list, op=ReduceOp.SUM, group=, async_op=False)[source]

torch.distributed.all_to_all(output_tensor_list, input_tensor_list, group=, async_op=False)[source]

torch.distributed.barrier(group=, async_op=False)[source]

class torch.distributed.ReduceOp

class torch.distributed.reduce_op[source]

多GPU收集函数

torch.distributed.broadcast_multigpu(tensor_list, src, group=, async_op=False, src_tensor=0)[source]

torch.distributed.all_reduce_multigpu(tensor_list, op=ReduceOp.SUM, group=, async_op=False)[source]

torch.distributed.reduce_multigpu(tensor_list, dst, op=ReduceOp.SUM, group=, async_op=False, dst_tensor=0)[source]

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/很楠不爱3/article/detail/420503

推荐阅读

相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

闽ICP备14008679号