o Meةم@sbddlZddlmmZddlmZddd„Zddd„Zdd „Zd d„Z ddd„Z dS)éN)ع collectiveécCs2t|ƒ}|d|<t|jƒ|krtdƒ‚dS)Nrz1The in_tensor for scatter is not correctly-sized.)عlistعshapeعRuntimeError)عtensorrعnranksZexpect_shape©r ْ_D:\Projects\ConvertPro\env\Lib\site-packages\paddle/distributed/communication/stream/scatter.pyع_check_tensor_shapes ےrcCs4t|ƒ|kr tdƒ‚|D]}|j|krtdƒ‚qdS)Nz3The tensor_list for scatter is not correctly-sized.)عlenrr)عtensor_listrrrr r r ع_check_tensor_list_shapesے ےےےrc Csٹ|durt ،n|}| |،}|dkrtdƒ‚|j}tj ،}||kr*t||j |ƒ|r4|j |||،S|j ||||،} |rC| ،| S)NéےےےےْSrc rank out of group.)rع_get_default_groupعget_group_rankrrعpaddleعdistributedعget_rankrrع process_groupZscatter_tensor_on_calc_streamZscatter_tensorعwait) Z out_tensorZ in_tensorعsrcعgroupعsync_opعuse_calc_streamعsrc_rankrعrankعtaskr r r ع_scatter_tensor_in_dygraph%s$ ےےrc s¶|durt ،n|}| |،}|dkrtdƒ‚|j}tj ،}||kr5t|ƒdkr-tdƒ‚t |ˆj |ƒn‡fdd„t|ƒDƒ}|rJ|j |ˆ|،S|j |ˆ||،} |rY| ،| S)Nrrrz0The tensor_list should not be empty on src rank.csg|]}ˆ‘qSr r )ع.0ع_©rr r ع Nsz'_scatter_in_dygraph..)rrrrrrrrrrrعrangerZscatter_on_calc_streamعscatterr) rr rrrrrrrrr r"r ع_scatter_in_dygraph>s* ےےr&TFcCsv|dur| ،stdƒ‚|s|rtdƒ‚|durtdƒ‚t ،r7t |،r.t||||||ƒSt||||||ƒStdƒ‚)a¼ Scatter a tensor (or a tensor list) across devices. Args: tensor (Tensor): The output tensor on each rank. The result will overwrite this tenor after communication. Support float16, float32, float64, int32, int64, int8, uint8 or bool as the input data type. tensor_or_tensor_list (Union[Tensor, List[Tensor]]): The input to scatter (default is `None`, must be specified on the source rank). If it is a tensor, it should be correctly-sized. If it is a list, it should contain correctly-sized tensors. src (int, optional): Rank of the source device. If none is given, use `0` as default. group (Group, optional): Communicate in which group. If none is given, use the global group as default. sync_op (bool, optional): Indicate whether the communication is sync or not. If none is given, use true as default. use_calc_stream (bool, optional): Indicate whether the communication is done on calculation stream. If none is given, use false as default. This option is designed for high performance demand, be careful to turn it on except you are clearly know its meaning. Returns: Return a task object. Warning: This API only supports the dygraph mode now. Examples: .. code-block:: python # required: distributed import paddle import paddle.distributed as dist dist.init_parallel_env() if dist.get_rank() == 0: data1 = paddle.to_tensor([7, 8, 9]) data2 = paddle.to_tensor([10, 11, 12]) dist.stream.scatter(data1, src=1) else: data1 = paddle.to_tensor([1, 2, 3]) data2 = paddle.to_tensor([4, 5, 6]) dist.stream.scatter(data1, [data1, data2], src=1) out = data1.numpy() # [1, 2, 3] (2 GPUs, out for rank 0) # [4, 5, 6] (2 GPUs, out for rank 1) NzjThe group should not be None and all ranks which invoke this operation should be the member of this group.z5use_calc_stream can only be true in sync op behavior.zThe input should be specified.zHpaddle.distributed.stream.scatter is only supported in dygraph mode now.)Z is_memberrع frameworkZin_dygraph_moderZ is_tensorrr&)rZtensor_or_tensor_listrrrrr r r r%[s,/ےے ‏ےےr%)r)NrNTF)rZpaddle.fluid.frameworkZfluidr'Zpaddle.distributedrrrrr&r%r r r r عs û