o MeIã@s†ddlZddlZddlZddlZddlmZmZddlm Z ddl m Z m Z ddl mZddlmZddlmZd d d „Zd d „ZdS)éN)Ú dygraph_onlyÚ _global_flagsé)ÚDistributedStrategy)ÚHybridParallelOptimizerÚHeterParallelOptimizer)Úcore)Úfleet)ÚloggercCsntj}||_|dur|jrt d¡t |¡|_i|_|  ¡dkr5|jj dkr/t ||j |jƒSt ||jƒS|S)a¼ Optimizer for distributed training. For the distributed training, this method would rebuild a new instance of DistributedOptimizer. Which has basic Optimizer function and special features for distributed training. Args: optimizer(Optimizer): The executor to run for init server. strategy(DistributedStrategy): Extra properties for distributed optimizer. It is recommended to use DistributedStrategy in fleet.init(). The strategy here is for compatibility. If the strategy in fleet.distributed_optimizer() is not None, then it will overwrite the DistributedStrategy in fleet.init(), which will take effect in distributed training. Returns: Fleet: instance of fleet. Examples: .. code-block:: python import paddle import paddle.distributed.fleet as fleet fleet.init(is_collective=True) strategy = fleet.DistributedStrategy() optimizer = paddle.optimizer.SGD(learning_rate=0.001) optimizer = fleet.distributed_optimizer(optimizer, strategy=strategy) Na$It is recommended to use DistributedStrategy in fleet_env.init(). The strategy here is only for compatibility. If the strategy in fleet_env.distributed_optimizer() is not None, then it will overwrite the DistributedStrategy in fleet_env.init(), which will take effect in distributed training.rF)r Zuser_defined_optimizerZ_is_collectiver ÚwarningÚcopyÚdeepcopyZ_user_defined_strategyÚ_contextZ worker_numZheter_ccl_moderZ_hcgr)Z optimizerZstrategyZ fleet_env©rúRD:\Projects\ConvertPro\env\Lib\site-packages\paddle/distributed/fleet/optimizer.pyÚ_dygraph_distributed_optimizers$ÿ   ÿÿrcOs,tjj ¡r t|i|¤ŽStjj|i|¤ŽS©N)ÚpaddleZfluidZ frameworkZ_non_static_moderr Údistributed_optimizer)ÚargsÚkwargsrrrrLs rr)r rÚosÚnumpyÚnpZpaddle.fluid.frameworkrrZbase.distributed_strategyrZmeta_optimizersrrZ paddle.fluidrZpaddle.distributedr Zutils.log_utilr rrrrrrÚs      1