o MeD%@svddlmZddlmZddlmZmZmZddlm Z ddl m Z m Z ddZ dd Zdd d Zd dZddZdS))core) LayerHelper)_non_static_mode_in_legacy_dygraphin_dygraph_mode)check_variable_and_dtype)_C_ops _legacy_C_opscCsptr t|d|Strtj|d|Sd}t|fit}|j|j d}|j |d|id|id|id|S)a calculate the expert count according to the gate index. Args: numbers (Tensor): Tensor. The input gate index whose data type should be int32 or int64. upper_range (int): The number of the experts. Returns: out (Tensor): The output expert count. Examples: .. code-block:: python # required: distributed import paddle numbers = [ [0, 2], [0, 2] ] upper_range = 6 numbers = paddle.to_tensor(numbers, dtype="int32") number_count = paddle.distributed.utils.number_count(numbers, upper_range) print(number_count) # the result: [2, 0, 2, 0, 0, 0] upper_range number_countdtypenumbersOuttypeinputsoutputsattrs) rr r rropsrlocals"create_variable_for_type_inferencer append_op)rr op_typehelperoutrSD:\Projects\ConvertPro\env\Lib\site-packages\paddle/distributed/models/moe/utils.py _number_countsrcCstr t|||dStrtj|||dSd}t|fit}|j|j d}|j ||g|g|dgdd|gid|S)a Assign pos decides which tokens should be fetched belong to specially expert orderingly. Args: x (Tensor): Tensor. Every element in the list must be a Tensor whose data type should be float16, float32, float64, int32 or int64. cum_count (Tensor): The cumulative sum tokens of counters. Every element in the list must be a Tensor whose data type should be int64. Returns: out (Tensor): Assemble numbers in the order of counters. Examples: .. code-block:: python # required: distributed import paddle number_count = [2, 0, 2, 0] numbers = [ [0, 2], [0, 2] ] number_count = paddle.to_tensor(number_count) numbers = paddle.to_tensor(numbers, dtype="int32") num_cum = paddle.cumsum(number_count) pos = paddle.distributed.utils.assign_pos(x=numbers, cum_count=num_cum) print(pos) # the result: (2, 0, 3, 1) assign_posr )X cum_countZ eff_num_lenr)rrr) rr r rrrrrrr r)xr"rrrrrr _assign_pos=sr$cCsB|dkrtrt|||Strtj|||Stdtd)a random routing topk gate idx ``` out = topk_idx for i in len(topk_idx): if topk * value[i][topk-1] < prob[i]: out[i][topk-1] = -1 ``` Args: topk_idx: gate idx, shape=(N, topk) topk_value: values, shape = topk_idx.shape prob: random prob, shape=(topk_idx.shape[0],) r%zNot supporting static mode nowzonly topk=2 is supported now)rr Zrandom_routingrrr RuntimeError)Ztopk_idxZ topk_valueZprobZtopkrrr_random_routingosr'cCsvtr t||d|Strtj||d|Sd}t|fit}|j|j d}|j |||dd|id|id|S)a limit the expert count by capacity. Args: expert_count (Tensor): Tensor. The input expert count whose data type should be int32 or int64. capacity (Tensor): Tensor. The input capacity whose data type should be int32 or int64 and the elements of capacity should be the same with expert_count.numel()/n_work. n_work (int): The number of the works. Returns: out (Tensor): The output expert count limit by capacity. Examples: .. code-block:: python # required: distributed import paddle expert_count = [1, 2, 2, 8, 3, 6] capacity = [5, 5, 5] n_work = 2 expert_count = paddle.to_tensor(expert_count, dtype="int32") capacity = paddle.to_tensor(capacity, dtype="int32") out = paddle.distributed.utils.limit_by_capacity(expert_count, capacity, n_work) print(out) # the result: [1, 2, 2, 4, 3, 3] n_workerlimit_by_capacityr ) expert_countcapacityrr) rr r)rrrrrrr r)r*r+r(rrrrrr_limit_by_capacitys* r,cCstr t||d|d|Strtj||d|d|St|dddgdt|dddgdtdit}|j |j d }|j d||d d |i||d d |S)u) prune gate by capacity(only support CUDA) Args: gate_idx (Tensor): Represents the gate_id sequence corresponding to the input data with type int32, int64. expert_count (Tensor): The quantity value counted on the gate_id sequence of the input data with type int32, int64. n_worker(int，optional): The number of workers on the trainer with type int64. Returns: new_gate_idx (Tensor): The gate_id sequence corresponding to the new input data after passing through prune. Examples: .. code-block:: python import paddle gate_idx = paddle.to_tensor([1, 3, 3, 3, 3, 2, 1, 1], dtype='int32') expert_count = paddle.to_tensor([0, 3, 1, 3, 0, 0, 0, 0], dtype='int32') n_worker = 1 new_gate_id = paddle.distributed.utils.prune_gate_by_capacity(gate_idx, expert_count, n_expert, n_worker) print(new_gate_id) # Tensor(shape=[8], dtype=int32, place=CUDAPlace(0), stop_gradient=True, [1, 3, 3, 3, -1, 2, 1, 1]) n_expertr(GateIdxZint32Zint64z/paddle.distributed.utils.prune_gate_by_capacity ExpertCountprune_gate_by_capacityr )r.r/Z NewGateIdx)r-r(rN)r0) rr r0rrrrrrrr r)Zgate_idxr*r-r(rZ new_gate_idxrrr_prune_gate_by_capacitys<    r1N)r%)Z paddle.fluidrZpaddle.fluid.layer_helperrZpaddle.fluid.frameworkrrrZpaddle.fluid.data_feederrZpaddlerr rr$r'r,r1rrrrs   ' 2 ,