o NeAV@sdZddlmZddlZddlZddlZddlZddlm Z ddl m Z ddl Z gdZ Gddde ZGd d d e Zd d Zd dZddZddZddZddZddZe aiaddZddZejddfdd Zejddfd!d"ZdS)#z# Utilities of Auto SParsity (ASP). )print_functionN)Enum) permutations) calculate_density check_mask_1d get_mask_1d check_mask_2dget_mask_2d_greedyget_mask_2d_best create_maskcheck_sparsityMaskAlgo CheckMethodc@seZdZdZdZdZdZdS)r z A collection of all mask generating algorithms. There currently are three algorithms, `MASK_1D`, `MASK_2D_GREEDY` and `MASK_2D_BEST` rr r N)__name__ __module__ __qualname____doc__MASK_1DZMASK_2D_GREEDYZ MASK_2D_BESTrrSD:\Projects\ConvertPro\env\Lib\site-packages\paddle/fluid/contrib/sparsity/utils.pyr $s r c@s$eZdZdZdZdZeddZdS)rzz A collection of all sparsity checking approaches. There currently are two methods, `CHECK_1D` and `CHECK_2D` rrcCs(t|ts Jd|tjkrtjStjS)a Get sparsity checking method by mask generating algorithm. Args: mask_algo (MaskAlgo): The algorithm of mask generating. Returns: CheckMethod: The corresponded sparsity checking method. Examples: .. code-block:: python import numpy as np from paddle.static.sparsity import MaskAlgo from paddle.fluid.contrib.sparsity import CheckMethod CheckMethod.get_checking_method(MaskAlgo.MASK_1D) # CheckMethod.CHECK_1D CheckMethod.get_checking_method(MaskAlgo.MASK_2D_GREEDY) # CheckMethod.CHECK_2D CheckMethod.get_checking_method(MaskAlgo.MASK_2D_BEST) # CheckMethod.CHECK_2D z!mask_algo should be MaskAlgo type) isinstancer rrCHECK_1DCHECK_2D)Z mask_algorrrget_checking_method6s  zCheckMethod.get_checking_methodN)rrrrrr staticmethodrrrrrr.s rcCs"|}tt|dj|jS)a Return the density of the input tensor. Args: x (nparray): The input tensor. Returns: float: The density of :attr:`x`. Examples: .. code-block:: python import paddle import numpy as np x = np.array([[0, 1, 3, 0], [1, 1, 0, 1]]) paddle.incubate.asp.calculate_density(x) # 0.625 r)flattenfloatnpnonzerosize)xZ x_flattenedrrrrWsrcCst|jdks Jd|jd|}|jd|dkrDt|jd|jd||f}||ddd|jdf<|j}|d||fS|d||jfS)a Reshape the input 2D matrix to shape (-1, m). If the second dimension of :attr:`mat` is not a multiples of :attr:`m`, then this function would pad the remainder with 0 before reshaping. .. math:: remainder = mat.shape[1] % m Args: mat (nparray): The input 2D matrix. m (int): The second dimension of reshaped matrix. Returns: tuple: A pair of the reshaped and padded matrix and the shape of padded matrix (non-reshaping). $The input mat should be a 2D matrix!rN)lenshaperzerosreshape)matm remainder mat_paddedr&rrr _reshape_1dls"r-cCsft|jdkrt|d|jd|\}}nt||\}}|D]}t|dj||kr0dSqdS)aX Check if every row of the input matrix :attr:`mat` is in 1D `n:m` sparse pattern. This function would pad the second dimension of :attr:`mat` by zero to be a multiples of :attr:`m` if necessary. 1D `n:m` sparse pattern: At least :attr:`n` zeros in every :math:`1 \times m` block. Args: mat (nparray): The input matrix. n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: bool: True if every row of :attr:`mat` is in 1D n:m sparse pattern, else False. Examples: .. code-block:: python import numpy as np import paddle.fluid.contrib.sparsity as sparsity x = np.array([[0, 1, 3, 0], [1, 0, 0, 1]]) sparsity.check_mask_1d(x, 2, 4) # True x = np.array([[0, 1, 5, 4], [1, 0, 0, 1]]) sparsity.check_mask_1d(x, 2, 4) # False # x would be padded to shape (2, 8) x = np.array([[0, 1, 0, 4, 6], [1, 0, 0, 1, 7]]) sparsity.check_mask_1d(x, 2, 4) # True r#rFT)r%r&r-r(rrr)r)nr* mat_flatternr&sub_matrrrrs!rc Cst||\}}t|}t|}t|jdD]}||}tt|} d||| d|f<q||}|ddd|jdf|ddddf<|S)a Generate 1D `n:m` sparse pattern mask of the input matrix :attr:`mat` in row-directory. This function would pad the second dimension of :attr:`mat` by zero to be a multiples of :attr:`m` before mask generation. 1D `n:m` sparse pattern: At least :attr:`n` zeros in every :math:`1 \times m` block. Args: mat (nparray): The input matrix. n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: nparray: The 1D `n:m` sparse mask of :attr:`mat`. Examples: .. code-block:: python import numpy as np import paddle.fluid.contrib.sparsity as sparsity mat = np.array([[0, 1, 5, 4], [2, 7, 3, 6]]) mask = sparsity.get_mask_1d(mat, 2, 4) # nparray([[0, 0, 1, 1], # [0, 1, 0, 1]]) sparsity.check_mask_1d(mask, 2, 4) # True rNr#) r-r ones_likeranger&argsortabsolutetolistr() r)r.r*r/r& mask_flatternmaskir0Zmin_order_indicesrrrrs   *rc Cs8t|jdks Jd|jd|}|jd|}|dkr"|jdn|jd|||dkr3|jdn|jd||f}t|}||d|jdd|jdf<t|d||}d}td|jd|D].}||} td|jd|D]} | |} t||| | | fd} | ||<|d7}qwqh||jfS)a4 Reshape the input 2D matrix to shape (-1, :math:`m \times m`). In each dimension of :attr:`mat`, if it is not a multiples of :attr:`m`, then this function would pad the remainder with 0 before reshaping. .. math:: remainder_0 = mat.shape[0] % m \\ remainder_1 = mat.shape[1] % m Args: mat (nparray): The input 2D matrix. m (int): The square root of second dimension of reshaped matrix. Returns: tuple: A pair of the reshaped and padded matrix and the shape of padded matrix (non-reshaping). r!r"rr#Nr$)r%r&rr'emptyr(r2squeeze) r)r*Z remainder_0Z remainder_1Z new_shaper,r/curr_idx row_startrow_end col_startcol_endr0rrr _reshape_2ds2     r@c Cs~t||\}}|D]3}tt|||dk}ttj|dd||kdkrasz&get_mask_2d_greedy..r#rg?N) r@rZ zeros_liker(r2r%r4r:r3 collectionsCounterr9r&)r)r.r*r,r&Z mask_paddedidxr0rCZmin_order_1d_indicesZmin_order_2d_indicesZ row_counterZ col_counterr8Z matrix_entryr7r;r<r=r>r?rrFrr 9s<      r cCsd||}|tvrt|St|}d|d|<ttt|}||}tttt||}|j dd|kj dd|k d d}t |j d||f}||dd|dd<t|t|<t|S)a Compute all vaild 2D `n:m` sparse patterns. 2D `n:m` sparse pattern: At least :math:`n \times n` zeros in every :math:`m \times m` block under the constraint of at least :attr:`n` zeros for each row and column. Args: n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: dictionary: A dictionary with key: *m_n* (string) and value: all vaild 2D `n:m` sparse patterns. z{}_{}r#NrArr$)format_valid_2d_patternsrr'listsetrr5ZasarrayrBrr(r9r&_valid_2d_patterns_lockacquirerelease)r.r*Z valid_keypatternsZvalidZvalid_patternsrrr_compute_valid_2d_patternss.   rSc Cst||}t||\}}t|d||}tjt|||jd||jdd}||dd|dd<t |}d} t d|d|D]%} | |} t d|d|D]} | |} || || | | | f<| d7} qTqF|d|jdd|jdfS)aC Generate 2D `n:m` sparse pattern mask of the input matrix :attr:`mat` to form sparse matrix with maximun L1 norm .This function would pad each dimension of :attr:`mat` by zero to be a multiples of :attr:`m` before mask generation. 2D `n:m` sparse pattern: At least :math:`n \times n` zeros in every :math:`m \times m` block under the constraint of at least :attr:`n` zeros for each row and column. *Note*: L1 norm of sparse matrix from `Best` API is greater than or equal to the one from `Greedy`. Args: mat (nparray): The input matrix. n (int): n of `n:m` sparse pattern. m (int): m of `n:m` sparse pattern. Returns: nparray: The 1D `n:m` sparse mask of :attr:`mat`. Examples: .. code-block:: python import numpy as np import paddle.fluid.contrib.sparsity as sparsity mat = np.array([[2, 8, 9, 9], [9, 1, 3, 9], [5, 6, 3, 9], [2, 4, 6, 9]]) mask_greedy = sparsity.get_mask_2d_greedy(mat, 2, 4) mask_best = sparsity.get_mask_2d_best(mat, 2, 4) print("L1 norm of `greedy` sparse matrix", np.multiply(mat, mask_greedy).sum()) # 56 print("L1 norm of `best` sparse matrix", np.multiply(mat, mask_best).sum()) # 61 r$rr#rAN) rSr@rr1r(Zargmaxmatmulr&Tr9r2)r)r.r*rRr/r&r6Zpmaxr7r;r<r=r>r?rrrr s&     r r!c CsZ|j}|j}|t}t|tsJdt|tt j t |j d}t |dkr2|d|d}nlt |dkrC||d|d}n[t |dkrX||d|d|d}nFt |dkr|gd|d|d|d|d}||||d }||d|d|d|dggd|Std t |||||d }|||S) aV Create `n:m` sparse pattern mask of the input tensor via function given by :attr:`func_name`. Currently only support tensor with dimension less than or equal to 4. Args: tensor (nparray): The input tensor. func_name (MaskAlgo, optional): The function name to generate spase mask. Default is `MaskAlgo.MASK_1D`. All options please refer to `MaskAlgo`. n (int, optional): n of `n:m` sparse pattern. Default is 2. m (int, optional): m of `n:m` sparse pattern. Default is 4. Returns: nparray: The `n:m` sparse mask of :attr:`tensor` generated by :attr:`func_name`. Examples: .. code-block:: python import numpy as np import paddle.fluid.contrib.sparsity as sparsity tensor = np.array([[2, 8, 9, 9], [9, 1, 3, 9], [5, 6, 3, 9], [2, 4, 6, 9]]) mask_1d = sparsity.create_mask(tensor, func_name=sparsity.MaskAlgo.MASK_1D) # nparray([[0 0 1 1], # [1 0 0 1], # [0 1 0 1], # [0 0 1 1]]) mask_2d = sparsity.create_mask(tensor, func_name=sparsity.MaskAlgo.MASK_2D_BEST) # nparray([[0 1 1 0], # [1 0 0 1], # [1 1 0 0], # [0 0 1 1]]) zNfunc_name argumet of create_mask is only accepted as type MaskAlgo. But got {}Nr#rr!rVrr#rWr!r.r*iThe dimension of input tensor is not supported in create_mask, Only dimension < 4 is supported but got {})r&dtypeastyperrr rKtypegetattrsysmodulesrvaluer%r( transpose ValueError) tensor func_namer.r*r&r[tfuncr7rrrr s:!       $ r cCs |j}|t}t|tksJdt|ttjt |j d}t |dkr0| d|d}nNt |dkrA| |d|d}n=t |dkrV| |d|d|d}n(t |dkru| gd |d|d|d|dg}n td t |||||d S) a= Check if input tensor is in `n:m` sparse pattern via function given by :attr:`func_name`. Currently only support tensor with dimension less than or equal to 4. Args: tensor (nparray): The input tensor. func_name (CheckMethod, optional): The function name to generate spase mask. Default is `CheckMethod.CHECK_1D`. All options please refer to `CheckMethod`. n (int, optional): n of `n:m` sparse pattern. Default is 2. m (int, optional): m of `n:m` sparse pattern. Default is 4. Returns: bool: True if tensor pass checking of function given by :attr:`func_name`, else False. Examples: .. code-block:: python import numpy as np import paddle.fluid.contrib.sparsity as sparsity tensor = np.array([[2, 8, 9, 9], [9, 1, 3, 9], [5, 6, 3, 9], [2, 4, 6, 9]]) mask_1d = sparsity.create_mask(tensor, func_name=sparsity.MaskAlgo.MASK_1D) # nparray([[0 0 1 1], # [1 0 0 1], # [0 1 0 1], # [0 0 1 1]]) sparsity.check_sparsity(mask_1d, func_name=sparsity.CheckMethod.CHECK_1D) # True sparsity.check_sparsity(mask_1d, func_name=sparsity.CheckMethod.CHECK_2D) # False zTfunc_name argumet of check_sparsity is only accepted as type CheckMethod. But got {}Nr#rr!rWrVrXrZrY)r&r\rr]rrKr^r_r`rrar%r(rbrc)rdrer.r*r&rfrgrrrr s*       " r ) r __future__rr_mathrHnumpyrenumr itertoolsr threading__all__r rrr-rrr@rr LockrOrLrSr rr rr rrrrs2    ),(+2B&5>