o Ne1@s|ddlmZddlZddlZddlZddlmZmZmZddl m Z ddl m Z ddgZ Gd ddeZGd ddeZdS) )print_functionN) InitializerXavierConstant)WeightDecayRegularizer) check_type ParamAttrWeightNormParamAttrc@sVeZdZdZ       dddZddZd d Zd d Zed dZ dddZ dS)r a Note: ``gradient_clip`` of ``ParamAttr`` HAS BEEN DEPRECATED since 2.0. Please use ``need_clip`` in ``ParamAttr`` to speficiy the clip scope. There are three clipping strategies: :ref:`api_paddle_nn_ClipGradByGlobalNorm` , :ref:`api_paddle_nn_ClipGradByNorm` , :ref:`api_paddle_nn_ClipGradByValue` . Create a object to represent the attribute of parameter. The attributes are: name, initializer, learning rate, regularizer, trainable, gradient clip, and model average. Parameters: name (str, optional): The parameter's name. Default None, meaning that the name would be created automatically. initializer (Initializer, optional): The method to initial this parameter. Default None, meaning that the weight parameter is initialized by Xavier initializer, and the bias parameter is initialized by 0. learning_rate (float, optional): The parameter's learning rate. The learning rate when optimize is the global learning rates times the parameter's learning rate times the factor of learning rate scheduler. Default 1.0. regularizer (WeightDecayRegularizer, optional): Regularization strategy. There are two method: :ref:`api_paddle_regularizer_L1Decay` , :ref:`api_paddle_regularizer_L2Decay` . If regularizer is also set in ``optimizer`` (such as :ref:`api_paddle_optimizer_SGD` ), that regularizer setting in optimizer will be ignored. Default None, meaning there is no regularization. trainable (bool, optional): Whether this parameter is trainable. Default True. do_model_average (bool, optional): Whether this parameter should do model average when model average is enabled. Only used in ExponentialMovingAverage. Default True. need_clip (bool, optional): Whether the parameter gradient need to be cliped in optimizer. Default is True. Returns: ParamAttr Object. Examples: .. code-block:: python import paddle weight_attr = paddle.ParamAttr(name="weight", learning_rate=0.5, regularizer=paddle.regularizer.L2Decay(1.0), trainable=True) print(weight_attr.name) # "weight" paddle.nn.Linear(3, 4, weight_attr=weight_attr) N?TcCstjjdkrt|dttdtfdn t|dttdfdt|dttfdt|dt dt|dt dt|dt dt|dt tdfdt|d t tdfd||_ |j d kr^t d ||_||_||_||_||_||_dS) Nnamer learning_rate trainabledo_model_average need_clip initializer regularizerz&name of ParamAttr can not be empty str)sys version_infomajorrstrtypeunicodefloatintboolrrr ValueErrorrrrrrr)selfr rrrrrrr GD:\Projects\ConvertPro\env\Lib\site-packages\paddle/fluid/param_attr.py__init__Ps,    zParamAttr.__init__cCs6|dur|jdur tddS|jdurdS||_dS)z Set the default initializer, the initializer should be Constant, Uniform, Normal, Xavier, MSRA. Args: initializer(Initializer): the initializer to set. Returns: None Nz ParamAttr.initializer is not set)rr)rrr r r!_set_default_initializerqs   z"ParamAttr._set_default_initializercCs|tdS)z Set the default initializer for the parameter with Xavier. Args: None. Returns: None. N)r#rrr r r!_set_default_param_initializers z(ParamAttr._set_default_param_initializercCs|tddS)z Set the default initializer for the bias with Constant(0.0). Args: None. Returns: None. gN)r#rr$r r r!_set_default_bias_initializers z'ParamAttr._set_default_bias_initializercCs|durtSt|tst|trdd|DSt|tr|St|tjr*t|dSt|tr4t|dSt|tr>t|dSt|trL|rJt dSdSt d t |) a Create ParamAttr[s]. Args: arg: Arguments to initialize ParamAttr[s]. arg's type can be str, Initializer, float, WeightDecayRegularizer, BaseGradientClipAttr, bool, ParamAttr, or a list of above type. Returns: ParamAttr[s]: ParamAttr[s] initialized with arg. Raises: arg can not initialize a ParamAttr. NcSsg|]}t|qSr )r _to_attr).0ar r r! sz&ParamAttr._to_attr..)r )r)rFz{0} cast to ParamAttr) r isinstancelisttuplesix string_typesrrrr' TypeErrorformatr)argr r r!r's        zParamAttr._to_attrFcCs4|jd|ji|j|j|j|jd}|r|j|d<|S)z Returns the attributes of this parameter. Args: with_initializer(bool): Whether to add initializer attr. Returns: Parameter attributes(map): The attributes of this parameter. r)r Z optimize_attrrrrrr)r rrrrrr)rZwith_initializerkwargsr r r! _to_kwargss  zParamAttr._to_kwargs)NNr NTTT)F) __name__ __module__ __qualname____doc__r"r#r%r& staticmethodr'r4r r r r!r s 1 !   cs6eZdZdZgZ        dfdd ZZS) r a Note: Please use 'paddle.nn.utils.weight_norm' in dygraph mode. Note: ``gradient_clip`` of ``ParamAttr`` HAS BEEN DEPRECATED since 2.0. Please use ``need_clip`` in ``ParamAttr`` to speficiy the clip scope. There are three clipping strategies: :ref:`api_paddle_nn_ClipGradByGlobalNorm` , :ref:`api_paddle_nn_ClipGradByNorm` , :ref:`api_paddle_nn_ClipGradByValue` . Parameter of weight Norm. Weight Norm is a reparameterization of the weight vectors in a neural network that decouples the magnitude of those weight vectors from their direction. Weight Norm has been implemented as discussed in this paper: `Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks `_. Args: dim(int, optional): Dimension over which to compute the norm. Dim is a non-negative number which is less than the rank of weight Tensor. For Example, dim can be chosen from 0, 1, 2, 3 for convolution whose weight shape is [cout, cin, kh, kw] and rank is 4. Default None, meaning that all elements will be normalized. name(str, optional): The parameter's name. Default None, meaning that the name would be created automatically. Please refer to :ref:`api_guide_Name` for more details. initializer(Initializer, optional): The method to initialize this parameter, such as ``initializer = paddle.nn.initializer.Constant(1.0)``. Default None, meaning that the weight parameter is initialized by Xavier initializer, and the bias parameter is initialized by 0. learning_rate(float32, optional): The parameter's learning rate when optimizer is :math:`global\_lr * parameter\_lr * scheduler\_factor`. Default 1.0. regularizer (WeightDecayRegularizer, optional): Regularization strategy. There are two method: :ref:`api_paddle_regularizer_L1Decay` , :ref:`api_paddle_regularizer_L2Decay`. If regularizer isralso set in ``optimizer`` (such as :ref:`api_paddle_optimizer_SGD` ), that regularizer setting in optimizer will be ignored. Default None, meaning there is no regularization. trainable(bool, optional): Whether this parameter is trainable. Default True. do_model_average(bool, optional): Whether this parameter should do model average. Default False. need_clip (bool, optional): Whether the parameter gradient need to be cliped in optimizer. Default is True. Examples: .. code-block:: python import paddle paddle.enable_static() data = paddle.static.data(name="data", shape=[3, 32, 32], dtype="float32") fc = paddle.static.nn.fc(x=data, size=1000, weight_attr=paddle.static.WeightNormParamAttr( dim=None, name='weight_norm_param', initializer=paddle.nn.initializer.Constant(1.0), learning_rate=1.0, regularizer=paddle.regularizer.L2Decay(0.1), trainable=True, do_model_average=False, need_clip=True)) Nr TFc s(tt|j|||||||d||_dS)N)r rrrrrr)superr r"dim) rr;r rrrrrr __class__r r!r"!s  zWeightNormParamAttr.__init__)NNNr NTFT)r5r6r7r8Zparams_with_weight_normr" __classcell__r r r<r!r sF) __future__rr.warningsrrrrrrrZpaddle.fluid.data_feederr__all__objectr r r r r r!s   :