o NeF[@sddlmZddlmZddlmZddlmZddlmZddlZddlZ ddl Z dd l m Z m Z mZmZgd ZGd d d eZGd ddeZGdddeZGdddeZGdddeZdS))print_function) control_flow)tensor)ops)nnN) convert_dtypecheck_variable_and_dtype check_type check_dtype)UniformNormal CategoricalMultivariateNormalDiagc@s@eZdZdZddZddZddZdd Zd d Zd d Z dS) DistributionzP Distribution is the abstract base class for probability distributions. cCt)zSampling from the distribution.NotImplementedErrorselfrQD:\Projects\ConvertPro\env\Lib\site-packages\paddle/fluid/layers/distributions.pysample#zDistribution.samplecCr)z The entropy of the distribution.rrrrrentropy'rzDistribution.entropycCr)z7The KL-divergence between self distributions and other.r)rotherrrr kl_divergence+rzDistribution.kl_divergencecCr)z&Log probability density/mass function.r)rvaluerrrlog_prob/rzDistribution.log_probcGs<d}d}|D] }t|tjrd}qd}q|r|rtd|S)z Argument validation for distribution args Args: value (float, list, numpy.ndarray, Variable) Raises ValueError: if one argument is Variable, all arguments should be Variable FTz=if one argument is Variable, all arguments should be Variable) isinstancerVariable ValueError)rargsZ is_variableZ is_numberargrrr_validate_args3s zDistribution._validate_argscGsg}g}d}|D]K}d}tttjtjfD] }t||rd}nq|s&Jdt|tr2td|}t|}|j } t | dvrJt d| d}||}||q|j } |D]}t||\} } tj| d } t| | || qYt|S) z Argument convert args to Variable Args: value (float, list, numpy.ndarray, Variable) Returns: Variable of args. FTzBtype of input args must be float, list, numpy.ndarray or Variable.r)float32zUdata type of argument only support float32, your argument will be convert to float32.r'dtype)floatlistnpndarrayrr!r zerosarrayr)strwarningswarnZastypeappendZbroadcast_arraysZ create_tensorZassigntuple)rr#Z numpy_argsZ variable_argstmpr$Z valid_argclsZarg_npZ arg_dtyper)Zarg_broadcasted_Z arg_variablerrr _to_variableIs:           zDistribution._to_variableN) __name__ __module__ __qualname____doc__rrrrr%r8rrrrrs rc@s2eZdZdZddZd ddZddZd d Zd S) r aUniform distribution with `low` and `high` parameters. Mathematical Details The probability density function (pdf) is, .. math:: pdf(x; a, b) = \\frac{1}{Z}, \ a <=x t |tr>d|_| ||\|_ |_ dS)Nlowr highFT)r r*r,r-rr!r+all_arg_is_floatbatch_size_unknownr%r=r>r r8)rr=r>rrr__init__  zUniform.__init__rcCst|dtdt|dtdt|j|jj}|jrL||}t|j|j|||jj d}t j ||jdd|d}|||j|j|j}t ||S||}t j ||dtj||jj d|j|j|j}|jrrt ||S|S) Generate samples of the specified shape. Args: shape (list): 1D `int32`. Shape of the generated samples. seed (int): Python integer number. Returns: Variable: A tensor with prepended dimensions shape.The data type is float32. shaperseedr&?)minmaxrE)rEr()r r+intr=r>rDr@rfill_constant_batch_size_liker)rZuniform_random_batch_size_likereshapeZuniform_randomr.r?)rrDrE batch_shape output_shapezero_tmpZuniform_random_tmpoutputrrrrs8       zUniform.samplecCsnt|dddgdt|j|}t||j}tj||jd}tj||jd}t ||t |j|jS)Log probability density/mass function. Args: value (Variable): The input tensor. Returns: Variable: log probability.The data type is same with value. rr'float64rr() r r less_thanr=r>rcastr)rlog)rrZlb_boolZub_boolZlbZubrrrrs  zUniform.log_probcCst|j|jS)zShannon entropy in nats. Returns: Variable: Shannon entropy of uniform distribution.The data type is float32. )rrTr>r=rrrrrszUniform.entropyNr)r9r:r;r<rArrrrrrrr ss ? " r c@s:eZdZdZddZdddZddZd d Zd d Zd S)raThe Normal distribution with location `loc` and `scale` parameters. Mathematical details The probability density function (pdf) is, .. math:: pdf(x; \mu, \sigma) = \\frac{1}{Z}e^{\\frac {-0.5 (x - \mu)^2} {\sigma^2} } .. math:: Z = (2 \pi \sigma^2)^{0.5} In the above equation: * :math:`loc = \mu`: is the mean. * :math:`scale = \sigma`: is the std. * :math:`Z`: is the normalization constant. Args: loc(float|list|numpy.ndarray|Variable): The mean of normal distribution.The data type is float32. scale(float|list|numpy.ndarray|Variable): The std of normal distribution.The data type is float32. Examples: .. code-block:: python import numpy as np from paddle.fluid import layers from paddle.fluid.layers import Normal # Define a single scalar Normal distribution. dist = Normal(loc=0., scale=3.) # Define a batch of two scalar valued Normals. # The first has mean 1 and standard deviation 11, the second 2 and 22. dist = Normal(loc=[1., 2.], scale=[11., 22.]) # Get 3 samples, returning a 3 x 2 tensor. dist.sample([3]) # Define a batch of two scalar valued Normals. # Both have mean 1, but different standard deviations. dist = Normal(loc=1., scale=[11., 22.]) # Complete example value_npdata = np.array([0.8], dtype="float32") value_tensor = layers.create_tensor(dtype="float32") layers.assign(value_npdata, value_tensor) normal_a = Normal([0.], [1.]) normal_b = Normal([0.5], [2.]) sample = normal_a.sample([2]) # a random tensor created by normal distribution with shape: [2, 1] entropy = normal_a.entropy() # [1.4189385] with shape: [1] lp = normal_a.log_prob(value_tensor) # [-1.2389386] with shape: [1] kl = normal_a.kl_divergence(normal_b) # [0.34939718] with shape: [1] cCst|dttjtjtfdt|dttjtjtfdd|_d|_| ||r1d|_||_ ||_ dSt |tr>t |tr>d|_| ||\|_ |_ dS)NlocrscaleFT)r r*r,r-rr!r+r@r?r%rVrWr r8rrVrWrrrrAArBzNormal.__init__rc Cst|dtdt|dtdt|j|jj}|jrL||}t|j|j|||jj d}t |}t j |dd|d}|||j|j}t ||S||}t j |dd|dtj ||jj d|j|j}|jrqt ||S|S)rCrDrrEr&rF)meanZstdrEr()r r+rIrVrWrDr@rrJr)rZgaussian_randomrKr.r?) rrDrErLrMrNZzero_tmp_shapeZnormal_random_tmprOrrrrRs2    z Normal.samplecCsVt|j|jj}t|j|j||jjd}ddtdtj t |j|S)zShannon entropy in nats. Returns: Variable: Shannon entropy of normal distribution.The data type is float32. r&?r) r+rVrWrDrrJr)mathrTpir)rrLrNrrrrwszNormal.entropycCsdt|dddgd|j|j}t|j}d||j||jd||ttdtjS)rPrr'rQrg@)r rWrrTrVr[sqrtr\)rrvarZ log_scalerrrrs    zNormal.log_probcCsVt|dtd|j|j}||}|j|j|j}||}d||dt|S)zThe KL-divergence between two normal distributions. Args: other (Normal): instance of Normal. Returns: Variable: kl-divergence between two normal distributions.The data type is float32. rrrZrF)r rrWrVrrT)rrZ var_ratiot1rrrrs  zNormal.kl_divergenceNrU) r9r:r;r<rArrrrrrrrrs= % rc@s(eZdZdZddZddZddZdS) ra Categorical distribution is a discrete probability distribution that describes the possible results of a random variable that can take on one of K possible categories, with the probability of each category separately specified. The probability mass function (pmf) is: .. math:: pmf(k; p_i) = \prod_{i=1}^{k} p_i^{[x=i]} In the above equation: * :math:`[x=i]` : it evaluates to 1 if :math:`x==i` , 0 otherwise. Args: logits(list|numpy.ndarray|Variable): The logits input of categorical distribution. The data type is float32. Examples: .. code-block:: python import numpy as np from paddle.fluid import layers from paddle.fluid.layers import Categorical a_logits_npdata = np.array([-0.602,-0.602], dtype="float32") a_logits_tensor = layers.create_tensor(dtype="float32") layers.assign(a_logits_npdata, a_logits_tensor) b_logits_npdata = np.array([-0.102,-0.112], dtype="float32") b_logits_tensor = layers.create_tensor(dtype="float32") layers.assign(b_logits_npdata, b_logits_tensor) a = Categorical(a_logits_tensor) b = Categorical(b_logits_tensor) a.entropy() # [0.6931472] with shape: [1] b.entropy() # [0.6931347] with shape: [1] a.kl_divergence(b) # [1.2516975e-05] with shape: [1] cCs@t|dtjtjtfd||r||_dS||d|_dS)z Args: logits(list|numpy.ndarray|Variable): The logits input of categorical distribution. The data type is float32. logitsrrN) r r,r-rr!r+r%rar8)rrarrrrAs   zCategorical.__init__c Cst|dtd|jtj|jddd}|jtj|jddd}t|}t|}tj|ddd}tj|ddd}||}tj||t||t|ddd} | S)aThe KL-divergence between two Categorical distributions. Args: other (Categorical): instance of Categorical. The data type is float32. Returns: Variable: kl-divergence between two Categorical distributions. rrTdimZkeep_dim) r rrar reduce_maxrexp reduce_sumrT) rrraZ other_logitse_logitsZother_e_logitszZother_zprobklrrrrs    zCategorical.kl_divergencecCs`|jtj|jddd}t|}tj|ddd}||}dtj||t|ddd}|S)zShannon entropy in nats. Returns: Variable: Shannon entropy of Categorical distribution. The data type is float32. rbTrcr])rarrerrfrgrT)rrarhrirjrrrrrs zCategorical.entropyN)r9r:r;r<rArrrrrrrs 0 rc@s8eZdZdZddZddZddZdd Zd d Zd S) ra A multivariate normal (also called Gaussian) distribution parameterized by a mean vector and a covariance matrix. The probability density function (pdf) is: .. math:: pdf(x; loc, scale) = \\frac{e^{-\\frac{||y||^2}{2}}}{Z} where: .. math:: y = inv(scale) @ (x - loc) Z = (2\\pi)^{0.5k} |det(scale)| In the above equation: * :math:`inv` : denotes to take the inverse of the matrix. * :math:`@` : denotes matrix multiplication. * :math:`det` : denotes to evaluate the determinant. Args: loc(list|numpy.ndarray|Variable): The mean of multivariateNormal distribution with shape :math:`[k]` . The data type is float32. scale(list|numpy.ndarray|Variable): The positive definite diagonal covariance matrix of multivariateNormal distribution with shape :math:`[k, k]` . All elements are 0 except diagonal elements. The data type is float32. Examples: .. code-block:: python import numpy as np from paddle.fluid import layers from paddle.fluid.layers import MultivariateNormalDiag a_loc_npdata = np.array([0.3,0.5],dtype="float32") a_loc_tensor = layers.create_tensor(dtype="float32") layers.assign(a_loc_npdata, a_loc_tensor) a_scale_npdata = np.array([[0.4,0],[0,0.5]],dtype="float32") a_scale_tensor = layers.create_tensor(dtype="float32") layers.assign(a_scale_npdata, a_scale_tensor) b_loc_npdata = np.array([0.2,0.4],dtype="float32") b_loc_tensor = layers.create_tensor(dtype="float32") layers.assign(b_loc_npdata, b_loc_tensor) b_scale_npdata = np.array([[0.3,0],[0,0.4]],dtype="float32") b_scale_tensor = layers.create_tensor(dtype="float32") layers.assign(b_scale_npdata, b_scale_tensor) a = MultivariateNormalDiag(a_loc_tensor, a_scale_tensor) b = MultivariateNormalDiag(b_loc_tensor, b_scale_tensor) a.entropy() # [2.033158] with shape: [1] b.entropy() # [1.7777451] with shape: [1] a.kl_divergence(b) # [0.06542051] with shape: [1] cCsdt|dtjtjtfdt|dtjtjtfd|||r&||_||_dS| ||\|_|_dS)NrVrrW) r r,r-rr!r+r%rVrWr8rXrrrrAZs  zMultivariateNormalDiag.__init__cCsPt|j}tj||jjd}ttj|dg|jjd}t|||}|S)NrDr)r) r+rDronesrVr)diagrZ reduce_prod)rrrLone_allone_diagZdet_diagrrr_detfs zMultivariateNormalDiag._detcCsRt|j}tj||jjd}ttj|dg|jjd}t||d|}|S)Nrlrr) r+rDrrmrVr)rnrZelementwise_pow)rrrLrorpZinv_diagrrr_invps zMultivariateNormalDiag._invcCs:d|jjddtdtjt||j}|S)zShannon entropy in nats. Returns: Variable: Shannon entropy of Multivariate Normal distribution. The data type is float32. rZrrFr)rWrDr[rTr\rrq)rrrrrrzs zMultivariateNormalDiag.entropycCst|dtdt||j|j}t|j|j||j}t||j|j}t|jj d}t | |jt | |j}d||||}|S)a%The KL-divergence between two Multivariate Normal distributions. Args: other (MultivariateNormalDiag): instance of Multivariate Normal. Returns: Variable: kl-divergence between two Multivariate Normal distributions. The data type is float32. rrrrZ) r rrrgrrrWmatmulrVr+rDrTrq)rrZ tr_cov_matmulZloc_matmul_covZ tri_matmulkZln_covrkrrrrs  $z$MultivariateNormalDiag.kl_divergenceN) r9r:r;r<rArqrrrrrrrrrsC   r) __future__rrrrrr[numpyr,r1Z data_feederr r r r __all__objectrr rrrrrrrs"     U*j