o Ne,@sddlmZddlmZddlZddlmZgdZGdddeZ Gd d d e Z Gd d d e Z dddZ Gddde Z dS))print_function)divisionN)core)SamplerSequenceSampler RandomSamplerWeightedRandomSamplerc@s"eZdZdZdddZddZdS)ra An abstract class to encapsulate methods and behaviors of samplers. All sampler used by :code:`paddle.io.BatchSampler` should be a subclass of :code:`paddle.io.Sampler`, BatchSampler subclasses should implement following methods: :code:`__iter__`: return sample index iterably, which iterate over indices of dataset elements :code:`__len__`: the number of sample in :attr:`data_source` Args: data_source(Dataset, optional): this could be an instance of :code:`paddle.io.Dataset` other Python object which implemented :code:`__len__` for Sampler to get indices as the range of :attr:`dataset` length. Default None. Returns: Sampler: an iterable object for sample indices iterating Examples: .. code-block:: python from paddle.io import Dataset, Sampler class RandomDataset(Dataset): def __init__(self, num_samples): self.num_samples = num_samples def __getitem__(self, idx): image = np.random.random([784]).astype('float32') label = np.random.randint(0, 9, (1, )).astype('int64') return image, label def __len__(self): return self.num_samples class MySampler(Sampler): def __init__(self, data_source): self.data_source = data_source def __iter__(self): return iter(range(len(self.data_source))) def __len__(self): return len(self.data_source) sampler = MySampler(data_source=RandomDataset(100)) for index in sampler: print(index) see `paddle.io.BatchSampler` see `paddle.io.DataLoader` NcC ||_dSN data_sourceselfr rOD:\Projects\ConvertPro\env\Lib\site-packages\paddle/fluid/dataloader/sampler.py__init__W zSampler.__init__cCstr )NotImplementedErrorrrrr__iter__ZszSampler.__iter__r )__name__ __module__ __qualname____doc__rrrrrrrs < rc@s(eZdZdZddZddZddZdS) rau Iterate samples sequentially, yield :code:`0, 1, 2, ..., len(data_source) -1` generally, Args: data_source(Dataset): dataset to sample, this could be an instance of :code:`paddle.io.Dataset` other Python object which implemented :code:`__len__`. Returns: Sampler: a Sampler yield sample index sequentially Examples: .. code-block:: python from paddle.io import Dataset, SequenceSampler class RandomDataset(Dataset): def __init__(self, num_samples): self.num_samples = num_samples def __getitem__(self, idx): image = np.random.random([784]).astype('float32') label = np.random.randint(0, 9, (1, )).astype('int64') return image, label def __len__(self): return self.num_samples sampler = SequenceSampler(data_source=RandomDataset(100)) for index in sampler: print(index) see `paddle.io.Sampler` cCr r r rrrrrrzSequenceSampler.__init__cCsttt|jSr )iterrangelenr rrrrrszSequenceSampler.__iter__cCs t|jSr )rr rrrr__len__rzSequenceSampler.__len__Nrrrrrrrrrrrras & rc@s<eZdZdZ   d ddZeddZdd Zd d ZdS) rah Iterate samples randomly, yield shuffled indices, if :attr:`replacement=False`, yield shuffled indices of the whole data souce, if :attr:`replacement=True`, :attr:`num_samples` can set to specify the sample number to draw. Args: data_source(Dataset): dataset to sample, this could be an instance of :code:`paddle.io.Dataset` other Python object which implemented :code:`__len__`. replacement(bool): If False, sample the whole dataset, If False, set :attr:`num_samples` for how many sample to draw. Default False. num_samples(int): set sample number to draw if :attr:`replacement` is True. Default None. generator(Generator): specify a generator to sample the data source. Default None Returns: Sampler: a Sampler yield sample index randomly Examples: .. code-block:: python from paddle.io import Dataset, RandomSampler class RandomDataset(Dataset): def __init__(self, num_samples): self.num_samples = num_samples def __getitem__(self, idx): image = np.random.random([784]).astype('float32') label = np.random.randint(0, 9, (1, )).astype('int64') return image, label def __len__(self): return self.num_samples sampler = RandomSampler(data_source=RandomDataset(100)) for index in sampler: print(index) see `paddle.io.Sampler` FNcCst||_||_||_||_t|jtstd|j|jdur%|s%tdt|j t r0|j dkr8td|j dS)Nznum_samples should not be specified while replacement is Falserz@num_samples should be a positive integer, but got num_samples={}) r replacement _num_samples generator isinstancebool TypeErrorformat ValueError num_samplesint)rr r r(r"rrrrs" zRandomSampler.__init__cCs|jdur t|jS|jSr )r!rr rrrrr(s  zRandomSampler.num_samplesc cst|j}|jr)t|jD]}zt|j}Wn ty"YdSw|VqdS|jrCtj j t ||jdd D]}|Vq;dStj j t ||dd D]}|VqQdS)NT)replaceF) rr r"rr(next StopIterationr nprandomchoiceZarangetolist)rniindexrrrrs6  zRandomSampler.__iter__cCs|jSr )r(rrrrrszRandomSampler.__len__)FNN) rrrrrpropertyr(rrrrrrrs.   rTcCsJt|tjr |}t|ttfrt|}t|tjs Jdt |j dks+Jd| d|j df}t |dks@Jdt |tjkrLJdt |tjkrXJdtj|dkd d }t |d kslJd |syt ||ksyJd ||jd d }g}t|j d D]}tj|j d ||||}||qt|S)Nz=weights should be paddle.Tensor, numpy.ndarray, list or tuplerz$weights should be a 1-D or 2-D arraygz weights should be positive valuezweights shoule not be INFzweights shoule not be NaN)Zaxisrz#weights should have positive valueszUweights positive value number should not less than num_samples when replacement=False)r#rZ LoDTensornumpylisttupler-arrayZndarrayrshapereshapeallanyinfnansumrr.r/append)weightsr(r Z non_zerosZretsr2retrrr_weighted_samplesH    rEc@s*eZdZdZd ddZddZddZd S) r a Random sample with given weights (probabilities), sampe index will be in range [0, len(weights) - 1], if :attr:`replacement` is True, index can be sampled multiple times. Args: weights(numpy.ndarray|paddle.Tensor|list|tuple): sequence of weights, should be numpy array, paddle.Tensor, list or tuple num_samples(int): set sample number to draw from sampler. replacement(bool): Whether to draw sample with replacements, default True Returns: Sampler: a Sampler yield sample index randomly by given weights Examples: .. code-block:: python from paddle.io import WeightedRandomSampler sampler = WeightedRandomSampler(weights=[0.1, 0.3, 0.5, 0.7, 0.2], num_samples=5, replacement=True) for index in sampler: print(index) TcCsBt|tr |dkr tdt|tstd||_||_||_dS)Nrz(num_samples should be a positive integerz%replacement should be a boolean value)r#r)r'r$rCr(r )rrCr(r rrrr3s  zWeightedRandomSampler.__init__cCs$t|j|j|j}t|dSNr5)rErCr(r rr<r0)rZidxsrrrr<s zWeightedRandomSampler.__iter__cCs$t|jj|jjd}|j|SrF)r-prodrCr;r()rmulrrrrAs zWeightedRandomSampler.__len__NTrrrrrr s   r rI) __future__rrr7r-r__all__objectrrrrEr rrrrs   G1 b"