o Me<"@sdZddlmZddlZddlZddlmZddlZddlm Z ddlm Z ddlm Z dd lm Z dd l mZmZdd lmZddlmmZddlZddlZdd lmZddlZdd lmZddlm Z gZ!dZ"dZ#dZ$dZ%dZ&dZ'dZ(dZ)dZ*ddZ+e,e+dZ-e,e+dZ.   d*ddZ/edd dd!d"e-dddfd#d$Z0edd dd!d"e.dddfd%d&Z1edd dd!d"e.ddfd'dZ2d(d)Z3dS)+a This module will download dataset from http://www.robots.ox.ac.uk/~vgg/data/flowers/102/index.html and parse train/test set intopaddle reader creators. This set contains images of flowers belonging to 102 different categories. The images were acquired by searching the web and taking pictures. There are a minimum of 40 images for each category. The database was used in: Nilsback, M-E. and Zisserman, A. Automated flower classification over a large number of classes.Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing (2008) http://www.robots.ox.ac.uk/~vgg/publications/papers/nilsback08.{pdf,ps.gz}. )print_functionN)download)load_image_bytes) load_image)simple_transform)batch_images_from_tar) map_readers xmap_readers)compat) cpu_count)cPickle) try_importz8http://paddlemodels.bj.bcebos.com/flowers/102flowers.tgzz9http://paddlemodels.bj.bcebos.com/flowers/imagelabels.matz3http://paddlemodels.bj.bcebos.com/flowers/setid.matZ 52808999861908f626f3c1f4e79d11faZ e0620be6f572b9609742df49c70aed4dZ a5357ecc9cb78c4bef273ce3793fc85cZtstidZtrnidvalidcCs8|\}}t|}t|dd|gdd}|d|fS)zB map image bytes data to type needed by model input layer )g\(Y@gR1]@gQ^@)meanZfloat32)rrflattenZastype)Zis_trainsampleimglabelrFD:\Projects\ConvertPro\env\Lib\site-packages\paddle/dataset/flowers.pydefault_mapperFsrTFc s6fdd}|rt||tdt|St||S)a 1. read images from tar file and merge images into batch files in 102flowers.tgz_batch/ 2. get a reader to read sample from batch file :param data_file: downloaded data file :type data_file: string :param label_file: downloaded label file :type label_file: string :param setid_file: downloaded setid file containing information about how to split dataset :type setid_file: string :param dataset_name: data set name (tstid|trnid|valid) :type dataset_name: string :param mapper: a function to map image bytes data to type needed by model input layer :type mapper: callable :param buffered_size: the size of buffer used to process images :type buffered_size: int :param cycle: whether to cycle through the dataset :type cycle: bool :return: data reader :rtype: callable c 3std}|dd}|d}i}|D]}d|}||d||<qt}|}d}|D]} | j|vrS|| } || j} | t| dfVq7dS)Nzscipy.iolabelsrzjpg/image_%05d.jpgr) rZloadmattarfileopen getmembersname extractfilereadint) ZsciorZindexesZ img2labelirtfZmemsZfile_idZmemimager data_file dataset_name label_file setid_filerrreaderys$   zreader_creator..reader)r minr r ) r'r)r*r(mapper buffered_sizeuse_xmapcycler+rr&rreader_creatorXs! r2z2.0.0zpaddle.vision.datasets.Flowersz>Please use new dataset API which supports paddle.io.DataLoader)ZsinceZ update_tolevelreasonc C0tttdtttdtttdtt||||dS)a8 Create flowers training set reader. It returns a reader, each sample in the reader is image pixels in [0, 1] and label in [1, 102] translated from original color image by steps: 1. resize to 256*256 2. random crop to 224*224 3. flatten :param mapper: a function to map sample. :type mapper: callable :param buffered_size: the size of buffer used to process images :type buffered_size: int :param cycle: whether to cycle through the dataset :type cycle: bool :return: train data reader :rtype: callable flowersr1) r2rDATA_URLDATA_MD5 LABEL_URL LABEL_MD5 SETID_URL SETID_MD5 TRAIN_FLAGr.r/r0r1rrrtrain   r@c Cr5)a3 Create flowers test set reader. It returns a reader, each sample in the reader is image pixels in [0, 1] and label in [1, 102] translated from original color image by steps: 1. resize to 256*256 2. random crop to 224*224 3. flatten :param mapper: a function to map sample. :type mapper: callable :param buffered_size: the size of buffer used to process images :type buffered_size: int :param cycle: whether to cycle through the dataset :type cycle: bool :return: test data reader :rtype: callable r6r7) r2rr8r9r:r;r<r= TEST_FLAGr?rrrtestrArCcCs,tttdtttdtttdtt|||S)a Create flowers validation set reader. It returns a reader, each sample in the reader is image pixels in [0, 1] and label in [1, 102] translated from original color image by steps: 1. resize to 256*256 2. random crop to 224*224 3. flatten :param mapper: a function to map sample. :type mapper: callable :param buffered_size: the size of buffer used to process images :type buffered_size: int :return: test data reader :rtype: callable r6) r2rr8r9r:r;r<r= VALID_FLAG)r.r/r0rrrrs   cCs(ttdtttdtttdtdS)Nr6)rr8r9r:r;r<r=rrrrfetchs  rE)rTF)4__doc__ __future__r itertools functoolscommonrrZpaddle.dataset.imagerrrrZ paddle.readerr r Zpaddler ZcptZpaddle.utils.deprecatedutils deprecatedosnumpynpmultiprocessingr sixZ six.movesr pickleZ paddle.utilsr__all__r8r:r<r9r;r=r>rBrDrpartialZ train_mapperZ test_mapperr2r@rCrrErrrrst             ;