o MePI@sddlmZddlZddlZddlmZddlmZm Z m Z ddlm Z ddl m Z ddlmZdd lmZgd Zd d ZGd ddeZGdddeZGdddeZGdddeZdS))print_functionN)layers)ProgramVariable program_guard) unique_name) LayerHelper)Constant) detection)ChunkEvaluator EditDistance DetectionMAPcCs.t|tsJ|j|j|j|j|j|jddS)NT)nameshapedtypetype lod_level persistable) isinstancerZ create_varrrrrr)blockvarrFD:\Projects\ConvertPro\env\Lib\site-packages\paddle/fluid/evaluator.py _clone_var_"src@s4eZdZdZddZd ddZd ddZd d ZdS) Evaluatora Warning: better to use the fluid.metrics.* things, more flexible support via pure Python and Operator, and decoupled with executor. Short doc are intended to urge new user start from Metrics. Base Class for all evaluators. Args: name(str): The name of evaluator. such as, "accuracy". Used for generate temporary variable name. main_program(Program, optional): The evaluator should be added to this main_program. Default default_main_program() startup_program(Program, optional):The parameter should be added to this startup_program. Default default_startup_program() Attributes: states(list): The list of state variables. states will be reset to zero when `reset` is invoked. metrics(list): The list of metrics variables. They will be calculate every mini-batch cKs>td|jj|jjftg|_g|_t|fi||_dS)NzThe %s is deprecated, because maintain a modified program inside evaluator cause bug easily, please use fluid.metrics.%s instead.) warningswarn __class____name__Warningstatesmetricsr helper)selfrkwargsrrr__init__DszEvaluator.__init__NcCs|durt}t|d'|jD]}t|tsJt||}tj|j d|j |dqWdn1s6wY| |dS)a reset metric states at the begin of each pass/user specified batch Args: executor(Executor|ParallelExecutor): a executor for executing the reset_program reset_program(Program): a single Program for reset process N main_programrvaluerout) rrr!rrr current_blockr fill_constantrrrun)r$executor reset_programrZg_varrrrresetLs   zEvaluator.resetcCst)z Evaluate the statistics merged by multiple mini-batches. Args: executor(Executor|ParallelExecutor): a executor for executing the eval_program eval_program(Program): a single Program for eval process )NotImplementedError)r$r0 eval_programrrrevalbszEvaluator.evalcCs8|jjdt|jj|gd||d}|j||S)z Create state variable. Args: suffix(str): the state suffix. dtype(str|core.VarDesc.VarType): the state data type shape(tuple|list): the shape of state Returns: State variable _T)rrrr)r#create_variablejoinrgeneraterr!append)r$suffixrrstaterrr _create_stateks  zEvaluator._create_stateN)r __module__ __qualname____doc__r&r2r5r=rrrrr,s    rcs.eZdZdZ dfdd ZdddZZS)r a Warning: This would be deprecated in the future. Please use fluid.metrics.ChunkEvaluator instead. Accumulate counter numbers output by chunk_eval from mini-batches and compute the precision recall and F1-score using the accumulated counter numbers. For some basics of chunking, please refer to 'Chunking with Support Vector Machines '. Args: input (Variable): prediction output of the network. label (Variable): label of the test data set. chunk_scheme (str): can be IOB/IOE/IOBES and IO. See the chunk_eval op for details. num_chunk_types (int): the number of chunk type. excluded_chunk_types (list): A list including chunk type ids, indicating chunk types that are not counted. Returns: tuple: tuple containing: precision, recall, f1_score Examples: .. code-block:: python exe = fluid.executor(place) evaluator = fluid.Evaluator.ChunkEvaluator(input, label) for epoch in PASS_NUM: evaluator.reset(exe) for data in batches: loss = exe.run(fetch_list=[cost]) distance, instance_error = distance_evaluator.eval(exe) Nc stt|d|jj}|jdkrtd|jddgdd|_ |jddgdd|_ |jddgd d|_ t j |||||d \}}} } } } t j|j | g|j d t j|j | g|j d t j|j | g|j d |j||| gdS) N chunk_evalr+You can only invoke Evaluator in root blockint64rnum_infer_chunksrrr;num_label_chunksnum_correct_chunks)inputlabel chunk_schemenum_chunk_typesexcluded_chunk_typesrIr,)superr r&r#r(r-idx ValueErrorr=rErGrHrrBsumsr"extend) r$rIrJrKrLrMr( precisionrecallf1_scorerErGrHrrrr&s@   zChunkEvaluator.__init__c s|durt}||j|fdd|jDd\}}}|d}|d}|d}|r1t||nd}|r;t||nd}|rKtd||||nd}tj|gddtj|gddtj|gddfS)Ncsg|]}t|qSr)r).0r<rrr sz'ChunkEvaluator.eval..Z fetch_listrfloat32)r)rr-r/r!floatnparray) r$r0r4rErGrHrTrUrVrrYrr5sJ zChunkEvaluator.evalr>rr?r@rAr&r5 __classcell__rrrWrr s &%r cs,eZdZdZdfdd ZdddZZS)r a@ Warning: This would be deprecated in the future. Please use fluid.metrics.EditDistance instead. Accumulate edit distance sum and sequence number from mini-batches and compute the average edit_distance and instance error of all batches. Args: input: the sequences predicted by network. label: the target sequences which must have same sequence count with input. ignored_tokens(list of int): Tokens that should be removed before calculating edit distance. Examples: .. code-block:: python exe = fluid.executor(place) distance_evaluator = fluid.Evaluator.EditDistance(input, label) for epoch in PASS_NUM: distance_evaluator.reset(exe) for data in batches: loss = exe.run(fetch_list=[cost]) distance, instance_error = distance_evaluator.eval(exe) In the above example: 'distance' is the average of the edit distance in a pass. 'instance_error' is the instance error rate in a pass. Nc s0tt|jdi||jj}|jdkrtd|jddgdd|_ |jddgd d|_ |jddgd d|_ t j |||d \}}t jdgd dd }t ||} t j| dd} t | } t j|| d} t |} t j|j | g|j dt j|j |g|j dt j|j | g|j d|j| |j| dS)N edit_distancerrCr]rtotal_distancerFrDseq_numinstance_error)rIrJignored_tokensr))rr+rxrriyrN)rc)rOr r&r#r(r-rPrQr=rdrerfrrcr.equalcastZ reduce_sumZelementwise_subrRr"r:)r$rIrJrgr%r(Z distancesrezeroZcompare_resultZcompare_result_intZseq_right_countZinstance_error_countrdrWrrr&sJ       zEditDistance.__init__c Cs|durt}|}t|d?t||j}t||j}t||j}tj|dd}tj|dd}tj ||d}tj ||d}|j |||gd} Wdn1sRwYt | dt | dfS)Nr'r]rhrjr[rr) rr-rrrdrerfrrmZelementwise_divr/r_r`) r$r0r4rrdrerfZ avg_distanceZavg_instance_errorresultrrrr5s$     zEditDistance.evalr>rarrrWrr s"r cs@eZdZdZ      d fdd Zd d Zdd d ZZS)ra Warning: This would be deprecated in the future. Please use fluid.metrics.DetectionMAP instead. Calculate the detection mean average precision (mAP). The general steps are as follows: 1. calculate the true positive and false positive according to the input of detection and labels. 2. calculate mAP value, support two versions: '11 point' and 'integral'. Please get more information from the following articles: https://sanchom.wordpress.com/tag/average-precision/ https://arxiv.org/abs/1512.02325 Args: input (Variable): The detection results, which is a LoDTensor with shape [M, 6]. The layout is [label, confidence, xmin, ymin, xmax, ymax]. gt_label (Variable): The ground truth label index, which is a LoDTensor with shape [N, 1]. gt_box (Variable): The ground truth bounding box (bbox), which is a LoDTensor with shape [N, 4]. The layout is [xmin, ymin, xmax, ymax]. gt_difficult (Variable|None): Whether this ground truth is a difficult bounding bbox, which can be a LoDTensor [N, 1] or not set. If None, it means all the ground truth labels are not difficult bbox. class_num (int): The class number. background_label (int): The index of background label, the background label will be ignored. If set to -1, then all categories will be considered, 0 by default. overlap_threshold (float): The threshold for deciding true/false positive, 0.5 by default. evaluate_difficult (bool): Whether to consider difficult ground truth for evaluation, True by default. This argument does not work when gt_difficult is None. ap_version (string): The average precision calculation ways, it must be 'integral' or '11point'. Please check https://sanchom.wordpress.com/tag/average-precision/ for details. - 11point: the 11-point interpolated average precision. - integral: the natural integral of the precision-recall curve. Examples: .. code-block:: python exe = fluid.executor(place) map_evaluator = fluid.Evaluator.DetectionMAP(input, gt_label, gt_box, gt_difficult) cur_map, accum_map = map_evaluator.get_map_var() fetch = [cost, cur_map, accum_map] for epoch in PASS_NUM: map_evaluator.reset(exe) for data in batches: loss, cur_map_v, accum_map_v = exe.run(fetch_list=fetch) In the above example: 'cur_map_v' is the mAP of current mini-batch. 'accum_map_v' is the accumulative mAP of one pass. Nr?Tintegralc  s.tt|dtj||jd}|r%tj||jd}tj|||gdd} n tj||gdd} tj|| ||||| d} |j dddd|j d dd d|j d dd dd|_ |j j d ddgd } |j j | ttddd| |_ tj|| |||||j |j|j| d } tj|j jd|j j|j d| |_| |_dS)NZmap_evalrhr)Zaxis)overlap_thresholdevaluate_difficult ap_versionZint32Zaccum_pos_countrFr]Zaccum_true_posZaccum_false_posT)rrrr)r+) initializer)rrrs has_stateZ input_statesZ out_statesrtr*)rOrr&rrmrconcatr Z detection_mapr=rvr#r7Zset_variable_initializerr intr!r.rcur_map accum_map)r$rIZgt_labelZgt_boxZ gt_difficultZ class_numZbackground_labelrrrsrtrJmaprrzrWrrr&hs`    zDetectionMAP.__init__cCs |j|jfSr>)ryrz)r$rrr get_map_vars zDetectionMAP.get_map_varcCsl|durt}t|dt||j}tj|jd|j|dWdn1s*wY| |dS)Nr'rr*) rrrr-rvrr.rrr/)r$r0r1rrrrr2s zDetectionMAP.reset)NNrrpTrqr>)rr?r@rAr&r|r2rbrrrWrr-s>?r) __future__rrnumpyr_rZ frameworkrrrrZ layer_helperr rur r __all__robjectrr r rrrrrs       TZS