o e6@sdZddlZddlmZddlZddlZddZddZGdd d eZd Z d Z d Z d Z dZ dZGdddeZGdddeZdS)zProgress tracker for uploader.N)datetimecCstdS)z1Get a human-readable time string for the present.z%Y-%m-%dT%H:%M:%S)rnowstrftimerrSD:\Projects\ConvertPro\env\Lib\site-packages\tensorboard/uploader/upload_tracker.pyreadable_time_stringsrcCs8|dkr dt|dS|dkrdt|dSd|S)z0Get a human-readable string for number of bytes.iz%.1f MBiz%.1f kBz%d B)float)bytesrrrreadable_bytes_strings r c@seZdZdZddZddZddZdd Zd d Ze d d Z e ddZ e ddZ e ddZ e ddZe ddZe ddZe ddZe ddZe ddZd d!Zd"d#Zd$d%Zd&d'Zd(d)Zd*d+Zd,S)- UploadStatszStatistics of uploading.cCsRt|_d|_d|_d|_d|_d|_d|_d|_d|_ d|_ d|_ t |_ dS)Nr)time_last_summarized_timestamp_last_data_added_timestamp _num_scalars _num_tensors_num_tensors_skipped _tensor_bytes_tensor_bytes_skipped _num_blobs_num_blobs_skipped _blob_bytes_blob_bytes_skippedset _plugin_namesselfrrr__init__*s  zUploadStats.__init__cCs||j|7_dS)zpAdd a batch of scalars. Args: num_scalars: Number of scalars uploaded in this batch. N)"_refresh_last_data_added_timestamprr num_scalarsrrr add_scalars8szUploadStats.add_scalarscCsT||ksJ||ks J||j|7_|j|7_|j|7_||_dS)aIAdd a batch of tensors. Args: num_tensors: Number of tensors encountered in this batch, including the ones skipped due to reasons such as large exceeding limit. num_tensors: Number of tensors skipped. This describes a subset of `num_tensors` and hence must be `<= num_tensors`. tensor_bytes: Total byte size of tensors encountered in this batch, including the skipped ones. tensor_bytes_skipped: Total byte size of the tensors skipped due to reasons such as size exceeding limit. N)rrrrr)r num_tensorsnum_tensors_skipped tensor_bytestensor_bytes_skippedrrr add_tensorsAs   zUploadStats.add_tensorscCsL||jd7_|j|7_|r$|jd7_|j|7_dSdS)zAdd a blob. Args: blob_bytes: Byte size of the blob. is_skipped: Whether the uploading of the blob is skipped due to reasons such as size exceeding limit. N)rrrrr)r blob_bytes is_skippedrrradd_blob\szUploadStats.add_blobcCs||j|dS)zPAdd a plugin. Args: plugin_name: Name of the plugin. N)rraddrZ plugin_namerrr add_pluginkszUploadStats.add_plugincC|jSN)rrrrrrtzUploadStats.num_scalarscCr-r.)rrrrrr!xr/zUploadStats.num_tensorscCr-r.)rrrrrr"|r/zUploadStats.num_tensors_skippedcCr-r.)rrrrrr#r/zUploadStats.tensor_bytescCr-r.)rrrrrr$r/z UploadStats.tensor_bytes_skippedcCr-r.)rrrrr num_blobsr/zUploadStats.num_blobscCr-r.)rrrrrnum_blobs_skippedr/zUploadStats.num_blobs_skippedcCr-r.)rrrrrr'r/zUploadStats.blob_bytescCr-r.)rrrrrblob_bytes_skippedr/zUploadStats.blob_bytes_skippedcCr-r.)rrrrr plugin_namesr/zUploadStats.plugin_namescCs|jdkp|jdkp|jdkS)zHas any data been tracked by this instance. This counts the tensor and blob data that have been scanned but skipped. Returns: Whether this stats tracking object has tracked any data. r)rrrrrrrhas_datas zUploadStats.has_datacCst|_g}|d|j|j|j}|j|j}||s!dnd|t|f|j |j }|j |j }||sr:rrrrrr 'sB           . r zzzzzzc@seZdZdZdZdddZddZefdd Zd d Z d d Z ddZ ddZ e jddZe jddZe jddZe jddZdS) UploadTrackerz)Tracker for uploader progress and status.)rr&FcCs:||jvrtd||jf||_t|_d|_||_dS)Nz5Unsupported verbosity value %s (supported values: %s)r)_SUPPORTED_VERBISITY_VALUES ValueError _verbosityr _stats _send_count _one_shot)r verbosityZone_shotrrrrs  zUploadTracker.__init__ccs dVq)NTrrrrrr_dummy_generatorszUploadTracker._dummy_generatorcCs<|jsdS|d7}tjt||tdtjdS)z3Overwrite the current line with a stylized message.Nz... )rGsysstdoutwrite_STYLE_ERASE_LINE _STYLE_RESETflush)rmessage color_coderrr_overwrite_line_messagesz%UploadTracker._overwrite_line_messagecCs6|jsdSdttt|f}tj|tjdS)z9Write a timestamped single line, with newline, to stdout.Nz %s[%s]%s %s )rG _STYLE_BOLDrrRrNrOrPrS)rrTZ start_messagerrr_single_line_message s z"UploadTracker._single_line_messagecCs |jS)zBDetermine if any data has been uploaded under the tracker's watch.)rHr4rrrrr4-s zUploadTracker.has_datacCsl|jsdS|js dS|j\}}dttt|f}tj ||r/tj dt |tftj dS)z>Write an update summarizing the data uploaded since the start.Nz%s[%s]%s Total uploaded: %s z%sTotal skipped: %s %s) rGrHr>r=rWrrRrNrOrP_STYLE_DARKGRAYrS)rZ uploaded_strZ skipped_strZuploaded_messagerrr_update_cumulative_status1s&  z'UploadTracker._update_cumulative_statuscCs|j|dSr.)rHr,r+rrradd_plugin_nameHszUploadTracker.add_plugin_namec cs|jd7_|jdkr|dz |ddVW||jr*|ddS|jdtddS||jr@|dw|jdtdw)z5Create a context manager for a round of data sending.r&zStarted scanning logdir.zData upload startingNzDone scanning logdir.z Listening for new data in logdir)rU)rIrXrVrZrJ _STYLE_YELLOWrrrr send_trackerKs*     zUploadTracker.send_trackerc cs8|d|z dVW|j|dS|j|w)zCreate a context manager for tracking a scalar batch upload. Args: num_scalars: Number of scalars in the batch. zUploading %d scalarsN)rVrHr rrrrscalars_tracker_s zUploadTracker.scalars_trackerc csv|rd||t|||t|f}nd|t|f}||zdVW|j||||dS|j||||w)aCreate a context manager for tracking a tensor batch upload. Args: num_tensors: Total number of tensors in the batch. num_tensors_skipped: Number of tensors skipped (a subset of `num_tensors`). Hence this must be `<= num_tensors`. tensor_bytes: Total byte size of the tensors in the batch. tensor_bytes_skipped: Byte size of skipped tensors in the batch (a subset of `tensor_bytes`). Must be `<= tensor_bytes`. z3Uploading %d tensors (%s) (Skipping %d tensors, %s)zUploading %d tensors (%s)N)r rVrHr%)rr!r"r#r$rTrrrtensors_trackerls4  zUploadTracker.tensors_trackerccs,|dt|z t|j|VWdSw)zCreates context manager tracker for uploading a blob. Args: blob_bytes: Total byte size of the blob being uploaded. zUploading binary object (%s)N)rVr _BlobTrackerrH)rr'rrr blob_trackers zUploadTracker.blob_trackerN)F)r?r@rArBrErrL _STYLE_GREENrVrXr4rZr[ contextlibcontextmanagerr]r^r_rarrrrrDs$       (rDc@seZdZddZddZdS)r`cCs||_||_dSr.) _upload_statsr)rZ upload_statsr'rrrrs z_BlobTracker.__init__cCs|jj|j| ddS)N)r()rer)r)rZ is_uploadedrrr mark_uploadeds z_BlobTracker.mark_uploadedN)r?r@rArrfrrrrr`s r`)rBrcrrNr rr objectr rRrWrbr\rYrQrDr`rrrrs$  S%