o e!a@sdZddlZddlZddlZddlZddlmZddlZddlm Z ddl m Z ddl m Z ddl m Z ddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddl mZddlmZddlmZdZdZdZ ddZ!d6ddZ"Gdddej#dZ$Gddde$Z%Gddde$Z&Gd d!d!e$Z'Gd"d#d#e$Z(d$d%Z)d&d'Z*Gd(d)d)e$Z+Gd*d+d+e$Z,d6d,d-Z-d.d/Z.d0d1Z/d2d3Z0Gd4d5d5ej1Z2dS)7z>Entry point and high-level logic for TensorBoard.dev uploader.N)logging)tf)experiment_pb2)export_service_pb2_grpc)write_service_pb2_grpc)auth) dry_run_stubs)exporter) flags_parser) formatters) server_info)uploader)server_info_pb2)program) base_pluginakYour use of this service is subject to Google's Terms of Service and Privacy Policy , and TensorBoard.dev's Terms of Service . This notice will not be shown again while you are logged into the uploader. To log out, run `tensorboard dev auth revoke`. diXcCsb|}d}d||tf}tj|tjdtd}|dvr)tdtjddS)z>Prompts for user consent, exiting the program if they decline.z" ***** TensorBoard Uploader *****  zContinue? (yes/NO) )yyesrN) get_ack_message_bodyjoin _MESSAGE_TOSsysstderrwriteinputlowerexit)intentbodyheaderZuser_ack_messageresponser"XD:\Projects\ConvertPro\env\Lib\site-packages\tensorboard/uploader/uploader_subcommand.py_prompt_for_user_ack;s    r$c Cs\ttjt||}t}t|tr%|t j dt j dS| }|sAt|tj|jd}t j d|||j\}}zt|}Wntjyg}z tt|WYd}~nd}~wwt|td||jjstd|tdt|t |} tj!|jj| |d} | |"|| WddS1swYdS) zRuns the main uploader program given parsed flags. Args: flags: An `argparse.Namespace`. experiment_url_callback: A function accepting a single string argument containing the full TB.dev URL of the uploaded experiment. zLogged out of uploader. N)Z force_consolerzReceived server info: <%r>zServer info response: %sz6Internal error: frontend did not specify an API server)options)#rZset_stderrthresholdWARNING _get_intentrZCredentialsStore isinstance_AuthRevokeIntentclearrrrflushZread_credentialsr$Zauthenticate_userZauth_force_consoleZwrite_credentialsZgrpc_creds_typeZchannel_config_get_server_infoserver_info_libZCommunicationError_diestr_handle_server_infoinfo api_serverendpointerrorgrpcZcomposite_channel_credentialsZid_token_call_credentialsZsecure_channelexecute) flagsexperiment_url_callbackrstore credentialsZ channel_credsZchannel_optionsr eZcomposite_channel_credschannelr"r"r#_runHsN          "r=c@s,eZdZdZejddZejddZdS)_IntentzA description of the user's intent in invoking this program. Each valid set of CLI flags corresponds to one intent: e.g., "upload data from this logdir", or "delete the experiment with that ID". cCdS)zGets the message to show when executing this intent at first login. This need not include the header (program name) or Terms of Service notice. Returns: A Unicode string, potentially spanning multiple lines. Nr"selfr"r"r#rs z_Intent.get_ack_message_bodycCr?)aCarries out this intent with the specified gRPC channel. Args: server_info: A `server_info_pb2.ServerInfoResponse` value. channel: A connected gRPC channel whose server provides the TensorBoard reader and writer services. Nr"rAr r<r"r"r#r6s z_Intent.executeN)__name__ __module__ __qualname____doc__abcabstractmethodrr6r"r"r"r#r>s   r>) metaclassc@s eZdZdZddZddZdS)r)z'The user intends to revoke credentials.cCtd)zMust not be called.z(No user ack needed to revoke credentialsAssertionErrorr@r"r"r#rsz&_AuthRevokeIntent.get_ack_message_bodycCrJ)zJExecute handled specially by `main`. Must not be called. z1_AuthRevokeIntent should not be directly executedrKrBr"r"r#r6sz_AuthRevokeIntent.executeN)rCrDrErFrr6r"r"r"r#r)s r)c@2eZdZdZedZddZddZddZ d S) _DeleteExperimentIntentz)The user intends to delete an experiment.aA This will delete the {num} experiment(s) on https://tensorboard.dev with the following experiment ID(s): {experiment_id_list} You have chosen to delete an experiment. All experiments uploaded to TensorBoard.dev are publicly visible. Do not upload sensitive data. cC ||_dSN)experiment_id_list)rArQr"r"r#__init__ z _DeleteExperimentIntent.__init__cCs|jjt|j|jdS)N)numrQ)_MESSAGE_TEMPLATEformatlenrQr@r"r"r#rsz,_DeleteExperimentIntent.get_ack_message_bodyc Cs<t|}|js tdi}d}d}t|jD]Z}|s#d|f||<qzt||d||f||<WqtjyEd||f||<Yqtj yVd||f||<Yqt j yr}zd||f|f||<WYd}~qd}~wwd } d } | D]\} } | |krt | | |kr| | d 7} d } q{| rt| dSdS) Nz2Must specify at least one experiment ID to delete. NO_ACTION DIE_ACTIONzSkipping empty experiment_id.zDeleted experiment %s.NNo such experiment %s. Either it never existed or it has already been deleted.zDCannot delete experiment %s because it is owned by a different user.z*Internal error deleting experiment %s: %s.FrT)rTensorBoardWriterServiceStubrQr FlagsErrorset uploader_libZdelete_experimentExperimentNotFoundErrorPermissionDeniedErrorr5RpcErrorvaluesprintr.) rAr r< api_clientresultsrXrY experiment_idr;Zany_die_actionerr_msgmsgactionr"r"r#r6sj       z_DeleteExperimentIntent.executeN rCrDrErFtextwrapdedentrUrRrr6r"r"r"r#rNs  rNc@4eZdZdZedZd ddZddZdd Z dS) _UpdateMetadataIntentz:The user intends to update the metadata for an experiment.aM This will modify the metadata associated with the experiment on https://tensorboard.dev with the following experiment ID: {experiment_id} You have chosen to modify an experiment. All experiments uploaded to TensorBoard.dev are publicly visible. Do not upload sensitive data. NcCs||_||_||_dSrP)rgname description)rArgrprqr"r"r#rRs z_UpdateMetadataIntent.__init__cC|jj|jdS)N)rg)rUrVrgr@r"r"r#rz*_UpdateMetadataIntent.get_ack_message_bodyc Cs4t|}|j}t|jt|j|stdz t j |||j|jdWnOt j y6t d|Yn@t j yEt d|Yn1t jy]}z t d|WYd}~nd}~wtjyu}z t d|WYd}~nd}~wwtd||jdurtd|j|jdurtd t|jdSdS) Nz1Must specify a non-empty experiment ID to modify.rprqrZzDCannot modify experiment %s because it is owned by a different user.z0Server cannot modify experiment as requested: %sz'Internal error modifying experiment: %szModified experiment %s.zSet name to %rzSet description to %r)rr\rg_die_if_bad_experiment_namerp"_die_if_bad_experiment_descriptionrqrr]r_Zupdate_experiment_metadatar`r.raZInvalidArgumentErrorr5rbrr1repr)rAr r<rergr;r"r"r#r6sT      z_UpdateMetadataIntent.execute)NNrkr"r"r"r#ros  roc@rn) _ListIntentz/The user intends to list all their experiments.z This will list all experiments that you've uploaded to https://tensorboard.dev. TensorBoard.dev experiments are visible to everyone. Do not upload sensitive data. NcC ||_dS)zConstructor of _ListIntent. Args: json: If and only if `True`, will print the list as pretty-formatted JSON objects, one object for each experiment. Njson)rAr{r"r"r#rRL z_ListIntent.__init__cCs|jSrP)_MESSAGEr@r"r"r#rUsz _ListIntent.get_ack_message_bodyc Cst|}tjdddddddddd }tj||d}d}|jr$t}nt }|D]}|d7}|j } t || } t ||| q*tj|sPtjdntjd|tjdS)NT) rprqZ create_timeZ update_timeZnum_runsZnum_tagsZ num_scalarsZtotal_tensor_bytesZtotal_blob_bytes) fieldmaskrz=No experiments. Use `tensorboard dev upload` to get started. zTotal: %d experiment(s) )rTensorBoardExporterServiceStubrZExperimentMask exporter_libZlist_experimentsr{r Z JsonFormatterZReadableFormatterrgr-experiment_urlrdZformat_experimentrstdoutr+rr) rAr r<rer~gencount formatterZ experimentrgurlr"r"r#r6Xs>    z_ListIntent.executerP) rCrDrErFrlrmr}rRrr6r"r"r"r#rxAs  rxcC,|rt|tkrtdt|fdSdSdS)NzFExperiment name is too long. Limit is %s characters. %r was provided.)rW_EXPERIMENT_NAME_MAX_CHARSr.)rpr"r"r#ru}rucCr)NzMExperiment description is too long. Limit is %s characters. %r was provided.)rW!_EXPERIMENT_DESCRIPTION_MAX_CHARSr.)rqr"r"r#rvrrvc@s@eZdZdZedZ      d ddZddZdd Z dS) UploadIntentz?The user intends to upload an experiment from the given logdir.z This will upload your TensorBoard logs to https://tensorboard.dev/ from the following directory: {logdir} This TensorBoard will be visible to everyone. Do not upload sensitive data. NcCsF||_||_||_||_|durdn||_|durdn||_||_dS)NF)logdirrprq verbositydry_runone_shotr8)rArrprqrrrr8r"r"r#rRs  zUploadIntent.__init__cCrr)N)r)rUrVrr@r"r"r#rrsz!UploadIntent.get_ack_message_bodyc Cs|jrt}nt|}t|jt|jt j ||j t |t ||j|j|j|jd}|jrGtjj|j sGtd|j tddS|}t ||}|jdur[|||jsbtd|jrjtdntd|d}zz|WnRt jytdYW|jr|std |j |jfd }|r|d 7}n|d 7}|js|r|d |7}tj|dtj dSt!yd}YnwW|jr|std |j |jfd }|r|d 7}n|d 7}|js|r|d |7}tj|dtj dS|jr|std |j |jfd }|r$|d 7}n|d 7}|js7|r7|d |7}tj|dtj w)N)allowed_plugins upload_limitsrprqrrz%s: No such directory.zbUser specified `one_shot` mode with an unavailable logdir. Exiting without creating an experiment.ztUpload started and will continue reading any new data as it's added to the logdir. To stop uploading, press Ctrl-C.zC ** This is a dry run. No data will be sent to tensorboard.dev. ** z6 New experiment created. View your TensorBoard at: %s Fz4Experiment was deleted; uploading has been cancelledzTensorBoard was run in `one_shot` mode, but did not find any uploadable data in the specified logdir: %s An empty experiment was created. To delete the empty experiment you can execute the following tensorboard dev delete --experiment_id=%sz z Interrupted.zDone.z View your TensorBoard at %srT)"rrZDryRunTensorBoardWriterStubrr\rurprvrqr_ZTensorBoardUploaderrr-rrrrrioZgfileisdirrdZcreate_experimentrr8Zstart_uploadingr`Zhas_datargrrrr+KeyboardInterrupt) rAr r<rer rgrZ interruptedZ end_messager"r"r#r6s                      zUploadIntent.execute)NNNNNNrkr"r"r"r#rs  rc@rM) _ExportIntentz7The user intends to download all their experiment data.a^ This will download all your experiment data from https://tensorboard.dev and save it to the following directory: {output_dir} Downloading your experiment data does not delete it from the service. All experiments uploaded to TensorBoard.dev are publicly visible. Do not upload sensitive data. cCrOrP output_dir)rArr"r"r#rRrSz_ExportIntent.__init__cCrr)Nr)rUrVrr@r"r"r#rrsz"_ExportIntent.get_ack_message_bodyc Cst|}|j}zt||}Wntjy!d|}t|wd}z|D] }|d7}t d|q)Wntj yQ} z t d| j WYd} ~ nd} ~ wwt d||fdS)Nz#Output directory already exists: %rrrzDownloaded experiment %sz Uploader has failed because of a timeout error. Please reach out via e-mail to tensorboard.dev-support@google.com to get help completing your export of experiment %s.z&Done. Downloaded %d experiments to: %s) rrrrZTensorBoardExporterZOutputDirectoryExistsErrorrr]ZexportrdZGrpcTimeoutExceptionrg) rAr r<reoutdirr riZnum_experimentsrgr;r"r"r#r6s6   z_ExportIntent.executeNrkr"r"r"r#rs  rc Csdt|tjd}|durtd|tjkr2|jr-ttj |j|j |j |j |j|j|dStd|tjkrX|jrS|j dusD|j durNt|j|j |j dStdtd|tjkrj|jret|jStd|tjkrut|jd S|tjkr|jrt|jStd |tjkrt|tjd}|durtd |tjkrtStd |ftd |f)aDetermines what the program should do (upload, delete, ...). Args: flags: An `argparse.Namespace` with the parsed flags. experiment_url_callback: A function accepting a single string argument containing the full TB.dev URL of the uploaded experiment. Returns: An `_Intent` instance. Raises: base_plugin.FlagsError: If the command-line `flags` do not correctly specify an intent. Nz%Must specify subcommand (try --help).)rprqrrrr8z0Must specify directory to upload via `--logdir`.rtz0Must specify either `--name` or `--description`.z8Must specify experiment to modify via `--experiment_id`.z;Must specify experiment(s) to delete via `--experiment_id`.rzz-Must specify output directory via `--outdir`.z$Must specify a subcommand to `auth`.zUnknown auth subcommand %rzUnknown subcommand %r) getattrr ZSUBCOMMAND_FLAGrr]ZSUBCOMMAND_KEY_UPLOADrrospath expanduserrprqverboserrZSUBCOMMAND_KEY_UPDATE_METADATArgroZSUBCOMMAND_KEY_DELETErNZSUBCOMMAND_KEY_LISTrxr{ZSUBCOMMAND_KEY_EXPORTrrZSUBCOMMAND_KEY_AUTHZAUTH_SUBCOMMAND_FLAGZAUTH_SUBCOMMAND_KEY_REVOKEr)rL)r7r8cmdZauth_cmdr"r"r#r'/sj              r'cCs\|jptj}t|dg}|jr|jst||j|St||}|jr,|jj r,|j|j_ |S)Nplugins) originr ZDEFAULT_ORIGINrZ api_endpointr-Zcreate_server_infoZfetch_server_infor2r3)r7rrr r"r"r#r,|s     r,cCsz|j}|jtjkrtjd|jtjdS|jtj kr(t d|jdS|jr;tjd|jtjdSdS)NzWarning [from server]: %s zError [from server]: %s%s ) compatibilityZverdictrZ VERDICT_WARNrrrdetailsr+Z VERDICT_ERRORr.)r1compatr"r"r#r0s  r0cCs*tjd|ftjtddS)Nrr)rrrr+r)messager"r"r#r.s r.c@s:eZdZdZd ddZddZddZd d Zd d ZdS)UploaderSubcommandz)Integration point with `tensorboard` CLI.NcCry)zConstructor of UploaderSubcommand. Args: experiment_url_callback: A function accepting a single string argument containing the full TB.dev URL of the uploaded experiment. N)_experiment_url_callback)rAr8r"r"r#rRr|zUploaderSubcommand.__init__cCr?)Ndevr"r@r"r"r#rpzUploaderSubcommand.namecCst|dSrP)r define_flags)rAparserr"r"r#rszUploaderSubcommand.define_flagscCs t||jSrP)r=r)rAr7r"r"r#runs zUploaderSubcommand.runcCr?)Nzupload data to TensorBoard.devr"r@r"r"r#helprzUploaderSubcommand.helprP) rCrDrErFrRrprrrr"r"r"r#rs   rrP)3rFrGrrrlZabslrr5Ztensorboard.compatrZtensorboard.uploader.protorrrZtensorboard.uploaderrrr rr r r r-r r_rZ tensorboardrZtensorboard.pluginsrrrrr$r=ABCMetar>r)rNrorxrurvrrr'r,r0r.ZTensorBoardSubcommandrr"r"r"r#sP                 7S?<  p 0M