o eM&@s6ddlZddlmZmZmZmZmZGdddZdS)N)np_to_triton_dtype raise_errorserialize_bf16_tensorserialize_byte_tensortriton_to_np_dtypec@s\eZdZdZddZddZddZdd Zd d Zdd dZ dddZ ddZ ddZ dS) InferInputaRAn object of InferInput class is used to describe input tensor for an inference request. Parameters ---------- name : str The name of input whose data will be described by this object shape : list The shape of the associated input. datatype : str The datatype of the associated input. cCs(||_||_||_i|_d|_d|_dS)N)_name_shape _datatype _parameters_data _raw_data)selfnameshapedatatyperND:\Projects\ConvertPro\env\Lib\site-packages\tritonclient/http/_infer_input.py__init__4s  zInferInput.__init__cC|jS)zGet the name of input associated with this object. Returns ------- str The name of input )rrrrrr<zInferInput.namecCr)zGet the datatype of input associated with this object. Returns ------- str The datatype of input )r rrrrrFrzInferInput.datatypecCr)zGet the shape of input associated with this object. Returns ------- list The shape of input r rrrrrPrzInferInput.shapecCs ||_|S)zSet the shape of input. Parameters ---------- shape : list The shape of the associated input. Returns ------- InferInput The updated input r)rrrrr set_shapeZs zInferInput.set_shapeTcCst|tjfs td|jdkr%|jt|jkr$td|jt|jnt|j}|j|kr8td||jd}t |j t |j krGd}nt t |j D]}|j ||j |kr\d}qN|sutdt |j dd t |j dd |jd d |jd d |jd d |s|jdd d |_|jdkrtd|jdkrg|_zI|jdkrtj|dgddD]7}|jtjkrt|tkr|jt |ddq|jt |q|jt |ddqW|SW|Stytd|dY|Swdd|D|_|Sd |_|jdkr1t|}|jdkr-||_n#d|_n|jdkrKt|}|jdkrG||_n d|_n||_t |j|jd<|S)aSet the tensor data from the specified numpy array for input associated with this object. Parameters ---------- input_tensor : numpy array The tensor data in numpy array format binary_data : bool Indicates whether to set data for the input in binary format or explicit tensor within JSON. The default value is True, which means the data will be delivered as binary data in the HTTP body after the JSON object. Returns ------- InferInput The updated input Raises ------ InferenceServerException If failed to set data for the tensor. z"input_tensor must be a numpy arrayZBF16zFgot unexpected datatype {} from numpy array, expected {} for BF16 typez8got unexpected datatype {} from numpy array, expected {}TFz4got unexpected numpy array shape [{}], expected [{}]shared_memory_regionNshared_memory_byte_sizeshared_memory_offsetbinary_data_sizezNBF16 inputs must be sent as binary data over HTTP. Please set binary_data=TrueZBYTESrZrefs_okC)flagsorderzutf-8)encodingzFailed to encode "zM" using UTF-8. Please use binary_data=True, if you want to pass a byte array.cSsg|]}|qSr)item).0valrrr sz2InferInput.set_data_from_numpy..) isinstancenpZndarrayrr dtyperformatrlenr rrangestrr popr r sizeZnditerZobject_typer$bytesappendUnicodeDecodeErrorflattenrrtobytes)rZ input_tensorZ binary_datar+Z valid_shapeiobjZserialized_outputrrrset_data_from_numpyjs         &%       zInferInput.set_data_from_numpyrcCsDd|_d|_|jdd||jd<||jd<|dkr ||jd<|S)a Set the tensor data from the specified shared memory region. Parameters ---------- region_name : str The name of the shared memory region holding tensor data. byte_size : int The size of the shared memory region holding tensor data. offset : int The offset, in bytes, into the region where the data for the tensor starts. The default value is 0. Returns ------- InferInput The updated input Nrrrrr)r r r r0)rZ region_nameZ byte_sizeoffsetrrrset_shared_memorys   zInferInput.set_shared_memorycCr)zReturns the raw binary data if available Returns ------- bytes The raw data for the input tensor )r rrrr_get_binary_datarzInferInput._get_binary_datacCsT|j|j|jd}|jr|j|d<|jddur(|jdur(|jdur(|j|d<|S)zRetrieve the underlying input as json dict. Returns ------- dict The underlying tensor specification as dict )rrr parametersrNdata)rr r r getr r )rZtensorrrr _get_tensors    zInferInput._get_tensorN)T)r) __name__ __module__ __qualname____doc__rrrrrr:r<r=rArrrrr&s     n r) numpyr*Ztritonclient.utilsrrrrrrrrrrs