o es7 @sddlZddlmZmZmZmZmZmZddlZ ddl m Z m Z m Z mZmZddlmZmZdeedeefddZd e jd eeeefde jfd d Zd+dedede jfddZd,de jdeedefddZde deefddZd-deedeedeede fddZde deeeffddZd,d eeefdeede fd!d"Zd#e deefd$d%Z  d-d&eedeedeede fd'd(Z deddfd)d*Z!dS).N)AnyDictListOptionalSequenceUnion)MapProto OptionalProto SequenceProto TensorProtomapping)load_external_data_for_tensoruses_external_datafareturncsfddttdDS)Ncs*g|]}t|d|ddqS))complex).0irAD:\Projects\ConvertPro\env\Lib\site-packages\onnx/numpy_helper.py s*z,combine_pairs_to_complex..r)rangelenrrrrcombine_pairs_to_complex srdatadimscCs&dd}||tj|tjS)z8Converts ndarray of bf16 (as uint32) to f32 (as uint32).cSs|d>S)Nr)xrrrsz%bfloat16_to_float32..)astypenpint32reshapeviewZfloat32)rrshiftrrrbfloat16_to_float32sr(tensorbase_dirc Cs|dr td|jtjkrtd|j}tj|}tj|}tj|}tj |}|j }|jtj krLt ||}t dd|D} t| ||St|rUt|||drtjdkrct||tjkrvtj|jtjd} t| |Stj|j|d|S|tjkrtj|jtjd|tj S|tjkrtj|jtj!d} t| |St ||} |tj"ks|tj#krt$| } tj| |d||S) zConverts a tensor def object to a numpy array. Inputs: tensor: a TensorProto object. base_dir: if external tensor exists, base_dir can help to find the path to it Returns: arr: the converted array. segmentz*Currently not supporting loading segments.z4The element type in the input tensor is not defined.css|]}|dVqdS)utf-8N)decode)rsrrr .zto_array..raw_databigdtype)%ZHasField ValueError data_typer UNDEFINED TypeErrorr TENSOR_TYPE_TO_NP_TYPEZ"TENSOR_TYPE_TO_STORAGE_TENSOR_TYPEZSTORAGE_TENSOR_TYPE_TO_FIELDrSTRINGgetattrlistr#Zasarrayr"r%rr sys byteorderconvert_endianZBFLOAT16 frombufferr2Zint16r(ZFLOAT16Z int32_dataZuint16r&Zfloat16r$Z COMPLEX64Z COMPLEX128r) r*r+ tensor_dtypenp_dtypeZ storage_typeZstorage_np_dtypeZ storage_fieldrZ utf8_stringsssrrrrto_arraysh                   rEarrnamecCs8t}|j|j|r||_|jtkrotj|j|_ | }|D]K}t |t r2|j |dq!t |tjrX|D]}t |t rK|j |dq:t |trV|j |q:q!t |trd|j |q!tdt t||Sztj|j}Wntytdt |jw||_ ||_tjdkrt||S)zConverts a numpy array to a tensor def. Inputs: arr: a numpy array. name: (optional) the name of the tensor. Returns: TensorProto: the converted tensor def. r-zMUnrecognized object in the object array, expect a string, or array of bytes: z$Numpy data type not understood yet: r3)r rextendshaperGr5objectr NP_TYPE_TO_TENSOR_TYPEr7flatten isinstancestrZ string_dataappendencoder#ndarraybytesNotImplementedErrortypeKeyError RuntimeErrortobytesr2r>r?r@)rFrGr*Z flat_arrayer/r5rrr from_array`sH             rYsequencecCsg}|j}tj|}t||}|D]1}|tjks|tjkr%|t|q|tj kr2|t |q|tj kr?|t |qt d|S)zConverts a sequence def to a Python list. Inputs: sequence: a SequenceProto object. Returns: list: the converted list. z8The element type in the input sequence is not supported.) elem_typer ZSTORAGE_ELEMENT_TYPE_TO_FIELDr<r TENSOR SPARSE_TENSORrOrESEQUENCEto_listMAPto_dictr9)rZlstr[ value_fieldvaluesvaluerrrr_s    r_rbr5c st}|r||_|r |}n#tdkr-d}t|tr tj}nt|tr)tj}ntj}ntj}||_ tdkrHt fddDsHt d|tjkr]D] }|j t|gqO|S|tjkrrD] }|j t|gqd|S|tjkrD] }|j t|gqy|St d)a^Converts a list into a sequence def. Inputs: lst: a Python list name: (optional) the name of the sequence. dtype: (optional) type of element in the input list, used for specifying sequence values when converting an empty list. Returns: SequenceProto: the converted sequence def. rc3s"|] }t|tdVqdS)rN)rMrT)relemrbrrr0s zfrom_list..zqThe element type in the input list is not the same for all elements and therefore is not supported as a sequence.zZThe element type in the input list is not a tensor, sequence, or map and is not supported.)r rGrrMdictr`r=r^r\r[allr9Z tensor_valuesrHrYZsequence_values from_listZ map_values from_dict) rbrGr5rZr[Z first_elemr*seqmaprrgrrjs:    "   rjrmcCs`g}|jtjkrt|j}nt|j}t|j}t|t|kr't d|j dt t ||}|S)zConverts a map def to a Python dictionary. Inputs: map: a MapProto object. Returns: dict: the converted dictionary. z2Length of keys and values for MapProto (map name: z) are not the same.) key_typer r;r= string_keyskeysr_rdr IndexErrorrGrhzip)rmZkey_listZ value_list dictionaryrrrras    rarhcst}|r||_t|}t|djtj}t j t j t j t j t jt jt jt jg}tfdd|Ds.zfThe key type in the input dictionary is not the same for all keys and therefore is not valid as a map.c3rtrurv)rval)raw_value_typerrr0r1zjThe value type in the input dictionary is not the same for all values and therefore is not valid as a map.)rrGr=rpr#arrayr5r rKr ZINT8ZINT16ZINT32ZINT64ZUINT8ZUINT16ZUINT32ZUINT64rir9rdrTrjrnr;rorHCopyFrom)rhrGrmrprnZvalid_key_int_typesrdZ value_seqr)rxrzrrks0          rkoptionalcCsd}|j}|tjkr |Stj|}t||}|tjks |tjkr&t|}|S|tj kr1t |}|S|tj kr.NzUThe element type in the input is not a tensor, sequence, or map and is not supported.)r rGZDataTyperdrMrhr`r=r^r8r\r[Z tensor_valuer|rYZsequence_valuerjZ map_valuerkr9)rrGr5r}Z valid_dtypesr[rrr from_optional@s6      rcCs.|j}tj|}tj|j|d|_dS)z Call to convert endianess of raw data in tensor. Arguments: tensor (TensorProto): TensorProto to be converted. r4N)r7r r:r#rAr2byteswaprW)r*rBrCrrrr@rs r@)r)ru)NN)"r>typingrrrrrrnumpyr#Zonnxrr r r r Zonnx.external_data_helperr rintrrrQr(rNrErYr_rjrarkr~rr@rrrrs4 &J:(2$) 2