o e@s\ddlZddlZddlZddlTddlmZddlmZmZGdddZ Gdd d Z dS) N)*) InferResult)get_error_grpc raise_errorc@sHeZdZdZddZddZddZdd Zd d Zd d Z ddZ dS) _InferStreamapSupports sending inference requests and receiving corresponding requests on a gRPC bi-directional stream. Parameters ---------- callback : function Python function that is invoked upon receiving response from the underlying stream. The function must reserve the last two arguments (result, error) to hold InferResult and InferenceServerException objects respectively which will be provided to the function when executing the callback. The ownership of these objects will be given to the user. The 'error' would be None for a successful inference. cCs&||_||_t|_d|_d|_dS)NT) _callback_verbosequeueQueue_request_queue_handler_active)selfcallbackverboserOD:\Projects\ConvertPro\env\Lib\site-packages\tritonclient/grpc/_infer_stream.py__init__7s   z_InferStream.__init__cCs |dSN)closerrrr__del__>s z_InferStream.__del__cCsF|jdur!|jd|jr|j|jrtdd|_dSdS)zGracefully close underlying gRPC streams. Note that this call blocks till response of all currently enqueued requests are not received. Nzstream stopped...)r r putis_alivejoinr printrrrrrAs     z_InferStream.closecCsF|jdur tdtj|j|fd|_|j|jr!tddSdS)zInitializes the handler to process the response from stream and execute the callbacks. Parameters ---------- response_iterator : iterator The iterator over the gRPC response stream. Nz7Attempted to initialize already initialized InferStream)targetargszstream started...)r r threadingThread_process_responsestartr r)rZresponse_iteratorrrr _init_handlerNs   z_InferStream._init_handlercCs"|jr |j|dStddS)zEnqueues the specified request object to be provided in gRPC request stream. Parameters ---------- request : ModelInferRequest The protobuf message holding the ModelInferRequest zThe stream is no longer in valid state, the error detail is reported through provided callback. A new stream should be started after stopping the current stream.N)rr rrrrequestrrr_enqueue_requestbs  z_InferStream._enqueue_requestcCs|j}|S)aYReturns the request details in the order they were added. The call to this function will block until the requests are available in the queue. InferStream._enqueue_request adds the request to the queue. Returns ------- protobuf message The ModelInferRequest protobuf message. )r getr$rrr _get_requestus z_InferStream._get_requestc Csz+|D]%}|jr t|d}}|jdkrt|jd}nt|j}|j||dqWdStjyO}z| |_ t |}|jd|dWYd}~dSd}~ww)aWorker thread function to iterate through the response stream and executes the provided callbacks. Parameters ---------- responses : iterator The iterator to the response from the server for the requests in the stream. N)msg)resulterror) r r error_messageZInferenceServerExceptionrZinfer_responsergrpcZRpcErrorZ is_activerr)r responsesresponser+r,Z rpc_errorrrrr!s     z_InferStream._process_responseN) __name__ __module__ __qualname____doc__rrrr#r&r(r!rrrrr's  rc@s(eZdZdZddZddZddZdS) _RequestIteratorzAn iterator class to provide data to gRPC request stream. Parameters ---------- stream : InferStream The InferStream that holds the context to an active stream. cCs ||_dSr)_stream)rstreamrrrrs z_RequestIterator.__init__cCs|Srrrrrr__iter__sz_RequestIterator.__iter__cCs|j}|dur t|Sr)r6r( StopIterationr$rrr__next__s z_RequestIterator.__next__N)r1r2r3r4rr8r:rrrrr5s   r5) r rr.Ztritonclient.utilsZ _infer_resultr_utilsrrrr5rrrrs |