o e}@sdZddlmZddlZddlZddlmZmZmZmZm Z m Z m Z m Z m Z mZddlZddlZddlmZmZmZddlmZmZeeZGdd d ZGd d d ZGd d d eZdejde ee e e!ffddZ"dS)a Read SPE files. This plugin supports reading files saved in the Princeton Instruments SPE file format. Parameters ---------- check_filesize : bool The number of frames in the file is stored in the file header. However, this number may be wrong for certain software. If this is `True` (default), derive the number of frames also from the file size and raise a warning if the two values do not match. char_encoding : str Deprecated. Exists for backwards compatibility; use ``char_encoding`` of ``metadata`` instead. sdt_meta : bool Deprecated. Exists for backwards compatibility; use ``sdt_control`` of ``metadata`` instead. Methods ------- .. note:: Check the respective function for a list of supported kwargs and detailed documentation. .. autosummary:: :toctree: SpePlugin.read SpePlugin.iter SpePlugin.properties SpePlugin.metadata )datetimeN) AnyCallableDictIteratorListMappingOptionalSequenceTupleUnion)RequestIOModeInitializationError)PluginV3ImagePropertiesc@sxeZdZdZdddddddZid d d d egd dfdddddddddddddddddd d!d"d#d$d%d&d'd(d)d*d+d,id-d.d/d0d1d2d3d4d5d6d7d8d9d:d;d<d=d>d?d@dAdBdCdDdEdFdGdHdIdJdKdLdMdNdOdPdQdRdSdTdUddVdWdXdYddZ Zd[Zeej eej eej eej eej d\Zgd]Zgd^Zd_gZd`S)aSpecaWSPE file specification data Tuples of (offset, datatype, count), where offset is the offset in the SPE file and datatype is the datatype as used in `numpy.fromfile`() `data_start` is the offset of actual image data. `dtypes` translates SPE datatypes (0...4) to numpy ones, e. g. dtypes[0] is dtype("ed d?d@d8dAed edBd=e dCdDed eddfdEd8dFed ed5dBe d0dGeded,d edHededdIe dJdKeded dedLededId=e d0ededdeeded=dMe d0ededdeeded/dNe d0ededMdOe d0ededOdPe d0ededPd.e d0dQdRed edSdTdUd8idVZ e dWe e dXeeeffdYdZZe dWe e d[eeefdXee efffd\d] Ze d^e d_e dXeed`fffdadb Ze dhddedee ffdfdg ZZS)iSDTControlSpeca&Extract metadata written by the SDT-control software Some of it is encoded in the comment strings (see :py:meth:`parse_comments`). Also, date and time are encoded in a peculiar way (see :py:meth:`get_datetime`). Use :py:meth:`extract_metadata` to update the metadata dict. uJänr`JanFebr uMärraMarAprr0ZMairSMayJunr)JulAugr2Sep ZOktr'OctNov ZDez DecstandardZTOCCSLrb arbitrary)ZSEQUZSETOZKINEZSEARc @szeZdZUdZeed< eed< eege fed< e de fed< edfdededeege fde e fddZ dS) zSDTControlSpec.CommentDescz>Describe how to extract a metadata entry from a comment stringnslicecvtNscalecCs||_||_||_||_dSN)rrrr)selfrrrrrprprq__init__s z#SDTControlSpec.CommentDesc.__init__)rcrdrerfint__annotations__rrstrrr floatr rrprprprq CommentDescs*  rZsdt_major_versionrEDZsdt_minor_versionFZsdt_controller_namerZ exposure_timerDIgư>Z color_coder-Zdetection_channelsr/Zbackground_subtractioncC|dkS)NBrpxrprprqzSDTControlSpec.Z em_active cCr)NErprrprprqr rZem_gainZmodulation_active!cCr)NArprrprprqr rZ pixel_sizeg?Z sequence_typecs j|Sr)sequence_typesr __class__rprqr gridZn_macroZ delay_macrogMbP?Zn_miniZ delay_mini%Or97)Zn_microZ delay_microZ n_subpicsZ delay_shutterZdelay_prebleachZ bleach_timeZ recovery_timeZbleach_piezo_active"#cCr)Nzrprrprprqrr))rSr)rSr`rQreturncCsX|ddddkr dSzt|dddt|dddfWSty+YdSw)a-Get the version of SDT-control metadata encoded in the comments Parameters ---------- comments List of SPE file comments, typically ``metadata["comments"]``. Returns ------- Major and minor version. ``-1, -1`` if detection failed. r0rLZCOMVER)rNP)r ValueError)rQrprprqget_comment_version"s * z"SDTControlSpec.get_comment_versionversionc s6i}t|ddD]\}z j|d|f}Wn ty Yq w|D]@\}}z|||j|j}|jdur?||j9}|||<Wq%tye}zt d|d|d||<WYd}~q%d}~wwq |jvrd t ddj} t d |dd |dd d | d |d|d} | |d<|S)aCExtract SDT-control metadata from comments Parameters ---------- comments List of SPE file comments, typically ``metadata["comments"]``. version Major and minor version of SDT-control metadata format Returns ------- Dict of metadata r`rNz-Failed to decode SDT-control metadata field `z`: z, cSs|dd|ddS)Nr.r`02rprrprprqr^z/SDTControlSpec.parse_comments..z)Unsupported SDT-control metadata version rrz. Only versions z@ are supported. Some or all SDT-control metadata may be missing.r comment)rangecomment_fieldsKeyErroritemsrrrr Exceptionwarningswarnjoinmapstrip) rQrZsdt_mdminorZcmtnamespecveZ supported_verrrrprqparse_comments6sB      zSDTControlSpec.parse_commentsr4timeNc sz1j|dd}tt|dd|t|ddt|ddt|ddt|ddWStyM}ztd|dWYd }~d Sd }~ww) aiTurn date and time saved by SDT-control into proper datetime object Parameters ---------- date SPE file date, typically ``metadata["date"]``. time SPE file date, typically ``metadata["time_local"]``. Returns ------- File's datetime if parsing was succsessful, else None. r rSr}rr0r)z1Failed to decode date from SDT-control metadata: rN)monthsrrrloggerinfo)r4rmonthrrrprq get_datetimeis zSDTControlSpec.get_datetimelatin1meta char_encodingcs|d}tdd|DrtddS|d|}|d|||d|d}|rC||d<|d|d|d }z|||d <|d Wnt yct d Ynw|d |d dS)arExtract SDT-control metadata from SPE metadata SDT-control stores some metadata in comments and other fields. Extract them and remove unused entries. Parameters ---------- meta SPE file metadata. Modified in place. char_encoding Character encoding used to decode strings in the metadata. rQcss|]}|dkVqdS)rNrp.0crprprq sz2SDTControlSpec.extract_metadata..zSDT-control comments not found.Nr4rFrrYZmodulation_scriptzHFailed to decode SDT-control laser modulation script. Bad char_encoding?rIr3) ranyrdebugrpopupdaterdecodeUnicodeDecodeErrorrr)rrZcomversdt_metadtZsp4rrprqextract_metadatas.       zSDTControlSpec.extract_metadata)r)rcrdrerfrrrrrrrr staticmethodr r rrrrr rrrr __classcell__rprprrqrrs         "   2$"rrc seZdZ   ddededeedeeddf fdd Zd d d ede j fd dZ de e j fddZ   d d ededededeeeff ddZdededeeeffddZdeeeffddZd!d edefddZdeeefdedeeeffddZZS)" SpePluginTNrequestcheck_filesizerrrc slt||jjtjkrtd|durtdt ||_ |dur(tdt ||_ |j |_zy|tjd}|d|_tj|d|_|d|d f|_|d |_|r|dd kra|d }n |jd tj|j}|tj}||jd |jd|jj}||jkrtd|j jd|jd|dt||j|_|jtjWdStytdw)aInstantiate a new SPE file plugin object Parameters ---------- request : Request A request object representing the resource to be operated on. check_filesize : bool If True, compute the number of frames from the filesize, compare it to the frame count in the file header, and raise a warning if the counts don't match. (Certain software may create files with char_encoding : str Deprecated. Exists for backwards compatibility; use ``char_encoding`` of ``metadata`` instead. sdt_meta : bool Deprecated. Exists for backwards compatibility; use ``sdt_control`` of ``metadata`` instead. zcannot write SPE filesNz{Passing `char_encoding` to the constructor is deprecated. Use `char_encoding` parameter of the `metadata()` method instead.ztPassing `sdt_meta` to the constructor is deprecated. Use `sdt_control` parameter of the `metadata()` method instead.rrrrrrrarrr`zThe file header of z claims there are z frames, but there are actually z frames.z)SPE plugin cannot read the provided file.) superrr1Zio_moderwriterrrDeprecationWarning_char_encoding _sdt_metarZget_file_file _parse_headerrrg_file_header_verrl_dtype_shape_lenseekosSEEK_ENDtellrkitemsizefilenameminr)rrrrrrZdata_endlinerrprqrsT            zSpePlugin.__init__.indexrcCs|turtj}|jd|jd|j}|jg|jR}n=|dkr)td|d||jkr:td|d|jdtj||jd|jd|jj}|jd|jd}|j}|j |t j |j|j|d}| |S)a<Read a frame or all frames from the file Parameters ---------- index : int Select the index-th frame from the file. If index is `...`, select all frames and stack them along a new axis. Returns ------- A Numpy array of pixel values. rr`zIndex `z` is smaller than 0.z5` exceeds the number of frames stored in this file (`z`).ricount) Ellipsisrrkrr IndexErrorrrrrrhfromfileZreshape)rr read_offsetrZ out_shapedatarprprqreads&   zSpePlugin.readcsfddtjDS)zrIterate over the frames in the file Yields ------ A Numpy array of pixel values. c3s|] }j|dVqdS)rN)r)rirrprqr1sz!SpePlugin.iter..)rrrrprrqiter)szSpePlugin.iterrexclude_applied sdt_controlcCs>|jdkr|jdur |j}|jdur|j}|||S|S)uaSPE specific metadata. Parameters ---------- index : int Ignored as SPE files only store global metadata. exclude_applied : bool Ignored. Exists for API compatibility. char_encoding : str The encoding to use when parsing strings. sdt_control : bool If `True`, decode special metadata written by the SDT-control software if present. Returns ------- metadata : dict Key-value pairs of metadata. Notes ----- SPE v3 stores metadata as XML, whereas SPE v2 uses a binary format. .. rubric:: Supported SPE v2 Metadata fields ROIs : list of dict Regions of interest used for recording images. Each dict has the "top_left" key containing x and y coordinates of the top left corner, the "bottom_right" key with x and y coordinates of the bottom right corner, and the "bin" key with number of binned pixels in x and y directions. comments : list of str The SPE format allows for 5 comment strings of 80 characters each. controller_version : int Hardware version logic_output : int Definition of output BNC amp_hi_cap_low_noise : int Amp switching mode mode : int Timing mode exp_sec : float Alternative exposure in seconds date : str Date string detector_temp : float Detector temperature detector_type : int CCD / diode array type st_diode : int Trigger diode delay_time : float Used with async mode shutter_control : int Normal, disabled open, or disabled closed absorb_live : bool on / off absorb_mode : int Reference strip or file can_do_virtual_chip : bool True or False whether chip can do virtual chip threshold_min_live : bool on / off threshold_min_val : float Threshold minimum value threshold_max_live : bool on / off threshold_max_val : float Threshold maximum value time_local : str Experiment local time time_utc : str Experiment UTC time adc_offset : int ADC offset adc_rate : int ADC rate adc_type : int ADC type adc_resolution : int ADC resolution adc_bit_adjust : int ADC bit adjust gain : int gain sw_version : str Version of software which created this file spare_4 : bytes Reserved space readout_time : float Experiment readout time type : str Controller type clockspeed_us : float Vertical clock speed in microseconds readout_mode : ["full frame", "frame transfer", "kinetics", ""] Readout mode. Empty string means that this was not set by the Software. window_size : int Window size for Kinetics mode file_header_ver : float File header version chip_size : [int, int] x and y dimensions of the camera chip virt_chip_size : [int, int] Virtual chip x and y dimensions pre_pixels : [int, int] Pre pixels in x and y dimensions post_pixels : [int, int], Post pixels in x and y dimensions geometric : list of {"rotate", "reverse", "flip"} Geometric operations sdt_major_version : int (only for files created by SDT-control) Major version of SDT-control software sdt_minor_version : int (only for files created by SDT-control) Minor version of SDT-control software sdt_controller_name : str (only for files created by SDT-control) Controller name exposure_time : float (only for files created by SDT-control) Exposure time in seconds color_code : str (only for files created by SDT-control) Color channels used detection_channels : int (only for files created by SDT-control) Number of channels background_subtraction : bool (only for files created by SDT-control) Whether background subtraction war turned on em_active : bool (only for files created by SDT-control) Whether EM was turned on em_gain : int (only for files created by SDT-control) EM gain modulation_active : bool (only for files created by SDT-control) Whether laser modulation (“attenuate”) was turned on pixel_size : float (only for files created by SDT-control) Camera pixel size sequence_type : str (only for files created by SDT-control) Type of sequnce (standard, TOCCSL, arbitrary, …) grid : float (only for files created by SDT-control) Sequence time unit (“grid size”) in seconds n_macro : int (only for files created by SDT-control) Number of macro loops delay_macro : float (only for files created by SDT-control) Time between macro loops in seconds n_mini : int (only for files created by SDT-control) Number of mini loops delay_mini : float (only for files created by SDT-control) Time between mini loops in seconds n_micro : int (only for files created by SDT-control) Number of micro loops delay_micro : float (only for files created by SDT-control) Time between micro loops in seconds n_subpics : int (only for files created by SDT-control) Number of sub-pictures delay_shutter : float (only for files created by SDT-control) Camera shutter delay in seconds delay_prebleach : float (only for files created by SDT-control) Pre-bleach delay in seconds bleach_time : float (only for files created by SDT-control) Bleaching time in seconds recovery_time : float (only for files created by SDT-control) Recovery time in seconds comment : str (only for files created by SDT-control) User-entered comment. This replaces the "comments" field. datetime : datetime.datetime (only for files created by SDT-control) Combines the "date" and "time_local" keys. The latter two plus "time_utc" are removed. modulation_script : str (only for files created by SDT-control) Laser modulation script. Replaces the "spare_4" key. bleach_piezo_active : bool (only for files created by SDT-control) Whether piezo for bleaching was enabled raN)rrr_metadata_pre_v3_metadata_post_v3)rrrrrrprprqrj3s M   zSpePlugin.metadatac s|tj|dd}|dkrdn|}tdd|d<fdddDd<fd dd Dd <fd dd Dd<fdddDd<dddDd<g}dd}|d@ri|d|d@rr|d|d@r{|d|d<d}d|krttjkrnn tj|dd<ndd<d}d|krttjkrnn tj|dd<ndd<dD] }t ||<q|rt |S)aZExtract metadata from SPE v2 files Parameters ---------- char_encoding String character encoding sdt_control If `True`, try to decode special metadata written by the SDT-control software. Returns ------- dict mapping metadata names to values. rNr`r cg|]}|dqSrrrkmrprq rz.SpePlugin._metadata_pre_v3..)r(r*Z chip_sizecrrr r r rprqr r)r,r.Zvirt_chip_sizecrrr r r rprqr!r)rZr\Z pre_pixelscrrr r r rprqr"r)r[r]Z post_pixelscSsg|]}t|qSrp)rrrprprqr%srQrTrrotater reverser0flipr^r_)r;r>r@rC) rrrjrroi_array_to_dictappendlenrmrnboolrrr) rrrnrgftrr rpr rqrs>      zSpePlugin._metadata_pre_v3cCs0|tjd}|j|d|j}d|iS)zExtract XML metadata from SPE v3 files Returns ------- dict with key `"__xml"`, whose value is the XML metadata rrZ__xml)rrrgrrr)rrxmlrprprqrMs zSpePlugin._metadata_post_v3cCs:|turt|jg|jR|j|jddSt|j|jddS)aStandardized ndimage metadata. Parameters ---------- index : int If the index is an integer, select the index-th frame and return its properties. If index is an Ellipsis (...), return the properties of all frames in the file stacked along a new batch dimension. Returns ------- properties : ImageProperties A dataclass filled with standardized image metadata. T)shaperiZn_imagesis_batchF)rrir)rrrrr)rrrprprq propertiesZszSpePlugin.propertiesrc si}tfdd}|D]`\}}|j|dt|dkr#dn|d}tj|j|d|d}|jjdkrV|t j vrVz||}Wnt yUt d |d Ynwz|}Wntyjt|}Ynw|||<q|S) aGet information from SPE file header Parameters ---------- spec Maps header entry name to its location, data type description and optionally number of entries. See :py:attr:`Spec.basic` and :py:attr:`Spec.metadata`. char_encoding String character encoding Returns ------- Dict mapping header entry name to its value cs |Sr)rrrrprqrrz)SpePlugin._parse_header..rrar`r rSzFailed to decode "z3" metadata string. Check `char_encoding` parameter.)rhZ vectorizerrrrrrikindrrorrritemrZsqueeze) rrrretrrspZcntrrprrqrts*      zSpePlugin._parse_header)TNN).TrT).)rcrdrerrr rrrrhndarrayrrrrrrjrrrrrr rrrprprrqrsVP%   UF   rarc Csdg}|gd}|D]%\}}}}}}t|t|gt|t|gt|t|gd}||q |S)a\Convert the `ROIs` structured arrays to :py:class:`dict` Parameters ---------- a Structured array containing ROI data Returns ------- One dict per ROI. Keys are "top_left", "bottom_right", and "bin", values are tuples whose first element is the x axis value and the second element is the y axis value. )r!r$r"r%r#r&)top_left bottom_rightbin)rr) r&Z dict_listsxZsyexZeyZgxgyZroi_dictrprprqrs  r)#rfrloggingrtypingrrrrrrr r r r rnumpyrhZ core.requestrrrZcore.v3_plugin_apirr getLoggerrcrrrrrr%rrrrprprprqs& #0  r(s