o e@s\UdZddlmZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZddlmZdd lmZmZmZdd lmZdd lmZdd lmZmZmZmZmZmZm Z dd l!m"Z"ddl#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*ddl+m,Z,m-Z-ddl.m/Z/m0Z0ddl1m2Z2m3Z3m4Z4ddl5m6Z6m7Z7e8Z9Gddde Z:Gddde Z;e ee:j>e:j?e:j?e:j@e:jAe:jBe:jCe:jDe:jEe:jFe:j>e:j?e:j@e:jCdZGdeHd<e;jIe;jJe;jKe;jLe;jIe;jJe;jMe;jNe;jOe;jPe;jQd ZRdeHd<e:j>de:j?de:j@de:jAde:jCdiZSd eHd!<d"ZTd#d$ZUGd%d&d&eZVGd'd(d(eZWGd)d*d*eZXGd+d,d,eZYGd-d.d.ZZGd/d0d0Z[d1d2Z\eTfdjd5d6Z]dkd9d:Z^Gd;d<d<Z_dld?d@Z`dmdBdCZadndEdFZbdodIdJZcddKdpdMdNZddqdRdSZedrdUdVZfdagdsdYdZZhd[d\Zi ]dtdudadbZjGdcddddZke-ZlGdedfdfZmdgdhZneodikrendSdS)vz/Extract information about the content of a PDF.) annotationsN) defaultdict) ExitStackDecimal)Enumauto)partial)hypotinfisclose)PathLike)Path) ContainerIterableIteratorMapping NamedTupleSequenceTuple)warn)ObjectPdfPdfImagePdfInlineImage PdfMatrixUnsupportedImageTypeErrorparse_content_stream)ExecutorSerialExecutor)EncryptedPdfErrorInputFileError) Resolutionavailable_cpu_countpikepdf_enable_mmap)get_page_analysisget_text_boxesc@sLeZdZdZeZeZeZeZeZ eZ eZ eZ eZ eZdS) Colorspacez1Description of common image colorspaces in a PDF.N)__name__ __module__ __qualname____doc__rgrayrgbcmyklabiccindexsepdevnpatternjpeg2000r6r6ED:\Projects\ConvertPro\env\Lib\site-packages\ocrmypdf/pdfinfo/info.pyr')s r'c@sFeZdZdZeZeZeZeZeZ eZ eZ eZ eZ dS)Encodingz/Description of common image encodings in a PDF.N)r(r)r*r+rccittjpegr5jbig2asciihexascii85lzwflate runlengthr6r6r6r7r89s r8)z /DeviceGrayz/CalGrayz /DeviceRGBz/CalRGBz /DeviceCMYKz/Labz /ICCBasedz/Indexedz /Separationz/DeviceNz/Patternz/Gz/RGBz/CMYKz/Izdict[str, Colorspace]FRIENDLY_COLORSPACE) z/CCITTFaxDecodez /DCTDecodez /JPXDecodez /JBIG2Decodez/CCFz/DCTz/AHxz/A85z/LZWz/Flz/RLzdict[str, Encoding]FRIENDLY_ENCODINGzdict[Colorspace, int] FRIENDLY_COMP)?rHrGrHrHcCs&tt|}t|t}tdd|DS)Ncss"|] \}}t||ddVqdS)gMbP?)Zrel_tolN)r ).0abr6r6r7 y z"_is_unit_square..)mapfloatzip UNIT_SQUAREall) shorthandvaluespairwiser6r6r7_is_unit_squarevs  rVc@*eZdZUdZded<ded<ded<dS) XobjectSettingsz%Info about an XObject found in a PDF.strname/tuple[float, float, float, float, float, float]rSint stack_depthNr(r)r*r+__annotations__r6r6r6r7rX|  rXc@rW) InlineSettingsz*Info about an inline image found in a PDF.riimager[rSr\r]Nr^r6r6r6r7rar`rac@s:eZdZUdZded<ded<ded<ded<d ed <d S) ContentsInfoz*Info about various objects found in a PDF.zlist[XobjectSettings]xobject_settingszlist[InlineSettings] inline_imagesbool found_vector found_textz#Mapping[str, list[XobjectSettings]] name_indexNr^r6r6r6r7rcs  rcc@s*eZdZUdZded<ded<ded<dS) TextboxInfoz%Info about a text box found in a PDF.z!tuple[float, float, float, float]bboxrf is_visible is_corruptNr^r6r6r6r7rjr`rjc@eZdZdZdS) VectorMarkerzCSentinel indicating vector drawing operations were found on a page.Nr(r)r*r+r6r6r6r7roroc@rn) TextMarkerzASentinel indicating text drawing operations were found on a page.Nrpr6r6r6r7rrrqrrccsH|D]\}}t|}td|r|D]}g|fVqq||fVqdS)z7Convert runs of qQ's in the stack into single graphobjszQ*q+$N)rYrematch)Z graphobjsoperandsoperatorcharr6r6r7_normalize_stacks    rx contentstreamrc Csg}t|}g}g}tdd}d}d}td} td} td} d| | B| B} ttt|| D]\} }|\}}|dkra||t |d kr`t |d kr\t d | t d q8|d kryz| }Wq8t yxt dYq8w|dkrt||}q8|dkr|d}t||jt |d}|||t||q8|dkr|d}t||jt |d}||q8|| vrd}q8|| vrd}q8t|||||dS)aInterpret the PDF content stream. The stack represents the state of the PDF graphics stack. We are only interested in the current transformation matrix (CTM) so we only track this object; a full implementation would need to track many other items. The CTM is initialized to the mapping from user space to device space. PDF units are 1/72". In a PDF viewer or printer this matrix is initialized to the transformation to device space. For example if set to (1/72, 0, 0, 1/72, 0, 0) then all units would be calculated in inches. Images are always considered to be (0, 0) -> (1, 1). Before drawing an image there should be a 'cm' that sets up an image coordinate system where drawing from (0, 0) -> (1, 1) will draw on the desired area of the page. PDF units suit our needs so we initialize ctm to the identity matrix. According to the PDF specification, the maximum stack depth is 32. Other viewers tolerate some amount beyond this. We issue a warning if the stack depth exceeds the spec limit and set a hard limit beyond this to bound our memory requirements. If the stack underflows behavior is undefined in the spec, but we just pretend nothing happened and leave the CTM unchanged. cSsgSNr6r6r6r6r7sz%_interpret_contents..FzS s f F f* B B* b b*z TJ Tj " 'zBI ID EI q Q Do cm q z5PDF graphics stack overflowed hard limit at operator z(PDF graphics stack overflowed spec limitQz5PDF graphics stack underflowed - PDF may be malformedcmZDor)rZrSr]z INLINE IMAGE)rbrSr]T)rdrergrhri)rrsetsplitjoin enumeraterxrappendlen RuntimeErrorrpop IndexErrorrXrSrYrarc)ryinitial_shorthandstackctmrdrerirgrhZ vector_opsZtext_showing_opsZ image_opsZoperator_whitelistnZgraphobjrurvZ image_namesettingsrbinliner6r6r7_interpret_contentssr                rreturnr"c sV|\}}}}}}t||t||fdddfddtdD\}}t||S) azGiven the transformation matrix and image size, find the image DPI. PDFs do not include image resolution information within image data. Instead, the PDF page content stream describes the location where the image will be rasterized, and the effective resolution is the ratio of the pixel size to raster target size. Normally a scanned PDF has the paper size set appropriately but this is not guaranteed. The most common case is a cropped image will change the page size (/CropBox) without altering the page content stream. That means it is not sufficient to assume that the image fills the page, even though that is the most common case. A PDF image may be scaled (always), cropped, translated, rotated in place to an arbitrary angle (rarely) and skewed. Only equal area mappings can be expressed, that is, it is not necessary to consider distortions where the effective DPI varies with position. To determine the image scale, transform an offset axis vector v0 (0, 0), width-axis vector v0 (1, 0), height-axis vector vh (0, 1) with the matrix, which gives the dimensions of the image in PDF units. From there we can compare to actual image dimensions. PDF uses row vector * matrix_transposed unlike the traditional matrix * column vector. The offset, width and height vectors can be combined in a matrix and multiplied by the transform matrix. Then we want to calculated magnitude(width_vector - offset_vector) and magnitude(height_vector - offset_vector) When the above is worked out algebraically, the effect of translation cancels out, and the vector magnitudes become functions of the nonzero transformation matrix indices. The results of the derivation are used in this code. pdfimages -list does calculate the DPI in some way that is not completely naive, but it does not get the DPI of rotated images right, so cannot be used anymore to validate this. Photoshop works, or using Acrobat to rotate the image back to normal. It does not matter if the image is partially cropped, or even out of the /MediaBox. R@cSs |dkr||nt}||}|SNr)r )ZdrawnZpixelsZ inches_per_ptscaledpir6r6r7calc<sz_get_dpi..calcc3s"|] }||VqdSrzr6rIrrZ image_drawn image_sizer6r7rLBrMz_get_dpi..N)r)r ranger") ctm_shorthandrrJrKcd_Zdpi_wZdpi_hr6rr7_get_dpis /  rc@seZdZUdZedZded<ded<ddddd d&ddZeddZ eddZ eddZ eddZ eddZ eddZeddZeddZed d!Zed"d#Zd$d%ZdS)' ImageInfoz*Information about an image found in a PDF.z1.000z int | None_comprY_nameN)rZpdfimagerrSr Object | NonerPdfInlineImage | Nonec Cst||_||_|durd|_|}n|durd|_t|}ntd|j|_|j|_ |j r1d|_ nd|_ t |j |_z t|jd|_Wn tyQd|_Ynwz t|jpYd|_Wn tyid|_Ynw|jtjkrttj|_|jtjkrz#|j}|jjdkrd |_n|jjd krd |_n d |_WdSWdSWdStt fy}zd|_t!"d |d|WYd}~dSd}~wwt#|jtrt$|j|_nd|_|jdur|jtj%tj&fvrt$tj'|_dSdSdS)NrZxobjectz%Either pdfimage or inline must be setZstencilimagerrZGRAYrCZCMYKrErDzrAn image with a corrupt or unreadable ICC profile was found. The output PDF may not match the input PDF visually: z. )(rYr _shorthandZ_originr ValueErrorwidth_widthheight_heightZ image_mask_typer\Zbits_per_component_bpcrBgetfilters_encrrAZ colorspace_colorNotImplementedErrorr8r5r'r0ZprofileZ xcolor_spacerAttributeErrorrloggerwarning isinstancerFr9r;r,)selfrZrrrSZpimZpim_iccexr6r6r7__init__Nsl             zImageInfo.__init__cC|jSrz)rrr6r6r7rZzImageInfo.namecCrrz)rrr6r6r7type_rzImageInfo.type_cCrrz)rrr6r6r7rrzImageInfo.widthcCrrz)rrr6r6r7rrzImageInfo.heightcCrrz)rrr6r6r7bpcrz ImageInfo.bpccC|jdur|jSdSN?)rrr6r6r7colorzImageInfo.colorcCrr)rrr6r6r7comprzImageInfo.compcCr)Nr)rrr6r6r7encrz ImageInfo.enccCs|jjo |jdko |jdkSr)r is_finiterrrr6r6r7 renderableszImageInfo.renderablecCst|j|j|jfSrz)rrrrrr6r6r7rrz ImageInfo.dpicCsNd|jd|jd|jd|jd|jd|jd|jd|jd|jdS)Nz ) rZrrrrrrrrrr6r6r7__repr__s"zImageInfo.__repr__)rrrr)r(r)r*r+rZDPI_PRECr_rpropertyrZrrrrrrrrrrr6r6r6r7rFs> H           r contentsinfoIterator[ImageInfo]ccs6t|jD]\}}td|d|j|jdVqdS)z'Find inline images in the contentstreamzinline-02d)rZrSrN)rrerrSrb)rrrr6r6r7_find_inline_imagess  rIterator[tuple[Object, str]]ccshd|vrdS|d}d|vrdS|d}|D]}||}d|vr$q|ddkr1|}||fVqdS)aSearch for all XObject-based images in the container Usually the container is a page, but it could also be a Form XObject that contains images. Filter out the Form XObjects which are dealt with elsewhere. Generate a sequence of tuples (image, xobj container), where container, where xobj is the name of the object and image is the object itself, since the object does not know its own name. /ResourcesN/XObject/Subtypez/Image)as_dict) container resourcesxobjsxobj candidaterr6r6r7_image_xobjectss     rrccs^t|D]'\}}||jvrq|j|D]}|jdkr!t|jr!qt|j||jdVqqdS)zFind images stored in the container's /Resources /XObject Usually the container is a page, but it could also be a Form XObject that contains images. Generates images with their DPI at time of drawing. r)rZrrSN)rrir]rVrSrrZ)rrrrZdrawr6r6r7_find_regular_imagess rpdfrc csd|vrdS|d}d|vrdS|d}|D](}||}|ddkr&q|}|jD]}|j|kr3q+|j} t||| dEdHq+qdS)z|Find any images that are in Form XObjects in the container The container may be a page, or a parent Form XObject. rNrr/FormrrrS)rrdrZrS_process_content_streams) rrrrrrrZ form_xobjectrrr6r6r7_find_form_xobject_imagess*    r)rS/Iterator[VectorMarker | TextMarker | ImageInfo]ccs|ddkrd|vr|pt}n-|ddkr<|ddkr<|r$t|nt}|dt}t|}||}|j}ndSt||}|jrJtV|jrQt Vt |EdHt ||EdHt |||EdHdS) aFind all individual instances of images drawn in the container Usually the container is a page, but it may also be a Form XObject. On a typical page images are stored inline or as regular images in an XObject. Form XObjects may include inline images, XObject images, and recursively, other Form XObjects; and also vector graphic objects. Every instance of an image being drawn somewhere is flattened and treated as a unique image, since if the same image is drawn multiple times on one page it may be drawn at differing resolutions, and our objective is to find the resolution at which the page can be rastered without downsampling. z/Typez/Pagez /Contentsrrrz/MatrixN) rrQridentityrSrrgrorhrrrrr)rrrSrrZform_shorthandZ form_matrixrr6r6r7r$s$  r text_blocksIterable[FloatRect]rfc Csjt|t|}}d}||d||d||||f}d dd }d }|D] } || |r2d }|Sq&|S)z3Smarter text detection that ignores text in marginsg?rCrJ FloatRectrKrrfcSs@|d|dko|d|dko|d|dko|d|dkS)z Where (a,b) are 4-tuple rects (left-0, top-1, right-2, bottom-3) https://stackoverflow.com/questions/306316/determine-if-two-rectangles-overlap-each-other Formula assumes all boxes are in first quadrant rrrCrDr6)rJrKr6r6r7rects_intersectds@z'_page_has_text..rects_intersectFTN)rJrrKrrrf)rO) rZ page_widthZ page_heightpwphZ margin_ratioZ interior_bboxrhas_textrkr6r6r7_page_has_textWs     rIterator[TextboxInfo]ccsN||D]}|jd}|jd}|jdk}|dk}t|j||VqdS)z|Extract only limited content from text boxes We do this to save memory and ensure that our objects are pickleable. rrDu�N)Z_objsZ rendermodeZget_textrjrk)minerZtextbox_getterbox first_line first_charvisiblecorruptr6r6r7simplify_textboxests     rinfilercCsBttd||durt|add}t|dSdS)NpdfminercSs tdSrz) worker_pdfcloser6r6r6r7on_process_closes z1_pdf_pageinfo_sync_init..on_process_close) r$logging getLoggersetLevelropenratexitregister)rrZpdfminer_loglevelrr6r6r7_pdf_pageinfo_sync_inits rc Csp|\}}}}}|dur |nt}t}|s|t|}t|||||}|WdS1s1wYdSrz)rr enter_contextrrPageInfo) argspagenoZ thread_pdfr check_pagesdetailed_analysisrrpager6r6r7_pdf_pageinfo_syncs$r FexecutorrSequence[PageInfo | None]c sdgt|jfdd}|durt}t|j}d} tdtd|} | dkr-d} | r1|ndfddt|D} | rK| dksSJd | dksSJd || | t|d d | d ttt d j t | |dS)Ncs&|}|std||j<|dS)NzCould read a page in the PDF)r!rupdate)resultZpbarr )pagesr6r7update_pageinfos   z1_pdf_pageinfo_concurrent..update_pageinfoFrCrETc3s|] }|fVqdSrzr6r)rrr initial_pdfr6r7rLs z+_pdf_pageinfo_concurrent..zNot multithreadablezScanning contentsr )totaldescunitdisabler) use_threads max_workersZ tqdm_kwargsZworker_initializertaskZtask_argumentsZ task_finished) rrr#minrdictr rrrlevelr ) rr rprogbarrrrrrrZ n_workersZcontextsr6)rrrrrr7_pdf_pageinfo_concurrents>   $  rc@seZdZUdZded<ded<ded< dddZed?ddZ ed?ddZ ed?ddZ ed@d d!Z ed@d"d#Z ed>d$d%Zed>d&d'Zed>d(d)Zejd*d)Zed+d,ZdAdBd0d1ZedCd3d4Zed@d5d6ZedDd8d9Zd:d;Zd-S)Erz9Information about type of contents on each page in a PDF. bool | None _has_text _has_vectorzlist[ImageInfo]_imagesFrrrr\rr rContainer[int]rrfcCs(||_||_||_||||||dSrz)_pageno_infile_detailed_analysis_gather_pageinfo)rrrrrrr6r6r7rszPageInfo.__init__cCs|j|}dd|jD}|d|d}|d|d} ||v} | rP|rPt|jdd} t||| } tt | t |_ d d |j D} t | || |_ ng|_ d|_ |d td }t|tsgt|}||_||td |_| |td |_z t|d|_Wn tyd|_Ynw|dd|ddf}| rd|_d|_ g|_t|||dD]"}t|trd|_qt|trd|_ qt|tr|j|qtn d|_d|_ g|_d|_|jr tdd dd |jD}||_tt!|j"t#|j|_$tt!|j%t#|j|_&dSdS)NcSsg|]}t|qSr6r)rIrr6r6r7 sz-PageInfo._gather_pageinfo..rrrDrCz/CreatorZPScript5cs|]}|jVqdSrzrk)rIrr6r6r7rLz,PageInfo._gather_pageinfo..z /UserUnitrGrz/RotateFrTrHcss|] }|jr|jVqdSrz)rr)rIrr6r6r7rL,s  )'rZMediaBoxas_listrYZdocinfor startswithr%listrr& _textboxesrrrr _userunit _width_inches_height_inchesr\_rotateKeyErrorr r!rrorrrrr_dpir"Ztake_maxroundrrOZ _width_pixelsyZ_height_pixels)rrrrrrr ZmediaboxZwidth_ptZ height_ptZcheck_this_pageZ pscript5_moderZbboxesuserunitZuserunit_shorthandinforr6r6r7r&sh         zPageInfo._gather_pageinforcCrrz)r#rr6r6r7r3rzPageInfo.pagenocC t|jSrz)rfrrr6r6r7r7 zPageInfo.has_textcCs"|jstdtdd|jDS)NzDid not do detailed analysiscsr(rz)rm)rIZtboxr6r6r7rL?r*z,PageInfo.has_corrupt_text..)r%ranyr.rr6r6r7has_corrupt_text;szPageInfo.has_corrupt_textcCr9rz)rfr rr6r6r7 has_vectorAr:zPageInfo.has_vectorrcCrrz)r0rr6r6r7 width_inchesErzPageInfo.width_inchescCrrz)r1rr6r6r7 height_inchesIrzPageInfo.height_inchescCttt|j|jjSrz)r\r5rOr>rrrr6r6r7 width_pixelsMzPageInfo.width_pixelscCr@rz)r\r5rOr?rr6rr6r6r7 height_pixelsQrBzPageInfo.height_pixelscCrrz)r2rr6r6r7rotationUrzPageInfo.rotationcCs|dvr ||_dStd)N)rZiihiiLiz!rotation must be a cardinal angle)r2r)rvaluer6r6r7rDYs cCrrz)r!rr6r6r7images`rzPageInfo.imagesNrrcsDdd|jsdurdurtd|jSfdd|jDS)NcSs4d}|dur |j|kr d}|dur|j|krd}|S)NTF)rlrm)objZ want_visibleZ want_corruptrr6r6r7 predicatees  z)PageInfo.get_textareas..predicatez#Incomplete information on textboxesc3s"|] }|r|jVqdSrzr))rIrIrrJrr6r7rLtrMz)PageInfo.get_textareas..)r.r)rrrr6rKr7 get_textareasds  zPageInfo.get_textareasr"cCs|jdur tddS|jS)NrH)r4r"rr6r6r7rvs  z PageInfo.dpicCrrz)r/rr6r6r7r7|rzPageInfo.userunitrYcCs|jdurdSdS)Nz1.6z1.5r7rr6r6r7 min_versions zPageInfo.min_versionc Cs6d|jd|jd|jd|jd|jd|jd S)Nzr?rDrrrr6r6r7rszPageInfo.__repr__F) rrrr\rr rr"rrf)rr\rrf)rr)NN)rrrrrr"rrY)r(r)r*r+r_rr&rrrr<r=r>r?rArCrDsetterrHrLrr7rNrr6r6r6r7rsL  E                rc@seZdZdZddddedd%d d Zed&ddZed'ddZed(ddZ ed(ddZ ed)ddZ ed(ddZ d*dd Z d!d"Zd#d$ZdS)+PdfInfoz#Get summary information about a PDFFN)rrrrr rrfrrr\r rc Cs||_|dur tdd}t|\}|jrtt|||||||d|_|j dd|_ d|_ d|jvrUt |jj dgdkrDd|_ n!d |jj vr]d|_ WddSWddSWddSWddS1spwYdS) Nriʚ;)rrz/NeedsRenderingFz /AcroFormz/FieldsTz/XFA)r$rrr is_encryptedr r_pagesZRootr_needs_rendering _has_acroformrZAcroForm)rrrrrrr rr6r6r7rs:      "zPdfInfo.__init__rr cCrrzrVrr6r6r7rrz PdfInfo.pagesrYcCtdd|jDS)Ncss|]}|r|jVqdSrz)rNrIr r6r6r7rLsz&PdfInfo.min_version..)maxrrr6r6r7rNszPdfInfo.min_versioncCrZ)Ncss|] }|r|jdkVqdS)rGNrMr[r6r6r7rLsz'PdfInfo.has_userunit..)r;rrr6r6r7 has_userunitrzPdfInfo.has_userunitcCrrz)rXrr6r6r7 has_acroformrzPdfInfo.has_acroform str | PathcCst|jttfs td|jS)Nzcan't get filename from stream)rr$rYrrrr6r6r7filenameszPdfInfo.filenamecCrrz)rWrr6r6r7needs_renderingrzPdfInfo.needs_renderingrcCs |j|SrzrY)ritemr6r6r7 __getitem__ zPdfInfo.__getitem__cCr9rz)rrVrr6r6r7__len__rdzPdfInfo.__len__cCsdt|dS)Nzr?r@rBrFrQrVrXrarcrjrorrrxrrrrrrrrrrrrr rrrfrTrmr(r6r6r6r7s       $ $     W>   ! 3   :6J