sg=k ddlmZmZmZmZmZmZmZddlZ ddl m Z m Z ddl mZmZmZmZddlmZmZmZmZmZmZmZmZmZmZmZmZddlm Z m!Z!m"Z"e"jFe$Z%e!r ddl&Z&ddl&m'Z'd ee(e(ffd Z)d ed eee jTfd Z+d eed eefdZ, ddeee jTdeee-efd ee(fdZ. dde jTdee(e(fdeee-efd e jTfdZ/ded efdZ0Gdde Z1y))AnyDictIterableListOptionalTupleUnionN)BaseImageProcessor BatchFeature) PaddingModepadresizeto_channel_dimension_format) IMAGENET_STANDARD_MEANIMAGENET_STANDARD_STDChannelDimension ImageInputPILImageResamplingget_image_sizeinfer_channel_dimension_formatis_scaled_imageis_valid_imageto_numpy_array valid_imagesvalidate_preprocess_arguments) TensorTypeis_vision_availablelogging)Imagereturnct||\}}|d}|d}||z }||k\r||kDr|}t||z }n||kDr||kDr|}t||z}t||}t||}||fS)a Get the output size of the image after resizing given a dictionary specifying the max and min sizes. Args: image (`np.ndarray`): Image to resize. size (`Dict[str, int]`): Size of the output image containing the keys "shortest_edge" and "longest_edge". input_data_format (`ChannelDimension` or `str`): The channel dimension format of the input image. Returns: The output size of the image after resizing.  channel_dim shortest_edge longest_edge)rintmax)imagesizeinput_data_formatheightwidthmin_lenmax_len aspect_ratios i/var/www/html/venv/lib/python3.12/site-packages/transformers/models/idefics2/image_processing_idefics2.pyget_resize_output_image_sizer20s#56GHMFE?#G>"G6>L 57?U\)* %FW,F\)*  !F w E 5=imagesclt|r|gg}|St|ttfr!t |dkDrt|dr|g}|St|ttfrLt |dkDr>t|dttfr%t |ddkDrt|ddr |St d)z Convert a single image or a list of images to a list of numpy arrays. Args: images (`ImageInput`): A single image or a list of images. Returns: A list of numpy arrays. rz]Invalid input type. Must be a single image, a list of images, or a list of batches of images.)r isinstancelisttuplelen ValueError)r4s r1make_list_of_imagesr;Psf(" M FT5M *s6{Q>RXYZR[C\ M 6D%=) K!O vay4- 0 q NQ  6!9Q< ( M k  r3valuescJt|Dcgc] }t|c}Scc}w)zO Return the maximum value across all indices of an iterable of values. )zipr()r<values_is r1max_across_indicesr@rs +.v, 7hCM 77 7s  images_listr+c|t|dd}g}|D]%}|D]}|jt|| 't|\}}||fS)zH Get the maximum height and width across all images in a batch. rr#)rappendrr@)rAr+ image_sizesr4r) max_height max_widths r1get_max_height_widthrGysw  :;q>!;LMKU UE   ~eARS T UU/{;J  ""r3r) output_sizect||\}}tj|tj}d|d|d|f<|S)a Make a pixel mask for the image, where 1 indicates a valid pixel and 0 indicates padding. Args: image (`np.ndarray`): Image to make the pixel mask for. output_size (`Tuple[int, int]`): Output size of the mask. r#dtypeN)rnpzerosint64)r)rHr+ input_height input_widthmasks r1make_pixel_maskrSsF!/uBS TL+ 88Krxx 0D()D,  $% Kr3c&t|tjjs|S|jdk(r|S|j d}tj d|j d}tj||}|j d}|S)z Converts an image to RGB format. Only converts if the image is of type PIL.Image.Image, otherwise returns the image as is. Args: image (Image): The image to convert. RGBRGBA)rWrW)r6PILr modeconvertnewr*alpha_composite)r) image_rgba backgroundr\s r1convert_to_rgbr_sz eSYY__ -  zzU v&J6:??ODJ++J CO%--e4O r3c eZdZdZdgZdddej dddddddf deded ee e fd ed ed e d ede e e ee fde e e ee fdededdffd Zej ddfdej"d ee e fd ede e e efde e e efdej"f dZ d'dej"dee e fde e ee fde ede e e efdej"f dZ d(deej"de e ee fdede e e efde ede e e efdefdZ d)dej"d e d!e d"e d#e de e e efdej"fd$Z d)dej"de e e effd%Zdddddddddddddej8fdede ede ed e ee e fd ed e ed e e d e ede e e ee fde e e ee fde ede ede e e efde ede efd&ZxZS)*Idefics2ImageProcessoraH Constructs a Idefics image processor. Args: do_convert_rgb (`bool`, *optional*, defaults to `True`): Whether to convert the image to RGB. This is useful if the input image is of a different format e.g. RGBA. Only has an effect if the input image is in the PIL format. do_resize (`bool`, *optional*, defaults to `True`): Whether to resize the image. The longest edge of the image is resized to be <= `size["longest_edge"]`, with the shortest edge resized to keep the input aspect ratio, with a minimum size of `size["shortest_edge"]`. size (`Dict`, *optional*): Controls the size of the output image. This is a dictionary containing the keys "shortest_edge" and "longest_edge". resample (`Resampling`, *optional*, defaults to `Resampling.BILINEAR`): Resampling filter to use when resizing the image. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image. If set to `True`, the image is rescaled to have pixel values between 0 and 1. rescale_factor (`float`, *optional*, defaults to `1/255`): Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. If set to `True`, the image is normalized to have a mean of `image_mean` and a standard deviation of `image_std`. image_mean (`float` or `List[float]`, *optional*, defaults to `IDEFICS_STANDARD_MEAN`): Mean to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_mean` parameter in the `preprocess` method. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`float` or `List[float]`, *optional*, defaults to `IDEFICS_STANDARD_STD`): Standard deviation to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_std` parameter in the `preprocess` method. Can be overridden by the `image_std` parameter in the `preprocess` method. do_pad (`bool`, *optional*, defaults to `True`): Whether or not to pad the images to the largest height and width in the batch and number of images per sample in the batch, such that the returned tensor is of shape (batch_size, max_num_images, num_channels, max_height, max_width). do_image_splitting (`bool`, *optional*, defaults to `False`): Whether to split the image into a sequence 4 equal sub-images concatenated with the original image. That strategy was first introduced in https://arxiv.org/abs/2311.06607. pixel_valuesTNgp?Fdo_convert_rgb do_resizer*resample do_rescalerescale_factor do_normalize image_mean image_stddo_paddo_image_splittingr!c  t |di| ||_||_||nddd|_||_||_||_||_||nt|_ | | nt|_ | |_ | |_y)Nizi)r%r&)super__init__rcrdr*rerfrgrhrrirrjrkrl)selfrcrdr*rerfrgrhrirjrkrlkwargs __class__s r1rpzIdefics2ImageProcessor.__init__s "6"," ,DCY\2]   $,((2(>*DZ&/&;AV "4r3r) data_formatr+c d|vrd|vrt|||}nd|vrd|vr |d|df}n tdt||f|||d|S)aZ Resize an image. The shortest edge of the image is resized to size["shortest_edge"], with the longest edge resized to keep the input aspect ratio. Args: image (`np.ndarray`): Image to resize. size (`Dict[str, int]`): Size of the output image. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BICUBIC`): Resampling filter to use when resiizing the image. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format of the image. If not provided, it will be the same as the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred. r%r&r,r-z_size must be a dictionary with keys 'shortest_edge' and 'longest_edge' or 'height' and 'width'.)rertr+)r2r:r)rqr)r*rertr+rrs r1rzIdefics2ImageProcessor.resizes{2 d "~'=/t=NOD  'T/NDM2Dq  4 "* Wh lr  r3rHconstant_valuesct||\}}|\}} ||z } | |z } d| fd| ff} t|| tj|||} | S)z< Pad an image with zeros to the given size. r#r)rYrvrtr+)rrr CONSTANT)rqr)rHrvrtr+rPrQ output_height output_width pad_bottom pad_rightpadding padded_images r1 _pad_imagez!Idefics2ImageProcessor._pad_image sp%35FW$X! k&1# |"\1  ;. z?Q N3  %%+#/  r3r4return_pixel_maskreturn_tensorsc jt||}t|}td|D} |t|ddn|}||n|}d} t |D cgc]"} t | D cgc] } | || c} $} } t |D cgc].} t | D cgc]} t j |c} 0} } t |D]H}t||D]5\}}|j|||||| ||<t|||| ||<7J|r| nd} | | fScc} wcc} wcc} wcc} w)ay For a list of images, for each images, pads a batch of images to the bottom and right of the image with zeros to the size of largest height and width. For each sample in the batch, pads the sample with empty images to the max_number of images per sample in the batch. Optionally returns a pixel mask. Args: images (`np.ndarray`): List of list of images to pad. Pads to the largest height and width in the batch. constant_values (`float` or `Iterable[float]`, *optional*): The value to use for the padding if `mode` is `"constant"`. return_pixel_mask (`bool`, *optional*, defaults to `True`): Whether to return a pixel mask. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format of the image. If not provided, it will be the same as the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred. )r+c32K|]}t|ywN)r9).0images_s r1 z-Idefics2ImageProcessor.pad..^s@gS\@sNrc |tjk(r)tjdg|tjS|tj k(r*tjg|dtjSt d)Nr rJz!Invalid channel dimension format.)rFIRSTrMrNuint8LASTr:)r*r+s r1 empty_imagez/Idefics2ImageProcessor.pad..empty_imagedsd $4$:$::xx T "((;;"&6&;&;;xx $  "((;;@A Ar3)rvrtr+)rHr+) rGr9r(rrangerMrN enumeraterrS)rqr4rvrrrtr+pad_size batch_sizemax_num_imagesr_padded_images_list padded_masks batch_idx sample_idxr)s r1rzIdefics2ImageProcessor.pad;sz@(BST[ @@@'K  BZ__iYj TU~9N OA[; / O  V[[eUfgPQU>5JK(+Kg gz* I%.vi/@%A ! E<@OO$3 +&7 =L="9-j97FxCT7 Y' 3  (9|d !<//% P Lgs0D&(D! 8D& D0D+ :D0!D&+D0imw1h1w2h2c|tjk(r|dd||||fS|tjk(r|||||ddfSyr)rrr)rqrrrrrr+s r1_cropzIdefics2ImageProcessor._cropsY  0 6 6 6aB2o& & "2"7"7 7beRUAo& &8r3c t||\}}|dz}|dz}|j|dd||||j||d||||j|d|||||j|||||||gS)a" Split an image into 4 equal sub-images, and the concatenate that sequence with the original image. That means that a single image becomes a sequence of 5 images. This is a "trick" to spend more compute on each image with no changes in the vision encoder. Args: image (`np.ndarray`): Images to split. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred. r)rr)rqr)r+r,r- mid_width mid_heights r1 split_imagez"Idefics2ImageProcessor.split_images 'u.?@ QJ q[ JJuaIz;L M JJuiE:?P Q JJuaY@Q R JJuiUFDU V    r3c R||n |j}||n |j}||n |j}||n |j}||n |j}||n |j }| | n |j } | | n |j} ||n |j}| | n |j} | | n |j} t|}t|ds tdt|||| | ||||r(|Dcgc]}|Dcgc] }t|c}}}}|Dcgc]}|Dcgc] }t!|c}}}}t#|ddr|rt$j'd|t)|dd}| rFg}|D]=}g}|D]#}|j+|j-||%|j/|?|}|r2|Dcgc]%}|Dcgc]}|j1||||c}'}}}|r1|Dcgc]$}|Dcgc]}|j3|||c}&}}}|r2|Dcgc]%}|Dcgc]}|j5|| | |c}'}}}d}| r|j7|d | | \}}|+|Dcgc]}|Dcgc]}t9||| c} }}}d | rt;j<|n|i}|| rt;j<|n||d <t?|| Scc}wcc}}wcc}wcc}}wcc}wcc}}wcc}wcc}}wcc}wcc}}wcc}wcc}}w)a Preprocess a batch of images. Args: images (`ImageInput`): A list of images to preprocess. do_convert_rgb (`bool`, *optional*, defaults to `self.do_convert_rgb`): Whether to convert the image to RGB. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`Dict[str, int]`, *optional*, defaults to `self.size`): Size of the image after resizing. Shortest edge of the image is resized to size["shortest_edge"], with the longest edge resized to keep the input aspect ratio. resample (`int`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. This can be one of the enum `PILImageResampling`. Only has an effect if `do_resize` is set to `True`. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `List[float]`, *optional*, defaults to `self.image_mean`): Image mean to use for normalization. Only has an effect if `do_normalize` is set to `True`. image_std (`float` or `List[float]`, *optional*, defaults to `self.image_std`): Image standard deviation to use for normalization. Only has an effect if `do_normalize` is set to `True`. do_pad (`bool`, *optional*, defaults to `self.do_pad`): Whether or not to pad the images to the largest height and width in the batch. do_image_splitting (`bool`, *optional*, defaults to `self.do_image_splitting`): Whether to split the image into a sequence 4 equal sub-images concatenated with the original image. That strategy was first introduced in https://arxiv.org/abs/2311.06607. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. NrzkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.)rfrgrhrirjrdr*rezIt looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.)r)r*rer+)r)scaler+)r)meanstdr+T)rrr+)input_channel_dimrbpixel_attention_mask)data tensor_type) rdr*rerfrgrhrirjrcrkrlr;rr:rr_rrlogger warning_oncerextendrrCrrescale normalizerrrMarrayr )rqr4rcrdr*rerfrgrhrirjrkrlrr+rtrAr)new_images_list new_imagesrrs r1 preprocessz!Idefics2ImageProcessor.preprocesssJ"+!6IDNN 'tTYY'38#-#9Zt +9+E4K^K^'3'?|TEVEV #-#9Zt !*!6IDNN +9+E4K^K^!-4;;3E3Q/W[WnWn)&1 KN+:  &!)%!  U`a6vFeN51FaKaR]]v6B%u-B] ] ;q>!, -*   s   $ >{1~a?P Q   O% 3 #RE%%d&6&6u>O&PQR&&z2  3 *K  *  "(KKe$]nKoK  *  "(LLuNVgLhK  *  "(NNZYbsNtK $ 04tNfw191 -K-  " *  "(0{VghK 5[Q  +MS2884H+IYmD' (>BBMGaC]. s/ K,8K' K, K7 K2 2K7 L K= )L8 L L L + L4L L; L#L L#'K,2K7=LL LL#)rNN)rTNNNr) __name__ __module__ __qualname____doc__model_input_namesrBILINEARboolrstrr'floatrr rrprMndarrayrrrrrrr rrrrrr __classcell__)rss@r1rarasr#J(( $#'9'B'B '!:>9=#(55538n 5 % 5  555U5$u+#5675E%e"45655!5 5@(:'B'B>BDH # zz# 38n# % # eC)9$9:; # $E#/?*?$@A #  # T:;26DH zz38_uhuo56  ./  $E#/?*?$@A  <:;"&;?26DHC0RZZ C0uhuo56C0 C0 !sJ!78 C0 ./ C0$E#/?*?$@AC0 C0XEI ' JJ '  '  '  '  '$E#/?*?$@A '  '"EI zz $E#/?*?$@A >*.$()-'+%)*.'+:>9=!%-1;?8<2B2H2H!kCkC!kCD> kC tCH~& kC % kCTNkC!kCtnkCU5$u+#567kCE%e"456kCkC%TNkC!sJ!78kC$$45kC ./!kCr3rar)2typingrrrrrrr numpyrMimage_processing_utilsr r image_transformsr rrr image_utilsrrrrrrrrrrrrutilsrrr get_loggerrrrXr r'r2rr;r@rrGrSr_rarnr3r1rse"EDDFUU    >=   H %E#s(O@ tD4D/ED8x}8c8fj#d2::&'#