sg'dZddlmZmZmZddlmZddlmZddl m Z ddl m Z m Z mZmZddlmZmZej&eZGd d ed ZGd de d ZGdde Zy)z Processor class for UDOP. )ListOptionalUnion)logging) BatchFeature) ImageInput)ProcessingKwargsProcessorMixin TextKwargsUnpack)PreTokenizedInput TextInputcleZdZUeeeeeeefed<eeeeeeeefed<y)UdopTextKwargs word_labelsboxesN)__name__ __module__ __qualname__rrrint__annotations__[/var/www/html/venv/lib/python3.12/site-packages/transformers/models/udop/processing_udop.pyrr sH%S 4S ? :;<< d3i$tDI"77 88rrF)totalc 6eZdZUeed<dddddddddd idZy)UdopProcessorKwargs text_kwargsTFr) add_special_tokenspadding truncationstridereturn_overflowing_tokensreturn_special_tokens_maskreturn_offsets_mapping return_lengthverbose)r images_kwargsN)rrrrr _defaultsrrrrr%s6#').*/&+"   Irrc eZdZdZddgZdZdZdgZfdZ dddd d e e d e e e ee ee fd eed efdZdZdZdZedZxZS) UdopProcessora Constructs a UDOP processor which combines a LayoutLMv3 image processor and a UDOP tokenizer into a single processor. [`UdopProcessor`] offers all the functionalities you need to prepare data for the model. It first uses [`LayoutLMv3ImageProcessor`] to resize, rescale and normalize document images, and optionally applies OCR to get words and normalized bounding boxes. These are then provided to [`UdopTokenizer`] or [`UdopTokenizerFast`], which turns the words and bounding boxes into token-level `input_ids`, `attention_mask`, `token_type_ids`, `bbox`. Optionally, one can provide integer `word_labels`, which are turned into token-level `labels` for token classification tasks (such as FUNSD, CORD). Additionally, it also supports passing `text_target` and `text_pair_target` to the tokenizer, which can be used to prepare labels for language modeling tasks. Args: image_processor (`LayoutLMv3ImageProcessor`): An instance of [`LayoutLMv3ImageProcessor`]. The image processor is a required input. tokenizer (`UdopTokenizer` or `UdopTokenizerFast`): An instance of [`UdopTokenizer`] or [`UdopTokenizerFast`]. The tokenizer is a required input. image_processor tokenizerLayoutLMv3ImageProcessor) UdopTokenizerUdopTokenizerFast text_pairc&t|||y)N)super__init__)selfr-r. __class__s rr5zUdopProcessor.__init__Ss )4rN)audiovideosimagestextkwargsreturncT|jtfd|jji||j|}|dj dd}|dj dd} |dj dd} |dj dd} |dj d d} |dj d d} |jjr | td |jjr | td | r | s td | |jdi|dS|jdd|i|d}|j dd}|j dd}|dj d d|dj dd| |dd<||n||dd<| |dd<|3|jjr| t|tr|g}||dd<|jdd||n|i|d}| dur|j|d|d|d<|j||S)a~ This method first forwards the `images` argument to [`~UdopImageProcessor.__call__`]. In case [`UdopImageProcessor`] was initialized with `apply_ocr` set to `True`, it passes the obtained words and bounding boxes along with the additional arguments to [`~UdopTokenizer.__call__`] and returns the output, together with the prepared `pixel_values`. In case [`UdopImageProcessor`] was initialized with `apply_ocr` set to `False`, it passes the words (`text`/``text_pair`) and `boxes` specified by the user along with the additional arguments to [`~UdopTokenizer.__call__`] and returns the output, together with the prepared `pixel_values`. Alternatively, one can pass `text_target` and `text_pair_target` to prepare the targets of UDOP. Please refer to the docstring of the above two methods for more information. tokenizer_init_kwargsrrNrr2r$Fr& text_targetzdYou cannot provide bounding boxes if you initialized the image processor with apply_ocr set to True.zaYou cannot provide word labels if you initialized the image processor with apply_ocr set to True.zKYou cannot return overflowing tokens without returning the offsets mapping.r:r)wordstext_pair_targetr;T pixel_valuesoverflow_to_sample_mappingr) _merge_kwargsrr. init_kwargs'prepare_and_validate_optional_call_argspopgetr- apply_ocr ValueError isinstancestrget_overflowing_imagesupdate)r6r:r;r8r9argsr< output_kwargsrrr2r$r&r@featuresfeatures_wordsfeatures_boxesencoded_inputss r__call__zUdopProcessor.__call__Vs8+**  "&.."<"<  ;d::DA  m,00$?#M266}dK !-044[$G $1-$@$D$DE`bg$h!!.}!=!A!ABZ\a!b#M266}dK    ) )u/@v     ) ){/Fs  %-Cjk k  "!4>> . ,t++\6\]?=[\H%\\'48N%\\'48N - ( , ,]D A - ( , ,-? F8AM- ( 5=B=NETbM- ( 1:EM- ( 7D$8$8$B$ByGXdC( 6D .N )D0+/+F+F^,n=Y.Z,( OON +Orcg}|D]}|j||t|t|k7r#tdt|dt||S)Nz`Expected length of images to be the same as the length of `overflow_to_sample_mapping`, but got z and )appendlenrK)r6r:rDimages_with_overflow sample_idxs rrNz$UdopProcessor.get_overflowing_imagessy!4 ruse)( 2%TTC   H %9Zu9 *%$^GN^Gr