sg$ddlZddlmZddlmZmZddlmZmZm Z m Z m Z m Z ddl mZmZe r ddlmZdd lmZe r ddlZdd lmZer dd lmZdd lmZe j6eZeed GddeZy)N)UserDict)ListUnion)add_end_docstringsis_tf_availableis_torch_availableis_vision_availableloggingrequires_backends)Pipelinebuild_pipeline_init_args)Image) load_image)6MODEL_FOR_ZERO_SHOT_IMAGE_CLASSIFICATION_MAPPING_NAMES)9TF_MODEL_FOR_ZERO_SHOT_IMAGE_CLASSIFICATION_MAPPING_NAMES)stable_softmaxT)has_image_processorcreZdZdZfdZd deeeededfffd Zd dZ d dZ dZ d Z xZ S) #ZeroShotImageClassificationPipelineaL Zero shot image classification pipeline using `CLIPModel`. This pipeline predicts the class of an image when you provide an image and a set of `candidate_labels`. Example: ```python >>> from transformers import pipeline >>> classifier = pipeline(model="google/siglip-so400m-patch14-384") >>> classifier( ... "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png", ... candidate_labels=["animals", "humans", "landscape"], ... ) [{'score': 0.965, 'label': 'animals'}, {'score': 0.03, 'label': 'humans'}, {'score': 0.005, 'label': 'landscape'}] >>> classifier( ... "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png", ... candidate_labels=["black and white", "photorealist", "painting"], ... ) [{'score': 0.996, 'label': 'black and white'}, {'score': 0.003, 'label': 'photorealist'}, {'score': 0.0, 'label': 'painting'}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This image classification pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"zero-shot-image-classification"`. See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=zero-shot-image-classification). c t|di|t|d|j|jdk(r t yt y)Nvisiontf)super__init__r check_model_type frameworkrr)selfkwargs __class__s h/var/www/html/venv/lib/python3.12/site-packages/transformers/pipelines/zero_shot_image_classification.pyrz,ZeroShotImageClassificationPipeline.__init__CsH "6"$) ~~% F H imagerc hd|vr|jd}| tdt| |fi|S)a Assign labels to the image(s) passed as inputs. Args: image (`str`, `List[str]`, `PIL.Image` or `List[PIL.Image]`): The pipeline handles three types of images: - A string containing a http link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly candidate_labels (`List[str]`): The candidate labels for this image. They will be formatted using *hypothesis_template*. hypothesis_template (`str`, *optional*, defaults to `"This is a photo of {}"`): The format used in conjunction with *candidate_labels* to attempt the image classification by replacing the placeholder with the candidate_labels. Pass "{}" if *candidate_labels* are already formatted. timeout (`float`, *optional*, defaults to None): The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: A list of dictionaries containing one entry per proposed label. Each dictionary contains the following keys: - **label** (`str`) -- One of the suggested *candidate_labels*. - **score** (`float`) -- The score attributed by the model to that label. It is a value between 0 and 1, computed as the `softmax` of `logits_per_image`. imageszSCannot call the zero-shot-image-classification pipeline without an images argument!)pop ValueErrorr__call__)r r%r!r"s r#r*z,ZeroShotImageClassificationPipeline.__call__MsB@ v JJx(E =rs sw000r$c i}d|vr|d|d<d|vr|d|d<d|vr|d|d<|tjdt||d<|iifS)Ncandidate_labelstimeouthypothesis_templatez^The `tokenizer_kwargs` argument is deprecated and will be removed in version 5 of Transformerstokenizer_kwargs)warningswarn FutureWarning)r r/r!preprocess_paramss r#_sanitize_parametersz8ZeroShotImageClassificationPipeline._sanitize_parametersss  '4:;M4N 0 1  +1)+< i ( F *7=>S7T 3 4  ' MMp 5E 0 1 "b((r$c|i}t||}|j|g|j}|jdk(r|j|j}||d<|Dcgc]}|j |}}|j jjdk(rdnd} |j|f|j| d|} | g|d <|Scc}w) N)r-)r'return_tensorsptr,siglip max_lengthT)r6padding text_inputs) rimage_processorrto torch_dtypeformatmodelconfig model_type tokenizer) r r%r,r.r-r/inputsx sequencesr:r;s r# preprocessz.ZeroShotImageClassificationPipeline.preprocesss  #! 5'2%%eWT^^%T >>T !YYt//0F%5!"">("J,PT$dnnYst~~W^sbrs !, } Ns#C c|jd}|jd}t|dtr|d}n|dd}|jdi||}||jd}|S)Nr,r;r)r,logitsr)r( isinstancerr@logits_per_image)r model_inputsr,r;outputs model_outputss r#_forwardz,ZeroShotImageClassificationPipeline._forwards'++,>?"&&}5 k!nh /%a.K&a.+K$**;{;l;!1.. r$c|jd}|dd}|jdk(rk|jjjdk(rHt j |jd}|j}t|ts|g}n|jdk(rE|jdjd}|j}t|tsW|g}nS|jdk(r,t|d }|jj}ntd |jtt!||d Dcgc] \}}||d  }}}|Scc}}w)Nr,rIrr7r8)dimr)axiszUnsupported framework: c|d S)Nrr)rEs r#zAZeroShotImageClassificationPipeline.postprocess..s_`ab_c^cr$)key)scorelabel)r(rr@rArBtorchsigmoidsqueezetolistrJlistsoftmaxrnumpyr)sortedzip) r rNr,rIprobsscoresrWcandidate_labelresults r# postprocessz/ZeroShotImageClassificationPipeline.postprocesssF(,,-?@x(+ >>T !djj&7&7&B&Bh&NMM&)11"5E\\^Ffd+  ^^t #NNrN*2226E\\^Ffd+  ^^t #"63E[[]))+F6t~~6FGH H+1V=M1NTc*d &o 6    sE*)N)NzThis is a photo of {}.NN)__name__ __module__ __qualname____doc__rrstrrr*r4rGrOrf __classcell__)r"s@r#rr!sS@ $1eCcGT']$JK$1L)(4 *"r$r)r0 collectionsrtypingrrutilsrrr r r r baserrPILr image_utilsrrYmodels.auto.modeling_autormodels.auto.modeling_tf_autortf_utilsr get_loggerrgloggerrrr$r#rxs 5(bh)   H %,FG_(_H_r$