sgœ BdZddlZddlmZmZmZmZmZddlZddl Zddlm Z ddl m Z m Z mZddlmZddlmZmZmZmZdd lmZdd lmZmZdd lmZmZmZmZm Z m!Z!d d l"m#Z#ejHe%Z&dZ'dZ(gdZ)dZ*dZ+dBde,de,dee,de,fdZ-Gdde j\Z/Gdde j\Z0Gdde j\Z1Gdde j\Z2Gdd e j\Z3Gd!d"e j\Z4Gd#d$e j\Z5Gd%d&e j\Z6Gd'd(e j\Z7Gd)d*e j\Z8Gd+d,e j\Z9Gd-d.e j\Z:Gd/d0eZ;d1Zed6e<Gd7d8e;Z?Gd9d:e j\Z@Gd;de j\ZBed?e<Gd@dAe;ZCy)CzPyTorch MobileViT model.N)DictOptionalSetTupleUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)BaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttentionSemanticSegmenterOutput)PreTrainedModel) find_pruneable_heads_and_indicesprune_linear_layer)add_code_sample_docstringsadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings torch_int)MobileViTConfigrzapple/mobilevit-small)rirztabby, tabby catvaluedivisor min_valuereturnc|||}t|t||dz z|z|z}|d|zkr||z }t|S)a Ensure that all layers have a channel count that is divisible by `divisor`. This function is taken from the original TensorFlow repo. It can be seen here: https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet/mobilenet.py g?)maxint)rrr new_values c/var/www/html/venv/lib/python3.12/site-packages/transformers/models/mobilevit/modeling_mobilevit.pymake_divisibler(>sS  Is57Q;#677BWLMI3;W y>ceZdZ ddedededededededed ed eeefd d ffd Zde jd e jfdZ xZ S)MobileViTConvLayerconfig in_channels out_channels kernel_sizestridegroupsbiasdilationuse_normalizationuse_activationr!Nc $t |t|dz dz |z} ||zdk7rtd|d|d||zdk7rtd|d|dt j ||||| |||d |_| r t j|d d d d |_nd|_| rdt| trt| |_ yt|jtrt|j|_ y|j|_ yd|_ y)Nrr#rzInput channels (z) are not divisible by z groups.zOutput channels (zeros) r-r.r/r0paddingr3r1r2 padding_modegh㈵>g?T) num_featuresepsmomentumaffinetrack_running_stats)super__init__r% ValueErrorrConv2d convolution BatchNorm2d normalization isinstancestrr activation hidden_act) selfr,r-r.r/r0r1r2r3r4r5r8 __class__s r'r@zMobileViTConvLayer.__init__Ns* {Q!+,x7  1 $/ }UV\U]]efg g99#%#    !#)$( "D "&D  .#."("8F--s3"():):";"("3"3"DOr)featuresc|j|}|j|j|}|j|j|}|SN)rCrErH)rJrLs r'forwardzMobileViTConvLayer.forwardsK##H-    )))(3H ?? &x0Hr))rrFrTT) __name__ __module__ __qualname__rr%boolrrGr@torchTensorrO __classcell__rKs@r'r+r+Ms"&+/4#4#4# 4#  4#  4#4#4#4# 4#dCi(4# 4#l r)r+c xeZdZdZ d dedededededdf fd Zd ejdejfd Z xZ S) MobileViTInvertedResidualzQ Inverted residual block (MobileNetv2): https://arxiv.org/abs/1801.04381 r,r-r.r0r3r!Nc @t|ttt ||j zd}|dvrt d|d|dk(xr||k(|_t|||d|_ t|||d||||_ t|||dd |_ y) Nr)rr#zInvalid stride .rr-r.r/r )r-r.r/r0r1r3Fr-r.r/r5) r?r@r(r%round expand_ratiorA use_residualr+ expand_1x1conv_3x3 reduce_1x1)rJr,r-r.r0r3expanded_channelsrKs r'r@z"MobileViTInvertedResidual.__init__s *3u[6CVCV5V/W+XZ[\  vha89 9#q[K{l/J,  :KYZ + )*$  - )%  r)rLc|}|j|}|j|}|j|}|jr||zS|SrN)rarbrcr`)rJrLresiduals r'rOz!MobileViTInvertedResidual.forwardsI??8,==*??8,&*&7&7x("EXEr)r) rPrQrR__doc__rr%r@rTrUrOrVrWs@r'rYrYsc jk % 47 GJ TW cf  BF FFr)rYc teZdZ d dedededededdf fd Zd ejdejfd ZxZ S) MobileViTMobileNetLayerr,r-r.r0 num_stagesr!Nct|tj|_t |D]5}t ||||dk(r|nd}|jj||}7y)Nrr)r-r.r0)r?r@r ModuleListlayerrangerYappend) rJr,r-r.r0rkirnrKs r'r@z MobileViTMobileNetLayer.__init__sh ]]_ z" 'A-')!"avQ E JJ  e $&K 'r)rLc8|jD] }||} |SrNrn)rJrL layer_modules r'rOzMobileViTMobileNetLayer.forwards$ JJ .L#H-H .r))rr rPrQrRrr%r@rTrUrOrVrWs@r'rjrjsVop'%'47'GJ'TW'il' '  r)rjceZdZdededdffd ZdejdejfdZdejdejfd Z xZ S) MobileViTSelfAttentionr, hidden_sizer!Nct|||jzdk7rtd|fd|jd|j|_t ||jz |_|j|j z|_tj||j |j|_ tj||j |j|_ tj||j |j|_ tj|j|_y)NrzThe hidden size z4 is not a multiple of the number of attention heads r[)r2)r?r@num_attention_headsrAr%attention_head_size all_head_sizerLinearqkv_biasquerykeyrDropoutattention_probs_dropout_probdropoutrJr,rxrKs r'r@zMobileViTSelfAttention.__init__s  33 3q 8";<.1334A7  $*#=#= #&{V5O5O'O#P !558P8PPYY{D,>,>V__U 99[$*<*<6??SYY{D,>,>V__U zz&"E"EF r)xc|jdd|j|jfz}|j|}|j ddddS)Nrr#rr )sizerzr{viewpermute)rJr new_x_shapes r'transpose_for_scoresz+MobileViTSelfAttention.transpose_for_scoressNffhsmt'?'?AYAY&ZZ AFFK yyAq!$$r) hidden_statesc|j|}|j|j|}|j|j|}|j|}t j ||j dd}|tj|jz }tjj|d}|j|}t j ||}|jddddj}|j!dd|j"fz} |j$| }|S)Nrdimrr#rr )rrrrrTmatmul transposemathsqrtr{r functionalsoftmaxrr contiguousrr|r) rJrmixed_query_layer key_layer value_layer query_layerattention_scoresattention_probs context_layernew_context_layer_shapes r'rOzMobileViTSelfAttention.forwards& JJ}5--dhh}.EF // =0IJ //0AB !<< Y5H5HR5PQ+dii8P8P.QQ--//0@b/I,,7 _kB %--aAq9DDF "/"4"4"6s";t?Q?Q>S"S* **,CD r)) rPrQrRrr%r@rTrUrrOrVrWs@r'rwrwsWGGSGTG&%ell%u||% U\\ellr)rwcdeZdZdededdffd ZdejdejfdZxZ S)MobileViTSelfOutputr,rxr!Nct|tj|||_tj |j |_yrNr?r@rr}denserhidden_dropout_probrrs r'r@zMobileViTSelfOutput.__init__s6 YY{K8 zz&"<"<= r)rcJ|j|}|j|}|SrNrrrJrs r'rOzMobileViTSelfOutput.forward s$ =1  ]3 r)rurWs@r'rrs8>>S>T> U\\ellr)rczeZdZdededdffd ZdeeddfdZdejdejfd Z xZ S) MobileViTAttentionr,rxr!Nct|t|||_t |||_t |_yrN)r?r@rw attentionroutputset pruned_headsrs r'r@zMobileViTAttention.__init__s4 / D)&+> Er)headsc>t|dk(ryt||jj|jj|j \}}t |jj||j_t |jj||j_t |jj||j_ t |jj|d|j_ |jjt|z |j_|jj|jjz|j_ |j j||_y)Nrrr)lenrrrzr{rrrrrrrr|union)rJrindexs r' prune_headszMobileViTAttention.prune_headss u:? 7 4>>55t~~7Y7Y[_[l[l u 2$..2F2FN/0B0BEJ1$..2F2FN.t{{/@/@%QO .2^^-O-ORUV[R\-\*'+~~'I'IDNNLnLn'n$ --33E:r)rcJ|j|}|j|}|SrN)rr)rJr self_outputsattention_outputs r'rOzMobileViTAttention.forward,s%~~m4 ;;|4r)) rPrQrRrr%r@rrrTrUrOrVrWs@r'rrsO""S"T" ;S;d;$ U\\ ell r)rcheZdZdedededdffd ZdejdejfdZxZ S) MobileViTIntermediater,rxintermediate_sizer!Nct|tj|||_t |j trt|j |_ y|j |_ yrN) r?r@rr}rrFrIrGr intermediate_act_fnrJr,rxrrKs r'r@zMobileViTIntermediate.__init__3sR YY{,=> f'' -'-f.?.?'@D $'-'8'8D $r)rcJ|j|}|j|}|SrN)rrrs r'rOzMobileViTIntermediate.forward;s& =1 00? r)rurWs@r'rr2sA99S9UX9]a9U\\ellr)rceZdZdedededdffd Zdejdejdejfd ZxZ S) MobileViTOutputr,rxrr!Nct|tj|||_tj |j |_yrNrrs r'r@zMobileViTOutput.__init__Bs7 YY0+> zz&"<"<= r)r input_tensorcT|j|}|j|}||z}|SrNr)rJrrs r'rOzMobileViTOutput.forwardGs. =1  ]3 % 4 r)rurWs@r'rrAsO>>S>UX>]a> U\\RWR^R^r)rcheZdZdedededdffd ZdejdejfdZxZ S) MobileViTTransformerLayerr,rxrr!Nc$t|t|||_t ||||_t ||||_tj||j|_ tj||j|_ y)Nr;) r?r@rrr intermediaterrr LayerNormlayer_norm_epslayernorm_beforelayernorm_afterrs r'r@z"MobileViTTransformerLayer.__init__Osq +FK@1&+GXY%fk;LM " [f>S>S T!||KV=R=RSr)rc|j|j|}||z}|j|}|j|}|j ||}|SrN)rrrrr)rJrr layer_outputs r'rOz!MobileViTTransformerLayer.forwardWs\>>$*?*? *NO(=8 ++M: ((6 {{<? r)rurWs@r'rrNsFTTSTUXT]aTU\\ellr)rcheZdZdedededdffd ZdejdejfdZxZ S) MobileViTTransformerr,rxrkr!Nc t|tj|_t |D]A}t ||t||jz}|jj|Cy)N)rxr) r?r@rrmrnrorr% mlp_ratiorp)rJr,rxrk_transformer_layerrKs r'r@zMobileViTTransformer.__init__bsh ]]_ z" 1A 9'"%kF4D4D&D"E!  JJ  / 0  1r)rc8|jD] }||} |SrNrs)rJrrts r'rOzMobileViTTransformer.forwardns% JJ 8L(7M 8r)rurWs@r'rras@ 1 1S 1c 1VZ 1U\\ellr)rceZdZdZ ddededededededed d ffd Zd ejd e eje ffd Z dejde d ejfdZ d ejd ejfdZ xZS)MobileViTLayerz; MobileViT block: https://arxiv.org/abs/2110.02178 r,r-r.r0rxrkr3r!Nct||j|_|j|_|dk(r)t ||||dk(r|nd|dkDr|dznd|_|}nd|_t||||j|_ t|||ddd|_ t||||_ tj||j|_t|||d|_t|d|z||j|_y) Nr#r)r-r.r0r3r\F)r-r.r/r4r5)rxrkr)r?r@ patch_size patch_width patch_heightrYdownsampling_layerr+conv_kernel_sizeconv_kxkconv_1x1r transformerrrr layernormconv_projectionfusion) rJr,r-r.r0rxrkr3rKs r'r@zMobileViTLayer.__init__ys !,,"-- Q;&?')!)QvA*2Q,QA 'D #'K&*D #* #$//   + #$#   0 #!  kv7L7LM1  +ST ) KkW]WnWn  r)rLc||j|j}}t||z}|j\}}}}tj j r$tt j||z |zn#ttj||z |z} tj j r$tt j||z |zn#ttj||z |z} d} | |k7s| |k7r'tjj|| | fdd}d} | |z} | |z} | | z}|j||z| z|| |}|jdd}|j||||}|jdd}|j||z|d}||f||| || | d }||fS) NFbilinearrmode align_cornersTrr#r r) orig_size batch_sizechannels interpolate num_patchesnum_patches_widthnum_patches_height)rrr%shaperTjit is_tracingrceilrrrrreshaper)rJrLrr patch_arearr orig_height orig_width new_height new_widthrnum_patch_widthnum_patch_heightrpatches info_dicts r' unfoldingzMobileViTLayer.unfoldings$($4$4d6G6G\ |34 8@5 Hk:yy##% ejj|!;<|K LTYY{\9:\IJ yy##% ejjk!9:[H ITYYzK78;FG    "jK&?}}00 I6ZW\1HK${2%5&8 ""  !$4 4lOU` ##Aq)//*h ZP##Aq)//*z"9;K&z2$ &&!0"2  !!r)rrc|j|j}}t||z}|d}|d}|d}|d} |d} |jj |||d} | j dd} | j ||z| z| ||} | j dd } | j ||| |z| |z} |d r&tjj| |d d d } | S)Nrrrrrrrr r#rrrFr) rrr%rrrrrrr) rJrrrrrrrrrrrLs r'foldingzMobileViTLayer.foldings&$($4$4d6G6G\ |34 |, Z( . $%9:#$78%%',,Z[RTU%%a+##  !$4 4o|U` %%a+## "2\"A?U`C`  ] #}}00y5JV[1Hr)c|jr|j|}|}|j|}|j|}|j|\}}|j |}|j |}|j ||}|j|}|jtj||fd}|SNrr) rrrrrrrrrrTcat)rJrLrfrrs r'rOzMobileViTLayer.forwards  " "..x8H==*==*"^^H5""7+..)<<3''1;;uyy(H)=1EFr)rg)rPrQrRrhrr%r@rTrUrrrrrOrVrWs@r'rrts8 8 8  8  8  8 8 8  8 t1"%,,1"5t9K3L1"fu||: r)rc deZdZdeddffd Z d dej dededee e ffdZ xZ S) MobileViTEncoderr,r!Nc t |||_tj|_d|_dx}}|jdk(rd}d}n|jdk(rd}d}t||jd|jddd}|j j|t||jd|jddd }|j j|t||jd|jd d|jdd }|j j||r|dz}t||jd |jd d|jdd | }|j j||r|dz}t||jd |jd d|jdd | } |j j| y)NFrTrr)r-r.r0rkr#r )r-r.r0rxrk)r-r.r0rxrkr3) r?r@r,rrmrngradient_checkpointing output_striderjneck_hidden_sizesrpr hidden_sizes) rJr,dilate_layer_4dilate_layer_5r3layer_1layer_2layer_3layer_4layer_5rKs r'r@zMobileViTEncoder.__init__s  ]]_ &+#+0/   1 $!N!N  ! !R '!N) 00311!4   '") 00311!4   '" 00311!4++A.   '"  MH 00311!4++A.  '"  MH 00311!4++A.  '"r)routput_hidden_states return_dictc|rdnd}t|jD]K\}}|jr)|jr|j |j |}n||}|sF||fz}M|st d||fDSt||S)Nc3&K|] }|| ywrNr).0vs r' z+MobileViTEncoder.forward..}sXq!-Xs)last_hidden_stater) enumeraternr training_gradient_checkpointing_func__call__tupler)rJrrrall_hidden_statesrqrts r'rOzMobileViTEncoder.forwardhs #7BD(4 IOA|**t}} $ A A ))!! !-] ; #$58H$H! IX]4E$FXX X- ]noor))FT) rPrQrRrr@rTrUrSrr$rrOrVrWs@r'rrsaH#H#4H#Z&+ p||p#p p u44 5 pr)rc~eZdZdZeZdZdZdZdgZ de e je je jfddfd Zy) MobileViTPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. mobilevit pixel_valuesTrmoduler!Nct|tjtjfrm|jj j d|jj|j%|jj jyyt|tjrJ|jj j|jj jdyy)zInitialize the weightsg)meanstdNg?) rFrr}rBweightdatanormal_r,initializer_ranger2zero_rfill_)rJr*s r' _init_weightsz&MobileViTPreTrainedModel._init_weightss fryy"))4 5 MM   & &CT[[5R5R & S{{&   &&('  - KK   " " $ MM   $ $S ).r))rPrQrRrhr config_classbase_model_prefixmain_input_namesupports_gradient_checkpointing_no_split_modulesrrr}rBrr4rr)r'r'r'sT #L#$O&*#)* *E"))RYY *L$M *RV *r)r'aK This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`MobileViTConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. aF Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Pixel values can be obtained using [`AutoImageProcessor`]. See [`MobileViTImageProcessor.__call__`] for details. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zWThe bare MobileViT model outputting raw hidden-states without any specific head on top.c eZdZd dedeffd ZdZeee e e e de  d deej deed eed eee ffd ZxZS)MobileViTModelr, expand_outputcLt||||_||_t ||j |j ddd|_t||_ |jr.t ||j d|j dd|_ |jy) Nrr r#)r-r.r/r0r rr\) r?r@r,r<r+ num_channelsr conv_stemrencoder conv_1x1_exp post_init)rJr,r<rKs r'r@zMobileViTModel.__init__s   *+ ++11!4  (/    2"44Q7#55a8 !D  r)c|jD]e\}}|jj|}t|ts0|j jD]}|j j|gy)zPrunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsrArnrFrrrr)rJheads_to_prune layer_indexrmobilevit_layerrs r' _prune_headszMobileViTModel._prune_headsss#1"6"6"8 C K"ll00=O/>:)8)D)D)J)JC%%//;;EBC Cr)vision) checkpoint output_typer5modalityexpected_outputr)rrr!c||n|jj}||n|jj}| td|j |}|j |||}|j r/|j|d}tj|ddgd}n|d}d}|s|||fn|f}||ddzSt|||j S) Nz You have to specify pixel_valuesrrrrrF)rkeepdimr)r pooler_outputr) r,ruse_return_dictrAr@rAr<rBrTr,rr) rJr)rrembedding_outputencoder_outputsr pooled_outputrs r'rOzMobileViTModel.forwards%9$D $++JjJj &1%>,7,, !5#'     $ 1 1/!2D E "JJ'8r2hPUVM / 2  M;H;T'7[lZnFOAB// /7/')77  r))T)NNN)rPrQrRrrSr@rIrMOBILEVIT_INPUTS_DOCSTRINGr_CHECKPOINT_FOR_DOCr_CONFIG_FOR_DOC_EXPECTED_OUTPUT_SHAPErrTrUrr$rOrVrWs@r'r;r;s t4C++EF&<$. 04/3&* ' u||,' 'tn' d^ ' u>> ? ' G' r)r;z MobileViT model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. ceZdZdeddffd Zeeeee e e  d de e jde ede e jd e edeee ff d ZxZS) MobileViTForImageClassificationr,r!Nc|t|||j|_t||_t j |jd|_|jdkDr-t j|jd|jnt j|_ |jy)NT)inplacerr)r?r@ num_labelsr;r(rrclassifier_dropout_probrr}rIdentity classifierrCrJr,rKs r'r@z(MobileViTForImageClassification.__init__s   ++'/zz&"@"@$O JPJ[J[^_J_BIIf..r2F4E4E Fegepeper  r))rKrLr5rNr)rlabelsrc6||n|jj}|j|||}|r |jn|d}|j |j |}d}||jj |jdk(rd|j_nl|jdkDrL|jtjk(s|jtjk(rd|j_nd|j_|jj dk(rIt} |jdk(r&| |j|j}n| ||}n|jj dk(r=t} | |jd|j|jd}n,|jj dk(rt!} | ||}|s|f|ddz} ||f| zS| St#|||j$ S) a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss). If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NrPr regressionsingle_label_classificationmulti_label_classificationrr#)losslogitsr)r,rSr(rRrbr problem_typer_dtyperTlongr%r squeezer rr rr) rJr)rrdroutputsrVrjriloss_fctrs r'rOz'MobileViTForImageClassification.forward%s(&1%* #%"!  r)rLc|jdd}|j|}|j|}tjj ||dd}|S)NrrFr)rrzrrrr)rJrL spatial_sizes r'rOzMobileViTASPPPooling.forwardssS~~bc* ##H-==*==,,XLzin,or)rurWs@r'rurucsA    S  PS  X\   r)rucdeZdZdZdeddffd ZdejdejfdZxZ S) MobileViTASPPzs ASPP module defined in DeepLab papers: https://arxiv.org/abs/1606.00915, https://arxiv.org/abs/1706.05587 r,r!Nc ~t||jd}|j}t |j dk7r t dtj|_ t|||dd}|jj||jj|j Dcgc]}t|||d|dc}t|||}|jj|t|d|z|dd|_tj|j |_ycc}w) Nrr z"Expected 3 values for atrous_ratesrrxr])r-r.r/r3r5r )p)r?r@raspp_out_channelsr atrous_ratesrArrmconvsr+rpextendruprojectraspp_dropout_probr)rJr,r-r. in_projectionrate pool_layerrKs r'r@zMobileViTASPP.__init__s( ..r2 // v"" #q (AB B]]_ * #%!   -( #// # +!- !!#)   *&+|L  *%) L 0|YZkq  zzF$<$<= ) s5D:rLcg}|jD]}|j||tj|d}|j |}|j |}|Sr)rrprTrrr)rJrLpyramidconvpooled_featuress r'rOzMobileViTASPP.forwards\JJ +D NN4> * +))G+,,w/,,7r) rPrQrRrhrr@rTrUrOrVrWs@r'r~r~{s7)>)>4)>V r)r~cdeZdZdZdeddffd ZdejdejfdZxZ S)MobileViTDeepLabV3zB DeepLabv3 architecture: https://arxiv.org/abs/1706.05587 r,r!Nc t|t||_t j |j |_t||j|jdddd|_ y)NrFT)r-r.r/r4r5r2) r?r@r~asppr Dropout2dr`rr+rr_rbrcs r'r@zMobileViTDeepLabV3.__init__s] !&) ||F$B$BC , 00**#  r)rcr|j|d}|j|}|j|}|S)Nr)rrrb)rJrrLs r'rOzMobileViTDeepLabV3.forwards699]2./<<)??8,r)rrWs@r'rrs6  4 U\\ellr)rzX MobileViT model with a semantic segmentation head on top, e.g. for Pascal VOC. ceZdZdeddffd Zeeeee  d de e jde e jde e d e e deeeff d ZxZS) MobileViTForSemanticSegmentationr,r!Nct|||j|_t|d|_t ||_|jy)NF)r<)r?r@r_r;r(rsegmentation_headrCrcs r'r@z)MobileViTForSemanticSegmentation.__init__sD   ++'eD!3F!; r))rLr5r)rdrrch||n|jj}||n|jj}|$|jjdk(r t d|j |d|}|r |j n|d}|j|}d}|Ytjj||jdddd } t|jj } | | |}|s|r |f|ddz} n |f|d dz} ||f| zS| St|||r|j d Sdd S) a labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth semantic segmentation maps for computing the loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1`, a classification loss is computed (Cross-Entropy). Returns: Examples: ```python >>> import requests >>> import torch >>> from PIL import Image >>> from transformers import AutoImageProcessor, MobileViTForSemanticSegmentation >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("apple/deeplabv3-mobilevit-small") >>> model = MobileViTForSemanticSegmentation.from_pretrained("apple/deeplabv3-mobilevit-small") >>> inputs = image_processor(images=image, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> # logits are of shape (batch_size, num_labels, height, width) >>> logits = outputs.logits ```Nrz/The number of labels should be greater than oneTrPrrFr) ignore_indexr#)rirjr attentions)r,rrSr_rAr(rrrrrrr semantic_loss_ignore_indexr) rJr)rdrrroencoder_hidden_statesrjriupsampled_logitsrprs r'rOz(MobileViTForSemanticSegmentation.forwardsqN%9$D $++JjJj &1%  !}}88V\\"#.Zu 9  (T[[5[5[\H,f5D# WQR[0 WQR[0)-)9TGf$ Ev E&3G'//   NR   r)rq)rPrQrRrr@rrWrrrYrrTrUrSrr$rOrVrWs@r'rrs4++EF+BQ`a04)-/3&* K u||,K &K 'tn K d^ K u-- . K bGK r)r)rN)DrhrtypingrrrrrrTtorch.utils.checkpointrtorch.nnr r r activationsr modeling_outputsrrrrmodeling_utilsr pytorch_utilsrrutilsrrrrrrconfiguration_mobilevitr get_loggerrPloggerrYrXrZrrrsr%r(Moduler+rYrjrwrrrrrrrrr'MOBILEVIT_START_DOCSTRINGrWr;r\rur~rrrr)r'rsE" 44 AA! .Q5   H %$.'21 #  HSM UX ==@-F -F`bii.0RYY0f "))   > BII  bii  &299&fRYYfRbpryybpJ**2  ]T -T T n K &>K K \29908BII8v8  X '?X  X r)