sgBdZddlmZmZmZmZddlZddlmZddlm Z ddl m Z ddl m Z dd lmZmZmZdd lmZd d lmZd ZGddej.ZGddej.ZGddej.ZGddej.ZGddej.ZGdde ZdZdZedeGddeZ y)zrPyTorch UperNet model. Based on OpenMMLab's implementation, found in https://github.com/open-mmlab/mmsegmentation.)ListOptionalTupleUnionN)nn)CrossEntropyLoss)SemanticSegmenterOutput)PreTrainedModel)add_start_docstrings%add_start_docstrings_to_model_forwardreplace_return_docstrings) load_backbone) UperNetConfigrceZdZdZ d dededeeeeeffdeeeeefefdedeeeeeffdd ffd Z d e jde jfd Z xZ S)UperNetConvModulez A convolutional block that bundles conv/norm/activation layers. This block simplifies the usage of convolution layers, which are commonly used with a norm layer (e.g., BatchNorm) and activation layer (e.g., ReLU). in_channels out_channels kernel_sizepaddingbiasdilationreturnNct|tj|||||||_tj ||_tj|_y)N)rrrrrr) super__init__rConv2dconv BatchNorm2d batch_normReLU activation)selfrrrrrr __class__s _/var/www/html/venv/lib/python3.12/site-packages/transformers/models/upernet/modeling_upernet.pyrzUperNetConvModule.__init__(sQ II#%#   ..6'')inputcl|j|}|j|}|j|}|SN)rr!r#)r$r(outputs r&forwardzUperNetConvModule.forward=s15!(( r')rFr)__name__ __module__ __qualname____doc__intrrstrboolrtorchTensorr, __classcell__r%s@r&rr"s5601$$$3c3h/0 $ sE#s(OS01 $  $U38_,-$ $*U\\ellr'rcheZdZdedededdffd Zdej dej fdZxZS) UperNetPyramidPoolingBlock pool_scalerchannelsrNct|tj|t ||dg|_t |j D] \}}|jt||"y)Nrr) rrrAdaptiveAvgPool2drlayers enumerate add_moduler2)r$r:rr;ilayerr%s r&rz#UperNetPyramidPoolingBlock.__init__Fsa    , k8 C  "$++. +HAu OOCFE * +r'r(c<|}|jD] }||} |Sr*)r?)r$r( hidden_staterCs r&r,z"UperNetPyramidPoolingBlock.forwardOs* [[ /E .L /r') r-r.r/r1rr4r5r,r6r7s@r&r9r9Es?+3+S+C+D+U\\ellr'r9c eZdZdZdeedfdedededdf fd Zd ejde ejfd Z xZ S) UperNetPyramidPoolingModulea} Pyramid Pooling Module (PPM) used in PSPNet. Args: pool_scales (`Tuple[int]`): Pooling scales used in Pooling Pyramid Module. in_channels (`int`): Input channels. channels (`int`): Channels after modules, before conv_seg. align_corners (`bool`): align_corners argument of F.interpolate. pool_scales.rr; align_cornersrNct|||_||_||_||_g|_t|D]I\}}t|||}|j j||jt||Ky)N)r:rr;) rrrHrIrr;blocksr@r9appendrAr2) r$rHrr;rIrBr:blockr%s r&rz$UperNetPyramidPoolingModule.__init__es &*&   &{3 +MAz.*R]hpqE KK  u % OOCFE * +r'xcg}|jD]Y}||}tjj||j ddd|j }|j |[|S)NbilinearsizemoderI)rKr functional interpolaterSrIrL)r$rNppm_outsppmppm_outupsampled_ppm_outs r&r,z#UperNetPyramidPoolingModule.forwardqsn;; /C!fG " 9 9affhqrl4K]K]!:!  OO- .  / r') r-r.r/r0rr1r3rr4r5rr,r6r7s@r&rGrGVs[  +E#s(O +# +QT +ei +nr +$u||*<r'rGcleZdZdZfdZdZdZdZdejdejfdZ xZ S) UperNetHeadz Unified Perceptual Parsing for Scene Understanding. This head is the implementation of [UPerNet](https://arxiv.org/abs/1807.10221). ct|||_|j|_||_|j |_d|_tj|j |jd|_ t|j|jd|j |j|_ t|jdt|j|j zz|j dd|_tj"|_tj"|_|jddD]s}t||j d}t|j |j dd}|j$j)||j&j)|utt|j|j z|j dd|_y)NFrr=)rIr rr)rrconfigrHr hidden_sizer;rIrr num_labels classifierrG psp_modulesrlen bottleneck ModuleList lateral_convs fpn_convsrLfpn_bottleneck)r$r`rl_convfpn_convr%s r&rzUperNetHead.__init__s  !--&** "))DMM63D3DRST7      R MM,,   ,   R 3t'7'7#84==#H H MM   ]]_++CR0 ,K&{DMMqQF( ST^_`H    % %f - NN ! !( +  , 0   !DMM 1 MM  r'c:|j|jyr*apply _init_weightsr$s r& init_weightszUperNetHead.init_weights 4%%&r'ct|tjrm|jjj d|j j|j%|jjjyyyNg)meanstd isinstancerrweightdatanormal_r`initializer_rangerzero_r$modules r&rpzUperNetHead._init_weightsa fbii ( MM   & &CT[[5R5R & S{{&   &&(' )r'c|d}|g}|j|j|tj|d}|j |}|S)Nr^rdim)extendrdr4catrf)r$inputsrNpsp_outsr+s r& psp_forwardzUperNetHead.psp_forwardsL 2J3((+,99X1-* r'encoder_hidden_statesrcPt|jDcgc]\}}|||}}}|j|j|t |}t |dz ddD]V}||dz j dd}||dz tjj|||d|jz||dz <Xt |dz Dcgc]}|j|||}}|j|dt |dz ddD]E}tjj|||dj ddd|j||<Gtj|d}|j|}|j|}|Scc}}wcc}w)Nrrr^rPrQrRr)r@rhrLrrerangeshaperrUrVrIrir4rrjrc) r$rrB lateral_convlateralsused_backbone_levels prev_shapefpn_outsr+s r&r,zUperNetHead.forwardsR[\`\n\nRopq,L!6q!9:pp(()>?@ #8}+a/B7 A!!a%..qr2J&q1uo 0I0I *:TM_M_1J1HQUO =BBVYZBZ<[\q%DNN1%hqk2\\ %+a/B7 A--33 (1+"3"3AB"7jX\XjXj4HQK 99X1-$$X.( 3q]s FF#) r-r.r/r0rrrrprr4r5r,r6r7s@r&r\r\|s8 % N') U\\ellr'r\c eZdZdZ d dededeeeeeffddffd ZdZd Z d e jde jfd Z xZ S) UperNetFCNHeada Fully Convolution Networks for Semantic Segmentation. This head is the implementation of [FCNNet](https://arxiv.org/abs/1411.4038>). Args: config: Configuration. in_channels (int): Number of input channels. kernel_size (int): The kernel size for convs in the head. Default: 3. dilation (int): The dilation rate for convs in the head. Default: 1. in_indexrrrNc Jt|||_|j|_|j |_|j|_|j|_ ||_ |dz|z}g}|jt|j|j |||t|jdz D]5}|jt|j |j |||7|jdk(rtj |_ntj$||_|jr8t|j|j z|j ||dz|_tj(|j |j*d|_y)NrP)rrrrrr_r=)rrr`auxiliary_in_channelsrauxiliary_channelsr;auxiliary_num_convs num_convsauxiliary_concat_input concat_inputrrLrrrIdentityconvs Sequentialconv_catrrbrc) r$r`rrr conv_paddingrrBr%s r&rzUperNetFCNHead.__init__s_  !7711 33"99  #q(H4    $--[R^iq  t~~)* A LL!MM4==kS_jr   >>Q DJ.DJ   -  4==0$--[bmqrbrDM))DMM63D3DRSTr'c:|j|jyr*rnrqs r&rrzUperNetFCNHead.init_weights rsr'ct|tjrm|jjj d|j j|j%|jjjyyyrurxrs r&rpzUperNetFCNHead._init_weightsrr'rc||j}|j|}|jr(|jt j ||gd}|j |}|S)Nrr)rrrrr4rrc)r$r hidden_statesr+s r&r,zUperNetFCNHead.forwardsX-dmm< M*   ]]599mV-D!#LMF( r')rPr r)r-r.r/r0r1rrrrrrpr4r5r,r6r7s@r&rrsv  hi"U #"U69"UINsTYZ]_bZbTcOcId"U "UH') U\\ellr'rc(eZdZdZeZdZgZdZdZ y)UperNetPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. pixel_valuesct|tr\|jj|jj|j |j jyyyr*)ryrbackbonerr decode_headauxiliary_headrs r&rpz$UperNetPreTrainedModel._init_weights*sW f4 5 OO ( ( *    + + -$$0%%2241 6r'c|jj|jj|j|jjyy)zInitialize the weightsN)rrrrrrqs r&rrz#UperNetPreTrainedModel.init_weights1sG ""$ %%'    *    , , . +r'N) r-r.r/r0r config_classmain_input_name_no_split_modulesrprrr'r&rr s# !L$O5/r'raI Parameters: This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) sub-class. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. config ([`UperNetConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. ax Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Padding will be ignored by default should you provide it. Pixel values can be obtained using [`AutoImageProcessor`]. See [`SegformerImageProcessor.__call__`] for details. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers in case the backbone has them. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers of the backbone. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zMUperNet framework leveraging any vision backbone e.g. for ADE20k, CityScapes.ceZdZfdZeej deee  d de e jde e de e de e jde e d eeeff d ZxZS) UperNetForSemanticSegmentationct||t||_t ||jj |_|jr t|nd|_ |jy)N)r) rrrrr\r;ruse_auxiliary_headrr post_init)r$r`r%s r&rz'UperNetForSemanticSegmentation.__init__Xs[  %f- 'v4==;Q;QR8>8Q8QnV4W[ r'zbatch_size, sequence_length) output_typerroutput_attentionsoutput_hidden_stateslabels return_dictrc|$|jjdk(r td||n|jj}||n|jj}||n|jj }|j j|||}|j}|j|}tjj||jdddd}d} |jA|j|} tjj| |jdddd} d} |Pt|jj } | ||} | %| | |} | |jj"| zz } |s|r |f|ddz} n |f|ddz} | | f| zS| St%| ||j&|j( S) a labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth semantic segmentation maps for computing the loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1`, a classification loss is computed (Cross-Entropy). Returns: Examples: ```python >>> from transformers import AutoImageProcessor, UperNetForSemanticSegmentation >>> from PIL import Image >>> from huggingface_hub import hf_hub_download >>> image_processor = AutoImageProcessor.from_pretrained("openmmlab/upernet-convnext-tiny") >>> model = UperNetForSemanticSegmentation.from_pretrained("openmmlab/upernet-convnext-tiny") >>> filepath = hf_hub_download( ... repo_id="hf-internal-testing/fixtures_ade20k", filename="ADE_val_00000001.jpg", repo_type="dataset" ... ) >>> image = Image.open(filepath).convert("RGB") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits # shape (batch_size, num_labels, height, width) >>> list(logits.shape) [1, 150, 512, 512] ```Nrz/The number of labels should be greater than one)rrrPrQFrR) ignore_index)losslogitsr attentions)r`rb ValueErroruse_return_dictrrrforward_with_filtered_kwargs feature_mapsrrrUrVrrrloss_ignore_indexauxiliary_loss_weightr rr)r$rrrrroutputsfeaturesrauxiliary_logitsrloss_fctauxiliary_lossr+s r&r,z&UperNetForSemanticSegmentation.forwarddsN  $++"8"8A"=NO O%0%R 'tn R & R d^ R  u-- .R bkR r'r)!r0typingrrrrr4rtorch.nnrmodeling_outputsr modeling_utilsr utilsr r rutils.backbone_utilsrconfiguration_upernetrrModulerr9rGr\rrUPERNET_START_DOCSTRINGrrrr'r&rsy// %7-kk10"  F"#"))#LZ"))ZzDRYYDN/_/2  Wa %;a a r'