sg@5ddlmZmZmZddlmZddlZddlmcm Z ddlmZGddeZ GddejZ d ejd ejd efd ZGd dej"ZGddej"ZGddej"ZGddej"ZGddej"ZGddej"ZGddej"ZGddej"ZGddej"ZGdd ej"ZGd!d"ej"ZGd#d$ej"ZGd%d&ej"Zy)')DictAnyTuple)BaseSparsifierN)nncheZdZdeeefddffd ZdejdedeeefddfdZ xZ S)ImplementedSparsifierkwargsreturnNc &t||y)N)defaults)super__init__)selfr __class__s Y/var/www/html/venv/lib/python3.12/site-packages/torch/testing/_internal/common_pruning.pyrzImplementedSparsifier.__init__ s &)module tensor_namec d|jjdjd<|jd}|j dddz|d<y)Nrzlinear1.weight step_count)parametrizationsweightmaskstateget)rrrr linear_states r update_maskz!ImplementedSparsifier.update_mask sN45&&q)..q1zz"23 %1%5%5lA%F%J \"r) __name__ __module__ __qualname__rstrrrrModuler __classcell__rs@rr r sR*c3h*D*K"))K#KcSVhK\`Krr c>eZdZdZedej ddfdZy)MockSparseLinearz This class is a MockSparseLinear class to check convert functionality. It is the same as a normal Linear layer, except with a different type, as well as an additional from_dense method. modr c@||j|j}|S)z ) in_features out_features)clsr)linears r from_densezMockSparseLinear.from_denses"S__%%' rN)r r!r"__doc__ classmethodrLinearr/rrr(r(s. RYY+=rr( subset_tensorsuperset_tensorr cd}|D]@}|t|kr/tj|||s|dz }n1|t|kr/yy)zW Checks to see if all rows in subset tensor are present in the superset tensor rrFT)lentorchequal)r4r5irows rrows_are_subsetr<"s] A#o&&;;sOA$67Q #o&&  rc\eZdZdZdfd Zdej dej fdZxZS) SimpleLinearzModel with only Linear layers without biases, some wrapped in a Sequential, some following the Sequential. Used to test basic pruned Linear-Linear fusion.r c Rt|tjtjdddtjdddtjddd|_tjddd|_tjddd|_y)NFbias )rrr Sequentialr2seqlinear1linear2rrs rrzSimpleLinear.__init__6sx == IIa ' IIa ' IIa '  yyAE2 yyBU3 rxcl|j|}|j|}|j|}|SN)rHrIrJrrLs rforwardzSimpleLinear.forward@. HHQK LLO LLOrr N r r!r"r0rr8TensorrPr%r&s@rr>r>2s)V4%,,rr>c\eZdZdZdfd Zdej dej fdZxZS) LinearBiaszModel with only Linear layers, alternating layers with biases, wrapped in a Sequential. Used to test pruned Linear-Bias-Linear fusion.r c :t|tjtjdddtjdddtjdddtjdddtjddd|_y) Nr@rATrBrDFrF)rrrrGr2rHrKs rrzLinearBias.__init__Ksn == IIa & IIa ' IIa & IIa & IIa% (  rrLc(|j|}|SrN)rHrOs rrPzLinearBias.forwardUs HHQKrrRrSr&s@rrVrVGs)O %,,rrVc\eZdZdZdfd Zdej dej fdZxZS)LinearActivationzModel with only Linear layers, some with bias, some in a Sequential and some following. Activation functions modules in between each Linear in the Sequential, and each outside layer. Used to test pruned Linear(Bias)-Activation-Linear fusion.r c t|tjtjdddtj tjdddtj tjddd|_tjddd|_tj |_ tjdd d|_ tj |_ y) Nr@rATrBrDFrErXrF) rrrrGr2ReLUTanhrHrIact1rJact2rKs rrzLinearActivation.__init___s == IIa & GGI IIa ' GGI IIa &  yyAD1 GGI yyBU3 GGI rrLc|j|}|j|}|j|}|j|}|j |}|SrN)rHrIr_rJr`rOs rrPzLinearActivation.forwardmH HHQK LLO IIaL LLO IIaLrrRrSr&s@rr[r[Zs)B %,,rr[c\eZdZdZdfd Zdej dej fdZxZS)LinearActivationFunctionala,Model with only Linear layers, some with bias, some in a Sequential and some following. Activation functions modules in between each Linear in the Sequential, and functional activationals are called in between each outside layer. Used to test pruned Linear(Bias)-Activation-Linear fusion.r c  t|tjtjdddtj tjdddtj tjddd|_tjddd|_tjdd d|_tjd d d|_ tj |_ y) Nr@rATrBrDFrErXrF) rrrrGr2r]rHrIrJlinear3r_rKs rrz#LinearActivationFunctional.__init__|s == IIa & GGI IIa ' GGI IIa &  yyAD1 yyAE2 yyBU3 GGI rrLc |j|}|j|}tj|}|j |}tj|}|j |}tj|}|SrN)rHrIFrelurJrgrOs rrPz"LinearActivationFunctional.forwardsb HHQK LLO FF1I LLO FF1I LLO FF1IrrRrSr&s@rrdrdvs)B %,,rrdc\eZdZdZdfd Zdej dej fdZxZS) SimpleConv2dzModel with only Conv2d layers, all without bias, some in a Sequential and some following. Used to test pruned Conv2d-Conv2d fusion.r c 4t|tjtjdddddtjddddd|_tjddddd|_tjddddd|_y) Nr rXFrB@04rrrrGConv2drHconv2d1conv2d2rKs rrzSimpleConv2d.__init__sx == IIaQ . IIb"a / yyRAE: yyRAE: rrLcl|j|}|j|}|j|}|SrNrHrtrurOs rrPzSimpleConv2d.forwardrQrrRrSr&s@rrlrls(1;%,,rrlc\eZdZdZdfd Zdej dej fdZxZS) Conv2dBiaszModel with only Conv2d layers, some with bias, some in a Sequential and some outside. Used to test pruned Conv2d-Bias-Conv2d fusion.r c ft|tjtjdddddtjdddddtjddddd|_tjddddd|_tjdd ddd|_y NrrnrXTrBroFrprqrrrKs rrzConv2dBias.__init__s == IIaQ - IIb"a . IIb"a /  yyRAD9 yyRAE: rrLcl|j|}|j|}|j|}|SrNrwrOs rrPzConv2dBias.forwardrQrrRrSr&s@rryrys(6;%,,rryc\eZdZdZdfd Zdej dej fdZxZS)Conv2dActivationaModel with only Conv2d layers, some with bias, some in a Sequential and some following. Activation function modules in between each Sequential layer, functional activations called in-between each outside layer. Used to test pruned Conv2d-Bias-Activation-Conv2d fusion.r c t|tjtjdddddtj tjdddddtj tjdddddtj |_tjddddd|_tjdd ddd|_ yr{) rrrrGrsr]r^rHrtrurKs rrzConv2dActivation.__init__s == IIaQ - GGI IIb"a . GGI IIb"a / GGI  yyRAE: yyRAD9 rrLc|j|}|j|}tj|}|j |}tj |}|SrN)rHrtrirjruhardtanhrOs rrPzConv2dActivation.forwardsH HHQK LLO FF1I LLO JJqMrrRrSr&s@rr~r~s)A :%,,rr~c\eZdZdZdfd Zdej dej fdZxZS) Conv2dPadBiasaQModel with only Conv2d layers, all with bias and some with padding > 0, some in a Sequential and some following. Activation function modules in between each layer. Used to test that bias is propagated correctly in the special case of pruned Conv2d-Bias-(Activation)Conv2d fusion, when the second Conv2d layer has padding > 0.r ct|tjtjddddddtj tjdddddtj tjddddddtj tjddddddtj tjdddddtj |_tjdd dddd|_tj |_ tjd d dddd|_ tj |_ y) NrrnrXT)paddingrCFrBrorprq) rrrrGrsr]r^rHrtr_rur`rKs rrzConv2dPadBias.__init__s == IIaQ14 8 GGI IIb"a / GGI IIb"aAD 9 GGI IIb"aAD 9 GGI IIb"a . GGI  yyRAqtD GGI yyRAqtD GGI rrLc|j|}|j|}|j|}|j|}|j |}|SrN)rHrtr_rur`rOs rrPzConv2dPadBias.forwardrbrrRrSr&s@rrrs)c &%,,rrc\eZdZdZdfd Zdej dej fdZxZS) Conv2dPoolzModel with only Conv2d layers, all with bias, some in a Sequential and some following. Activation function modules in between each layer, Pool2d modules in between each layer. Used to test pruned Conv2d-Pool2d-Conv2d fusion.r c t|tjtjdddddtj dddtj tjdddddtjtjddd|_ tjdd ddd|_ tj ddd|_ tj |_ tjd d ddd|_ tjd d ddd|_y) NrrnrXT kernel_sizerrCrstriderrorprq)rrrrGrs MaxPool2dr]r^ AvgPool2drHrtmaxpoolaf1ruconv2d3rKs rrzConv2dPool.__init__s == IIaAD A LLQq! < GGI IIb"!QT B GGI LLQq! <  yyRQM ||!QG 779yyRQM yyRQM rrLc.|j|}|j|}|j|}|j|}|j |}t j |ddd}t j|}|j|}|SNrrr) rHrtrrruri avg_pool2drjrrOs rrPzConv2dPool.forwardsv HHQK LLO LLO HHQK LLO LL!Q ? FF1I LLOrrRrSr&s@rrrs)8N  %,, rrc\eZdZdZdfd Zdej dej fdZxZS)Conv2dPoolFlattenFunctionalaModel with Conv2d layers, all with bias, some in a Sequential and some following, and then a Pool2d and a functional Flatten followed by a Linear layer. Activation functions and Pool2ds in between each layer also. Used to test pruned Conv2d-Pool2d-Flatten-Linear fusion.r c |t|tjtjdddddtj dddtj tjdddddtjtjddd|_ tjddddd|_ tj |_ tjdd ddd|_ tjd |_tjd d d |_y) NrrXTrrrrAr@ )rr rB)rrrrGrsrr]r^rrHrtrruAdaptiveAvgPool2davg_poolr2fcrKs rrz$Conv2dPoolFlattenFunctional.__init__#s == IIa14 @ LLQq! < GGI IIa14 @ GGI LLQq! <  yyA1adK 779yyBAqtL ,,V4 ))B.rrLc0|j|}|j|}tj|ddd}|j |}|j |}|j |}tj|d}|j|}|Sr) rHrtri max_pool2drrurr8flattenrrOs rrPz#Conv2dPoolFlattenFunctional.forward3sz HHQK LLO LL!Q ? HHQK LLO MM!  MM!Q  GGAJrrRrSr&s@rrrs)@ /  %,, rrc\eZdZdZdfd Zdej dej fdZxZS)Conv2dPoolFlattenaModel with Conv2d layers, all with bias, some in a Sequential and some following, and then a Pool2d and a Flatten module followed by a Linear layer. Activation functions and Pool2ds in between each layer also. Used to test pruned Conv2d-Pool2d-Flatten-Linear fusion.r c t|tjtjdddddtj dddtj tjdddddtjtjddd|_ tjddddd|_ tj |_ tjdd ddd|_ tjd |_tj|_tj"d d d |_y)NrrXTrrrrAr@r)rr,rrB)rrrrGrsrr]r^rrHrtrrurrFlattenrr2rrKs rrzConv2dPoolFlatten.__init__Es == IIa14 @ LLQq! < GGI IIa14 @ GGI LLQq! <  yyA1adK 779yyBAqtL ,,V4 zz| ))B.rrLc&|j|}|j|}tj|ddd}|j |}|j |}|j |}|j|}|j|}|Sr) rHrtrirrrurrrrOs rrPzConv2dPoolFlatten.forwardVsw HHQK LLO LL!Q ? HHQK LLO MM!  LLO GGAJrrRrSr&s@rrr?s)@ /"  %,, rrc eZdZdZdededededdf fd Zd ejdeejejffd Z xZ S) LSTMLinearModelzCContainer module with an encoder, a recurrent module, and a linear. input_dim hidden_dim output_dim num_layersr Nct|tj||||_tj |||_yrN)rrrLSTMlstmr2r.rrrrrrs rrzLSTMLinearModel.__init__es6 GGIz:> ii J7 rinputcT|j|\}}|j|}||fSrN)rr.)rroutputhiddendecodeds rrPzLSTMLinearModel.forwardls,5)++f%r r r!r"r0intrr8rTrrPr%r&s@rrrbs\M88*-8;>8LO8 8U\\eELL%,,4N.Orrc eZdZdZdededededdf fd Zd ejdeejejffd Z xZ S) LSTMLayerNormLinearModelz9Container module with an LSTM, a LayerNorm, and a linear.rrrrr Nct|tj||||_tj ||_tj|||_yrN) rrrrr LayerNormnormr2r.rs rrz!LSTMLayerNormLinearModel.__init__usF GGIz:> LL, ii J7 rrLcv|j|\}}|j|}|j|}||fSrN)rrr.)rrLrs rrPz LSTMLayerNormLinearModel.forward}s699Q<5 IIaL KKN%xrrr&s@rrrrs\C88*-8;>8LO8 8% ell0J*Krr)typingrrrtorch.ao.pruningrr8torch.nn.functionalr functionalrir r2r(rTboolr<r$r>rVr[rdrlryr~rrrrrrr3rrrs1$#+ KNK ryy  5<< %,, SW  299*&ryy8>299(*ryy8BIIDB"))D  F bii  ryyr