sg5A ddlmZddlmZmZddlmZddlmZm Z m Z m Z m Z m Z mZmZddlZddlmZddlmcmZddlmZddlZddlmZddlmcmZddlZddl Z ddl!Z!ddl"m#Z#de$d e e$e$ffd Z%d e ed ejLd e e$e ffdZ'd ejLd e e$e fdej$jPfdZ)d'dej$jPd ej$jPfdZ*dejPd ejPfdZ+dejPdeejLdeejLdeejLfdZ,ejZej\ej^ej`ejbejdejfejhejjejlejhejnejpg Z9ejtejvgZ (`foo.bar`, `baz`) .r)rsplit)rparentnames U/var/www/html/venv/lib/python3.12/site-packages/torch/fx/experimental/optimization.py _parent_namers3 MM#q)MVT6!9 ,,B ,,patternnodemodulescjt|jdk(ry|jd|f}t||D]z\}}t|tj sy|j dk7ryt|jtsy|j|vryt||j|uszyy)NrF call_moduleT) lenargszip isinstancefxNodeoprstrtype)rrrnodes expected_type current_nodes rmatches_module_patternr.s 499~"&))A,!5E'*7E': # |,0 ??m +,--s3   g -  ++, -] B  r new_modulect|jtsJt|j\}}|||j<t ||||yN)r%rr)rsetattr)rrr/ parent_namers rreplace_node_moduler4-sH dkk3 '' '$T[[1K%GDKK GK $ 3rmodelctjtjftjtjftj tj fg}|stj|}|r$t|tjjstj|}n|}t|j}tj|j }|D]}|j"D]}t%|||st'|j(dj*dkDr7||j(dj,} ||j,} | j.sot1| | } t3|j(d|| |j5|j(d|j7|tj||S)z Fuses convolution/BN layers for inference purposes. Will deepcopy your model by default, but can modify the model inplace as well. rr)nnConv1d BatchNorm1dConv2d BatchNorm2dConv3d BatchNorm3dcopydeepcopyr%torchr& GraphModulesymbolic_tracedict named_modulesgraphr+r.r"r#usersrtrack_running_statsrr4replace_all_uses_with erase_node) r5inplaceno_tracepatternsfx_modelr new_graphrrconvbn fused_convs rfuserR3ss BNN+BNN+BNN+-H  e$ :eUXX-A-AB$$U+8))+,G hnn-I +OO +D%gtW=tyy|))*Q.tyy|223T[[)--.tR8 #DIIaL':F**499Q<8$$T* + + >>(I ..rctj|}Gddtjj}||j S)z5 Removes all dropout layers from the module. cDeZdZdedeedfdeeefdeffd Z xZ S)&remove_dropout..DropoutRemoverrr#.kwargsrct|j|tjrt |dk(sJ|dSt ||||S)Nrr)r% submodulesr7Dropoutr"superr!)selfrr#rV __class__s rr!z2remove_dropout..DropoutRemover.call_moduleZsG$//&12::>4yA~%~Aww*64@@r) __name__ __module__ __qualname__rr rrr)rr! __classcell__)r\s@rDropoutRemoverrUYsF Av AeHcM6J AUYZ]_bZbUc Ahk A Arra)r&rBr@ Transformer transform)r5rMras rremove_dropoutrdSsB  'HA--A ( # - - //r orig_moduler+inputsoutputscZ tj}i |D]"}|j|j}| |<$|D]}|j | fd}| |<|j |Dcgc]} | c}|j tj||Scc}w)z Given lists of nodes from an existing graph that represent a subgraph, returns a submodule that executes that subgraph. c|Sr1)xenvs rz"extract_subgraph..ls s1vr)r&Graph placeholderr node_copyoutputlintrA) rer+rfrgrNinputnew_noderrqrls @rextract_subgraphrubs I"$C((4E &&t-=>D 8fc&k89 NN >>+y 119s/ B(c,tj|Sr1) th_mkldnnMkldnnBatchNorm)a_s rrmrms!:!:1!=rci}|D]}|jdk(st|jtsJ||j}t |t vsPt t ||t j}t|tjsJtj|||<t||||S)z For each node, if it's a module that can be preconverted into MKLDNN, then we do so and create a mapping to allow us to convert from the MKLDNN version of the module to the original. r!) r(r%rr)r* mkldnn_mapr@floatr7Moduler>r?r4)r+r old_modulesr cur_moduler/s rmodules_to_mkldnnrs /1K? 77m #dkk3/ // -JJ:-'Z(89*ekkR !*bii888*.-- *C J'#D':>? rrc|D]R}|jdk(st|jtsJ||j}||vsCt ||||Ty)za Maps each module that's been changed with `modules_to_mkldnn` back to its original. r!N)r(r%rr)r4)r+rrrrs r reset_modulesrs\ L 77m #t{{C0 10 -J[(#D';z3JK Lrc,eZdZdejfdZy) MklSubgraphfx_graphc<||_g|_g|_g|_yr1)rr+ start_nodes end_nodes)r[rs r__init__zMklSubgraph.__init__s  $& *,(*rN)r]r^r_r&rnrrjrrrrs++rrcDdddtdtffd }|S)aW This generates a heuristic that can be passed into `optimize_for_inference` that determines whether a subgraph should be run in MKL by running it with the example_inputs. Example usage: heuristic = gen_mkl_autotuner(example_inputs, iters=10) fast_model = optimization.optimize_for_inference(model, heuristic) NrErc|j} F|jj |jj t j |Dcgc]!}t j|j#c}tttj|jDcgc]}|jdc}}t |j || fd}|fd}t#j$j t'j) |fd}||kScc}wcc}w)NrctD] }| tj}tD] }|} tj|z Sr1)rangetime)frzbeginoutiterswarmups r benchmarkz?gen_mkl_autotuner..use_mkl_heuristic..benchmarksQ6]  IIKE5\ c 99;& &rcDcgc]}|jc}Dcgc]}|jc}Scc}wcc}wr1) to_mkldnnto_dense)i sample_inputs submodules rrmz>gen_mkl_autotuner..use_mkl_heuristic..s6I_lGmZ[ Gmgen_mkl_autotuner..use_mkl_heuristic..s =(Ar)rr owning_modulerr propagater@randnshaper r r&r'rr#rur+rrErCrD)rE input_nodesr output_argsrmkl_time no_mkl_timerrexample_inputsrMrrrs @@ruse_mkl_heuristicz,gen_mkl_autotuner..use_mkl_heuristics''  ~~33H..44K h  ) ). 9=HITTZZ0I 4=EOO*TD499Q<*TU $Xu{{KU  'opioo++T)2I2I2K-LkZ AB +%%!J*Ts &D>.E )rbool)rrrrrMrs``` @@rgen_mkl_autotunerrs/HK&&&&0 rrEc2t|jdkDS)z This is a heuristic that can be passed into `optimize_for_inference` that determines whether a subgraph should be run in MKL by checking if there are more than 2 nodes in it )r"r+)rEs ruse_mkl_lengthrs u{{ a rc>eZdZdZdefdZdedefdZdedefdZy ) UnionFindc0dg|z|_dg|z|_y)Nrrsize)r[ns rrzUnionFind.__init__s,06A: !sQw rvc@||j|<d|j|<y)Nrr)r[rs rmake_setzUnionFind.make_sets A ! rrc|j|}||k(r|S|J|j||j|<tt|j|Sr1)rfindr int)r[rpars rrzUnionFind.findsOkk!n 8H3 ACQ((rrybc|j||j|}}||k(r|S|j||j|kr||}}||j|<|j|xx|j|z cc<yr1)rrr)r[ryrs rjoinzUnionFind.joinsoyy|TYYq\1 6H 99Q<$))A, &aqA A !  ! $ rN)r]r^r_rrrrrrjrrrrs9'#)c)c)%c%c%rr pass_configtracercdddtid}|i}|j||dr t|}|dr t|}|ddur|St |dt s t d d|dvr t d |dd}|}|jtj|tj|j}t |j}Gd d t}tj D]} |j"} | j$d k(r|| j&} t)| t*vr|j,} t/| j1d} | | j2t4j6k(sJd| j8t5j8dk(sSJd| j$dk(r=| j&t*vr |j,} n| j&t:vr |j<} | |j"k7s| |j<k(rt?d| j@Ds>jC| 5tjD| j@fd} dddtGtHtjJjL | _ jO| 5jQdd| f}| jS|| f|_ dddtUtj |}|_+j D]} | j$dk(s| j&dk(s#| j@d}t| jX}|D]D}|j$dk(s|j&dk(s#|jS|j[|Ft]| jXdk(sj[| t]j }t_|fd}taj D]'\}} | j$dk(r(| j&dk(r|| _1je|>| j$dk(rA| j&dk(r2|| j@dJ|| j@d| _3| jhDcgc].}t |tjjs||'||0}}t]|dk(rt?d|DrJtm|}|d| _7|ddD]}jq|d|*tsfd}j D]} tu| dr7|jw| jnj jy| tu| dr7|jw| jbjzjy| tu| ds|jw| jfj|jy| |jD]q}||r |jz|j|zD]3} | j@d}| jS|j[| 5t|j ||sd}j D]&} | j&dk(s| j&dk(s"|dz }(tjtjd|jtj|}|S#1swYxYw#1swYTxYwcc}w) a Performs a set of optimization passes to optimize a model for the purposes of inference. Specifically, the passes that are run are: 1. Conv/BN fusion 2. Dropout removal 3. MKL layout optimizations The third optimization takes a function `use_mkl_heuristic` that's used to determine whether a subgraph should be explicitly run in MKL layout. Note: As FX does not currently handle aliasing, this pass currently assumes nothing aliases. If that isn't true, use at your own risk. T heuristic) conv_bn_fuserdmkldnn_layout_optimizeNrrdrFz+mkldnn_layout_optimize config is not a dictz4Heuristic not found in mkldnn_layout_optimize configceZdZdZdZdZy)*optimize_for_inference..MklSupportrrN)r]r^r_NOYESUNKNOWNrjrr MklSupportrs rrr!z)this pass is only for torch.float modulescpuz!this pass is only for CPU modules call_functionc3:K|]}|jdk(yw)rN)r).0args r z)optimize_for_inference..7sI3::3Isc*jd|fS)Nr) call_method)rrs rrmz(optimize_for_inference..:sh>R>RS^ab`e>frrrrrct|drj|jSt|drj|jSy)Ncolor start_color)hasattrrrr)rufs r get_colorz)optimize_for_inference..get_colorWs@ 1g 77177# # 1m $771==) )rc3$K|]}|du ywr1rj)rrs rrz)optimize_for_inference..vs9199srctSr1)r)rsrrmz(optimize_for_inference..}s H@Urrr end_colorzmkldnn conversions: %s)FrupdaterRrdr%rC RuntimeErrortracer>r?r&rArootrDrlistr+rr(rr*mkldnn_supportedrnext parametersdtyper@r}devicemkldnn_supported_unknownranyr#inserting_beforemap_argr r rrinserting_after create_noderHrrrFrIr"r enumeraterrrall_input_nodesr'sortedrrrrrappendrrvaluesrlogging getLoggerr]inforr) r5rrdefault_pass_configr cur_tracerrMrrrsupports_mkldnnrsample_parameter mkldnn_argsdense_xrprv_noderFuser num_nodesrcur_idxr cur_colors other_color mkldnn_graphsrEprvmkldnn_conversionsresultrrs @@roptimize_for_inferencers&#."?  {+>*U +,u%34= )*BCT JHII-.FGGQRR+,DEkRJ e 45H~~joox8H$()<)<)>$?GTX^^$'$-- 77m # -JJ#33",..#' (=(=(?#F #/+11U[[@mBmm@+22ell56IInKnnI WW '{{..",.. 88","4"4 jmm +*"4"44ItyyII**40 h jj4fg  hU277#3#34kBDI))$/ '"..}j4'R**73 $w  ' '3'>$D$8'BK&H * 77m # z(Ayy|H$E .77m+ {0J..x8''- .4::!###D) *HNN#I 9 B&#8>>24  77m # {(B&D  KK WW %$++*CTYYq\*6 66&tyy|4DN040D0D{1 STVXV]V]H^bklmbnbz)A,{J{:!#9j99 99 +J#ADJ)!"~ 4  1 {3 44&-88U,VMJ 4 ! "''$**- . 4 4 ; ;D A 4 ' "''$"2"23 4 @ @ G G M 4 % "''$..1 2 < < C CD I J%%'= '))EOO; *iil**3/##D) * %++w < =$ ;;+ % )B ! # $ h$$%=?QR MMO ^^E8 ,F MA h h  ' 'h|s*$]  .]]$&]$/ ]$ ] ]! )FF) r)Itorch.fxr& torch.fx.noderrtorch.nn.utils.fusionrtypingrrrr r r r r r@torch.nnr7torch.nn.functional functionalFtorch.fx.passes.shape_proprr> collectionsrtorch.utils.mkldnnutilsmkldnnrwoperatorrrenumrr)rr'r.r~r4rRrdrur:Linearr;ReLU MaxPool2d AvgPool2dAdaptiveAvgPool2drelu transposesigmoid avg_pool2dadaptive_avg_pool2draddmulr MkldnnConv2d MkldnnLinearr|rrrrrrrTracerrrjrrr"s*3III 0 #&& -#-%S/-HTN"''DQTVYQYN$4bgg4S#X4EHHOO4 //588??/@ 0")) 0 02"))2DM24PRPWPW=2cghjhohocp2"IIryy".."''2<<rOcOc JJFFALL!//%LL(,,7IIy%%IIy%%NN= T"'']T#ryy.5I$ Lbgg Lc299n1E LTXY[YbYbdfdmdmYmTn L++$L + $ %%8-1 iii 88??i$sCx.)i Oi XX__ ir