sg~ddlZddlmZddlmZmZmZmZmZm Z ddl Z ddl Z ddl m Z ddlmZddlmZddlmZdd lmZmZmZmZmZmZmZgd Zej:eZe d Gd de Z!e d Gdde Z"e d Gdde Z#eGddZ$GddZ%y)N) dataclass)AnyCallableDictListOptionalTuple) compatibility)map_arg) ShapeProp) split_by_tags)CALLABLE_NODE_OPSFxNetAccFusionsFinderNamesNodeListNodeSetTensorOrTensorsTensors)FxNetMinimizerBadModuleErrorFxNetMinimizerRunFuncError!FxNetMinimizerResultMismatchErrorF)is_backward_compatibleceZdZdZy)rz9 Raised if failed to split out a minimize module N__name__ __module__ __qualname____doc__O/var/www/html/venv/lib/python3.12/site-packages/torch/fx/passes/net_min_base.pyrr!r!rceZdZdZy)rz@ Raised if error occurs during run_a or run_b functions Nrr r!r"rr)r#r!rceZdZdZy)rzJ Raised if comparing function thinks the results are mismatching. Nrr r!r"rr1r#r!rcPeZdZUdZdZeed<dZeed<dZ eed<dZ eed<dZ y ) _MinimizerSettingBasea; Args: `accumulate_error`: Instead of using a's input for both converted module to verify , use the previous outputs of each converted module as input to accumulate the errors. `traverse_method`: "sequential" or "binary" or "accumulate" Determine the way of traverse the nodes in FX module. `find_all`: Minimizer will go through the entire model and return all problematic nodes. `return_intermediate`: If true, when using `run_nodes()` function to run the model, intermediate results of all the ops will be returned as output. Faccumulate_error sequentialtraverse_methodfind_allreturn_intermediatecdd}t|jD]\}}|d|d|dz }|S)NzFX Minimizer Settings:  :  )varsitems)self settings_strkvs r"__str__z_MinimizerSettingBase.__str__OsG1 J$$& ,DAq b2aSO +L ,r!N) rrrrr(bool__annotations__r*strr+r,r7r r!r"r'r'9s8 #d"'OS'Hd %%r!r'ceZdZdZ d5dej j dedee e e ge e e ffdedeeeej j egdfdeeeeegdff d Zd6d ej j d ed ed e fdZd6d ej j d ed ed e fdZde de dej j fdZdej j ded e eeffdZdefdZded e ej j effdZ d6dej j dede d efdZded ed!ed efd"Zded efd#Zded efd$Zded ed!ed%e d ef d&Zded%ee d efd'Z ded efd(Z!ded efd)Z"ded ed!ed efd*Z#ded+e$d efd,Z%d-eed.eed efd/Z&d5d-eed.eefd0Z'd1e$efd2Z(d3Z) d7d-eed.eed+ee$d%ee d ef d4Z*y)8_MinimizerBasea This class is used to automatically find problematic nodes in a model. It takes a FX graphmodule and generate some submodules while traverse the graph. Then two functions `run_a` and `run_b` will be used to run the same submodule and a function `compare_fn` will be used to compare the results. Currently we provides two ways to traverse the graph and generate submodules. 1. Sequential traversal: this will traverse the graph node by node and generate one submodule with one sigle node. 2. Binary searching: this will do a binary search style traversal on the graph. For internal Users, a guide can be found here https://fb.quip.com/HDtuAgiKGfkP. Nmodule sample_input compare_fnsettingsmodule_exporter exclusion_fncXt|tjjsJ||_||_||_||_||_||_ i|_ i|_ i|_ g|_ d|_|jjj Dchc]}|j"t$vs|}}t'|jj(|j t+|j||_|jjj Dcgc]}|j"dk(s|j. } }t1| t1|j k(sJt3| D])\} } || |j| <|| |j| <+ycc}wcc}w)Nr placeholder) isinstancetorchfx GraphModuler=r>r?rAr@rB a_outputs b_outputsresultsreports iterationgraphnodesoprr propagaterfusionsnamelen enumerate) r3r=r>r?r@rArBnodecallable_nodes placeholdersirSs r"__init__z_MinimizerBase.__init__gs$&%(("6"6777 ($.  (*,*,(* )+  "[[..44 CT8TD   ) $++(($*;*;<I,T[[.IK #'++"3"3"9"9 TWW =UDII  < C(9(9$::::!. 3GAt#/?DNN4 #/?DNN4  3  sF".F""F'7F'modinputs report_idxreturnctd)zz Run `mod` with `inputs` and generate output. The output will be compared with output of run_b(). zrun_a() is not implemented. RuntimeErrorr3r[r\r]s r"run_az_MinimizerBase.run_a 899r!ctd)zz Run `mod` with `inputs` and generate output. The output will be compared with output of run_a(). zrun_b() is not implemented.r`rbs r"run_bz_MinimizerBase.run_brdr!a_resultb_result submodulectd|jjD}t|jdt j jrM||j|jdj<||j|jdj<yt|jdD]=\}}|||j|j<|||j|j<?y)a Store the outputs of self.run_a() and self.run_b() into self.a_outputs and self.b_outputs, so that we can use them when execute preceding nodes that use those outputs as inputs. Args: a_result: Output of self.run_a(). Could be a tensor or tensors. b_result: Output of self.run_b(). Could be a tensor or tensors. submodule: The module that generates a_result and b_result. c3@K|]}|jdk(s|yw)outputN)rP).0rVs r" z0_MinimizerBase._store_outputs..s  dgg6ID srN) nextrNrOrEargsrFrGNoderIrSrJrU)r3rgrhri output_noderYargs r"_store_outputsz_MinimizerBase._store_outputss  &__22  k&&q)588== 97?DNN;++A.33 47?DNN;++A.33 4$K$4$4Q$78 73+3A;sxx(+3A;sxx( 7r! main_module submod_pathc g g}t||}|jjDcgc]}|jdk(s|j }}t ||j jkrD|D]>} j|j ||j|j|@n}|jjrtd|ddtjjdt f fd }|j#|} ||j$| j' }|jjs fS |fScc}w)al Try get submodule inputs from stored outputs. If not found then use torch_glow.get_submod_inputs to get the inputs. If accumulate_error is False, use a_input for run_a() and run_b() otherwise use a_input for run_a and b_input for run_b. Args: main_module: Top-levlel fx module. submod_path: Path to the submodule we want to run and compare results. Returns: a_input: List of tensor(s) that will be used by run_a() as submodule inputs. b_input: List of tensor(s) that will be used by run_b() as submodule inputs. rDz)Can't find previous stored outputs named !r3r\c |yNr )r3r\a_inputs r" get_inputsz5_MinimizerBase._get_submod_inputs..get_inputss r!)getattrrNrOrPrSsetrIkeysappendrJr@r(printrFnnModulerregister_forward_pre_hookr>remove) r3rurvb_inputrirVrXrSr|handler{s @r"_get_submod_inputsz!_MinimizerBase._get_submod_inputss5$K5 "+//"7"7 477m;SDII   |  3 3 5 5$ 5t~~d34t~~d34 5}}--A,qQR ! !# ! 88DF ** + MMOG}}--G# #; s EEselected_nodesc|jjjD]L}|jtvr||vrd|_"t d|jDrd|_Fd|_Ny)ag Tag selected nodes with tag "minimize". Nodes with the same tags will be split to the same submodule afterwards. Args: selected_nodes: Nodes that we want to minimize. We will tag those nodes with "minimize", all preceding nodes with "main_0" and all following nodes with "main_1". minimizec3\K|]$}|jtvr|jdv&yw)>main_1rN)rPrtag)rmns r"rnz,_MinimizerBase._tag_nodes..s/44,,//s*,rmain_0N)r=rNrOrPrranyall_input_nodes)r3rrVs r" _tag_nodesz_MinimizerBase._tag_nodessmKK%%++ $Dww//~%%-- $# $r!rOc|j|t|jgd}d}|jD]\}}d|vr |dk(r|}t d||dk(rt d|||fS)ak Split self.module so that one submodule consists of `nodes` and only `nodes`. Args: nodes: Nodes that we want to include in the minimize submodule. Returns: split_module (torch.fx.GraphModule): the module after split. submodule_name (str): the name of the submodule that consists of `nodes`. )rrrrz0Expected only one minimize submodule with nodes z,Minimize submodule was not found with nodes )rrr=named_childrenr)r3rO split_modulesubmodule_name child_name_s r"_build_submodulez_MinimizerBase._build_submodules %T[[2RS !)88: MJ+#!+2FugN  R .>ugF ^++r!r submod_name output_namesct||}|j||\}}t|jdk(r"|jj gd|_|j|dk\r|n|j dz }|j d|rg} |j jD]L} | jdk(r|j j| | j|vs<| j | N|j jt| dk(r| dn t| |j j|j|j jD])} | jdk(st| j d} + |j#|||} |j%|||} |j'| | ||}| Dcgc] }t-|}}|j/| | |\}}||j0 <|j d ||sr|j d | |j2rD|j3||t-| dd z|j3||t-| dd zt5d | y#t($r-}|j d|d|t+d|d|d}~wwxYwcc}w) a Run the submodule in `split_module` that has name `submod_name` using `self.run_a` and `self.run_b` and compare their results. Args: split_module: Main module that contains the minimize submodule. submod_name: Name of the minimize submodule. output_names: Names of the node we want to output. If None, we will use the original output. rr zRun and compare ...rlc|jSrz)rS)xs r"z1_MinimizerBase._run_and_compare..ns !&&r!zException raised when running r/NzNumerical accuracy = zResult mismatch for _cpu_acc)r}rrTrLrrMrNrOrP erase_noderSrltuplelint recompiler rprcrfrt Exceptionrr:r?rKrAr)r3rrrr]rir{rreport output_nodesrV result_keyrgrhenamesr6numeric_result bool_results r"_run_and_comparez_MinimizerBase._run_and_compareAs"L+6 22<M t||  ! LL   #DNJ!OjRSAST +, %'L!-- .77h&OO..t499 , ''-  . OO " "#&|#4#9 Qu\?R  OO "    !OO)) BDww("$TYY0@A  B zz)WjAHzz)WjAH   (I >$  %/0SV0E0&*ooh%&P# #1 Z  -n-=>? MM0 = >##$$YJqM(:V(C$$YJqM(:V(C46J:,4WX X  MM:;-r!M N,0 RsC  1s9J K K)(KK all_nodes start_idxend_idxct}|||}g}|jA|j|||t|dk(r dg}|jj ||S|dj }|dj }|xj dz c_|jj ||j d|j |j d|d|d|dz d|d t| t|} |j| \} } |j| | |g|j d |j|tS#ttf$rt|dk(r*|j d | d |j|| cYS|j d |j|t|dz} |j|||| z}t|dk7r|jjs|cYS|j||| z|}t|dk(r%|j d||j||cYSwxYw)z9 Recursive binary search implementation. rAll nodes are excluded by userr zBinary search iteration From node index : to &. Size of the interested node list is No discrepancy found.zcThis is the last node in the sub-module. Search in the current branch is successful with culprit = .MProceed to split and lower the halves of the current sub-module individually.zaFurther split and lowering found no errors. Unable to minimize the submodule with list of nodes: )r~rBrTrLrrSrMrr print_reportrr_binary_search_implr@r+) r3rrrculpritsrOrfirst_node_nameoutput_node_name cur_nodesrrmids r"rz"_MinimizerBase._binary_search_implsm  E#Ig6    (   eY 85zQ:; ##F+(-- 9>> ! F# 00@AB yk?*;4 {!L\K]^336u:, @ !Z ' (,(=(=i(H %L+  ! !, >N=O PF MM1 2   f %5LG+,MN 5zQ QQZP[[\^!!&)  MM+    f %e*/C// 9iRUoVH8}!$--*@*@// 9s?GTH8}! LLQ7T!!&)O? s!(EAIA,I A IIc:|j|dt|S)z7 Binary search on `nodes` for culprit. r)rrT)r3rOs r"_binary_traversez_MinimizerBase._binary_traverses''q#e*==r!ct}|D]q}g}|jj||xjdz c_|jd|jd|jd|jt j d|j|g}|jl|j|ddt|dk(rK|jd|j|j||jjs|cS|h}||jvr|j|} |j|\}}|j|||jg|j|t|S#t$rV|j!||jd ||j||jjs|cYcSYt"$rV|j%||jd ||j||jjs|cYcSY2wxYw) zX Traverse `nodes` one by one and determine if any of them is a culprit. r zSequential traverse iteration rz Visit node: zVisit node: %srrzUser exclusion : "Found culprit from numeric error: Found culprit from run error: )r~rLrrMrS_LOGGERinforBrTrr@r+rRrrraddrupdate) r3rOrrVr node_listrrrs r"_sequential_traversez#_MinimizerBase._sequential_traverses E' $D "F LL   ' NNa N MM:4>>:J!L M MML 4 5 LL)499 5#'&I  ,!!)R4y>Q&MM$5dii["AB%%f-==11' "&It||# LL.  $,0,A,A),L) k%%lK$))M!!&)7' $R6 $ T" B4&IJ!!&)}}--#O.. $ * >tfEF!!&)}}--#O.  $s ! F# )$..)9:   fGDDUV"9+QuY/?/D/D.ET'RSTYZaTbTgTgShhjk*?*;4?OP"#3"456w7JQ7N6OP   &* 2 X(,(=(=i(H %L+  ! !, n=Mz Z4 MMB3%qsHYYZ[ \C CN$+]53C3H3H2IK!!&)(6y1}IIMI MM+    f %00a.YY00 3WW_23MN Z MM5o5Fd>JZZ[\ ]C CP$+]53C3H3H2IK!!&)  MM+    f %00 3WW00a.YY/ Zs()HA&K 68K 0K  K ct}|dj}|dj}d|d|g}|jd|jj|d}t |dz }|dnd} | s|rm|jd |j ||j |||d}|jd d |d ||jg|j || s|sd g} |j | |j |d|dz||d}| jd|jj| | jdd|d ||jg|j | |j|||dzd||d||dg} |jj| |j | |S)a Traverse topologically sorted node list Find minimium block (start_idx, end_idx) which contains the culprit 1st pass: search for end_idx by finding the last node in culprit block where Numerical accuracy (0, end_idx) > threshold 2nd pass: search for start_idx by finding the first node in culprit block where Numerical accuracy (start_idx, end_idx) < threshold Form minimum block by (start_idx - 1, end_idx) rrzBlock search from rz2**************************************************r TFz(Start searching for last node in culpritz Finish Pass 1zFind end_idx = rz)Start searching for first node in culpritz Finish Pass 2zFind start_idx = z'Finish searching, found minimum block (,)) r~rSrrLrTrrrr) r3rOrrrrlast_node_reportrrrun_bothfirst_node_report result_reports r"_block_traversez_MinimizerBase._block_traversegs E(--r00AnEUVW) ,- e*q.)14u ~  # #$N O   . ///y'4PG  # ##%gYag0C0C/DE    . / >!L M    / 011%'A+2F SZ\abI  $ $X . LL   1 2  $ $#' {!E)4D4I4I3JK    / 0 i! 45B5CSBTTUV[\cVdUeefgh  M* -(r!ct}|j|j|ddt|dk(r dg}|jj ||S|dj }|dj }d|d|g}t|} |j |\}}|j|||g|j||S#ttf$r*|j d||j||cYSwxYw)zL run user defined `nodes` and determine if it is a culprit. rrrzDefined graph from rFound culprit ) r~rBrTrLrrSrrrrr) r3rOrrrrrrrs r"_defined_traversez _MinimizerBase._defined_traverses E    (   eR , u:?67F LL   'O(-- 9>>''8=MN=O P   f %  23MN  MMN9+6 7   f %O s 9C6C?>C?ct}t}|jjr td|S|D]}g}|jj ||xj dz c_|j d|j d|j||j}|t|tr|d}|t|ts Jd||j d| |j|\}}|j|||g|j||S#tt f$r=|j||j d||j||cYcSwxYw) Nz9'Find All' mode is not supported in accumulate traversal.r zAccumulate traverse iteration rrzminimize: node_name: z Add node: r)r~r@r+rrLrrMrrSrErr:rrrrr) r3rOr nodes_to_runrVr node_namerrs r"_accumulate_traversez#_MinimizerBase._accumulate_traverseswE #  == ! ! M NO D "F LL   ' NNa N MM:4>>:J!L M   T " I$Iu)E%aL (Z3. 3&yk2 3 MMJyk2 3 ,0,A,A,,O) k%%lK)M!!&)) 6 67QR  T" tf56!!&)  s39D00AE<;E<c t}|||}t|}|j|j|||t|}n4|D]/}||jvs|j|j|1g}|jj ||xj dz c_|j d|j d|j d|d|dz dt| |j|\} } |j| | g|j d|j|tS#t$r;|j||j d||j||cYSt$r;|j||j d ||j||cYSwxYw) ? Skip certain nodes in graph based on settings r z Nodes block rrrrrrr) r~rBrRrrLrrMrTrrrrr) r3rrrrrOrrVrrrs r"_skip_traverse_implz"_MinimizerBase._skip_traverse_impls E#Ig6 Z    (   eY 8E I 94<<'$$T\\$%78 9 F# ! dnn%5Q78 ykgai[9336u:, @  (,(=(=i(H %L+  ! !, R @ MM1 2   f %5L2  OOI & MM>ykJ K   f %O*  OOI & MM:9+F G   f %O  s2'EAG AG  G  skip_nodescd}t|}d}t}||kr_||}|j|vr||kDr|j|||}|dz}n#||dz k(r||kr|j|||dz}|dz }||kr_|S)rrr )rTr~rSr)r3rrr num_nodesidxrrVs r"_skip_traversez_MinimizerBase._skip_traverse s  N 5IoS>D Z'?#77 9cRH!G  A %)s*:33Iy#PQ'R 1HCIor!startendcg}|du}|jjjD]K}|jtvr|j |k(rd}|r|j ||j |k(sJ|S|S)z Collect nodes in the model that between nodes with name of `start` and `end`. These two nodes are also included. NT)r=rNrOrPrrSr)r3rrrOadd_noderVs r"_collect_nodesz_MinimizerBase._collect_nodes"s| D=KK%%++ Dww//yyE! T"yyC   r!c|j||}t|}|D]/}||jvs|j|j|1g}|jj r|Dcgc]}|j }} |j|\}}|j|||ycc}w#ttf$r} t| Yd} ~ yd} ~ wwxYw)a] Run part of the model from `start` node to `end` node. If `start` is None then we start from the beginning of the model. If `end` is None then we stop at the end of the model. Args: start: The name of the node which is the first node of the submodule we want to run. If set to None, then we'll start with the first node of the model. end: The name of the node which is the last node of the submodule we want to run. If set to None, we'll end with the last node of the model. N) rr~rRrr@r,rSrrrrr) r3rrrOrrVrrrrs r" run_nodesz_MinimizerBase.run_nodes9s##E3/J  5Dt||#  d!34 5 == , ,278$DII8L8 (,(=(=i(H %L+  ! !, \ J 9 ' -   !HH  s.B,'B11C CCrctt|D]'}|dkDrtd||zt||)y)Nrz . )rangerTr)r3rrYs r"rz_MinimizerBase.print_report[s>s6{# !A1uefQi'(fQi  !r!cH|jD]}|j|yrz)rLr)r3rs r" print_reportsz_MinimizerBase.print_reportsbs#ll &F   f % &r!ct|jt|jj|j ||}|jj dk(r|j |S|jj dk(r|j|S|jj dk(r|j|S|jj dk(r| td|j||S|jj dk(r|j|S|jj dk(r|j||Std|jj d ) az Minimizing the model from node with name `start` to node with name `end` base on self.settings. Find culprits that causes FxNetMinimizerRunFuncError or FxNetMinimizerResultMismatchError errors. Args: start: The name of the node where we want to start minimizing. If set to None, then we'll start with the first node of the model. end: The name of the node where we want to terminate minimizing. If set to None, we'll end with the last node of the model. skip_nodes: The names of nodes where we want to skip during minimizing. It'll create subgraphs without these skip nodes under the hood. Only applicable in mode "skip". find_last_node: True if only last_node of a culprits is needed in mode "block". False if only the first_node of a culprits is needed. Only applicable in mode "block". Returns: nodes: A list of nodes that causes FxNetMinimizerRunFuncError or FxNetMinimizerResultMismatchError errors during minimizing. r)binary accumulateskipz<'skip_nodes' can't be None when 'traverse_method' is 'skip'.definedblockzUnknown traverse method rx) rr@r=rNrr*rrrrarrr)r3rrrrrOs r"rz_MinimizerBase.minimizefs=: dmm dkk ##E3/ == ( (L 8,,U3 3 == ( (H 4((/ / == ( (L 8,,U3 3 == ( (F 2""#abb&&uj9 9 == ( (I 5))%0 0 == ( (G 3''~> >5dmm6S6S5TTUVWWr!)NN)r)NNNN)+rrrrrFrGrHrrrrr floatr8r'rr:rintrZrcrfrtrrrrrrrrrrrrrrrrrrrrr r!r"r<r<Xs 6  !93$$9393 ou 5uUD[7I I 93(93" %((..4  93 hS)4/ 0 93v:--:w:C:Yh::--:w:C:Yh:7!7"788'' 7<2  88//2 >A2 w 2 h$$2$,g$,%8L8Lc8Q2R$,V KYhh**KYKY KY  KYZC!C.1Ch>7> /(/w/dYX(YXsYXSYXbfYXknYXx6X6x~6RY6rxG6%(%w%N'X'#'PS'X_'Tdw,HSM (. x} (3- D!49!& $!%))- 6X}6Xc]6XTN 6X ! 6X  6Xr!r<)&logging dataclassesrtypingrrrrrr rFtorch.fxtorch.fx._compatibilityr torch.fx.noder shape_propr split_utilsr tools_commonrrrrrrr__all__ getLoggerrrrrrrr'r<r r!r"rs!== 1!!&  '  H %e,9-e,-e, -