sg0lddlmZmZmZmZmZmZmZddlZddl m Z ddl m cm Z ddlmZmZddlmZddlmZmZmZgdZded eeeeffd Zded eed ffd Zded eed ffd Zded eeeeefffdZded eeejj>eejj>ffdZ ded eeee jBefffdZ"ded eeeejj>ffdZ#ded eeeffdZ$ded eeeffdZ%d efdZ&d efdZ'd efdZ(ded efdZ)ded efdZ*y))AnyCallableDictListTupleTypeUnionN) _reverse2 _reverse3)Pattern) BackendConfigBackendPatternConfig DTypeConfig) get_pattern_to_dtype_configsget_qat_module_classesget_fused_module_classes"get_pattern_to_input_type_to_index-get_root_module_to_quantized_reference_moduleget_fuser_method_mappingget_module_to_qat_module&get_fusion_pattern_to_root_node_getter)get_fusion_pattern_to_extra_inputs_getter!remove_boolean_dispatch_from_namepattern_to_human_readableentry_to_pretty_strbackend_configreturncli}|jjD]\}}|j||<|SN)!_pattern_complex_format_to_configitems dtype_configs)rpattern_to_dtype_configspatternconfigs ]/var/www/html/venv/lib/python3.12/site-packages/torch/ao/quantization/backend_config/utils.pyrrsGBD)KKQQSA,2,@,@ )A ##.cg}|jD]*}|j|j|j,tt |Sr )configs qat_moduleappendtupleset)rqat_module_classesr&s r'rr&sP ((9    (  % %f&7&7 89 '( ))r(cg}|jD]*}|j|j|j,tt |Sr )r* fused_moduler,r-r.)rfused_module_classesr&s r'rr.sP ((=    * ' '(;(; <= )* ++r(cli}|jjD]\}}|j||<|Sr )r!r"_input_type_to_index)rpattern_to_input_type_to_indexr%r&s r'rr6sGEG")KKQQSN282M2M&w/N ))r(ci}|jD]5}|j|j|j||j<7|Sr )r* root_modulereference_quantized_module)rmappingr&s r'rr?sWCEG ((L    *11=*0*K*KGF&& ' L Nr(ci}|jjD]"\}}|jt|}|||<$|Sr )r!r" fuser_method1_get_fuser_method_in_reversed_nested_tuple_format)rfuser_method_mappingr%r&r;s r'rrLsZKM)KKQQS9    *MVTL,8  ) 9 r(ci}|jjD]!\}}|j|j||<#|Sr )r!r"r+)rmodule_to_qat_moduler%r&s r'rrZsSBD)KKQQS>    (,2,=,=  )> r(ci}|jjD]!\}}|j|j||<#|S)aGet a map from fusion pattern to a function that returns the root node from the fusion pattern, e.g. the most common one is: def get_root_node(node_pattern): while not isinstance(node_pattern[-1], Node): node_pattern = node_pattern[-1] return node_pattern[-1] This can work for all patterns whose root node is the "last node" in the pattern, e.g. (torch.add, MatchAllNode, (torch.ReLU, torch.Conv2d)) )r!r"_root_node_getter)rroot_node_getter_mappingr%r&s r'rrdsS9;)KKQQSI  # # /060H0H $W -I $#r(ci}|jjD]!\}}|j|j||<#|S)aGet a map from fusion pattern to a function that returns extra input nodes from the fusion pattern, in the order required by the root node. This is optional, if not specified, we will not copy over any extra inputs for the root node. Example: # Let's say we have the pattern (torch.add, MatchAllNode, (torch.nn.BatchNorm2d, torch.nn.Conv2d)) # and root node is torch.nn.Conv2d, and the node in MatchAllNode would be an extra # argument to the fused module, we can unpack the pattern and return the node at # MatchAllNode here # we can implement extra_inputs_getter as follows: def extra_inputs_getter(pattern) -> List[Any]: add, extra_input, conv_pattern = pattern return [extra_input] )r!r"_extra_inputs_getter)rextra_inputs_getter_mappingr%r&s r'rrwsS <>)KKQQSO  & & 2393N3N ' 0O '&r(cj|tjury|tjury|tjury|tjury|tj ury|tj ury|tjury|tjuryd t|vs J|d d z|S) z Some ops have a default string representation such as '.fn at 0x7ff1106bf280>', this function replaces them with the hardcoded function names. z)torch.nn.functional.fractional_max_pool2dz)torch.nn.functional.fractional_max_pool3dztorch.nn.functional.max_pool1dztorch.nn.functional.max_pool2dztorch.nn.functional.max_pool3dz'torch.nn.functional.adaptive_max_pool1dz'torch.nn.functional.adaptive_max_pool2dz'torch.nn.functional.adaptive_max_pool3dboolean_dispatchz2 does not have a human readable representation in zquantization documentation) Ffractional_max_pool2dfractional_max_pool3d max_pool1d max_pool2d max_pool3dadaptive_max_pool1dadaptive_max_pool2dadaptive_max_pool3dstrps r'rrs  A # ##: a%% %: all / all / all / a## #8 a## #8 a## #8 SV + # ?@ & ' + Hr(ct|trtd|DSt|tr|St|}|S)Nc32K|]}t|ywr )r).0inner_ps r' z,pattern_to_human_readable..sIG.w7Is) isinstancer-rQrrRs r'rrs;!UIqIII As  -a 0r(c~d}d|vrt|d}|d|dz }d|vrB|dz }|dD]0}|dz }|jD]\}}|d|d |dz }|d z }2|d z }d |vr1|d z }|d jD]\}}|d|d|dz }|dz }gd}|D]}||vr|d|d ||dz }|dz }|S)zo Given a backend_config_dict entry, returns a string with the human readable representation of it. z{ r%z 'pattern': z, r#z 'dtype_configs': [ z { z 'z': z }, z ], #num_tensor_args_to_observation_typez+ 'num_tensor_args_to_observation_type': { z z: z }, )r%r#r[z '})rr")entrys pattern_str dtype_configkvcustom_handled_fields field_names r'rrs] AE/i0@A  }[M --% %%!/2 L NA$**, ,1wqcQCs++ , OA   X -5 ;;?@FFH $DAq 4s"QCs# #A $ X  9 . .  s:,c% "3!4C 889 HA Hr(r&c|j |jS|j tdt|jts |jSt |jdk(r|j\}}||fSt |jdk(r|j\}}}|||ffStd|j)a Return the pattern specified in the given config in the reversed nested tuple format used internally in the quantization pattern matching code. If the pattern is not a tuple, or the pattern is already specified in the reversed nested tuple format, return the pattern as is. Otherwise: For 2-tuples (a, b), return (b, a). For 3-tuples (a, b, c), return (c, (b, a)). For example: * Given nn.Linear, return nn.Linear * Given (nn.Linear, nn.ReLU), return (nn.ReLU, nn.Linear) * Given (nn.Conv2d, nn.BatchNorm2d, nn.ReLU), return (nn.ReLU, (nn.BatchNorm2d, nn.Conv2d)) For context, the reason why this is needed is the user-facing BackendConfig API accepts the flat 2-or-3-tuple format in forward order. While this simple format handles the vast majority of use cases, it does not handle the more complex ones, and so the internal pattern matching code for quantization uses the following, more general reversed nested tuple format instead: operator = module_type | functional | torch op | native op | MatchAllNode Pattern = (operator, Pattern, Pattern, ...) | operator In the future, we expect to replace the above complex format with the one used by the subgraph rewriter in torch.fx, so we don't have to maintain our own complex pattern matching code. Then we won't need this helper function anymore. z>Either 'pattern' or 'pattern_complex_format' must be specified,Expected a tuple with 2 or 3 elements, got: )_pattern_complex_formatr% ValueErrorrYr-len)r&abcs r',_get_pattern_in_reversed_nested_tuple_formatros@%%1--- ~~ L   fnne ,~~ 6>>aA1v V^^  !NN AqAq6{GXXr(c|jJ|j |jSt|jtst d|jt |jdk(rt|jSt |jdk(rt|jSt d|j)a Return the fuser method specified in the given config in the reversed nested tuple format used internally in the quantization pattern matching code. If pattern is specified in the reversed nested tuple format, we assume the fuser method is also specified in this format and simply return it as is. Otherwise, we convert the fuser method as follows: * Given f(is_qat, conv, relu), return f'(is_qat, relu, conv) * Given f(is_qat, conv, bn, relu), return f'(is_qat, relu, bn_conv), where bn_conv is a 2-tuple (bn, conv) The first argument of a fuser method is always `is_qat` and is not affected in the conversion. We currently only support functions with 3 or 4 arguments. z%Expected pattern to be a tuple, got: rfrgrh) r;rirYr%r-rjrkr r )r&s r'r<r<s$    ** * %%1""" fnne ,@&..QQ 6>>a,,-- V^^  !,,--GXXr()+typingrrrrrrr torchtorch.nnnntorch.nn.functional functionalrH+torch.ao.quantization.fuser_method_mappingsr r torch.ao.quantization.utilsr rrrr__all__rtyperrrQintrModuler Sequentialrrrrrrrror<r(r'rs@@@ L/LL  $!$ '4 $ $%$*=*U49=M*,],uT3Y?O,*!* '4S> !"* !  $uxx ehhoo!6 67   !   '501 12   !  '4( () $!$ '8 $&'!' '8 '. C : C * #* Z1Y 1Y 1YhY Y Yr(