sg4XddlmZddlmZmZmZmZmZmZm Z ddl Z ddl m Z ddlmZe rddlZeGddeZGddZeGd d eZGd d Zd(d Zd(dZ d)dZGddeZGddeZGddeZGddeZGddeZGddeZGddeZGddeZ Gd d!eZ!Gd"d#eZ"Gd$d%eZ#Gd&d'eZ$y)*) annotations)AnyCallableMappingProtocolruntime_checkableSequence TYPE_CHECKINGN)_pytreec*eZdZdZ d ddZy)InputAdaptStepaA protocol that defines a step in the input adapting process. The input adapting process is a sequence of steps that are applied to the PyTorch model inputs to transform them into the inputs format expected by the exported ONNX model. Each step takes the PyTorch model inputs as arguments and returns the transformed inputs. This serves as a base formalized construct for the transformation done to model input signature by any individual component in the exporter. NcyNself model_args model_kwargsmodels R/var/www/html/venv/lib/python3.12/site-packages/torch/onnx/_internal/io_adapter.pyapplyzInputAdaptStep.apply&s 36rr Sequence[Any]rzMapping[str, Any]r@torch.nn.Module | Callable | torch_export.ExportedProgram | Nonereturnz'tuple[Sequence[Any], Mapping[str, Any]]__name__ __module__ __qualname____doc__rrrrr r s: SW 6!6(6P 6 1 6rr c6eZdZdZdddZd dZdd d dZy) InputAdapterzRA class that adapts the PyTorch model inputs to exported ONNX model inputs format.Nc|xsg|_yr_stepsrstepss r__init__zInputAdapter.__init__1 kr rc:|jj|y)zfAppends a step to the input adapt steps. Args: step: The step to append. Nr&appendrsteps r append_stepzInputAdapter.append_step4 4 rrcf|}|}|jD]}|j|||\}}|rJ|S)aKConverts the PyTorch model inputs to exported ONNX model inputs format. Args: model_args: The PyTorch model inputs. model: The PyTorch model. model_kwargs: The PyTorch model keyword inputs. Returns: A sequence of tensors converted from PyTorch model inputs. r2r&r)rrrrargskwargsr/s rrzInputAdapter.apply<sH)$0KK AD::dF%:@LD& Az rr)r(zlist[InputAdaptStep] | None)r/r rNone)rrrzFSequence[int | float | bool | str | torch.Tensor | torch.dtype | None]rrr r!r)r0rrrrr#r#.s1\"!SWP P rr#c&eZdZdZ d ddZy)OutputAdaptStepaA protocol that defines a step in the output adapting process. The output adapting process is a sequence of steps that are applied to the PyTorch model outputs to transform them into the outputs format produced by the exported ONNX model. Each step takes the PyTorch model outputs as arguments and returns the transformed outputs. This serves as a base formalized construct for the transformation done to model output signature by any individual component in the exporter. Ncyrrr model_outputsrs rrzOutputAdaptStep.apply`srrr=rrrrrrrrrr:r:Ss0 SWP rr:c8eZdZdZdddZddZ d d dZy) OutputAdapterzTA class that adapts the PyTorch model outputs to exported ONNX model outputs format.Nc|xsg|_yrr%r's rr)zOutputAdapter.__init__jr*rc:|jj|y)zhAppends a step to the output format steps. Args: step: The step to append. Nr,r.s rr0zOutputAdapter.append_stepmr1rcN|jD]}|j||}|S)aConverts the PyTorch model outputs to exported ONNX model outputs format. Args: model_outputs: The PyTorch model outputs. model: The PyTorch model. Returns: PyTorch model outputs in exported ONNX model outputs format. r2r4)rr=rr/s rrzOutputAdapter.applyus1KK CD JJ}EJBM Crr)r(zlist[OutputAdaptStep] | None)r/r:rr7)r=rrrrz1Sequence[torch.Tensor | int | float | bool | str]r8rrrr@r@gs8^"!SWP ; rr@c |jtk(rtn |j}tj||j tt t|jSr) typetuplelistpytreeTreeSpeccontextmap_replace_tuple_with_listchildren_specs)spec_types rrLrLsGII&DDIIE ?? t||T#&>@S@S"TU rch|jtk(r|jdk(r|jdS|S)Nr)rErG num_childrenrM)rNs r&_open_top_level_list_if_single_elementrSs1 yyDT..!3""1%% Krcfdfdfdfdg}td|Dst|dddy ) aZAssert the two `TreeSpec` objects are identical. Args: spec1: The first `TreeSpec` object. spec2: The second `TreeSpec` object. error_message: The error message to raise if the two `TreeSpec` objects are not identical. Raises: ValueError: If the two `TreeSpec` objects are not identical. ck(Srrspec1spec2srz/_assert_identical_pytree_spec..s rc2ttk(Sr)rLrVsrrYz/_assert_identical_pytree_spec..s(/3KE3RRrc tk(SrrSrVsrrYz/_assert_identical_pytree_spec..s6u=Frc tk(Srr\rVsrrYz/_assert_identical_pytree_spec..s?FFrc3*K|] }| ywrr).0checks r z0_assert_identical_pytree_spec..s75uw7sz Expect z . Actual .N)any ValueError)rWrX error_messagepass_if_any_checkss`` r_assert_identical_pytree_specrgsM RFF 8 7$67 7M?)E7*UG1MNN 8rc2eZdZdZddZ d ddZy) BindInputStepz0Bind the input arguments to the model signature.c||_yr)_model_signature)rmodel_signatures rr)zBindInputStep.__init__s /rNc|jj|i|}|j|jr t dd|j fS)a&Bind the input arguments to the model signature. We hope the input kwargs will be mapped to bound.args after binding. If not, we will raise an error. Args: model_args: The model args. model_kwargs: The model kwargs. model: The PyTorch model. Returns: A tuple of the model args and kwargs. args is always empty. Raises: ValueError: If there are keyword-only arguments left after binding args and kwargs to model signature. z)Keyword-only arguments are not supported.r)rkbindapply_defaultsr6rd arguments)rrrrbounds rrzBindInputStep.applysQ.+%%**JG,G  <<HI I5??""r)rlzinspect.Signaturerrrrr r!r)rrrrriris=:0SW #!#(#P # 1 #rric*eZdZdZ d ddZy)MergeKwargsIntoArgsInputStepz+Merge the input kwargs into the input args.NcPt|t|jzifS)aMerge the input kwargs into the input args. Args: model_args: The model args. model_kwargs: The model kwargs. model: The PyTorch model. Returns: A tuple of the model args and kwargs. kwargs is always empty. )rFvaluesrs rrz"MergeKwargsIntoArgsInputStep.applys' Z 5)<)<)>#??CCrrrrrrrrtrts=5 SW D!D(DP D 1 Drrtc2eZdZdZddZ d ddZy))LiftParametersAndBuffersIntoArgsInputStepzBAppend parameters and buffers to model's positional argument list.c||_yrinputs)rr{s rr)z2LiftParametersAndBuffersIntoArgsInputStep.__init__s  rNc(g||j|fS)aAppend model's parameters and buffers into its input. Args: model_args: The model args. model_kwargs: The model kwargs. model: The PyTorch model. Returns: A tuple of the model args + appended inputs and kwargs. rzrs rrz/LiftParametersAndBuffersIntoArgsInputStep.applys +*dkk*L88r)r{ztuple[torch.Tensor, ...]rr7rrrrrrrrxrxs=LSW 9!9(9P 9 1 9rrxc*eZdZdZ d ddZy)+ConvertComplexToRealRepresentationInputStep8Convert complex dtype tensors to real representation tensors. ONNX does not support complex dtype tensors. Thus, we convert complex dtype tensors to real representation tensors (i.e., float dtype tensors with an extra dimension representing the real and imaginary parts of the complex number). Nc*td|D|fS)Convert complex tensors to float tensors. Args: model_args: The model args. model_kwargs: The model kwargs. model: The PyTorch model. Returns: A tuple of the model args and kwargs. c3K|]S}t|tjr3|jr#tj|j n|Uywr isinstancetorchTensor is_complex view_as_real resolve_conjr_args rrazDConvertComplexToRealRepresentationInputStep.apply..%sOc5<<0S^^5E""3#3#3#56sAArFrs rrz1ConvertComplexToRealRepresentationInputStep.applys*" &     rrrrrrrr~r~ s:SW  ! ( P  1  rr~c*eZdZdZ d ddZy)RemoveNoneInputStepzRemove `None` from arguments. This adapt step assumes ``model_kwargs`` is empty. It also assumes ``model_args`` is flattened, i.e. it does not check `None` inside nested collections. Nc2|rJtd|DifS)a<Remove `None` from arguments. Args: model_args: The model args. model_kwargs: The model kwargs. model: The PyTorch model. Returns: A tuple of the model args and kwargs. Raises: ValueError: If `model_kwargs` is not empty. c3&K|] }|| ywrrrs rraz,RemoveNoneInputStep.apply..JsBS#/SBsrrs rrzRemoveNoneInputStep.apply6s#& BJBBBFFrrrrrrrrr/s?SW G!G(GP G 1 Grrc*eZdZdZ d ddZy)RemoveNonTensorInputStepaNRemove the non-tensor input arguments. Dynamo does not support non-tensor input arguments (https://github.com/pytorch/pytorch/issues/99534). Specifically, it does put the input into graph with an empty node, but consumed by no ones. The concrete value is embedded into the graph as a constant arg of a target node. Meta suggests in this case that one should rewrite the model code to make it tensor if the input value is supposed to change at runtime. We might need to further investigate the feasibility of that suggestion. For example, def func(x, b=1.0): y = x + b z = y.relu() return (y, z) x = torch.randn(1, 1, 2, dtype=torch.float32) gm_fun, _ = dynamo.export(func, x, b=8.0, aten_graph=True, tracing_mode="real") # class GraphModule(torch.nn.Module): # def forward(self, x, b): # arg0: f32[1, 1, 2], arg1, = fx_pytree.tree_flatten_spec(([x, b], {}), self._in_spec) # # File: path/to/pytorch/test_constant_input.py:5, code: y = x + b # add_tensor: f32[1, 1, 2] = torch.ops.aten.add.Tensor(arg0, 8.0); arg0 = None # # File: path/to/pytorch/test_constant_input.py:6, code: z = y.relu() # relu_default: f32[1, 1, 2] = torch.ops.aten.relu.default(add_tensor) # return pytree.tree_unflatten([add_tensor, relu_default], self._out_spec) Empty torch.fx.Node input leading to a mismatched number of input with PyTorch, as it's ignored in ONNX graph. Thus, we delete the useless input here. Nc2|rJtd|DifS)a>Remove Constant from arguments. Args: model_args: The model args. model_kwargs: The model kwargs. model: The PyTorch model. Returns: A tuple of the model args and kwargs. Raises: ValueError: If `model_kwargs` is not empty. c3`K|]&}t|ttttfs|(ywr)rintfloatboolstrrs rraz1RemoveNonTensorInputStep.apply..s+!#UD#'>?s,.rrs rrzRemoveNonTensorInputStep.applyqs1&  %     rrrrrrrrrMs;!NSW  ! ( P  1  rrc:eZdZUdZdZded< d ddZy)+FlattenInputWithTreeSpecValidationInputStepaWFlatten nested collection types and return a flat list of elements. ONNX can't represent collection types (e.g., dictionary, tuple of tuple of tensor, etc). This class stores the `SpecTree` output produced when `adapt` was called the first time. It then validates the `SpecTree` output produced from later `adapt` calls. Npytree.TreeSpec | None_specctj||f\}}|j ||_|ifSt|j|d|ifS)ayFlatten the model args and kwargs and validate the `SpecTree` output. Args: model_args: The model args. model_kwargs: The model kwargs. model: The PyTorch model. Returns: A tuple of the flattened model args and kwargs. The kwargs is empty, because they are flattened and merged into the args. Raises: ValueError: If the `SpecTree` output produced from the current `model_outputs` is not identical to the `SpecTree` output produced from the first `model_outputs` that was passed to this method. z=Model inputs incompatible with the format that was exported. rerH tree_flattenrrg)rrrrflattened_argsrNs rrz1FlattenInputWithTreeSpecValidationInputStep.applys`, &22J 3MN :: DJr!! * ]  r!!rrrrrr r!r__annotations__rrrrrrsG%)E !( SW "!"("P " 1 "rrc&eZdZdZ d ddZy)FlattenOutputStepaFlatten nested collection types and return a flat list of elements. ONNX can't represent collection types (e.g., dictionary, tuple of tuple of tensor, etc). NOTE: Ideally we would want to use ``FlattenOutputWithTreeSpecValidationOutputStep``, such that `SpecTree` can be validate for new model outputs. However, this is not possible currently because we never have access to real PyTorch model outputs during export. Only traced outputs may be available, but they are not an accurate reflection of the original PyTorch model outputs format as they are typically in their own unique format, depending on the tracing strategy. Nc,tj|S)zFlatten the model outputs. Args: model_outputs: The model outputs to flatten. model: The PyTorch model. Returns: A tuple of the flattened model outputs. )rH tree_leavesr<s rrzFlattenOutputStep.applys!!-00rrr=rrrrrrrrrrrs0  SW11P1  1rrc&eZdZdZ d ddZy),ConvertComplexToRealRepresentationOutputSteprNc|Dcgc]V}t|tjr8tj|r#tj|j n|Xc}Scc}w)zConvert float tensors to complex tensors. Args: model_output: The model output. model: The PyTorch model. Returns: A tuple of the model output. r)rr=routputs rrz2ConvertComplexToRealRepresentationOutputStep.applys_$(  &%,,/E4D4DV4L   v224 5    sAA#rr>rrrrrrs0SW  P   rrc6eZdZUdZdZded< d ddZy)-FlattenOutputWithTreeSpecValidationOutputStepzSame as ``FlattenOutputStep``, with additional `TreeSpec` validation. This class stores the `SpecTree` output produced when `adapt` was called the first time. It then validates the `SpecTree` output produced from later `adapt` calls. Nrrctj|\}}|j ||_|St|j|d|S)aFlatten the model outputs and validate the `SpecTree` output. Args: model_outputs: The model outputs to flatten. model: The PyTorch model. Returns: flattened_outputs: The flattened model outputs. Raises: ValueError: If the `SpecTree` output produced from the current `model_outputs` is not identical to the `SpecTree` output produced from the first `model_outputs` that was passed to this method. z>Model outputs incompatible with the format that was exported. rr)rr=rflattened_outputsrNs rrz3FlattenOutputWithTreeSpecValidationOutputStep.applysR&#)"5"5m"D4 :: DJ! * ^  ! rrrrrrrrrs= %)E !( SW!!P!  !rrc*eZdZdZ d ddZy)0PrependParamsBuffersConstantAotAutogradInputStepaEPrepend model parameters, buffers and constants to the user input. :func:`torch.export.export` lifts model parameters, buffers and constants as model input, thus, they must be added to the user input before the model is executed. Args: model: The PyTorch model with embedded parameters and buffers. NctfdjjD}tjj}g}jj D]C}||vr|j j|&|j j|EtfdjjD}g||||} |rtj| |S| ifS)rc3<K|]}j|ywr) state_dictr_namers rrazIPrependParamsBuffersConstantAotAutogradInputStep.apply..=s$    T " c3<K|]}j|ywr) constants)r_fqnrs rrazIPrependParamsBuffersConstantAotAutogradInputStep.apply..Hs")  OOC ) rr2) rFgraph_signature parameterssetnon_persistent_buffersbuffersr-rrlifted_tensor_constantsrtr) rrrrordered_paramsrordered_buffersrordered_constant_tensors updated_argss ` rrz6PrependParamsBuffersConstantAotAutogradInputStep.apply-s  --88  "%U%:%:%Q%Q!R))11 ?D--&&ut'<=&&u'7'7'=>  ? $)) ,,DD) $     &   /177l%8 Rrrrrrrrrr#s:SW , !, (, P , 1 , rrc&eZdZdZ d ddZy),PrependParamsAndBuffersAotAutogradOutputStepaPrepend model's mutated buffers to the user output. :func:`torch.export.export` lifts model's mutated buffers as outputs, thus, they must be added to the user output after the model is executed. Args: model: The PyTorch model with mutated buffers. NcttjsJdtfdjj j D}g||}|S)aFlatten the model outputs and validate the `SpecTree` output. Args: model_outputs: The model outputs to flatten. model: The PyTorch model. Returns: flattened_outputs: The flattened model outputs. z,'model' must be torch_export.ExportedProgramc3vK|]0}|jvrj|nj|2ywr)rrrs rrazEPrependParamsAndBuffersAotAutogradOutputStep.apply..xsG u'''   T "& ' s69)r torch_exportExportedProgramrFrbuffers_to_mutaterv)rr=rrupdated_outputss ` rrz2PrependParamsAndBuffersAotAutogradOutputStep.applyfsq <//  : 9 :   --??FFH   =OrsD" #) 6X66(""Jh&H O O#2OCFO O8%#N%#PD>D,992! .! HG.G ? >%!O%!P6 ~6 r%?%r