sg3 >ddlZddlmZddlmZmZmZddlZddlmZeje Z dZ dZ GddeZd Zd eeej"eej"d ffd eeej"eej"d fffd ZeGddZy)N) dataclass)ListTupleUnion)fxcZgfd}tjj||}|fS)z\ Flatten the args into a list form and detach the tensors from computational graph. ct|tjr<|jj |j }j ||Sj ||SN) isinstancetorchTensordetachrequires_grad_ requires_gradappend)avalflat_detached_argss V/var/www/html/venv/lib/python3.12/site-packages/torch/distributed/pipelining/_utils.pyextract_tensor_argsz0flatten_args_detach..extract_tensor_argssQ a &((*++AOO.extract_tensor_args,srr)rrr s @r flatten_argsr!&s1I GG  rceZdZdZy)PipeliningShapeErrorz5Shape mismatch between configured and runtime values.N)__name__ __module__ __qualname____doc__rrr#r#9s?rr#c|j|jk(s't|d|jd|j|j|jk(s't|d|jd|j|j|jk(s/t|d|jd|jy)Nz has a shape mismatch: expected z actual z has a dtype mismatch: expected z! has a stride mismatch: expected )shaper#dtypestride)descexpectedgivens rvalidate_tensor_metadatar0=s >>U[[ ("f4X^^4DHU[[M Z   >>U[[ ("f4X^^4DHU[[M Z   ??   ."f5hoo6G5HQVQ]Q]Q_P` a   /rexpected_tensors.actual_tensorsc t|t|k7r&t|dt|dt|dtt|D]}t|d|||||y)Nz: Number of values (z") does not match expected number ()z: value )lenr#ranger0)r-r1r2is rvalidate_tensors_metadatar8Ls  N 33"f(^)<(==_`cdt`u_vvw x  3'( )  fHQC "21"5~a7H  rcDeZdZUdZej ed<eed<eed<y)PipeInfoz> Captures information for a pipeline (`Pipe` object). graph num_stageshas_loss_and_backwardN) r$r%r&r'rGraph__annotations__intboolr(rrr:r:[s 88OOrr:)logging dataclassesrtypingrrrr r getLoggerr$loggerrr! RuntimeErrorr#r0r r8r:r(rrrHs!%%    8 $(0&@<@    D.ellC6G0HHI  $u||,eELL#4E.FFG       r