sgddlZddlmZmZddlmZddlmZddlm Z m Z m Z m Z m Z ddlmZmZmZmZddlZddlmZej,j/edZd Zed Zed Zd e ee efefd e ee efeffdZd e ee efefd e ee efeffdZGddZGddeZ e Z!y)N)cached_propertywraps)chain)median)AnyCallableDictListTuple) Concatenate ParamSpecSelfTypeVar)counters benchmarkingiPTfnreturnc tjjjj dsSt dt dtjdtjdtffd }|S)aYWrapper that logs the duration of `fn`, in milliseconds, along with a representation of the function's args and kwargs, if logging is enabled. It is expected that `fn` is a method of `Benchmarker` or one of its subclasses; typing limitations prevent us from declaring this directly. If logging is disabled, this becomes a no-op. rselfargskwargsrc tj}|i|}tjd|jj j ||tj|z t z|S)NzFCall `benchmarking.%s.%s(*args=%r, **kwargs=%r)` took %f milliseconds.)time perf_counterloggerdebug __class____name__MILLISECONDS_PER_SECOND)rrrstart_tresultrs W/var/www/html/venv/lib/python3.12/site-packages/torch/_inductor/runtime/benchmarking.pywrapperzmaybe_time..wrapper"sg##%T$V$ T NN # # KK      7 *.E E   ) torch_logging _internal log_stateis_artifact_enabledrrrrrrrr%s` r$ maybe_timer-si >> # # - - A A. Q  2Y c !&& AHH    Nr&c tdtdtjdtjdt ffd }|S)aWrapper that increments relevant dynamo counters on `fn` call. It is expected that `fn` is a method of `Benchmarker` or one of its subclass; typing limitations prevent us from declaring this directly. The counter incrementation follows the formula, `counters["inductor"]["benchmarking.Foo.bar] += 1` where `Foo` is the class whose' instance called the function, and `bar` is the function name. rrrrctdd|jjzdzjzxxdz cc<|g|i|S)Ninductorz benchmarking..)rrr )rrrrs r$r%zcount..wrapper=sT dnn55 5 ;bkk I   $((((r&)rrrrrrr,s` r$countr33sF 2Y)c)!&&)AHH))) Nr&ceZdZdeddfdZeedededefde ede e efd ede f d Z ee dded egefd ed ede f dZededed ede fdZy) BenchmarkerrrNcyN)rs r$__init__zBenchmarker.__init__Hs r&r.fn_args fn_kwargsrc vd}tjD]F}t|tjs| |j }-|j |k7s=t d| t dfd}|tj dk(r|j|fi|S|j|fi|S)aBenchmark `fn(*fn_args, *fn_kwargs)` and return the runtime, in milliseconds (the actual runtime calculation is dictated by the benchmarking implementation, but may be one of [mean, median, minimum, etc.]). Functions as a convenience wrapper around device-specific implementations, like `benchmark_cpu` and `benchmark_gpu`. Raises `ValueError(...)` if we can't safely infer the device type of `fn`; for example, if multiple device types are found in `fn_args` and `fn_kwargs`, or if no device types are found. Arguments: - fn: The function to benchmark. - fn_args: The function's arguments. - fn_kwargs: The function's kwargs. Keyword Arguments: - **kwargs: The benchmarking implementation's kwargs. Returns: - The runtime of `fn(*fn_args, **fn_kwargs)`, in milliseconds. NzcCan't safely infer the device type of `fn` with multiple device types in `fn_args` and `fn_kwargs`!zCan't safely infer the device type of `fn` with no device types in `fn_args` or `fn_kwargs`! You should be calling `.benchmark_cpu` or `.benchmark_gpu` directly.ciSr7r8)rr:r;sr$z'Benchmarker.benchmark..usB595r&cpu) rvalues isinstancer'Tensordevice ValueError benchmark_cpu benchmark_gpu)rrr:r;rinferred_device arg_or_kwarg _callables ``` r$ benchmarkzBenchmarker.benchmarkKs8!'9+;+;+=> LlELL9&"."5"5$$7 y    "t 6 ell51 1%4%%i:6: :"t!!)6v66r&rIwarmuprepchdtdttffd }||t||S)aBenchmark the CPU callable, `_callable`, and return the median runtime, in milliseconds. Arguments: - _callable: The CPU callable to benchmark. Keyword Arguments: - warmup: Optionally, the duration, in milliseconds, to run `_callable` before benchmarking starts. - rep: Optionally, the duration, in milliseconds, to run `_callable` during benchmarking. Returns: - The median runtime of `_callable`, in milliseconds. msrcg}tj} tj}tj}|j||z tz||z tz|kDr |S]r7)rrappendr!)rNtimings run_start_tr"end_trIs r$run_forz*Benchmarker.benchmark_cpu..run_forspG++-K++- ))+3JJK[(,CCrINr&)intr floatr)rrIrKrLrTs ` r$rEzBenchmarker.benchmark_cpu}s2*  U  gcl##r&rctr7)NotImplementedError)rrrs r$rFzBenchmarker.benchmark_gpus!!r&)d)r __module__ __qualname__rr9r-r3rrr r strrVrJrUrErFr8r&r$r5r5Gs t   .7.7 S#X .7s.7S> .7  .7  .7 .7` OR $ $'C0 $:= $IL $  $  $D "D"""" "r&r5c zeZdZeeedededeffdZ eededegefdede fdZ y) TritonBenchmarkerrr.cN ddlm}|S#t$r}td|d}~wwxYw)z"Lazily import Triton's `do_bench`.r)do_benchzrequires TritonN)triton.testingra ImportErrorrX)rraes r$triton_do_benchz!TritonBenchmarker.triton_do_benchs4  @ / @%&78a ? @s $ $rIrc d|vr|j|fi|dSd|vr|j|fi|S|j|fi|ddiS)aBenchmark the GPU callable, `_callable`, and return the runtime, in milliseconds. Arguments: - _callable: The GPU callable to benchmark. Keyword Arguments: - quantiles: Optionally, a tuple of floats denoting the requested quantiles. - return_mode: Optionally, the requested return mode. Currently, Triton's `do_bench` supports min, max, mean, and median return modes. - **kwargs: Additional kwargs passed to Triton's `do_bench`. Returns: - The runtime of `callable`, in milliseconds. If `kwargs["quantiles"]` is specified, this is the first requested quantile. Else, if `kwargs["return_mode"]` is specified, this is the requested return mode. Otherwise, this is the median. quantilesr return_moder)re)rrIrs r$rFzTritonBenchmarker.benchmark_gpusf& & '4'' O#ORWO Or&r_)"r functoolsrr itertoolsr statisticsrtypingrrr r r typing_extensionsr r rrr'torch._dynamo.utilsrr(getArtifactLoggerr rr!rrr-r3r5r_ benchmarkerr8r&r$rqs ,33CC (  ) )(N C cN CLS!V$a'( k#q&!1$%<h{36*A-.8KQ