sgEdZddlZddlmZmZmZmZmZddlZ ddl m Z m Z m Z mZddlmZddlmZddlmZmZmZej.eZGd d eZy) z%Feature extractor class for SpeechT5.N)AnyDictListOptionalUnion)mel_filter_bankoptimal_fft_length spectrogramwindow_function)SequenceFeatureExtractor) BatchFeature)PaddingStrategy TensorTypeloggingc eZdZdZddgZ d#dededededed ed ed ed ed ededededeffd Z e d$de e jde e jdede e jfdZde jde jfdZ d%deee je ee e je e efdeee je ee e je e efdeeeefdeededeedeedeeeefdeedefdZ d&dee je ee e je e efd edeeeefdeededeedeedeeeefdefd!Zdeeefffd" ZxZS)'SpeechT5FeatureExtractora Constructs a SpeechT5 feature extractor. This class can pre-process a raw speech signal by (optionally) normalizing to zero-mean unit-variance, for use by the SpeechT5 speech encoder prenet. This class can also extract log-mel filter bank features from raw speech, for use by the SpeechT5 speech decoder prenet. This feature extractor inherits from [`~feature_extraction_sequence_utils.SequenceFeatureExtractor`] which contains most of the main methods. Users should refer to this superclass for more information regarding those methods. Args: feature_size (`int`, *optional*, defaults to 1): The feature dimension of the extracted features. sampling_rate (`int`, *optional*, defaults to 16000): The sampling rate at which the audio files should be digitalized expressed in hertz (Hz). padding_value (`float`, *optional*, defaults to 0.0): The value that is used to fill the padding values. do_normalize (`bool`, *optional*, defaults to `False`): Whether or not to zero-mean unit-variance normalize the input. Normalizing can help to significantly improve the performance for some models. num_mel_bins (`int`, *optional*, defaults to 80): The number of mel-frequency bins in the extracted spectrogram features. hop_length (`int`, *optional*, defaults to 16): Number of ms between windows. Otherwise referred to as "shift" in many papers. win_length (`int`, *optional*, defaults to 64): Number of ms per window. win_function (`str`, *optional*, defaults to `"hann_window"`): Name for the window function used for windowing, must be accessible via `torch.{win_function}` frame_signal_scale (`float`, *optional*, defaults to 1.0): Constant multiplied in creating the frames before applying DFT. This argument is deprecated. fmin (`float`, *optional*, defaults to 80): Minimum mel frequency in Hz. fmax (`float`, *optional*, defaults to 7600): Maximum mel frequency in Hz. mel_floor (`float`, *optional*, defaults to 1e-10): Minimum value of mel frequency banks. reduction_factor (`int`, *optional*, defaults to 2): Spectrogram length reduction factor. This argument is deprecated. return_attention_mask (`bool`, *optional*, defaults to `True`): Whether or not [`~SpeechT5FeatureExtractor.__call__`] should return `attention_mask`. input_valuesattention_mask feature_size sampling_rate padding_value do_normalize num_mel_bins hop_length win_length win_functionframe_signal_scalefminfmax mel_floorreduction_factorreturn_attention_maskc t|d |||d|||_||_||_||_||_||_| |_| |_ | |_ | |_ | |_ ||zdz|_ ||zdz|_t|j|_|j dzdz|_t%|j|jd|_t)|j"|j|j|j|j*dd|_| d k7rt/j0d t2| d k7rt/j0d t2yy)N)rrriT) window_lengthnameperiodicslaney)num_frequency_binsnum_mel_filters min_frequency max_frequencyrnorm mel_scale?zeThe argument `frame_signal_scale` is deprecated and will be removed in version 4.30.0 of Transformersg@zcThe argument `reduction_factor` is deprecated and will be removed in version 4.30.0 of Transformers)super__init__rr#rrrrrrr r!r" sample_size sample_strider n_fftn_freqsr windowr r mel_filterswarningswarn FutureWarning)selfrrrrrrrrrrr r!r"r#kwargs __class__s k/var/www/html/venv/lib/python3.12/site-packages/transformers/models/speecht5/feature_extraction_speecht5.pyr4z!SpeechT5FeatureExtractor.__init__NsQ$ wl-_lwpvw(%:"($$("4  " 0% 5='-74?'(8(89  a1, %D4D4D4K\K\gkl *#|| --)))),,   $ MMw  s " MMu  #returnc|tj|tj}g}t||j dD]m\}}||d|j z tj |d|jdzz }||jdkr|||d|j|o|S|Dcgc]<}||j z tj |jdzz >}}|Scc}w)z[ Every array in the list is normalized to have zero mean and unit variance NgHz>r) nparrayint32zipsummeansqrtvarshapeappend)rrrnormed_input_valuesvectorlength normed_slicexs rAzero_mean_unit_var_normz0SpeechT5FeatureExtractor.zero_mean_unit_var_norms  %XXnbhh?N"$ "%lN4F4Fr4J"K 9 &)=)=)? ?2776RYSY?K^K^K`cgKgChh L..q11,9L)#**<8  9#"Vb"bPQALBGGAEEGdN4K#K"b "b""#cs:AC? one_waveformc t||j|j|j|j|j |j d}|jS)zZ Extracts log-mel filterbank features for one waveform array (unbatched). log10)r9 frame_lengthr fft_lengthr:r!log_mel)r r9r5r6r7r:r!T)r>rV log_mel_specs rA_extract_mel_featuresz.SpeechT5FeatureExtractor._extract_mel_featuressP# ;;))))zz((nn  ~~rBaudio audio_targetpadding max_length truncationpad_to_multiple_ofreturn_tensorsc | | td| ;| |jk7rAtd|d|jd|jd| d tjd||j|d ||||||fi| } nd} |>|j|d ||||||fi| } | | S| d | d <| j d } | | | d<| S)aA Main method to featurize and prepare for the model one or several sequence(s). Pass in a value for `audio` to extract waveform features. Pass in a value for `audio_target` to extract log-mel spectrogram features. Args: audio (`np.ndarray`, `List[float]`, `List[np.ndarray]`, `List[List[float]]`, *optional*): The sequence or batch of sequences to be processed. Each sequence can be a numpy array, a list of float values, a list of numpy arrays or a list of list of float values. This outputs waveform features. Must be mono channel audio, not stereo, i.e. single float per timestep. audio_target (`np.ndarray`, `List[float]`, `List[np.ndarray]`, `List[List[float]]`, *optional*): The sequence or batch of sequences to be processed as targets. Each sequence can be a numpy array, a list of float values, a list of numpy arrays or a list of list of float values. This outputs log-mel spectrogram features. padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*, defaults to `False`): Select a strategy to pad the returned sequences (according to the model's padding side and padding index) among: - `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single sequence if provided). - `'max_length'`: Pad to a maximum length specified with the argument `max_length` or to the maximum acceptable input length for the model if that argument is not provided. - `False` or `'do_not_pad'` (default): No padding (i.e., can output a batch with sequences of different lengths). max_length (`int`, *optional*): Maximum length of the returned list and optionally padding length (see above). truncation (`bool`): Activates truncation to cut input sequences longer than *max_length* to *max_length*. pad_to_multiple_of (`int`, *optional*): If set will pad the sequence to a multiple of the provided value. This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. return_attention_mask (`bool`, *optional*): Whether to return the attention mask. If left to the default, will return the attention mask according to the specific feature_extractor's default. [What are attention masks?](../glossary#attention-mask) return_tensors (`str` or [`~utils.TensorType`], *optional*): If set, will return tensors instead of list of python integers. Acceptable values are: - `'tf'`: Return TensorFlow `tf.constant` objects. - `'pt'`: Return PyTorch `torch.Tensor` objects. - `'np'`: Return Numpy `np.ndarray` objects. sampling_rate (`int`, *optional*): The sampling rate at which the `audio` or `audio_target` input was sampled. It is strongly recommended to pass `sampling_rate` at the forward call to prevent silent errors. Nz9You must provide either `audio` or `audio_target` values.z3The model corresponding to this feature extractor: z& was trained using a sampling rate of zB. Please make sure that the provided audio input was sampled with z and not .zIt is strongly recommended to pass the ``sampling_rate`` argument to this function. Failing to do so can result in silent errors that might be hard to debug.FTrlabelsrdecoder_attention_mask) ValueErrorrloggerwarning_process_audioget)r>r_r`rarbrcrdr#rerr?inputs inputs_targetris rA__call__z!SpeechT5FeatureExtractor.__call__sI~ =\1XY Y  $ 2 22 I$P**+,**+9]O1F NN\   (T(("%  FF  #/D//"%  M~$$#0#@x )6):):;K)L&)57MF34 rBspeech is_targetc Zt|tjxrt|jdkD} | r&t|jdkDrt d|| xs@t|t tfxr(t|dtjtt f} | r4|Dcgc]'}tj|tj)c}}n| s@t|tjs&tj|tj}nht|tjrN|jtjtjur|jtj}| s|g}|j} |r=|D cgc]} |j| }} td|i}|j |_ n td|i}|j"|f|||||d| }| |_ |d}t|dtjs8|Dcgc]'}tj|tj)c}|d<nt|tjst|dtjrc|djtjtjur1|Dcgc]!}|jtj#c}|d<nkt|tjrQ|jtjtjur"|jtj|d<|j%d}|6|Dcgc]'}tj|tj&)c}|d<|sW|j(rK|j+|| t,j.ur|nd}|j1|d||j2 |d<||j5|}|Scc}wcc} wcc}wcc}wcc}w) Nr&r%z2Only mono-channel audio is supported for input to r)dtyper)rarbrcrdr#r)rb)rr) isinstancerFndarraylenrNrjlisttupleasarrayfloat32rufloat64astyperr^rrpadrnrHr_get_padding_strategiesr DO_NOT_PADrUrconvert_to_tensors)r>rrrsrarbrcrdr#rer?is_batched_numpy is_batchedfeature_size_hackwaveformfeaturesencoded_inputs padded_inputsrrGrs rArmz'SpeechT5FeatureExtractor._process_audio)s|&fbjj9Sc&,,>ORS>S FLL 1A 5QRVQWXY Y% ve} - d:fQi"**V[]aIb3c  IOPvbjjrzz:PFJvrzz$BZZbjj9F  + @T0T]]2::.FXF!-- MST228<THT)>8*DEN $ 1 1D )>6*BCN   !!1"7   .%^4 ,q/2::6^j,kUZRZZRZZ-P,kM. )<4<?BJJ7Q%%"**)==S_,`%U\\"**-E,`M. )  bjj 1l6H6HBHHUWU_U_L`6`,8,?,? ,KM. )'**+;<  %^l.mUZrzz%rxx/P.mM* +T..//J/OWfWqWqq  -1,H,Hn-n\`\n\n-I-M. )  %)<<^LMCQU*-l -a/ns',PP$,P&P#4,P(cJt|}gd}|D] }||vs||= |S)N)r9r:r5r6r7r8)r3to_dict)r>outputnamesr(r@s rArz SpeechT5FeatureExtractor.to_dicts;"^ !Dv~4L ! rB)r&i>FP@ hann_windowr1rig|=r%T)r) NNFNFNNNN)FFNFNNN)__name__ __module__ __qualname____doc__model_input_namesintfloatboolstrr4 staticmethodrrFrwrUr^rrrrrrqrmrrr __classcell__)r@s@rArrsI*X()9:""")$' !&*::: :  :  ::::"::::: $:xbe#2::&#8NPTUYZ_U`Pa%abcstS/12 s SM s  s%SMs (~s!sJ!78s }s sp 5:$( ,004;?Ubjj$u+tBJJ/?d5kARRSUUtS/12 U SM U  U%SMU (~U!sJ!78U Un c3h  rBr)rr;typingrrrrrnumpyrF audio_utilsr r r r !feature_extraction_sequence_utilsr feature_extraction_utilsrutilsrrr get_loggerrrkrr2rBrArsK,33\\I499   H %j7jrB