sg;dZddlZddlmZddlmZmZmZmZddl Z ddl Z ddl m Z ddl m Z ddlmZmZmZmZdd lmZdd lmZdd lmZdd lmZmZmZmZmZmZm Z d dl!m"Z"erddl#m$Z$ejJe&Z'dZ(eGddeZ)eGddeZ*eGddeZ+Gdde jXZ-Gdde jXZ.Gdde jXZ/Gdde jXZ0Gdd e jXZ1Gd!d"e jXZ2d#Z3dGd$Z4Gd%d&e jXZ5d'e jld(e7d)e jlfd*Z8Gd+d,e jXZ9Gd-d.e9Z:Gd/d0e9Z;e9e:e;d1Z<Gd2d3e jXZ=Gd4d5e jXZ>Gd6d7e jXZ?Gd8d9e jXZ@Gd:d;e jXZAGd<d=e jXZBGd>d?e jXZCGd@dAeZDdBZEdCZFedDeEGdEdFeDZGy)HzPyTorch Mimi model.N) dataclass)ListOptionalTupleUnion)nn)ACT2FN)Cache DynamicCacheSlidingWindowCache StaticCache)AttentionMaskConverter)BaseModelOutputWithPast)PreTrainedModel) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardis_flash_attn_2_available#is_flash_attn_greater_or_equal_2_10loggingreplace_return_docstrings) MimiConfig)_flash_attention_forwardrceZdZUdZdZej ed<dZejed<dZ e e e eejfed<dZe e e eejfed<y) MimiOutputa Args: audio_codes (`torch.LongTensor` of shape `(batch_size, num_quantizers, codes_length)`, *optional*): Discret code embeddings computed using `model.encode`. audio_values (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*) Decoded audio values, obtained using the decoder part of Mimi. encoder_past_key_values (`Cache`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks) that can be used to speed up sequential decoding of the encoder transformer. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user can optionally input only the last `audio_values` or `audio_codes (those that don't have their past key value states given to this model). decoder_past_key_values (`Cache`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks) that can be used to speed up sequential decoding of the decoder transformer. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user can optionally input only the last `audio_values` or `audio_codes (those that don't have their past key value states given to this model). N audio_codes audio_valuesencoder_past_key_valuesdecoder_past_key_values)__name__ __module__ __qualname____doc__rtorch LongTensor__annotations__r FloatTensorr rrr rr!Y/var/www/html/venv/lib/python3.12/site-packages/transformers/models/mimi/modeling_mimi.pyrr4ss0%)K!!(&*L%##*OSXeE48I8I3J,J&KLSOSXeE48I8I3J,J&KLSr+rcleZdZUdZdZej ed<dZe e e e ejfed<y)MimiEncoderOutputaY Args: audio_codes (`torch.LongTensor` of shape `(batch_size, num_quantizers, codes_length)`, *optional*): Discret code embeddings computed using `model.encode`. encoder_past_key_values (`Cache`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks) that can be used to speed up sequential decoding of the encoder transformer. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user can optionally input only the last `audio_values` or `audio_codes (those that don't have their past key value states given to this model). Nrr )r"r#r$r%rr&r'r(r rrr rr)r*r+r,r.r.Ts> %)K!!(OSXeE48I8I3J,J&KLSr+r.cleZdZUdZdZej ed<dZe e e e ej fed<y)MimiDecoderOutputaU Args: audio_values (`torch.FloatTensor` of shape `(batch_size, segment_length)`, *optional*): Decoded audio values, obtained using the decoder part of Mimi. decoder_past_key_values (`Cache`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks) that can be used to speed up sequential decoding of the decoder transformer. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user can optionally input only the last `audio_values` or `audio_codes (those that don't have their past key value states given to this model). Nrr!) r"r#r$r%rr&r)r(r!rrr rr*r+r,r0r0hs> '+L%##*OSXeE48I8I3J,J&KLSr+r0ceZdZdZ ddededededededeffd Zd Zd Zd e jd e jfdZ e dd e jde eefdedefdZdZxZS) MimiConv1dz;Conv1d with asymmetric or causal padding and normalization. in_channels out_channels kernel_sizestridedilationgroupsbiasc bt ||j|_| |jn||_|dkDr$|dkDrt j d|d|d|dtj||||||| |_ |jjd}tj|jjdtj}|jjd}tj|dz |zdztj}|j!d |d |j!d |d |j!d tj||z tjd |j"dz|_|j"|j$z |_y)NrzNMimiConv1d has been initialized with stride > 1 and dilation > 1 (kernel_size=z stride=z , dilation=).)r7r8r9rdtyper6F persistentr5 padding_total)super__init__use_causal_convcausalpad_modeloggerwarningrConv1dconvr5r&tensorr6int64r7register_bufferr@ padding_right padding_left) selfconfigr3r4r5r6r7r8rFr9 __class__s r,rCzMimiConv1d.__init__s ,, +3+;  A:(Q, NN!!, XfX[ RTV  II {FXV\cg  ii++A. dii..q1E99%%a(llK!Ox#?!#C5;;W  Xv%@ ]KEJ _ell;;OW\WbWb.cpuv"//14 ..1C1CCr+ctjj}ttjjdr$tjjj}||j yN weight_normrutilsrUhasattrparametrizationsrJrPrUs r,apply_weight_normzMimiConv1d.apply_weight_normFhh** 288,,m <((33??KDIIr+cVtjj|jyNrrWremove_weight_normrJrPs r,r`zMimiConv1d.remove_weight_norm ##DII.r+ hidden_statesreturnc>|jd}||jz |jz|jz dz}t j |j tjdz }||jz|jz|jz }||z S)zSee `pad_for_conv1d`.r)shaper5r@r6r&ceiltorL)rPrclengthn_frames ideal_lengths r,_get_extra_padding_for_conv1dz(MimiConv1d._get_extra_padding_for_conv1ds $$R(T---0B0BBdkkQTUU::h'**5;;7!;$++-0@0@@4CUCUU f$$r+paddingsmodevaluecl|jd}|\}}|dk(s"tjj||||St ||}d}||kr*||z dz}tjj|d|f}tjj||||} | jd|z } | dd| fS)zTiny wrapper around torch.nn.functional.pad, just to allow for reflect padding on small input. If this is the case, we insert extra 0 padding to the right before the reflection happens. rfreflectrr.N)rgr functionalpadmax) rcrnrorprjrOrNmax_pad extra_padpaddedends r,_pad1dzMimiConv1d._pad1ds $$R(&.# my ==$$]HdEJ JlM2 W &(1,IMM--ma^LM""=(D%Hll2*c4C4i  r+c&|j|}|jr+|j||j|f|j}n7|j||j |j |zf|j}|j|}|S)N)ro)rmrErzr@rFrOrNrJ)rPrc extra_paddings r,forwardzMimiConv1d.forwards::=I ;; KK 8J8JM7ZaeananKoM KK 1 143E3E 3UV]a]j]j(M -0 r+)rrrNT)zero)r"r#r$r%intboolrCr[r`r&Tensorrm staticmethodrstrfloatrzr} __classcell__rRs@r,r2r2|sE(D(D (D  (D  (D(D(D(DT/ %|| %  %!ell!eCHo!S!bg!!$ r+r2c ReZdZdZ d dededededef fd ZdZd Zd ZxZ S) MimiConvTranspose1dzDConvTranspose1d with asymmetric or causal padding and normalization.r3r4r5r6r8ct ||j|_|j|_t j |||||||_|js|jdk(s td|jjd}|jjd}||z }|jr(tj||jz|_ n |dz|_ ||jz |_y)N)r8r9?zB`trim_right_ratio` != 1.0 only makes sense for causal convolutionsrrA)rBrCrDrEtrim_right_ratiorConvTranspose1drJ ValueErrorr5r6mathrhrNrO) rPrQr3r4r5r6r8r9r@rRs r,rCzMimiConvTranspose1d.__init__s ,, & 7 7&&{L+v^dkop  t44;ab bii++A. !!!$#f, ;;"&=4;P;P+P!QD "/!!3D )D,>,>>r+ctjj}ttjjdr$tjjj}||j yrTrVrZs r,r[z%MimiConvTranspose1d.apply_weight_norm r\r+cVtjj|jyr^r_ras r,r`z&MimiConvTranspose1d.remove_weight_normrbr+c|j|}|jd|jz }|d|j|f}|S)Nrf.)rJrgrNrO)rPrcrys r,r}zMimiConvTranspose1d.forwardsM -0 !!"%(:(::%c4+<+) rBrCrmax_position_embeddingsbaser&arangerLrrirM)rPrrrdevicerrRs r,rCzMimiRotaryEmbedding.__init__os '>$ $)) Q!5;;(W(](](_(b(bci(jmqmumu(uvw ZeDr+c|jddddfjj|jddd}|dddddfj}|jj }t |tr|dk7r|nd}tj|d5|j|jzjdd}tj||fd }|j}|j} dddj|j  j|j fS#1swYAxYw) NrrfrmpscpuF) device_typeenabledrArr<)rrexpandrgrtype isinstancerr&autocast transposecatcossinrir=) rPr position_idsinv_freq_expandedposition_ids_expandedrfreqsembrrs r,r}zMimiRotaryEmbedding.forwardxs% !MM$4-8>>@GG HZHZ[\H]_acde ,QaZ 8 > > @hhmm %/ S%AkUZFZk`e ^^ U C &,,.1F1L1L1NNYYZ[]^_E))UEN3C'')C'')C   vvAGGv$cff177f&;;;   s !A+E  E)ii'N)r"r#r$rCr&no_gradr}rrs@r,rrns%EU]]_ <