sgYddlZddlZddlmZddlmZddlmZddl m Z ddl m Z e e ZdZGddZGd d ZGd d eZGd deZGddeZGddeZGddeZGddeZGddeZGddeZGddeZGddeZGdd eZGd!d"eZy)#N)sparse)add_start_docstrings) get_loggerad Args: input_ids (`jnp.ndarray` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) scores (`jnp.ndarray` of shape `(batch_size, config.vocab_size)`): Prediction scores of a language modeling head. These can be logits for each vocabulary when not using beam search or log softmax for each vocabulary token when using beam search kwargs (`Dict[str, Any]`, *optional*): Additional logits processor specific kwargs. Return: `jnp.ndarray` of shape `(batch_size, config.vocab_size)`: The processed prediction scores. cveZdZdZeedejdejdejfdZy)FlaxLogitsProcessorzSAbstract base class for all logit processors that can be applied during generation. input_idsscoresreturnc2t|jd)z"Flax method for processing logits.H is an abstract class. Only classes inheriting this class can be called.NotImplementedError __class__selfr r s ^/var/www/html/venv/lib/python3.12/site-packages/transformers/generation/flax_logits_process.py__call__zFlaxLogitsProcessor.__call__6!"~~f g  N __name__ __module__ __qualname____doc__r!LOGITS_PROCESSOR_INPUTS_DOCSTRINGjnpndarrayrrrrr3s>];< #++ s{{ s{{ = rrcveZdZdZeedejdejdejfdZy)FlaxLogitsWarperzjAbstract base class for all logit warpers that can be applied during generation with multinomial sampling.r r r c2t|jd)zFlax method for warping logits.r rrs rrzFlaxLogitsWarper.__call__ArrNrrrrr!r!>s>t;< #++ s{{ s{{ = rr!c zeZdZdZeedejdejdedejfdZ y)FlaxLogitsProcessorLista. This class can be used to create a list of [`FlaxLogitsProcessor`] or [`FlaxLogitsWarper`] to subsequently process a `scores` input tensor. This class inherits from list and adds a specific *__call__* method to apply each [`FlaxLogitsProcessor`] or [`FlaxLogitsWarper`] to the inputs. r r cur_lenr c r|D]}tj|jj}t |dkDrmt fdt |jddDs3tdt |jd|jd||||fi}||||}|S)Nc3&K|]}|v ywNr).0argkwargss r z3FlaxLogitsProcessorList.__call__..UsSS3&=Ssrz,Make sure that all the required parameters: z for z$ are passed to the logits processor.) inspect signaturer parameterslenalllistkeys ValueErrorr)rr r r%r, processor function_argss ` rrz FlaxLogitsProcessorList.__call__Ps ?I#--i.@.@ALLM=!A%SD9K9K9M4Nqr4RSS$FtML^L^L`GaFbbg$..//SU#9fgHH"9fg> ? rN) rrrrrrrrintrrrrr$r$IsL ;< #++ s{{ S _b_j_j = rr$cpeZdZdZdefdZdejdejdedejfdZ y ) FlaxTemperatureLogitsWarperz [`FlaxLogitsWarper`] for temperature (exponential scaling output probability distribution). Args: temperature (`float`): The value used to module the logits distribution. temperaturecXt|tr|dkDstd|||_y)Nrz:`temperature` has to be a strictly positive float, but is ) isinstancefloatr5r;)rr;s r__init__z$FlaxTemperatureLogitsWarper.__init__is/+u-kAoYZeYfgh h&rr r r%r c$||jz }|Sr))r;rr r r%s rrz$FlaxTemperatureLogitsWarper.__call__os$*** rN) rrrrr>r?rrr8rrrrr:r:`sC'E' #++s{{SUXU`U`rr:ceZdZdZed dfdededefdZdejd ejd ed ejfd Z y )FlaxTopPLogitsWarpera= [`FlaxLogitsWarper`] that performs top-p, i.e. restricting to top tokens summing to prob_cut_off <= prob_cut_off. Args: top_p (`float`): If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation. filter_value (`float`, *optional*, defaults to -inf): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. Inftop_p filter_valuemin_tokens_to_keepct|tr |dks|dkDrtd|t|tr|dkrtd|||_||_||_y)Nrg?z.`top_p` has to be a float > 0 and < 1, but is rEz:`min_tokens_to_keep` has to be a positive integer, but is )r=r>r5r8rFrGrH)rrFrGrHs rr?zFlaxTopPLogitsWarper.__init__sj%'EAIMeWUV V,c27IA7MYZlYmno o ("4rr r r%r cXtj||jd\}}tj||j }t jj|djd}||jk}tj|d}||jdddfjdz}|jddd|jfjd}tj|||} t jj!|| d} | S)NaxisrErT)laxtop_kshaper full_likerGjaxnnsoftmaxcumsumrFrollatsetrHwhere sort_key_val) rr r r% topk_scores topk_indices mask_scorescumulative_probs score_masktopk_next_scores next_scoress rrzFlaxTopPLogitsWarper.__call__s$'IIffll26F$G! \mmFD,=,=> 66>>+B>?FFBFO% 2 XXj!, jmmAqD)--d33  ]]1&?(?(?&?#?@DDTJ 99ZkJgg**<9IJ2N rN rrrrr>r8r?rrrrrrrCrCtsa =B%L=de5e555^a5#++s{{SUXU`U`rrCceZdZdZed dfdededefdZdejd ejd ed ejfd Z y )FlaxTopKLogitsWarpera [`FlaxLogitsWarper`] that performs top-k, i.e. restricting to the k highest probability elements. Args: top_k (`int`): The number of highest probability vocabulary tokens to keep for top-k-filtering. filter_value (`float`, *optional*, defaults to -inf): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. rDrErOrGrHczt|tr|dkrtd|t|||_||_y)Nrz6`top_k` has to be a strictly positive integer, but is )r=r8r5maxrOrG)rrOrGrHs rr?zFlaxTopKLogitsWarper.__init__s>%%!UV[U\]^ ^ 23 (rr r r%r c|j\}}tj||z|j}t |j |jd}t j ||\}} tjtj||zdddf||fj} |j} | j| z} |j| j| }|j||} | S)NrK) rPrfullrGminrOrN broadcast_toarangeflattenrWrXreshape)rr r r% batch_size vocab_sizenext_scores_flattopkr[r\shifttopk_scores_flattopk_indices_flatras rrzFlaxTopKLogitsWarper.__call__s!' J88J$;T=N=NO4::v||B/0$'IIfd$;! \  #**Z"8:"Eq$w!OR\^bQcdlln&..0(002U:+../@AEEFVW&..z:F rNrbrrrrdrdsa ;@,bc)c))\_) #++ s{{ S UXU`U` rrdcpeZdZdZdefdZdejdejdedejfdZy ) !FlaxForcedBOSTokenLogitsProcessorz [`FlaxLogitsProcessor`] that enforces the specified token as the first generated token. Args: bos_token_id (`int`): The id of the token to force as the first generated token. bos_token_idc||_yr))rw)rrws rr?z*FlaxForcedBOSTokenLogitsProcessor.__init__s (rr r r%r ctj|jtd }dtj|dz z }tj ||j dd|jfjd|}|SNinfrEr) rrhrPr>bool_rYrWrwrXrr r r% new_scores apply_penaltys rrz*FlaxForcedBOSTokenLogitsProcessor.__call__skXXfllU5\M: CIIgk22 =*--4;L;L8L*M*Q*QRS*TV\] rN rrrrr8r?rrrrrrrvrvsC)S)#++s{{SUXU`U`rrvcteZdZdZdedefdZdejdejdedejfd Zy ) !FlaxForcedEOSTokenLogitsProcessorae [`FlaxLogitsProcessor`] that enforces the specified token as the last generated token when `max_length` is reached. Args: max_length (`int`): The maximum length of the sequence to be generated. eos_token_id (`int`): The id of the token to force as the last generated token when `max_length` is reached. max_length eos_token_idc ||_||_yr))rr)rrrs rr?z*FlaxForcedEOSTokenLogitsProcessor.__init__s$(rr r r%r c,tj|jtd }dtj||j z dzz }tj ||jdd|jfjd|}|Srz) rrhrPr>r|rrYrWrrXr}s rrz*FlaxForcedEOSTokenLogitsProcessor.__call__suXXfllU5\M: CIIg&?!&CDD =*--4;L;L8L*M*Q*QRS*TV\] rNrrrrrrsJ)3)c)#++s{{SUXU`U`rrcteZdZdZdedefdZdejdejdedejfd Zy ) FlaxMinLengthLogitsProcessora3 [`FlaxLogitsProcessor`] enforcing a min-length by setting EOS probability to 0. Args: min_length (`int`): The minimum length below which the score of `eos_token_id` is set to `-float("Inf")`. eos_token_id (`int`): The id of the *end-of-sequence* token. min_lengthrct|tr|dkrtd|t|tr|dkrtd|||_||_y)Nrz2`min_length` has to be a positive integer, but is z4`eos_token_id` has to be a positive integer, but is )r=r8r5rr)rrrs rr?z%FlaxMinLengthLogitsProcessor.__init__s\*c*j1nQR\Q]^_ _,, q0@ST`Sabc c$(rr r r%r cdtj||jz ddz }tj||jdd|j fj td |}|S)NrErr{)rcliprrYrWrrXr>rr r r%rs rrz%FlaxMinLengthLogitsProcessor.__call__s`CHHWt%>1EE =&))At7H7H4H*I*M*MuUZ|m*\^de rNrrrrrrsJ)3)c)#++s{{SUXU`U`rrc"eZdZdZdZdefdZy)(FlaxSuppressTokensAtBeginLogitsProcessora [`FlaxLogitsProcessor`] supressing a list of tokens as soon as the `generate` function starts generating using `begin_index` tokens. This should ensure that the tokens defined by `begin_suppress_tokens` are not sampled at the beginning of the generation. Args: begin_suppress_tokens (`List[int]`): Tokens to not sample. begin_index (`int`): Index where the tokens are suppressed. c2t||_||_yr))r3begin_suppress_tokens begin_index)rrrs rr?z1FlaxSuppressTokensAtBeginLogitsProcessor.__init__s%)*?%@"&rr%cdtj||jz z }tj||jdd|j fj td |}|S)NrEr{)rr|rrYrWrrXr>rs rrz1FlaxSuppressTokensAtBeginLogitsProcessor.__call__!saCIIg0@0@&@AA =&))At7Q7Q4Q*R*V*VX]^cXdWd*egmn rN)rrrrr?r8rrrrrrs '3rrcpeZdZdZdefdZdejdejdedejfdZ y ) !FlaxSuppressTokensLogitsProcessorz [`FlaxLogitsProcessor`] suppressing a list of tokens at each decoding step. The processor will set their log probs to be `-inf` so they are not sampled. Args: suppress_tokens (`list`): Tokens to not sample. suppress_tokensc$t||_yr))r3r)rrs rr?z*FlaxSuppressTokensLogitsProcessor.__init__3s#O4rr r r%r cn|jd|jfjtd }|S)N.r{)rWrrXr>rAs rrz*FlaxSuppressTokensLogitsProcessor.__call__6s13 4 445995<-H rN) rrrrr3r?rrr8rrrrrr)sC55#++s{{SUXU`U`rrcjeZdZdZdZdej dej dedej fdZy) FlaxForceTokensLogitsProcessora [`FlaxLogitsProcessor`] that takes a list of pairs of integers which indicates a mapping from generation indices to token indices that will be forced before sampling. The processor will set their log probs to 0 and all other tokens to `-inf` so that they are sampled at their corresponding index. Args: force_token_map (`list`): Map giving token ids and indices where they will be forced to be sampled. cDt|}tjt|j dztj dz}|j D]&\}}| |j|j|}(tj ||_ y)NrEdtyperK) dictronesrfr4int32itemsrWrXforce_token_array)rforce_token_maprindextokens rr?z'FlaxForceTokensLogitsProcessor.__init__Gs/ HHc/*>*>*@&AA&EciiX[]]+113 KLE5 $5$8$8$?$C$CE$J! K"%+zerosrNdynamic_update_slice)generation_idxrn current_tokenr~updatesr rs r _force_tokenz=FlaxForceTokensLogitsProcessor.__call__.._force_tokenSsvaJ 22>BMvV\\BeEl]RJiiQv||DG11*g=GYZJ rrcSr)rr srz9FlaxForceTokensLogitsProcessor.__call__.._sFrc`tjjdk\fdfdS)NrcSr)r)rr%srrzKFlaxForceTokensLogitsProcessor.__call__....ds W-rcSr)rrsrrzKFlaxForceTokensLogitsProcessor.__call__....fsr)rNcondr)rr%r rsrrz9FlaxForceTokensLogitsProcessor.__call__..as)CHH&&w/14- r)rNrrrP)rr r r%rs` ``@rrz'FlaxForceTokensLogitsProcessor.__call__Rs@  t--33A6 6     rN) rrrrr?rrr8rrrrrr<s< >#++s{{SUXU`U`rrceZdZdZdZdZy)#FlaxWhisperTimeStampLogitsProcessora{ Whisper specific Processor. This processor can be used to force a list of tokens. The processor will set their log probs to `inf` so that they are sampled at their corresponding index. Args: generate_config (`GenerateConfig`): The generate config used to generate the output. The following parameters are required: eos_token_id (`int`, *optional*, defaults to 50257): The id of the *end-of-sequence* token. no_timestamps_token_id (`int`, *optional*, defaults to 50363): The id of the `"<|notimestamps|>"` token. max_initial_timestamp_index (`int`, *optional*, defaults to 1): Used to set the maximum value of the initial timestamp. This is used to prevent the model from predicting timestamps that are too far in the future. c`|j|_|j|_|jdz|_|dz|_|jr|xjdz c_t |dr|j |_n|j|_|j |j|_yy)NrErmax_initial_timestamp_index)rno_timestamps_token_idtimestamp_beginris_multilingualhasattrrro)rgenerate_config model_configdecoder_input_lengths rr?z,FlaxWhisperTimeStampLogitsProcessor.__init__}s+88&5&L&L#.EEI/!3  * *    !  ?$A B/>/Z/ZD ,/;/F/FD ,  + + 3/;/F/FD , 4rc|jddjfjtd }fd}t j |||}t jjk(dd}t jjdu|d}jjz}t j||jdd|dzdfjtd |}tjj|d}fd}t j |||}|S) Nr{c ftjjz dk\dd}tj|dz jk\|d}tjjz dkdd}tj|dz jk\d|}tj|tj|dkD|jjdj t d |jdjj t d |S)NrETFrrr{)rrYrrrWrXr>r) input_ids_kscores_klast_was_timestamppenultimate_was_timestampr%rs r handle_pairszBFlaxWhisperTimeStampLogitsProcessor.__call__..handle_pairss"!$Gd6F6F,F1+LdTY!Z !$GaK(D,@,@@+"  ), 7T=M=M3MQR2RTXZ_(` %(+ GaK(D,@,@@)) % 99" -1KK 4 4 67;;U5\MJKK 3$"3"3488%,G  rTFrErKrLc8tjj|jdd}t j |dj}t j ||kD|jdjjtd |S)NrKrLr{) rRrS logsumexprrrfrYrWrXr>) logprobs_krtimestamp_logprobmax_text_token_logprobrs rhandle_cumulative_probszMFlaxWhisperTimeStampLogitsProcessor.__call__..handle_cumulative_probss # 0 0DrRvmaprrYrrrrS log_softmax) rr r r%rapply_max_initial_timestamp last_allowedlogprobsrs ` ` rrz,FlaxWhisperTimeStampLogitsProcessor.__call__s"1d999:>>e }M 2(,' 6:&)ii4;K;K0KTSX&Y#&)ii  , ,D 80 ' # ++d.N.NN  ' IIa)++ , 0 0%, ?  66%%f2%6 3128VD rN)rrrrr?rrrrrrls G"<rrceZdZdZdefdZdejdedefdZdejd ejfd Z dejd ejded ejfd Z y ) FlaxNoRepeatNGramLogitsProcessora9 [`FlaxLogitsProcessor`] that enforces no repetition of n-grams. See [Fairseq](https://github.com/pytorch/fairseq/blob/a07cb6f40480928c9e0548b737aadd36ee66ac76/fairseq/sequence_generator.py#L345). Args: ngram_size (`int`): All ngrams of size `ngram_size` can only occur once. ngram_sizecXt|tr|dkrtd|||_y)Nrz;`ngram_size` has to be a strictly positive integer, but is )r=r8r5r)rrs rr?z)FlaxNoRepeatNGramLogitsProcessor.__init__s.*c*jAoZ[eZfgh h$rr ror%c  j\ }|jdz z }|jdz z } fd} |zjdzf}tjj d |z|t j |j} t j |z |zkjd} tj| | f f|fjzzS)a get a matrix of size (batch_size,) + (vocab_size,)*n (for n-grams) that represent the n-grams that occurred previously. The BCOO representation allow to store only the few non-zero entries, instead of the full (huge) matrix rEc |z}|z}|j|jtj|gt j Dcgc]}tj|||zf!c}zScc}wr))rWrXrarrayranger)ivalbposjrnr rs rbody_funzFFlaxNoRepeatNGramLogitsProcessor.get_previous_ngrams..body_funsJAz/C66!9== BGtAWXAsyy+AsQwJ7XY  Ys $A=rrfloat32)rP) rPrrRrN fori_looprrrrkastyperBCOO) rr ror%seq_len seq_ngrams cur_ngramsrrPall_update_indicesdatarns `` @rget_previous_ngramsz4FlaxNoRepeatNGramLogitsProcessor.get_previous_ngramss (oo G! 34 ! 34  j($//A*=> WW.. zJ&#))E2Y   : 23j:6MMUUV_`{{D"45j]j]]a]l]lMl=lmmr latest_tokensr ctjtjd}tj|||S)zt Determines which tokens must be banned given latest tokens and the previously seen ngrams. c|t|Sr))tuple)rprevious_ngramss rinner_fnzIFlaxNoRepeatNGramLogitsProcessor.get_banned_tokens_mask..inner_fns#5#78 8r)rsparsifyrRr bcoo_todense)rrrrs rget_banned_tokens_maskz7FlaxNoRepeatNGramLogitsProcessor.get_banned_tokens_masks@   9   9""8M?#KLLrr cfd}tjjjdz k\|fd}|S)Nc "j\}}j|}tjjdjdz fj }t jj|t jjdjdz z fjdjdz fd}j||jd}tj|td S)NrrEr)rrboolr{)rPrrrrrrRrNr dynamic_slicerrrYr>) _rorrbanned_tokens_indices_maskr%r r rs rtrue_fnz:FlaxNoRepeatNGramLogitsProcessor.__call__..true_fn s"LLMAz"66y*gVO IIyq'94??Q;N&OW`WfWfgMGG88%%7doo.A#BCiooVWFX[_[j[jmn[nEp M*.)D)D]Tc)d)k)klr)s &997%,O OrrEcSr)rrsrrz;FlaxNoRepeatNGramLogitsProcessor.__call__..sQWr)rRrNrr)rr r r%routputs```` rrz)FlaxNoRepeatNGramLogitsProcessor.__call__ s4 P&w$//A*==X rN) rrrrr8r?rrrrrrrrrrs%3% nS[[ nc nTW nD MCKK MUXU`U` M#++s{{SUXU`U`rr)r.rRjax.laxrN jax.numpynumpyrjax.experimentalrutilsr utils.loggingrrloggerrrr!r3r$r:rCrdrvrrrrrrrrrrrs  #(& H %!*    d."2()+)X + F(;,(;2#6</B2(;&-%8-`^*=^BS':Sr