sg) ddlmZmZddlmZddlmZmZmZer ddl m Z m Z m Z GddeZ dded ed ed efd Zd ede d efdZded eedfdZy))c_floatsizeof)Enum) TYPE_CHECKINGOptionalUnionAutoFeatureExtractor AutoProcessor AutoTokenizerc&eZdZeZedefdZy)ParameterFormatreturnc,t|jS)zf Number of byte required for this data type Returns: Integer > 0 )rvalue)selfs J/var/www/html/venv/lib/python3.12/site-packages/transformers/onnx/utils.pysizezParameterFormat.sizesdjj!!N)__name__ __module__ __qualname__rFloatpropertyintrrrrrs E "c""rr dimensionfixed_dimensionnum_token_to_addrc|dkr|}||z}|S)ze Args: dimension: fixed_dimension: num_token_to_add: Returns: rr)rrr s r compute_effective_axis_dimensionr"&s!A~#  !!I rnum_parametersdtypec ||jzS)a7 Compute the size taken by all the parameters in the given the storage format when serializing the model Args: num_parameters: Number of parameters to be saved dtype: The data format each parameter will be saved Returns: Size (in byte) taken to save all the parameters )r)r#r$s r"compute_serialized_parameters_sizer&9s EJJ &&r model_name)r r r c`ddlm}m}m} |j|S#t t tf$r{d\}} |j|}n#t tf$rYnwxYw |j|}n#t tf$rYnwxYw||t d|d||Yy||cYS|cYSwxYw)a> Gets a preprocessor (tokenizer, feature extractor or processor) that is available for `model_name`. Args: model_name (`str`): Name of the model for which a preprocessor are loaded. Returns: `Optional[Union[AutoTokenizer, AutoFeatureExtractor, AutoProcessor]]`: If a processor is found, it is returned. Otherwise, if a tokenizer or a feature extractor exists, it is returned. If both a tokenizer and a feature extractor exist, an error is raised. The function returns `None` if no preprocessor is found. r r )NNNz&Couldn't auto-detect preprocessor for z1. Found both a tokenizer and a feature extractor.)r r r from_pretrained ValueErrorOSErrorKeyError)r'r r r tokenizerfeature_extractors rget_preprocessorr0GsFE%,},,Z88  *%'1$ $ 5 55jAI"     D 4 D DZ P "     %6%B8 Duv  #4#<  " $ $)%sbB-A  B- AB-AB-!A43B-4BB-BB-"B-(B-,B-N)r)ctypesrrenumrtypingrrrr)r r r rrr"r&strr0rrrr5s#11EE "d "c]`il& 's '? 'WZ '&%&%%8p2q)r&%r