sg!ZdZddlmZddlmZej eZGddeZy)zTVLT model configuration)PretrainedConfig)loggingcjeZdZdZdZdddddgddgddd d d d d d ddddddddd dddddddffd ZxZS) TvltConfiga This is the configuration class to store the configuration of a [`TvltModel`]. It is used to instantiate a TVLT model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the TVLT [ZinengTang/tvlt-base](https://huggingface.co/ZinengTang/tvlt-base) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: image_size (`int`, *optional*, defaults to 224): The size (resolution) of each image. spectrogram_length (`int`, *optional*, defaults to 2048): The time length of each audio spectrogram. frequency_length (`int`, *optional*, defaults to 128): The frequency length of audio spectrogram. image_patch_size (`List[int]`, *optional*, defaults to `[16, 16]`): The size (resolution) of each image patch. audio_patch_size (`List[int]`, *optional*, defaults to `[16, 16]`): The size (resolution) of each audio patch. num_image_channels (`int`, *optional*, defaults to 3): The number of input image channels. num_audio_channels (`int`, *optional*, defaults to 1): The number of input audio channels. num_frames (`int`, *optional*, defaults to 8): The maximum number of frames for an input video. hidden_size (`int`, *optional*, defaults to 768): Dimensionality of the encoder layers and the pooler layer. num_hidden_layers (`int`, *optional*, defaults to 12): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 12): Number of attention heads for each attention layer in the Transformer encoder. intermediate_size (`int`, *optional*, defaults to 3072): Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder. hidden_act (`str` or `function`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"selu"` and `"gelu_new"` are supported. hidden_dropout_prob (`float`, *optional*, defaults to 0.0): The dropout probability for all fully connected layers in the embeddings, encoder, and pooler. attention_probs_dropout_prob (`float`, *optional*, defaults to 0.0): The dropout ratio for the attention probabilities. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. layer_norm_eps (`float`, *optional*, defaults to 1e-06): The epsilon used by the layer normalization layers. qkv_bias (`bool`, *optional*, defaults to `True`): Whether to add a bias to the queries, keys and values. use_mean_pooling (`bool`, *optional*, defaults to `False`): Whether to mean pool the final hidden states instead of using the final hidden state of the [CLS] token. decoder_num_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each attention layer in the decoder. decoder_hidden_size (`int`, *optional*, defaults to 512): Dimensionality of the decoder. decoder_num_hidden_layers (`int`, *optional*, defaults to 8): Number of hidden layers in the decoder. decoder_intermediate_size (`int`, *optional*, defaults to 2048): Dimensionality of the "intermediate" (i.e., feed-forward) layer in the decoder. pixel_mask_ratio (`float`, *optional*, defaults to 0.75): Image patch masking ratio. audio_mask_ratio (`float`, *optional*, defaults to 0.15): Audio patch masking ratio. audio_mask_type (`str`, *optional*, defaults to `"frame-level"`): Audio patch masking type, choose between "frame-level" and "patch-level". task_matching (`bool`, *optional*, defaults to `True`): Whether to use vision audio matching task in pretraining. task_mae (`bool`, *optional*, defaults to `True`): Whether to use the masked auto-encoder (MAE) in pretraining. loss_type (`str`, *optional*, defaults to `"classification"`): Loss types including regression and classification. Example: ```python >>> from transformers import TvltConfig, TvltModel >>> # # Initializing a TVLT ZinengTang/tvlt-base style configuration >>> configuration = TvltConfig() >>> # # Initializing a model (with random weights) from the ZinengTang/tvlt-base style configuration >>> model = TvltModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```tvltii i gelugg{Gz?gư>TFig?g333333? frame-levelclassificationc t|di||dvrtd|||_||_||_||_||_||_||_ ||_ | |_ | |_ | |_ | |_| |_||_||_||_||_||_||_||_||_||_||_||_||_||_||_||_||_y)N)r patch_levelz^audio_mask_type must be one of two acceptable strategies - {'frame_level', 'patch-level') got ) super__init__ ValueError image_sizespectrogram_lengthfrequency_lengthimage_patch_sizeaudio_patch_sizenum_image_channelsnum_audio_channels num_frames hidden_sizenum_hidden_layersnum_attention_headsintermediate_size hidden_acthidden_dropout_probattention_probs_dropout_probinitializer_rangelayer_norm_epsqkv_biasuse_mean_poolingdecoder_num_attention_headsdecoder_hidden_sizedecoder_num_hidden_layersdecoder_intermediate_sizepixel_mask_ratioaudio_mask_ratioaudio_mask_type task_matchingtask_mae loss_type) selfrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4kwargs __class__s i/var/www/html/venv/lib/python3.12/site-packages/transformers/models/deprecated/tvlt/configuration_tvlt.pyrzTvltConfig.__init__ps B "6" "@ @&')  %"4 0 0 0"4"4$&!2#6 !2$#6 ,H)!2,  0+F(#6 )B&)B& 0 0.*  ")__name__ __module__ __qualname____doc__ model_typer __classcell__)r7s@r8rrs~S jJbb%($&"#"&%"=H#H#r9rN) r=configuration_utilsrutilsr get_loggerr:loggerrrr9r8rDs34   H %`#!`#r9