sgkO (
d dl mZ d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dl m
Z
d dl m Z m
Z
mZmZmZmZmZmZmZmZ d dlZd dlmZ d dlZd dlm Z d dl m Z d dl m Z d dl!m"Z" ddl#m$Z$ ddl%m&Z&m'Z' d Z(d
Z)d Z*d Z+d
Z,d
Z-dZ.dZ/ G d de Z0 G d de1 Z2 G d de' Z3ddZ4ddZ5ejl G d d Z7 G d de Z8 G d d e8 Z9 dd Z: dd Z; dd Z< d dd"Z= dd#Z> dd$Z? dd%Z@ dd&ZA d dd'ZB d dd(ZCdd)ZD dd*ZE dd+ZF dd,ZG dd-ZH dd.ZI dd/ZJ dd0ZK dd1ZL dd2ZMddd3ZN d dd4ZO d dd5ZPdd6ZQ eQd7 dd8 ZR eQd9 dd: ZS eQd; dd< ZT eQd= dd> ZU eQd? dd@ ZV eQdA ddB ZW eQdC ddD ZX eQdE ddF ZY eQdG ddH ZZ eQdI ddJ Z[ eQdK ddL Z\ eQdM ddN Z] eQdO ddP Z^ eQdQ ddR Z_ eQdS ddT Z` eQdU ddV Za eQdW ddX Zb eQdY ddZ Zc eQd[ dd\ Zd eQd] dd^ Ze eQd_ dd` Zf eQda ddb Zg eQdc ddd Zh e5de dddddfdgdhdidjd!g dddkdkdldjd dmdndddf ddo Zi e5dp e5dq dddddfdgdhdidjd!g dddkdkdldjd dmdndddf ddr Zj G ds dt Zk G du dvek Zl G dw dxek Zm G dy dzek Zn G d{ d|ek Zo G d} d~ek ZpepZq G d dek Zr e5d dddddfdgdhdidjd!g dddkdkdldjd dmdndddddf dd Zsy) ) annotationsN) ExitStack)
AnyDictIteratorListLiteralOptionalTupleUnionProtocolcast) ImmutableSandboxedEnvironment )logger)suppress_stdout_stderr Singletonz{% for message in messages %}{{'<|im_start|>' + message['role'] + '
' + message['content'] + '<|im_end|>' + '
'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
' }}{% endif %}
<|im_end|>a {{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token + ' ' }}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}a {{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}a {% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>
'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>
' }}{% endif %}c e Zd ZdZddddddddg ddddddddd d
d
d d d
ddddd ddZy)LlamaChatCompletionHandlera Base Protocol for a llama chat completion handler.
Very generic protocol that can be used to implement any chat format.
The only hard requirement is that it must return a ChatCompletion when
stream=False and an iterator of ChatCompletionChunks when stream=True.N皙?ffffff?( F 皙?皙? ?r @皙?) functions
function_calltools tool_choice temperaturetop_ptop_kstreamstopseedresponse_format
max_tokenspresence_penaltyfrequency_penaltyrepeat_penaltymodel
logit_biasmin_p typical_ptfs_z
mirostat_mode mirostat_tau mirostat_etalogits_processorgrammarlogprobs top_logprobsc y N ) selfllamamessagesr" r# r$ r% r& r' r( r) r* r+ r, r- r. r/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 r: r; r< kwargss N/var/www/html/venv/lib/python3.12/site-packages/llama_cpp/llama_chat_format.py__call__z#LlamaChatCompletionHandler.__call__B s R )) J) ) ) ) ) . ) !) "
#) ( ")) * +) , !-) . /) 0 1) 2 /3) 6 7) 8 9) : ;) < =) > ?) @ A) B >C) D .E) F !G) H $I) L
M) rF r c e Zd Zy)+LlamaChatCompletionHandlerNotFoundExceptionN)r[ r\ r] r? rF rD r` r` n s rF r` c J e Zd ZU i Zded< d ddZd dZ d
dZy) "LlamaChatCompletionHandlerRegistryz%Dict[str, LlamaChatCompletionHandler]_chat_handlersc ` |s || j v rt d| d || j |< y )NzFormatter with name 'z>' is already registered. Use `overwrite=True` to overwrite it.rc
ValueError)r@ name chat_handler overwrites rD register_chat_completion_handlerzCLlamaChatCompletionHandlerRegistry.register_chat_completion_handleru s@ TT%8%8 8 'v-kl
%1D!rF c X || j v r| j |= y t d| d )Nz(No formatter registered under the name 'z'.re )r@ rg s rD unregister_chat_handlerz:LlamaChatCompletionHandlerRegistry.unregister_chat_handler s3 4&& &##D) GvR PQ QrF c
| j | }|S # t $ r4 t d| dt | j j d w xY w)NzInvalid chat handler: z (valid formats: ))rc KeyErrorr` listkeys)r@ rg rh s rD #get_chat_completion_handler_by_namezFLlamaChatCompletionHandlerRegistry.get_chat_completion_handler_by_name sb ..t4L =(.?TEXEXE]E]E_@`?aabc
s =ANF)rg strrh r ri rO rg rt rg rt rX r )r[ r\ r] rc __annotations__rj rl rr r? rF rD rb rb r sN <>N9>
1
1 1
1
1R # rF rb c 4 t j | S r> )rb rr )rg s rD get_chat_completion_handlerry s - / S S rF c d fd }|S )Nc < t j | | S r> )rb rj )frg s rD decoratorz3register_chat_completion_handler..decorator s *,MMdTUVrF )r| r r? rg r} s ` rD rj rj s rF c F e Zd ZU dZded< dZded< dZded< d Zd
ed < y) ChatFormatterResponsea Dataclass that stores completion parameters for a given chat format and
create_chat_completion request.
prompt contains the formatted prompt generated from the chat format and messages.
stop contains the stop token or list of stop tokens to use for the chat format.rt promptNrP r* z$Optional[llama.StoppingCriteriaList]stopping_criteriaFrO
added_special)r[ r\ r] r^ rw r* r r r? rF rD r r s0 W
K,0D
)0>B;B M4 rF r c $ e Zd ZdZ ddZy)
ChatFormattera Base Protocol for a chat formatter. A chat formatter is a function that
takes a list of messages and returns a chat format response which can be used
to generate a completion. The response can also include a stop token or list
of stop tokens to use for the completion.c y r> r? )r@ rB rC s rD rE zChatFormatter.__call__ s
!$rF NrB rH rC r rX r rZ r? rF rD r r s+ 1
$ A$ $
$rF r c b e Zd Z d ddZddddd ddZd dZy)
Jinja2ChatFormatterNc || _ || _ || _ || _ | t | nd| _ t
t j dd j | j | _
y)zAA chat formatter that uses jinja2 templates to format the prompt.NT)loader trim_blocks
lstrip_blocks) template eos_token bos_tokenadd_generation_promptsetstop_token_idsr jinja2
BaseLoader from_string _environment)r@ r r r r r s rD __init__z Jinja2ChatFormatter.__init__ sl !
""%:"#1#=C 4
:$$&
+dmm
$
rF )r" r# r$ r% c $ dd} j j | j j | j |||| }d } j
" d fd }
t
j |
g } t | j g| d S )Nc t | r> )rf )messages rD raise_exceptionz5Jinja2ChatFormatter.__call__..raise_exception s
W% %rF ) rB r r r r r" r# r$ r% c & | d j v S )N)r )tokenslogitsr@ s rD stop_on_last_tokenz8Jinja2ChatFormatter.__call__..stop_on_last_token s bzT%8%888rF T)r r* r r )r rt )r znpt.NDArray[np.intc]r znpt.NDArray[np.single]rX rO ) r renderr r r r rA StoppingCriteriaListr ) r@ rB r" r# r$ r% rC r r r r s ` rD rE z Jinja2ChatFormatter.__call__ s & "")) nnnn +"&"<"< '# *
! *
9,
96L
9
9
!& : : ))chat_formatter_to_chat_completion_handlerr@ s rD to_chat_handlerz#Jinja2ChatFormatter.to_chat_handler s
8>>rF )TN)
r rt r rt r rt r rO r zOptional[List[int]])rB rH r" rI r# rJ r$ rK r% rL rC r rX r )rX r )r[ r\ r] r rE r r? rF rD r r s '+.2
%% % %
$ % ,
%6 IMQU@DLP(
A(
F (
O (
>
(
J(
(
(
T?rF r c | y t | d | d | d D cg c]2 \ }}}|d ||j D cg c] \ }}||d d
c}}d4 c}}}}}d dS c c}}w c c}}}}}w )Nr token_logprobsr< )tokenlogprobbytes)r r r r< )contentrefusal)zipitems)r; r r r< top_token top_logprobs rD )_convert_text_completion_logprobs_to_chatr s 588JHUeLfhpq iA 5B
15'< " 3?2D2D2F
! / ; "+#.!%!
! !
s A' A!A' !A' c
d| v sJ d| d z d| d | d dd| d d d
d t | d d d | d d d
dg| d dS )Nusagechatidchat.completioncreatedr1 r assistantchoicestextroler r;
finish_reasonindexr r; r r objectr r1 r r r )
completions rD _convert_text_completion_to_chatr s j z$''# i(G$ ' )) 4Q 7 ? FjQZF[\]F^_iFj k!+I!6q!9/!J
G$! rF c # K t | D ]w \ }}|dk( r d|d z |d |d ddddid d d gd
d|d z |d |d dd|d d d
d
|d d d ini t |d d d |d d d d gd
y y w)Nr r r r1 r chat.completion.chunkr r r deltar; r r r1 r r r r r r r r; ) enumerater )chunksichunks rD '_convert_text_completion_chunks_to_chatr 7 s f% %
5 6 uT{* w +1 "# "K" %))-
" 5;&7^ Y'-
!+A.?G &uY'7':6'B I%PYJZ[\J]^hJi j%*9%5a%8 %I
%%
s BB Fc : |r
| }t | S | }t | S r> )r r )completion_or_chunksr) r r s rD _convert_completion_to_chatr b s' G[6v>>-A
/
;;rF c
|sj|}d|v sJ d z dz |d z }d|d z d|d |d d d
d |d d d d
|d |d d d d
dgdt |d d d ddg|d d}|S |} d fd } || S )Nr call__0__r r r r r1 r r r r rg argumentsfunctionr typer r r r#
tool_callsr; r r r c 3 , K d}d }d }d }d }| D ] }|rd|d z }|d }|d }dz dz |d z }|d||d d d d
d d d d d gd
d|d z d|d |d d d t |d d d d d |d d d dd |d|d d d ddgd d gd
d}|J d|d z d|d |d d d t |d d d d d |d d d dd |d|d d d ddgd d gd
| |||d||d dd d d d d d d gd
y y y y w)NTr r r r1 r r r r r r r r r; r r r r r1 r r r; r r r r r r r Fr r )r firstid_r r1 tool_idr tool_names rD #_stream_response_to_function_streamzQ_convert_completion_to_chat_function.._stream_response_to_function_stream ss ECGEG ]
5;.C #I .G !'NE - 9C ?%+ MG !"9#*!& *+15,0,7/35926 *"
$ ( %uT{ 2"9#(#3!&w *+15,UV[\eVfghVijtVu,v,0/3095:95Ea5H5P6& 67294>8A=B9=Ma=P4:>29.
)* 3&*" $ ! !D "E *** 5;. 5 $Y / "7^ &'-1(QRWXaRbcdRefpRq(r(,+/,516y1A!1DV1L2" 23.50:4=9>y9I!9LV9T5* %&
/"& }]
~ 7#65;L 5 & " &'-9(,(,+/15.2 &
q)A&)I* '.(2,51;I1Fq1I&1Q-" ' $ !J*U^J_`aJbcmJn o%1+2 (= E
@ G[{ H{
E{ z 36::rF c d d d d ddddddg d d d ddddd d
d d d d d d d d d fd
}|S )Nr r r r r Fr r r r r! r" r# r$ r% r& r' r( r3 r4 r) r* r+ r, r- r. r/ r0 r5 r6 r7 r8 r1 r9 r: r2 r; r< c z ( )||||| } | j | j j d | j d } | j Q| g nt | t r| gn| } t | j t r | j n | j g} | | z } d }!| j | j }!| |d dk( rt || j }||D "cg c] }"d|"d }}"|:t |t r |d
k( s|d k( r|}t |t rd |v r
dd |d id }d }#|t |t rw|u|d d (t (fd
|D d }#|#t d( d |#d d }$ t j j! t# j$ |$ | j } | j2 d*i d| d|d|d|d| d|
d|r| nd d| d| d|
d |d |d |d |d |d!|d"|d#|d$|d%|d&|!d'|d(|}&|#|#d d }'t5 |'|&| S t7 |&| ) S c c}"w # t& $ rs}%| j r$t) t
|% t* j, t j j/ t j0 | j }Y d }%~%d }%~%ww xY w)+N)rB r" r# r$ r% zutf-8Tadd_bosspecialr json_objectverboser r r noneautorg c 3 : K | ] }|d d k( s| ywr rg Nr? .0trg s rD z]chat_formatter_to_chat_completion_handler..chat_completion_handler.. ! KqQz]6-Bd-JK
Tool choice '' not found in tools.
parametersfiler r& r' r( r3 r4 r; r) r* r+ r- r. r/ r0 r5 r6 r7 r8 r1 r9 r r: r2 r) r? ) tokenizer encoder r*
isinstancert rp r _grammar_for_response_formatr dictnextrf
llama_grammar LlamaGrammarfrom_json_schemajsondumps Exceptionprintsysstderrr JSON_GBNFcreate_completionr r )*rA rB r" r# r$ r% r& r' r( r3 r4 r) r* r+ r, r- r. r/ r0 r5 r6 r7 r8 r1 r9 r: r2 r; r< rC resultr rstopr r toolschemaer r rg chat_formatters* @rD chat_completion_handlerzJchat_formatter_to_chat_completion_handler..chat_completion_handler' s L '#
MM ) ,,,
;; " 2Zc5JD6PTD#-fkk4#@FKKv{{mE%>H> > J:A)J55J:
MP JMP MP MP MP MP MP MP . MP MP
!MP& "'MP( )MP* !+MP, -MP. /MP0 1MP2 3MP4 5MP6 7MP8 >9MP: .;MP< /=MP> !?MP@ $AMPD
EMP^ #"rF c N ddl m} |j | dfd }|S )Nr
AutoTokenizerc d_ j | d }t |t sJ t |j
d S )NFr Tr r* r )use_default_system_promptapply_chat_templater rt r r )rB rC r tokenizers rD format_autotokenizerz@hf_autotokenizer_to_chat_formatter..format_autotokenizer sL /4 + 33Hu3M&#&&&$ 3 34
rF r ) transformersr from_pretrained) pretrained_model_name_or_pathr r& r% s @rD "hf_autotokenizer_to_chat_formatterr* s@ + --.KLI
@
rF c . t | }t | S r> )r* r )r) r s rD +hf_autotokenizer_to_chat_completion_handlerr, s 88UVN 4^ DDrF c H t | t sJ d| v sJ t | d t sJ | d }d| v sJ t | d t sJ | d d| v sJ t | d t sJ | d t dd j | dfd }|S )N
chat_templater r Tr r c rg | t j dd } j | }t |gd S )Nr r )rB r r Tr" ) llama_types%ChatCompletionRequestAssistantMessager r )rB rC r r r envr s rD format_tokenizer_configzFhf_tokenizer_config_to_chat_formatter..format_tokenizer_config sh !AA$bH
%I 6d
rF r )r r rt r r )tokenizer_configr r. r5 r r4 r s ` @@@rD %hf_tokenizer_config_to_chat_formatterr7 s & -- - . .. . &7 == =$_5M * ** * &{3S 99 9 -I * ** * &{3S 99 9 -I
' k-
@
, #"rF c 2 t | | }t | S )N)r )r7 r )r6 r r s rD .hf_tokenizer_config_to_chat_completion_handlerr9 s! ;0EN 5^ DDrF c t d| vry | d t k( ry| d t k( s | d t k( ry| d t k( ryy )Nztokenizer.chat_templatechatmlmistral-instructllama-3)CHATML_CHAT_TEMPLATE MISTRAL_INSTRUCT_CHAT_TEMPLATE MIXTRAL_INSTRUCT_CHAT_TEMPLATE LLAMA3_INSTRUCT_CHAT_TEMPLATE)metadatas rD $guess_chat_format_from_gguf_metadatarC sW 0)*.BB *+/MM - .2P P!)*.KK rF c 8 | D ] }|d dk( s |d xs dc S y)z Get the first system message.r systemr r1 r? )rB r s rD _get_system_messagerF * s4 , 6?h &9%+ +, rF c g }| D ]< }|d }||v s
t |d t r|d nd}|j || |f > |S )zMap the message roles.r r N)r rt append)rB role_mapoutputr r r s rD
_map_rolesrK 4 sd
/1F 5v 8 &01CS&I "t
MM8D>73 4
5 MrF c ||g}| |z }t | D ]C \ }\ }}| r|dk( r|xs d} || ||dz z z
}%|r|||z dz ||dz z z
}<||dz z
}E |S )z(Format the prompt with the llama2 style.r r1 r )
system_messagerB sepsep2sepsretr r r ms
rD _format_llama2rV D s
;D
3
C ' 1 ?D' a1f
2A 1tAE{? "C
4'>C'$q1u+5 5C 4#: C JrF c V | |z }|D ] \ }}|r||dz |z |z z
}||dz z
} |S )z2Format the prompt with the add-colon-single style.: :r? rP rB rQ rT r r s rD _format_add_colon_singler[ U sQ 3
C!
g 4$;(3. .C 4#: C
JrF c ||g}| |d z }t | D ]' \ }\ }}|r||dz |z ||dz z z
} ||dz z
}) |S )z/Format the prompt with the add-colon-two style.r rX rM rY rO ) rP rB rQ rR rS rT r r r s rD _format_add_colon_twor] b sp
;D
47
"C ' 1 ?D' 4$;(4A;6 6C 4#: C
JrF c D | }|D ] \ }}|r |||z |z z
}||z
} |S )z1Format the prompt with the no-colon-single style.r? rZ s rD _format_no_colon_singler_ p sB C!
g 4'>C' 'C 4KC
JrF c V | |z }|D ] \ }}|r||dz |z |z z
}||dz z
} |S )z8Format the prompt with the add-colon-space-single style.rX r? rZ s rD _format_add_colon_space_singlera } sQ 3
C!
g 4$;(3. .C 4$; C
JrF c p | dk( rdn| |z dz }|D ]! \ }}|r||dz |z |z dz z
}||dz z
}# |S )z(Format the prompt with the chatml style.r1
r? rZ s rD _format_chatmlrd sd "$".3*>*EC!
g 4$;(3.5 5C 4$; C
JrF c X d}| r|| z
}|D ] \ }}|r||dz dz |z z
}||z
} |S )z*Format the prompt with the chatglm3 style.r1 rc rN r? rZ s rD _format_chatglm3rf sT
C ~ !
g 4$;$w. .C 4KC
JrF c ` t j j t j | S Nr )r
r r r r s rD _grammar_for_jsonri s+ % % 1 1 2 rF c t j j | | S # t $ r}|rt | cY d }~S |d }~ww xY wrh )r
r r r ri )r r fallback_to_jsonr s rD _grammar_for_json_schemarl sF ))::67:SS $W5 5G s # A
AAAAc z | d dk7 ry d| v r#t t j | d | S t | S )Nr r r r )rl r
r ri )r, r s rD r r sI v -/?"' JJx0 17
!11rF c d fd }|S )Nc R t | }t j | | S r> )r rb rj )r| r rg s rD r} z'register_chat_format..decorator s, "KA"N *,MM )
rF )r| r r? r~ s ` rD register_chat_formatrp s rF zllama-2c d}t dd }t | | }t | }|r|j | }t ||dd dz }t | S ) Nz([INST] <>
{system_message}
<>z [INST]z[/INST]userr rP rN r r )r rK rF formatrV r )rB rC _system_template_roles _messagesrP _prompts rD
format_llama2r{ sc
D
{i
8F8V,I(2N)000O ^YVDyPG 00rF r= c t ddd }d}t | | }|j |d d f t d|| }t || S ) Nz,<|start_header_id|>system<|end_header_id|>
z*<|start_header_id|>user<|end_header_id|>
z/<|start_header_id|>assistant<|end_header_id|>
)rE rs r z
<|eot_id|>r r1 r r* )r rK rH r_ r )rB rC rx _sepry rz s rD
format_llama3r s]
?
;EF
D8V,I
f[)401%b)T:G d ;;rF alpacac t dd }d}d}t | }t | | }t |||| }t | S )Nz### Instruction ### Responserr
r ru )r rF rK r] r )rB rC rx r~ _sep2rP ry rz s rD
format_alpacar sL
(N
CF D E(2N8V,I#NItUKG 00rF qwenc t dd }t | xs d}d}|j | }t | | }|j |d d f d}t ||| }d }t
||
S ) N<|im_start|>user<|im_start|>assistantrr You are a helpful assistant.#<|im_start|>system
{system_message}rt r r z
<|endoftext|>r} )r rF rv rK rH rd r ) rB rC rx rP system_templatery r~ rz r s rD format_qwenr s
)5L
MF(2T6TN+JN8V,I
f[)401 D ^Y=G E e <>
{system_message}
<>
<|prompter|>
<|assistant|>rr r rt r ru r rF rv rK rH r_ r rB rC rw rx r~ rP ry rz s rD format_oasst_llamar ( sv
H
~
AF D(2N%,,N,KN8V,I
f[)401%niFG 00rF z
baichuan-2c d}t dd }d}t | }|j | }t | | }|j |d d f t ||| }t
| S ) N{system_message}zzrr r1 rt r ru r r s rD format_baichuan2r 8 w
*
'3C
DF
D(2N%,,N,KN8V,I
f[)401%niFG 00rF baichuanc d}t dd }d}t | }|j | }t | | }|j |d d f t ||| }t
| S ) Nr zzrr r1 rt r ru r r s rD format_baichuanr H r rF openbuddyc d}t dd }d}|}t | | }|j |d d f t ||| }t | S )Na You are a helpful, respectful and honest INTP-T AI Assistant named Buddy. You are talking to a human User.
Always answer as helpfully and logically as possible, while being safe. Your answers should not include any harmful, political, religious, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
You can speak fluently in many languages, for example: English, Chinese.
You cannot access the internet, but you have vast knowledge, cutoff: 2021-09.
You are trained by OpenBuddy team, (https://openbuddy.ai, https://github.com/OpenBuddy/OpenBuddy), you are based on LLaMA and Falcon transformers model, not related to GPT or OpenAI.
User Assistantrr rc r ru r rK rH r[ r )rB rC r rx r~ rP ry rz s rD format_openbuddyr X s^
O v
5F D$N8V,I
f[)401&~y$GG 00rF zredpajama-incitec t | }t dd }d}d}|}t | | }|j |d d f t ||| }t || S )Nzzrr rc r r} )rF r rK rH r[ r ) rB rC r rx r~ _stoprP ry rz s rD format_redpajama_inciter n sh
*(3O
yG
4F D E$N8V,I
f[)401&~y$GG e <F D E$N8V,I
f[)401&~y$GG e <
?F D5O8V,I
f[)401& 4HG 00rF z open-orcac d}d}d}d}d}|j | }t | t t || }|j |d d f t ||| }t
|| S ) Nr a You are a helpful assistant. Please answer truthfully and write out your thinking step by step to be sure you get the right answer. If you make a mistake or encounter an error in your thinking, say so out loud and attempt to correct it. If you don't know or aren't sure about something, say so clearly. You will act as a professional logician, mathematician, and physicist. You will also act as the most appropriate type of expert to answer any particular question or solve the relevant problem; state which expert type your are, if so. Also think of any particular named expert that would be ideal to answer the relevant question or solve the relevant problem; name and act as them, if appropriate.)r r z<|end_of_turn|>
r rt r r} )rv rK r r rH ra r ) rB rC r rP rolesrQ stop_strry rz s rD format_open_orcar s
)O B
"E
CH$++>+JN8T#eU*;%<=I
eAh%&,^YLG h ??rF mistrallitec t dd }d}d}t | }|j | }t | | }|j |d d f t ||| }t
| S ) Nr z
<|assistant|>rr rN z <|system|>{system_message}rt r ru r )rB rC rx r~ r rP ry rz s rD format_mistralliter su
~1F
GF D:O(2N$++>+JN8V,I
f[)401%niFG 00rF zephyrc d}t | }|j | }t dd }d}t | | }|j |d d f t ||| }t
|| S ) N<|system|>
{system_message}rt z <|user|>
z<|assistant|>
rr r r r} rF rv r rK rH rd r rB rC r rP rx r~ ry rz s rD
format_zephyrr sw
O(2N$++>+JN
|/@
AF D8V,I
f[)401 ^Y=G d ;;rF pygmalionc d}t | }|j | }t dd }d}t | | }|j |d d f t ||| }t
|| S ) Nz<|system|>{system_message}rt <|user|>z <|model|>rr rc r r} r r s rD format_pygmalionr sv
7O(2N$++>+JN
z[
9F D8V,I
f[)401 ^Y=G d ;;rF r; c d}t | }|j | }t dd }d}t | | }|j |d d f t ||| }t
|| S ) Nr rt r r rr r r r} r r s rD
format_chatmlr sx
O(2N$++>+JN
)5L
MF D8V,I
f[)401 ^Y=G d ;;rF r< c d}|}d}| D ]K }|d dk( r$|d t |d t r |d|d z z
}/|d dk( s8|d >|d|d z |z z
}M |dz
}t || S )
Nr r1 r rs r z[INST] r z [/INST]r} )r rt r )rB rC eosr* r r s rD format_mistral_instructr
s
C D
F < FOv % ".79-s3 i')"44 4F
V_
+ 0B0N j79#55; ;F< jF T ::rF chatglm3c d}t | }|j | }t dd }d}t | | }|j |d d f t ||| }t
|| S ) Nr rt r r rr r r r} )rF rv r rK rH rf r r s rD format_chatglm3r " sv
O(2N$++>+JN
z_
=F D8V,I
f[)401 ~y$?G d ;;rF openchatc d}t | }|j | }t dd }d}t | | }|j |d d f t ||| }t
|| S ) Nz {system_message}<|end_of_turn|>rt zGPT4 Correct User: z'<|end_of_turn|>GPT4 Correct Assistant: rr z<|end_of_turn|>r r} r r s rD format_openchatr 3 s|
8O(2N$++>+JN
".WF D8V,I
f[)401 ^Y=G d ;;rF saigac d}t ddd }t | | }d}|D ]' \ }}|r||j || z
} |d| d z
}) |d
z
}t |j S ) Nz{role}
{content}rs botrE )rs r rE r1 r r rc zbotru )r rK rv r strip)rB rC _message_templaterx ry rz r r s rD format_saigar G s
3
v5
:F8V,IG" &
g (//T7/K KG TF"~ %G & xG
88rF gemmac t | }|dk7 rt j d t dd }d}t | | }|j |d d f t
d|| }t || S )
Nr1 zB`role='system'` messages are not allowed on Google's Gemma models.zuser
zmodel
rr z
r )rP rB rQ r} )rF r debugr rK rH r_ r )rB rC rP rx r~ ry rz s rD format_gemmar ] sz
)2N P
.:R
SF D8V,I
f[)401%R)QUVG d ;;rF functionaryr r r r r r r r r! c %&'( d% d<(fd (d=(fd 'd>d?'(fd
& d@ dA%&fd
}||D cg c] }|d dk( s |d }}|t |t r|n|d } |||| } |k|t | dk( r[ | j dBi d | d
z d |d |d
|d| d|
d| dddgd|d|d|d|d|d|d|d|d |d |d |} t | | S |t |t rG|d k( rBd!} | j | | d"# } | d$ d d% } | j d& d' d d' }| | z | z } nBt |t r |d(k7 r| d
z } n't |t r| d)|d* d
z } |d* }n| d
z } d }!|xs g D ] }"|"d* |k( s |"d+ }! n |xs g D ] }|d dk( s |d d* |k( s|d d+ }! n |! t | j , 5 t j t j |! }#t j j t j t j |! | j - }t |# d d d nXt | j , 5 t j j t j" | j - }d d d | j dBi d | dddgdd"d |d|d |d |d
|d| d|
d|d|d|d|d|d|d|d |d |} d/| v sJ t |t sJ | d"u sJ | j r t | t | d$ d d% t% j& d0| d1 z d2| d3 | d dd4d || d$ d d% d5|d|| d$ d d% d5d6gd7t) | d$ d d8 d9d:g| d/ ; S c c}w # 1 sw Y `xY w# t $ r}$| j rt d. t |$ t | j , 5 t j j t j" | j - }d d d n # 1 sw Y nxY wY d }$~$d }$~$ww xY w# 1 sw Y xY w)CNA chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. The assistant calls functions with appropriate input when necessaryc d|z }d| v r| d j d d }|S | j d dk( r%| j di }
||dz | }d | d
S | j d d k( rQ| j d i }d
}|j D ] \ } }
|
|dz | } || d| d| dz
}" ||dz z
}|S d| v r&dj | d D cg c] } d| d
c} S | j dd S c c} w N z$ref/r r arrayr r zArray<>r
propertiesz{
rX ,
}enum | "anysplitgetr joinparam indent_level shared_defsindentref_namer item_typer
nested_schemanested_param_name nested_paramnested_param_type
enum_valuegenerate_type_definitions rD r z:functionary_chat_handler..generate_type_definition Z $ U? V}**3/H O
YYv
'
)IIgr*E0 q8H+VII;a( (
YYv
(
* <4J !M3=3C3C3E
/!<$< ,"2K%! hb!2 326G5HL
Vc\ )M
u_::5= QZ:,a 0 QR R 99VU+ + R
C>c
d|z }d}| j D ]d \ }}|| d| dz
}|j d dk( r| ||| z
}n-d|v r)|dj |d D cg c] }d | d
c} z
}|d
z
}f |S c c}w Nr r1 type z = r r r r r z;
r r r r r r shared_definitionsdef_namedef_propertiesr r s rD generate_shared_definitionsz=functionary_chat_handler..generate_shared_definitions $ (3(9(9(; ( $Hn VHE(3"? ? !!&)X5"&>"L+' " >)"ejj9G9OP:q A&P' "
% ' ( "! Q #
B c N d}|d| dz
}i }| D ]5 }|j di }|j |j di 7 | |d z
}| D ] }|d }|j dd }|j di }|j d
g }|d | d z
}|d
| dz
}|j di j D ]@ \ } }
|
j dd } |
d| } | |v rd nd}
|d| d z
}|d| |
d| dz
}B |dz
} |dj | z
}|S )NH// Supported function definitions that should be called when necessary.
namespace {
r $defsr rg descriptionr1 requiredz // rc z type = (_: {
r rM ?z // z rX r z }) => any;
z}} // namespace {}
r updater rv r" namespacer r r r
function_namer required_params
param_namer param_description
param_typeoptional_indicatorr r s rD generate_schema_from_functionsz@functionary_chat_handler..generate_schema_from_functions s W Jyk11 ! CH! lB7J % %jnnWb&A B C -.@!DD! )H $V ,M",,}b9K! lB7J (nnZ
z
user:
z
r assistant:
z
assistant to=z
assistant to=r r r zUnsupported role: )lenrf )msg tool_calls rD message_to_strzXfunctionary_chat_handler..prepare_messages_for_inference..message_to_str sE 6{h&"3y>"2"55V
*v}'F }CI7GrJJV
*#/E'O(- +C,?+@C NCSSUVV +C,?+@CCV&y>). (Y (8??V +y>-/S2H )#i.)9 _I]^dIeHffijmn}j~ @K kL jM MS T T$+ *3+?+G*HCP_L`alLmKnntuu!S(S\1B-Ca-G%($& n "/y.?s9ZCXYdCeBffl mmn ^+& )#i.)9<< #5c&k]!CDDrF r1 )r$ z(llama_types.ChatCompletionRequestMessage)rH r2 "ChatCompletionRequestSystemMessager3 r )
rB r" r$ all_messagesr r r& r$ SYSTEM_MESSAGEr s
rD prepare_messages_for_inferencez@functionary_chat_handler..prepare_messages_for_inference s
HJ >>!+I)+T
>>! : ). $ #F|z 9 ! ,
: : ~
)Gv*,71B$.wv.?"@ ') ! !9&!A BC (
( ) = = $
! EF ww|Ds+DEEGF Es D2 D7r r r r r r& r' r( r3 r4 r) r* zuser:r r- r. r/ r0 r5 r6 r7 r8 r1 r9 r: r r rc F)r r* r) r r .r r z to=functions.rg r disabler MFailed to parse function body as JSON schema, falling back to default grammarr r r r r r r r r r; r r r r z Dict[str, llama_types.JsonType]r rN rX rt r rN rX rt r" rX rt NN)rB rH r" 3Optional[List[llama_types.ChatCompletionFunctions]]r$ rK r? )r rt r# r r r r r r r
json_schema_to_gbnfr
r r r r r r r2 CreateChatCompletionResponser ))rA rB r" r# r$ r% r& r' r( r3 r4 r) r* r, r- r. r/ r0 r5 r6 r7 r8 r1 r9 r: rC r* r r completion_or_completion_chunksr completion_text
new_prompt
function_bodyr grammar_textr r) r r r s) @@@@rD functionary_chat_handlerr< r s< : zN ,. ,>A , ,B"" H JN@DUF@UFFUF >UFn
27V$4<:;UT*%V V %k37K[=T ,Hi
GF )"3s9~7J*A%*A*A +
E>+
#+
+
+
+
+
+
6"+
"+
.+
0+
*+
+
( +
& +
&!+
" #+
$ .%+
& '+
'* ++JSYZZ =#&=F+B-DU-D-DU.
%Y/26:'--c226s;
o-4
M3 'MV,Ce^
M4 (}V/D.ES II
%f-
e^
M O F } , $\ 2M <: %$z*:6*Bm*S ,\ :M
'
>
$ , @ @JJ}- (44@@!55djj 6OP !MM A l#
$* $EMM
: #00<< '' = G
*A)@)@ **v
* * *
*
* * * * * ** ,* &* * $ * "!*" "#*$ %*& *'*J, j mS )) ) U?? }}
j
j#A&v./ 3 3 Jt$ $ 9%! ' # -%/ %:1%=f%E& #0$.(5-7 -B1-Ef-M) # $ FjQZF[\]F^_iFj k!-+
2 != s W@
$
$
}}c a'
>
'44@@!++ !MM A
s`
N4N4 O BN9O ::Q$9O>O Q!8Q :Q
Q
Q Q Q!$Q.zfunctionary-v1zfunctionary-v2c
456789:;<=> d8 j }t |d sJ d ddlm} d|j j
v rd>d} d } d
} d } d} d }!n
d
>d6d5d7d4 dV| }$|dk( st dk( r>dk( r } n7} |$dz
}$ j d\i d|$d d d d d
d! d"| d#d$d%d&d'd(d)d*d+d,d-|}% d.u r"|%d/ d d0 j |%d/ d d0< t |% 1 S fd2=
fd39d4}&g g }(}'d})45679=>fd5}* d.ur |*||$6 S >dk( r_t
|t r
|dk( rd7 g}+nDt
|t r.|$ |d8 d9z
}$!}+|d8 }|'j | =| }n|$}$d7 g}+ 9|$|+|: },|,d/ d d0 }-|)|,d; d< z
}) |$vr| |-vr|)|,d; d<< t |, 1 S | |$vr| |-v r|$|-j | d=| d7z z
}$|'j |-j | d> d d> j! =|'d> } 9|$!|: },|)|,d; d< z
})|(j |,d/ d d0 j! n|(j |-j! n}t
|t rq|$|d8 d74 z
}$|d8 }|'j | =| }75g}+ 9|$|+|: },|,d/ d d0 }-|)|,d; d< z
})|(j |-j! nt
|t r|dk( r d }4}+ 9|$|+|: },|,d/ d d0 }-|)|,d; d< z
})|-j! }.|.d?k( r|$dz
}$n1|-j! }|$| d@z
}$|'j | =| }67g}+ 9|$|+|: },|,d/ d d0 }-|)|,d; d< z
})|.d?k( r|-j# dA r|&|-d t dA z
}&|-j# dB r|&|-t dB z
}&n|&|-z
}&|&j }&dC|-v sdD|-v rw|-j# dA r |-d t dA j! }/n?|-j# dB r |-t dB j! }/n|-j! }/|$|/ dEz
}$nlnm|(j |-j! |$|-j! z
}$d } 9|$|+|: },|)|,d; d< z
})dC|,d/ d d0 v s
dD|,d/ d d0 v r|$dEz
}$nnd;,v sJ t |' t |( k( sJ g }0t% |'|( D ]x \ }}1|0j dFd4j' t) dG D 2cg c]6 }2t+ j, t. j0 t. j2 z 8 c}2 z d||1dHdI z i }3t |0 dkD r"|0|3dJ< n|0d d d8 |0d d dK dH|3dL< |)|,d; d<< t5 j6 dM|,dN z dO|,dP |,d+ dt9 |,d/ d dQ dR|&d4k( rd n|&dS|3t |0 dkD rdJnd"dTg|,d; U S c c}#w c c}2w )]Nr hf_tokenizerznPlease provide a valid hf_tokenizer_path from https://huggingface.co/meetkai when initializing the Llama classr r z<|START_OF_FUNCTION_CALL|>v1z<|END_OF_SYSTEM|>z<|END_OF_USER|>z<|END_OF_ASSISTANT|>z<|END_OF_FUNCTION_RESULT|>z<|END_OF_FUNCTION_CALL|>v2z
<|recipient|>z<|from|>z<|stop|>z <|content|>c d|z }d| v r| d j d d }|S | j d dk( r%| j di }
||dz | }d | d
S | j d d k( rQ| j d i }d
}|j D ] \ } }
|
|dz | } || d| d| dz
}" ||dz z
}|S d| v r&dj | d D cg c] } d| d
c} S | j dd S c c} w r r r s rD r z@functionary_v1_v2_chat_handler..generate_type_definition
r r c
d|z }d}| j D ]d \ }}|| d| dz
}|j d dk( r| ||| z
}n-d|v r)|dj |d D cg c] }d | d
c} z
}|d
z
}f |S c c}w r r r s rD r zCfunctionary_v1_v2_chat_handler..generate_shared_definitions. r r c L d}|d| dz
}i }| D ]5 }|j di }|j |j di 7 | |d z
}| D ] }|d }|j dd }|j di }|j d
g }|d | d z
}|d
| dz
}|j di j D ]? \ } }
|
j dd } |
d| } | |v rd nd}
|d | d z
}|| |
d| dz
}A |dz
} |dj | z
}|S )Nr
r r r r
r rg r r1 r z// rc r r r rM r rX r z }) => any;
z}} // namespace {}r r s rD r zFfunctionary_v1_v2_chat_handler..generate_schema_from_functions? s W Jyk11 ! CH! lB7J % %jnnWb&A B C -.@!DD! 'H $V ,M",,}b9K! lB7J (nnZ assistant
<|recipient|>Fr! )rH r2 r' r> r$ ) rB r% versionr" r$ r% r( r r suffixr) r s rD r* zFfunctionary_v1_v2_chat_handler..prepare_messages_for_inferencea s HJ & >>!+I"+M
$ ## BB %/Mi/X
"{f'< ## BB % > -2 $(#'<:#= !%Z 0 ! : : ~
)Gv*,71B$.wv.?"@ ') ! !9&!A BC (
( ) d?#F7F
" " 6 6|e 6 T
? s
Er r r all
<|content|>r r& r' r( r3 r4 r) r* r- r. r/ r0 r5 r6 r7 r8 r1 r9 r: Fr r r c d }xs g D ] }|d | k( s |d } n xs g D ] }|d dk( s |d d | k( s|d d } n t j 5 t j t j
| }t j j t j t j
| }t | d d d |S # 1 sw Y S xY w# t $ r}j rt d t | t j 5 t j j t j j }d d d n # 1 sw Y nxY wY d }~S d }~ww xY w)Nrg r r r r, r. r ) r r r
r5 r
r r r r r r )
r# r: r r r; r: r r" rA r$ s
rD get_grammarz3functionary_v1_v2_chat_handler..get_grammar s|
!R H =0 ( 6
KR DF|z)d:.>v.F-.W $Z 0 >
'
>
$ , @ @JJ}- (44@@!55djj 6OP l#
$& '
$& }}c a'
>
'44@@!++U]] A
sO C. A9C!C. !C+&C. +C. . F 78F/:E2) F2E; 7FF c t t j j di d| ddddddd|d d
d d d
d
d d d dd| }|S )Nr r& r' r( r3 r4 r) r* r- r. r/ r0 r5 r6 r7 r8 r1 r9 r: r? )r r2
Completionr )r r* r: r r/ rA r9 r- r3 r8 r6 r7 r1 r. r0 r) r& r5 r( r' r4 s rD r z9functionary_v1_v2_chat_handler..create_completion s " " #E # #
'
$
&
"2
#4
.
,
*
*!
" #
$ "2%
& '
2 rF r1 c 3 V K dk( sJ d d\ }}t |t r||d d z
} |d }g}dj t d D cg c]6 }t j
t j t j z 8 c} } ||| }
d} d } |
D ] }
| rT| d
d d | z d
|d dddgi}n
d|d ddi}t j d|
d z d|
d |
d d d d d d|dg d} | (d
d d | z d
d |
d d d j ddgi}n dd |
d d d j di}t |
d d d j d kD st j d|
d z d|
d |
d d t |
d d d d d d|dg t j d
d z d|
d |
d d | d
ndd d d d d d d g y t |t r|d k( r
d } d }} ||| }
d} |
D ] }
| |
d d d z
} |
d }|
d }| j }|d k( r1|d!z
}t j d|z
d |dd d"ddd d d#g$ n|| d%z
} | }dj t d D cg c]6 }t j
t j t j z 8 c} } | d
|d | z d
|dddgi}nd|ddi}t j d|z d|
d d t |
d d d d"d d|dg g} ||| }
|d k( rbd} d&g d}}}t! |
D ] \ }}
| |
d d d z
} |r|j# |
d d d j d' |j% dj | ra|j' t | d kD rZt j d|z d||
d d t |
d d d d"|j' d ddg t | d kD rZd}n;|
d d d dk( r-d }|j# |
d d d j d' t | d k( s't |
d d d d kD s@t j d|z d||
d d t |
d d d d"|d kD r |
d d d n|
d d d j) ddg d(| v sd)| v rx| j+ d* r | d t d* j }n?| j+ d+ r | d t d+ j }n| j }|| d&z
}n~t j d|z |
d |dd i d d,d#g$ y d} |
D ] }
| |
d d d z
} t |
d d d j d kD s7| (d
|d z d
d |
d d d j ddgi}n dd |
d d d j di}t j d|z d||
d d t |
d d d d d d|dg || j z
}d } ||| }
| dj |
D
cg c]
}
|
d d d c}
z
} d(| v sd)| v r
| |d&z
}|d-z
}n2t j d|z d|
d d | d
ndd d d d d d d g y
y y c c}w c c}w c c}
w w).Nr@ z!Streaming for v1 is not supportedr3 rg rc r1 r r* r: Tr r call_r r r r# r r r r r1 r )r r; r r Fr r r; r r r allrG r r r
<|content|>
<|from|>assistant
<|recipient|>rN <|from|> assistant<|from|>assistant
<|from|>assistant
<|from|> assistant
r* r )r r r rangerandomchoicestring
ascii_lettersdigitsr2 "CreateChatCompletionStreamResponserstripr# r rt r r rH
startswithpoplstripendswith) r$ r" r# r chunk_id
chunk_createdr: stopsr r r r8 r r func_call_dict
tool_indexr
stop_sequencebufferis_endr cleaned_completion_text
CONTENT_TOKEN
FROM_TOKENRECIPIENT_TOKEN
STOP_TOKENr rI rE s rD generate_streamingz:functionary_v1_v2_chat_handler..generate_streaming si
$C CC", - mT * v./r- A AF!-"78G ,EggNSTViX v 3 3fmm CDXG +&ugVJ OE# P
( (-.*1G*;,60=f0E571& !"
+ * ,(5f(=-/. * &HH!E$K/ 6 %i 0 #Gn *+,0,0/3*" '5*"
! " "E$$)*&-&7(2,016y1A!1DV1L1S1S1U-"
' &N ($().y)9!)v+FF! "MM% *:1*=f*E*K*KC*P Q , 7 7 H ( &
&)&kAo*5*X*X+1H+(5&+Gn 234]^cdm^nop^qr|^}4~4? 45q5 16i0@0CF0K5:95Ea5H4:6228&( 2*
%&)" # A5 p -?./A*334KL6E ?3'>#?"?7 #eg 4 - 5 56N O6E @3'?#@"@7 #eg 4 7F6K6K6M 3 %<$==_"`` * L L % 0"'.$1#: ./-/045; !"%
')O!+ , '5+;A+>v+FF uY / 26 : A A CDqH $ 0$05?29G2C4>8<=B9=Ma=P4:>228&( 9.
)* 3&2"" %4045:95Ea5H,26**0&( 6&2" #."P"P#)H#4'>(5&+Gn 234]^cdm^nop^qr|^}4~487;2* /=2*%&
)" # 9, Z o 3 3 55F "G!2 %E7"J $rwwBLMy)!,V4M( O -?./A + "FF"a
* L L % 0#:$1"'. ./ ,1+< )5-<0404379=6: .&!"% . O 1H
+[ YfD NsO A\);\C \)5D;\)0;\ +E \) A \)\)2E \) additional_special_tokensr rt r# r ra r r rH replacer r rb r r rW rX rY rZ r[ r\ r2 r6 r )?rA rB r" r# r$ r% r& r' r( r3 r4 r) r* r, r- r. r/ r0 r5 r6 r7 r8 r1 r9 r: rC r% r END_SYSTEM_TOKENEND_USER_TOKENEND_ASSISTANT_TOKENEND_FUNCTION_RESULT_TOKENSTART_FUNCTION_CALL_TOKENEND_FUNCTION_CALL_TOKENr* r r r7 r function_callsfunction_bodiesrq rp re r r8 r rk r r: r function_call_dictrl rm rn ro r) r r r r rI rE s? ` ` ` `````` `````````` @@@@@@@@@@@rD functionary_v1_v2_chat_handlerr s < zN I > xwx +#y'='='W'WW.* 4$@!$@!"< )
%
,. ,>A , ,B"" L JN@D(.
=
@=
=
%=
G =
> =
&
=
~
27V$4<:;UT*%V %k37K[=T
"
+)Wi
F
)"3s9~7J d?&D D ( (F*A%*A*A +
+
#+
+
+
+
+
+
+
"+
.+
0+
*+
+
( +
& +
&!+
" #+
$ .%+
& '+
'* U?/ :1=fELLN
,I 6q 9& A ++JSYZZ B 8 G&("ONA A F U!9MRX
d?---62I 23M40 6 7
f8M7NcRR/ -f 5
%%m4%m4 23*&ugVJ ( 3A 6v >O G!45H!I I *7-_D;L
7#$782:fMM *7- @#++45Q79R %%#))*CDRH"MSSU &nR&89 .!(?
"Z%89L%MM! &&z)'? -.]6 2 32m_EE -f 5
%%m4%m4#Z0 .!w
#-Y"7":6"B!Z%89L%MM! &&'<'<'>?M3/MV4K "G )E!2 %E7"J '1&;A&>v&FO%G)<=P)QQ%$3$9$9$;M$- "44(7(=(=(?
]O="AA&--m<"-m"< ,j 9E!2 %E7"J '1&;A&>v&FO%G)<=P)QQ%$-*334KL #7V#>U:V9V'W WG*334LM #"c/&::::FH
,/,P (M= !gg &+2Y ! #MM&*>*> *N O ' -%2!
2
z?Q 3="<0 'qM* 5f =!+A z!:;!G7"?3 4E
7 /077
4(($ y) W% I*U^J_`aJbcmJn o ++2b=4g -
69_q5H\f W%#
M WPs -
[2;[2 ;[7c ^ e Zd ZU dZded< dZdddZd dZd d dZddddd d
d d d
dg ddddddd
dddddddddd d dZ e
d d Z e
d d Z e
d!d Z
e d" d#d Zy)$Llava15ChatHandlerzA chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.rS DEFAULT_SYSTEM_MESSAGEa% {% for message in messages %}{% if message.role == 'system' %}{{ message.content }}{% endif %}{% if message.role == 'user' %}{% if message.content is string %}
USER: {{ message.content }}{% endif %}{% if message.content is iterable %}
USER: {% for content in message.content %}{% if content.type == 'image_url' and content.image_url is string %}{{ content.image_url }}{% endif %}{% if content.type == 'image_url' and content.image_url is mapping %}{{ content.image_url.url }}{% endif %}{% endfor %}{% for content in message.content %}{% if content.type == 'text' %}{{ content.text }}{% endif %}{% endfor %}{% endif %}{% endif %}{% if message.role == 'assistant' and message.content is not none %}
ASSISTANT: {{ message.content }}{% endif %}{% endfor %}{% if add_generation_prompt %}
ASSISTANT: {% endif %}c < dd l m} | _ | _ | _ t _ d _ d _ t j j | st d| t j 5 j j j j d }|t d| | _ fd} j j# | d d d fd} j j# | y # 1 sw Y *xY w)Nr z Clip model path does not exist: r, zFailed to load clip model: c t j 5 j j j d d d y # 1 sw Y y xY wNr, )r r
_llava_cpp clip_freeclip_ctxr s rD r z.Llava15ChatHandler.__init__..clip_free
s: +DLLA =OO--dmm<= = =s &AAc t j 5 j , j j j d _ d d d y # 1 sw Y y xY wr )r r _last_image_embedr llava_image_embed_freer s rD last_image_embed_freez:Llava15ChatHandler.__init__..last_image_embed_free
sP ' =
2))5OO::4;Q;QR-1D*
2
2
2s 9AA#)llama_cpp.llava_cpp llava_cppclip_model_pathr r r _exit_stackr _last_image_hashospathexistsrf r clip_model_loadr r callback)r@ r r r r r r s ` rD r zLlava15ChatHandler.__init__
s / . #$;
04 ww~~o. ??P QR R
#DLL
9 166t7K7K7R7R7TVWXH #>>O!PQQ $DM
=
% %i 0 1 2
!!"78) 1 1s
=A,DDc $ | j | S r> ) _load_image)r@ image_urls rD
load_imagez Llava15ChatHandler.load_image
s **rF c
H | j 0| j $t | | j k( r | j S t | j 5 | j 3| j
j
| j d | _ d | _ | j
j | j |t j t | z j t | t | }|| _ t | | _ |cd d d S # 1 sw Y y xY wr )r r hashr r r r !llava_image_embed_make_with_bytesr ctypesc_uint8r# from_buffer bytearray)r@ image_bytesn_threads_batchembeds rD _embed_image_bytesz%Llava15ChatHandler._embed_image_bytes
s " " .%%1[!T%:%::)) )
#DLL
9 %%166t7M7MN)-&(,%OOEE
#k"22?? k* K
E &+D "$($5D !! s
B;DD!Nr r r r r Fr r r r r! r c
2 | j J t | } | dk( r1| j %t j d| j g|z }| j | } t
dd j | j }!|!j |d |j |j g |j |j g }"| j |"| }#| j rt |"t j" |j$ |j& j) |#D ] \ }$}%|$dk( r |j* |%j- d d
d }&|j. t1 |& z |j2 kD r6t5 d |j. t1 |& z d
|j2 |j6 |& | j9 |% }'| j; |'|j< j> }(|j. |(j@ jB z |j2 kD rAt5 d |j. |(j@ jB z d
|j2 tE jF |j. })tE jH |) }*tK | j 5 | jL jO |jP |(|jR |* d d d d|jT |j. |)jV |)jV |_ |jT d |j. jY }+||d dk( r t[ | }||D ,cg c] },d|,d }},|:t] |t^ r |dk( s|dk( r|}t] |t` rd|v r
dd|d id}d }-|t] |t` rw|u|d d 2tc 2fd|D d }-|-t5 d2 d |-d d }. td jf ji tk jl |. |j } |jr d3i d |+d |d |d | d |
d!| d"| r| nd d#| d$|
d%|d&|d'|d(|d)|d*|d+|d,|d-|d.|d/|d0|d1|}0|-|-d d }1tu |1|0| S tw |0| 2 S # 1 sw Y xY wc c},w # tn $ rt}/|j r$t t_ |/ t j" td jf j td jp |j }Y d }/~/d }/~/ww xY w)4Nr1 rE r Tr/ )rB r r r r r utf8Fr zPrompt exceeds n_ctx: z > r, r r r r r r r rg c 3 : K | ] }|d d k( s| ywr r? r s rD r z.Llava15ChatHandler.__call__..N r r r r r r r r& r' r( r3 r4 r; r) r* r+ r- r. r/ r0 r5 r6 r7 r8 r1 r9 r: r2 r r? )#>#J>>!4+F+F H ((2
0
+d&&
' "&&e&&(9':;&e&&(9':;
224D
<< $SZZ (
!!# & .LE5' LL(% >>CK/+%++-?$0 #f+1M0NcR]RWR]R]R_Q`a
6" "ooe 4 // U=Q=Q=a=ab>>ENN$>$>> N$0 %..B\B\1\0]]`alafalalan`op enn5!>>&1+DLLA OO::
BD >!' 9 .> !15>>299; &?6+Bm+S2?CG !*
' (E $-- '=F+B + -.6]3J &
f 5! #;-! z*62DKEKTRD| =6K!LMM*%l3F
'44EEJJv&
F 7u 6 6
#
&.\4
"
.
0
*
(!
" &#
$ &%
& '
( .)
* +
, "-
0 Z(0I7/
++?OO{ &L
==#a&szz2'44@@!++U]] A
s+ 3S S-(>S2 S* 2 U/;A)U**U/c | j d r)dd l}|j | j d d }|S dd l}|j
j
| 5 }|j }|cd d d S # 1 sw Y y xY w)Nzdata:r ,r )r_ base64 b64decoder urllib.requestrequesturlopenread)r r r urllibr| s rD r z Llava15ChatHandler._load_image} ss ( **9??3+?+BCK ! '' 2
#a ffh "
#
#
#s A66A?c g }| D ] }|d dk( s |d |d D ]g }t |t sd|v s|d dk( s"t |d t r d|d v r|j |d d T|j |d i |S )Nr rs r r r url)r r rH )rB r r r s rD r z!Llava15ChatHandler.get_image_urls s "
HGv&(9%- &y 1 HG!'40Vw5F"6?k9 *7;+? F$)W[-A$A * 1 1'+2Fu2M N * 1 1'+2F G H
H rF c dd}g }| }|rn ||| \ }}|H|F|dkD r|j d|d | f |j d|| f ||t || z d }n|j d|f d}|rn|S )Nc b t | D ]! \ }}| j | }|dk7 s ||fc S y)Nr r3 )r find)ssubstrsr substrposs rD
find_firstz?Llava15ChatHandler.split_text_on_image_urls..find_first s< &w /
" 6ffVn"9 6M
" rF r r r r1 )r rt r List[str])rH r# )r r r r remainingr r s rD r z+Llava15ChatHandler.split_text_on_image_urls s FH
:6FC1=7 %%vy#&?@!!;
1
">? %cC
1
,>&>&@ A !!69"56 rF c
dd l }ddlm} ddlm} m}
ddlm} | | |
} | j | D
cg c] }
t |
t r|
d n|
}}
g }|D ]3 }
||
j | }|j t | 5 |D
cg c] }
|j |
| s|
}}
t | dk( r't! d| d| d t# j$ | t | d
kD r't! d | d | d t# j$ | |\ }t || j& } || j( } | |||t+ t, t |d f | |t+ t, t |d f |
|( | |||||t+ t, t |d f | d }n t. j0 j3 || } | dd|i|S # t $ r t d w xY wc c}
w c c}
w )Nr )Path)hf_hub_download HfFileSystem)validate_repo_idzrLlama.from_pretrained requires the huggingface-hub package. You can install it with `pip install huggingface-hub`.rg zNo file found in z that match z
Available Files:
r zMultiple files found in z
matching )repo_idfilename subfolder local_dirlocal_dir_use_symlinks cache_dirT)r r r r r r local_files_onlyr r? )fnmatchpathlibr huggingface_hubr r huggingface_hub.utilsr ImportErrorlsr r relative_torH rt r# rf r
r parentrg r r r r r )clsr r r r r rC r r r r r hffsr files file_listrel_pathmatching_files
matching_filer
model_paths rD r( z"Llava15ChatHandler.from_pretrained sL E > !~ (
'tT 2DL <
" ,DDz--g6H S] + , ,5X4h8W$XX ~ ! # #G9L
C%%)ZZ %:$;=
~ " *7):hZ H%%)ZZ%6$79
* ]+223
&++ 5dD 19=#95dD 19=
( !##'= uS$_ 5yA!%J i:J
&
I
Ys G% G=H2H%G:T)r rt r rO )r rt rX r )r )r r r rN r )rB rH )r rt r r )Nr N)r rt r rS r &Optional[Union[str, os.PathLike[str]]]r z Union[bool, Literal['auto']]r r rC r rX z'Llava15ChatHandler')r[ r\ r] r rw r r r r rE staticmethodr r r classmethodr( r? rF rD r r \
s h M
F$9L+< IMQU@DLP 02 "
$("%#& # ! ! #@D0415#'&*CuP uP A uP
F uP O
uP >uP JuP uP uP uP uP uP uP . uP !uP"
#uP( ")uP* +uP, !-uP. /uP0 1uP2 3uP4 5uP6 7uP8 9uP: >;uP< .=uP> /?uP@ !AuPB $CuPF
GuPn # # $ .
=A?E<@
Q
Q
Q
: Q
!= Q
:
Q
Q
Q
Q
rF r c e Zd ZdZy)ObsidianChatHandlera< {% for message in messages %}{% if message.role == 'system' %}<|im_start|>system
{{ message.content }}
###
{% endif %}{% if message.role == 'user' %}<|im_start|>user
{% if message.content is string %}{{ message.content }}{% endif %}{% if message.content is iterable %}{% for content in message.content %}{% if content.type == 'image_url' and content.image_url is string %}{{ content.image_url }}{% endif %}{% if content.type == 'image_url' and content.image_url is mapping %}{{ content.image_url.url }}{% endif %}{% endfor %}{% for content in message.content %}{% if content.type == 'text' %}{{ content.text }}{% endif %}{% endfor %}{% endif %}###
{% endif %}{% if message.role == 'assistant' %}<|im_start|>assistant
{{ message.content }}###
{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
{% endif %}Nr[ r\ r] r r? rF rD r r s ( rF r c e Zd ZdZy)MoondreamChatHandlera {% for message in messages %}{% if message.role == 'user' %}{% if message.content is iterable %}{% for content in message.content %}{% if content.type == 'image_url' %}{% if content.image_url is string %}{{ content.image_url }}
{% endif %}{% if content.image_url is mapping %}{{ content.image_url.url }}
{% endif %}{% endif %}{% endfor %}{% for content in message.content %}{% if content.type == 'text' %}Question: {{ content.text }}
{% endif %}{% endfor %}{% endif %}{% if message.content is string %}Question: {{ message.content }}
{% endif %}{% endif %}{% if message.role == 'assistant' %}Answer:{{ message.content }}
{% endif %}{% endfor %}{% if add_generation_prompt %}Answer:{% endif %}Nr r? rF rD r r D s " rF r c e Zd ZdZdZy)Llava16ChatHandlerzA chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions. a {% for message in messages %}{% if message.role == 'system' %}{{ message.content }}{% endif %}{% if message.role == 'user' %}{% if message.content is iterable %}{% for content in message.content %}{% if content.type == 'image_url' %}{% if content.image_url is string %}{{ content.image_url }}
{% endif %}{% if content.image_url is mapping %}{{ content.image_url.url }}
{% endif %}{% endif %}{% endfor %}{% for content in message.content %}{% if content.type == 'text' %}{{ content.text }}{% endif %}{% endfor %}{% endif %}{% if message.content is string %}{{ message.content }}{% endif %}{% endif %}{% if message.role == 'assistant' %}{{ message.content }}{% endif %}{% endfor %}{% if add_generation_prompt %}Answer:{% endif %}Nr[ r\ r] r r r? rF rD r r n s } % rF r c e Zd ZdZdZy)NanoLlavaChatHandlerzAnswer the questionaM {% for message in messages %}{% if message.role == 'system' %}<|im_start|>system
{{ message.content }}<|im_end|>{% endif %}{% if message.role == 'user' %}<|im_start|>user
{% if message.content is string %}{{ message.content }}{% endif %}{% if message.content is iterable %}{% for content in message.content %}{% if content.type == 'image_url' and content.image_url is string %}{{ content.image_url }}{% endif %}{% if content.type == 'image_url' and content.image_url is mapping %}{{ content.image_url.url }}{% endif %}{% endfor %}{% for content in message.content %}{% if content.type == 'text' %}{{ content.text }}{% endif %}{% endfor %}{% endif %}<|im_end|>{% endif %}{% if message.role == 'assistant' %}<|im_start|>assistant
{{ message.content }}<|im_end|>{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
{% endif %}Nr r? rF rD r r s 3( rF r c e Zd ZdZdZy) Llama3VisionAlphaChatHandlerNa {% for message in messages %}<|start_header_id|>{% if message.role == 'user' %}user<|end_header_id|>
{% if message.content is iterable %}{% for content in message.content %}{% if content.type == 'image_url' %}{% if content.image_url is string %}{{ content.image_url }}{% endif %}{% if content.image_url is mapping %}{{ content.image_url.url }}{% endif %}{% endif %}{% endfor %}{% for content in message.content %}{% if content.type == 'text' %}{{ content.text }}{% endif %}{% endfor %}{% endif %}{% if message.content is string %}{{ message.content }}{% endif %}{% endif %}{% if message.role == 'assistant' %}assistant<|end_header_id|>
{{ message.content }}{% endif %}<|eot_id|>{% endfor %}{% if add_generation_prompt %}<|start_header_id|>assistant<|end_header_id|>
{% endif %}r r? rF rD r r s "& rF r c e Zd ZdZdZy)MiniCPMv26ChatHandlerr a {% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}<|im_start|>system
You are a helpful assistant.<|im_end|>
{% endif %}<|im_start|>{{ message['role'] }}
{% if message['content'] is iterable %}{% for content in message['content'] %}{% if content.type == 'image_url' %}{% if content.image_url is string %}{{ content.image_url }}{% endif %}{% if content.image_url is mapping %}{{ content.image_url.url }}{% endif %}{% endif %}{% endfor %}{% for content in message['content'] %}{% if content.type == 'text' %}{{ content.text }}{% endif %}{% endfor %}{% endif %}{% if message['content'] is string %}{{ message['content'] }}{% endif %}<|im_end|>
{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
{% endif %}Nr r? rF rD r r
s ; rF r zchatml-function-callingc - d} t t j ddg t j j | } ||D cg c] } d| d }} |:t |t r |dk( s|dk( r|}t |t rd |v r
dd |d id}t | t r| d
gn
| r| d
gz nd
g} |%t |t r|dk( s|t | d k( r| j |g d d
} |
|
d dk( r t |
}t | j dOi d| d|d|d|d| d|
d| d| d|d|d|d|d |d |d |d |d |d!|d"|d#|r|nd | $ S t |t r|d d -t -fd%|D d } | t d&- d' | j ||d d
} | d(- d)z
} t j j# t% j& | d d* | j( + } | j dOi d| d|d|d|d| d|
d| d| d|d|d|d|d |d |d |d |d |d!|d"|}"t1 -|"| S t |t r|dk( sJ d-j3 |D cg c] } d.| d d d/ c} }#d0|# d1}$d2|# d1}%| j ||d d
} | j dOi d| dd d|d|d| d|
dd3dd4gdd d|d|d|d |d |d |d |d |d!|d"t j j |$| j( + }"|"}&|&d5 d d6 }'d7|'v rt | j dOi d| d8z d|d|d|d| d|
d| dd
gd#|r|nd dd d|d|d|d |d |d |d |d |d!|d"t j j |%| j( + | $ S |'t d( d -t -fd9|D d } | sg }(g })| | d(- d)z
} t j j# t% j& | d d* | j( + } | j dOi d| d|d|d|d| d|
dd3d| dd d|d|d|d |d |d |d |d |d!|d"|}"t5 t6 j8 |" }"|(j; |" |)j; - | |"d5 d d6 z
} | d1z
} | j dOi d| d|d|d|d| d|
dd3d| dd d|d|d|d |d |d |d |d |d!|d"t j j |%| j( + }*t5 t6 j8 |* }*|*d5 d d6 t d( d -t -fd:|D d } | t |( d;k( rd<-|(d d5 d d6 d=ini }+d>|&d? z d@|&dA |&d dBd t= |&d5 d d# dCd t? tA |)|( D ,-&cg c]- \ },\ }-}&dDdE|, dEz |-z dEz |&d? z d|-|&d5 d d6 d=dF/ c}&}-},dG|+dHgtC dI |(D tC dJ |(D tC dK |(D dLdMS t dN c c} w # t* $ rf}!t j j t j, | j( + }| j( rt/ d, t/ |! Y d }!~!d }!~!ww xY wc c} w # t* $ rf}!t j j t j, | j( + }| j( rt/ d, t/ |! Y d }!~! d }!~!ww xY wc c}&}-},w )PNa {% for message in messages %}<|im_start|>{{ message.role }}
{% if message.role == 'system' %}{{ message.content }}{% if tool_calls %}
You have access to the following functions:
{% for tool in tools %}
functions.{{ tool.function.name }}:
{{ tool.function.parameters | tojson }}
{% endfor %}
You can respond to users messages with either a single message or one or more function calls.
To respond with a message begin the message with 'message:', use the following format:
message:
To respond with one or more function calls begin the message with 'functions.:', use the following format:
functions.:
{ "arg1": "value1", "arg2": "value2" }
functions.:
{ "arg1": "value1", "arg2": "value2" }{% endif %}<|im_end|>
{% endif %}{% if message.role == 'user' %}{{ message.content }}<|im_end|>
{% endif %}{% if message.role == 'assistant' %}{% if message.content and message.content | length > 0 %}{% if tool_calls %}message:
{% endif %}{{ message.content }}<|im_end|>
{% endif %}{% if 'tool_calls' in message %}{% for tool_call in message.tool_calls %}functions.{{ tool_call.function.name }}:
{{ tool_call.function.arguments }}{% endfor %}<|im_end|>
{% endif %}{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
{% endif %}htmlxml)
autoescape undefinedr r r r rg r r T)rB r$ r r r r r r& r' r( r3 r4 r) r* r- r. r/ r0 r5 r6 r7 r8 r1 r9 r: r; r c 3 : K | ] }|d d k( s| ywr r? r r r s rD r z*chatml_function_calling..
s" MdtJ'7'?9'LT Mr zTool with name 'z' not found in toolsr r r r r. r z "functions.z:"z0root ::= functions | "message:"
functions ::= rc z2root ::= functions | "<|im_end|>"
functions ::= FrY r r r z message:
c 3 : K | ] }|d d k( s| ywr r? r s rD r z*chatml_function_calling..S " Q$4
+;F+Cy+PQr c 3 : K | ] }|d d k( s| ywr r? r s rD r z*chatml_function_calling.. r r r r# r r r r r r r rO r r )r r r )r r r; r c 3 8 K | ] }d |v r|d d nd yw)r rq r Nr? r r s rD r z*chatml_function_calling.. s7 ) # #j0 #7+,?@ ) c 3 8 K | ] }d |v r|d d nd yw)r
prompt_tokensr Nr? r s rD r z*chatml_function_calling.. s0 %" =Dz. s0 $" 'K mT *v /F"M& 1 K dC
| &*T\N
" {C ([F-B = u:?")) "& *
&?6+Bm+S2?CG* #E # #
'
$
&
"2
#4
.
,
*
*!
" #
$ "2%
& '
( *2 t)
, /
6 +t$
+F3 Me Mt
< / {:N OP P")) "& *
Jyk-- #00AA
4
+L9:EMM B G 7u 6 6
#
"
.
0
*
(
&
&!
" #
$ .%
& '
* 4 +V
k3 'K6,AA AZZDIJD[j)&12" 7JN )*" 0
)*" 0
%
% " & F 3522
U * , & $ " "! " # $ *% & **66 %u}} 7
' . 8LJ i #F +DD* #E # #
,
'
$
#^
*2 t
"2
#4
.
,
*!
" *#
$ %
& "2'
( &22>>/ ? )
0 3
: S &()I Q%QSW XD BD +-
9+S1 1F
'44EEJJtJ / =>
F $;5#:#: $ $ '$ $ $
$ $
$ $ $ $ "2$ #4$ .$ $ , $ * $ *!$" #$$ "2%$& '$ * $(446J$
3 4 ! ( ( 3 *95a8@ @F dNF.u.. '
$
"2 #4 . , * *!" #$ "2%& &22>>/ ? 'H. K @ @(KH +A.v6s<7H7JKIQ%QSWDM p ; 1$ %!,Q !:1!=f!E"
" :d++'!), ( &2 I*U^J_`aJbcmJn o +#' ?H #$9; G? ' ' !;#:Iz '.$%aS('+"+', #&'& #-T"2 '3
)3,51;I1Fq1I&1Q-" ' ( -) < &) ) '2
) & "% %&1% " !$ $&1$ !E4
4
l G
HHy
t #00<< '' = G }}c a J Kd
'44@@!++U]] A ==g !H
D'sF Y4;AY9 ;[+9A[0 42]"9 [(A[##[(0 ] 9A]] rv ru )r; z(Optional[llama_types.CompletionLogprobs]rX z"llama_types.ChatCompletionLogprobs)r zllama_types.CompletionrX zllama_types.ChatCompletion)r r rX z)Iterator[llama_types.ChatCompletionChunk]rs )r aUnion[llama_types.CreateCompletionResponse, Iterator[llama_types.CreateCompletionStreamResponse]]r) rO rX zZUnion[llama_types.CreateChatCompletionResponse, Iterator[llama_types.ChatCompletionChunk]])r rt r r' r) rO )r r rX r )r) Union[str, os.PathLike[str]]rX r )r) r( rX r r )r6 Dict[str, Any]r rO rX r )r6 r) r rO rX r )rB Dict[str, str]rX rS )rB rH rX rt )rB rH rI r* rX List[Tuple[str, Optional[str]]])
rP rt rB r+ rQ rt rR rt rX rt )rP rt rB r+ rQ rt rX rt )r rO )FT)r rt r rO rk rO )r, z/llama_types.ChatCompletionRequestResponseFormatr rO r )rB z.list[llama_types.ChatCompletionRequestMessage]rC r rX r )4rA rG rB rH r" rI r# rJ r$ rK r% rL r& rM r' rM r( rN r3 rM r4 rM r) rO r* rP r, rR r- rQ r. rM r/ rM r0 rM r5 rM r6 rN r7 rM r8 rM r1 rS r9 rU r: rV rX zLUnion[llama_types.ChatCompletion, Iterator[llama_types.ChatCompletionChunk]])8rA rG rB rH r" rI r# rJ r$ rK r% rL r& rM r' rM r( rN r3 rM r4 rM r) rO r* rP r, rR r- rQ r. rM r/ rM r0 rM r5 rM r6 rN r7 rM r8 rM r1 rS r9 rU r: rV r; rW r< rQ rX rY )t
__future__r r r r
r dataclassesrX rZ
contextlibr typingr r r r r r
r r r
r r jinja2.sandboxr numpynp numpy.typingnptllama_cpp.llamarA llama_cpp.llama_typesr2 llama_cpp.llama_grammarr
_loggerr _utilsr r r> CHATML_BOS_TOKENCHATML_EOS_TOKENr? MISTRAL_INSTRUCT_BOS_TOKENMISTRAL_INSTRUCT_EOS_TOKENr@ rA r r r` rb ry rj dataclassr r r r r r r r r r* r, r7 r9 rC rF rK rV r[ r] r_ ra rd rf ri rl r rp r{ r r r rv r r r r r r r r r r r r r r r r r r r< r r r r r r r Llama3VisionAlphar r& r? rF rD r@ s
"
8 + / 5
a "A " # "z !l
0 0 f ) B
$H $ C?- C?L6'2& 0(
@(
.(
` <<
<
<"n;n;n;
n;bR#!R# R#j #? 2E#?E E #'-#$-# -# -#d #'E$E E E.<
<
%
#BILTW"
#B
IL
#B IL TW
#B
IL
#B
IL
#B
IL
#B IL BF :> 2D 2
2$ i 1< 1 1 1 ! 1 i
<<
<
<
< !
< h
1<
1
1
1
1 f
=<
=
=
=
= h 1< 1 1 1 1 m$ 1< 1 1 1 % 1 l# 1< 1 1 1 $ 1 j! 1< 1 1 1 " 1 k"1<11 1 #1* () =< = = = * = h =<== = =* g
1<
1
1
1
1 g
1<
1
1
1
1 k"@<@@ @ #@4 m$ 1< 1 1 1 % 1 h
<<
<
<
<
< k" << < < < # < h
<<
<
<
<
< ();<;; ; *;( j!
<<
<
<
< "
< j!<<<< < "<& g 9<99 9 9* g <<<< < <( "-0 EIMQ<@HL ,.QU $ ! " <@,03b b<b Bb K b
: b F
b b b b b b
b *b O b b !b" #b$ %b& 'b( )b* +b, -b. /b0 :1b2 *3b6 R7b 1bJ ""23!"23 EIMQ<@HL ,.QU $ ! " <@,03
<
B
K
:
F
*
O
!
" #
$ %
& '
( )
* +
, -
. /
0 :1
2 *3
6 R7
4 4
D m
m
`
5, 5p'- 'T-+ -`4- 4n.#5 .d 1 # . #L "";< EIMQ<@HL ,.QU $ ! " <@,0 #"&7wI wI<wI BwI K wI
: wI F
wI wI wI wI wI wI
wI *wI O wI wI !wI" #wI$ %wI& 'wI( )wI* +wI, -wI. /wI0 :1wI2 *3wI4 5wI6 7wI:;wI =wIrF