sg[dddlmZmZddlmZmZmZddlmZm Z m Z ddl m Z ddl mZ ddlmZmZmZdd lmZdd lmZdd lmZdd lmZd Ze j>dZ defdZ!GddeZ"GddeZ#GddeZ$GddeZ%GddeZ&y#eef$r eZdZdZY`wxYw))ArgumentParser Namespace)AnyListOptional)Pipelineget_supported_taskspipeline)logging)BaseTransformersCLICommand)BodyFastAPI HTTPException)APIRoute) BaseModel) JSONResponse)runTcy)N)xys P/var/www/html/venv/lib/python3.12/site-packages/transformers/commands/serving.pyrr"s Fztransformers-cli/servingargsct|j|jr |jnd|j|j|j }t ||j|j|jS)z~ Factory function used to instantiate serving server from provided command line arguments. Returns: ServeCommand N)taskmodelconfig tokenizerdevice) r rrr r!r" ServeCommandhostportworkers)rnlps rserve_command_factoryr(+sX  YY JJdjjD{{..{{  C TYY 4<< @@rceZdZUdZeed<y)ServeModelInfoResultz" Expose model information infosN)__name__ __module__ __qualname____doc__dict__annotations__rrrr*r*;s Krr*c8eZdZUdZeeed<eeeed<y)ServeTokenizeResultz Tokenize result model tokens tokens_idsN) r,r-r.r/rstrr1rintrrrr3r3Cs" Ic##rr3ceZdZUdZeed<y)ServeDeTokenizeResultz! DeTokenize result model textN)r,r-r.r/r6r1rrrr9r9Ls Irr9ceZdZUdZeed<y)ServeForwardResultz Forward result model outputN)r,r-r.r/rr1rrrr<r<Ts Krr<ceZdZedefdZdedededefdZ dZ d Z e d d e d d fdede fdZe d d e d d e d d fdeede de fdZe d d fdZy )r#parserc|jdd}|jdttd|jdtdd |jd td d |jdtdd |jdtd|jdtd|jdtd|jdtdd |j t y)z Register this command to argparse so it's available for the transformer-cli Args: parser: Root parser to register command-specific arguments servezFCLI tool to run inference requests through REST and GraphQL endpoints.)helpz--taskzThe task to run the pipeline on)typechoicesrBz--host localhostz$Interface the server will listen on.)rCdefaultrBz--porti"z Port the serving will listen to.z --workersr zNumber of http workersz--modelz%Model's name or path to stored model.)rCrBz--configz,Model's config name or path to stored model.z --tokenizerzTokenizer name to use.z--devicezSIndicate the device to run onto, -1 indicates CPU, >= 0 indicates GPU (default: -1))funcN) add_parser add_argumentr6r r7 set_defaultsr()r? serve_parsers rregister_subcommandz ServeCommand.register_subcommand]s(( b)  !! ')2 " !!(kPv!w!!(dIk!l!!+CIa!b!!)# > 8^^F+F%V4 4 8gs1v%67 7 8s%A$<A$ A!AA!!A$)r,r-r. staticmethodrrMr r6r7rcrr]rboolr^rr_r`rrrr#r#\s>N>>rs/&&??(*44("0$(!   6 7 A A 9$)$IH8-H8{ ^$*I %*! *s$BB/.B/