+sg LddlmZddlZddlmZmZerddlmZGddZy)) annotationsN) TYPE_CHECKINGAny)SentenceTransformercVeZdZdZdZ d d dZd dZd dZed dZ y) SentenceEvaluatorzl Base class for all evaluators Extend this class and implement __call__ for custom evaluators. c d|_d|_y)a Base class for all evaluators. Notably, this class introduces the ``greater_is_better`` and ``primary_metric`` attributes. The former is a boolean indicating whether a higher evaluation score is better, which is used for choosing the best checkpoint if ``load_best_model_at_end`` is set to ``True`` in the training arguments. The latter is a string indicating the primary metric for the evaluator. This has to be defined whenever the evaluator returns a dictionary of metrics, and the primary metric is the key pointing to the primary metric, i.e. the one that is used for model selection and/or logging. TN)greater_is_betterprimary_metric)selfs e/var/www/html/venv/lib/python3.12/site-packages/sentence_transformers/evaluation/SentenceEvaluator.py__init__zSentenceEvaluator.__init__s"&"Ncy)a This is called during training to evaluate the model. It returns a score for the evaluation with a higher score indicating a better result. Args: model: the model to evaluate output_path: path where predictions and metrics are written to epoch: the epoch where the evaluation takes place. This is used for the file prefixes. If this is -1, then we assume evaluation on test data. steps: the steps in the current epoch at time of the evaluation. This is used for the file prefixes. If this is -1, then we assume evaluation at the end of the epoch. Returns: Either a score for the evaluation with a higher score indicating a better result, or a dictionary with scores. If the latter is chosen, then `evaluator.primary_metric` must be defined N)r model output_pathepochstepss r __call__zSentenceEvaluator.__call__s2 rc|s|S|jDcic]\}}|dz|z|}}}t|dr5|jj|dzs|dz|jz|_|Scc}}w)N_r )itemshasattrr startswith)r metricsnamekeyvalues r prefix_name_to_metricsz(SentenceEvaluator.prefix_name_to_metrics9s|N=D]]_MzsE4#:#U*MM 4) *43F3F3Q3QRVY\R\3]"&*t/B/B"BD NsA1c<|jj||y)N)model_card_dataset_evaluation_metrics)r rrs r store_metrics_in_model_card_dataz2SentenceEvaluator.store_metrics_in_model_card_dataAs 44T7Crc|jj} |jd}|d|}t j dd|S#t$rY"wxYw)z Returns a human-readable description of the evaluator: BinaryClassificationEvaluator -> Binary Classification 1. Remove "Evaluator" from the class name 2. Add a space before every capital letter EvaluatorNz([a-z])([A-Z])z \g<1> \g<2>) __class____name__index IndexErrorresub)r class_namer)s r descriptionzSentenceEvaluator.descriptionDs^^^,,  $$[1E#FU+Jvv'DD   sA AA)Nr/) rrrstrrintrr1returnzfloat | dict[str, float])rdict[str, float]rr0r2r3)rrrzdict[str, Any]r2None)r2r0) r( __module__ __qualname____doc__rrr r$propertyr.rrr rr sa #bd ( 7: JM [^ ! 6DEErr) __future__rr+typingrr)sentence_transformers.SentenceTransformerrrrrr r<s"" %MIEIEr