j>:dZddlmZmZmZmZmZmZmZddl m Z ddl Z ddl m Z ddlmZddlmZmZmZdd lmZmZedddddddfd ed eed eed eedeedeedeedeedeeedefdZeddddddfd ed eed eed eedeedeedeedeedefdZeddddfd ed eed eed eedeedeedefdZeddfd ed eed eed eedef dZefd ed eedefdZde fdZdefde dddedeed eedef dZde deeeeffd Z d+de deeeefd!ed eefd"Z d,de d#ed$ed%ed&eeeefd'eeef d(Z!de deeeefd)eege"ffd*Z#y)-z= A logger that logs training activity to Weights and Biases. )DictAnyTupleCallableListIOOptional) ModuleTypeN)Language)SimpleFrozenList) dict_to_dot dot_to_dictmatcher_for_regex_patterns)setup_default_console_loggerLoggerT project_nameremove_config_valuesmodel_log_intervallog_dataset_direntityrun_name log_best_dirlog_latest_dirlog_custom_statsreturnc  t tjtjfdddtdtdt t tttfgdft gdfff f d } | S)aCreates a logger that interoperates with the Weights & Biases framework. Args: project_name (str): The name of the project in the Weights & Biases interface. The project will be created automatically if it doesn't exist yet. remove_config_values (List[str]): A list of values to exclude from the config before it is uploaded to W&B. Defaults to []. model_log_interval (Optional[int]): Steps to wait between logging model checkpoints to the W&B dasboard. Defaults to None. log_dataset_dir (Optional[str]): Directory containing the dataset to be logged and versioned as a W&B artifact. Defaults to None. entity (Optional[str]): An entity is a username or team name where you're sending runs. If you don't specify an entity, the run will be sent to your default entity, which is usually your username. Defaults to None. run_name (Optional[str]): The name of the run. If you don't specify a run name, the name will be created by the `wandb` library. Defaults to None. log_best_dir (Optional[str]): Directory containing the best trained model as saved by spaCy, to be logged and versioned as a W&B artifact. Defaults to None. log_latest_dir (Optional[str]): Directory containing the latest trained model as saved by spaCy, to be logged and versioned as a W&B artifact. Defaults to None. log_custom_stats (Optional[List[str]]): A list of regular expressions that will be applied to the info dictionary passed to the logger. Statistics and metrics that match these regexps will be automatically logged. Defaults to None. Returns: LoggerT: Logger instance. nlpr stdoutstderrrNc t t| rj_ rt  dddt t ttff fd }d fd }||fS)NrrdatasetpathnametypeinfocTt|t|t|yN) _log_scores_log_model_artifact_log_custom_stats)r( match_statrrunwandbs ]/home/spazio/kingdom_calls/work5/transenv/lib/python3.12/site-packages/spacy_loggers/wandb.pylog_stepz7wandb_logger_v5..setup_logger..log_stepJs( t $ tS2D E eT: 6cjrtddrtddjyN model_bestmodelr$ model_last_log_dir_artifactjoin)rrr0sr1finalizez7wandb_logger_v5..setup_logger..finalizeOs>!%% !'%  JJLr3rN) r _setup_wandbr/r&r:r rstrr)rrr r2r<r.r/rrrrrrrrrr0s @@r1 setup_loggerz%wandb_logger_v5..setup_logger7s}00@A    !5   %EIIN  O))  78DcN3 7 7  $!!r3 _import_wandbsysrr rrrrr?r) rrrrrrrrrr@r0s ````````` @r1wandb_logger_v5rDsxH OE'*jjszz*" *"!#*":<*" xc3h($./"d(1CC D*"*"X r3c  t tjtjfdddtdtdt t tttfgdft gdfff f d }|S)aCreates a logger that interoperates with the Weights & Biases framework. Args: project_name (str): The name of the project in the Weights & Biases interface. The project will be created automatically if it doesn't exist yet. remove_config_values (List[str]): A list of values to exclude from the config before it is uploaded to W&B. Defaults to []. model_log_interval (Optional[int]): Steps to wait between logging model checkpoints to the W&B dasboard. Defaults to None. log_dataset_dir (Optional[str]): Directory containing the dataset to be logged and versioned as a W&B artifact. Defaults to None. entity (Optional[str]): An entity is a username or team name where you're sending runs. If you don't specify an entity, the run will be sent to your default entity, which is usually your username. Defaults to None. run_name (Optional[str]): The name of the run. If you don't specify a run name, the name will be created by the `wandb` library. Defaults to None. log_best_dir (Optional[str]): Directory containing the best trained model as saved by spaCy, to be logged and versioned as a W&B artifact. Defaults to None. log_latest_dir (Optional[str]): Directory containing the latest trained model as saved by spaCy, to be logged and versioned as a W&B artifact. Defaults to None. Returns: LoggerT: Logger instance. rr rr rNc t|||\t| rj_ rt  dddt t ttff fd }d fd }||fS)Nr"r#r$r(cJ|t|t|yr*r+r,r(console_log_steprr/r0s r1r2z7wandb_logger_v4..setup_logger..log_step$ T " t $ tS2D Er3cxrtddrtddjyr5r9)console_finalizerrr0sr1r<z7wandb_logger_v4..setup_logger..finalizesD!%% !'%    JJLr3r= rr>r/r&r:r rr?r)rrr r2r<rMrJr/rrrrrrrrr0s @@@r1r@z%wandb_logger_v4..setup_loggers.J . **   !5   %EIIN  O))  F8DcN3 F F   &!!r3rA) rrrrrrrrr@r0s ```````` @r1wandb_logger_v4rOgsxB OE'*jjszz-" -"!#-":<-" xc3h($./"d(1CC D-"-"^ r3c ttjtjfdddtdtdt t tttfgdft gdffffd }|S)auCreates a logger that interoperates with the Weights & Biases framework. Args: project_name (str): The name of the project in the Weights & Biases interface. The project will be created automatically if it doesn't exist yet. remove_config_values (List[str]): A list of values to exclude from the config before it is uploaded to W&B. Defaults to []. model_log_interval (Optional[int]): Steps to wait between logging model checkpoints to the W&B dasboard. Defaults to None. log_dataset_dir (Optional[str]): Directory containing the dataset to be logged and versioned as a W&B artifact. Defaults to None. entity (Optional[str]): An entity is a username or team name where you're sending runs. If you don't specify an entity, the run will be sent to your default entity, which is usually your username. Defaults to None. run_name (Optional[str]): The name of the run. If you don't specify a run name, the name will be created by the `wandb` library. Defaults to None. Returns: LoggerT: Logger instance. rr rr rNct|||\t|  r j_ rt  dddt t ttff fd }dfd }||fS)Nr"r#r$r(cJ|t|t|yr*rHrIs r1r2z7wandb_logger_v3..setup_logger..log_steprKr3c4jyr*r;rMr0sr1r<z7wandb_logger_v3..setup_logger..finalize   JJLr3r=rN)rrr r2r<rMrJr/rrrrrrr0s @@@r1r@z%wandb_logger_v3..setup_loggers.J . **   !5   %EIIN  O))  F8DcN3 F F  !!r3rA)rrrrrrr@r0s`````` @r1wandb_logger_v3rWsv6 OE'*jjszz" "!#":<" xc3h($./"d(1CC D""@ r3c ttjtjfdddtdtdt t tttfgdft gdffffd }|S)aCreates a logger that interoperates with the Weights & Biases framework. Args: project_name (str): The name of the project in the Weights & Biases interface. The project will be created automatically if it doesn't exist yet. remove_config_values (List[str]): A list of values to exclude from the config before it is uploaded to W&B. Defaults to []. model_log_interval (Optional[int]): Steps to wait between logging model checkpoints to the W&B dasboard. Defaults to None. log_dataset_dir (Optional[str]): Directory containing the dataset to be logged and versioned as a W&B artifact. Defaults to None. Returns: LoggerT: Logger instance. rr rr rNct|||\t | rt dddttt t ff  fd }d fd }||fS)Nrr#r$r(cJ|t|t|yr*rHrIs r1r2z7wandb_logger_v2..setup_logger..log_step$rKr3c4jyr*rTrUsr1r<z7wandb_logger_v2..setup_logger..finalize)rVr3r=)rr>r:r rr?r) rrr r2r<rMrJr/rrrrr0s @@@r1r@z%wandb_logger_v2..setup_loggers}.J . ** 3 ;O   O))  F8DcN3 F F  !!r3rA)rrrrr@r0s```` @r1wandb_logger_v2r]su* OE'*jjszz" "!#":<" xc3h($./"d(1CC D""4 r3c ttjtjfdddtdtdt t tttfgdft gdffffd }|S)aCreates a logger that interoperates with the Weights & Biases framework. Args: project_name (str): The name of the project in the Weights & Biases interface. The project will be created automatically if it doesn't exist yet. remove_config_values (List[str]): A list of values to exclude from the config before it is uploaded to W&B. Defaults to []. Returns: LoggerT: Logger instance. rr rr rNct|||\t |dtttt ff fd }d fd }||fS)NrZr(c.|t|yr*)r+)r(rJr0s r1r2z7wandb_logger_v1..setup_logger..log_stepMs T " t $r3c4jyr*rTrUsr1r<z7wandb_logger_v1..setup_logger..finalizeQrVr3r=)rr>r rr?r) rrr r2r<rMrJrrr0s @@r1r@z%wandb_logger_v1..setup_loggerCs].J . **  3 ;O  %8DcN3 % !!r3rA)rrr@r0s`` @r1wandb_logger_v1rb3sp OE'*jjszz" "!#":<" xc3h($./"d(1CC D"( r3cT ddl}ddlm}m}m}|S#t$r t dwxYw)Nr)initlogr;zThe 'wandb' library could not be found - did you install it? Alternatively, specify the 'ConsoleLogger' in the 'training.logger' config section, instead of the 'WandbLogger'.)r0rdrer; ImportError)r0rdrer;s r1rBrBZs9   *)   N   s'r0rr projectc|jj}t|}|D]}||=t|}|j |||d}|S)NT)rgconfigrreinit)ri interpolaterrrd) r0rrgrrri config_dotfieldr/s r1r>r>js\ZZ # # %FV$J% u   $F **WVF4* PC Jr3r(c|}|d}|d}|d}|jd|i|r6|j|jDcic] \}}d|| c}}t|tr|j|yyycc}}w)Nscore other_scoreslossesloss_)reitems isinstancedict)r0r(rorprqkvs r1r+r+zs W N+ h 7E"#  II&,,.A$!Qqc{A~A B lD ) IIl # * BsB r/c|t|rq|jdr_|d|zdk(rS|ddk7rJt||dd|jzd|d|dd|dd |d t|d dk(rd nd gyyyyyy)N output_pathstepr pipeline_ checkpointzepoch epochz step latestro checkpointsbest)r%r&r'metadataaliases)getr:idmax)r0r(r/rs r1r,r,s   $((="9F|00A5$v,!:K!m,$svv-%! gvd6l^D "&w-3tM7J3KA3N"NTV ;L5#: r3r%r&r'rrcx|j|||}|j|||j||y)N)r'r)r&)r)Artifactadd_dir log_artifact)r0r%r&r'rrdataset_artifacts r1r:r:s@~~d~IT- '9r3matchercr|5|jD]!\}}||s|j||i#yyr*)rsre)r0r(rrvrws r1r-r-s? JJL "DAqqz 1a&! "r3r*)NN)$__doc__typingrrrrrrr typesr rCspacyr spacy.utilr utilrrrrrr?intrDrOrWr]rbrBr>r+r,r:boolr-r3r1rsBAA 'FF7 '7&8(,%) ""&$(,0RRs)R! Rc] R SM R sm R3-RSMRtCy)R Rp'7&8(,%) ""&$(RRs)R! Rc] R SM R sm R3-RSMR Rp'7&8(,%) " ==s)=! =c] = SM = sm = =F'7&8(,%) 11s)1! 1c] 1  1l:J9K$$-1#Y$ $N  z  (!&6&8       SM  s)    $z $$sCx.)A $ )-   4S> " ! 8*.#' :  :  :  :  : tCH~& : d3i :" "%d38n5"@H#PT@U"r3