bj:ddlZddlZddlZddlmZddlmZddlm Z m Z m Z m Z m Z mZmZmZmZmZddlmZmZmZmZmZddlmZddlmZdd lmZdd lm Z m!Z!m"Z"d d l#m$Z$e rdd l%m&Z&dZ'dZ( d3dejRejTddddeede+de de dedeeff dZ,dddede-de+de+de+de+d ee.d!ee.d"ee de e.e fgdffd#Z/d$Z0ddd%e d&e e.e-fde gee-e e.e-ffffd'Z1ddd(e dgee$fd)e ee$gee$fd*e+fd+Z2d,ee e.e fefddd-e e.e fddfd.Z3d/ee dgdfde dgdffd0Z4d1eeddfd2Z5y)4N)Path) default_timer) IO TYPE_CHECKINGAnyCallableDictIterableListOptionalTupleUnion)Config Optimizerconstantfix_random_seedset_gpu_allocator)Printer)Errors)ConfigSchemaTraining)loggerregistryresolve_dot_names)ExampleLanguagez model-bestz model-last)use_gpustdoutstderrnlpr output_pathr r!r"returncvtd}jj}|ddt|dd|dd}|dk\r |r t |t j |dt}|d |d g} t|| \} } |d |d } |d } |d}t|d|d}fd}|d}|d}tt| | |dt| | |d|d|d|d|d||| }t|j|jdj dz|r&|j|jd|dz|r&|j|jd|dz|j|jdj"dzj%| 5|||\}}ddd |D][\}}}|Gj%| 5t'||ddd||t)t*z |d!<||nd]  |d# j2rj5j26|j|j7d$t*z dzt*z fSdfS#1swYxYw#1swYxYw#t,$r8}/|j|j/d"t1|dz|d}~wwxYw# |d#wwxYw)%aUTrain a pipeline. nlp (Language): The initialized nlp object with the full config. output_path (Optional[Path]): Optional output path to save trained model to. use_gpu (int): Whether to train on GPU. Make sure to call require_gpu before calling this function. stdout (file): A file-like object to write output messages. To disable printing, set to io.StringIO. stderr (file): A second file-like object to write output messages. To disable printing, set to io.StringIO. RETURNS (tuple): The final nlp object and the path to the exported model. T)no_printtrainingseedN gpu_allocatorr)schema train_corpus dev_corpus optimizer score_weightsbatcherrbefore_to_disk before_updatecVjj5jtz ddd|rXtz j rt jtz t jtz tz yy#1swYdxYwN) use_paramsaveragesto_diskDIR_MODEL_LASTDIR_MODEL_BESTexistsshutilrmtreecopytree)is_bestr1r#r.r$s ]/home/spazio/kingdom_calls/work5/transenv/lib/python3.12/site-packages/spacy/training/loop.pysave_checkpointztrain..save_checkpointMs ^^I.. / F 3  ' ' n(D E F n,446 kN:; OOK.8+:V W  F Fs BB(frozen_componentsannotating_components max_epochsdropoutaccumulate_gradientpatience max_stepseval_frequency)rDrErFrGrHexcluderBr2z Pipeline:  zFrozen components: zSet annotations on update for: zInitial learn rate: )disabler$zAAborting and saving the final best model. Encountered exception: Fz"Saved pipeline to output directory)rconfig interpolaterrrresolverrcreate_before_to_disk_callbacktrain_while_improvingcreate_train_batchescreate_evaluation_callbackclean_output_dirwriteinfo pipe_names learn_rate select_pipes update_metastrr8 Exceptionwarnreprr6r5good)r#r$r r!r"msgrL allocatorT dot_namesr,r-r/r0 train_loggerr2r@rArBtraining_step_iteratorlog_stepfinalize_loggerbatchrUis_best_checkpointer1r.s`` @@r?trainrj#s, 4 C ZZ # # %F j&!-z*623z"?3I!| )$ +4HIA>"AlO4I0CL*+Io&M lGX;L3A6F4GHNo&MX-.562 S,<I"3 MB) 34:K.)*!3# [! LLJs~~&6784?@ SXX 34E3FGH4OP HH67L6MN ORV V  LL01E1E0FGH4OP  "3  4F$0ff$E!/F#/E G +E4+!-%%.?%@.3-.*#$67*-kN.J*KD' /;T F G&   " E " y))* HH9;;W X  [>122T{GFF ..    " LL..21gY8      " E " #sHK2K K5K KK K L!)3LL!!L$$L8r.rDrHrErFrGrIrBr2c #Kt|tr t|} n|} g} i}d}t}t |D]\}\}}| r||d}| ||t | }t ||D]}|j|||d| | |jD]D\}}|| vs t|ds|js%|jdvs4|j|F|j||zsj|jr/|j|j5|\}}dddn |\}}| j!|f|t#| dk(}nd\}}d}|t%d |Dz }||||| t't|z |d }|||f|i}t#d | D}|d  }|r ||z |k\ry|s||k\syy#1swYxYww) aTrain until an evaluation stops improving. Works as a generator, with each iteration yielding a tuple `(batch, info, is_best_checkpoint)`, where info is a dict, and is_best_checkpoint is in [True, False, None] -- None indicating that the iteration was not evaluated as a checkpoint. The evaluation is conducted by calling the evaluate callback. Positional arguments: nlp: The spaCy pipeline to evaluate. optimizer: The optimizer callable. train_data (Iterable[Batch]): A generator of batches, with the training data. Each batch should be a Sized[Tuple[Input, Annot]]. The training data iterable needs to take care of iterating over the epochs and shuffling. evaluate (Callable[[], Tuple[float, Any]]): A callback to perform evaluation. The callback should take no arguments and return a tuple `(main_score, other_scores)`. The main_score should be a float where higher is better. other_scores can be any object. Every iteration, the function yields out a tuple with: * batch: A list of Example objects. * info: A dict with various information about the last update (see below). * is_best_checkpoint: A value in None, False, True, indicating whether this was the best evaluation so far. You should use this to save the model checkpoints during training. If None, evaluation was not conducted on that iteration. False means evaluation was conducted, but a previous evaluation was better. The info dict provides the following information: epoch (int): How many passes over the data have been completed. step (int): How many steps have been completed. score (float): The main score from the last evaluation. other_scores: : The other scores from the last evaluation. losses: The accumulated losses throughout training. checkpoints: A list of previous results, where each result is a (score, step, epoch) tuple. r)stepepochF)droplossessgdrI annotates is_trainable)TFNN)NNc32K|]}t|ywr4)len).0egs r? z(train_while_improving..s2b#b'2s)rmrlscore other_scoresro checkpointssecondswordsc3,K|] \}}|| fywr4)rur_scorer_steps r?rwz(train_while_improving.. sL&7VG,Lsr) isinstancefloatrtimer enumeratenextsubdivide_batchupdatepipelinehasattrrrmodel finish_updatestep_schedulesr6r5appendmaxsumint)r#r. train_dataevaluaterDrHrErFrGrIrBr2dropoutsresultsro words_seen start_timerlrmrgbefore_update_argssubbatchnameprocrxryrhrU best_result best_steps r?rPrPs/j'5!G$G!FJJ )* 5:nue *.!?  #1 2x.'/BC H JJ/   ,, .JD$G#D.1%%JJ&99""9- .   "~%!!^^I$6$675*2*'E<55'/j#| NNE4= )!&#g,q/!9 ". Ez!subdivide_batch..sc",,/)keyr)listsortrtrange)rgrEsub_lenstartirs r?rrs KE JJ/J0%j//G E & '1 N X  UV}HsA(A*r-weightscjDcic] \}}| || c}}dtttttffffd }|Scc}}w)Nr%c : jj Dcic] \}}| || c}} j Dcic] \}}|vs ||c}} j D]S\}}| vs t|ttfr"ttjj |t| t fd D}|fS#t$r9}ttjj j |d}~wwxYwcc}}wcc}}w#t$rU}tj}tj j dt#||}t|dd}~wwxYw)N)r)r score_typec3hK|])}j|dj|dz+yw)N)get)russcoresrs r?rwz?create_evaluation_callback..evaluate..6s1!=> 1c"W[[C%88!s/2r/)dictrkeys)rKeyErrorrE900formatrVitemsrrr ValueErrorE915typerrrE983rZ) rirvalueweighted_scorererrrr-r#rs @r?rz,create_evaluation_callback..evaluate'sk O\\*S/2F 06||~SeAR#u*S07 Q*#u3&=3:Q ,,. WJCg~je &E !3!3e!3!UVV W * !BI!Nv%%% O6;;--s~~-FGQ N O TQ * &D++$$/s1vD$QC3-T ) *sFC+ D0D0 D6"D6D<+ D-44D((D-< FAFF)rr rr rZ)r#r-rrrrs``` r?rRrR"sT-4MMOQjc5u?PsEzQG&eE4U #334&0 O5Rs AAcorpusr0rCc#Kd}|dk\r,t||}|sttj|dks||k7rH|dk\rt j n||}||D]}||f |dz }|dkrA||k7rGyyw)Nrr)rrrE986randomshuffle)r#rr0rCrmexamplesrgs r?rQrQBs EQs $V[[) ) q.EZ/ ? NN8 $c{HX& E,     q.EZ/sA>B B B r(rUci|jd<|dD]*}||dj|d|jd|<,|jD]%}||dvs |d||jd|d<'y)N performancer/ryrro_loss)metarrV)r(r#rUmetric pipe_names r?rYrYXs!CHH]?+T  .2>.B.F.Fvs.SCHH] #F +T^^U X &;?>);TCHH] #yk$7 8Urcallbackc,ddlmddffd }|S)Nrrr#r%cs|S|}t|s5tjjdt |}t ||S)Nr1)rr)rrE914rrr)r# modified_nlprrrs r?r1z6create_before_to_disk_callback..before_to_diskisMJ} ,1++$$*:$|BT$UCS/ !r)languager)rr1rs` @r?rOrOds#$H rpathc\|o|jr^|tz |tz fD]H}|js tjt |t jd|Jyyy#t$r/}ttjj||d}~wwxYw)a Remove an existing output directory. Typically used to ensure that that a directory like model-best and its contents aren't just being overwritten by nlp.to_disk, which could preserve existing subdirectories (e.g. components that don't exist anymore). Nz%Removed existing output directory: %s)r) r:r9r8r;r<rZrdebugr[IOErrorrE901r)rsubdirris r?rSrSus  DKKMn,d^.CD HF}}HMM#f+.LL!H&Q  H* !H!&++"4"4$"4"?@aGHs4A33 B+<*B&&B+r4)6rr;syspathlibrtimeitrrtypingrrrrr r r r r r thinc.apirrrrrwasabirerrorsrschemasrutilrrrexamplerrrr9r8r!r"rrjrrZrPrrRrQrYrOrSr~rr?rs )   VU*66# #'s s s$s s  s  s :x~ %&slw ww  wwwww#Yw 9wHj$sCx.%A4%GHIwt  !)48e4D b%tCJ//001@  j\8G#44 5x()8G+<< = , UDcNF*+ U2< UDHcN U  Ux j 89: zlJ&'" H8D> Hd Hr