sg_}dZddlZddlmZmZddlmZddlZ ddl m Z ddl m Z dd lmZdd lmZmZmZdd lmZdd lmZdd lmZGddZGddZ ddZy)z This module contains the TreeGrower class. TreeGrower builds a regression tree fitting a Newton-Raphson step, based on the gradients and hessians of the training data. N)heappopheappush) default_timer)_openmp_effective_n_threads) sum_parallel)!set_raw_bitset_from_binned_bitset)PREDICTOR_RECORD_DTYPEX_BITSET_INNER_DTYPEMonotonicConstraint)HistogramBuilder) TreePredictor)Splitterc(eZdZdZdddZdZdZy)TreeNodeaTree Node class used in TreeGrower. This isn't used for prediction purposes, only for training (see TreePredictor). Parameters ---------- depth : int The depth of the node, i.e. its distance from the root. sample_indices : ndarray of shape (n_samples_at_node,), dtype=np.uint32 The indices of the samples at the node. partition_start : int start position of the node's sample_indices in splitter.partition. partition_stop : int stop position of the node's sample_indices in splitter.partition. sum_gradients : float The sum of the gradients of the samples at the node. sum_hessians : float The sum of the hessians of the samples at the node. Attributes ---------- depth : int The depth of the node, i.e. its distance from the root. sample_indices : ndarray of shape (n_samples_at_node,), dtype=np.uint32 The indices of the samples at the node. sum_gradients : float The sum of the gradients of the samples at the node. sum_hessians : float The sum of the hessians of the samples at the node. split_info : SplitInfo or None The result of the split evaluation. is_leaf : bool True if node is a leaf left_child : TreeNode or None The left child of the node. None for leaves. right_child : TreeNode or None The right child of the node. None for leaves. value : float or None The value of the leaf, as computed in finalize_leaf(). None for non-leaf nodes. partition_start : int start position of the node's sample_indices in splitter.partition. partition_stop : int stop position of the node's sample_indices in splitter.partition. allowed_features : None or ndarray, dtype=int Indices of features allowed to split for children. interaction_cst_indices : None or list of ints Indices of the interaction sets that have to be applied on splits of child nodes. The fewer sets the stronger the constraint as fewer sets contain fewer features. children_lower_bound : float children_upper_bound : float N)valuec8||_||_|jd|_||_||_||_d|_d|_d|_ |jtdtdd|_ d|_ d|_d|_||_||_y)NrFz-infz+inf)depthsample_indicesshape n_samples sum_gradients sum_hessiansris_leafallowed_featuresinteraction_cst_indicesset_children_boundsfloat split_info left_child right_child histogramspartition_startpartition_stop)selfrrr$r%rrrs b/var/www/html/venv/lib/python3.12/site-packages/sklearn/ensemble/_hist_gradient_boosting/grower.py__init__zTreeNode.__init__Ws ,'--a0*(  $'+$   vf > /,c ||_||_y)z`. interaction_cst : list of sets of integers, default=None List of interaction constraints. l2_regularization : float, default=0. The L2 regularization parameter penalizing leaves with small hessians. Use ``0`` for no regularization (default). feature_fraction_per_split : float, default=1 Proportion of randomly chosen features in each and every node split. This is a form of regularization, smaller values make the trees weaker learners and might prevent overfitting. rng : Generator Numpy random Generator used for feature subsampling. shrinkage : float, default=1. The shrinkage parameter to apply to the leaves values, also known as learning rate. n_threads : int, default=None Number of OpenMP threads to use. `_openmp_effective_n_threads` is called to determine the effective number of threads use, which takes cgroups CPU quotes into account. See the docstring of `_openmp_effective_n_threads` for details. Attributes ---------- histogram_builder : HistogramBuilder splitter : Splitter root : TreeNode finalized_leaves : list of TreeNode splittable_nodes : list of TreeNode missing_values_bin_idx : int Equals n_bins - 1 n_categorical_splits : int n_features : int n_nodes : int total_find_split_time : float Time spent finding the best splits total_compute_hist_time : float Time spent computing histograms total_apply_split_time : float Time spent splitting nodes with_monotonic_cst : bool Whether there are monotonic constraints that apply. False iff monotonic_cst is None. NgMbP?Fg?c<|j|||t|}| | dz } t| tjr7t j | g|jdzt j} n%t j| t j} t| tr| g|jdz} t j| t j} | Bt j|jdtjt j} n%t j| t j} t j | tjk7|_| 3t j$|jdt j} n%t j| t j} t j t j&| dk(| tjk7r t)d|jddk(}t+|| |||||_| dz }t/|| || | | |||||||||_||_||_||_||_||_| |_||_| |_ | |_!| |_"||_#||_$||_%|jd|_&||_'g|_(g|_)d|_*d|_+d|_,d|_-|j]||d|_/y) Nr dtype)r fill_valuer?)rr?z7Categorical features cannot have monotonic constraints.r)X_binnedn_bins_non_missingmissing_values_bin_idxhas_missing_valuesis_categorical monotonic_cstl2_regularizationmin_hessian_to_splitmin_samples_leafmin_gain_to_splithessians_are_constantfeature_fraction_per_splitrng n_threadsr;)0_validate_parametersr isinstancenumbersIntegralnparrayruint32asarraybooluint8fullr NO_CSTint8anywith_monotonic_cstzeros logical_and ValueErrorrhistogram_builderrsplitterrAmax_leaf_nodes max_depthrIrJrBrCrDrErFinteraction_cstrG shrinkage n_featuresrNsplittable_nodesfinalized_leavestotal_find_split_timetotal_compute_hist_timetotal_apply_split_timen_categorical_splits_initialize_rootn_nodes)r&rA gradientshessiansrcrdrIrJrHn_binsrBrDrErFrerGrLrMrfrNrKrCs r'r(zTreeGrower.__init__s, !!    0 :  %!'!  ('*:*: ;!##$x~~a'88 " "$,>bii!P  ($ /"4!5q8I!I ZZ(:"((K  GGnnQ'.55ggM JJ}BGGDM"$&&:M:T:T)T"U  !XXHNN1,=RXXNNZZbhhGN 66 NN!#]6I6P6P%P  VW W (q 1Q 6!1 fi3H)" "(! 1#91)'/!5-/"7'A  ! ," 0!2"4&<#"4,*.!2""..+" " "%("'*$&)#$%! i2 r)c|jtjk7r td|jj s t d|dkrt dj||dkrt dj|y)zfValidate parameters passed to __init__. Also validate parameters passed to splitter. zX_binned must be of type uint8.zMX_binned should be passed as Fortran contiguous array for maximum efficiency.rz&min_gain_to_split={} must be positive.z)min_hessian_to_split={} must be positive.N)r?rSrXNotImplementedErrorflags f_contiguousr`format)r&rArJrHs r'rOzTreeGrower._validate_parametersgs >>RXX %%&GH H~~**0  q 8??@QR  ! #;BBCWX  $r)cv|jr|j|jr|jy)z#Grow the tree, from root to leaves.N)rh split_next_apply_shrinkage)r&s r'growzTreeGrower.grows.## OO ## r)cd|jD]!}|xj|jzc_#y)aMultiply leaves values by shrinkage parameter. This must be done at the very end of the growing process. If this were done during the growing process e.g. in finalize_leaf(), then a leaf would be shrunk but its sibling would potentially not be (if it's a non-leaf), which would lead to a wrong computation of the 'middle' value needed to enforce the monotonic constraints. N)rirrf)r&leafs r'rzzTreeGrower._apply_shrinkages+)) )D JJ$.. (J )r)c |jjd}d}t||j}|jj r |d|z}nt||j}t ||jjd|||d|_ |jjd|jzkr|j|jy||jjkr|j|jy|jtt!|j|j_t%j&|j}t)j*|t(j,t!||j_t1}|jj3|jj4|jj.|j_|xj8t1|z z c_t1}|j;|j|xj<t1|z z c_y)z/Initialize root node and finalize it if needed.rrrr$r%rrrNr?count)rArrrNrarKrrb partitionrootrrI_finalize_leafrHrerangelenrsetunionrSfromiterrUrtimecompute_histograms_bruterr#rk_compute_best_split_and_pushrj) r&rprqrrrrrtics r'rnzTreeGrower._initialize_rootsMM''* $Y?  ! ! 7 7#A;2L'$..AL==22$'%   99  T%:%:!: :    *  $--<< <    *     +05c$:N:N6O0PDII -*su{{D,@,@A )+  =M9N*DII &f#55NN II $ $dii&@&@   $$ 4$f ))$))4 ""dfsl2"r)c v|jj|j|j|j|j |j |j|j|j|_ |jjdkr|j|yt|j|y)a]Compute the best possible split (SplitInfo) of a given node. Also push it in the heap of splittable nodes if gain isn't zero. The gain of a node is 0 if either all the leaves are pure (best gain = 0), or if no split would satisfy the constraints, (min_hessians_to_split, min_gain_to_split, min_samples_leaf) )rr#rrr lower_bound upper_boundrrN)rbfind_node_splitrr#rrrr+r,rr r0rrrhr&nodes r'rz'TreeGrower._compute_best_split_and_pushs--77nn,,****1111!228   ??  1 $    % T**D 1r)c  t|j}t}|jj |j |j \}}}|xjt|z z c_|jdz}t|jt|jz}|dz }t|||j|j|z|j j|j j|j j}t|||j |j |j j"|j j$|j j&} | |_||_|j,@|j/|\|_|_|j2| _|j0| _|j4|j j6s(|j8| j8kD|j _|xj<dz c_|xj>|j j@z c_|jBE||jBk(r6|jE||jE| |jG|| fS|jH5||jHk(r&|jE||jE| || fS|j8|jJdzkr|jE|| j8|jJdzkr|jE| |jLr|jN|j j6tPjRk(r|jTx} } |jVx} } n|jX| jXzdz }|jN|j j6tPjZk(r|jT|} } ||jV} } n||jV} } |jT|} } |j]| | | j]| | |j^ }| j^ }|s|rN|j j`d}| j j`d}||kr|}| }n| }|}t}|jbje|j |j0|_3|jbji|jf|jf|j0|_3d|_3|xjjt|z z c_5t}|r|jm||r|jm| |xjnt|z z c_7|| fD]}|j^s|`3|`3|| fS)zSplit the node with highest potential gain. Returns ------- left : TreeNode The resulting left child. right : TreeNode The resulting right child. r rrNr)8rrhrrb split_indicesr rrlrrrirr$sum_gradient_leftsum_hessian_left value_leftr%sum_gradient_rightsum_hessian_right value_rightr"r!re_compute_interactionsrrrD feature_idxrmissing_go_to_leftrormrErcr_finalize_splittable_nodesrdrIr]rFr rZr+r,rPOSrrrrarr#compute_histograms_subtractionrkrrj)r&rrsample_indices_leftsample_indices_rightright_child_posr n_leaf_nodesleft_child_noderight_child_node lower_left lower_right upper_left upper_rightmidshould_split_leftshould_split_rightn_samples_leftn_samples_rightsmallest_child largest_childchilds r'ryzTreeGrower.split_nexts<t,,-f MM ' '9L9L M    ##tv|3# Q4001C8M8M4NN  ". 00///A//;;99//,, $/+::..//<<:://-- ,)    + **40 07 77  41@0P0P  -&&t'B'BC  )),<,F,FF OO .   !!T__%C%CC!    *|t?R?R/R    0    0 1  + + -"$44 4 >> %%4>>*A    0    0 1"$44 4  $ $t'<'#>?&--.,0+D+DD [+/+D+DD [&,,/?/E/EEJ&&t'B'BC*../.2-F-F J/2D4M4MK-0$2K2K J/3/H/H#K  / / J G  0 0k J!0 7 77!1!9!99  2-;;AA!DN.==CCAFO/!0 0 !1 / &C(,(>(>(W(W--~/N/N)N %&&EEOO"--"33  $#DO  ( (DFSL 8 (&C 11/B!112BC  & &$&3, 6 &*+;< )==( ) O 000r)cHt}g}|jD]W}|jj|j|vs)|j ||j |j|Ytj|tjt||fS)a]Compute features allowed by interactions to be inherited by child nodes. Example: Assume constraints [{0, 1}, {1, 2}]. 1 <- Both constraint groups could be applied from now on / \ 1 2 <- Left split still fulfills both constraint groups. / \ / \ Right split at feature 2 has only group {1, 2} from now on. LightGBM uses the same logic for overlapping groups. See https://github.com/microsoft/LightGBM/issues/4481 for details. Parameters: ---------- node : TreeNode A node that might have children. Based on its feature_idx, the interaction constraints for possible child nodes are computed. Returns ------- allowed_features : ndarray, dtype=uint32 Indices of features allowed to split for children. interaction_cst_indices : list of ints Indices of the interaction sets that have to be applied on splits of child nodes. The fewer sets the stronger the constraint as fewer sets contain fewer features. r) rrr rreappendupdaterSrrUr)r&rrris r'rz TreeGrower._compute_interactionss>5"$-- AA**d.B.B1.EE'..q1 ''(<() rSr^ror rmr _fill_predictor_arraysrrBr)r&binning_thresholdspredictor_nodesbinned_left_cat_bitsetsraw_left_cat_bitsetss r'make_predictorzTreeGrower.make_predictors ((4<<7MN"$((  & & *2F#  "xx  & & *2F    # II   # #   46J  r))r3r4r5r6rSrandom default_rngr(rOr{rzrnrryrrrrr7r)r'r9r9s^J! #& II ! ! #)ob4  )*3X22d1L( T+ &! r)r9c ||}|j|d<|j|d<|j|jj|d<nd|d<|j|d<|j r d|d<|dz|fS|j} | j | j} } | |d <| |d <| j|d <| j|d <| j|| dz k(rtj|d <nP| jr9|| } ||d<| j||<t||| j| |dz }n || | |d <|dz }||d<t||||j||||\}}||d<t||||j ||||S)z>Helper used in make_predictor to set the TreePredictor fields.rrr0rTrr r bin_thresholdrrE num_threshold bitset_idxleft)rrBnext_free_node_idxnext_free_bitset_idxright)rrr r0rrrbin_idxrrErSinfleft_cat_bitsetr rr!r") rrr grower_noderrBrrrr rr categoriess r'rrs - .D))DM%%DM)"--22V V %%DMY!A%';;;''J%11:3E3EK%D#D!+!>!>D '66D / rs`#(E,6 ($sAsAlD  D  \Kr)