o il @sJdZddlZddlmZmZddlmZmZddlZ ddl m Z m Z ddl mZmZmZmZmZmZddlmZdd lmZmZdd lmZmZdd lmZdd lmZm Z m!Z!gd Z"ddZ# d(ddZ$ddZ%d)ddZ&ddZ'GdddeeeeeedZ(Gd d!d!e(Z)Gd"d#d#e(Z*Gd$d%d%e(Z+Gd&d'd'eeeZ,dS)*zG The :mod:`sklearn.pls` module implements Partial Least Squares (PLS). N)ABCMetaabstractmethod)IntegralReal)pinvsvd) BaseEstimatorClassNamePrefixFeaturesOutMixinMultiOutputMixinRegressorMixinTransformerMixin _fit_context)ConvergenceWarning) check_arraycheck_consistent_length)Interval StrOptions)svd_flip) FLOAT_DTYPEScheck_is_fitted validate_data)PLSSVD PLSCanonical PLSRegressionc Cst|ddd\}}}|jj}ddd}t|||t|j}t||k}|ddd|f}||d|}t t t ||d|S)NF) full_matrices check_finiteg@@g.A)fd) rdtypecharlowernpmaxfinfoepssumZ transpose conjugatedot)ausZvhtfactorZcondZrankr.o/var/www/html/eduruby.in/lip-sync/lip-sync-env/lib/python3.10/site-packages/sklearn/cross_decomposition/_pls.py _pinv2_old s   r0Aư>Fc st|jjztfdd|jD}Wnty&}ztd|d}~wwd}|dkr6t|t|} } t|D]z} |dkrGt | |} n t |j|t ||} | t t | | } t || } |dkrrt | | }nt |j| t | j| }|r|t t ||}t ||t ||}| |}t |||ks|j ddkrn| }q:| d}||krt dt| ||fS) a?Return the first left and right singular vectors of X'y. Provides an alternative to the svd(X'y) and uses the power method instead. With norm_y_weights to True and in mode A, this corresponds to the algorithm section 11.3 of the Wegelin's review, except this starts at the "update saliences" part. c3s(|]}tt|kr|VqdSN)r"anyabs).0colr%r.r/ ?s&z;_get_first_singular_vectors_power_method..y residual is constantNdBz$Maximum number of iterations reached)r"r$rr%nextT StopIterationr0ranger(sqrtshapewarningswarnr)Xymodemax_itertolnorm_y_weightsZy_scoreeZ x_weights_oldZX_pinvZy_pinvi x_weightsZx_score y_weightsZx_weights_diffZn_iterr.r9r/(_get_first_singular_vectors_power_method2s<      rQcCs@t|j|}t|dd\}}}|dddf|dddffS)zbReturn the first left and right singular vectors of X'y. Here the whole SVD is computed. FrNr)r"r(r@r)rGrHCU_Vtr.r.r/_get_first_singular_vectors_svdms rWTcCs|jdd}||8}|jdd}||8}|r9|jddd}d||dk<||}|jddd}d||dk<||}nt|jd}t|jd}||||||fS)z{Center X, y and scale if the scale parameter==True Returns ------- X, y, x_mean, y_mean, x_std, y_std raxisr>)rYZddofg?)meanZstdr"ZonesrD)rGrHscaleZx_meanZy_meanZx_stdZy_stdr.r.r/_center_scale_xyws     r]cCs2tt|}t||}||9}||9}dS)z7Same as svd_flip but works on 1d arrays, and is inplaceN)r"Zargmaxr6sign)r*vZbiggest_abs_val_idxr^r.r.r/ _svd_flip_1ds r`c seZdZUdZeeddddgdgeddhged d hged d hgeeddddgeed dddgdgdZe e d<e d$ddd d ddddddZ e ddddZd%ddZd&ddZd'ddZd&d d!Zfd"d#ZZS)(_PLSaPartial Least Squares (PLS) This class implements the generic PLS algorithm. Main ref: Wegelin, a survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case https://stat.uw.edu/sites/default/files/files/reports/2000/tr371.pdf r>Nleftclosedboolean regression canonicalr1r=rnipalsr n_componentsr\deflation_moderI algorithmrJrKcopy_parameter_constraintsrTr2r3)r\rkrIrlrJrKrmc Cs4||_||_||_||_||_||_||_||_dSr4)rjrkrIr\rlrJrKrm) selfrjr\rkrIrlrJrKrmr.r.r/__init__s  z _PLS.__init__Zprefer_skip_nested_validationc Cst||t||tjd|jdd}t|dtjd|jdd}|jdkr,d|_|dd}nd|_|j d }|j d}|j d}|j }|j d krKt ||nt |||}||kr`t d |d |d |j dk|_|j}t|||j\} } |_|_|_|_t||f|_t||f|_t||f|_t||f|_t||f|_t||f|_g|_t| jj} t |D]} |j!dkrtj"t#| d| kd d} d| dd| f<zt$| | |j%|j&|j'|d\}}}Wn#t(y }zt)|dkrt*+d| WYd}~nd}~ww|j,|n |j!dkr!t-| | \}}t.||t/| |}|r2d}nt/||}t/| ||}t/|| t/||}| t0||8} |j dkrnt/|| t/||}| t0||8} |j d krt/|| t/||}| t0||8} ||jdd| f<||jdd| f<||jdd| f<||jdd| f<||jdd| f<||jdd| f<qt/|jt1t/|jj2|jdd|_3t/|jt1t/|jj2|jdd|_4t/|j3|jj2|_5|j5|jj2|j|_5|j|_6|j3j d|_7|S)Fit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. y : array-like of shape (n_samples,) or (n_samples, n_targets) Target vectors, where `n_samples` is the number of samples and `n_targets` is the number of response variables. Returns ------- self : object Fitted model. Trrforce_writeablermZensure_min_samplesrHF input_namerrtrm ensure_2dr>rrf`n_components` upper bound is . Got instead. Reduce `n_components`.rgrh rXrZN)rIrJrKrLr;z$y residual is constant at iteration r)r)8rrr"float64rmrndim _predict_1dreshaperDrjrkmin ValueErrorZ_norm_y_weightsr]r\_x_mean_y_mean_x_std_y_stdZzeros x_weights_ y_weights_ _x_scores _y_scores x_loadings_ y_loadings_n_iter_r$rr%rBrlallr6rQrIrJrKrAstrrErFappendrWr`r(outerrr@ x_rotations_ y_rotations_coef_ intercept__n_features_out)rorGrHnpqrjrank_upper_boundrLZXkZykZy_epskZyk_maskrOrPrrMx_scoresZy_ssy_scoresZ x_loadingsZ y_loadingsr.r.r/fits               z_PLS.fitcCst|t|||tdd}||j8}||j}t||j}|durKt|dd|td}|j dkr6| dd}||j 8}||j }t||j }||fS|S)a.Apply the dimension reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to transform. y : array-like of shape (n_samples, n_targets), default=None Target vectors. copy : bool, default=True Whether to copy `X` and `y`, or perform in-place normalization. Returns ------- x_scores, y_scores : array-like or tuple of array-like Return `x_scores` if `y` is not given, `(x_scores, y_scores)` otherwise. FrmrresetNrH)rvrwrmrr>rx)rrrrrr"r(rrr~rrrr)rorGrHrmrrr.r.r/ transformps        z_PLS.transformcCst|t|dtd}t||jj}||j9}||j7}|dur>t|dtd}t||j j}||j 9}||j 7}||fS|S)akTransform data back to its original space. Parameters ---------- X : array-like of shape (n_samples, n_components) New data, where `n_samples` is the number of samples and `n_components` is the number of pls components. y : array-like of shape (n_samples,) or (n_samples, n_components) New target, where `n_samples` is the number of samples and `n_components` is the number of pls components. Returns ------- X_original : ndarray of shape (n_samples, n_features) Return the reconstructed `X` data. y_original : ndarray of shape (n_samples, n_targets) Return the reconstructed `X` target. Only returned when `y` is given. Notes ----- This transformation will only be exact if `n_components=n_features`. rG)rvrNrH) rrrr"matmulrr@rrrrr)rorGrHZX_reconstructedZy_reconstructedr.r.r/inverse_transforms    z_PLS.inverse_transformcCsHt|t|||tdd}||j8}||jj|j}|jr"|S|S)aLPredict targets of given samples. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples. copy : bool, default=True Whether to copy `X` or perform in-place normalization. Returns ------- y_pred : ndarray of shape (n_samples,) or (n_samples, n_targets) Returns predicted values. Notes ----- This call requires the estimation of a matrix of shape `(n_features, n_targets)`, which may be an issue in high dimensional space. Fr) rrrrrr@rrZravel)rorGrmZy_predr.r.r/predicts  z _PLS.predictcC|||||S)aLearn and apply the dimension reduction on the train data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. y : array-like of shape (n_samples, n_targets), default=None Target vectors, where `n_samples` is the number of samples and `n_targets` is the number of response variables. Returns ------- self : ndarray of shape (n_samples, n_components) Return `x_scores` if `y` is not given, `(x_scores, y_scores)` otherwise. rrrorGrHr.r.r/ fit_transformz_PLS.fit_transformcst}d|j_d|j_|S)NTF)super__sklearn_tags__Zregressor_tagsZ poor_scoreZ target_tagsrequired)rotags __class__r.r/rs z_PLS.__sklearn_tags__r)NTr4T)__name__ __module__ __qualname____doc__rrrrrndict__annotations__rrprrrrrrr __classcell__r.r.rr/ras<        # ' , ra) metaclasscsdeZdZUdZiejZeed<dD]Ze eq d dddddfd d Z fd d Z Z S)ra PLS regression. PLSRegression is also known as PLS2 or PLS1, depending on the number of targets. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, n_features]`. scale : bool, default=True Whether to scale `X` and `y`. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `y` in :term:`fit` before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `y`. x_scores_ : ndarray of shape (n_samples, n_components) The transformed training samples. y_scores_ : ndarray of shape (n_samples, n_components) The transformed training targets. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `y`. coef_ : ndarray of shape (n_target, n_features) The coefficients of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. Examples -------- >>> from sklearn.cross_decomposition import PLSRegression >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> pls2 = PLSRegression(n_components=2) >>> pls2.fit(X, y) PLSRegression() >>> y_pred = pls2.predict(X) For a comparison between PLS Regression and :class:`~sklearn.decomposition.PCA`, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_pcr_vs_pls.py`. rnrkrIrlrTr2r3r\rJrKrmc  tj||ddd|||ddS)Nrfr1rhrirrprorjr\rJrKrmrr.r/rpm zPLSRegression.__init__cs"t|||j|_|j|_|S)rr)rrrZ x_scores_rZ y_scores_rrr.r/r{szPLSRegression.fitr) rrrrrarnrrparampoprprrr.r.rr/rs g  rcsZeZdZUdZiejZeed<dD]Ze eq d dddddd fd d Z Z S) ra^ Partial Least Squares transformer and regressor. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `y`. algorithm : {'nipals', 'svd'}, default='nipals' The algorithm used to estimate the first singular vectors of the cross-covariance matrix. 'nipals' uses the power method while 'svd' will compute the whole SVD. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. Empty if `algorithm='svd'`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- CCA : Canonical Correlation Analysis. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import PLSCanonical >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> plsca = PLSCanonical(n_components=2) >>> plsca.fit(X, y) PLSCanonical() >>> X_c, y_c = plsca.transform(X, y) rn)rkrIrTrhr2r3)r\rlrJrKrmc s tj||dd||||ddS)Nrgr1rir)rorjr\rlrJrKrmrr.r/rps  zPLSCanonical.__init__r rrrrrarnrrrrrprr.r.rr/rs b  rcsXeZdZUdZiejZeed<dD]Ze eq d dddddfd d Z Z S) CCAa Canonical Correlation Analysis, also known as "Mode B" PLS. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `y`. max_iter : int, default=500 The maximum number of iterations of the power method. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import CCA >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [3.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> cca = CCA(n_components=1) >>> cca.fit(X, y) CCA(n_components=1) >>> X_c, y_c = cca.transform(X, y) rnrrTr2r3rc r)Nrgr=rhrirrrr.r/rpxrz CCA.__init__rrr.r.rr/rs Z rc@speZdZUdZeeddddgdgdgdZeed<dd d d d d Z e d dddZ dddZ dddZ dS)raPartial Least Square SVD. This transformer simply performs a SVD on the cross-covariance matrix `X'y`. It is able to project both the training data `X` and the targets `y`. The training data `X` is projected on the left singular vectors, while the targets are projected on the right singular vectors. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 The number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `y`. copy : bool, default=True Whether to copy `X` and `y` in fit before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. y_weights_ : ndarray of (n_targets, n_components) The right singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. CCA : Canonical Correlation Analysis. Examples -------- >>> import numpy as np >>> from sklearn.cross_decomposition import PLSSVD >>> X = np.array([[0., 0., 1.], ... [1., 0., 0.], ... [2., 2., 2.], ... [2., 5., 4.]]) >>> y = np.array([[0.1, -0.2], ... [0.9, 1.1], ... [6.2, 5.9], ... [11.9, 12.3]]) >>> pls = PLSSVD(n_components=2).fit(X, y) >>> X_c, y_c = pls.transform(X, y) >>> X_c.shape, y_c.shape ((4, 2), (4, 2)) r>Nrbrcrerjr\rmrnrT)r\rmcCs||_||_||_dSr4r)rorjr\rmr.r.r/rps zPLSSVD.__init__rqc Cs&t||t||tjd|jdd}t|dtjd|jdd}|jdkr(|dd}|j}t |j d |j d|j d}||krIt d |d |d t |||j \}}|_|_|_|_t|j|}t|dd \}}}|ddd|f}|d|}t||\}}|j} ||_| |_|jj d|_|S)aJFit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets) Targets. Returns ------- self : object Fitted estimator. TrrsrHFrur>rxrryrzr{rRN)rrr"r}rmrr~rrjrrDrr]r\rrrrr(r@rrrrr) rorGrHrjrrSrTr+rVVr.r.r/rsP    z PLSSVD.fitcCst|t||tjdd}||j|j}t||j}|durGt|ddtjd}|j dkr4| dd}||j |j }t||j }||fS|S)a  Apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to be transformed. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- x_scores : array-like or tuple of array-like The transformed data `X_transformed` if `y is not None`, `(X_transformed, y_transformed)` otherwise. F)rrNrH)rvrwrr>rx)rrr"r}rrr(rrr~rrrr)rorGrHZXrryrrr.r.r/rs  zPLSSVD.transformcCr)aLearn and apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- out : array-like or tuple of array-like The transformed data `X_transformed` if `y is not None`, `(X_transformed, y_transformed)` otherwise. rrr.r.r/r7rzPLSSVD.fit_transformrr4)rrrrrrrnrrrprrrrr.r.r.r/rs D  @ r)r1r2r3Fr)-rrEabcrrnumbersrrnumpyr"Z scipy.linalgrrbaser r r r r r exceptionsrutilsrrZutils._param_validationrrZ utils.extmathrZutils.validationrrr__all__r0rQrWr]r`rarrrrr.r.r.r/sF    ;  bn