o i.@sddlmZddlmZddlmZddlZddlm Z m Z m Z m Z m Z ddlmZddd d d Zd"d d ZGdddeZddZGdddeZddZddZddddZd#ddZGdddeZd d!ZdS)$)Counter)suppress) NamedTupleN)_isin _searchsorteddevice get_namespacexpx is_scalar_nanFreturn_inverse return_countscCs&|jtkr t|||dSt|||dS)aHelper function to find unique values with support for python objects. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : ndarray Values to check for unknowns. return_inverse : bool, default=False If True, also return the indices of the unique values. return_counts : bool, default=False If True, also return the number of times each unique item appears in values. Returns ------- unique : ndarray The sorted unique values. unique_inverse : ndarray The indices to reconstruct the original array from the unique array. Only provided if `return_inverse` is True. unique_counts : ndarray The number of times each of the unique values comes up in the original array. Only provided if `return_counts` is True. r )dtypeobject_unique_python _unique_np)valuesrrrd/var/www/html/eduruby.in/lip-sync/lip-sync-env/lib/python3.10/site-packages/sklearn/utils/_encode.py_uniques rc Cst|\}}d\}}|r|r||\}}}}n|r"||\}}n|r,||\}}n||}|jrgt|drgt||j|d}|d|d}|rR||||k<|rg| ||d||<|d|d}|f} |rq| |f7} |rx| |f7} t | dkr| dS| S)zHelper function to find unique values for numpy arrays that correctly accounts for nans. See `_unique` documentation for details.)NNxpNrr) r Z unique_allZunique_inverseZ unique_counts unique_valuessizer rnansumlen) rrrr_ZinversecountsuniquesZnan_idxretrrrr=s.     rc@s*eZdZUdZeed<eed<ddZdS) MissingValuesz'Data class for missing data informationrnonecCs*g}|jr |d|jr|tj|S)z3Convert tuple to a list where None is always first.N)r%appendrnp)selfoutputrrrto_listjs   zMissingValues.to_listN)__name__ __module__ __qualname____doc__bool__annotations__r*rrrrr$ds  r$cCsndd|D}|s|tdddfSd|vr)t|dkr"tddd}n tddd}ntddd}||}||fS)a.Extract missing values from `values`. Parameters ---------- values: set Set of values to extract missing from. Returns ------- output: set Set with missing values extracted. missing_values: MissingValues Object with missing value information. cSs h|] }|dus t|r|qSNr .0valuerrr sz#_extract_missing..F)rr%NrT)r$r)rZmissing_values_setZoutput_missing_valuesr)rrr_extract_missingts  r6cs(eZdZdZfddZddZZS)_nandictz!Dictionary with support for nans.cs6t||D]\}}t|r||_dSq dSr1)super__init__itemsr nan_value)r(mappingkeyr4 __class__rrr9s z_nandict.__init__cC t|dr t|r |jSt|)Nr;)hasattrr r;KeyErrorr(r=rrr __missing__z_nandict.__missing__)r+r,r-r.r9rD __classcell__rrr>rr7s r7csDt||\}}tddt|D|jfdd|Dt|dS)z,Map values based on its position in uniques.cSsi|]\}}||qSrr)r3ivalrrr sz#_map_to_integer..csg|]}|qSrrr3vtablerr z#_map_to_integer..)r)r r7 enumerateZasarrayr)rr"rr rrLr_map_to_integers rQcCszt|}t|\}}t|}||tj||jd}Wnty=tddtdd|DD}td|w|f}|rK|t ||f7}|rU|t ||f7}t |dkr_|dS|S)Nrcss|]}|jVqdSr1)r-)r3trrr sz!_unique_python..css|]}t|VqdSr1)typerJrrrrTszPEncoders require their input argument must be uniformly strings or numbers. Got rr) setr6sortedextendr*r'arrayr TypeErrorrQ _get_countsr)rrr uniques_setZmissing_valuesr"typesr#rrrrs(   rT) check_unknownc Cs~t||\}}||jds(zt||WSty'}ztd|d}~ww|r8t||}|r8td|t|||dS)aHelper function to encode values into [0, n_uniques - 1]. Uses pure python method for object dtype, and numpy method for all other dtypes. The numpy method has the limitation that the `uniques` need to be sorted. Importantly, this is not checked but assumed to already be the case. The calling method needs to ensure this for all non-object values. Parameters ---------- values : ndarray Values to encode. uniques : ndarray The unique values in `values`. If the dtype is not object, then `uniques` needs to be sorted. check_unknown : bool, default=True If True, check for values in `values` that are not in `unique` and raise an error. This is ignored for object dtype, and treated as True in this case. This parameter is useful for _BaseEncoder._transform() to avoid calling _check_unknown() twice. Returns ------- encoded : ndarray Encoded values numericz%y contains previously unseen labels: Nr)r isdtyperrQrB ValueError_check_unknownr)rr"r^rr ediffrrr_encodes  recst||\}}d}||jdspt|}t|\}}t|t\|}|jo.j } |jo5j } fdd|r\|sE| sE| rR|fdd|D}n |jt ||j d}t |}| rg| d| ro| t jnM||} tj| |d|d }|r|jrt|||}n |jt ||j d}|||r||} || r|jr|r||} d || <|| }t |}|r||fS|S) a Helper function to check for unknowns in values to be encoded. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : array Values to check for unknowns. known_values : array Known values. Must be unique. return_mask : bool, default=False If True, return a mask of the same shape as `values` indicating the valid values. Returns ------- diff : list The unique values present in `values` and not in `know_values`. valid_mask : boolean array Additionally returned if ``return_mask=True``. Nr_cs$|vpjo |dupjot|Sr1)r%rr )r4)missing_in_uniquesr\rris_valids   z _check_unknown..is_validcsg|]}|qSrrr2)rgrrrN&rOz"_check_unknown..rRT) assume_uniquerr)r r`rrVr6rr%rYZonesrr/listr&r'rr Z setdiff1drranyisnan)rZ known_valuesZ return_maskrr Z valid_maskZ values_setZmissing_in_valuesrdZ nan_in_diffZ none_in_diffrZ diff_is_nanis_nanr)rgrfr\rrbsL           rbcs0eZdZdZfddZddZddZZS) _NaNCounterz$Counter with support for nan values.cst||dSr1)r8r9_generate_items)r(r:r>rrr9Msz_NaNCounter.__init__ccs>|D]}t|s |Vqt|dsd|_|jd7_qdS)z>Generate items without nans. Stores the nan counts separately. nan_countrrN)r rAro)r(r:itemrrrrnPs z_NaNCounter._generate_itemscCr@)Nro)rAr rorBrCrrrrDZrEz_NaNCounter.__missing__)r+r,r-r.r9rnrDrFrrr>rrmJs   rmc Cs|jjdvr9t|}tjt|tjd}t|D]\}}tt ||||<Wdn1s1wYq|St |dd\}}tj ||dd}t |dr[t |dr[d|d<t |||} tj|tjd}|| ||<|S)zGet the count of each of the `uniques` in `values`. The counts will use the order passed in by `uniques`. For non-object dtypes, `uniques` is assumed to be sorted and `np.nan` is at the end. ZOUrRNT)r)rhr)rkindrmr'ZzerosrZint64rPrrBrisinrkZ searchsortedZ zeros_like) rr"counterr)rGrprr!Zuniques_in_valuesZunique_valid_indicesrrrr[`s"   r[)FF)F) collectionsr contextlibrtypingrnumpyr'Z _array_apirrrr r Z_missingr rrr$r6dictr7rQrrerbrmr[rrrrs"     )'& +T