Close
Sidebar
Search tutorials
Get Started
Documentation
XCurve.Metrics.AUROC
Compute Area Under the ROC Curve (AUC). Note that, this function can not only be used to binary but also multiclass AUC (either 'ova' or 'ovo'). For binary classifications, we have $$ \text{AUROC}(f) = \frac{1}{n_+n_-}\sum_{i=1}^{n_+}\sum_{j=1}^{n_-}\mathbb{I}[f(x_i^+) > f(x_j^-)], $$ where $f$ is a binary classifier, $n_+$ ($n_-$) is the number of positive (negative) samples. For multiclass AUC (MAROC) problems, there are two schemes to compute MAUROC score. (1) One vs. All (OVA) Regime: $$ \text{MAUROC}_{\texttt{ova}}(f) = \frac{1}{N_{C}}\sum_{n_c=1}^{N_C}\text{AUROC}_{n_c|\neg n_c}(f^{(n_c)}), $$ where $f = \{f^{(1)}, f^{(2)}, \dots, f^{(N_C)}\}$ is a classifier with $N_C$ channels and $N_C$ is the number of classes, $\text{AUROC}_{n_c|\neg n_c}(f^{(n_c)})$ refers to a pairwise AUC score for each $f^{(n_c)}$, where the positive samples are drawn from the $n_c$-class and the negative ones are drawn from the others conditioned on $y \neq n_c$. (2) One vs. One (OVO) Regime: $$ \text{MAUROC}_{\texttt{ovo}}(f) = \frac{1}{N_{C}(N_{C} - 1)}\sum_{i=1}^{N_C}\text{AUROC}_{i|j}(f^{(i)}), $$ where, in terms of $\text{AUROC}_{i|j}(f^{(i)})$, the positive samples are drawn from the $i$-class and the negative ones are drawn from $j$-class. Note that $\text{AUROC}_{i|j}(f^{(i)}) \neq \text{AUROC}_{j|i}(f^{(j)})$ since they employ different score functions. For more details, please refer to the literature:Learning with Multiclass AUC: Theory and Algorithms. Zhiyong Yang, Qianqian Xu, Shilong Bao, Xiaochun Cao and Qingming Huang. T-PAMI, 2021.
|
|
Number of components to keep. Should be in True labels or binary label indicators (numpy array with shape (n_samples,)). Number of components to keep. Should be in Prediction score (numpy array with shape (n_samples,) for binary case, (n_samples, n_classes) for multiclass case). Number of components to keep. Should be in {'ovo', 'ova'}, default='ova. Only worked for multiclass cases. Number of components to keep. Should be in True or False, default=True. Only used for 'ovo' regime. If True, an accelerated calculation version for MAUORC is employed. |
|---|---|
|
|
return corresponding AUROC score. |
Example:
from Metrics import AUROC
import numpy as np
# binary cases
y_true = np.asarray([1, 0] * 50)
y_pred = np.random.rand(100)
binary_auc=AUROC(y_true=y_true, y_pred=y_pred)
print(binary_auc)
# multiclass cases
y_true = np.asarray([0, 1, 3, 2, 4] * 10)
y_pred = np.random.rand(50, 5)
# normalized [0,1]
y_pred = np.exp(y_pred)/ np.sum(np.exp(y_pred), axis=1, keepdims=True)
# ova
mauc_ova = AUROC(y_true=y_true, y_pred=y_pred, multi_type='ova')
print(mauc_ova)
# ovo
mauc_ovo = AUROC(y_true=y_true, y_pred=y_pred, multi_type='ovo', acc=True)
print(mauc_ovo)