The types of evaluation metrics used in measuring the accuracy of your predictive query will vary depending on your target type (i.e., what kind of prediction problem you’re solving).

​
Binary Classification

If your predictive query is determining one of two values (e.g., true/false, yes/no), Kumo provides the following evaluation metrics:
  • Accuracy
  • Area Under The Receiver Operating Characteristic Curve (AUROC)
  • Area Under The Precision-Recall Curve (AUPRC)
  • Confusion Matrix
  • Gain Chart

​
Multiclass Classification

If your predictive query is distinguishing between three or more values (e.g., the LASTvalue from a high cardinality categorical column in the target table), Kumo provides the following evaluation metrics:
  • Accuracy

​
Multilabel Classification

If your predictive query returns one or more values from a list of categorical value candidates (e.g., the LIST_DISTINCT of values from a high cardinality categorical column in the target table), Kumo provides the following macro/micro/per-label versions of the following evaluation metrics:
  • AUPRC
  • AUROC
  • AP
If your predictive query uses LIST_DISTINCT over a foreign key column, it will trigger a multilabel classification task.
If your pQuery returns a list of distinct values associated with your entity (e.g. LIST_DISTINCT), Kumo provides top-k retrieval metrics (for K = 1, 10, and 100):
  • F1@K
  • MAP@K
  • Precision@K
  • Recall@K
Your pQuery is deemed a link prediction task if it uses LIST_DISTINCT over a foreign key column; otherwise it will trigger a multilabel classification task.

​
Regression

If your pQuery returns a numeric value (e.g. the SUM/MAX/etc. of values from a numerical column in the target table), Kumo provides the following metrics:
  • Mean Absolute Error (MAE)
  • Mean Squared Error (MSE)
  • Root Mean Square Error (RMSE)
  • Mean Absolute Percentage Error (MAPE)
  • Symmetric Mean Absolute Percentage Error (SMAPE)
  • Distribution of Predictions Histogram