tracker can be configured to collect API prediction metrics and display real-time stats in cortex get <api_name>. The tracker looks for scalar values in the response payload. If the response payload is a JSON object, key can be set to extract the desired scalar value.

- kind: api
  name: <string>  # API name (required)
  ...
  tracker:
    key: <string>  # the JSON key in the response to track (required if the response payload is a JSON object)
    model_type: <string>  # model type, must be "classification" or "regression" (required)
  ...

For classification models, the tracker should be configured with model_type: classification to collect integer or string values and display the class distribution. For regression models, the tracker should be configured with model_type: regression to collect float values and display regression stats such as min, max and average.

Example

- kind: api
  name: iris
  predictor:
    path: predictor.py
  tracker:
    model_type: classification