Model serving at scale

Cortex is a platform for deploying, managing, and scaling machine learning in production.

Run realtime inference, batch inference, and training jobs.

Configure inference endpoints, model retraining, and A/B testing programmatically with Python or interactively with a CLI.

Cortex supports TensorFlow, PyTorch, ONNX and other models.

Cortex is built on top of Kubernetes to support large-scale machine learning workloads.

Export metrics and logs to your favorite monitoring tools.

Model serving on your cloud infrastructure

Pay only for AWS or GCP resources

Cluster management

Realtime and batch workloads

Workload and cluster autoscaling

Spot or preemptible instances

Traffic splitting and A/B testing

Community support

Fully-managed model serving platform

Everything in Core

User management

Admin dashboard

Multi-cloud clusters

Uptime SLAs

Enterprise security

Priority support

GitHub