Cortex uses Docker images to deploy your models. These images can be replaced with custom images that you can augment with your system packages and libraries. You will need to push your custom images to a container registry that your cluster has access to (e.g. Docker Hub or AWS ECR).
Create a Dockerfile to build your custom image:
mkdir my-api && cd my-api && touch Dockerfile
The Docker images used to deploy your models are listed below. Based on the Cortex Predictor and compute type specified in your API configuration, choose a Cortex image to use as the base for your custom Docker image.
Python (CPU): cortexlabs/python-serve:0.15.1
Python (GPU): cortexlabs/python-serve-gpu:0.15.1
TensorFlow (CPU or GPU): cortexlabs/tf-api:0.15.1
ONNX (CPU): cortexlabs/onnx-serve:0.15.1
ONNX (GPU): cortexlabs/onnx-serve-gpu:0.15.1
Note that the Docker image version must match your cluster version displayed in
The sample Dockerfile below inherits from Cortex's Python CPU serving image and installs the
tree system package.
# DockerfileFROM cortexlabs/python-serve:0.15.1RUN apt-get update \&& apt-get install -y tree \&& apt-get clean && rm -rf /var/lib/apt/lists/*
Create a repository to store your image:
# We create a repository in ECRexport AWS_ACCESS_KEY_ID="***"export AWS_SECRET_ACCESS_KEY="***"eval $(aws ecr get-login --no-include-email --region us-west-2)aws ecr create-repository --repository-name=org/my-api --region=us-west-2# take note of repository url
Build the image based on your Dockerfile and push to its repository in ECR:
docker build . -t org/my-api:latest -t <repository_url>:latestdocker push <repository_url>:latest
Update your cluster configuration file to point to your image:
# cluster.yaml# ...image_python_serve: <repository_url>:latest# ...
Update your cluster for the change to take effect:
cortex cluster update --config=cluster.yaml
Cortex will use your custom image to launch workloads and you will have access to any packages you added:
# predictor.pyimport subprocessclass PythonPredictor:def __init__(self, config):subprocess.run(["tree"])...