System packages

Bash script
Cortex looks for a file named dependencies.sh in the top level Cortex project directory (i.e. the directory which contains cortex.yaml). For example:
1
./my-classifier/
2
├── cortex.yaml
3
├── predictor.py
4
├── ...
5
└── dependencies.sh
Copied!
dependencies.sh is executed with bash shell during the initialization of each replica (before installing Python packages in requirements.txt or conda-packages.txt). Typical use cases include installing required system packages to be used in your Predictor, building Python packages from source, etc.
Here is an example dependencies.sh, which installs the tree utility:
1
apt-get update && apt-get install -y tree
Copied!
The tree utility can now be called inside your predictor.py:
1
# predictor.py
2
import subprocess
3
​
4
class PythonPredictor:
5
    def __init__(self, config):
6
        subprocess.run(["tree"])
7
    ...
Copied!
Custom Docker image
You can also build a custom Docker image for use in your APIs, e.g. to avoid installing dependencies during replica initialization.
Create a Dockerfile
Create a Dockerfile to build your custom image:
1
mkdir my-api && cd my-api && touch Dockerfile
Copied!
Cortex's base Docker images are listed below. Depending on the Cortex Predictor and compute type specified in your API configuration, choose one of these images to use as the base for your Docker image:
Python Predictor (CPU): cortexlabs/python-predictor-cpu-slim:0.22.1
Python Predictor (GPU): cortexlabs/python-predictor-gpu-slim:0.22.1-cuda10.1 (also available in cuda10.0, cuda10.2, and cuda11.0)
Python Predictor (Inferentia): cortexlabs/python-predictor-inf-slim:0.22.1
TensorFlow Predictor (CPU, GPU, Inferentia): cortexlabs/tensorflow-predictor-slim:0.22.1
ONNX Predictor (CPU): cortexlabs/onnx-predictor-cpu-slim:0.22.1
ONNX Predictor (GPU): cortexlabs/onnx-predictor-gpu-slim:0.22.1
Note: the images listed above use the -slim suffix; Cortex's default API images are not -slim, since they have additional dependencies installed to cover common use cases. If you are building your own Docker image, starting with a -slim Predictor image will result in a smaller image size.
The sample Dockerfile below inherits from Cortex's Python CPU serving image, and installs 3 packages. tree is a system package and pandas and rdkit are Python packages.
1
# Dockerfile
2
​
3
FROM cortexlabs/python-predictor-cpu-slim:0.22.1
4
​
5
RUN apt-get update \
6
    && apt-get install -y tree \
7
    && apt-get clean && rm -rf /var/lib/apt/lists/*
8
​
9
RUN pip install --no-cache-dir pandas \
10
    && conda install -y conda-forge::rdkit \
11
    && conda clean -a
Copied!
Build your image
1
docker build . -t org/my-api:latest
Copied!
Push your image to a container registry
If you are only running Cortex locally, you can skip this section
You can push your built Docker image to a public registry of your choice (e.g. Docker Hub), or to a private registry on ECR or Docker Hub (for private Docker Hub, also follow this guide to configure access in your cluster).
For example, to use ECR, first create a repository to store your image:
1
# We create a repository in ECR
2
​
3
export AWS_REGION="***"
4
export AWS_ACCESS_KEY_ID="***"
5
export AWS_SECRET_ACCESS_KEY="***"
6
export REGISTRY_URL="***"  # this will be in the format "<aws_account_id>.dkr.ecr.<aws_region>.amazonaws.com"
7
​
8
aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $REGISTRY_URL
9
​
10
aws ecr create-repository --repository-name=org/my-api --region=$AWS_REGION
11
# take note of repository url
Copied!
Build and tag your image, and push it to your ECR repository:
1
docker build . -t org/my-api:latest -t <repository_url>:latest
2
​
3
docker push <repository_url>:latest
Copied!
Configure Cortex
Update your API configuration file to point to your image:
1
# cortex.yaml
2
​
3
- name: my-api
4
  ...
5
  predictor:
6
    image: <repository_url>:latest
7
  ...
Copied!
Note: for TensorFlow Predictors, two containers run together to serve predictions: one runs your Predictor code (cortexlabs/tensorflow-predictor), and the other is TensorFlow serving to load the SavedModel (cortexlabs/tensorflow-serving-gpu or cortexlabs/tensorflow-serving-cpu). There's a second available field tensorflow_serving_image that can be used to override the TensorFlow Serving image. Both of the default serving images (cortexlabs/tensorflow-serving-gpu and cortexlabs/tensorflow-serving-cpu) are based on the official TensorFlow Serving image (tensorflow/serving). Unless a different version of TensorFlow Serving is required, the TensorFlow Serving image shouldn't have to be overridden, since it's only used to load the SavedModel and does not run your Predictor code.
Deploy your API as usual:
1
cortex deploy
Copied!

Advanced - Previous

Python packages

Next - Advanced

Networking

Last modified 10mo ago

Copy link

Contents

Push your image to a container registry

Configure Cortex