Install

Install the CLI
1
pip install cortex
Copied!
You must have Docker installed to run Cortex locally or to create a cluster on AWS.
See here to install Cortex CLI without the Python Client.
Deploy an example
1
# clone the Cortex repository
2
git clone -b 0.22 https://github.com/cortexlabs/cortex.git
3
​
4
# navigate to the Pytorch text generator example
5
cd cortex/examples/pytorch/text-generator
Copied!
Using the CLI
1
# deploy the model as a realtime api
2
cortex deploy
3
​
4
# view the status of the api
5
cortex get --watch
6
​
7
# stream logs from the api
8
cortex logs text-generator
9
​
10
# get the api's endpoint
11
cortex get text-generator
12
​
13
# generate text
14
curl <API endpoint> \
15
  -X POST -H "Content-Type: application/json" \
16
  -d '{"text": "machine learning is"}'
17
​
18
# delete the api
19
cortex delete text-generator
Copied!
In Python
1
import cortex
2
import requests
3
​
4
local_client = cortex.client("local")
5
​
6
# deploy the model as a realtime api and wait for it to become active
7
deployments = local_client.deploy("./cortex.yaml", wait=True)
8
​
9
# get the api's endpoint
10
url = deployments[0]["api"]["endpoint"]
11
​
12
# generate text
13
print(requests.post(url, json={"text": "machine learning is"}).text)
14
​
15
# delete the api
16
local_client.delete_api("text-generator")
Copied!
Running at scale on AWS
Run the command below to create a cluster with basic configuration, or see cluster configuration to learn how you can customize your cluster with cluster.yaml.
See EC2 instances for an overview of several EC2 instance types. To use GPU nodes, you may need to subscribe to the EKS-optimized AMI with GPU Support and file an AWS support ticket to increase the limit for your desired instance type.
1
# create a Cortex cluster on your AWS account
2
cortex cluster up
3
​
4
# set the default CLI environment (optional)
5
cortex env default aws
Copied!
You can now run the same commands shown above to deploy the text generator to AWS (if you didn't set the default CLI environment, add --env aws to the cortex commands).
Next steps
Try the tutorial to learn more about how to use Cortex.
Deploy one of our examples.
See our exporting guide for how to export your model to use in an API.
View documentation for realtime APIs or batch APIs.
See uninstall if you'd like to spin down your cluster.

Deploy machine learning models to production

Last modified 10mo ago

Copy link

Contents

Running at scale on AWS

Next steps