Development

Remote development
Unless your internet connection is very fast or you will only be working on the CLI, it is recommended to run your development environment on a cloud instance, e.g. an AWS EC2 instance or GCP VM (due to frequent docker registry pushing). There are a variety of ways to develop on a remote VM, feel free to reach out on our gitter and we can point you in the right direction based on your operating system and editor preferences.
Prerequisites
1.Go (>=1.14)
2.Docker
3.eksctl
4.kubectl
5.aws-cli
Also, please use the VS Code yaml extension and enable auto-formatting for YAML files.
Go
To install Go on linux, run:
1
wget https://dl.google.com/go/go1.14.7.linux-amd64.tar.gz && \
2
sudo tar -xvf go1.14.7.linux-amd64.tar.gz && \
3
sudo mv go /usr/local && \
4
rm go1.14.7.linux-amd64.tar.gz
Copied!
Docker
To install Docker on Ubuntu, run:
1
sudo apt install docker.io && \
2
sudo systemctl start docker && \
3
sudo systemctl enable docker && \
4
sudo groupadd docker && \
5
sudo gpasswd -a $USER docker
Copied!
eksctl
To install eksctl run:
1
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp && \
2
sudo mv /tmp/eksctl /usr/local/bin
Copied!
kubectl
To install kubectl on linux, run:
1
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl && \
2
chmod +x ./kubectl && \
3
sudo mv ./kubectl /usr/local/bin/kubectl
Copied!
aws-cli (v1)
Follow these instructions to install aws-cli (v1).
E.g. to install it globally, run:
1
sudo python -m pip install awscli
2
​
3
aws configure
Copied!
Cortex dev environment
Clone the repo
Clone the project:
1
git clone https://github.com/cortexlabs/cortex.git
2
cd cortex
Copied!
Run the tests:
1
make test
Copied!
Image Registry
Create a config directory in the repo's root directory:
1
mkdir dev/config
Copied!
Next, create dev/config/build.sh. Add the following content to it (you may use a different region for REGISTRY_REGION):
1
export CORTEX_VERSION="master"
2
​
3
export REGISTRY_REGION="us-west-2"
Copied!
Create the AWS Elastic Container Registry:
1
make registry-create
Copied!
Take note of the registry URL, this will be needed shortly.
Create the S3 buckets:
1
aws s3 mb s3://cortex-cluster-<your_name>
2
aws s3 mb s3://cortex-cli-<your_name>  # if you'll be uploading your compiled CLI
Copied!
Cluster
Update dev/config/build.sh. Paste the following config, and update CLI_BUCKET_NAME, CLI_BUCKET_REGION, REGISTRY_URL (the), and REGISTRY_REGION accordingly:
1
export CORTEX_VERSION="master"
2
​
3
export REGISTRY_REGION="us-west-2"
4
export REGISTRY_URL="XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com"
5
​
6
# optional, only used for dev/build_cli.sh
7
export CLI_BUCKET_NAME="cortex-cli-<your_name>"
8
export CLI_BUCKET_REGION="us-west-2"
Copied!
Create dev/config/cluster.yaml. Paste the following config, and update cortex_bucket, cortex_region, and all registry URLs accordingly:
1
instance_type: m5.large
2
min_instances: 2
3
max_instances: 5
4
bucket: cortex-cluster-<your_name>
5
region: us-west-2
6
cluster_name: cortex
7
​
8
image_operator: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/operator:latest
9
image_manager: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/manager:latest
10
image_downloader: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/downloader:latest
11
image_request_monitor: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/request-monitor:latest
12
image_cluster_autoscaler: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/cluster-autoscaler:latest
13
image_metrics_server: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/metrics-server:latest
14
image_inferentia: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/inferentia:latest
15
image_neuron_rtd: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/neuron-rtd:latest
16
image_nvidia: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/nvidia:latest
17
image_fluentd: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/fluentd:latest
18
image_statsd: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/statsd:latest
19
image_istio_proxy: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/istio-proxy:latest
20
image_istio_pilot: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs/istio-pilot:latest
Copied!
Building
Add this to your bash profile (e.g. ~/.bash_profile, ~/.profile or ~/.bashrc):
1
export CORTEX_DEV_DEFAULT_PREDICTOR_IMAGE_REGISTRY="XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/cortexlabs"  # set the default image for APIs
2
export CORTEX_TELEMETRY_SENTRY_DSN="https://[email protected]/1848098"  # redirect analytics to our dev environment
3
export CORTEX_TELEMETRY_SEGMENT_WRITE_KEY="0WvoJyCey9z1W2EW7rYTPJUMRYat46dl"  # redirect error reporting to our dev environment
4
​
5
export AWS_ACCESS_KEY_ID=""
6
export AWS_SECRET_ACCESS_KEY=""
7
​
8
alias cortex-dev='<path/to/cortex>/bin/cortex'  # replace <path/to/cortex> with the path to the cortex repo that you cloned
Copied!
Refresh your bash profile:
1
. ~/.bash_profile  # or: `. ~/.bashrc`
Copied!
Build and push all Cortex images:
1
make registry-all
Copied!
Build the Cortex CLI:
1
make cli  # the binary will be placed in <path/to/cortex>/bin/cortex
2
cortex-dev version  # should show "master"
Copied!
If you wish to parallelize the build process, the parallel GNU utility needs to be installed. Once installed, set the NUM_BUILD_PROCS environment variable to the desired number of parallel jobs. For ease of use, export NUM_BUILD_PROCS in dev/config/build.sh.
Cortex cluster
Start Cortex:
1
make cluster-up
Copied!
Tear down the Cortex cluster:
1
make cluster-down
Copied!
Deploy an example
1
cortex deploy examples/pytorch/iris-classifier --env aws
Copied!
Off-cluster operator
If you're making changes in the operator and want faster iterations, you can run an off-cluster operator.
1.make tools to install the necessary dependencies to run the operator
2.make operator-stop to stop the in-cluster operator
3.make devstart to run the off-cluster operator (which rebuilds the CLI and restarts the Operator when files change)
If you want to switch back to the in-cluster operator:
1.<ctrl+c> to stop your off-cluster operator
2.make cluster-configure to install the operator in your cluster
Dev workflow
1.make cluster-up
2.make devstart
3.Make changes
4.make registry-dev
5.Test your changes with projects in examples or your own
See Makefile for additional dev commands.
Feel free to chat with us if you have any questions.