Set up VPC peering

If you are using an internal operator load balancer (i.e. you set operator_load_balancer_scheme: internal in your cluster configuration file before creating your cluster), you can use VPC Peering to enable your Cortex CLI to connect to your cluster operator from another VPC so that you may run cortex commands. Note that because the operator validates that the CLI user is an active IAM user in the same AWS account as the Cortex cluster, it is usually unnecessary to configure the operator's load balancer to be internal.
If you are using an internal API load balancer (i.e. you set api_load_balancer_scheme: internal in your cluster configuration file before creating your cluster) and you disabled API Gateway for your API (i.e. you set api_gateway: none in the networking field of your api configuration), you can use VPC Peering to enable prediction requests from another VPC.
This guide illustrates how to create a VPC Peering connection between a VPC of your choice and the Cortex load balancers.
Step 1
Navigate to AWS's EC2 Load Balancer dashboard and locate the Cortex operator's load balancer. You can determine which is the operator load balancer by inspecting the kubernetes.io/service-name tag:
step 1a
Click back to the "Description" tab and note the VPC ID of the load balancer and the ID of each of the subnets associated with the load balancer:
step 1b
Navigate to AWS's VPC dashboard and identify the ID and CIDR block of Cortex's VPC:
step 1c
The VPC ID here should match that of the load balancer.
Step 2
Identify the ID and CIDR block of the VPC from which you'd like to connect to the Cortex VPC.
In my case, I have a VPC in the same AWS account and region, and I can locate its ID and CIDR block from AWS's VPC dashboard:
step 2
Step 3
From AWS's VPC dashboard, navigate to the "Peering Connections" page, and click "Create Peering Connection":
step 3a
Name your new VPC Peering Connection (I used "cortex-operator", but "cortex" or "cortex-api" may make more sense depending on your use case). Then configure the connection such that the "Requester" is the VPC from which you'll connect to the Cortex VPC, and the "Accepter" is Cortex's VPC (from step 1).
step 3b
Click "Create Peering Connection", navigate back to the Peering Connections dashboard, select the newly created peering connection, and click "Actions" > "Accept Request":
step 3c
step 3d
Step 4
Navigate to the VPC Route Tables page. Select the route table for the VPC from which you'd like to connect to the Cortex cluster (in my case, I just have one route table for this VPC). Select the "Routes" tab, and click "Edit routes":
step 4a
Add a route where the "Destination" is the CIDR block for Cortex's VPC (identified in Step 1), and the "Target" is the newly-created Peering Connection:
step 4b
Do not create new route tables or change subnet associations.
Step 5
Navigate back to the VPC Route Tables page. There will be a route table for each of the subnets associated with the Cortex operator load balancer (identified in Step 1):
step 5a
For each of these route tables, click "Edit routes" and add a new route where the "Destination" is the CIDR block for the VPC from which you will be connecting to the Cortex cluster (identified in Step 2), and the "Target" is the newly-created Peering Connection:
step 5b
Repeat adding this route for each route table associated with the Cortex operator's subnets; in my case there were three. Do not create new route tables or change subnet associations.
You should now be able to use the Cortex CLI and make prediction requests from your VPC.
Cleanup
Delete the VPC Peering connection before spinning down your Cortex cluster:
cleanup

Guides - Previous

Set up a custom domain

Next - Guides

SSH into worker instance

Last modified 10mo ago

Copy link

Contents