Background
This article introduces the steps to connect a private cluster setup in your GCP account(BYOA).
Prerequisite
- A private Cluster deployed in a GCP project, we call it Cluster GCP project. In our example, it's staging-data
- A VM with psql or edb-psql installed in the same GCP project or a different one, we call it the client GCP project. The example here is a different project: development-data
- The VPC of the VM has a subnet in the same region as Cluster.
Main Steps
Publish a Service
Step 1 Get the Private IP Address
- Get hostname from EDB Postgres AI console, Cluster page - Connect
- Get the internal IP address of the host, choose one of the tools: ping or nslookup or dig +short <host>, for example
❯ ping p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
PING p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io (10.247.204.8): 56 data bytes
Request timeout for icmp_seq 0
2 packets transmitted, 0 packets received, 100.0% packet loss
❯ dig +short p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
10.247.204.8
❯ nslookup p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
Server: 202.106.0.20
Address: 202.106.0.20#53
Non-authoritative answer:
Name: p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
Address: 10.247.204.8
In this example, the IP address is 10.247.204.8.
Step 2 Get Load Balancer with IP
Go to Cluster GCP project. Here is staging-data
- Go to Network Services / Load balancing
- Find the load balancer name with the internal IP, here is 10.247.204.8
- Copy the name of the load balancer and we will use it in the next step.
Step 3 Publish a Service with Load Balancer
Go to Cluster GCP project. Here is staging-data
- Go to Private Service Connect - PUBLISHED SERVICES - + PUBLISH SERVICE This step is also called creating a service attachment.
- Type: Internal passthrough Network Load Balancer and fill in the Internal Load Balancer
-
Reserve a subnet.
Assign the CIDR for the field IPv4 range from the below 3 subnets.
Recommendations for IP Range:1. allocate at least 8 IP addresses to the CIDR, so the subnet mask should not be greater than 29.2. no overlap with the other reserved IP range, so don't allocate too many IP addresses at one time.
"10.247.214.0/23""10.247.216.0/21""10.247.224.0/19"
**If it returns the below error, get another CIDR range from the above 3 subnets which has no overlaps.
This IPv4 address range overlaps with a subnet you already added. Enter an address range that doesn't overlap. - (Optional) Add the consumer GCP project ID(GCP development-data-381310) to accept connections automatically.
-
ADD SERVICE and get the name of the service attachment.
Create Endpoint from Client Application
Go to the client GCP project, here is development-data
Step 6 Get Client Application's VPC
If there is no existing VM in GCP, you are recommended to create a VM in the same region as Cluster.
Go to your VM, Network Interface - Network, (below default is my VM’s VPC)
Step 7 Create an endpoint with VPC
Go to the client GCP project, here is development-data
- Go to Private Service Connect - CONNECTED ENDPOINTS - + CONNECT ENDPOINT
-
Target: Published Service, and paste the service attachment name from Published Service
- Choose the VPC(of your VM) and select the subnet (if no subnet is available, then please check VPC requirement below )
- Create an IP address, or choose an existing IP that is not used by the other endpoints
- Enable Global Access – Please note, if your VM is running in a different region from the cluster, then Global Access should be enabled.
- Create the endpoint.
- Check the endpoint status is accepted and get the IP address.
If it’s pending, then please check Endpoint is pending, Needs attention or error state
Step 7 Connect to the cluster with IP
Setup Private DNS
This step will allow you to connect the cluster with the host. Go to the client GCP project. Here is development-data
Step 8 Create a Private DNS Zone
- Go to Network services, Cloud DNS, + CREATE ZONE
- Zone Type: Private
- Give a Zone name with your preference
-
DNS name is from your cluster
- Options: Default
-
Add Networks: with the VPC of the application, and CREATE
Step 9 Add Standard Record
- Go to the created DNS Zone and + ADD STANDARD record
- DNS name: fill in cluster id,
-
IP Address: IP address from the created endpoint (here is 10.3.0.2)
Step 10 Connect to the cluster with the connection string
Appendix A - Possible issues
VPC Requirement
If the application side VPC has not subnet in the same region as the cluster, then it will return the below errors when creating the endpoint.
Solution: add a subnet in that region
Endpoint is Pending, Needs attention, or Error state
Pending state - possible cause 1
It’s due to the published service has not accepted this project.
Solution: go to Cluster GCP Project-1(published service side), and accept the project. When the project is accepted. then the following endpoints will be accepted automatically.
Pending state - possible cause 2
You have set connection limit when accepting the project. And currently connected endpoints exceed the limit.
For example, when the connection limit is 1 and my 2nd private endpoint will be pending.
Needs Attention state
IP addresses in the reserved IP range we set before being used up. The reserved IP range decides how many endpoints(CIDR - 4 reserved by EDB Postgres AI) can connect to this Published Service.
For example, if we allocate 8 IP addresses to be reserved before(the subnet mask is 29), then there will be at most 4 endpoints that can connect to this Service. The fifth endpoint will not be accepted.
Error state
If the IP address is used by the other endpoint, then it will be in an error state. Solution: create a new IP when creating the endpoint.