GCP | Connect to private cluster using GCP Private Endpoint – EnterpriseDB

Background

This article introduces the steps to connect a private cluster setup in your GCP account(BYOA).

Prerequisite

A private Cluster deployed in a GCP project, we call it Cluster GCP project. In our example, it's staging-data
A VM with psql or edb-psql installed in the same GCP project or a different one, we call it the client GCP project. The example here is a different project: development-data
The VPC of the VM has a subnet in the same region as Cluster.

Main Steps

Publish a Service
Create Endpoint from Client Application
1. Get Client Application's VPC
2. Create an endpoint with VPC
Setup Private DNS
1. Create a Private DNS Zone
2. Add Standard Record

Publish a Service

Step 1 Get the Private IP Address

Get hostname from EDB Postgres AI console, Cluster page - Connect

Get the internal IP address of the host, choose one of the tools: ping or nslookup or dig +short <host>, for example

❯ ping p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
PING p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io (10.247.204.8): 56 data bytes
Request timeout for icmp_seq 0

2 packets transmitted, 0 packets received, 100.0% packet loss
❯ dig +short p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
10.247.204.8

❯ nslookup p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
Server: 202.106.0.20
Address: 202.106.0.20#53

Non-authoritative answer:
Name: p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
Address: 10.247.204.8

In this example, the IP address is 10.247.204.8.

Step 2 Get Load Balancer with IP

Go to Cluster GCP project. Here is staging-data

Go to Network Services / Load balancing
Find the load balancer name with the internal IP, here is 10.247.204.8
Copy the name of the load balancer and we will use it in the next step.

Step 3 Publish a Service with Load Balancer

Go to Cluster GCP project. Here is staging-data

Go to Private Service Connect - PUBLISHED SERVICES - + PUBLISH SERVICE This step is also called creating a service attachment.
Type: Internal passthrough Network Load Balancer and fill in the Internal Load Balancer
Reserve a subnet.

Assign the CIDR for the field IPv4 range from the below 3 subnets.

Recommendations for IP Range:1. allocate at least 8 IP addresses to the CIDR, so the subnet mask should not be greater than 29.2. no overlap with the other reserved IP range, so don't allocate too many IP addresses at one time.
```
"10.247.214.0/23""10.247.216.0/21""10.247.224.0/19"
```
**If it returns the below error, get another CIDR range from the above 3 subnets which has no overlaps.
This IPv4 address range overlaps with a subnet you already added. Enter an address range that doesn't overlap.
(Optional) Add the consumer GCP project ID(GCP development-data-381310) to accept connections automatically.
ADD SERVICE and get the name of the service attachment.

Create Endpoint from Client Application

Go to the client GCP project, here is development-data

Step 6 Get Client Application's VPC

If there is no existing VM in GCP, you are recommended to create a VM in the same region as Cluster.

Go to your VM, Network Interface - Network, (below default is my VM’s VPC)

Step 7 Create an endpoint with VPC

Go to the client GCP project, here is development-data

Go to Private Service Connect - CONNECTED ENDPOINTS - + CONNECT ENDPOINT
Target: Published Service, and paste the service attachment name from Published Service
Choose the VPC(of your VM) and select the subnet (if no subnet is available, then please check VPC requirement below )
Create an IP address, or choose an existing IP that is not used by the other endpoints
Enable Global Access – Please note, if your VM is running in a different region from the cluster, then Global Access should be enabled.
Create the endpoint.
Check the endpoint status is accepted and get the IP address.
If it’s pending, then please check Endpoint is pending, Needs attention or error state

Step 7 Connect to the cluster with IP

Setup Private DNS

This step will allow you to connect the cluster with the host. Go to the client GCP project. Here is development-data

Step 8 Create a Private DNS Zone

Go to Network services, Cloud DNS, + CREATE ZONE
Zone Type: Private
Give a Zone name with your preference
DNS name is from your cluster
Options: Default
Add Networks: with the VPC of the application, and CREATE

Step 9 Add Standard Record

Go to the created DNS Zone and + ADD STANDARD record
DNS name: fill in cluster id,
IP Address: IP address from the created endpoint (here is 10.3.0.2)

Step 10 Connect to the cluster with the connection string

Appendix A - Possible issues

VPC Requirement

If the application side VPC has not subnet in the same region as the cluster, then it will return the below errors when creating the endpoint.

Solution: add a subnet in that region

Endpoint is Pending, Needs attention, or Error state

Pending state - possible cause 1

It’s due to the published service has not accepted this project.

Solution: go to Cluster GCP Project-1(published service side), and accept the project. When the project is accepted. then the following endpoints will be accepted automatically.

Pending state - possible cause 2

You have set connection limit when accepting the project. And currently connected endpoints exceed the limit.

For example, when the connection limit is 1 and my 2nd private endpoint will be pending.

Needs Attention state

IP addresses in the reserved IP range we set before being used up. The reserved IP range decides how many endpoints(CIDR - 4 reserved by EDB Postgres AI) can connect to this Published Service.

For example, if we allocate 8 IP addresses to be reserved before(the subnet mask is 29), then there will be at most 4 endpoints that can connect to this Service. The fifth endpoint will not be accepted.

Error state

If the IP address is used by the other endpoint, then it will be in an error state. Solution: create a new IP when creating the endpoint.