Skip to main content

GCP | Connect to BigAnimal private cluster using GCP Private Endpoint

shanshan zhao
  • Updated

Background

This article introduces the steps to connect a private BigAnimal cluster setup in your GCP account(BYOA).

For BAH private connection setup, please kindly refer to GCP BAH | Connect to BigAnimal hosted private cluster using GCP Private Endpoint

Prerequisite

  • A private BigAnimal Cluster deployed in a GCP project, we call it BigAnimal GCP project. In our example, it's staging-data 
  • A VM with psql or edb-psql installed in the same GCP project or a different one, we call it the client GCP project. The example here is a different project: development-data 
  • The VPC of the VM has a subnet in the same region as BigAnimal Cluster. 

Main Steps

  1. Publish a Service from BigAnimal
    1. Get the Private IP Address
    2. Get Load Balancer with IP
    3. Publish a Service with Load Balancer
  2. Create Endpoint from Client Application
    1. Get Client Application's VPC
    2. Create an endpoint with VPC
  3. Setup Private DNS 
    1. Create a Private DNS Zone
    2. Add Standard Record

Publish a Service from BigAnimal

Step 1 Get the Private IP Address

  1. Get hostname from BigAnimal portal, Cluster page - Connect
  2. Get the internal IP address of the host, choose one of the tools: ping or nslookup or dig +short <host>, for example 
    ❯ ping p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
    PING p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io (10.247.204.8): 56 data bytes
    Request timeout for icmp_seq 0

    2 packets transmitted, 0 packets received, 100.0% packet loss
    ❯ dig +short p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
    10.247.204.8

    ❯ nslookup p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
    Server: 202.106.0.20
    Address: 202.106.0.20#53

    Non-authoritative answer:
    Name: p-7pz5nk2dov.private.qom5sdktpyhpsq94.s.edbcloud.io
    Address: 10.247.204.8

In this example, the IP address is 10.247.204.8.

Step 2 Get Load Balancer with IP

Go to BigAnimal GCP project. Here is staging-data 

  1. Go to Network Services / Load balancing
  2. Find the load balancer name with the internal IP,  here is 10.247.204.8
  3. Copy the name of the load balancer and we will use it in the next step.

Step 3 Publish a Service with Load Balancer

Go to BigAnimal GCP project. Here is staging-data 

  1. Go to Private Service Connect - PUBLISHED SERVICES - + PUBLISH SERVICE This step is also called creating a service attachment.
  2. Type: Internal passthrough Network Load Balancer and fill in the Internal Load Balancer

  3. Reserve a subnet.

    Assign the CIDR for the field IPv4 range from the below 3 subnets.

    Recommendations for IP Range:1. allocate at least 8 IP addresses to the CIDR, so the subnet mask should not be greater than 29.2. no overlap with the other reserved IP range, so don't allocate too many IP addresses at one time.

    "10.247.214.0/23""10.247.216.0/21""10.247.224.0/19"
    **If it returns the below error, get another CIDR range from the above 3 subnets which has no overlaps.
    This IPv4 address range overlaps with a subnet you already added. Enter an address range that doesn't overlap.
  4. (Optional) Add the consumer GCP project ID(GCP development-data-381310) to accept connections automatically.
  5. ADD SERVICE and get the name of the service attachment.

Create Endpoint from Client Application

Go to the client GCP project, here is development-data

Step 6 Get Client Application's VPC

If there is no existing VM in GCP, you are recommended to create a VM in the same region as BigAnimal Cluster.

Go to your VM, Network Interface - Network, (below default is my VM’s VPC)

 

Step 7 Create an endpoint with VPC

Go to the client GCP project, here is development-data

  1. Go to Private Service Connect - CONNECTED ENDPOINTS - + CONNECT ENDPOINT
     
  2. Target: Published Service, and paste the service attachment name from Published Service
  3. Choose the VPC(of your VM) and select the subnet (if no subnet is available, then please check VPC requirement below )
  4. Create an IP address, or choose an existing IP that is not used by the other endpoints
  5. Enable Global Access – Please note, if your VM is running in a different region from BigAnimal, then Global Access should be enabled. 
  6. Create the endpoint.
  7. Check the endpoint status is accepted and get the IP address.
    If it’s pending, then please check Endpoint is pending, Needs attention or error state

Step 7 Connect to BigAnimal with IP

 

Setup Private DNS

This step will allow you to connect BigAnimal with the host. Go to the client GCP project, here is development-data

Step 8 Create a Private DNS Zone

  1. Go to Network services, Cloud DNS, + CREATE ZONE
  2. Zone Type: Private
  3. Give a Zone name with your preference
  4. DNS name is from your BigAnimal Cluster’s host, remove the cluster id.
  5. Options: Default
  6. Add Networks: with the VPC of the application, and CREATE

 

Step 9 Add Standard Record

  1. Go to the created DNS Zone and + ADD STANDARD record
  2. DNS name: fill in BigAnimal cluster id, 
  3. IP Address: IP address from the created endpoint (here is 10.3.0.2)

Step 10 Connect to BigAnimal with the connection string 

Appendix A - Possible issues

VPC Requirement

If the application side VPC has not subnet in the same region as BigAnimal Cluster, then it will return the below errors when creating the endpoint. 

Solution: add a subnet in that region

Endpoint is Pending, Needs attention, or Error state

Pending state - possible cause 1

It’s due to the published service has not accepted this project. 

Solution: go to BigAnimal GCP Project-1(published service side), and accept the project. When the project is accepted. then the following endpoints will be accepted automatically.

 

Pending state - possible cause 2 

You have set connection limit when accepting the project. And currently connected endpoints exceed the limit.

For example, when the connection limit is 1 and my 2nd private endpoint will be pending.

 

Needs Attention state

IP addresses in the reserved IP range we set before being used up. The reserved IP range decides how many endpoints(CIDR - 4 reserved by BigAnimal) can connect to this Published Service. 

For example, if we allocate 8 IP addresses to be reserved before(subnet mask is 29), then there will be at most 4 endpoints that can connect to this Service. Then the 5th endpoint will not be accepted.

Error state

If the IP address is used by the other endpoint, then it will be in an error state. Solution: create a new IP when creating the endpoint.