First of all, you would use cloud services, because you don't want to store your devices and maintain them.
We have many cloud providers like AWS, GCP, Azure Digital Ocean etc. But whatever provider you will be using – should be cheap and reliable. Let's go then point by point what kind of application do we have and what can we get for it.
— 20 min. read by Marcin Szewczyk
— 01.02.2021
#Terraform, #Pulumi, #Saltstack, #Ansible, #Cloudify
So, let's begin
Introduction
Our application will be selling cooking products from a manufactuer that wants to go worldwide. So, we have a marketing service that has a frontend with SEO capabilities, and a frontend for customer management - both are built in NodeJS. An online shop and a backend in Java Spring-Boot that creates reports and puts it to a database. Besides services we have also static files like compiled js and pictures that should be served on websites. What kind of configuration should we consider then? Well, let's go point by point what comes to our mind:
First of all
Our customer wants to go worldwide, so the cloud provider should have good edge networking.
Second
We will need a contenerization services in order to manage deployments
Third
We want to have low costs - so we need to use managed services
One of the latest innovations in these cases is GCP Cloud Run for managing containers, GCP global load balancing for good edge networking and GCP Firestore for database. Here I provided some guidelines on how to connect those things together.
The easiest part comes with GCP Cloud Run, as we want to build containers and put them online. Cloud Run is the best solution here, because it is like fully manged kubernetes but only with the part to deploy the service and the rest - creating a deployment, scaling and starting a service - is done automatically within Cloud Run. Briefly speaking - you just point to a container and the outcome is a link for your service.
#2
Setup
Before going to deployments we need to setup our workspace, what we need:
Google Cloud account with a project that has billing enabled. Our project will have name cooking-sales
for this example
gcloud
SDK for command-line use. It needs to be downloaded and initiated, guide can be found here. For the default region we will be using europe-west1
#3
Building and publishing images
We have three services to deploy:
marketing-service
marketing service built with NodeJS
customer-service
Customer management - NodeJS
shop-service
Online shop - Java Spring-Boot
Each of them must be built locally and you should create a docker image out of every service:
$ cd marketing-service
$ docker build -t marketing-service .
$ cd ../customer-service
$ docker build -t customer-service .
$ cd ../shop-service
$ docker build -t shop-service .
Try to build the images as small as possible.
For Cloud Run you need to store the image on GCR (Google Container Registry) if you want your images to be private. You can also store them in any other public registry as long as the image is publicly available - no privacy here.
To publish images to GCR you need to enable the registry, retag the image and push the image:
1. Enable the service
$ gcloud services enable containerregistry.googleapis.com
2. Before you can push images you need to authenticate yourself within the docker daemon.
Best way is to use credential helper
3. Now you can retag the images and push them, for each of the services just do:
$ docker tag marketing-service eu.gcr.io/cooking-sales/marketing-service:latest
$ docker tag customer-service eu.gcr.io/cooking-sales/customer-service:latest
$ docker tag shop-service eu.gcr.io/cooking-sales/shop-service:latest
#4
Deploying basic Cloud Run services
Now, we have our images in the registry, then it's time to launch first cloud run services.
$ gcloud run deploy "marketing-service" \
--image eu.gcr.io/cooking-sales/marketing-service:latest \
--region europe-west1 \
--project cooking-sales \
--platform managed \
--memory=512Mi \
--port=3000
Deploying...
Creating Revision..........................................done
Routing traffic..................done
Done.
Service [marketing-service] revision [marketing-service-00081-aac] has been deployed and is serving 100 percent of traffic.
Service URL: https://marketing-service-ad5yllhq1b-es.a.run.app
$ gcloud run deploy "customer-service" \
--image eu.gcr.io/cooking-sales/customer-service:latest \
--region europe-west1 \
--project cooking-sales \
--platform managed \
--memory=512Mi \
--port=3000
Deploying...
Creating Revision...................................done
Routing traffic...................done
Done.
Service [customer-service] revision [customer-service-00081-bvs] has been deployed and is serving 100 percent of traffic.
Service URL: https://customer-service-293fj019jhfaa.a.run.app
$ gcloud run deploy "shop-service" \
--image eu.gcr.io/cooking-sales/shop-service:latest \
--region europe-west1 \
--project cooking-sales \
--platform managed \
--memory=1024Mi \
--port=8080
Deploying...
Creating Revision..........................done
Routing traffic.................done
Done.
Service [shop-service] revision [shop-service-00081-bvs] has been deployed and is serving 100 percent of traffic.
Service URL: https://shop-service-0fdf250vmasss.a.run.app
#5
Issues so far
As it seems we have our services deployed, but the setup is not finished yet, we still have some things to do:
1.
Our static resources are served by services, but this is not efficient as this might be our bottleneck.
2.
Service URLs are not correct and we don't want our customers to be vising strange run.app
hosts.
3.
The shop-service
has long cold start and eventually will make users wait to enter the shop for about 1 minutes and 30 seconds.
#6
Static resources
Best way of serving static resources is to save them on Google Storage in some bucket and making this bucket publicly visible. Also we want the static resources to be accessible within the /static
path in the address
1. Using command line enable google cloud storage.
$ gcloud services enable storage-api.googleapis.com
$ gcloud services enable storage-component.googleapis.com
2. Compile static resources with any tool that you have (e.g. yarn build
etc.)
3. Create a bucket that will serve the resurces
$ gsutils mb \
-p cooking-sales \
-l EU \
gs://cooking-sales-website-static-resources
With this command we have created a bucket that is multiregion and placed in datacenters in EU. Please, mind that bucket name should be globally unique.
4. Copy all resources to your bucket. In this point static
is the directory and favicon.ico is the file with website icon.
$ gsutil -m cp -r static favicon.ico gs://cooking-sales-website-static-resources/
5. Make all of the resources readable on the internet with public visibility.
$ gsutil acl ch AllUsers:objectViewer gs://cooking-sales-website-static-resources/
#7
Load Balanced Services
We have sent static resources to the bucket, but the URL of the bucket does not satisfy us and also we need to serve our services with proper URLs. With succor comes Global Load Balancer which we will be using not only to load balance our resources, but also to properly set traffic routes.
1. For services we need to create Network Endpoint Groups, use the command line:
$ gcloud compute network-endpoint-groups create \
marketing-service-neg \
--network-endpoint-type SERVERLESS \
--cloud-run-service marketing-service \
--project cooking-sales \
--region europe-west1
$ gcloud compute network-endpoint-groups create \
customer-service-neg \
--network-endpoint-type SERVERLESS \
--cloud-run-service customer-service \
--project cooking-sales \
--region europe-west1
$ gcloud compute network-endpoint-groups create \
shop-service-neg \
--network-endpoint-type SERVERLESS \
--cloud-run-service shop-service \
--project cooking-sales \
--region europe-west1
2. Next, Google Load Balancers require us to set the backend services, as we can have multiple services and they can be balanced on different terms. For those services there should set proper backends which will be our network endpoint groups we created in previous point.
$ gcloud compute backend-services create \
marketing-service-bservice \
--global \
--project cooking-sales
$ gcloud compute backend-services add-backend \
marketing-service-bservice \
--network-endpoint-group marketing-service-neg \
--global \
--project cooking-sales
$ gcloud compute backend-services create \
customer-service-bservice \
--global \
--project cooking-sales
$ gcloud compute backend-services add-backend \
customer-service-bservice \
--network-endpoint-group customer-service-neg \
--global \
--project cooking-sales
$ gcloud compute backend-services create \
shop-service-bservice \
--global \
--project cooking-sales
$ gcloud compute backend-services add-backend \
shop-service-bservice \
--network-endpoint-group shop-service-neg \
--global \
--project cooking-sales
3. Now we need also to create backend service for the static resources in bucket.
$ gcloud compute backend-buckets create \
static-resources-bbucket \
--gcs-bucket-name cooking-sales-website-static-resources \
--enable-cdn
#8
Launching global load balancer
The last stage is to setup the global load balancer, which requires:
• Proper URL Map
• TLS certificates
• Forwarded ports (because we can have services internally load balanced)
To setup URL maps we have to first organize how our application should be visited. We have a domain spicy-pot.com
and also we have complete control over it. So:
spicy-pot.com
- this will be the page that will access marketing-service
.customer.spicy-pot.com
- this page will show the customer-service
.shop.spicy-pot.com
- this will access the shop-service
.
Don't forget to serve /static
for all of the services.
1. Create a yaml file with name main-url-map.yaml
with contents:
defaultService: https://www.googleapis.com/compute/v1/projects/cooking-sales/global/backendServices/marketing-service-bservice
hostRules:
- hosts:
- `spicy-pot.com`
pathMatcher: marketingpaths
- hosts:
- 'customer.spicy-pot.com'
pathMatcher: customerpaths
- hosts:
- 'shop.spicy-pot.com'
pathMatcher: shoppaths
pathMatchers:
- name: marketingpaths
defaultService: https://www.googleapis.com/compute/v1/projects/cooking-sales/global/backendServices/marketing-service-bservice
pathRules:
- paths:
- '/static'
- 'favicon.ico'
service: https://www.googleapis.com/compute/v1/projects/cooking-sales/global/backendBuckets/static-resources-bbucket
- name: customerpaths
defaultService: https://www.googleapis.com/compute/v1/projects/cooking-sales/global/backendServices/customer-service-bservice
pathRules:
- paths:
- '/static'
- 'favicon.ico'
service: https://www.googleapis.com/compute/v1/projects/cooking-sales/global/backendBuckets/static-resources-bbucket
- name: shoppaths
defaultService: https://www.googleapis.com/compute/v1/projects/cooking-sales/global/backendServices/shop-service-bservice
pathRules:
- paths:
- '/static'
- 'favicon.ico'
service: https://www.googleapis.com/compute/v1/projects/cooking-sales/global/backendBuckets/static-resources-bbucket
2. Upload this URL map
$ gcloud compute url-maps import main-url-map --source main-url-map.yaml
3. You need also to setup SSL certificates in order for HTTPS to work. The certificates is managed by google, so it also needs to be verified which is explained further.
$ gcloud compute ssl-certificates create \
general-cooking-sales-certificate \
--domains"sales-pot.com,customer.sales-pot.com,shop.sales-pot.com" \
--global
4. Obtain a global IP address and create a DNS record within your provider with a type A. The address can be obtained like this:
$ gcloud compute addresses describe \
general-cooking-sales-address \
--global \
--format="get(address)"
For all the domains set the same IP address:
— sales-pot.com
— customer.sales-pot.com
— shop.sales-pot.com
5. With URL map created, SSL certificate added and IP addresses set in DNS we can now create the load balancer entity which is also called "target HTTPS proxy"
$ gcloud compute target-https-proxies create \
general-cooking-sales-https-lb \
--url-map main-url-map \
--ssl-certificates general-cooking-sales-certificate \
--global \
--project cooking-sales
6. One more thing in the end - Load balancer needs to be forwarded.
$ gcloud compute forwarding-rules create \
global-https-fw-rule \
--global \
--address general-cooking-sales-address \
--target-https-proxy general-cooking-sales-https-lb \
--ports 443
From now on you must wait approx. 15 to 45 minutes in order to validate certificates and after validation sites should be reachable within addresses:
— spicy-pot.com
— customer.spicy-pot.com
— shop.spicy-pot.com
Final thoughts
Summary
We have setup a cheap infrastructure for our application which should be reachable globally. Fixed costs related to the infrastructure are billed for bucket, other cost regarding services and load balancing may vary depending on the application usage by clients.
This is still a cheap situation, because with this configuration we can:
easily deploy patched and updated services with no downtime - Cloud Run does the blue/green deployment automatically.
scalability is also done automatically by Cloud Run as it increases the number of service instances depending on the traffic.
application is easily reachable, because it is globally load balanced.
And a tip - recommended way to setup all of the infrastructure is to use IaaS tools like Terraform, Pulumi, Ansible, Saltstack etc. in order not to loose or confuse yourself during setup.