Welcome to issue #225 January 18th, 2021

News

Introducing Ruby on Google Cloud Functions - With Cloud Functions support for Rub Functions Framework, you can write idiomatic Ruby functions and deploy them in a fully managed Ruby environment.

Powering open source healthcare research on Google Cloud - Google Cloud partners with OHDSI on standardizing health data analytics with open source tools.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Gitlab — Exploring CICD-AutoDevOps to GKE - Exploring Gitlab CICD Pipeline — Auto DevOps to deploy to GKE.

Building a Kubernetes cluster on Google Cloud with Rancher and Terraform - Using Terraform and Rancher custom node setup to create GKE cluster.

Custom Gitlab CICD using Auto DevOps template + Add Cloud Build in GCP - Gitlab CICD with Custom Auto DevOps template + Cloud Build stage in GCP.

Migrating applications between Kubernetes clusters - A simple way to gradually migrate applications between Kubernetes deployments without affecting intra-cluster traffic.

Secure access to your VMs in Google Cloud - A way to organize a secure access to your VM fleet hosted in Google Cloud without additional costs.

How to Deploy a GCP VPN to our Datacenter using Terraform - Using GCP Cloud VPN to securely connect on-premises datacenter to GCP Virtual Private Cloud (VPC) network through an IPsec VPN connection.

Provisioning Cloud Run with Cloud NAT using Terraform - Use Terraform to automate the creation of a Cloud Run service that directs egress traffic through a Cloud NAT gateway.

How GitOps and the KRM make multi-cloud less scary. - An example of how to provision and manage infrastructure, apps, and managed services in a consistent way, on any cloud using Anthos.

App Development, Serverless, Databases, DevOps

Google Cloud and SAP demonstrate massive scalability for financial services customers - Engineers ran a SAP S/4HANA for financial product subledger (FPSL) solution using more than 20TB of actual transaction data on a scale-out Google Cloud environment and found it delivered performance, scale and reliability.

Service Account Authentication on GCP via Node.js App - Providing secure access for server-to-server communication on Google Cloud Platform.

Eventarc: A unified eventing experience in Google Cloud - Eventarc provides a unified eventing experience in Google Cloud so you can send events to Cloud Run from more than 60 Google Cloud sources.

Cloud Profiler provides app performance insights, without the overhead - Cloud Profiler helps troubleshoot nagging performance problems in your code, without adding any additional overhead.

4 best practices for ensuring privacy and security of your data in Cloud Storage - Beyond the fundamentals, Cloud Storage offers several security features, such as bucket-level access, service account HMAC keys, IAM conditions, Delegation tokens, and V4 signatures. Here are 4 tips to help get you started.

Compute Engine explained: Scheduling the OS patch management service - Here’s how to patch and update your VMs on a schedule with Compute Engine’s OS patch management service.

FastAPI: Deploy Containerized Apps On Google Cloud Run - Example of deploying FastAPI webapp to Cloud Run.

Build a Vue app with Firebase and Cloud Run - Developing and deploying a Vue application communicating with the Spring Boot application hosted on Cloud Run.

GCP API Gateway with gRPC - Simple gRPC client/server for GCP API Gateway and Cloud Run with authentication and authorization!

How to Scale Up Web Check-in for Millions using Microservices & DevOps - How AirAsia is using tens of microservices in Node.JS/Python deployed on Google App Engine.

Save money by scheduling Cloud SQL - How to schedule Cloud SQL instances.

Cloud Run and Load balancing: go beyond your own project! - Using Load Balancer with Cloud Run application which is in another GCP project.

Lock Statistics: Diagnose performance issues in Cloud Spanner - This post introduces a newly released introspection tool: Lock statistic, and guides our readers through a simple example of using it to diagnose transaction performance issues caused by lock conflict.

Implementing leader election on Google Cloud Storage - Controlling coordination among distributed systems can be complex. This article showcases how to build your own leader election system using only Google Cloud Storage.

Big Data, Analytics, ML&AI

Comparing Custom Model Development With GCP BQML and AutoML Tables - Comparing Custom Model Development on Python Jupyter notebook with Google Cloud Platform BigQuery Machine Learning and AutoML Tables (beta).

Read/Write From Any Google API To/From BigQuery In 1 Minute Using BQ Flow - Use BQ Flow to transfer data between any Google API (Campaign Manager, Adwords API, Display Video) and.

Counting unique elements faster in Google BigQuery with HyperLogLog - With HyperLogLog algorithm, you trade off some accuracy to get the job done faster. But how much faster and at the expense of what level of accuracy?

A Batch Driven CDC (Change Data Capture) Approach using Google Cloud Platform - Implementing Change Data Capture system on GCP.

Loading complex CSV files into BigQuery using Google Sheets - Building an ELT pipeline using Google Sheets as an intermediary.

BigQuery Hack: 1000x More Efficient Aggregation Using Materialized View - Learn how to supercharge your aggregation queries using Materialized View.

Sentiment classification using BigQuery ML - Training machine learning model without a single line of code!

Designing a Data Lake on GCP with Data Fusion and Composer - A scalable data lake architecture for storage and consumption of structured data on GCP using data integration and orchestration services.

Building a Data Lake on GCP using Data Fusion and Composer - Part 2: An overview of the detailed solution design based on architecture from the first part.

Demystifying Machine Learning by Building an ML Pipeline 1 & 2 - Intro to machine learning concepts with examples.

Google Cloud AI Platform Notebooks and Cloud Source Repositories - Saving Jupyter Notebook to Cloud Repository.

Computing Time Series metrics at scale in Google Cloud - This blog post shows how data scientists and engineers can use GCP Dataflow to compute time-series metrics in real-time or in batch to backfill data at scale, for example, to detect anomalies in market data or IoT devices.

An Accelerated Big Data Workflow for the Data Analyst - Explore and analyze 1B loan records with RAPIDS & Nvidia A100 GPUs on Cloud AI Platform.

Powerful yet familiar workflow for the Data Analyst - Accelerating data processing using GPUs.

Various

Re-certifications on Google Cloud - Experiences of re-certifying on Google Cloud.

Preparing For The Google Cloud Professional Data Engineer Exam - Tips and tricks from personal experience of passing Data Engineer exam, followed by a list of study materials.

How I cracked the GCP Professional ML Engineer certification in 8 days! - Areas to focus on when studying for Machine Learning Engineer certification.

Think big: Why Ricardo chose Bigtable to complement BigQuery - See how ecommerce company Ricardo.ch chose Cloud Bigtable as its database to complement its data warehouse and save costs with scalability.

Slides, Videos, Audio

Kubernetes Podcast - #133 Cilium, with Thomas Graf.

Simpler, faster, cheaper Enterprise Apps using only Spring Boot on GCP - Toronto Serverless meetup hosting Ray Tsang.

Releases

BigQuery - BigQuery's Cloud Console UI has been updated with many usability improvements for analyzing data, including multi-tab navigation, a new resource panel, and a new SQL editor.

Cloud Build - Regionalized builds from Cloud Functions and App Engine deployments are now visible in the Cloud Build History UI. Users can now specify their own service accounts for Cloud Build to run builds.

Cloud Composer - Preview: You can now restart the Airflow web server using the command gcloud beta composer environments restart-web-server or the Beta API. GA: Network ACL support for the Airflow web server is now generally available. New versions of Cloud Composer images: composer-1.13.4-airflow-1.10.9, composer-1.13.4-airflow-1.10.10, and composer-1.13.4-airflow-1.10.12. You can now set the machine type for the Airflow web server and Cloud SQL database using the to v1 Composer API. Fixed an issue where connections were inserted into the Airflow database twice.

Compute Engine - You can now create N2D VM instances in us-east4-c Northern Virginia.

Dataproc - Announcing the Beta release of Dataproc Service Account Based Secure Multi-tenancy, which allows you to share a cluster with multiple users. New sub-minor versions of Dataproc images: 1.3.81-debian10, 1.3.81-ubuntu18, 1.4.52-debian10, 1.4.52-ubuntu18, 1.5.27-centos8, 1.5.27-debian10, 1.5.27-ubuntu18, 2.0.0-RC23-debian10, and 2.0.0-RC23-ubuntu18. Image 2.0 preview: Upgraded Spark to version 3.1.0 RC1. The gcloud_dataproc_personal_cluster.py tool for the personal auth beta is no longer supported for new images. Added support for user configuration of Compute Engine Shielded VMs in a Dataproc Cluster.

Cloud Networking Products - The API for updating ResourceRecordSets in Cloud DNS is now available in Beta.

Cloud Functions - Cloud Functions has added support for a new runtime, Node 14, in Preview. Cloud Functions has added support for a new runtime, Ruby, in Preview.

Networking Interconnect - Support for 1500 MTU for Cloud Interconnect is now available in General Availability.

Cloud Logging - Cloud Logging now lets you share your saved queries with other users of a project.

Managed Microsoft AD - Managed Microsoft AD now supports audit logging.

Recommendations AI - Recommendations AI has migrated to the Retail API, which is now generally available.

Cloud Spanner - You can now run SQL queries to retrieve lock statistics to investigate lock conflicts in your database. Query statistics now includes information about queries that failed, queries that timed out, and queries that were canceled by the user.

Cloud SQL - Cloud SQL now exposes the metric database/memory/total_usage.

Virtual Private Cloud - Support for 1500 MTU for Cloud Interconnect is now available in General Availability.

Network Intelligence Center - The Network Topology graph now includes a checkbox, Show connections for child nodes only on focus, to display only the traffic paths between top-level entities, such as regions.

Migrate for Compute Engine 4.8 - 4.11. #171638373: General stability improvements. #171638373: Fixed Windows adaptation issue when boot partition and Windows partition were on different volumes. Performance improvement during detach phase. #175196444: Fixed Windows adaptation issue with network interface detection. #174330790: Linux adaptations now archive ifcfg-* scripts to avoid Network Manager conflicts with iSCSI boot. Security fixes applied.

Service Mesh - 1.6.x. 1.6.14-asm.0 is now available.

AI Platform Training - AI Platform Training now provides pre-built PyTorch containers for PyTorch 1.6.

Service Directory - Configuring an internal load balancer in Service Directory is available in Preview.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko