Welcome to issue #188 May 4th, 2020

We've got a new region in Las Vegas. Various articles demonstrating the power and flexibility of Cloud Build.

In case you didn't sign up yet, don't miss the opportunity for one-month free access to learning platforms.

News

Viva Las Vegas, with our latest Google Cloud Platform region - The new Las Vegas GCP region helps get capacity close to local users.

What’s happening in BigQuery: Efficient new views and Cloud AI integrations - Cloud data warehouse BigQuery now offers materialized views for improved query efficiency, plus new ML models and column-level security.

Discover, understand and manage your data with Data Catalog, now GA - New cloud metadata management service available to discover, understand, and manage data from one interface for data discovery and more.

Learn 3 in-demand cloud skills in 30 days at no cost during the month of May - Sign up by May 31, 2020 and get Google Cloud training on both Pluralsight and Qwiklabs at no cost for 30 days.

Security, simplified: Making Shielded VM the default for Compute Engine - Unified Extensible Firmware Interface (UEFI) and Shielded VM are now the default for everyone using Google Compute Engine—still at no additional charge.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Anthos in depth: What new AWS multi-cloud support means for you - Dive deeper into one of Anthos’ most exciting new features: support for multi-cloud. Now, you can use Anthos to consolidate all your operations across on-premises, Google Cloud, and other clouds, starting with AWS.

Explore Anthos with a sample deployment - You can now launch a test environment with multiple clusters and various Anthos components. It also includes a sample microservices application that runs in a multi-cluster Kubernetes environment, so you can explore all of Anthos’ advanced capabilities.

Improving your security posture with centralized secrets management - Secret Manager is now generally available and has many third-party integrations to help improve your security posture.

Your top network performance problems and how to fix them - Whether you want to troubleshoot a performance problem or optimize your deployment decisions, Google Cloud has a comprehensive set of tools for network monitoring, verification and optimization. Here's how to use them.

Designing distributed systems using NALSD flashcards - Get to know the SRE-inspired principles and numbers, plus handy flashcards, to help you design non-abstract large scale design (NALSD) distributed systems.

Understanding forwarding, peering, and private zones in Cloud DNS - Cloud DNS private zones, peering, and logging and auditing enhance security and manageability of your private GCP DNS environment.

Deploying Kubernetes Cluster on GCloud using Terraform - Guide to deploy modular Google Cloud Kubernetes Engine Cluster with Latest Terraform version 0.12.

Forseti Terraform Validator: Enforcing resource policy compliance in your CI pipeline - Using Policy as a Code with Forseti Terraform Validator.

Dynamic routing with Cloud Router - How dynamic routing saves you from pager duty and how to set it up (with VPN) on Google Cloud.

Google Kubernetes Engine (GKE) Scalability Options and Sizing Optimization - An overview of the scaling options offered by Google Kubernetes Engine and some quick tips to rightsize a cluster.

Secure Access to Web Apps with Identity-Aware Proxy - Using Identity-Aware Proxy to secure application on GKE.

VPN tunneling between an AWS VPC & a GCP VPC - Setting up VPN tunneling between VPCs in AWS and GCP.

Deploy Kubernetes Load Balancer Service with Terraform on GCP - In this blog, we will see how to set up a Kubernetes cluster and deploy Load Balancer type NGINX service on it using Terraform.

GCP Network Architecture for MicroServices - Sharing of architectural thinking, planning, and designing a network architecture to create an environment suitable for hosting large-scale microservices-based application in GCP.

Designing your Company Architecture on Google Cloud Platform - The article explains the basic aspects of organizing a company's resources hierarchy.

App Development, Serverless, Databases, DevOps

Managing cost and reliability in fully managed applications - Learn simple tasks and checks you can perform to both minimize downtime and mitigate unexpected costs for your serverless applications.

Using Google Cloud to Serve 10,000s of Personalized Recs Per Second - Improving recommendation product system for lower latency.

Google Cloud MemoryStore with Node JS on Google Kubernetes Engine - Cache improves your app performance terrifically, check how Google provides fully-managed service for Redis & Memcached for Nodejs.

Understanding Google Cloud Triad : Compute Engine , Kubernetes Engine and App Engine - An overview and description of Compute options on GCP.

What You Need to Know About Google Cloud Healthcare API - An overview of Cloud Healthcare API.

NFS Filestore on GCP for free - Setting up cheaper Filestore alternative.

Cloud Run with Bitbucket pipeline - An example of deploying Cloud Run application using Bitbucket pipelines.

Deploying a PyPI Server in minutes with GCP - PyPI is a repository of software for Python; well, if you are here you already know it.

How to build a Python package with Cloud Build in GCP - Deploying PyPI server on Cloud Run for Python packages.

Executing bash scripts with a webhook in Google Cloud - Learn how to trigger a bash script off a webhook in GCP.

How to CI/CD on Google Cloud Platform - Using Cloud Build, Google Container Registry, and Cloud Run to continuously build and deploy a simple Java application.

Paginating Firestore collections with snapshot listeners - Solution on how to paginate Firestore collection when using snapshot listeners.

Effectively specifying environment variables for Cloud Run - Tips and tricks when working with many environmental variables and deploying to Cloud Run.

Continuous Deployment on Google Cloud Platform — App Engine (Flexible) and Cloud Run using Cloud Build - How to set CI/CD pipeline for App Engine FLex and Cloud Run with Cloud Build.

Sapper, Google Cloud Run, Continuous Deployment - A boilerplate template - Deploying website to Cloud Run built with Sapper framework.

Calculating Google Cloud Storage Bucket Size - An inexpensive way to get Google Cloud Storage Bucket size using python code to read Stackdriver metrics.

Firebase Hosting for static assets of a Sapper web app on Cloud Run - Using Firebase to host static files and CDN for Cloud Run web app.

Big Data, Analytics, ML&AI

Optimize Dataproc costs using VM machine type - Try optimizing big data clusters from Spark, Hadoop, and Presto with fully managed Dataproc. Choosing VMs wisely can save on Dataproc costs.

Migrating Data Processing Hadoop Workloads to GCP - Intro to Dataproc as well as tips for best usage.

Calling Google Cloud Machine Learning APIs from Batch and Stream ETL pipelines - Making requests from a Beam pipeline to Cloud Natural Language API.

Troubleshooting cloud composer - Is cloud composer suddenly starting to miss deadlines? this blog might give hints to why composer is misbehaving and how to fix it!

Testing Airflow jobs on Google Cloud Composer using pytest - A reliable CI/CD for Airflow pipelines using Cloud Build.

Google Analytics in BigQuery 1: Getting Started - Intro to Google Analytics analysis using BigQuery and BigQuery public dataset.

Send Google Analytics Hit Level Data to BigQuery - How to send standard Google Analytics hit level data to BigQuery.

Yet More Google Compute Cluster Trace Data - Dump of 29 days Borg (Google's cluster management system) traces in BigQuery.

Connecting your Visualization Software to Hadoop on Google Cloud - The article explains how to set up architecture for visualization with Hadoop ecosystem on GCP.

Connecting your Visualization Software to Hadoop on Google Cloud - In part 2, steps to set up an environment that will hold data for visualization are explained.

Explaining model predictions on image data - A conceptual overview and technical deep dive into how XAI works on image data.

Getting started with Hyperparameter Tuning with AI Platform on GCP - Intro to hyperparameters and how to use AI Platform to get the best set of hyperparameters.

Baking with machine learning - Training an ML model to predict based on ingredients what kind of type food recipe is for.

Various

Google’s Thomas Kurian on COVID-19, customers in crisis and the big cloud fight - An interview with GCP CEO, Thomas Kurian.

Slides, Videos, Audio

GCP Podcast - #218 Chronicle Security with Dr. Anton Chuvakin and Ansh Patniak.

Kubernetes Podcast - #101 Open Policy Agent, with Tim Hinrichs and Torin Sandall.

Releases

BI Engine - BigQuery BI Engine is now available in the Las Vegas (us-west4) region.

BigQuery - A new function, JSON_EXTRACT_ARRAY, has been added to the list of JSON functions. The ORDER BY clause now supports the NULLS FIRST and NULLS LAST clauses. The BigQuery Data Transfer Service is now available in the Taiwan (asia-east1) region. BigQuery is now available in the Las Vegas (us-west4) region.

BigQuery ML - BigQuery ML is now available in the Las Vegas (us-west4) region.

BigQuery Transfer - BigQuery Data Transfer Service is now available in the Taiwan (asia-east1) region. BigQuery Data Transfer Service is now available in the Las Vegas (us-west4) region.

Cloud Composer - New versions of Cloud Composer images: composer-1.10.2-airflow-1.10.2, composer-1.10.2-airflow-1.10.3 and composer-1.10.2-airflow-1.10.6. Fixed an issue with the CloudSQL Proxy HealthCheck that caused the Proxy Pod to restart repeatedly. The oldest supported version of Composer is now composer-1.6.0-airflow-x.x.x.

Compute Engine - You can now suspend and resume VM instances, available in Beta. SSD persistent disks now have increased write throughput limits on instances with 1 to 15 vCPUs.

Config Connector - Fixes for the examples for the following resources: CloudBuildTrigger, AccessContextManager, ComputeDisk, and ComputeSubNetwork. Reduced memory requirements for deletion defender, recorder, and webhook. Ensure the webhook process does not signal it is ready until it is serving HTTP traffic.

Data Catalog - v1. Data Catalog is now generally available (GA).

Data Catalog Resources - v1. Data Catalog is now generally available (GA).

Dataproc - Announcing the Beta release of Dataproc Component Gateway, which provides secure access to web endpoints for Dataproc default and optional components. Dataproc on GKE version 1.4.27-beta is available with minor fixes.

Dialogflow - Beta launch of a one-click integration with a new telephony partner: Avaya.

Google Kubernetes Engine - Ingress for Anthos is now Generally Available (GA) for all GKE versions 1.14 and up. GKE cluster versions have been updated. Note: Your clusters might not have these versions available. The following Kubernetes versions are now available for new clusters and for opt-in master upgrades and node upgrades for existing clusters. No channel 1.14.10-gke.37 is now available. To improve the safety of upgrades and reduce disruption, all new node pools have surge upgrades turned on by default with the configuration: maxSurge=1 maxUnavailable=0. The following notable changes are coming in 1.17: The RunAsUsername feature in 1.17 is now beta and allows specifying the username when running a Windows container. The RuntimeClass scheduler in 1.17 simplifies scheduling Windows Pods to appropriate nodes. The following node labels are deprecated in 1.17: Cluster Versions Deprecated Label New Label 1.14+ beta.kubernetes.io/os kubernetes.io/os 1.14+ beta.kubernetes.io/arch kubernetes.io/arch 1.17+ beta.kubernetes.io/instance-type node.kubernetes.io/instance-type 1.17+ failure-domain.beta.kubernetes.io/zone topology.kubernetes.io/zone 1.17+ failure-domain.beta.kubernetes.io/region topology.kubernetes.io/region You must identify any node selectors using beta labels and modify them to use GA labels. RBAC in the apps/v1alpha1 and apps/v1beta1 API versions are deprecated in 1.17 and will no longer be served in 1.20. A known kernel bug in Linux kernel 4.18, 4.19, 4.20 and 5.0 may cause softlockup when running eBPF workloads. Google Kubernetes Engine will gradually upgrade clusters in the regular channel to GKE 1.16. 1.15.9-gke.24. 1.15.9-gke.26. 1.15.11-gke.1.

Google Kubernetes Engine Rapid - 1.17.4-gke.10 is now available in the Rapid release channel. Although clusters in the Rapid channel upgrade automatically, you should still review: Known issues. Urgent upgrade notes. The RunAsUsername feature is now beta and allows specifying the username when running a Windows container. The RuntimeClass scheduler simplifies scheduling Windows Pods to appropriate nodes. The following node labels are deprecated: Cluster Versions Deprecated Label New Label 1.14+ beta.kubernetes.io/os kubernetes.io/os 1.14+ beta.kubernetes.io/arch kubernetes.io/arch 1.17+ beta.kubernetes.io/instance-type node.kubernetes.io/instance-type 1.17+ failure-domain.beta.kubernetes.io/zone topology.kubernetes.io/zone 1.17+ failure-domain.beta.kubernetes.io/region topology.kubernetes.io/region You must identify any node selectors using beta labels and modify them to use GA labels. RBAC in the apps/v1alpha1 and apps/v1beta1 API versions are deprecated in 1.17 and will no longer be served in 1.20.

Load Balancing - Google-managed SSL certificates are available in General Availability.

Cloud Logging - The Logs Viewer (Preview) is now GA.

Cloud Text-to-Speech - Cloud Speech-to-Text now offers 25 new voices (both Standard and WaveNet) in the following languages.

Virtual Private Cloud - Google Cloud now encrypts VPC traffic within the boundaries of the data centers in asia-east2.

AI Platform Prediction - AI Platform Prediction now supports several regional endpoints for online prediction. You can now deploy scikit-learn and XGBoost models for online prediction using Compute Engine (N1) machine types. The europe-west4 (Netherlands) and asia-east1 (Taiwan) regions are now available for online prediction. We recommend against using Compute Engine (N1) machine types on the AI Platform Prediction global endpoint.

Dialogflow Enterprise - Beta launch of a one-click integration with a new telephony partner: Avaya.

Service Mesh - The Anthos Service Mesh dashboard in the Google Cloud Console is generally available for Anthos Service Mesh installations on Google Kubernetes Engine clusters.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko