Welcome to issue #247 June 21st, 2021

News

New Tau VMs deliver leading price-performance for scale-out workloads - Compute Engine’s new Tau VMs based on AMD EPYC processors provide leading price/performance for scale-out workloads on an x86-based architecture.

Cloud SQL extends PostgreSQL 9.6 version support beyond end-of-life - Plan your upgrade without stress with at least one year of support for PostgreSQL 9.6 on Cloud SQL.

Expanding partner solutions at the network edge - Google Cloud’s Anthos for Telecom initiative plus the availability of 5G lets our partners run their apps on Anthos, at the edge.

Improved accessibility in the Maps JavaScript API - Today I’m sharing some recent work by the Google Maps Platform JavaScript team focused on improving accessibility in the Maps JavaScript API. Last year we began a renewed effort to become more accessible ‘out of the box’, and to provide more hooks for developers to make accessible experiences.

Ubuntu Pro lands on Google Cloud - The availability of Ubuntu Pro on Google Cloud gives customers running open-source access to a number of compelling features.

New training helps you get started with GKE—for free! - At Cloud OnBoard and beyond, learn why and how to use Google Kubernetes Engine (GKE) as your business’s container orchestration platform.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Are we there yet? Thoughts on assessing an SRE team’s maturity - Examining the key indicators that signal a mature SRE team.

From multiple clouds to multicloud: Key factors that influence success - There’s a difference between running on multiple clouds and having a multicloud strategy -- Google Cloud can help.

Up or out—or both: A simple framework for plotting your cloud migration - When migrating applications to Google Cloud, it helps to think about whether you are moving “up” or “out.” A new whitepaper explains.

Build a platform with KRM: Part 2 - How the Kubernetes resource model works - The Kubernetes API is a powerful declarative engine for your configuration. Learn how to deploy resources, and how to set up Continuous Deployment for your Kubernetes environment.

Cloud CISO Perspectives: June 2021 - Google Cloud CISO Phil Venables shares his thoughts on ransomware, software supply chains, and RSA retrospectives.

Streamline your application migration journey with Migrate for Anthos and GKE - Migrate for Anthos allows developers to take existing VM-based workloads running on Linux or Windows and move them easily to either GKE or Anthos clusters.

Tutorial: Publishing GitHub Findings to Security Command Center - Publishing custom findings related to your GCP Cloud assets which are outside of GCP to Security Command Center.

App Development, Serverless, Databases, DevOps

Learn Cloud Functions in a snap! - Cloud Functions is a fully managed event-driven serverless function-as-a-service (FaaS). It is a small piece of code that runs in response to an event. Because it is fully managed, developers can just write the code and deploy it without worrying about managing the servers or scaling up/down with traffic spikes.

Build a Serverless Delivery Command Center on GCP - Combine Maps, OR-Tools, SendGrid and Cloud Functions to commandeer a delivery fleet.

How to get the most from Cloud SQL for SQL Server - Better understand when and how to use Google Cloud’s managed SQL Server offering with these best practices.

2 ways to migrate your SAP HANA database to Google Cloud - Learn two methods for moving your current SAP environment unchanged onto Google Cloud as a lift and shift or rehost.

Flutter Web on Google App Engine using Cloud Build - Deployment process to Google App Engine for a Flutter Web application, orchestrated by Google Cloud Build.

Serverless Redis on Google Cloud Functions - This tutorial shows how to build a Serverless API with Redis on Google Cloud Functions.

How to use CloudSQLProxy in Google Cloud Platform - The article explains various ways how Cloud SQL Proxy can be used and how to decide which one to choose.

Cloud Run: A story of serverless containers - Mindful Containers is a fictitious company that is creating containerized microservice applications. They need a fully managed compute environment for deploying and scaling serverless containerized microservices. So, they are considering Cloud Run.

Serverless Jenkins Pipelines with Google Cloud Run - Deploying Jenkins to Cloud Run.

Did we market Knative wrong? - A reflection on the Knative development from position/marketing point of view.

Big Data, Analytics, ML&AI

Bootstrap a Modern Data Stack in 5 minutes with Terraform - Setup Airbyte, BigQuery, dbt, Metabase, and everything else you need to run a Modern Data Stack using Terraform.

Orchestrating your data workloads in Google Cloud - The Data Orchestration is becoming more important as workflows expand and become more complex on the Cloud. This blog touches on how to tackle data orchestration in GCP using Cloud Composer!

Creating custom financial indices with Dataflow and Apache Beam - How CME Group and Google Cloud built an index publication pipeline to glean the sort of real time value insights today’s financial firms require.

Dataproc best practices guide - Best practices for Storage, Compute and Operations when using Dataproc for running Hadoop- or Spark-based workloads.

Export Datastores from multiple projects to BigQuery - How to export datastores from multiple projects using Google Dataflow — with additional filtering of entities.

Apache Beam Hack — Streaming into Sharded BQ Tables - Dealing with issues when streaming to hourly sharded BigQuery tables.

Google Data Fusion: Securing your Access Keys and Passwords - Using Data Fusion’s Secure Storage to store sensitive information like keys and passwords.

Google Data Fusion: Delete Pipelines via REST API - Example of using Data Fusion REST API to delete old pipeline.

Slots to the (Wallet) Rescue - Example of reducing costs in BigQuery using Slots.

A Primer on JavaScript UDFs for Spatial Analysis in BigQuery - Succinct guide to writing JavaScript User-Defined Functions for Geospatial Operations in BigQuery.

Calculating Internal Rate of Return (IRR) in BigQuery - A step-by-step guide to implementing Excel’s IRR function in BigQuery.

3 ways to divide safely in BigQuery - Examples of how to handle division by zero in BigQuery.

Unit Testing in dbt — Part 1 - Custom testing within dbt for data in BigQuery.

Using Cloud Pub/Sub on Node.js from Elm - This article will walk you through building an Elm application that runs in Node.js, and uses the Node.js Pub/Sub libraries for communicating with Pub/Sub.

The Multi-Cloud Future (5) — How To Do “Data” The Multi-Cloud Way - Some ways Google Cloud lets you make the best use of all of your data — wherever they are.

Machine Learning Pipelines with Google Cloud Platform (Part 2) - A path from data ingestion to model evaluation in Vertex AI.

How to Generate Logs with Vertex AI in Google Cloud - Use Vertex AI notebooks to communicate with centralized logging in Google Cloud.

Various

Expanding access to quantum today for a better tomorrow - You can gain access to IonQ’s trapped-ion quantum computer is available through the Google Cloud Marketplace.

How I cleared Google cloud — Professional Data Engineer Certification in 2021 - Preparing and taking the GCP Certification exam.

Slides, Videos, Audio

Kubernetes Podcast - #152 SRE for Everyone Else, with Steve McGhee.

Releases

AI Platform - Deep Learning VMs - M72 Release Added PyTorch 1.9 and PyTorch/XLA 1.9 images.

Anthos clusters on VMware 1.7 - When you upgrade an unregistered Anthos cluster on VMware from a version earlier than 1.7.0 to a version 1.7.0 or later, you need to manually install and configure the Anthos Config Management operator.

GKE on-prem 1.7 - When you upgrade an unregistered Anthos cluster on VMware from a version earlier than 1.7.0 to a version 1.7.0 or later, you need to manually install and configure the Anthos Config Management operator.

Cloud Composer - Cloud Composer 1.16.7 release started on June 17, 2021. New versions of Cloud Composer images: composer-1.17.0-preview.3-airflow-2.0.1 composer-1.16.7-airflow-1.10.15 composer-1.16.7-airflow-1.10.14 (default) composer-1.16.7-airflow-1.10.12. GCSfuse version was updated to 0.35.1 (latest release). (Airflow 2) Preinstalled packages changed. For DAG runs with long-running tasks, task level logs are now periodically updated in the Airflow UI. It is now possible to create environments with CMEK encryption in projects with enabled domain restricted sharing. (Airflow 2) In the Airflow UI, you can now create connection types from the installed custom Airflow provider packages. Fixed a problem where the Airflow worker health was calculated incorrectly because of leftover queued tasks without DAGs being present in the Airflow database. Fixed the cause of Liveness probe errored events that appeared in the scheduler and worker pod logs.

Compute Engine - Generally available: You can now create application consistent snapshots of disks attached to Linux VMs. You can now customize E2 shared-core machine types. Memory-optimized M2 machine types are now available in Belgium, europe-west1-b,c.

Data Fusion - The SAP accelerator for the order to cash process is now available.

Cloud Dataflow - In addition to scalar functions, Dataflow SQL now supports aggregate user-defined functions (UDFs) for Java.

Datastore - Support for Identity and Access Management custom roles. Support for the following additional locations: asia-southeast1 Singapore us-west1 Oregeon asia-east1 Taiwan See the full list of locations.

Deep Learning Containers - M72 Release Added PyTorch 1.9 and PyTorch/XLA 1.9 containers.

Deep Learning VM - M72 Release Added PyTorch 1.9 and PyTorch/XLA 1.9 images.

Dialogflow Enterprise - Final reminder: The Dialogflow V1 API shutdown will be finalized during the week of June 21, 2021.

Dialogflow - Final reminder: The Dialogflow V1 API shutdown will be finalized during the week of June 21, 2021.

Cloud Firestore - Support for Identity and Access Management custom roles. Support for the following additional locations: asia-southeast1 Singapore us-west1 Oregeon asia-east1 Taiwan See the full list of locations.

Cloud Functions - Cloud Functions is now available in the following region: asia-southeast1 (Singapore) See Cloud Functions Locations for details.

Google Kubernetes Engine - (2021-R20) Version updates GKE cluster versions have been updated. The issue affecting the Datadog Agent on Autopilot has been resolved in Datadog version 2.13.1.

GKE - (2021-R20) Version updates Version 1.19.10-gke.1600 is now the default version.

Google Kubernetes Engine Rapid - (2021-R20) Version updates Version 1.20.7-gke.1800 is now available in the Rapid channel.

Google Kubernetes Engine Regular - (2021-R20) Version updates Version 1.19.10-gke.1600 is now the default version in the Regular channel.

Google Kubernetes Engine Stable - (2021-R20) Version updates Version 1.18.17-gke.1901 is now the default version in the Stable channel.

KF - 2.4. Kf Operator to manage Kf installation. Added Operator diagnostics to kf doctor. Allow target command to take arg instead of flag. Config Connector can manage the Kf Google Service Account (GSA). Removed internal routing dependency on internal-gateway. Inline environment variable printing in kf env. Config Connector is now required. Updated Tekton to 0.23.0. Only check for timeout error for deprovisioning service instances. Make targeting a non-existent Space an error. Fixes manifest parsing bug.

Network Intelligence Center - The Connectivity Tests dynamic verification feature is now generally available.

Cloud Scheduler - Cloud Scheduler is now available in us-west1, asia-east1, and asia-southeast1.

Service Mesh - 1.9.x. Google-managed control plane is now a generally available (GA) feature.

Cloud SQL Postgres - Query Insights is now supported for read replicas.

Cloud Storage Transfer - Storage Transfer Service offers Preview support for transferring data from Azure ADLS Gen 2 to Cloud Storage.

Cloud Tasks - v2. Cloud Tasks is now available in us-west1, asia-east1, and asia-southeast1.

Cloud Text-to-Speech - Text-to-Speech now offers voices in the following new languages.

Cloud TPU - Cloud TPU team just released TF-2.1.4, TF-2.2.3 and TF-2.3.3 on Cloud TPUs.

VMware Engine - Added autoscale policies that can automatically expand or shrink a cluster in your private cloud based on factors like CPU utilization or storage capacity thresholds. Preview: vSAN data encryption for data at rest now uses keys generated by Cloud Key Management Service for all new private clouds. Removed vCenter privilege Host > Configuration > Storage partition configuration for role Cloud-Owner-Global-Role. Enabled TRIM/UNMAP support on vSAN at the time of private cloud creation for more efficient vSAN storage by default. Added the following vCenter privileges to the Cloud-Owner-Global-Role role: Guest operation alias modification Guest operation alias query Guest operation modifications Guest operation program execution Guest operation queries. Added vSphere content library management privileges to the Cloud-Global-VM-admin-group group. The Quotas page in the Cloud Console no longer shows VMware Engine node usage as 0 when you have an active private cloud.

Virtual Private Cloud - Private Service Connect endpoints in consumer networks now won't become unresponsive if they are connected to a service attachment that references a load balancer without backend VMs. Enabling or disabling PROXY protocol after a Private Service Connect service attachment is created does not change the configuration.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko