Welcome to issue #255 August 16th, 2021

News

Troubleshoot GKE apps faster with monitoring data in Cloud Logging - View contextual Monitoring data in your GKE log lines. Easily see the relevant pod, node and cluster events and metrics for your pod.

Use log buckets for data governance, now supported in 23 regions - If you have data governance and compliance requirements, Cloud Logging can help you meet them with regional log buckets, available now in 23 regions.

What’s new with Splunk Dataflow template: Automatic log parsing, UDF support, and more! - Announcing new features for Splunk Dataflow template with improved compatibility with Splunk Add-on for GCP, more extensibility using user-defined functions (UDFs), and general pipeline reliability enhancements.

Monitor and troubleshoot your VMs in context for faster resolution - Save time when accessing critical data for troubleshooting in Virtual Machines. Metrics, charts, and logs now available in the VM console.

Announcing our first-ever Google Cloud Startup Summit on September 9, 2021 - Registration is open for our first-ever Google Cloud Startup Summit, an event filled with inspiration, education, and discussion on September 9, 2021.

Upcoming Webinar: “How to plan your migration the right way” - Register today for free upcoming webinar, “How Google Cloud Customers Can Plan Their Migration the Right Way”.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Streaming VR and AR content from Google Cloud with NVIDIA CloudXR - How NVIDIA CloudXR in Google Cloud data centers enables high-fidelity VR and AR applications.

Running Anthos inside Google - How we run our third-party software on Anthos at Google.

Zero trust with reverse proxy - A reverse proxy stands in front of your data, services, or virtual machines, catching requests from anywhere in the world and carefully checking each one to see if it is allowed.

Verify GKE Service Availability with new dedicated uptime checks - Uptime checks for GKE Loadbalancer services help to ensure your services are running, and quickly isolate the root causes of failures.

Private Service Connect to connect privately with Google APIs and Services - Private Service Connect to connect privately with Google APIs and Services.

GKE and private registries with Self Signed Certificates - This article provides a workaround to pull images from a registry that is not signed by a trusted CA.

Missing data points in your monitoring API response? Use page iterators! - Iterators simplify the process of paging through API responses. Learn to use page iterators to get all data points from the `list_time_series` call.

Get real time alerts for “Open To Internet” firewall rules with Cloud Asset Inventory - This article will show how you can subscribe to real-time notifications for changes in our assets and get an alert when an “Open to Internet” firewall rule is made.

Integrating GCP with Datadog with terraform - Setting up Datadog collection on GCP with Terraform.

OWASP Top 10 mitigation options on Google Cloud - This document helps you identify Google Cloud products and mitigation strategies that can help you defend against common application-level attacks that are outlined in OWASP Top 10.

App Development, Serverless, Databases, DevOps

Saving the day: How Cloud SQL makes data protection easy - Backup, high availability, and replication all play key roles in making your applications fault-tolerant.

Data Validation for Oracle to Cloud Spanner Migration - Data Validation Tool is an open sourced python CLI based tool that compares heterogeneous data source tables and provides multi-level validation functions. DVT can connect to different types of source and target databases and create validation reports based on the criteria defined. In this blog, we will see how to set up the Data Validation Tool to compare data between Oracle and Spanner tables and create validation reports in BigQuery.

Deep dive into the new Firebase JS SDK design - Explanation of changes in a new Firebase Javascript SDK.

Migrating to Cloud Spanner using HarbourBridge and Data Validation Tool - There has recently been a lot of interest in migrating from existing open source relational databases such as MySQL or PostgreSQL to Google Cloud Spanner to enable horizontal scaling. This type of heterogeneous migration can be complicated, however there are two open source tools that can help: HarbourBridge and the Data Validation Tool.

Getting Started with Spring Cloud GCP: Overview - An overview of Spring Cloud GCP.

Examine Google Cloud Platform security vulnerabilities using Cloud Functions. - API to examine your Google Cloud Platform security vulnerabilities using Cloud Functions.

Mastering the Google Cloud Platform SDK tools - A look at some lesser-known GCP SDK settings and features that make your day-to-day interactions with GCP more enjoyable.

Step-by-Step Guide to Getting Started with Scylla Cloud - Step-by-step into what it takes to get your Scylla Cloud cluster up and running quickly.

ES Modules in Cloud Functions - With recent updates, Google Cloud Functions for Node now supports ES modules out of the box!

Big Data, Analytics, ML&AI

Extend your Dataflow template with UDFs - Learn how to easily extend a Cloud Dataflow template with user-defined functions (UDFs) to transform messages in-flight, without modifying or maintaining Apache Beam code.

Building a unified analytics data platform on Google Cloud - Data-driven transformation starts with a modern data platform, built on Google Cloud.

Entity Resolution using Google Cloud Dataflow - This article illustrates how data platform was modernized by implementing an entity resolution pipeline using Cloud Dataflow.

Let’s get it started! Triggering ML pipeline runs - Learn two approaches to automate your Vertex Pipeline runs using Cloud Scheduler and Cloud Functions.

Get that crucial report in Slack Channel - Python code to post visualized data from BigQuery to Slack channel.

Google Analytics 4 dataset for learning in BigQuery - Google Merchandise Store is an online store that sells Google-branded merchandise. The site uses Google Analytics 4's standard web ecommerce implementation along with enhanced measurement. Dataset available through the BigQuery Public Datasets program contains a sample of obfuscated BigQuery event export data for three months from 2020-11-01 to 2021-01-31.

BigQuery Admin reference guide: Data governance - Learn how to ensure your data is discoverable, secure and usable inside of BigQuery.

Optimizing your BigQuery tables using clustering - Besides the table partitioning to improve performance and costs in BigQuery, there is also another technique available called clustering.

The Brexit vote: A case study in causal inference using machine learning - This article introduces the concept of causal inference, applied to the Brexit vote, using both statistical methods and Vertex AutoML.

ISO Dates + BigQuery ML ARIMA models = less errors - Correct week extraction in datetime field for ARIMA model in BigQuery.

Fuel your custom models with Vertex AI - Learn how to use Vertex AI - Google Cloud's newly announced managed ML platform - to build end-to-end ML workflows. You'll learn how to go from raw data to deployed model, and will leave this workshop ready to develop and productionize your own ML projects with Vertex AI.

Vertex AI Notebooks With JetBrains IDEs (PyCharm/IDEA/etc) - This article explains how to bootstrap Vertex AI Notebook with JetBrains IDE instead of the JupyterLab.

Adapting to changes of data by building MLOps pipeline in Vertex AI - This post shows how to build a machine learning pipeline for object detection tasks. The objective is to demonstrate how to realize MLOps to prepare data drift situations using Vertex AI’s AutoML and Cloud Function.

A technical solution producing highly-personalized investment recommendations using ML - The implementation details behind Softserve’s use of Google Cloud to improve retail investing with the Investment Products Recommendation Engine (IPRE).

Various

Google Cloud study: Cloud adoption increasing in financial services, but regulatory hurdles remain - Survey results from Google Cloud and the Harris Poll on challenges and opportunities of cloud adoption in financial services, based on responses of >1,300 IT leaders.

Slides, Videos, Audio

GCP Podcast - #271 The Future of Service Networking with Ryan Przybyl.

Kubernetes Podcast - #158 Telekom (with a K), with Vuk Gojnic.

Software Engineering Daily Podcast - Cloud Run: Serverless Applications with Steren Giannini.

Releases

AI Platform Training - You can use TPU Pods for training.

Anthos clusters on bare metal - 1.6. Release 1.6.4 Anthos clusters on bare metal 1.6.4 is now available. Fixes: The following container image security vulnerabilities have been fixed: CVE-2021-3520 CVE-2021-20305 CVE-2021-22924 CVE-2021-22925 CVE-2021-31535 CVE-2021-33560 CVE-2021-33910. Known issues: When you upgrade Anthos clusters on bare metal from a version with a security patch to the next minor release, we recommend you upgrade to the highest patch version to ensure you have the latest security fixes.

Anthos clusters on VMware 1.7 - Anthos clusters on VMware 1.7.3-gke.2 is now available. Fixes: This security vulnerability has been fixed: CVE-2021-3520 Fixed the issue that the /etc/cron.daily/aide` script uses up all existing space in /run, causing a crashloop in Pods.

GKE on-prem 1.7 - Anthos clusters on VMware 1.7.3-gke.2 is now available. Fixes: This security vulnerability has been fixed: CVE-2021-3520 Fixed the issue that the /etc/cron.daily/aide` script uses up all existing space in /run, causing a crashloop in Pods.

Cloud Build - Users can now build repositories from GitHub Enterprise, including on-premises instances.

Config Connector - Config Connector 1.58.1 is now available. Miscellaneous bug fixes.

Dataproc - New sub-minor versions of Dataproc images: 1.4.67-debian10, 1.4.67-ubuntu18, 1.5.42-centos8, 1.5.42-debian10, 1.5.42-ubuntu18, 2.0.16-centos8, 2.0.16-debian10, and 2.0.16-ubuntu18. SPARK-28290: Fixed an issue where Spark History Server failed to serve because of a wild card certificate in the 1.4 and 1.5 images.

Deep Learning Containers - M77 release TensorFlow Enterprise 2.6.0 is now available and includes Long Term Version Support.

Dialogflow Enterprise - Dialogflow CX GA (generally available) launch of Experiments to compare the performance of flow versions to a control version while handling live traffic.

Dialogflow - Dialogflow CX GA (generally available) launch of Experiments to compare the performance of flow versions to a control version while handling live traffic.

Eclipse - 1.8.5. Fixed an issue where avatar images of logged-in accounts were not being displayed.

Cloud Functions - The use of private worker pools is now available for building your functions.

Google Kubernetes Engine - (2021-R26) Version updates GKE cluster versions have been updated.

GKE - (2021-R26) Version updates The following control plane and node versions are now available: 1.18.20-gke.901 1.18.20-gke.3300 1.19.13-gke.1200 1.20.9-gke.1000 The following control plane versions are no longer available: 1.17.17-gke.3700 1.17.17-gke.4400 1.17.17-gke.4900 1.17.17-gke.5400 1.17.17-gke.6000 1.17.17-gke.6700 1.17.17-gke.7200 1.17.17-gke.7800 1.17.17-gke.8200 Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.20 to 1.20.8-gke.900 with this release.

Google Kubernetes Engine Rapid - (2021-R26) Version updates The following control plane and node versions are now available in the Rapid channel: 1.20.8-gke.2100 1.21.3-gke.901 1.21.3-gke.1100 The following control plane and node versions are no longer available in the Rapid channel: 1.20.9-gke.700 1.21.3-gke.100 Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.3-gke.900 with this release.

Google Kubernetes Engine Regular - (2021-R26) Version updates Version 1.20.8-gke.2100 is now available in the Regular channel.

Google Kubernetes Engine Stable - (2021-R26) Version updates Version 1.19.12-gke.2100 is now the default version in the Stable channel.

Cloud Monitoring - Cloud Monitoring now supports creating an uptime check against a Kubernetes LoadBalancer Service hosted on GKE. When you create a line chart on a dashboard, you can now specify whether the selected metric is charted against the left Y-axis or the right Y-axis.

Security Command Center - Event Threat Detection, a built-in service of Security Command Center Premium, has launched new detectors to protect your Google Workspace domains in general availability. Container Threat Detection, a built-in service of Security Command Center Premium, has launched a new detector, Malicious Script Executed, in general availability. Security Command Center findings now include two new attributes that provide additional information about the type of finding and the activity that triggered it. Documentation Security Command Center documentation now includes a page that maps resource type formats between Cloud Asset Inventory and Security Command Center.

Cloud SQL - The Cloud SQL Admin API v1 is now generally available.

Cloud TPU - Cloud TPU now supports Tensorflow 2.6.0.

VPC Service Controls - General availability for the following integration: Cloud DNS.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko