Welcome to issue #246 June 14th, 2021

News

Multi-Project Cloud Monitoring made easier - We're providing more flexibility in Cloud Monitoring by replacing Workspaces with Metrics Scopes. Metrics Scopes handle all of the same operational tasks as the previous construct, and you can associate a project with multiple Metrics Scopes.

Hola, South America! Announcing the Firmina subsea cable - The new Firmina subsea cable will run from the eastern U.S. to Argentina, and will be the world’s longest cable cable powered by a single power source.

Introducing container-native Cloud DNS: Global DNS for Kubernetes - The new container-native Cloud DNS integrates Cloud DNS with Google Kubernetes Engine (GKE) to provide in-cluster Service and Pod DNS resolution.

Node, Python and Java repositories now available in Artifact Registry - Expanded language support lets you store Java, Node and Python artifacts in Artifact Registry, for a more secure software supply chain.

New Cloud Asset Inventory capabilities help assess your Google Cloud environment - Analyze access, assess anomalies, identify insights and discover assets.

Google Cloud VMware Engine now HIPAA compliant - VMware customers looking to move to the cloud can now build a HIPAA-compliant architecture with Google Cloud VMware Engine.

New This Month: From leadership in real-time streaming to intelligent data fabric and analytics exchanges - May was a very busy month for data analytics product innovation. If you didn’t have the chance to attend our inaugural Data Cloud Summit, video replays of all our sessions are now available so feel free to watch them at your own pace.

Join us: Google Workspace for the Public Sector Sector events - Google Workspace helps the public sector tap into tools to improve collaboration, supercharge workflows, and accelerate innovation.

Google Dataflow is a Leader in The 2021 Forrester Wave™: Streaming Analytics - Forrester gave Dataflow a score of 5 out of 5 across 12 different criteria in its 2021 Forrester Wave: Streaming Analytics.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

All about cables: A guide to posts on our infrastructure under the sea - All posts on Google’s global subsea cable system in one handy location.

Build a platform with KRM: Part 1 - What’s in a platform? - Building an extensible, secure developer platform is hard. This post introduces how to use Kubernetes and its declarative resource model as the foundation for your platform.

Intro to Kf: Cloud Foundry apps on Kubernetes - While many companies are writing brand-new Kubernetes-based applications, it’s still quite common to find companies who want to migrate existing workloads. A common source platform for these applications is Cloud Foundry. However, getting an existing Cloud Foundry application running on Kubernetes can be non-trivial, especially if you want to avoid making code changes in your applications, or taking on big process changes across teams. That is, if you’re not using Kf to do a lot of that heavy lifting for you.

How Lowe’s meets customer demand with Google SRE practices - Lowe’s has adopted Google SRE practices to help developer and operations teams keep up with ecommerce demand.

Four steps to scale your quantitative investment research - Cloud-based technologies like data analytics, AI, Kubernetes and more can lead to better, faster investment research workflows for portfolio managers and researchers.

Provision GKE Cluster with Terraform - A Beginners Guide to Deploying Kubernetes with Terraform.

State of Managed Kubernetes 2021 - EKS vs. AKS vs. GKE from a Developer’s Perspective (2021 Edition).

Getting started with Google Cloud Monitoring APIs— Part 1 - Creating custom metrics and dashboard using Python's client library for Cloud Monitoring.

How someone used my Google cloud account for mining cryptocurrency - Description of getting compromised API keys and getting misused GCP project.

App Development, Serverless, Databases, DevOps

Accelerate Your App Development with Firebase Extensions - Firebase Extensions allow you to add pre-built and tested functionality with just a few clicks.

Simulate your Microsoft and Windows modernization journey with these demos - See for yourself how Google Cloud supports Windows virtual machines and containers at our online Microsoft and Windows on Google Cloud Demo Center.

Python Packages in Artifact Registry - Example of using Artifact Registry for Python package.

GCP Cloud Functions in Java - Debugging locally Java Cloud Function (with IntelliJ and Postman), and deploying to GCP.

Dart and Cloud Run - Example of deploying Dart web application on Cloud Run.

Use multiple paths in Cloud Functions, Python and Flask - Modifying Flask application to handle multiple URL paths for Cloud Function.

CloudSQLProxy — the wiser choice than tunneling to CloudSQL via bastion - A secure way to connect to your CloudSQL DBs - without sharing any IP address, without any netowrk allow-listing, without any bastion host.

On-prem MySQL to Cloud SQL using DMS — The Migration Story - A configuration process for ongoing migration for on-prem MySQL DB to Cloud SQL using Database Migration Service.

Stream your data: On-Prem MS-SQL to CloudSQL SQL Server to BigQuery (Part-1) - Online replication from On-Prem MS-SQL to CloudSQL SQL Server.

Stream your data: On-Prem MS-SQL to CloudSQL SQL Server to BigQuery (Part-2) - Build Pipeline from CloudSQL SQL Server to BigQuery.

Monitoring and Optimizing performance in PostgreSQL and Google Cloud SQL - SQL queries can be elusive in the area of performance…how can we optimize them using modern tools in our ecosystem today?

Database Migration and Replication: On Consistency - This article discusses database replication and migration from the viewpoint of data consistency.

How BIG is Cloud Bigtable? - “Building an application that needs low latency and high throughput?” You will need a database that can scale for a large number of reads and writes. Cloud Bigtable is designed to handle just that.

Big Data, Analytics, ML&AI

Apache Airflow + GCP Dataproc via DataProcSparkOperator - Doing integration with Cloud Dataproc and exploring DataProcSparkOperator running Airflow.

How to categorise text in a Pandas dataframe using Google’s Natural Language API - Using Cloud Natural API for text categorization.

Monitoring BigQuery reservations and slot utilization with INFORMATION_SCHEMA - BigQuery Reservations help manage your BigQuery workloads. Learn how to use BigQuery’s INFORMATION_SCHEMA system tables to create the System Tables Reports Dashboard.

Get notified when an expensive BigQuery job executes using Eventarc and SendGrid - Example of using Eventarc to get notified when expensive BigQuery job executes.

Why you need to explain machine learning models - Why explainable AI (XAI) is essential to widespread AI adoption, common XAI methods, and how Google Cloud can help.

AI Simplified: Managing ML data sets with Vertex AI - Learn how to create and manage data sets for training machine learning models in Vertex AI.

Getting model predictions from Vertex AI and how it compares to AI Platform - The article focuses on how to deploy models, get predictions and how this way of working compares to the workflow on the soon-to-be-outdated AI Platform.

Testing the different AutoML options in GCP - Experience in using different AutoML alternatives that GCP offers.

Machine Learning Pipelines with Google Cloud Platform - Example of using Vertex AI.

Accessing your TPUs in Docker Containers with TPU VM - Issues when connecting to TPU within Docker container.

How to detect machine-learned anomalies in real-time foreign exchange data - Model the expected distribution of financial technical indicators to detect anomalies and show when the Relative Strength Indicator is unreliable.

Various

Leveraging Google geospatial AI to prepare for climate resilience - Google Cloud's environmental initiatives.

Google Cloud {Code_Love_Hack} Winners! - This blog will recaps the hackathon, recognize the great winners, and provide a diverse array of use cases for GKE Autopilot and Cloud Code.

How BBVA is using Cloud SQL for it’s next generation IT initiatives - BBVA prioritizes managed services for speed, ease of maintenance, and centralized control features. Learn how Cloud SQL fits perfectly within that strategy.

How Zebra Technologies manages security & risk using Security Command Center - SCC provides visibility that helps Zebra Technologies manage security and risk in their cloud environment.

GCP Outpaces Azure, AWS in the 2021 Cloud Report - GCP delivered the most throughput (i.e. the fastest processing rates) on 4/4 of the Cloud Report’s throughput benchmarks: network throughput, storage I/O read throughput, storage I/O write throughput, and maximum tpm throughput – a measure of throughput-per-minute (tpm).

Slides, Videos, Audio

GCP Podcast - #263 SAP + Apigee: The Power of APIs with Benjamin Schuler and Dave Feuer.

Kubernetes Podcast - #151 Multi-Instance GPUs, with Kevin Klues and Pradeep Venkatachalam.

Releases

Google Kubernetes Engine Stable - (2021-R19) Version updates Version 1.18.17-gke.1900 is now the default version in the Stable channel.

Load Balancing - Network Load Balancing now supports load-balancing ESP (Encapsulating Security Payload) and ICMP (Internet Control Message Protocol) traffic.

Cloud Memorystore - Added support for Upgrading the Redis version of an instance with the Google Cloud Console. Released support for Redis version 6.x (Preview) on Memorystore for Redis.

Migrate for Compute Engine 4.8 - 4.11. Transition the underlying OS used by Migrate for Compute Engine components (Manager, Cloud Extensions, Importers, and Exporters) to use Ubuntu Advantage.

Network Connectivity Center - If you use a Router appliance spoke to connect more than 1,000 VMs, you might be unable to establish BGP sessions between the router appliance instance and Cloud Router.

Resource Manager - The Resource Settings API has entered general availability.

Security Command Center - Security Command Center Legacy, previously known as Cloud Security Command Center, and Event Threat Detection Legacy have been permanently disabled.

SAP Solutions - SAP NetWeaver high-availability cluster documentation for SLES A new load-balancer-based configuration guide for SAP NetWeaver high-availability clusters on SUSE Linux Enterprise Server (SLES) is available for use: HA cluster configuration guide for SAP NetWeaver on SLES.

Cloud Spanner - You can now find common queries for monitoring and troubleshooting on the Query page in the Cloud Console.

Cloud SQL - Cloud SQL now offers faster maintenance, with connectivity dropping for less than 60 seconds on average.

Cloud TPU - Cloud TPU now supports Tensorflow 2.5.0.

VPC Service Controls - Integration with Document AI VPC Service Controls is now generally available.

Virtual Private Cloud - If you enable PROXY protocol for a Private Service Connect service attachment, the PROXY protocol header value was previously either 0xEA or 0xE0.

Cloud VPN - You can check for VPN tunnel overutilization using the VPN tunnel utilization recommender.

Workflows - v1. String processing functions are now available in the text module of the Workflows standard library.

AI Platform Unified - You can now use a pre-built container to serve predictions from TensorFlow 2.5 models. You can now use a pre-built container to serve predictions from XGBoost 1.4 models.

AI Platform Prediction - Runtime version 2.5 is now available.

Anthos clusters on VMware 1.7 - Anthos clusters on VMware 1.5.4-gke.2 is now available. Fixes These security vulnerabilities have been fixed: Fixed CVE-2021-25735 mentioned in the GCP-2021-003 Security Bulletin, CVE-2021-31535, and other medium and low vulnerability CVEs with fixes available. Anthos clusters on VMware 1.6.3-gke.3 is now available. Fixes These security vulnerabilities have been fixed: Fixed CVE-2021-25735 mentioned in the GCP-2021-003 Security Bulletin, CVE-2021-31535, and other medium and low vulnerability CVEs with fixes available.

Cloud Asset Inventory - New resource types are now available.

BigQuery - BigQuery now supports parameterized types.

Billing - Committed use discounts for Google Kubernetes Engine (GKE) are now Generally Available to purchase for workloads running on GKE Autopilot.

Cloud Composer - New versions of Cloud Composer images: composer-1.17.0-preview.2-airflow-2.0.1 composer-1.16.6-airflow-1.10.15 composer-1.16.6-airflow-1.10.14 (default) composer-1.16.6-airflow-1.10.12. You can now store values for the smtp_password Airflow configuration option in Secret Manager. Increased the timeout for environment upgrade operations to support upgrades for databases up to 16 GB in size. Fixed memory issues that occurred while syncing files on machine types with more than 8 vCPUs. DAG parsing and task processing in Airflow no longer fails because of incorrectly formatted Airflow logs. (New environments only) Some log messages related to Airflow web server access were previously missing in Cloud Logging. (Available without upgrading) Updating environment labels now correctly overrides previous labels in billing reports. Cloud Composer 1.10.4 has reached its end of full support period.

Compute Engine - NVIDIA® T4 GPUs are now available in the following additional regions and zones: St. Ghislain, Belgium: europe-west1-b,c,d. Generally available: You can configure how your regional managed instance group distributes instances across zones by using capacity-aware distribution shapes, which can automatically deploy instances to zones where capacity is available and optionally prioritize the use of reservations. Preview: When rolling out configuration or application updates to a stateful or stateless managed instance group, use the minimum and most disruptive allowed actions to control disruption to your workload.

Cloud Dataflow - Dataflow SQL now supports user-defined functions (UDFs) written using Java. Dataflow is now able to use workers, Dataflow Shuffle, Streaming Engine, FlexRS, and regional endpoints in zones in Melbourne (australia-southeast2).

Dataproc - Custom image limitation: Currently, the following Dataproc image versions are the latest images that can be used as the base for custom images: 1.3.89-debian10, 1.3.89-ubuntu18 1.4.60-debian10, 1.4.60-ubuntu18 1.5.35-debian10, 1.5.35-ubuntu18, 1.5.35-centos8 2.0.9-debian10, 2.0.9-ubuntu18, 2.0.11-centos8.

Document AI - v1. VPC Service Controls Integration with Document AI VPC Service Controls is now generally available.

Cloud Functions - Cloud Functions now supports Ruby 2.6 and 2.7 at the General Availability release level.

IAM - The documentation for IAM role recommendations now has more detail about how insights are used to generate recommendations.

Google Kubernetes Engine - GKE Multi-cluster Services support for pod-specific addressing is now generally available. Volume snapshots is now generally available. Committed use discounts are now generally available to purchase for Google Kubernetes Engine (Autopilot Mode). For GKE clusters running Windows Server node pools, you can see the version mapping between GKE versions and Windows Server versions for all available GKE versions by using a gcloud command. (2021-R19) Version updates GKE cluster versions have been updated. If you manually upgrade your cluster from 1.18 to 1.19 and the network tier configuration on an existing external network load balancer does not match the network tier annotation in the service spec (if unspecified, defaults to Premium), the load balancer will be deleted and recreated, and the network tier configuration will be enforced. A domain-scoped project is not supported in GKE version 1.20. 1.20 is now generally available Kubernetes 1.20 is now generally available (GA). The node.k8s.io/v1beta1 RuntimeClass API has graduated to node.k8s.io/v1 with no changes. As of version 1.20, the kubelet no longer creates the target_path for NodePublishVolume in accordance with the CSI spec. Starting in version 1.20, timeouts on exec probes are honored, and default to 1 second if unspecified. Non-deterministic treatment of objects with invalid ownerReferences was fixed in version 1.20. The metadata.selfLink field, deprecated since version 1.16, is no longer populated in version 1.20. Reminder: Future beta API removals in versions 1.22 and 1.25 Kubernetes versions 1.22 and 1.25 will stop serving several deprecated beta APIs. You can now specify the default image type to use for new auto-provisioning node pools.

GKE - (2021-R19) Version updates Version 1.18.17-gke.1900 is now the default version.

Google Kubernetes Engine Rapid - (2021-R19) Version updates Version 1.20.6-gke.1400 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - (2021-R19) Version updates Version 1.19.10-gke.1600 is now available in the Regular channel.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko