Welcome to issue #250 July 12th, 2021

News

Cloud SQL for MySQL launches IAM database authentication - Simplify administration and security in Cloud SQL for MySQL with IAM database authentication, now in GA.

New This Month in Data Analytics: Simple, Sophisticated, and Secure - We used all the hours we could in June to deliver a flurry of new features across BigQuery, Dataflow, Data Fusion, and more. Let’s take a look!

Learn to code for the cloud: Earn native app development skills badges for free - Google Cloud’s new Native App Development skills challenge lets you earn badges that demonstrate your ability to create cloud-native apps.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Build a platform with KRM: Part 5 - Manage hosted resources from Kubernetes - Learn how to manage your Google Cloud-hosted resources with GKE and Config Connector.

Accidental Network Outage? Here’s How You Can Regain Access to Your Google Cloud Windows Instance - Reconnecting when Windows server is unexpectedly disconnected from the internet.

First steps in Pulumi with Python & Google Cloud Platform - Getting started with Pulumi on GCP.

App Development, Serverless, Databases, DevOps

Authenticating to Cloud SQL for PostgreSQL with IAM service accounts - This blog post’s goal is to provide an ELI5 procedural outline of the steps that one needs to take in order to that led to perform: Configure Cloud SQL for IAM Authentication Configure a user agent to connect to a given database using assigned service account credentials.

Measuring Cloud Spanner performance for your workload - In this post, we will explore a middle ground to performance testing using JMeter. Performance test Cloud Spanner for a custom workload before making application code changes and executing data migration.

All you need to know about Firestore: A cheatsheet - Building applications is a heavy lift due to the technical complexity, which includes the complexity of backend services that are used to manage and store data. Firestore alters this by having Google Cloud manage your backend complexity through a complete backend-as-a-service!

Saving $30000 a month by improving Garbage Collection - Improving storage costs at Mixpanel.

Case of Cost Spike in GCP Stackdriver Monitoring - Optimizing Cloud Monitoring costs.

Session Management on Google Cloud Run with Serverless Redis - A basic web application that runs on Google Cloud Run and keeps the user sessions in Upstash Redis.

Setting Up Gmail Pub/Sub - This article is a basic walkthrough of setting up Pub/Sub notifications when new emails arrive in Gmail.

Uploading an Image to Firebase Cloud Storage and returning URL with Express / Nodejs - A step by step tutorial to create and deploy web application in Firebase to upload images.

Big Data, Analytics, ML&AI

How to build an open cloud datalake with Delta Lake, Presto & Dataproc Metastore - Building an Open Data Lake with Apache Spark for data processing, Presto as a query engine and Open Formats such as Delta Lake for storing all data.

Exploring Pub/Sub and Pub/Sub Lite - Exploring and comparing Pub/Sub and Pub/Sub Lite.

Using NPM Library in Google BigQuery UDF - Javascript UDF’s are cool and using with NPM library is a whole new world to explore!

Why We Shouldn’t Ignore Null Values and How to Treat Them - Data cleaning using Google BigQuery SQL.

Working with Times and Dates in BigQuery - Common operations with dates in BigQuery.

Version control BigQuery with Terraform (with CI/CD too) - Using Terraform to version BigQuery views.

Data Status Time Machine on Persisted dbt Artifacts - Storying and analyzing dbt artifacts in BigQuery.

Step-by-step guide to run dbt in production with Google Cloud Platform - A step-by-step guide to run dbt in production with GCP.

Data Layers in Data Studio - Understanding the difference between DWH data, data source, and chart data.

Unsupervised Anomaly Detection using BigQuery ML - Three unsupervised Machine learning models provided by BigQuery ML to predict anomalies in both time series and non-time series data.

Various

Passing the Google Cloud Certified Professional Architect 2021 exam - Sharing experience on the preparation for the GCP Architect certification exam and the actual exam itself.

GCP Associate Cloud Engineer Exam Preparation App for iOS, android, Windows10 - GCP Associate Cloud Engineer Exam Preparation App for iOS, Android, Windows10.

Notes from my beta Google Cloud Digital Leader certification exam - An overview of Cloud Digital leader certification exam.

Slides, Videos, Audio

GCP Podcast - #266 Data Analytics Launches with Bruno Aziza and Eric Schmidt.

Kubernetes Podcast - #153 Komodor, with Itiel Shwartz.

Releases

AI Platform - Deep Learning VMs - M74 Release Re-released all CUDA 11 images to fix an issue in which VM instances are unable to detect GPU accelerators after a reboot. In Debian 10 GPU images, updated NVIDIA drivers to 460.73.01 and CUDA to 11.0.3.

Anthos clusters on AWS - Anthos clusters on AWS aws-1.8.0-gke.8 is now available. This release fixes an issue mentioned in the entry on July 2, 2021. When you upgrade or update a user cluster, the Connect agent is automatically updated to the latest version.

BigQuery - BigQuery now supports materialized views without aggregation and materialized views with inner join.

Cloud Composer - Cloud Composer 1.16.9 release started on July 8, 2021. New versions of Cloud Composer images: composer-1.17.0-preview.5-airflow-2.0.1 composer-1.16.9-airflow-1.10.15 (default) composer-1.16.9-airflow-1.10.14 composer-1.16.9-airflow-1.10.12. When PyPI modules installation fails with certain error types in VPC SC environments, an additional attempt to perform the environment update operation is made using an in-cluster build. Cloud Composer 1.10.6 has reached its end of full support period.

Dataproc - Custom image limitation: New images announced in the Dataproc release notes are not available for use as the base for custom images until one week from their announcement date. The Dataproc v1beta2 APIs are deprecated. The end date of support for Dataproc image version 1.4 has been extended from August, 2021 to November, 2021. New sub-minor versions of Dataproc images: 1.3.92-debian10, 1.3.92-ubuntu18, 1.4.63-debian10, 1.4.63-ubuntu18, 1.5.38-centos8, 1.5.38-debian10, 1.5.38-ubuntu18, 2.0.12-centos8, 2.0.12-debian10, and 2.0.12-ubuntu18. Upgraded Spark version to 2.4.8 in the following images: Image 1.4 Image 1.5. Minimum boot disk sizes for Dataproc images: Image 2.0: 30GB Image 1.5: 20GB Image 1.4: 15GB Image 1.3: 15GB. Fixed stdout/stderr links on Spark History Server Web UI of the Persistent History Server in the following images: Image 1.4 Image 1.5. Fixed a bug where personal auth credentials would not propagate to every VM in the cluster if VPC service controls were enabled.

Deep Learning VM - M74 Release Re-released all CUDA 11 images to fix an issue in which VM instances are unable to detect GPU accelerators after a reboot. In Debian 10 GPU images, updated NVIDIA drivers to 460.73.01 and CUDA to 11.0.3.

Error Reporting - Error Reporting notifications has been upgraded to use the Alerting infrastructure.

Cloud Functions - Cloud Functions now logs pending queue requests abort error messages.

Cloud Healthcare API - The Cloud Healthcare API offers single-region support in the europe-west3 (Frankfurt) region. The Cloud Healthcare API offers single-region support in the asia-northeast3 (Seoul) region. The Cloud Healthcare API offers single-region support in the asia-south1 (Mumbai) region.

Google Kubernetes Engine - (2021-R22) Version updates GKE cluster versions have been updated. Microsoft published a security bulletin on a Remote code execution (RCE) vulnerability, CVE-2021-34527, that affects the print spooler in Windows servers.

GKE - (2021-R22) Version updates The following versions are now available: 1.18.18-gke.1101 1.18.18-gke.1701 1.18.19-gke.1701 1.18.19-gke.2101 1.18.20-gke.900 1.19.10-gke.1001 1.19.10-gke.1601 1.19.10-gke.1701 1.19.11-gke.1701 1.19.11-gke.2101 1.19.12-gke.700 1.20.8-gke.700 The following versions are no longer available: 1.18.18-gke.1100 1.18.18-gke.1700 1.18.19-gke.1700 1.18.19-gke.2100 1.19.10-gke.1600 1.19.10-gke.1700 1.19.11-gke.1700 1.19.11-gke.2100.

Google Kubernetes Engine Rapid - (2021-R22) Version updates Version 1.20.7-gke.2200 is now the default version.

Google Kubernetes Engine Regular - (2021-R22) Version updates Version 1.19.11-gke.1701 is now available in the Regular channel.

Google Kubernetes Engine Stable - (2021-R22) Version updates Version 1.18.19-gke.1701 is now available in the Stable channel.

Load Balancing - External TCP/UDP Network Load Balancing now allows you to configure a connection tracking policy.

Cloud Logging - The Share link feature for queries in the Logs Explorer now lets you choose whether to include an absolute time range or a relative time range. Preview: Cloud Logging now supports alerts based on matching the content of your logs.

Cloud Monitoring - Preview: Cloud Logging now supports alerts based on matching the content of your logs.

Network Connectivity Center - Network Connectivity Center now includes hard limits on the number of resources that can be linked to an individual spoke.

Cloud Spanner - The NUMERIC data type is now supported as a valid key column type, so you can now use NUMERIC type columns when specifying primary keys, foreign keys, and secondary indexes.

Cloud SQL MySQL - IAM database authentication for Cloud SQL for MySQL is now generally available.

Cloud Trace - Cloud Trace announces that the OpenTelemetry library for Java is now generally available.

Vertex AI - You can now containerize and run your training code locally by using the new gcloud beta ai custom-jobs local-run command.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko