News

Protect your running VMs with new OS patch management service - New OS patch management service protects your Compute Engine VMs.

Kotlin, meet gRPC: a new open-source project for modern apps - You can now use gRPC with Kotlin, including in Cloud Run.

Introducing Dataflow template to stream data to Splunk - Learn how to set up a streaming pipeline for Google Cloud data into Splunk Cloud or Enterprise with this new Pub/Sub to Splunk Dataflow template.

Find and fix issues faster with our new Logs Viewer - Find and fix cloud infrastructure issues faster, and stay compliant too, with help from the new logging interface in Google Cloud’s Logs Viewer.

How Google Cloud is helping COVID-19 academic research - COVID-19 researchers are in a race to understand more, and we’re hoping to support them with a number of initiatives designed to help.

Supporting not-for-profit COVID-19 response efforts with Google Maps Platform credits

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

How GKE surge upgrades improve operational efficiency - You can ensure GKE node upgrades complete successfully with surge upgrades.

Setting up Elasticsearch and Kibana on Google Kubernetes Engine with ECK - The article goes through a process of setting up Elasticsearch on GKE.

Getting Started with Artifact Registry: Deploying to Google Kubernetes Engine - The article explains how to deploy containers from Artifact Registry to GKE with Cloud Build.

A multi-cluster gRPC architecture on GKE - This post explains how to load-balance a gRPC application across many GKE clusters in different regions to increase performance and availability.

Ubiquiti and GCP Site-to-Site VPN: The How-To Guide - How to set up a direct, encrypted connection between Google Cloud and your on-premise network.

Creating a Helm repo on Google Cloud - A look at how Mettle created a helm repository in Google Cloud to store their custom helm charts.

Terraform on GoogleCloud — impersonating with short-lived AccessTokens & ServiceAccounts - Using ServiceAccounts with limited IAM roles to request AccessTokens with privileged IAM roles for GCloud resources using Terraform.

Google Cloud Inter-region latency and throughput dashboard in Data Studio

Viewing cluster autoscaler events - The Google Kubernetes Engine (GKE) cluster autoscaler emits visibility events, which are available as log entries in Cloud Logging. This page shows how to view those logged events to gain insight on when and why the GKE cluster autoscaler makes autoscaling decisions.

App Development, Serverless, Databases, DevOps

My cheat sheet for choosing the right database on GCP - Cheatsheet table with Database options on GCP.

How to Deploy a Cloud SQL DB with a Private IP only, using Terraform - Setting up Cloud SQL with Terraform.

Spanning the Globe without Google Spanner - The article explains how to set up YugabyteDB (a database inspired by Spanner) on GKE cluster.

Google Cloud Spanner driver for Go - Google Cloud Spanner driver for database/sql.

Cloud NVMes: the blind side of them - Exploring the performance of Compute Engine local SSDs (NVMe).

Private Access With Serverless — Managing Serverless Application Access in Google Cloud Platform - Setting up Google App Engine and Cloud Run to run as private web applications.

Build a serverless REST API with Node.js and MySQL on Google Cloud — Serverless Toolbox - An example of serverless REST API with Cloud Run and Cloud SQL, using Node.js and MySQL.

Choosing the right load balancer - The article goes through different load balancing options on GCP and how to choose the right one.

Application Capacity Optimizations with Global Load Balancing - Setting up Load Balancer on Instance Group of Compute Engine instances.

How can Chatbots help during global pandemic (COVID-19)? - A list of chatbot applications that can be used in COVID-19 time.

Building a FAQ bot in DialogFlow in 15 minutes - Simple chatbot in Dialogflow which responses to questions about GCP products.

How to Send Emails From Firebase With the Trigger Email Extension - An overview of Firebase extension for sending emails.

A Ktor API on Google App Engine’s Java 11 Standard Environment - A simple, quick walk-through to setup a basic Ktor API on Google’s App Engine Java 11 Standard Environment.

Big Data, Analytics, ML&AI

9 Common Mistakes with Cloud Data Fusion - 9 common mistakes that you will probably deal with when working with Data Fusion.

Redacting sensitive information from doctors’ patient notes - Implementing pipeline on GCP to process medical notes about patients affected by COVID-19 (from the public dataset on GCP from cases in Italy).

Exporting data from Firebase (Firestore) to BigQuery - Two ways how to get data from Cloud Firestore to BigQuery.

How to find the interior centroid of US counties using BigQuery GIS - How to handle the problem of centroids being in the Great Lakes.

BigQuery Materialized Views and Streaming Data - TL;DR: BigQuery materialized views and streaming data can be used together for building cost-effective near real-time dashboards.

From raw data to machine learning model, no coding required - How to build an entire ML pipeline, including data transformation and model training, without code.

Automation of data wrangling and Machine Learning on Google Cloud - Using Cloud Dataprep, Cloud Composer and BigQuery ML.

Various

Why we chose Google Cloud as the infrastructure platform for AgileData.io - Pick a few things that really matter, not thousands of “requirements”.

The Digitization of Twiga Takuwa Farm - Precision Agriculture with Liquid Telecom’s 0G (Sigfox) Network and Google Cloud IoT.

Slides, Videos, Audio

GCP Podcast - #216 Rugby and ML with Capgemini.

Kubernetes Podcast - #99 kpt, with Morten Torkildsen.

Releases

Access Approval - Access Approval API reached General Availability (GA) status.

AppEngine Standard Python - Quotas for sockets have been removed.

Google Cloud Armor - Google Cloud Armor support for CDN origins and hybrid origins is now available in General Availability. Update to rules language syntax.

BigQuery - BigQuery Reservations is now Generally Available (GA). Around the end of April 2020, INFORMATION_SCHEMA (Beta) views for dataset metadata will return metadata about all datasets in a region.

BigQuery ML - BigQuery ML now supports Matrix Factorization models for recommendations, as a beta release.

BigQuery Transfer - BigQuery Data Transfer Service now supports Google Merchant Center data transfers for best sellers data.

Billing - Discount sharing for committed use discounts is now available in beta. Cloud Billing console now has a Pricing report, providing a tabular view of the prices of Google’s cloud services SKUs, including Google Cloud, Google Maps Platform, and G Suite.

CDN - Cloud CDN request logs now include a cacheId field, which captures the location and cache node the client connected to. Signed Cookies are available in General Availability.

Cloud Composer - Composer version 1.10.1 has been rolled back.

Config Connector - Added readiness probes to Config Connector pods.

Cloud Dataflow - Cloud Dataflow SQL is now generally available.

Dataprep - TD-47149: Cannot edit settings when importing Google Sheets.

Dataproc - Announcing the Beta release of Dataproc on Google Kubernetes Engine. Image 1.5 Jupyter on Dataproc now supports exporting notebooks as PDFs. Image 1.5 Presto now includes two default connectors: * bigquery pointing to the datasets of the cluster's project * bigquery_public_data pointing to the public datasets. Image 1.3, 1.4 and 1.5 Added Component Gateway support for Datarpoc clusters secured with Kerberos. New sub-minor versions of Dataproc images: 1.2.95-debian9, 1.3.55-debian9, 1.4.26-debian9, 1.3.55-debian10, 1.4.26-debian10, 1.5.1-debian10, 1.3.55-ubuntu18, 1.4.26-ubuntu18, 1.5.1-ubuntu18. Image 1.5 Updated Presto to version 331. Created cloud-sql-proxy log file for the Cloud SQL Proxy initialization action and for Dataproc clusters with Ranger that use Cloud SQL Proxy. Image 1.3 and 1.4 Debian 10 images will become default images for 1.3 and 1.4 image tracks and Debian 9 images will not be released for these tracks anymore after June 30, 2020. Images 1.4 and 1.5 SPARK-29080: Support R file extension case-insensitively when submitting Spark R jobs. Image 1.3, 1.4 and 1.5 Fixed a bug where Jupyter was unable to read and write notebooks stored in Cloud Storage buckets with CMEK enabled. Image 1.3, 1.4 and 1.5 HIVE-17275: Auto-merge fails on writes of UNION ALL output to ORC file with dynamic partitioning.

Deployment Manager - Added support for Cloud Scheduler through gcp-types/cloudscheduler-v1:projects.locations.jobs. You can now apply granular IAM Permissions to the Google APIs service account used by Deployment Manager, as we've removed the requirement for roles/editor being assigned to the service account. Updates on Cloud Functions resources using gcp-types/cloudfunctions-v1 now retry on 429 errors. GKE clusters and node pools will wait for maintenance to complete before attempting to apply any updates. Deployment Manager now acquires existing GKE cluster resources of type gcp-types/container-v1:projects.locations.clusters. Added support for updating the following properties on gcp-types/container-v1:projects.zones.clusters and gcp-types/container-v1:projects.locations.clusters: binaryAuthorization databaseEncryption masterAuthorizedNetworksConfig autoscaling resourceUsageExportConfig verticalPodAutoscaling Additionally, for gcp-types/container-v1beta1:projects.zones.clusters and gcp-types/container-v1beta1:projects.locations.clusters the following fields can also be updated: podSecurityPolicyConfig privateClusterConfig shieldedNodes workloadIdentityConfig. Deployment Manager now correctly updates autoscaling properties for resources of type gcp-types/container-v1:projects.locations.clusters.nodePools and gcp-types/container-v1beta1:projects.locations.clusters.nodePools. Deployment Manager now correctly acquires Access Context Manager resources of type gcp-types/accesscontextmanager-v1:accessPolicies.accessLevels and gcp-types/accesscontextmanager-v1beta:accessPolicies.accessLevels if the resources already exist. Added support for updating Cloud Pub/Sub subscriptions using gcp-types/pubsub-v1:projects.subscriptions. Deployment Manager now correctly deletes Compute Engine forwarding rules of type compute.v1.forwardingRule, compute.beta.forwardingRule, gcp-types/compute-v1:forwardingRules and gcp-types/compute-beta:forwardingRules when the resource name does not match the forwarding rule name. Performance improvements when handling large Swagger / OpenAPI specs when adding an API as a type provider.

Cloud Data Loss Prevention - Add PDF and WORD FileTypes and PDF and WORD_DOCUMENT BytesTypes. Added additional infoType detectors: IRELAND_PPSN IRELAND_PASSPORT.

KMS - Cloud External Key Manager (Cloud EKM) is generally available.

Google Kubernetes Engine - GKE cluster versions have been updated. The following Kubernetes versions are now available for new clusters and for opt-in master upgrades and node upgrades for existing clusters. No channel 1.14.10-gke.36 is now available. Google Kubernetes Engine will gradually upgrade clusters in the regular channel to GKE 1.16 during the next release.

Google Kubernetes Engine Rapid - 1.16.8-gke.9 is now available in the Rapid release channel.

Load Balancing - External HTTP(S) load balancers now support URL rewrites and redirects.

Marketplace Partners - You can now create private quotes for VM solutions (alpha).

Cloud Spanner - Cloud Spanner Backup and Restore is now generally available, enabling you to create backups of Cloud Spanner databases on demand, and restore them. Query Optimizer Versioning is now generally available, enabling you to select which version of the optimizer to use for your database, application or query.

Traffic Director - A new document, Traffic Director features, is published.

Network Intelligence Center - Performance Dashboard is now available in General Availability. Firewall Insights is now in Beta.

AI Platform Prediction - The pricing of Compute Engine (N1) machine types for online prediction in the us-central1 region has changed.

Event Threat Detection - Event Threat Detection is now in general availability.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko