Welcome to issue #207 September 14th, 2020

News

Faster, more powerful apps for everyone: What happened at Next OnAir this week - Google Next business application platform week highlights include API Gateway, Apigee and AppSheet capabilities, and digital transformation insights.

Google Cloud API Gateway is now available in public beta - Google Cloud API Gateway makes it easy to securely share and manage serverless APIs.

Introducing interactive code samples in Google Cloud documentation - With interactive code samples in Google Cloud documentation, you can replace the variables inline, before you even copy the snippet.

Analytics get smarter for SAP customers with Informatica and Google Cloud - Informatica and Google Cloud have partnered to help SAP customers better integrate and manage their data and gain new, powerful capabilities like world class analytics and AI at scale.

Accelerate digital transformation with business application platform - Google Cloud launches API Gateway, Apigee data source for G Suite, AppSheet Automaton, and Business Application Platform.

Expanding Google Cloud’s Confidential Computing portfolio - Google Cloud Confidential Computing is now GA and including Confidential GKE Nodes.

Google Cloud named a Leader in first Forrester Wave: Public Cloud Development & Infrastructure Platforms for ANZ - Forrester noted Google Cloud has built a compelling enterprise strategy using artificial intelligence (AI) and machine learning (ML), analytics, containers, and functions to build new applications or modernise legacy ones.

Google a leader in Gartner Magic Quadrant for Cloud Infrastructure and Platform Services - For the third consecutive year, Google Cloud is a leader in Gartner Infrastructure as a Service (IaaS) Magic Quadrant.

New capabilities for Assured Workloads for Government - Assured Workloads for Government is now GA with new features, including FedRAMP Moderate support.

Spanning the globe with Google Cloud VMware Engine - With the addition of London, Frankfurt and Tokyo regions, Google Cloud VMware Engine is now available around the globe.

Announcing Google Cloud Next OnAir EMEA: 29 Sep - 27 Oct - Google Cloud Next OnAir EMEA begins 29 Sep and offers a full roster of curated content, including more than 30 new sessions specially tailored to the region.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Lost in translation: encryption, key management, and real security - How encryption key management is an important part of data security, and best practices to follow in your implementation.

What is Google Kubernetes Engine (GKE)? - Sketchnote about GKE.

GCP Service Account + HashiCorp Vault - Using HashiCorp Vault to manage the keys of service accounts.

Reducing costs by leveraging GKE Optimizations and GCP Preemptible VMs - In this article we will show an example of how the thought process works when we tackle the resource optimization problem.

Restricting Write Permissions on Folders in Google Cloud Storage with IAM Conditions - Setting access for Cloud Storage on the "folder" level.

How to install monitoring agent for cloud monitoring on multiple VMs - Simplifying installation of monitoring agents on multiple VMs.

App Development, Serverless, Databases, DevOps

Optimize your RDS MySQL to GCP Cloud SQL Migration - Tips and tricks when migrating from AWS RDS to Cloud SQL.

Migrate data from Firestore to Cloud SQL on Google Cloud - Migrating data from Firestore to Cloud SQL.

A first look at serverless orchestration with Workflows - Examples of using new GCP product - Workflows.

How to deploy a simple Flask app on Cloud Run with Cloud Endpoint - Deploying Python web app on Cloud Run and Cloud Endpoints.

Analyze running queries in Cloud Spanner to help diagnose performance issues - This article discusses how the Oldest Active Queries complements other introspection tools of Spanner and helps users troubleshoot system performance issues while they are ongoing.

Manage Serverless APIs With API Gateway in GCP - Example of using API Gateway to connect various serverless applications on GCP.

Authorizing end users in Cloud Run with Pomerium - This guide covers how to deploy Pomerium to Cloud Run, providing end-user authentication and authorization to other endpoints.

Big Data, Analytics, ML&AI

Setup DBT with Cloud Composer - Google Cloud Composer, and dbt can work together to develop ETL processes. This article will show you how to set up the two together.

Long-Running Spark Jobs on GCP using Dataproc with Preemptible Instances - Validating Spark Jobs on Dataproc Endure Preemptible Instance Recycling.

Google Cloud Pub/Sub - A brief overview of Cloud PubSub.

BigQuery Explained: Storage Overview - This post dives into BigQuery storage organization and format, how to partition, and cluster data to optimize performance and costs.

Publish Cloud Dataprep Profile Results to BigQuery - This article describes how to use webhooks and Cloud Functions to automatically publish Dataprep-generated profile information into BigQuery.

Use Google Sheets as a ‘Data Creek’ for your Data Lake - How to use Google Sheets for your Data Integration to BigQuery.

Finding the Closest Weather Stations — BigQuery Public Datasets - Use BigQuery Public Datasets, Geography Functions, and ARRAY_AGG & UNNEST to locate and query local historical weather data near any address.

Cool things you can do using window functions in BigQuery - Using windowing functions in BigQuery.

BigQuery: Populating a development environment with hashed data - Using BigQuery’s hashing functions to create test data for use in development/testing process.

Various

3 reasons why 2020 is a good time to re-evaluate your Google Cloud Partnership - Recent changes impacting Google Cloud partnerships are making 2020 a compelling time to re-assess your practice and opportunity.

2020 Google Cloud Certified Professional Data Engineer Certification - Preparing for the Data Engineer certification exam.

Get Certified: Google Cloud Platform Professional Data Engineer - In this blog post series, you will find some personal suggestions about how to get certified on the Google Cloud Platform.

Slides, Videos, Audio

GCP Podcast - #235 Active Assist with Chris Law + MariaDB SkySQL with Robert Hedgepeth.

Kubernetes Podcast - #120 Airbnb, with Melanie Cebula.

Releases

BigQuery - You can now use the BQ.JOBS.CANCEL system procedure to cancel a job.

CDN - Added a new tutorial: Faster web performance and improved web protection for load balancing.

Compute Engine - You can build highly available deployments of stateful workloads on VM instances using stateful managed instance groups (stateful MIGs).

Dataproc - Added the PrivateIpv6GoogleAccess API field to allow configuring IPv6 access to Dataproc cluster. New sub-minor versions of Dataproc images: 1.3.68-debian10, 1.3.68-ubuntu18, 1.4.39-debian10, 1.4.39-ubuntu18, 1.5.14-debian10, 1.5.14-ubuntu18, 2.0.0-RC10-debian10, and 2.0.0-RC10-ubuntu18. 1.3-1.5 Images: HIVE-18323: Vectorization: add the support of timestamp in VectorizedPrimitiveColumnReader for parquet. 1.5 and 2.0 preview images: Upgraded the jupyter-core and jupyter-client packages in the 1.5 and 2.0 images to be compatible with the installed notebook package version. 2.0 preview image: HIVE-21085: Materialized views registry starts non-external Tez session. Fixed a regression that could cause clusters to fail to start if user-supplied keystore/truststore are provided when enabling Kerberos.

Cloud Data Loss Prevention - STREET_ADDRESS infoType detector is now available in all regions.

Cloud Firestore - You can now view your Firestore Security Rules in the Cloud Console.

Cloud Healthcare API - v1. Two new permissions, healthcare.locations.get and healthcare.locations.list, have been added to the IAM permissions.

IAM - You cannot undelete most service accounts at this time.

Google Kubernetes Engine - GKE clusters in the ERROR state will be automatically deleted. Kubernetes 1.18 is now available in the Rapid channel. TaintBasedEvictions are generally available in GKE in 1.18 clusters. Consumers of the certificatesigningrequests/approval API must now have permission to approve certificate signing requests (CSRs) for the specific signer requested by the CSR. GKE now allows clusters of up to 15,000 nodes when using GKE 1.18. In GKE 1.18, Shielded Nodes are enabled by default. While the GKE API does support the use of the ingressClassName and ingressClass resources, the Compute Engine ingress controller does not. Cluster Autoscaler for GKE 1.18 could have problems with very large clusters or scale ups where there are over 5,000 nodes in the cluster or over 1,000 nodes being added at the same time. GKE cluster versions have been updated. The following Kubernetes versions are now available for new clusters and for opt-in master upgrades and node upgrades for existing clusters. No channel Note: Your clusters might not have these versions available.

Load Balancing - Added a new tutorial: Faster web performance and improved web protection for load balancing.

Cloud Monitoring - The API for creating and managing alerting policies is now Generally Available.

Security Command Center - Security Command Center Premium is now in general availability (Container Threat Detection remains in beta). Improved Summary Dashboard A new set of interactive charts and tables provide a high-level overview of all threats and vulnerabilities. Onboarding and configuration upgrades A streamlined interface lets you manage organization-wide service enablement settings. Security Health Analytics now supports real-time detections, with some exceptions. Managed Web Security Scans are now available to all Security Command Center Premium users. gcloud integration with new, simplified Beta APIs (Alpha) The gcloud command line interface can now access configuration functionality through new Beta APIs. Documentation New documentation includes details on onboarding and enablement in the Security Command Center latency overview and updates on billing tiers.

Cloud Spanner - Cloud Spanner introduces a new introspection tool that provides insights into queries that are currently running in your database.

Cloud Storage - New conditions available for Object Lifecycle Management: Noncurrent time conditions allow you to define lifecycle actions based on when an object became noncurrent. Custom time metadata is now available for objects.

Cloud TPU - Compute Engine TPU Metrics and Logs In-Context New Monitoring tab for TPUs provides key TPU Metrics and access to logs at a glance.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko