Welcome to issue #252 July 26th, 2021

News

Bringing Kubernetes’ goodness to Windows Server apps with Anthos - You can now run Windows Server apps on Anthos in your on-premises data center, in addition to running them on GKE.

Extending the power of Chronicle with BigQuery and Looker - Google Cloud security analytics platform, Chronicle, now integrated with BigQuery and Looker improving security operations.

Introducing the Data Validation Tool for EDW migrations - New open-sourced Python CLI tool helps take the headache out of data validation.

Give your data processing a boost with Dataflow GPU - With Dataflow GPU, customers can leverage the power of NVIDIA GPUs in their data pipelines.

Reimagining the way we experience data with new Looker product features - Google Cloud introduces new features to Looker to help organizations deliver data-driven experiences at scale.

The Ops Agent is now GA and it leverages OpenTelemetry - Today, we’re happy to announce the General Availability of the new Ops Agent, which replaces both the Logging and Monitoring agents and simplifies installation, management, and configuration across the board.

Faster Cloud Storage transfers using the gcloud command-line - The new gcloud storage enables super-fast data transfers using a new parallelization strategy and hashing library.

Cloud CISO Perspectives: July 2021 - Keep reading below for the highlights and learnings from our Security and Government Security Summits, Google-wide efforts to protect users from online threats and our continued progress securing the software supply chain and open source software security.

Advancing our trusted cloud with engineered-in, invisible security - A vision for invisible security that helps stay ahead of evolving threats.

New Google Cloud Security offerings, just announced in the Government Security Summit keynote

Modernizing SOC ... Introducing Autonomic Security Operations - The Autonomic Security Operations solution is a new approach to transforming Security Operations to protect against modern-day security threats, built on Chronicle and Google Cloud.

Extending our Trusted Cloud: Introducing Cloud IDS for Network-based Threat Detection - Cloud IDS (Intrusion Detection System) helps detect malware, spyware, and command-and-control attacks.

Registration is open for Google Cloud Next: October 12–14 - Register now for Google Cloud Next on October 12–14, 2021.

Announcing the winners of our Google Cloud 2020 Partner Awards - Announcing the winners of our Google Cloud 2020 Partner Awards.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

New Paper: Assuring Compliance in the Cloud - Today we are releasing the new paper by the Office of the CISO of Google Cloud. In the paper we reveal a new approach for modernizing your compliance approach using modern approaches and Google Cloud toolsets.

What you need to know about Confidential Computing - How Google Cloud uses Confidential VMs and GKE Nodes to encrypt data even when it’s in use.

Private Catalog: Improving Terraform deployment management experiences - With this release, Private Catalog admins can use Terraform configurations to keep end users informed about updates.

How to put your company on a path to successful cloud migration - A new white paper to help you put your company on a path to successful cloud migration.

GKE Workload Identity — A secure way for GKE applications to access GCP services - Using Workload Identity in GKE for secure access.

GKE best practices: Create a cost-optimized cluster in just a few clicks - Follow the prompts in our new GKE cost optimized cluster setup guide to build a cluster based on best practices.

Best reliability designs for Google Kubernetes Engine - Configurations that really add to the reliability level of GKE clusters.

Retrieve recommendation details across all GCP Projects - Getting data from Recommender API.

App Development, Serverless, Databases, DevOps

Where should I run my stuff? Choosing a Google Cloud compute option - Choosing the right infrastructure options to run your application is critical, both for the success of your application and for the team that is managing and developing it. This post breaks down some of the most important factors that you need to consider when deciding where you should run your stuff!

Use Cloud SQL Read Replicas to separate your analytics and production workloads - Learn the steps to use Cloud SQL Read Replicas to separate your analytics and production workloads.

Setting up Firebase token authentication with GCP API Gateway - Using API Gateway to authenticate requests from Firebase to backend.

Using The GCP Firestore Emulator for Local Go Application Development - This article provides a short introduction to using the Cloud Firestore emulator on a local machine to develop Go application.

Calling a private Google Cloud Function from on-prem - How to invoke a Google Cloud Function from on-prem servers in a private way, without exposing it to the internet.

Showcasing dynamic resource management in E2 VMs - In an internal analysis, Google Cloud’s cost-optimized E2 VMs displayed strong, consistent performance, making it a fit for a variety of workloads.

Big Data, Analytics, ML&AI

How we are streaming thousands of rows per second into BigQuery — Part I: Google Cloud Dataflow - Experience of using Cloud Dataflow to feed BigQuery tables.

Bridge data silos with Data Fusion - An overview of Cloud Data Fusion.

BigQuery Admin reference guide: Storage internals - Learn how BigQuery stores your data for optimal analysis, and what levers you can pull to further improve performance.

Query BIG with BigQuery: A cheat sheet - Organizations rely on data warehouses to aggregate data from disparate sources, process it, and make it available for data analysis in support of strategic decision-making. BigQuery is the Google Cloud enterprise data warehouse designed to help organizations to run large scale analytics with ease and quickly unlock actionable insights.

How we Reduced Google BigQuery Cost by 50% - Some practices to reduce BigQuery costs.

Make informed decisions with Google Trends data - Walk through example queries and workflows to blend the newly launched Google Trends public dataset with other actional data.

Kickstart your organization’s ML application development flywheel with the Vertex Feature Store - A Feature Store is a key ingredient for MLOps, helping accelerate development, deployment, and management of production ML applications. Google's new Vertex Feature Store provides a unified solution for ML feature discovery, sharing, and serving at scale.

Scaling deep learning workloads with PyTorch / XLA and Cloud TPU VM - This article addresses challenges associated with scaling deep learning workloads to distributed training jobs that use remote storage. We demonstrate how to stream training data from Cloud Storage to PyTorch / XLA models running on Cloud TPU Pods.

Vertex Matching Engine: Blazing fast and massively scalable nearest neighbor search

Various

Blue-sky thinking: how Sky is reimagining their FinOps journey - How Sky saved millions with Google Cloud and Google FinOps strategies.

Exclusive preview: Google Cloud fireside chat with Dmitri Alperovitch, Founder and Former CTO of CrowdStrike

Crux chose BigQuery for rock-solid, cost-effective data delivery - Crux uses BigQuery as central cloud data warehouse and data hub.

How Cloud Bigtable helps Ravelin detect retail fraud with low latency - Detecting fraud with low latency and accepting payments at scale is made easier thanks to Bigtable.

Grow your ML skills with free offer from Coursera - Google Cloud ML Academy provides free machine learning training and a one month free offer from Coursera.

Slides, Videos, Audio

GCP Podcast - #268 Secure Software Supply Chain with Nikhil Kaul and Victor Szalvay.

Kubernetes Podcast - #155 Software Supply Chain Security, with Priya Wadhwa.

Cloud Security Podcast - #23 - Threat Detection at Google Cloud Security Summit.

The Firebase Podcast - The state of Firebase for the Web.

Releases

AI Platform Training - You can now use an interactive shell to inspect your training container while it runs.

Anthos Config Management - 1.8.1. An issue introduced in 1.8.0 nomos hydrate that breaks support for --clusters has been fixed. An issue that caused Config Sync monitoring Pods fail to start in a cluster with PodSecurityPolicy enabled has been fixed. Cluster selectors and namespace selectors annotations are removed from the result of nomos hydrate so that it can pass nomos vet and can be synced directly to the cluster by Config Sync.

GKE on-prem 1.7 - Anthos clusters on VMware 1.8.1-gke.7 is now available. Fixes: The issue that the etc/cron.daily/aide script uses up all existing space in /run, causing a crashloop in Pods, has been fixed.

AppEngine Flexible - Specifying a user-managed service account for each App Engine version during deployment is now available in preview.

AppEngine Standard - Egress settings are now available for Serverless VPC Access.

Artifact Registry - v1beta2. Artifact Registry now supports Cloud External Key Manager (Cloud EKM) when using customer-managed encryption keys.

BigQuery ML - The end-to-end user journey for BigQuery ML documents an overview of the complete machine-learning flow for each available model including feature preprocessing, model creation, hyperparameter tuning, inference, evaluation, model export, etc.

BigQuery - BigQuery now supports workload management data control language (DCL) statements: CREATE CAPACITY CREATE RESERVATION CREATE ASSIGNMENT DROP CAPACITY DROP RESERVATION DROP ASSIGNMENT This feature is generally available GA. BigQuery now supports the following SQL query operators: PIVOT operator UNPIVOT operator This feature is generally available (GA). BigQuery standard SQL now supports the CONTAINS_SUBSTR function.

CDN - Cloud CDN now treats HTTP responses with a max-age or s-maxage directive as cacheable, even if those responses do not have a Cache-Control: public directive.

Cloud Composer - Cloud Composer 1.16.11 release started on July 22, 2021. Cloud Composer environments with Airflow 2 can run more than one Airflow scheduler. New versions of Cloud Composer images: composer-1.16.11-airflow-1.10.12 composer-1.16.11-airflow-1.10.14 composer-1.16.11-airflow-1.10.15 (default) composer-1.17.0-preview.7-airflow-2.0.2. Airflow 2.0.1 is no longer included in Cloud Composer images.

Compute Engine - Preview: You can use the Help Assistant in the Google Cloud Console to find answers to questions about Compute Engine.

Dataproc Metastore - v1. Avro based imports and exports are now in GA.

Dataproc - Announcing the General Availability (GA) release of Dataproc Enhanced Flexibility Mode. New sub-minor versions of Dataproc images: 1.3.93-debian10, 1.3.93-ubuntu18, 1.4.64-debian10, 1.4.64-ubuntu18, 1.5.39-centos8, 1.5.39-debian10, 1.5.39-ubuntu18, 2.0.13-centos8, 2.0.13-debian10, and 2.0.13-ubuntu18. Upgraded Cloud Storage connector to version 2.2.2 on 2.0 images. Fixed Hue installation on Ubuntu 2.0 images. Fixed an issue on 1.4 and 1.5 images where temporary shuffle data could be leaked when running Enhanced Flexibility Mode (EFM) with Spark.

Datastore - The DATA_READ and DATA_WRITE Data Access audit logs feature has been moved to a future release.

Dialogflow Enterprise - The root CA used for Dialogflow's client certificates for mutual TLS will change to GTS Root R1 in the week of July 26 2021. On July 26, 2021, two new Dialogflow IAM permissions will become effective: dialogflow.changelogs.get and dialogflow.changelogs.list.

Dialogflow - The root CA used for Dialogflow's client certificates for mutual TLS will change to GTS Root R1 in the week of July 26 2021. On July 26, 2021, two new Dialogflow IAM permissions will become effective: dialogflow.changelogs.get and dialogflow.changelogs.list.

Cloud Firestore - The DATA_READ and DATA_WRITE Data Access audit logs feature has been moved to a future release.

IAM - A C++ client library for IAM is now available. You can now set limits on the Cloud Storage roles that a member can grant and revoke.

Google Kubernetes Engine - Google Groups for RBAC is now generally available. (2021-R23) Version updates GKE cluster versions have been updated. Legacy Logging and Monitoring was deprecated December 12, 2019 and was decommissioned March 31, 2021.

GKE - (2021-R23) Version updates The following control plane and node versions are now available: 1.19.12-gke.900 1.19.12-gke.1100 1.20.8-gke.900 The following control plane versions are no longer available: 1.18.17-gke.1900 1.19.9-gke.1400 1.20.6-gke.1000 1.20.6-gke.1400 Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to version 1.18.17-gke.1901 with this release.

Google Kubernetes Engine Rapid - (2021-R23) Version updates Version 1.20.8-gke.700 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - (2021-R23) Version updates Version 1.19.12-gke.1100 is now available in the Regular channel.

Google Kubernetes Engine Stable - (2021-R23) Version updates Version 1.18.19-gke.1701 is now the default version in the Stable channel.

Cloud Memorystore - Added support for Maintenance Windows for Memorystore for Redis.

Network Connectivity Center - Network Connectivity Center now supports VPC Service Controls.

Network Intelligence Center - Connectivity Tests now includes a feature that verifies connectivity to and from Google-managed services, such as Google Kubernetes Engine (GKE) control planes or Cloud SQL instances.

Private Catalog - v1.1. Private Catalog launches improvements for using Terraform, including updating solutions, noting version highlights, and updating deployments.

Cloud Run - Cloud Run is now covered by FedRAMP Moderate.

Secret Manager - Secret Manager now supports using a filter to customize the output of ListSecrets and ListSecretVersions.

Security Command Center - Security Health Analytics, a built-in service of Security Command Center, has launched a new detector, DATASET_CMEK_DISABLED, in general availability. Event Threat Detection, a built-in service of Security Command Center Premium, has launched a public preview of new detectors to protect your Google Workspace domains.

Anthos Service Mesh - 1.7.x & 1.8.x & 1.9.x & 1.10.x. The 1.x version of kpt breaks Anthos Service Mesh installations and upgrades.

Cloud Spanner - Time to live (TTL) is now available in public preview. Granular instance sizing is now available in public preview. Key Visualizer for Cloud Spanner is now available.

Cloud Speech-to-Text - Speech-to-Text has launched a GA version of the Spoken Emoji and Spoken Puncuation features.

Cloud Storage - gcloud alpha storage commands are now available.

Vertex AI - Private endpoints for online prediction are now available in preview. You can now use an interactive shell to inspect your custom training container while it runs.

VPC Service Controls - Preview stage support for the following integration: Network Connectivity Center. Beta stage support for the following integration: Eventarc.

Virtual Private Cloud - External IPv6 addresses for VM instances is now available in General Availability in supported regions.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko