Welcome to issue #304 July 25th, 2022

News

A new Google Cloud region is coming to Mexico - The new Google Cloud region in Mexico will be the third in Latin America, joining Chile and Brazil, and bringing the total of regions and zones to 34 and 103.

The next generation of Dataflow: Dataflow Prime, Dataflow Go, and Dataflow ML - Dataflow is GCP’s Cloud Native way for all data processing workloads, powered by the universal batch and streaming model of Apache Beam.

Streamline data management and governance with the unification of Data Catalog and Dataplex - Data Catalog will be unified with Dataplex, providing an enterprise-ready data fabric that enables data management and governance at scale.

Standing shoulder to shoulder - building a resilient healthcare ecosystem with Health-ISAC - Google Cloud has joined the Health Information Sharing and Analysis Center (Health-ISAC) as its first Ambassador Partner in the cloud.

Google Workspace earns DOD IL4 authorization - Google Workspace has achieved the U.S. Department of Defense’s (DOD) Impact Level 4 (IL4) authorization.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

How to overcome 5 common SecOps challenges - Here are 5 common issues that many SecOps teams struggle with—and how to fix them.

Data security in Google Cloud - Data security is a huge part of an organization's security posture. Encryption is a core control for data security, and Google Cloud offers multiple encryption options for data at-rest, in-transit, and even in-use.

Quick Access: Intelligence behind the Google Clouds new homepage - Quick Access intelligent shortcuts simplify Google Cloud console navigation by predicting the destinations you will most likely want to open next.

Google supports CSRB call for open source security improvements in wake of log4j report - Google offers an open source security roadmap to industry that aligns with a new report from CISA’s Cyber Safety Review Board on the log4j vulnerabilities.

Deploying high-throughput workloads on GKE Autopilot with the Scale-Out compute class - GKE Autopilot now offers compute classes for running containerized workloads on specialized compute platforms such as the Arm architecture.

Using Google Kubernetes Engine’s GPU sharing to search for neutrinos - Native support for GPU time sharing and A100 Multi-Instance GPU partitioning allowed many more IceCube ray-tracing simulations from the same hardware.

Setting up NATS Streaming Server on GKE: a Technical Guide - A technical guide to setting up NATS on Google Kubernetes Engine.

Using kOps with GCE and NVMe Scratch Disks - How to enable kOps support for adding NVMe scratch disks to GCE instances.

Using a host project to simplify your Google Cloud network - An example of using host project to provide centralized network resources, firewall rules, configurations etc.

Stop using Network Appliances in Google Cloud - Explanation and using VPC Firewalls.

DNS on GKE: Everything you need to know - Everything you always wanted to know about DNS on GKE but you were too afraid to ask. This article for you.

App Development, Serverless, Databases, DevOps

Using Cloud Bigtable with IAM Conditions and Tags - Learn about limiting team members' access to Bigtable resources including more advanced techniques like conditional permissions.

SAP backup, the blended way - How to implement cost-efficient backups with low RTO and RPO for SAP.

Using Pacemaker for SAP high availability on Google Cloud - Part 1 - This blog introduces some basic terminology and concepts about the Red Hat and SUSE HA implementation of Pacemaker cluster software for SAP HANA and NetWeaver.

Access Google’s AlloyDB with pgAmdin, psql, and pgbench from a private IP - A walk through the steps to set up an AlloyDB cluster and access the Postgres database from a GCE instance with a private IP.

4 Steps to Automate deployment of a static website with GCP and Firebase - Website Deployment Automation workflow.

Single Sign On with GCP Identity Platform/Identity Providers and Okta using SAML Standard - Implementation of Single Sign on (SSO) with SAML standard using Okta as IDP and Identity Platform service which uses Firebase.

How to Setup a sFTP Server in Google Cloud Platform and Restrict Access - This article explains steps necessary to properly setup a sFTP Server in Google Compute Engine, so that a user can send files and only have access to a specific folder.

Protecting your Application on Cloud Run with API Gateway and Identity Aware Proxy - Learn how to protect your Cloud Run applications with IAP and API Gateway.

Translate Any Retro Game on the fly with Google Cloud AI and Go - Using the Google Cloud Vision and Google Translate APIs to translate which translates anything on screen to your preferred language.

Big Data, Analytics, ML&AI

New 20+ pipeline operators for BQML - We describe the new BigQuery and BigQuery ML (BQML) components now available for Vertex AI Pipelines, enabling data scientists and ML engineers to orchestrate and automate any BigQuery and BigQuery ML functions. We also showed an end-to-end example of using the components for demand forecasting involving BigQuery ML and Vertex AI Pipelines.

The Diversity Annual Report is now a BigQuery public dataset - Google’s 2022 Diversity Annual Report is now available as a BigQuery public dataset, making it easy for researchers and community groups to pull, analyze, and share diversity data.

Scalable Python on BigQuery using Dask and NVIDIA GPUs - To accelerate data analytics and machine learning workflows, we introduce the Dask BigQuery connector to read data through BigQuery storage API and deploy at scale on NVIDIA GPUs using Dask and RAPIDS on Google Dataproc.

Data Contracts — The Mesh glue - A practical definition and implementation guidelines.

How to use variables in BigQuery using SQL — Part 1 - A step towards flexibility and reusability using parameters and variables.

How to Set Up dbt, Google BigQuery, and Github(2022) - A step-by-step walkthrough to get you up and running.

Pivot and Unpivot Functions in BigQuery For Better Data Manipulation - A detailed tutorial.

Machine Learning Batch Prediction Architecture Using Vertex AI - Batch prediction architecture implemented with Vertex AI.

Various

Google Cloud Data Heroes Series: Meet Tomi, a data engineer based in Germany and creator of the ‘Not So BigQuery Newsletter’ - In the Data Heroes series we share stories of people who use data analytics tools to do incredible things. In this month’s edition, Meet Tomi.

The Invisible Cloud: How this Googler keeps the internet moving worldwide - Meet Stacey Cline and hear how she came to enable the worldwide movement of Google Cloud’s global technical infrastructure.

Slides, Videos, Audio

GCP Podcast - #312 Managed Service for Prometheus with Lee Yanco and Ashish Kumar.

Kubernetes Podcast - #186 Gateway API Beta, with Rob Scott.

Security Podcast - #75 How We Scale Detection and Response at Google: Automation, Metrics, Toil.

Releases

Anthos Config Management - 1.12.1. The constraint template library includes a new template: K8sRequireCosNodeImage. Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: c370036). Fixed the resource name length validation issue caused by long RepoSync names or long namespace names.

Anthos clusters on VMware - Anthos clusters on VMware 1.9.7-gke.8 is now available. Fixed a known issue in which the cluster backup feature affected the inclusion of always-on secrets encryption keys in the backup.

AppEngine Standard Java - Updated the Java SDK to version 1.9.98.

AppEngine Standard PHP7 - The App Engine legacy bundled services for PHP 7+ are now available at the General Availability release level.

BigQuery - Analytics Hub is now available in additional regions across the Americas, Asia Pacific, and Europe.

Billing - Secure the link between a project and its billing account In the Cloud Billing Console, you can now lock the link between a project and its Cloud Billing account, in order to prevent accidental changes to the billing state, such as disabling billing or moving the project to a different billing account.

Chronicle - The default parsers have changed.

Cloud Composer - Cloud Composer 1.19.4 and 2.0.21 release started on July 18, 2022. (Cloud Composer 2) Fixed a problem where an environment creation in the PSC configuration might fail with the "Composer backend timed out" message. Cloud Composer 1.19.4 and 2.0.21 images are available: composer-1.19.4-airflow-1.10.15 (default) composer-1.19.4-airflow-2.1.4 composer-1.19.4-airflow-2.2.5 composer-2.0.21-airflow-2.1.4 composer-2.0.21-airflow-2.2.5. Cloud Composer versions 1.16.10 and 1.17.0.preview.6 have reached their end of full support period.

Compute Engine - Generally available: NVIDIA® T4 GPUs are now available in the following additional regions and zones: Montréal, Québec, North America : northamerica-northeast1-c For more information about using GPUs on Compute Engine, see GPU platforms.

Data Catalog - Data Catalog is now a part of Dataplex to provide a complete data management and governance experience with built-in data intelligence and automation capabilities.

Dataflow - Dataflow Prime is now in General Availability.

Dataproc Metastore - Dataproc Metastore is available in the following regions: us-west2 (Los Angeles), us-west3 (Salt Lake City), europe-west4 (Netherlands), europe-west6 (Zürich), and asia-east1 (Taiwan).

Datastore - Time-to-live (TTL) policies now available in Preview.

Cloud Data Loss Prevention - A new detection model is available for the PERSON_NAME infoType detector.

Cloud Firestore - Time-to-live (TTL) policies now available in Preview.

Google Kubernetes Engine - GKE Gateway integration with Cloud Certificate Manager is now available as Public Preview in GKE versions 1.20 and later. If you start a credential rotation or an IP address rotation, ensure that you manually complete the rotation. Kubernetes control plane metrics are now Generally Available. (2022-R17) Version updates GKE cluster versions have been updated.

Google Kubernetes Engine Rapid - (2022-R17) Version updates The following versions are now available in the Rapid channel: 1.21.14-gke.700 1.22.11-gke.400 1.23.8-gke.400 1.24.2-gke.300 The following versions are no longer available in the Rapid channel: 1.21.12-gke.2200 1.22.9-gke.1500 1.23.6-gke.1501 1.24.1-gke.1400 Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to version 1.21.13-gke.900 with this release.

Google Kubernetes Engine Regular - (2022-R17) Version updates The following versions are now available in the Regular channel: 1.20.15-gke.9900 1.21.13-gke.900 1.22.10-gke.600 The following versions are no longer available in the Regular channel: 1.20.15-gke.8700 1.21.12-gke.1700 Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to version 1.21.12-gke.2200 with this release.

Google Kubernetes Engine Stable - (2022-R17) Version updates The following versions are now available in the Stable channel: 1.20.15-gke.8700 1.21.12-gke.2200 1.23.6-gke.2200 The following versions are no longer available in the Stable channel: 1.20.15-gke.8200 Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.20 to version 1.21.12-gke.1500 with this release.

Cloud Logging - You can now search your correlated log entries in the Logs Explorer.

Cloud Monitoring - A new version of Managed Service for Prometheus is now available.

reCAPTCHA Enterprise - You can now find legacy secret keys for all reCAPTCHA Enterprise keys in the Google Cloud console.

Cloud Run - Cloud Run now supports container images in the Open Container Initiative (OCI) image format.

Security Command Center - The container and kubernetes attributes were added to the Finding object. Virtual Machine Threat Detection, a built-in service of Security Command Center Premium, is generally available (GA).

Anthos Service Mesh - 1.14.x. 1.14.1-asm.3 is now available. Anthos Service Mesh allows you to configure the minimum TLS version for your Istio workloads. Managed Anthos Service Mesh isn't rolling out to the rapid release channel at this time. 1.11.x. Anthos Service Mesh 1.11 is no longer supported. 1.12.x. 1.12.8-asm.2 is now available. 1.13.x. 1.13.5-asm.1 is now available.

Cloud Storage Transfer - Detailed logging for objects copied between AWS S3, Azure Blob Storage, ADLS Gen 2, and Cloud Storage with Storage Transfer Service is now generally available (GA).

Vertex AI - NFS support for custom training is GA.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko