Welcome to issue #313 September 26th, 2022

News

Enabling real-time AI with Streaming Ingestion in Vertex AI - Starting this month, Vertex AI Matching Engine and Feature Store will support real-time Streaming Ingestion as Preview features.

Google Cloud Firewall introduces Network Firewall Policies, IAM-governed Tags and more - Google Cloud’s new Network Firewall Policies and IAM-governed Tags create a more secure, reliable, and scalable configuration while pursuing Zero Trust architecture.

Reduce operational complexity and costs with a central load balancer - Reduce operational complexity and costs with Cloud Load Balancing’s new cross-project service referencing feature. This feature allows you to create one central load balancer and route traffic across projects.

Container analysis support for Maven and Go Automatic Scanning of Containers in Public Preview - Customers can now list existing vulnerabilities in Maven and Go packages automatically via Container Scanning upon upload to Artifact Registry.

Introducing Custom Organization Policy for GKE to harden security - Google Cloud’s preview of the new Custom Organization Policy for GKE can help create guardrails to harden security and boost efficiency.

Extending Anthos to manage on-premises edge VMs: now generally available - VM support in Anthos extends Anthos on bare metal (Google Distributed Cloud Virtual) to run and manage both containers and VMs on a single, unified platform in the data center or edge.

New Google Maps Platform launch stages and what they mean for you - Starting today, all new Google Maps Platform offerings will launch in Experimental, Preview or General Availability.

Google launches dedicated cloud training program for Ukrainians - Google launches a new, dedicated cloud training program to support Ukrainian businesses and IT professionals, starting October 4, 2022.

Join Google Workspace at Cloud Next ’22 - Join business leaders, IT experts, and Google executives to discover what comes next in the world of collaboration and the future of work.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Security Roundup - stories and launches from second quarter 2022 - What's new in the Google Cloud Security world, highlighting and summarizing some key stories and new content.

View policy enforcement metrics for ACM Policy Controller - Policy Controller enables the enforcement of programmable policies for Anthos clusters. We have made it easier to export and visualize these metrics.

Schedule Connectivity Tests for continuous networking reachability diagnostics - Automating Connectivity Tests for continuous cloud networking diagnostics.

Deploy OCI artifacts and Helm charts the GitOps way with Config Sync - Getting started with deploying your OCI artifacts and Helm charts the GitOps way with Config Sync.

Nuts and bolts of NEGs (Network Endpoint Groups) in GCP - An overview of Network Endpoint Groups.

Gitlab and Workload Identity Federation on Google Cloud - This article explains how to set Workload Identity Federation to securely consume Google Cloud APIs from your Gitlab CI pipeline, for example for pushing a Docker container image to Artifact Registry.

Jobs and Deployment Strategies on Google Kubernetes Engine - An example of deploying a job on Kubernetes cluster with a canary and Blue/Green deployment strategies.

App Development, Serverless, Databases, DevOps

Cloud Wisdom Weekly: 3 ways serverless can save money and accelerate app development - How serverless compute platforms like Google Cloud’s Cloud Run and Cloud Functions can reduce costs and improve developer productivity.

Multi-environment service orchestrations - This blog provides three examples of how to implement multi-environment workflows.

Handle Firebase User Roles without a Firestore Collection - Using Custom claims in Firebase to provide access control.

Cloud Deploy with Cloud Run - An example of deploying Cloud Run application with the Cloud Deploy.

Google Firebase with DotNet 6 - An overview of Firebase with some examples in C#.

Big Data, Analytics, ML&AI

What’s New with Google’s Unified, Open and Intelligent Data Cloud - Google’s unified, open and intelligent data cloud provides insights at every level of the enterprise to empower leaders to drive results.

Benchmarking your Dataflow jobs for performance, cost and capacity planning - Learn how to test your own Dataflow pipelines for performance optimization, capacity planning and TCO estimation using open-source PerfKit Benchmarker.

Meet Optimus, Gojek’s open-source cloud data transformation tool - Gojek’s open-source Optimus is workflow orchestrator for data transformation, data modeling, data pipelines, and data quality management.

Exporting and analyzing billing data using BigQuery - Looking for more details on your cloud spending? Export your billing data to BigQuery!

Building trust in the data with Dataplex - American Eagle® and Aerie® is a leading global specialty retailer offering high-quality clothing using Dataplex for logistics/delivery partners.

Built with BigQuery: Retailers drive profitable growth with SoundCommerce - SoundCommerce uses Analytics Hub to increase the pace of innovation by sharing datasets with its customers in real-time by using the streaming functionality of BigQuery.

Simplify streaming ELT pipeline using BigQuery Subscriptions with Pub/Sub - Example of streaming data to BigQuery directly with Pub/Sub and NodeJS code.

Replicate RDS PostgreSQL to BigQuery using Datastream CDC - A workflow setup to migrate data from AWS RDS Postgress to BigQuery via Datastream.

BigQuery SQL Functions For Data Cleaning - A few use cases and functions to apply to clean data in BigQuery.

Quickly restore BigQuery dataset with time travel and Cloud Workflows - You just made a mistake in BigQuery and you want to restore the previous state? Keep calm, you have seven days and a Workflows solution!

Perform real time anomaly detection using Google Cloud’s Timeseries Insights API — Part I - Overview of an easy to use API to scale billions of timeseries with low latency anomaly detection and forecasting.

Building a Machine Learning Platform with Kubeflow and Ray on Google Kubernetes Engine - How Kubeflow and Ray can be deployed together on Google Kubernetes Engine to provide a production-ready ML system.

Google Vertex AI Model Versioning - Google released support for versioning models earlier this year. However, the official documentation on how to use it is still very….

Various

Jason Wellman is bullish on Cloud’s ability to transform healthcare - here’s why - Google Cloud’s Jason Wellman has watched cloud computing evolve over his past 15 years at Google. Hear why he’s bullish on Cloud’s ability to transform healthcare, creating resiliency and a stronger patient experience.

10/10 GCP Certs the CTS Way - How company is enabling its' employees to get certifications.

7 Google Certification preparation mistakes to avoid. - Sharing mistakes from a personal experience of taking GCP Certification exams.

Slides, Videos, Audio

Listen up: Meet our Multicloud Mindset series on Twitter Spaces - We’re excited to introduce our Multicloud Mindset series, featuring engaging, live conversations about multicloud topics that takes place on Twitter Spaces.

Kubernetes Podcast - #189 Ambient Mesh, with Justin Pettit and Ethan Jackson.

Security Podcast - #84 EP84 How to Secure Artificial Intelligence (AI): Threats, Approaches, Lessons So Far.

GCP Podcast - #320 Vertex AI Experiments with Ivan Nardini and Karthik Ramachandran.

Releases

Google Cloud Armor - Google Cloud Armor Threat Intelligence (Threat Intel) is generally available.

Batch - The Quotas and limits are now documented for Batch.

BigQuery - BigQuery Omni has introduced support for on-demand pricing model (GA) for a limited duration. You can now view shuffle usage ratios in the admin resource charts.

BigTable - A weekly digest of client library updates from across the Cloud SDK.

CDN - Cloud CDN support for dynamic compression is now Generally Available.

Cloud Composer - Cloud Composer 1.19.10 and 2.0.27 release started on September 21, 2022. The Monitoring Dashboard of an environment now displays periods of maintenance operations and other environment operations such as creating snapshots and updating environment configuration. Improved file synchronization performance in Airflow workloads. (Available without upgrading) Fixed a problem that caused package 'namespecifier' could not be found errors when installing PyPI packages in private IP environments. The apache-airflow-providers-google package was upgraded to 2022.9.6+composer. Cloud Composer 1.19.10 and 2.0.27 images are available: composer-1.19.10-airflow-1.10.15 (default) composer-1.19.10-airflow-2.1.4 composer-1.19.10-airflow-2.2.5 composer-2.0.27-airflow-2.1.4 composer-2.0.27-airflow-2.2.5. Cloud Composer versions 1.17.0, and 2.0.0-preview.1 have reached their end of full support period.

Compute Engine - Generally available: View the VM placement topology information to determine how close a VM is located in relation to another VM. Generally available: Reduce licensing costs by customizing the number of visible CPU cores. Generally Available: E2 shared-core custom VMs are now generally available. The quota limits displayed in the Cloud console might be incorrect in the me-west1 region.

Dataproc Metastore - Metadata federation now supports BigQuery datasets as a metadata source (in preview).

Dataproc Serverless - Dataproc Serverless for Spark now uses runtime version 1.0.18 and 2.0.0-RC3. Dataproc Serverless for Spark: You can now use the spark.dynamicAllocation.executorAllocationRatio property to configure how aggressively to scale up Serverless workloads. Dataproc Serverless for Spark: Reduced the latency between batch workload completion and when a batch is marked SUCCEEDED. Dataproc Serverless for Spark: Increased initial and maximum Spark executor limits to 500 and 2,000, respectively. Dataproc Serverless for Spark: Sets a maximum limit of 500 workers per scale up or scale down operation. Dataproc on Compute Engine: Stop all master and worker VMs when starting a cluster fails due to stockout or insufficient quota.

Deep Learning VM - M96 Release TensorFlow 2.10.0 is now available.

Cloud Deploy - Google Cloud Deploy now provides the ability to verify your deployment, supported in preview.

IAM - Conceptual and reference information for IAM basic and predefined roles has been improved.

Networking Interconnect - Dedicated Interconnect support is available in the following colocation facilities: Telecom Italia Rozzano Data Center, Milan Bulk Oslo Internet Exchange - OS-IX, Oslo For more information, see the Locations table.

Google Kubernetes Engine - (2022-R23) Version updates GKE cluster versions have been updated. 2022-09-22 update: Updated new default versions for the 2022-R22 release in the Stable channel. The a2-ultragpu machine family is available in Preview for node pools in clusters running GKE version 1.24 and later.

GKE - (2022-R23) Version updates Version 1.22.12-gke.2300 is now the default version.

Google Kubernetes Engine Rapid - (2022-R23) Version updates Version 1.24.3-gke.2100 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - (2022-R23) Version updates Version 1.22.12-gke.2300 is now the default version in the Regular channel.

Google Kubernetes Engine Stable - (2022-R23) Version updates Version 1.22.12-gke.1200 is now the default version in the Stable channel. 2022-09-22 update: Updated new default versions for the 2022-R22 release in the Stable channel.

Load Balancing - Regional internal and external HTTP(S) load balancers now support Shared VPC configurations where the load balancer's forwarding rule, target proxy, and URL map, can be created in a host or service project, while the backend services and backends can be distributed across multiple service projects in the Shared VPC environment.

Cloud Logging - The Cloud Logging API now supports the following region: Israel: me-west1.

Migrate for Compute Engine - #243583591:Fixed an issue with Yum configuration affecting RHEL migrations from Azure to Google Cloud.

StratoZone - Added ability to generate a Container Fit Assessment report. Added VMware Horizon workload detection. Increased timeout of HSTS HTTP response header policy. StratoProbe 5.0.4.1 - Added alternative method for collection of netstat data as netstat is being deprecated on Debian distros. Enhanced detection of Citrix workloads. Fixed an issue where the manual import fails if a VM has >50 IP addresses assigned. Fixed an issue where the Windows OS cost was not being calculated correctly for E2-micro/small/medium instances. Fixed an issue where RAM right-sizing values could be negative in some cases. Fixed an issue where out-of-scope Database instances were not being excluded in reports. Fixed an issue where a user could select an invalid filter combination impacting Sole Tenant pricing. StratoProbe 5.0.4.1 - Fixed an issue where Download Results was not producing a CSV in the IP Range Scan. StratoProbe 5.0.4.1 - Fixed an issue with memory utilization collection on older CentOs distributions.

Cloud Interconnect - Dedicated Interconnect support is available in the following colocation facilities: Telecom Italia Rozzano Data Center, Milan Bulk Oslo Internet Exchange - OS-IX, Oslo For more information, see the Locations table.

Cloud Router - You can now use the Google Cloud Console to configure MD5 authentication for BGP sessions.

Cloud PubSub - Pub/Sub introduces new monitoring dashboards for topics and subscriptions which you can access from the Topics and Subscriptions console pages.

Retail Recommendations AI - Recommendations AI now provides a Page-Level Optimization model.

Cloud Run - You can now configure an HTTP liveness healthcheck probe.

Secret Manager - A weekly digest of client library updates from across the Cloud SDK.

Service Mesh - 1.14.x. 1.14.4-asm.0 is now available. 1.13.x. 1.13.8-asm.1 is now available.

Anthos Service Mesh - 1.14.x. 1.14.4-asm.0 is now available. 1.13.x. 1.13.8-asm.1 is now available.

Cloud SQL - Cloud SQL allows the re-use of an instance name immediately after the instance is deleted.

Traffic Director - Traffic Director now supports outlier detection for proxyless service mesh deployments. The Traffic Director service routing APIs are now in General Availability.

Vertex AI - Vertex AI Matching Engine Vertex AI Matching Engine now offers Preview support for updating your indices using Streaming Update, which is real-time indexing for the Approximate Nearest Neighbor (ANN) service. The M96 release of Vertex AI Workbench managed notebooks includes the following: Fixed a problem where users were not able to save large Notebooks.

VMware Engine - Dell PowerScale is now available for in-guest file share access for Google Cloud VMWare Engine VMs: NFS and SMB shares supported PowerScale file shares may be accessed across Google Cloud VMWare Engine and other Google Cloud services Private connection configured via the VMware Engine UI Learn more about PowerScale for Google Cloud VMWare Engine.

VPC Service Controls - General availability for the following integration: Analytics Hub.

Virtual Private Cloud - VPC Service Controls ingress and egress rules are no longer required to establish Private Service Connect connections from inside a VPC Service Controls perimeter.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko