Welcome to issue #390 March 18th, 2024

News

Save up to 40 percent with Dataflow streaming committed use discounts - Today, we are announcing the general availability of Dataflow streaming committed use discounts (CUDs), providing a new way for you to save money on a key driver of your streaming costs: streaming compute.

Introducing Security Command Center Enterprise: The first multicloud risk management solution fusing AI-powered SecOps with cloud security - To help our customers manage and mitigate risk in their multicloud environments, Google Cloud is announcing Security Command Center Enterprise, the industry’s first cloud risk management solution that fuses proactive cloud security and enterprise security operations — supercharged by Mandiant expertise.

Optimize costs for Windows workloads using Persistent Disk Async Replication - This blog post introduces Persistent Disk Async Replication and its key features.

Streamlining data integration with SQL Server source support in Datastream - Datastream, Google Cloud's fully managed change data capture (CDC) service, has expanded its capabilities to include support for SQL Server sources.

Announcing SQL Server Reporting Services (SSRS) in Google Cloud SQL - With Google Cloud SQL for SQL Server, our fully managed relational database service to run SQL Server workloads, you can now build and run SSRS reports with databases hosted on Cloud SQL for SQL Server (including the report server database required to set up and run SSRS).

What’s new with Google Cloud network observability partner ecosystem - Today, we’re excited to introduce a number of new network observability solutions and feature enhancements from our partners, as well as two new partners with customized solutions for GKE network observability: Selector and Tigera.

Announcing Cloud HPC Toolkit blueprint for AI/ML with the NeMo Framework on A3 VMs - The Cloud HPC Toolkit is a Google product that helps simplify the creation and management of HPC systems for AI/ML and traditional HPC workloads.

A window into protein folding: Lowering the barriers for AlphaFold Inferencing - Google Cloud introduces AlphaFold Portal, protein modeling for beginners to make Vertex AI AlphaFold Inference Pipeline easier to use.

Healthcare's AI transformation is already underway

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Cloud CISO Perspectives: Easing the psychological burden of leadership

How to set up Google Cloud VMware Engine regional disaster recovery with VMware Site Recovery Manager - In this blog post, we present a guide to setting up SRM within GCVE, enabling failover and failback of your VMs between Google Cloud regions for DR purposes.

Google Cloud Architecture Framework : Performance Optimization

Design your Landing Zone — Design Considerations Part 2 — Kubernetes and GKE - This blog post covers considerations related to running Kubernetes in your landing zone.

Secure authentication on GCP with Workload Identity Federation - This article describes how to configure GitLab CI/CD pipelines to securely access Google Cloud Platform (GCP) resources using Workload Identity Federation (WIF).

Connect Google Cloud Build with a private GKE cluster - Enable your GCB triggers to talk to private GKE cluster.

Use Google Managed Prometheus and Triton Inference Server on GKE to Simplify LLM observability and monitoring - This blog post explains a method for monitoring large language models (LLMs) deployed on Google Kubernetes Engine (GKE).

Secrets Management: Using External Secret Operator for Google Secret Manager on GKE - ESO (External Secret Operator ) — is a Kubernetes operator that integrates external secret management systems, in this blog post, an example using a Secret Manager in GKE is provided.

Properly Running Kubernetes Jobs with Sidecars in 2024 (K8s 1.28+) - This article introduces the easiest way to run Jobs with sidecars using the latest Kubernetes features.

App Development, Serverless, Databases, DevOps

AlloyDB fuels 50% faster margin calculations for Apex FinTech - Apex FinTech Solutions Inc. ("Apex") enables modern investing and wealth management tools through an ecosystem of frictionless platforms. To transform this traditionally on-premises process, the company migrated to AlloyDB for PostgreSQL to enable real-time decision-making and risk management for clients and investors.

TPG Telecom reduces API delivery time by 50% with Apigee

Chess.com boosts performance, cuts response times by 71% with Cloud SQL Enterprise Plus

Jib-Gradle-Plugin, Cloud Build: Containerize & Store Effortlessly - Example of building Spring Boot application with Jib and Gradle in Cloud Build.

Integrate Open Telemetry with Cloud Ops in a Linux Environment - A tutorial about using Otel on Linux for Google Cloud.

Troubleshoot query latency changes using SPANNER_SYS.QUERY_PROFILES_TOP_* tables - One of the best tools in Cloud Spanner to understand why query latency changed is Query plan samples. Query plan samples allow you to view samples of historic query plans and compare the performance of a query over time.

Unlocking Secrets: Navigating Cloud Run’s Secret Access Methods - In Cloud Run, there are three distinct methods for accessing secrets. This article will delve into how these methods can be implemented using Python and deployed via YAML files, examining their respective advantages and limitations.

How to make Cloud Run talk to Cloud Run - The private way - This blog post covers the Cloud Run networking options and details how to make a source Cloud Run service access a destination Cloud Run service using a private network.

Big Data, Analytics, ML&AI

How Livesport activates data and saves engineering resources with BigQuery and Dataddo

Efficient BigQuery Data Modeling: A Storage and Compute Comparison - BigQuery storage and compute comparison for normalized, denormalized, and nested design: an in-depth analysis with actionable optimizations.

Build a security analytics data warehouse with BigQuery and Redpanda - In this tutorial, you’ll integrate Google BigQuery with Redpanda (a performant C++ drop-in for Apache Kafka) to build a simple security analytics data warehouse.

How we expose data in BigQuery - Transitioning to a self-service data platform.

Dive deeper into Gemini with BigQuery and Vertex AI - In this blog we will dive deeper into some of the recent innovations for Vertex AI and BigQuery and show you how to use Gemini 1.0 Pro in BigQuery.

Getting Started with Document AI: Introduction, Processors & Evaluation Metrics - Document AI turns unstructured content into structured data making it easier to understand, analyze, and consume. It extracts & classifies information from unstructured documents.

Transforming data science with Vertex AI: Telepass journey into MLOps - This article is about Telepass’ journey in implementing MLOps: from initial challenges to resulting architecture.

How FibroGen achieved 40x ROI by automating invoice processing

Generative AI in insurance: How Cytora helps insurers prioritize underwriting risks - In this article, we'll explore how Cytora (a startup that helps insurers transform their workflows with AI) leveraged Google Cloud's generative AI foundation models to deliver cutting-edge services to their clients.

Various

Maximize your cloud potential at Next ‘24: Lead with learning at any skill level

Join our first Security Talks of 2024: Trends, gen AI security, and augmenting defenses

Slides, Videos, Audio

Security Podcast - #163 Cloud Security Megatrends: Myths, Realities, Contentious Debates and Of Course AI.

Releases

Access Approval - Access Approval supports Google Distributed Cloud Edge in the GA stage.

AlloyDB - The Back up and restore AlloyDB Omni documentation is updated to include pgBackRest, an open-source solution included with the AlloyDB Omni Docker container. AlloyDB now supports continuous backup and recovery, and scheduled backups on secondary clusters. AlloyDB Language Connectors are now generally available (GA).

AppEngine Standard - You can't use the latest version of dev_appserver.py to locally run your applications for runtimes that reached end of support.

Artifact Registry - Artifact Registry remote repositories support basic authentication to user-defined and preset upstream sources for Docker, Maven, npm, and Python formats. Images copied to Artifact Registry from Container Registry with the automatic migration tool are failing to propagate their upload time to Artifact Registry, and instead have their upload time value set to zero, resulting in an upload time of early 1970.

Billing - Tags data for Google Cloud Storage buckets is available in both the Standard usage cost export and the Detailed usage cost export.

Carbon Footprint - Emissions reported for Google Kubernetes Engine (GKE) declined starting in February 2024.

Certificate Manager - Certificate Manager supports integration with regional external Application Load Balancers and regional internal Application Load Balancers.

Chronicle - Chronicle has expanded Cloud Threat Detections to create a detection when findings from Security Command Center Event Threat Detections, Cloud Armor, Sensitive Actions Service, and Custom modules for Event Threat Detection are identified. The following supported default parsers have changed. Forwarder troubleshooting guide is now available to help you diagnose and resolve common issues that may arise while using the Chronicle Linux forwarder.

Key Access Justifications - Access Approval supports Google Distributed Cloud Edge in the GA stage.

Access Transparency - Access Transparency supports the following services in the GA stage: Google Distributed Cloud Edge IAM workforce identity pools.

Cloud Composer - In Airflow 2.6.3 offered in Cloud Composer versions earlier than 2.6.2, task statuses can be deleted as a result of the #31179 Airflow issue. All preview versions of Cloud Composer 2 are past their security notifications end date and are not supported. If you see frequent scheduler restarts in your Airflow 2.6.3 environment and the [scheduler]job_heartbeat_sec Airflow configuration option is set to a non-default value, you can fix this issue either by upgrading to Cloud Composer version 2.6.4 or by removing this option's override, so that it uses the default value. Fixed creation and upgrades in environments that have environment variables with special symbols. Cloud Composer 2.6.4 images are available: composer-2.6.4-airflow-2.6.3 (default) composer-2.6.4-airflow-2.5.3. Cloud Composer versions 2.1.8 and 1.20.8 have reached their end of full support period.

Compute Engine - Generally available: You can use SSH-in-browser to connect to TPU VMs. Generally available: You can scale a single VM into a managed instance group (MIG), which is a group of VMs that you can manage as a single entity. Generally available: Hyperdisk Balanced is available with C3 and H3 VMs.

Container Registry - New tooling is available to upgrade from Container Registry to Artifact Registry.

Data Fusion - The Cloud Data Fusion version 6.9.2.3 patch revision is generally available (GA).

Dataflow - You can now use worker utilization hints to tune horizontal autoscaling for streaming pipelines. Added new autoscaling metrics: Autoscaling rationale chart: explains the factors driving autoscaling decisions Worker CPU utilization chart: shows current user worker CPU utilization and customer autoscaling hint value Timer backlog per stage: shows an estimate of time needed to materialize the output for windows whose timer has expired Parallel processing: the number of keys available for parallel processing. You can now use committed use discounts (CUDs) with Dataflow streaming jobs.

Dataproc Serverless - New Dataproc Serverless for Spark runtime versions: 1.1.54 2.0.62 2.1.41 2.2.0-RC14. Added the bigframes (BigQuery DataFrames) Python package in the Dataproc Serverless for Spark 2.1 runtime.

Dataproc - New Dataproc on Compute Engine subminor image versions: 2.0.95-debian10, 2.0.95-rocky8, 2.0.95-ubuntu18 2.1.43-debian11, 2.1.43-rocky8, 2.1.43-ubuntu20, 2.1.43-ubuntu20-arm 2.2.9-debian12, 2.2.9-rocky9, 2.2.9-ubuntu22. New Dataproc Serverless for Spark runtime versions: 1.1.54 2.0.62 2.1.41 2.2.0-RC14. Added the bigframes (BigQuery DataFrames) Python package in the Dataproc Serverless for Spark 2.1 runtime.

Cloud Deploy - Cloud Deploy support for deploy automation is now generally available.

Dialogflow - From March 18 to April 15, 2024 (new dates for the same migration announced earlier), for certain language tag and speech setting combinations, the Dialogflow CX and Dialogflow ES traffic with audio will gradually route away from the classic Speech-to-Text models behind the command_and_search, default, phone_call, and video model identifiers to the new conformer-based speech models.

Error Reporting - Error Reporting can now analyze logs routed by project sinks to different projects than the source project.

IAM - You can use the iam.serviceAccountKeyExposure organization policy constraint to help manage leaked service account credentials.

Google Kubernetes Engine - The following vulnerabilities were discovered in the Linux kernel that can lead to a privilege escalation on Container-Optimized OS and Ubuntu nodes: CVE-2024-1085 For more information, see the GCP-2024-018 security bulletin. A previous version of the GKE logging agent that rolled out in GKE version 1.28.7-gke.1100000 contained a security vulnerability. Private clusters created on GKE versions 1.29.0-gke.1384000 and later use Private Service Connect (PSC) for nodes to privately communicate with the control plane. Secret Manager add-on for GKE is now available. Opportunistic bursting and lower Pod minimums are now available on newly created GKE Autopilot clusters at version 1.29.2-gke.1060000 or later, and on existing clusters created at 1.26 or later that have been fully upgraded (including all nodes) to 1.29.2-gke.1060000 or later.

Load Balancing - The global external Proxy Network Load Balancer is implemented on globally distributed GFEs and supports advanced traffic management capabilities. Regional external Application Load Balancers and regional internal Application Load Balancers now support Certificate Manager certificates.

Cloud Logging - You can now use SQL JOIN and UNION operators on the Log Analytics page to combine tables in multiple Google Cloud projects. A weekly digest of client library updates from across the Cloud SDK.

Resource Manager - You can add tags at the time of creating folders and projects.

Retail Recommendations AI - Vertex AI Search for retail: Renamed in the console and documentation The Google Cloud console has been updated to show the current product name for Vertex AI Search for retail.

Cloud Run - Direct VPC egress now supports Cloud NAT with Public NAT IP addresses (in Preview).

Service Mesh - Managed Anthos Service Mesh. The rollout of managed Anthos Service Mesh version 1.17 to the stable channel has completed. 1.20.x. 1.20.4-asm.0 is now available for in-cluster Anthos Service Mesh. 1.19.x. 1.19.8-asm.2 is now available for in-cluster Anthos Service Mesh. 1.18.x. 1.18.7-asm.11 is now available for in-cluster Anthos Service Mesh. Managed Anthos Service Mesh. There is a known issue where new installations of Managed Anthos Service Mesh in the rapid channel on GKE Autopilot clusters may fail.

Cloud Spanner - Table renaming is now generally available.

Cloud SQL SQL Server - A new maintenance version rollout is currently underway for all supported SQL Server versions. Cloud SQL now supports SQL Server Reporting Services (SSRS) on your instances.

Cloud Storage Transfer - Support for transfers from cloud and on-premises Hadoop Distributed File System (HDFS) sources is now generally available (GA).

Cloud Storage - You can now view granular bucket-level cost data in the Cloud Billing Detailed data export.

Cloud TPU - Cloud TPU now supports TensorFlow 2.16.1.

VMware Engine - Google Cloud VMware Engine now leverages Cloud Logging to provide status updates about hardware health and VMware management components.