Welcome to issue #376 December 11th, 2023

News

Introducing Cloud SQL Authentication via IAM groups: Simplify database authentication and access at scale - With this feature, you can take advantage of better security, simplify user management and database authentication at scale, and empower database and security administrators to manage database access via familiar IAM-based authentication.

Troubleshoot your network with Connectivity Tests - Connectivity Tests allows you to quickly pinpoint where network connectivity may be broken, and verify if your firewall rules and other network configurations are working as intended.

Enabling next-generation AI workloads: Announcing TPU v5p and AI Hypercomputer - AI Hypercomputer is a groundbreaking supercomputer architecture that employs an integrated system of performance-optimized hardware, open software, leading ML frameworks, and flexible consumption models.

Dynamic Workload Scheduler: Optimizing resource access and economics for AI/ML workloads - Dynamic Workload Scheduler is a resource management and job scheduling platform designed for AI Hypercomputer.

Announcing general availability of Cloud Armor for regional application load balancers - A General Availability of Cloud Armor for Regional External Application Load Balancers, which can help create regionally-scoped Cloud Armor security policies.

Standard Tier Data Transfer now offers a 99.9% availability SLA

Chrome Enterprise 2023: A Year of Innovation Wrapped Up - Chrome Enterprise 2023: A year of innovation wrapped up.

Google is a Leader in Gartner Magic Quadrant for Strategic Cloud Platform Services

Google Cloud Is a Leader in the 2023 Forrester Wave: Streaming Data Platforms

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Simplifying MLOps using Weights & Biases with Google Kubernetes Engine - In this blog, we show you how to use W&B Launch to set up access to either GPUs or Cloud Tensor Processing Units (TPUs) on GKE.

GCVE networking: understanding Google Cloud VPC to Google Cloud VMware Engine connectivity - Exploring how the Google Cloud VPC communicates with the VMware Engine private cloud network.

4 dimensions for driving FinOps adoption - A simple approach consisting of 4 dimensions to help structure and drive Cloud FinOps adoption.

Downgrading Node Pools in Standard GKE Clusters: Restoring to Earlier Kubernetes Versions - This article shows how to perform GKE node pool downgrades if your applications are experiencing issues in the current version or the node pool upgrade has failed.

App Development, Serverless, Databases, DevOps

Python packages via GCP’s Artifact Registry - This blog post shows how to publish and install packages from GCP’s Artifact Factory using pip and poetry.

Cloud Run: The Spring Boot rebirth with GraalVM native compilation - Deploying Spring Boot applications to Cloud Run and evaluation cold start time.

Apollo24|7: Migrating a complex microservices application to Google Cloud with zero downtime - This blog post explains a migration of a critical application used 24x7 in the country, including 97 services and 40+ SQL databases to Google Cloud with zero downtime.

Tracing Firestore Queries: Unlock Insights with Google Cloud Audit Logs and Log Analytics - Comprehensive Guide to Enhancing Firestore Debugging.

Leveraging serverless Google Cloud Platform features for fun and profit - How Telegraph Engineering used the benefits of the serverless paradigm to get rid of technical debt and reduce operational costs.

Big Data, Analytics, ML&AI

Utilizing BigQuery to Analyze Exported Chronicle SIEM Archives - This post explores how to use the Data Export API, and effectively query exported raw logs using SQL statements in GCP BigQuery.

Autoscaling Dataproc for Trino workloads - Autoscaler for Trino on Dataproc solution provides reliable autoscaling for Trino on Dataproc without compromising workload execution.

Using Spark on Dataproc & Apache Iceberg To Build an Open Lakehouse - Using Spark on Dataproc in GCP for reading and writing from a Lakehouse.

Inside the eDreams ODIGEO Data Mesh — a platform engineering view - Read on to learn about how OeDreams (one of the biggest online travel companies in the world) modernized their legacy data warehouse environment to a data mesh built on BigQuery.

Accelerate data-driven growth with Google Cloud and Fivetran - This blog post explores how Google Cloud and Fivetran, a SaaS data integration platform, can help you centralize and analyze your marketing data in real time.

Reduce DBT Incremental Materialization Compute Cost in BigQuery - utilizing partitioned tables and partition pruning to reduce BigQuery cost when using DBT.

How Rittman Analytics Does Web Analytics and Marketing Attribution, using Google BigQuery, Looker, dbt and Segment - Creating a web performance dashboard that brings together key web performance metrics in one place.

Unveiling Data Skewness: A Practical Guide to Distribution Analysis with BigQuery and Looker Studio - This article presents an approach to address the limitations of averages by examining the distribution of data in Google Analytics.

More Ways to Create Incremental Tables in Dataform - The article considers Dataform’s built-in capabilities for creating incremental tables and some ways to improve them.

Turn customer feedback into opportunities using generative AI in BigQuery DataFrames - This blog post goes through an example of building a solution for transforming raw customer feedback into actionable intelligence.

Built-with Google AI: Meet Vivien, EPAM’s digital assistant with the power of Google Cloud gen AI - Using the power of generative AI, Vivien, a state-of-the-art Digital Human, is built on the powerful combination of Unreal Engine and Vertex AI.

Fine-tune and deploy an LLM on Google Colab Notebook with QLoRA and VertexAI - An example of fine-tuning and deploying MistralAI 7B model using QLoRA on your data and VertexAI endpoint, in Google Colab Notebook .

Vertex AI Grounding Large Language Models - Grounding allows Google’s large Language models to use your specific data to produce more accurate and relevant responses.

Unleash Vertex AI Power from ABAP: Effortless AI Integration - This article demonstrates the remarkable ease of calling a Vertex AI Foundation Model directly from ABAP.

Various

Announcing the </Code> with Maps Hackathon Winners

How to become certified Google Cloud Professional DevOps Engineer - Passing DevOps GCP certification.

Passing 10x GCP certifications: A return on experience (Part 1: How) - This article will try to give insights on how to study for and pass GCP certifications.

Slides, Videos, Audio

Kubernetes Podcast - #214 KubeCon NA 2023.

Security Podcast - #151 Cyber Insurance in the Cloud Era: Balancing Protection, Data and Risks.

Releases

Advisory Notifications - Advisory Notifications now sends mandatory security and privacy notifications for users using Google Cloud without an organization.

Anthos clusters on VMware - The StatefulSet CSI Migration Tool is now available.

AppEngine Flexible - .NET - .NET 8 is now available in preview.

AppEngine Flexible Java - Java 21 is now available in preview.

Google Cloud Armor - DDoS attack visibility is now Generally Available.

Bare Metal Solution - You can now set up encryption keys and SSH keys for your Bare Metal Solution server while provisioning or reimaging it. You can now select the pod for your Bare Metal Solution resources through the Google Cloud console intake form.

BigQuery ML - The following BigQuery ML data preprocessing features are now in preview: The ML.TRANSFORM function, which you can use to preprocess feature data.

Chronicle - Chronicle Curated Detections has been enhanced with new detection content for Google Cloud threats. Chronicle now has an additional mechanism to set up the ingestion of Google Workspace Activities logs (WORKSPACE_ACTIVITY).

Confidential VM - Confidential Space. You can now use custom attestation tokens to authenticate a workload to relying parties outside of Google Cloud. A new image (confidential-space-231200) is now available.

Compute Engine - Preview: Managed instance groups (MIGs) let you create pools of suspended and stopped virtual machine (VM) instances. Generally available: The following location and scale enhancements for Persistent Disk Asynchronous Replication are generally available: Larger disk capacity: the maximum disk size has increased from 2 TiB to 5 TiB.

Config Connector - Config Connector version 1.112.0 is now available. Added support for AlloyDBUser (v1beta1) resource. Added support for EdgeContainerCluster (v1beta1) and EdgeContainerNodePool (v1beta1) resources. Added support for EdgeNetworkNetwork (v1beta1) and EdgeNetworkSubnet (v1beta1) resources. Resource BigtableAppProfile(v1beta1): Added spec.standardIsolation field. Fixed the SecretKeyRef in the Go client.

Container Registry - mirror.gcr.io is hosted on Artifact Registry.

Data Fusion - Cloud Data Fusion versions 6.5 and 6.6 are no longer supported.

Dataflow - You can now archive completed Dataflow jobs. The Dataflow web-based monitoring interface now includes a dashboard that monitors your Dataflow jobs at the project level.

Dataproc Serverless - Announcing the Preview release of Dataproc Serverless for Spark 2.2 runtime: Spark 3.5.0 BigQuery Spark Connector 0.34.0 Cloud Storage Connector 3.0.0-RC1 Conda 23.10 Java 17 Python 3.12 R 4.3 Scala 2.13.

Dataproc - Added the Confidential Computing option on the "Manage Security" panel on the "Create a Dataproc cluster on Compute Engine" page in the Google Cloud console. New Dataproc on Compute Engine subminor image versions: 2.0.85-debian10, 2.0.85-rocky8, 2.0.85-ubuntu18 2.1.33-debian11, 2.1.33-rocky8, 2.1.33-ubuntu20, 2.1.33-ubuntu20-arm 2.2.0-RC3-debian12. Updated the Zookeeper component version from 3.8.0 to 3.8.3 in the latest Dataproc on Compute Engine 2.1 image version. Fixed Dataproc Hub issue in latest Dataproc on Compute Engine 2.1 image. Backported HIVE-21698 in Hive 3.1.3 component in latest Dataproc on Compute Engine image versions.

Cloud Deploy - Google Cloud Deploy is now available in the following regions: europe-southwest1 (Madrid) europe-west8 (Milan) europe-west9 (Paris) me-west1 (Israel) us-east5 (Columbus) us-south1 (Dallas).

Dialogflow - The Speech-to-Text API used by Dialogflow now supports two new speech models for the en and en-us language tags: telephony and telephony_short.

Cloud Domains - You can now turn off automatic renewal after you've registered your domain.

Cloud Functions - Cloud Functions (2nd gen) now supports the Java 21 runtime at the Preview release level. Cloud Functions (2nd gen) now supports the .NET 8 runtime at the Preview release level.

Google Kubernetes Engine - This is an update to the release note regarding the Dataplane V2 issue published on September 07, 2023. (2023-R25) Version updates GKE cluster versions have been updated.

GKE - (2023-R25) Version updates The following control plane and node versions are now available: 1.24.17-gke.2347000 1.25.16-gke.1020000 1.26.10-gke.1235000 1.27.7-gke.1293000 1.28.3-gke.1203001 1.28.3-gke.1286000 The following control plane versions are no longer available: 1.24.15-gke.1700 1.24.17-gke.2113000 1.24.17-gke.2155000 1.24.17-gke.2230000 1.25.11-gke.1700 1.25.14-gke.1421000 1.25.14-gke.1474000 1.25.15-gke.1083000 1.26.5-gke.2700 1.26.9-gke.1437000 1.26.9-gke.1507000 1.26.10-gke.1073000 1.27.2-gke.2100 1.27.6-gke.1248000 1.27.6-gke.1445000 1.27.7-gke.1088000 Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.23 to version 1.24.16-gke.500 with this release.

Google Kubernetes Engine Rapid - (2023-R25) Version updates Version 1.28.3-gke.1203001 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - (2023-R25) Version updates The following versions are now available in the Regular channel: 1.24.17-gke.2266000 1.25.15-gke.1115000 1.26.10-gke.1101000 1.27.7-gke.1121000 1.28.3-gke.1203001 The following versions are no longer available in the Regular channel: 1.24.16-gke.500 1.25.12-gke.500 1.26.7-gke.500 1.27.4-gke.900 Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.23 to version 1.24.17-gke.200 with this release.

Google Kubernetes Engine Stable - (2023-R25) Version updates The following versions are now available in the Stable channel: 1.24.17-gke.200 1.25.13-gke.200 1.26.8-gke.200 1.27.5-gke.200 The following versions are no longer available in the Stable channel: 1.24.15-gke.1700 1.26.5-gke.2700 Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.23 to version 1.24.16-gke.500 with this release.

Cloud Logging - You can now query for a specific error group in the Logs Explorer and Log Analytics pages by using the error group ID.

Media CDN - The Media CDN capability to use a private S3-compatible bucket as an origin is now Generally Available.

Cloud Monitoring - You can now create a broken-link checker, which periodically validates the links contained in your website.

Cloud Router - Cloud Router support for IPv6 BGP sessions is in Public Preview.

Security Command Center - New goal-based query presets for identity and access misconfigurations New goal-based query presets on the Security Command Center Vulnerabilities page are released to Preview. Policy Controller integration released to General Availability The integration of Policy Controller for Kubernetes clusters with Security Command Center is released to General Availability.

Service Mesh - Managed Anthos Service Mesh. Managed Anthos Service Mesh 1.18 is rolling out in the rapid channel. If you use Gateway API Automated Deployment, note the following breaking change.

Cloud Spanner - Cloud Spanner now supports the following PostgreSQL functions: unnest array_length array(subquery) date_trunc extract spanner.date_bin spanner.timestamptz_add spanner.timestamptz_subtract For more information, see working with arrays in PostgreSQL-dialect databases.

Cloud SQL MySQL - You can now upgrade your Cloud SQL instances to use the new network architecture to get additional capabilities not available in the old network architecture. You can now configure Cloud SQL for MySQL instances for IAM group authentication. The rollout of the following MySQL versions is currently underway: MySQL 5.7.43 is upgraded to MySQL 5.7.44.

Cloud SQL Postgres - You can now upgrade your Cloud SQL instances to use the new network architecture to get additional capabilities not available in the old network architecture.

Cloud Storage - Cloud Storage FUSE now provides the ability to configure log rotation. The restrict unencrypted HTTP requests organization policy constraint is now generally available (GA).

Cloud TPU - Cloud TPU now supports TensorFlow 2.14.1.

Vertex AI - Version @002 of the models for text, chat, code, and code chat are available. Version 2 of the stable version of the Codey code completion foundation model, named code-gecko@002, is available. Grounding with Vertex AI Search Model grounding is available in (Preview).

Cloud Vision API - Improved models are now available for the following features: Text detection and documentation text detection (OCR) Web detection Logo detection Object localization Specify "builtin/latest" in the model field of a Feature object to use the new models.

VPC Service Controls - The ability to allow access to protected resources from an internal IP address is available in Preview.