Welcome to issue #383 January 29th, 2024

News

Monitoring for every runtime: Managed Service for Prometheus now works with Cloud Run

Announcing general availability of Custom Org Policy to help tailor resource guardrails with confidence

---

Give your people seamless, secure access to all the apps they need to do their jobs - from anywhere and on any device - with Cameyo’s Virtual App Delivery (VAD) platform and Google Cloud. Download the white paper here to learn more.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

4 ways to reduce cold start latency on Google Kubernetes Engine - In this post, we’ll discuss four different techniques to reduce cold start latency on GKE, so you can deliver responsive services.

IDC study: Customers cite 407% ROI with Chronicle Security Operations

Cloud CISO Perspectives: How new SEC rules can help business leaders

Cloud Armor with Google Cloud GKE - An overview of Cloud Armor on GKE.

Google Cloud quotas — with Terraform - Gain full control over your Google Cloud costs by specifying quotas on your resource usage.

How to reduce your Google Cloud Compute Engine Bill by 50% with Committed Use Discounts — Part 2 - This article explains how to size, purchase, and analyze the effectiveness of Committed Use Discounts.

App Development, Serverless, Databases, DevOps

Spanner integration testing with the emulator - This post demonstrates how to set up integration testing for Spanner using GitHub Actions and the Spanner emulator.

Backup & Restore Neo4j Graph Database via GKE Cronjob and Google Cloud Storage - Automated, cloud-centric solution for the backup and restoration of Neo4j databases using GKE and Cloud Storage.

5 ways platform engineers can help developers create winning APIs - How can platform engineers influence API development?

How to build a gen AI application - UX considerations for generative AI apps and agents.

High availability with Memorystore for Redis Cluster, Part 1: Four nines - Exploring how the Memorystore for Redis Cluster architecture helps achieve its' 99.99% availability.

Best practices for migrating auto-incrementing keys to Spanner - This post shows how to migrate schemas and data from other databases to Spanner, minimizing changes to downstream applications and ensuring Spanner best practices.

Seamless Migration: From managed Cloud Run domains to Google Cloud Load Balancer - Migrating Cloud Run application to Cloud Load Balancer.

Create and deploy a new web app to Cloud Run with Duet AI - A journey of creating and deploying a new web application to Cloud Run with Duet AI’s help.

Big Data, Analytics, ML&AI

How Vodafone puts customers first with an environment built on data intelligence

Real-time data processing for machine learning with Striim and BigQuery - Integrating Striim with BigQuery ML for real-time data processing in machine learning.

How (and why) To Add Clustering To Your Tables In Bigquery (2024) - A part 2 of in-depth series on maximizing efficiency in Google BigQuery.

How to Use the Google Trends Open Dataset on BigQuery - Example of accessing Google Trends from public datasets in BigQuery.

Unleashing BigQuery Power with Spring Boot Starter - Spring sample configurations for BigQuery.

Vertex AI and BigQuery for Natural Language Exploration of GCP Billing Data - This article is your guide to unleashing Google’s new Gemini Pro model on your Google Cloud billing data that has been exported to BigQuery.

How to break down your query costs in BigQuery - Using information schema in BigQuery to breakdown cost of the executed queries.

Mastering Feature Preprocessing in BigQuery ML: A Comprehensive Guide - BigQuery ML’s Impact on Data Analytics.

Implementing SCD Type 2 Data Acquisition Pipelines to BigQuery Using GCP Datastream & dbt - This article explores a practical approach to building lowly Changing Dimensions (SCD) Type 2 data acquisition pipelines from multiple external PostgreSQL databases to Google BigQuery using GCP Datastream and dbt.

GCP Data Governance: Column Level Security Best Practices — Taxonomies, Data Class, Policies, and IAM Roles - Implementing policy tags in BigQuery for a robust data governance.

RLHF Tuning with Vertex AI - Vertex AI offers a pipeline template that encapsulates the RLHF algorithm.

Cost savings in VertexAI Notebooks using Terraform - Implement 2 auto-shutdown cost control features as part of your IaC.

Pixel guessing : using Gemini Pro Vision with Go - Let’s have fun with the vision powers of AI!

Machine Learning Pipeline Development on Google Cloud - Pipeline development best practices and field experience from Google Cloud Consulting. Part 2 of our series on MLOps.

Navigating Google Cloud’s Vertex AI Auto SxS - A Technical Deep Dive - An innovative tool for AI model evaluation.

How Apollo 24|7 leverages MedLM with RAG to revolutionize healthcare

Building a Clinical Intelligence Engine using MedLM augmented Clinical Knowledge Graphs - Designing an expert clinical assistant with a deep understanding of clinical knowledge base.

Slides, Videos, Audio

Kubernetes Podcast - #217 Cilium and eBPF, with Bill Mulligan.

Security Podcast - #156 Living Off the Land and Attacking Critical Infrastructure: Mandiant Incident Deep Dive.

Releases

AlloyDB - AlloyDB Language Connectors are now available in Preview. AlloyDB support for Cloud regions in the events timeline is now Generally available. AlloyDB now supports cross-project restoration.

Anthos Config Management - 1.17.1. The constraint template library includes a new template: K8sDisallowInteractiveTTY. Policy Controller bundles have been updated to the following versions: cis-k8s-v1.5.1: 202312.1, cost-reliability-v2023: 202312.0, nist-sp-800-190: 202312.1, nist-sp-800-53-r5: 202312.1, nsa-cisa-k8s-v1.2: 202312.1, pci-dss-v3.2.1: 202312.1, psp-v2022: 202312.0. Upgraded bundled Helm version from v3.13.1 to v3.13.3 to pick up vulnerability fixes. Upgraded bundled Kustomize version from v5.1.1 to v5.3.0 to pick up vulnerability fixes. Fixed a race condition that could cause deadlock when uninstalling Config Sync. Fixed an issue that could cause RootSyncs and RepoSyncs to be reported as Current before they had been reconciled. Fixed an issue where the log level could not be set for the otel-agent container. Fixed an issue where RepoSync Secrets were not garbage collected when switching between Secret refs or types. Fixed a performance issue where the config-management-operator was continuously updating the webhook definition, causing unnecessary churn in the apiserver and etcd.

Anthos clusters on VMware - Google Distributed Cloud Virtual for VMware 1.28.100-gke.131 is now available. The following issues are fixed in 1.28.100-gke.131: Fixed an issue where duplicate Service IP addresses caused the Seesaw load balancer to fail. Google Distributed Cloud Virtual for VMware 1.16.5-gke.28 is now available. The following issues are fixed in 1.16.5-gke.28: Fixed an issue where duplicate Service IP addresses caused the Seesaw load balancer to fail.

Apigee X - On January 22, 2023, we released an updated version of Apigee (1-11-0-apigee-14). Bug ID Description 316093865 Fixed issue where empty LoadBalancer configuration in the Target Endpoint results in a failed proxy deployment with NullPointerException.

AppEngine Standard - The option to update a Serverless VPC Access connector is now available at general availability (GA).

Google Cloud Armor - The following features are now Generally Available: Parsing of the GraphQL content-type Support for User IP request headers Support for JA3 fingerprints For more information about parsing GraphQL content, see Apply parsing on custom Content-Type header values.

Cloud Asset Inventory - The following resource types are now publicly available through the ExportAssets, ListAssets, BatchGetAssetsHistory, QueryAssets, Feed, and Search (SearchAllResources, SearchAllIamPolicies) APIs. The following resource types are now publicly available through the analyze policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning).

Batch - Jobs that try to consume reserved VMs might be incorrectly delayed or prevented from running.

BigQuery ML - BigQuery now natively supports the Delta Lake format for Amazon S3 and Azure tables. BigQuery ML has added a new residual column to the output of the ML.EXPLAIN_FORECAST function for ARIMA_PLUS and ARIMA_PLUS_XREG models. To improve BigQuery ML training performance, the APPROX_GLOBAL_FEATURE_CONTRIB argument now defaults to TRUE when you set the ENABLE_GLOBAL_EXPLAIN argument to TRUE, and you set the NUM_PARALLEL_TREE argument to greater than 10 for boosted tree models or greater than 50 for random forest models. BigQuery now supports the ST_LINEINTERPOLATEPOINT geography function, which gets a point at a specific fraction in a linestring. A weekly digest of client library updates from across the Cloud SDK. BigQuery is now available in the Berlin (europe-west10) region.

BigQuery - BigQuery now natively supports the Delta Lake format for Amazon S3 and Azure tables. BigQuery ML has added a new residual column to the output of the ML.EXPLAIN_FORECAST function for ARIMA_PLUS and ARIMA_PLUS_XREG models. To improve BigQuery ML training performance, the APPROX_GLOBAL_FEATURE_CONTRIB argument now defaults to TRUE when you set the ENABLE_GLOBAL_EXPLAIN argument to TRUE, and you set the NUM_PARALLEL_TREE argument to greater than 10 for boosted tree models or greater than 50 for random forest models. BigQuery now supports the ST_LINEINTERPOLATEPOINT geography function, which gets a point at a specific fraction in a linestring. A weekly digest of client library updates from across the Cloud SDK. BigQuery is now available in the Berlin (europe-west10) region.

Billing - Recommendations for Compute Engine Flexible committed use discounts are now Generally Available Flexible CUDs add flexibility to your spending capabilities by eliminating the need to restrict your commitments to a single project, region, or machine series. Subscription IDs for your committed use discounts are now available in the Detailed cost data export Note: The new Detailed cost data export column has started rolling out to customers.

Cloud Build - Cloud Build is now available in the following regions: europe-west10 europe-west12 me-central1 me-central2 For more information, see Cloud Build locations.

Chronicle - The Detection Engine added support for event variable joins on or expressions and function calls. Chronicle Curated Detections has been enhanced with new detection content for Linux Threats.

Cloud Composer - Cloud Composer 2.5.5 release started on January 25, 2024. Increased the amount of memory available to the Redis component in environments with Medium and Large environment sizes. Sensitive Airflow configuration options are now hidden in Airflow UI by default. (Available without upgrading) In VPC Service Controls setups, device policy attributes in access levels are now respected when accessing the Airflow web server. (Available without upgrading) Auxiliary Cloud Pub/Sub topics and subscriptions used by save and load snapshot operations are now cleaned up if these operations fail. Cloud Composer versions 2.1.4, 2.1.3, 1.20.4, and 1.20.3 have reached their end of full support period.

Compute Engine - Generally available: Hyperdisk Balanced is available with M3 VMs. Generally available: In the Google Cloud console, in the Observability tab on the VM instances page, you can customize the predefined dashboard to monitor specific VM metrics that you want.

Data Fusion - Cloud Data Fusion is available in the following regions: europe-southwest1 europe-west8 europe-west9 europe-west12 me-central1 southamerica-west1 For more information, see Pricing. Cloud Data Fusion supports patch revision upgrades in the Google Cloud console.

Dataproc Serverless - New Dataproc Serverless for Spark runtime versions: 1.1.47 2.0.55 2.1.34 2.2.0-RC7.

Dialogflow - Vertex AI Conversation data stores now support unstructured CSV. With data store agents, you can now provide a custom summarization prompt. Dialogflow CX generators has added support for text-bison@002 and gemini-pro models. The Dialogflow CX name collection prebuilt component removed use of the deprecated system entities @sys.given-name and @sys.last-name. Speech model migration announced eariler will include two additional language tags: en-in and nl.

Cloud Domains - On September 7, 2023, Squarespace acquired all domain registrations and related customer accounts from Google Domains. Consequently, some Cloud Domains features are deprecated and removed. For more information, see Deprecations and shut down features and Cloud Domains FAQ.

Eventarc - Eventarc support for creating triggers for direct events from Cloud Firestore with authentication context, is available in Preview.

Cloud Functions - The option to update a Serverless VPC Access connector is now available at the General Availability release level.

Google Kubernetes Engine - (2024-R02) Version updates GKE cluster versions have been updated. Clusters that are running GPUs and are upgraded from 1.26 to a 1.27 patch version earlier than 1.27.8 might experience issues with their nodes' GPU device plugins (nvidia-gpu-device-plugin). The following vulnerability was discovered in the Linux kernel that can lead to a privilege escalation on Container-Optimized OS and Ubuntu nodes: CVE-2023-6817 For instructions and more details, see the GCP-2024-004 security bulletin. We have identified several clusters where users have granted Kubernetes privileges to the system:authenticated group, which includes all users with a Google account.

Load Balancing - External passthrough Network Load balancers now support zonal NEGs with GCE_VM_IP endpoints.

Cloud Monitoring - You can now create and manage your uptime checks and synthetic monitors by using the Google Cloud CLI.

Cloud PubSub - BigQuery subscriptions now support the ability to parse JSON messages by using the schema of a BigQuery table.

Cloud Run - The option to update a Serverless VPC Access connector is now available at general availability (GA).

Security Command Center - Security Command Center Management API released to General Availability The Security Command Center Management API, which provides API support for managing settings and custom modules, is released to General Availability. New Container Threat Detection service account deferred The new service account for Container Threat Detection that was included with new activations of Security Command Center after December 7, 2023 was temporarily removed from new activations on Dec 19, 2023 due to issues with older GKE clusters.

Cloud Spanner - Cloud Spanner now supports the GoogleSQL INSERT OR IGNORE and INSERT OR UPDATE clauses. Cloud Spanner now supports COSINE_DISTANCE() and EUCLIDEAN_DISTANCE() functions (in Preview).

Cloud SQL Postgres - Cloud SQL Enterprise Plus edition now supports the following regions: asia-northeast2 (Osaka) asia-south2 (Delhi) europe-north1 (Finland) europe-southwest1 (Madrid) us-east5 (Columbus) us-south1 (Dallas).

Cloud Storage - You can now work with the Object Retention Lock feature within the Cloud Console.