Welcome to issue #375 December 4th, 2023

News

12 days of no-cost training to learn generative AI this December

What’s new with Filestore: Enhancing your stateful workloads on GKE - Three new features, which have all recently become generally available to improve integration of Filestore and GKE.

Nobl9's Reliability AI, Powered by Google - Customers who want to leverage AI technology in Google Cloud to define and understand SLOs can now do so through Vertex AI, thanks to Nobl9 and the new tool they developed, SLOgpt.ai.

Introducing sample GenAI Databases Retrieval App – augment your LLMs with Google Cloud databases

Build, Hack and Learn at the 2023 Looker Hackathon

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE - With Cloud TPUs on Google Kubernetes Engine (GKE), the leading Kubernetes service in the industry, customers can orchestrate AI workloads efficiently and cost effectively with best-in-class training and inference capabilities.

Securing Google Cloud Resources at Scale using Tags - Manage access to Google Cloud resources at scale using the power of Tags.

Exporting GCP Projects to Terraform

Cloud CISO Perspectives: How security validations can help organizations stay secure - This month, Mandiant Consulting’s Earl Matthews discusses Security Validation, a vital tool that can give CISOs better information for making security decisions, and can help organizations understand their true security posture and risk profile.

Developer-friendly networking that keeps up with your development velocity - This blog discusses design considerations for managing network connectivity through categorized VPCs and how they can be used to offer a developer centric networking while maintaining security.

Installing Apigee Hybrid with Helm: A Kubernetes-native approach - A new way to install Apigee Hybrid using the Helm package manager.

Mapping Your As-Is to Google Cloud, To Leverage Cloud Value - Google Cloud Adoption Series: Part 4B: As-Is to Cloud — Infrastructure.

App Development, Serverless, Databases, DevOps

How DORA DevOps best practices helped Circles launch a telco-as-a-service in under two months

Migrating Terraform resources to Cloud Run API v2 - This blog post describes the new Terraform resource for Cloud Run, and describes the process of migrating an example Cloud Run service to this new resource.

Saving costs while improving KPIs with AMD EPYC - Improving Redis cluster performance and cost savings.

Driving success through open communication - Distilling years of Google research into five dimensions that you can apply to drive success within your own organization.

Gen AI apps: Deploy LangChain on Cloud Run with LangServe - This blog post shows how to get started with LangServe and deploy a template to Cloud Run that calls the VertexAI PaLM 2 for chat model.

Deploying CloudRun application with custom domain using Cloudflare - This blog explains how to deploy an application using a custom domain on Cloud Run which acts as a backend to an HTTP(S) Classic Load Balancer.

Authentication for Multi-Regional Cloud Run Deployments with Custom Audiences - Learn how to use custom audiences for Cloud Run to authenticate clients in a multi-regional deployment where the service URI is unknown.

Cloud SQL for PostgreSQL: Faster similarity search performance with pgvector indexes - This post explains pgvector indexes, clarify different configurations, and give hands-on coding examples for improving the performance and viability of a pgvector-based application using HNSW indexes.

Deploy and manage Kubernetes applications with Workflows

Session leak detection solutions: Debug non-responsive applications on Cloud Spanner

Resolve and optimize many-to-many relationships in Spanner - The best practices to detection session leaks when using Cloud Spanner.

Not Everything Is Google’s Fault (Just Most Things) - (Not so good) experience with GCP.

Docker Hub Remote Repositories in GCP - Creating Docker Hub remote repositories in Google Cloud Platform.

Big Data, Analytics, ML&AI

A guide to RAID multiple Local SSDs & mount it to Dataproc - A guide to RAID multiple Local SSDs & mount it to Dataproc.

Data-driven decisions with YugabyteDB and BigQuery - YugabyteDB, a distributed SQL database, when combined with BigQuery, tackles data fragmentation, data integration, and scalability issues businesses face.

Built with BigQuery: LiveRamp’s open approach to optimizing customer experiences

Cost management in BigQuery: how to control spending with budgets and custom quotas - Managing BigQuery costs with budgets and custom quotas.

Media CDN Custom Dashboard using BigQuery and Looker Studio - This document will guide you through the steps to deploy a Media CDN custom Dashboard in the Google Cloud Console using Log Sink, BigQuery and Looker Studio.

Multi Instance Deployment with Looker - Using different environments in Looker.

The New Generative AI Function in BigQuery - How to use BigQuery GENERATE_TEXT remote function.

How Delivery Hero connected GitHub with Vertex AI to manage 20+ voucher fraud detection models - MLOps setup in Delivery Hero.

How generative AI is transforming the customer service experience

Ask your documents: Document AI and PaLM2 for question answering - Building "Ask your documents" tool for employees by leveraging Google Cloud Document AI, text embedding models, and PaLM 2.

How to launch an MLFlow server with Continuous Deployment on GCP in minutes - A step-by-step guide to deploying an MLFlow server using GitHub Actions and Cloud Run.

Google Generative AI Evaluation Service - A service to evaluate the performance of Generative AI Models using metrics like BLEU or ROUGE among others.

Dealing with delayed data using dbt on BigQuery - This article shows how to implement a partitioned based incremental approach for dbt backed by BigQuery.

Powering a GraphQL API over your BigQuery dataset via Hasura - Using Hasura, an open-source solution, to create an API around BigQuery dataset.

Slides, Videos, Audio

Kubernetes Podcast - #213 Kubernetes Pen Testing, with Jesper Larsson.

Security Podcast - #150 EP150 Taming the AI Beast: Threat Modeling for Modern AI Systems with Gary McGraw.

GCP Life Podcast - #54 “They sacked themselves!“ – In this episode we discuss; Google Next, Udemy & Google, Manifest v3, Broadcom & VMWARE, Google AI Search, Memory Store For Redis Cluster, Google Workspace Hack, Atomic Stealer, New CPU Vulnerability, Open AI, Spotify and Google AI.

Releases

AlloyDB - The AlloyDB columnar engine now supports columns with the array data type.

Anthos clusters on bare metal - 1.16. Release 1.16.3 Anthos clusters on bare metal 1.16.3 is now available for download. Functionality changes: Increased the certificate time to live (TTL) for metrics-providers-ca and stackdriver-prometheus-scrape for third-party monitoring. Supported node pool versions: If you use selective worker node pool upgrades to upgrade a cluster to version 1.16.3, the following versions are supported for the worker node pools: 1.16.3 1.16.2 1.16.1 1.16.0 1.15.6 1.15.5 1.15.4 1.15.3 1.15.2 1.15.1 1.15.0. Fixes: Fixed an issue where CoreDNS Pods can get stuck in an unready state. Fixes: The following container image security vulnerabilities have been fixed in 1.16.3: Critical container vulnerabilities: CVE-2023-38408 CVE-2023-45871 High-severity container vulnerabilities: CVE-2023-1989 CVE-2023-4244 CVE-2023-4622 CVE-2023-4623 CVE-2023-4921 CVE-2023-42753 Medium-severity container vulnerabilities: CVE-2023-3772 CVE-2023-37453 CVE-2023-39189 CVE-2023-39192 CVE-2023-39193 CVE-2023-39194 CVE-2023-42754 CVE-2023-42755 Low-severity container vulnerabilities: CVE-2023-3773 CVE-2023-6176 CVE-2023-42756. Known issues: For information about the latest known issues, see Anthos clusters on bare metal known issues in the Troubleshooting section.

Apigee X - On December 1, 2023, we released an updated version of Apigee (1-11-0-apigee-8). Configure a forward proxy at the environment level using forwardProxyUri. Dynamic endpoint target metrics aggregated into a single metric. Bug ID Description 294882858 Fixed issue with ServiceCallout policy overriding target_ip value in proxy.

Cloud Asset Inventory - The following resource types are now publicly available through the ExportAssets, ListAssets, BatchGetAssetsHistory, QueryAssets, Feed, and Search (SearchAllResources, SearchAllIamPolicies) APIs. The following resource types are now publicly available through the ExportAssets, ListAssets, BatchGetAssetsHistory, QueryAssets, Feed, and Search (SearchAllResources, SearchAllIamPolicies) APIs.

Batch - Documentation has been added to explain how to configure jobs that can run on reserved VMs.

BigQuery - You can use configuration YAML files to transform SQL code when you translate SQL queries from your source database. The slot estimator now supports project level cost-optimal commitment and autoscale recommendations for on-demand workloads.

Chronicle - Supported default parsers have changed, click on the link for more details.

Cloud Composer - Cloud Composer 2.5.2 release started on November 30, 2023. (Available without upgrading) The dags list-import-errors Airflow CLI command is now supported. Improved the validation of zone values. The apache-airflow-providers-google package is upgraded to version 10.11.1 in images with Airflow 2.6.3 and 2.5.3. The apache-airflow-providers-cncf-kubernetes package was upgraded to version 7.9.0. Cloud Composer 2.5.2 images are available: composer-2.5.2-airflow-2.5.3 composer-2.5.2-airflow-2.6.3 (default). Cloud Composer versions 2.0.32 and 1.19.15 have reached their end of full support period. You can specify a preferred Cloud SQL zone for the environment's database when creating a standard resilience environment. Quotas for snapshot operations are increased.

Database Migration Service - Database Migration Service supports creating Cloud SQL for MySQL and Cloud SQL for PostgreSQL instances with customer-managed encryption keys (CMEK) enabled.

Dataproc Serverless - Dataproc Serverless runtime image tracks have upgraded the Cloud Storage connector to version 2.2.18.

Dataproc - New Spark runtime versions: 1.1.40, 2.0.48, 2.1.27.

Dialogflow - Dialogflow CX answer feedback is now generally available (GA) and has new configurations in Dialogflow Messenger, conversation history, and BigQuery export.

Google Kubernetes Engine - Starting in GKE version 1.27.6-gke.1248000, clusters in Autopilot mode detect nodes that can't fit all DaemonSets and, over time, migrate workloads to larger nodes that can fit all DaemonSets. The following GKE versions fix an issue that could cause the NVIDIA GPU driver installer image to be garbage collected on Container-optimized OS nodes: 1.25.15-gke.1040000 and later 1.26.10-gke.1030000 and later 1.27.6-gke.1513000 and later 1.28.3-gke.1061000 and later. Starting in GKE 1.27.7, you can configure your workloads to use TPU reservations with node auto-provisioning.

Cloud Logging - Ops Agent version 2.44.0 introduces the following features: Support for Compute Engine VMs that are running Ubuntu 23.10 (Mantic Minotaur). You can now specify an inclusion filter or exclusion filter that is applied to the _Default sinks of new resources.

Cloud Monitoring - Ops Agent version 2.44.0 introduces the following features: Support for Compute Engine VMs that are running Ubuntu 23.10 (Mantic Minotaur).

Service Mesh - Managed Anthos Service Mesh. The rollout of managed Anthos Service Mesh version 1.17 to the rapid channel has completed. If you use Gateway API Automated Deployment, note the following upcoming change.

SAP Solutions - Google Cloud's Agent for SAP version 2.8 Version 2.8 of Google Cloud's Agent for SAP is generally available (GA).

Cloud Storage Transfer - Storage Transfer Service supports transfers from cloud and on-premises Hadoop Distributed File System (HDFS) sources.

Cloud Text-to-Speech - Cloud Text-to-Speech now offers de-DE and fr-FR Studio voices: de-DE-Studio-B, de-DE-Studio-C, fr-FR-Studio-A, and fr-FR-Studio-D.

Vertex AI - The following Vertex AI Model Garden updates are available: Updated default model deployment settings with L4 GPUs, such as LLaMA2, falcon-instruct, openllama, Stable Diffusion 1.5, 2.1, and XL models. The Unicorn model size for PaLM 2 for Text is generally available (GA). Vertex AI's integration of model and dataset metadata into Dataplex's Data Catalog service is now generally available (GA).