Welcome to issue #428 December 9th, 2024

News

¡Hola Mexico! Google Cloud region in Querétaro now open - Google Cloud has opened its 41st cloud region in Querétaro, Mexico, offering fast, reliable cloud services to businesses and public sector organizations.

Registration is open for Google Cloud Next 2025 - Google Cloud Next 2025, the annual flagship conference, is set to take place in Las Vegas from April 9-11, 2025. With early bird pricing available until February 14th, attendees can register now for an unforgettable week of hands-on experiences, inspiring content, and networking opportunities with industry experts and peers.

Get cost-effective protection for SAP HANA with Backup and DR Service - Google Cloud Backup and DR Service for SAP HANA offers cost-effective protection with incremental forever backups and HANA Savepoints integration. It reduces storage costs by up to 50% compared to traditional methods and simplifies DR setup with pre-configured networks and dependencies.

Google Cloud named a Leader in the 2024 Gartner Magic Quadrant for Data Integration Tools - Google Cloud has been recognized as a Leader in the 2024 Gartner Magic Quadrant for Data Integration Tools. Google Cloud's unified data and AI capabilities, combined with its comprehensive suite of fully managed services, empower organizations to ingest, process, transform, orchestrate, analyze, and activate their data with unprecedented speed and efficiency.

Vertex AI grounding: More reliable models, fewer hallucinations - Vertex AI grounding helps businesses build more reliable generative AI models by connecting them with sources of truth, such as internal data and the latest information from the internet. This reduces hallucinations and improves the accuracy, relevance, and up-to-dateness of AI-generated responses.

Registration is open for Partner Summit at Google Cloud Next - Google Cloud Next Partner Summit 2025 registration is now open. The event will take place in Las Vegas from April 8 to 11, 2025, and will offer partners the opportunity to accelerate their business, build new connections, and get a look at what's next from Google Cloud leadership. Early bird pricing is available until February 14, 2025.

Build agentic RAG on Google Cloud databases with LlamaIndex - LlamaIndex integrates with AlloyDB and Cloud SQL for PostgreSQL, enabling developers to build agentic applications that can connect with Google databases. These integrations empower developers to leverage the data in their operational databases to easily build complex agentic RAG workflows. Key features include streamlined knowledge retrieval, complex document parsing, secure authentication and authorization, fast prototyping, and flow control.

Veo and Imagen 3: Announcing new video and image generation models on Vertex AI - Google Cloud introduces Veo, its most advanced video generation model, and Imagen 3, its highest quality image generation model, both available on Vertex AI. Veo generates high-quality videos from text or image prompts, while Imagen 3 produces photorealistic images with fewer visual artifacts. Both models prioritize safety and responsibility with built-in precautions like digital watermarking, safety filters, and data governance.

(Re)Introducing IBM Power for Google Cloud - Converge Enterprise Cloud with IBM Power for Google Cloud (IP4G) now supports all three major environments for Power: AIX, IBM i, and Linux. It's also available in four new regions in production, bringing the total to six. Customers like Infor have successfully run mission-critical IBM Power workloads in IP4G. If you're considering moving your IBM Power workloads to the cloud, reach out to power4gcp@googlegroups.com for custom cloud plans.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Locking down Cloud Run: Inside Commerzbank's adoption of custom org policies - Commerzbank, a leading financial institution, shares their journey of adopting Google Cloud's custom organization policies to fortify their Cloud Run environments.

Is Google Cloud Anthos Service Mesh a Mess? - Today we are going to learn how Google Cloud is using service mesh in the cloud, and it is not a simple way.

Moloco: 10x faster model training times with TPUs on Google Kubernetes Engine - Moloco, an AI-powered advertising solutions provider, achieved remarkable success by leveraging Google Cloud's solutions, particularly Google Kubernetes Engine (GKE) and Cloud TPUs. By migrating its ML workloads to GKE, Moloco reduced ML training times by up to tenfold, enabling rapid model iteration and experimentation.

Bridging the Gap: Elevating Red Team Assessments with Application Security Testing - Mandiant integrates AppSec expertise into Red Team assessments to address this gap, providing a more comprehensive and realistic evaluation of an organization's security.

(QR) Coding My Way Out of Here: C2 in Browser Isolation Environments - Browser isolation is a security technology that separates web browsing activity from the user's local device by running the browser in a secure environment. Mandiant demonstrates a novel technique to circumvent all three current types of browser isolation (remote, on-premises, and local) for the purpose of controlling a malicious implant via C2.

Integrating GCP Secret Manager with Kafka Connect Using External Secrets - This guide shows how to securely inject secrets stored in GCP Secret Manager into a Kafka Connect deployment running on Kubernetes.

Scaling New Heights: Addressing AI/ML Workload Scale Challenges in GKE GCSFuse CSI Driver - This article summarizes findings and optimizations made to address challenges for AI/ML workloads running on GKE and using GCSFuse CSI Driver.

A phased approach supporting FinOps adoption - An easy-to-use approach to facilitate creating and following up your FinOps strategy and action plan.

Deploy Slurm on GKE - Slurm is a powerful open-source workload manager designed for Linux and Unix-like systems, can be deployed on Google Kubernetes Engine (GKE). This guide is intended for platform administrators and AI/ML teams who want to set up Slurm clusters on Kubernetes or GKE.

App Development, Serverless, Databases, DevOps

Building a Geospatial Application with AlloyDB, PostGIS, and Python - This article demonstrates how to build a geospatial application using Google Cloud's AlloyDB for PostgreSQL, the PostGIS extension, and Python. It covers setting up the environment, creating a database and table, generating and inserting sample data, and visualizing the data on a map.

How HighLevel built an AI marketing platform with Firestore - HighLevel, an all-in-one sales and marketing platform, migrated its workloads to Firestore, a serverless NoSQL document database, to handle volatile write loads and enable AI-powered services. Firestore's scalability, reliability, and real-time capabilities have significantly improved developer productivity, enhanced scalability, and ensured consistent performance under peak load.

How Current leveraged Spanner to build a resilient platform for banking services - Current, a fintech company, leveraged Spanner, Google's globally distributed database, to build a resilient platform for banking services. The migration to Spanner resulted in improved resilience, reduced recovery time objectives and recovery point objectives, and optimized cloud budget and operational efficiency.

Getting Cloud Tasks Working with Firebase Emulator - This article describes how to set up Cloud Tasks to work with Firebase emulator for local testing.

Cloud SQL: Real-World Scenarios and Solutions - This article discusses real-world scenarios and solutions for Cloud SQL. It covers two scenarios: performance degradation during peak hours and data corruption recovery. For the first scenario, it suggests implementing connection pooling, setting up query performance monitoring, and implementing read replicas. For the second scenario, it provides steps for point-in-time recovery implementation, data verification and synchronization.

Google Cloud Storage Signed URLs with Workload Identity Federation - Securely create Signed URLs from an AWS Lambda Function without service account keys.

Stay Ahead of the Curve with Google Cloud’s Service Health API - The Service Health API from Google Cloud Platform provides real-time and historical data on the operational status of GCP services. It allows you to receive immediate notifications about service disruptions, integrate service health data into monitoring dashboards, and trigger automated actions in response to service events. By leveraging this API, you can proactively monitor your GCP services, minimize downtime, and ensure a smooth experience for your users.

Deploying a Streamlit App on Cloud Run with Cloud SQL Postgres - This blog post demonstrates how to deploy a Streamlit application on Cloud Run with Cloud SQL Postgres integration using service account authentication. It covers creating a Cloud SQL for PostgreSQL database, uploading data, interacting with the database from Streamlit, and deploying the application securely. The setup ensures robust and secure deployment for data-driven Streamlit applications on Google Cloud Platform.

Gain Insights into AI Applications with Distributed Tracing on Google Cloud - A tutorial about using Otel to trace AI apps on Google Cloud.

Restricting GCS buckets by IP address - This blog post demonstrates how to use newly introduced IP filtering for Cloud Storage buckets.

Big Data, Analytics, ML&AI

Build and refine your audio generation end-to-end with Gemini 1.5 Pro - Gemini 1.5 Pro and the Text-to-Speech API on Google Cloud can help you create podcasts with diverse voices and generate podcast scripts with custom prompts. Gemini's multimodal capabilities, combined with the Text-to-Speech API, offer 380+ voices across 50+ languages and custom voice creation. This approach helps content creators reach a wider audience and streamline the content creation process.

Faster food: How Gemini helps restaurants thrive through multimodal visual analysis - Gemini, a multimodal AI solution from Google Cloud, helps restaurants optimize operations through visual analysis. It offers real-time meal preparation tracking, inventory management, safety assessments, and multilingual capabilities. By automating mundane tasks, Gemini enables staff to focus on delivering exceptional service and creating culinary masterpieces. It also helps businesses grow by improving cost savings and creating a safer work environment.

Fireworks.ai: Lighting up gen AI through a more efficient inference engine - Fireworks AI, a startup born from Meta AI, has developed the fastest and most efficient gen AI inference engine to date. Partnering with Google Cloud, Fireworks AI addresses the scale, cost, and complexity challenges of gen AI by utilizing services like Google Kubernetes Engine (GKE), Compute Engine, Cloud Pub/Sub, Cloud Functions, Cloud Monitoring, and BigQuery.

PayPal's Real-Time Revolution: Migrating to Google Cloud for Streaming Analytics - PayPal migrated its streaming analytics platform to Google Cloud's Dataflow to overcome challenges with reliability, efficiency, security, and scalability. The migration resulted in significant cost savings, enhanced stability, and accelerated development cycles, empowering PayPal to focus on high-value initiatives and deliver exceptional customer experiences.

BigQuery CDC with PubSub: Overcoming limitations - BigQuery CDC with PubSub is a great feature but with some scary limitations. Discover a small open source tool to go over them.

Reverse ETL for Granular Billing Data with BigQuery and Dataflow - Reverse ETL for granular billing data with BigQuery and Dataflow. This involves utilizing the Billing export job in BigQuery, leveraging CTEs and nested records in analytics queries, and writing a Dataflow job with custom transformation logic to make the data digestible for other parts of your organization. At Real Kinetic, we’ve applied this process to display cost data for our customer environments, integrating it directly into our Konfigurate platform.

Deploying vLLM on Google Cloud: A Guide to Scalable Open LLM Inference - This guide explores deploying a production-ready LLM inference service on Google Cloud Platform using vLLM. It includes a step-by-step deployment guide, configuration considerations, and production best practices for memory management, request handling, Kubernetes infrastructure setup, and security.

GenAIOps: Operationalize Generative AI - A Practical Guide - This blog post offers a comprehensive guide to navigating the complexities of Generative AI deployment.

Metadata-Driven Insights in Data Mesh - The article discusses the challenges of maintaining transparency, coordination, and governance in complex data platforms for Tchibo's data platform team.

Building and Running an AI YouTube and Video Processing as a Python Streamlit Web Application, on Serverless Google Cloud Run

GenAI App Starter Pack — Now with RAG Pattern & Vertex AI Search! - Have you ever dreamt of building a production ready RAG application in seconds while keeping it fully configurable?

Deploying LlamaIndex Workflows to Cloud Run with Llama Deploy - This guide provides a comprehensive walkthrough of deploying custom LLM workflows on Google Cloud Run with Llama Deploy. It covers containerization, building an interactive Flask app, and empowering users to deploy and scale AI solutions with ease. The full code for the sample application is available in the provided repository.

Serverless, Location-Aware Search for web and mobile apps with Agent Builder & BigQuery - This blog post introduces a method for building a serverless, location-aware search engine using BigQuery and Google Cloud's Agent Builder.

Building product recommendation bot using Gemini — Part 1 — Basics - List of posts describing process of building chatbot for product recommendation.

Deploying AI Agents on Google Cloud Platform - Deploying AI agents with large language models (LLMs) can be challenging, but this article demonstrates how to do it cost-effectively on Google Cloud Platform using LangChain and LangGraph. The technology stack includes Firestore for the vector store, Vertex AI for text embedding and the LLM, Cloud Run for deployment, Cloud Functions for preprocessing, and Cloud SQL for persistence.

Slides, Videos, Audio

Security Podcast - #201 Every CTO Should Be a CSTO (Or Else!) - Transformation Lessons from The Hoff.

Releases

AlloyDB - The AlloyDB database performance snapshot report feature is generally available (GA).This feature lets you improve your database performance by using a report that compares snapshots of system metrics between two different points in time.

Anthos Config Management - 1.20.0. Hierarchy Controller is no longer available to install. Improved the manual installation process for Config Sync. When you use Config Sync to manage configurations that are stored in OCI repositories (such as Artifact Registry), you can now enhance your security posture with custom signature verification. Introduced a new field for stopping and resuming syncing. To optimize resource use, Config Sync installations managed through Fleet no longer include the ConfigManagement Operator or the ConfigManagement CRD. Upgraded the git-sync dependency from v4.2.4 to v4.3.0 to pick up a fix for lingering Git lock files and other vulnerability fixes. Fixed a bug that prevented the applyset.kubernetes.io/part-of label from being correctly removed from managed objects when they were no longer managed by Config Sync. Fixed an issue that could cause sync delays due to retry backoff problems.

Cloud Architecture Center - (New guide) Infrastructure for a RAG-capable generative AI application using Vertex AI and Vector Search: Describes how to design infrastructure for a generative AI application with retrieval-augmented generation (RAG) by using Vector Search. Google Cloud Architecture Framework: Performance optimization: Major update to align the recommendations with core principles of performance optimization.

Backup and DR Service - Backup and DR service added support for immutable and indelible backups with the new backup vault feature. Backup and DR service added centralized backup management within Google Cloud console, with support for Compute Engine VM backup to backup vaults. Backup and DR service added integration with the Compute Engine VM creation experience, enabling the application of Backup and DR backup policies when VMs are created.

Bigtable - To create a Bigtable instance, a user or account must be a principal in a role with the permission bigtable.clusters.create.

Certificate Manager - Certificate Manager has passed HIPAA compliance validation and is listed as a covered product in HIPPA compliance on Google Cloud.

Chronicle SOAR - Release Notes 6.3.27 is in Preview. In order to align with our flagship Google SecOps platform, we are unifying our themes. Release 6.3.26 is now in General Availability.

Cloud Composer - Scheduled snapshots are available in Cloud Composer 3. Cloud Composer 2 is now available in Mexico (northamerica-south1). All Cloud Composer environment's GKE clusters are set up with maintenance exclusions from December 20, 2024 to January 2, 2025.

Compute Engine - Generally available: Hyperdisk Balanced High Availability provides cross-zonal, synchronous replication for your disk data, offering the best set of options for RPO, RTO, and performance.

Cloud Firestore - You can now Manage Firestore resources using Organization Policy Service custom constraints. You can monitor performance using client-side traces in Java and Node.js.

Identity Platform - The Identity Platform integration with reCAPTCHA Enterprise API now supports bot protection and SMS toll fraud protection for SMS-based flows on iOS platforms.

Google Kubernetes Engine - In GKE version 1.31.1-gke.2105000 or later, you can now configure custom compute classes to consume Compute Engine reservations.

Resource Manager - You can now manage Firestore resources using Organization Policy Service custom constraints.

Cloud Spanner - Spanner Graph is Generally Available (GA). A predefined Identity and Access Management (IAM) role is available to enable Spanner permission to query a Spanner database using Data Boost.

Cloud SQL MySQL - Cloud SQL Enterprise Plus edition now supports the following regions: africa-south1 (Johannesburg) asia-east2 (Hong Kong) europe-west10 (Berlin). Cloud SQL for MySQL now supports minor version 8.0.40. You can now use the Network Connectivity Center hub to propagate Private Service Connect endpoints of Cloud SQL instances in a VPC network.

Cloud Text-to-Speech - Journey Voices now supports the Journey-O speaker for de-de, en-au, en-in, en-gb, es-es, es-us, fr-ca, fr-fr, and it-it.

Vertex AI - Preview: You can consume reservations of VMs that have GPUs attached with your custom training jobs or prediction jobs.