Welcome to issue #429 December 16th, 2024

News

Introducing Gemini 2.0: our new AI model for the agentic era - An Introduction to Gemini 2.0 Flash and other AI-related projects.

Announcing the general availability of Trillium, our sixth-generation TPU - Trillium TPU, Google Cloud's sixth-generation TPU, is now generally available. It offers significant improvements over the previous generation, including over 4x improvement in training performance, up to 3x increase in inference throughput, and a 67% increase in energy efficiency.

Introducing Google Agentspace: Bringing AI agents and AI-powered search to enterprises - Google Agentspace is a new platform that brings together advanced reasoning, Google-quality search, and enterprise data to help employees accomplish complex tasks more efficiently. It offers new ways to interact with enterprise data, information discovery across the enterprise, and expert agents to automate business functions.

Tailor your search engine with AI-powered hybrid search in Spanner - Spanner, Google’s multi-model database, now offers AI-powered hybrid search capabilities. Developers can combine vector search, full-text search, and machine learning (ML) model reranking capabilities in a unified platform directly integrated with the operational data store, using a familiar SQL interface.

Improve your security posture with expanded Custom Org Policy - Google Cloud is now adding support for more than 30 additional Google Cloud services for custom Organizational Policy. This expansion unlocks many new use cases and expands the scope of cloud governance.

Understand how your users are using Gemini for Google Cloud with Cloud Logging and Monitoring - Gemini for Google Cloud now supports Cloud Logging and Monitoring, providing visibility into how users interact with the AI-powered assistant. Cloud Logging records requests and responses between Gemini and individual users, while Cloud Monitoring reports active users and response counts. This information can be used to track the provenance of AI-generated content, record user usage, and configure dashboards and alerts.

Google Cloud first CSP to join BRC, MFG-ISAC, and affiliates to advance security - Google Cloud has joined the Business Resilience Council (BRC), the Manufacturing Information Sharing and Analysis Center (MFG-ISAC), the Operational Technology Information Sharing and Analysis Center (OT-ISAC), and the Energy Analytic Security Exchange (EASE) as part of the Global Resilience Federation (GRF).

Unwrap 12 days of training to learn generative AI this December - Get ready for 12 Days of Learning, a festive digital advent calendar packed with courses, hands-on labs, videos, and community opportunities—all designed to boost your generative AI expertise. Discover a new learning resource on Google Cloud’s social channels every day for twelve days this December.

Boost your Looker Studio Pro skills with new on-demand course from Google Cloud - The new Looker Studio Pro Essentials on-demand course from Google Cloud provides comprehensive training on Looker Studio Pro's advanced features and capabilities. It covers topics such as enhancing collaboration, scheduling and sharing reports, ensuring data security and control, and leveraging Google Cloud customer care.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Orchestrating GPU-based distributed training workloads on AI Hypercomputer - Google Cloud offers various GPU orchestration options for AI Hypercomputer, including GKE, Cluster Toolkit, and Vertex AI.

XRefer: The Gemini-Assisted Binary Navigator - XRefer is a tool that helps analysts navigate and understand binary files. It uses a combination of static analysis and natural language processing to provide a high-level view of the binary's architecture and detailed information about its components.

Scaling to zero on Google Kubernetes Engine with KEDA - This article demonstrates how to integrate the KEDA ( Kubernetes Event-driven Autoscaler), an open-source Kubernetes-native solution, that enables scaling deployments based on various metrics and events, including scaling workloads to zero.

How Google Cloud helps navigate your DPIA and AI privacy compliance journey - Google Cloud offers a DPIA Resource Center to help customers understand and comply with data protection obligations when using AI offerings integrated in Google Cloud services. Customers can use the center to determine if a DPIA is needed, get guidance on preparing one, and access downloadable templates.

Network Performance Decoded: A brief look at performance limiters - A new whitepaper from Google Cloud explores the often overlooked factors that can limit network performance, such as packet size and offloading techniques. It also delves into TCP Round Trip Time (RTT) and how it can be affected by various factors, including the receiving application's behavior. Finally, the whitepaper examines Path MTU discovery and how it helps prevent IP fragmentation, offering solutions to common problems.

Cloud CISO Perspectives: Our 2025 Cybersecurity Forecast report - Google Cloud’s Cloud CISO Perspectives report for 2025 predicts that AI will continue to evolve rapidly, with attackers increasingly using it for sophisticated phishing, vishing, and social engineering attacks. Cyber-espionage and cybercrime actors will use deepfakes for identity theft, fraud, and bypassing know-your-customer (KYC) security requirements. Generative AI will also be used by defenders to bolster security operations, with AI tools supporting human awareness and ingenuity.

To avoid “bill shocks,” Palo Alto Networks deploys custom AI-powered cost anomaly detection - Palo Alto Networks deployed a custom AI-powered cost anomaly detection solution on Google Cloud to proactively manage cloud costs and avoid unexpected expenses. The solution leverages BigQuery ML, Looker, Pub/Sub, and Cloud Run functions to detect anomalies, notify stakeholders, and facilitate root-cause analysis.

Going Cloud: EFB vs Fabric FAST — Which one for GCP? - Did you know that Google provides multiple approach to enable a fast deploy on the Cloud? Learn the key aspects and difference!

Modules and relations - Learn how to troubleshoot common issues that arise when using Terraform modules for network configurations in Google Cloud Platform. Discover the importance of understanding module interactions and resource referencing, and explore real-world examples of VPCs, Cloud NAT gateways, and VPN gateways.

Replacing GCP Secondary Backup with a Cost-Effective Kubernetes Operator Solution - Optimizing Database Backups: Cutting Costs, Increasing Flexibility, and Enhancing Observability with Kubernetes.

App Development, Serverless, Databases, DevOps

How Dun & Bradstreet is transforming software development with Gemini Code Assist - Dun & Bradstreet, a leading global provider of business data and analytics, has transformed its software development lifecycle by adopting Gemini Code Assist, an AI-powered coding assistance tool from Google Cloud.

4 Years with Google Cloud Platform (GCP): The Good, the Not-So-Good, and the “Needs Work” - Honest take on GCP services — what shines, what lags, and what needs a serious overhaul.

How Ford Pro uses Bigtable to harness connected vehicle telemetry data - Ford Pro Intelligence, built on Google Cloud's Bigtable NoSQL database, is a cloud-based platform used for managing and supporting fleet operations of commercial customers. It collects connected vehicle data to help fleet operators streamline operations, increase productivity, reduce costs, and improve fleet performance.

Unleash a new era of PostgreSQL: AlloyDB redefines what's possible - A brief e-book about AlloyDB, Google's PostgreSQL-compatible database.

Cloud SQL: Real-World Scenarios and Solutions Part 2 - And so we are back here with our part of scenarios and issues which we end up facing due to CloudSQL here.

Cloud Build + Cloud Deploy - Google Cloud Build and Cloud Deploy are powerful tools for automating and streamlining modern software development workflows. In this article, a CI/CD pipeline is implemented using these services to ensure automated, reliable, and efficient application deployment.

Benchmarking AlloyDB with TPC-H: A Deep Dive - This blog post presents a detailed benchmark of Google Cloud's AlloyDB for PostgreSQL using the industry-standard TPC-H benchmark. The setup, data loading process, query execution, and results analysis are covered. The results show that AlloyDB exhibits strong performance for TPC-H queries, making it a strong contender for applications requiring high-throughput data analysis. Further investigation could focus on optimizing specific queries and exploring the impact of the columnar engine on overall performance.

Enjoy a Stress-Free Holiday Season with Deploy Policy - Use deny policy to avoid unwanted releases during holidays.

Big Data, Analytics, ML&AI

Looking back on a year of deeper connectivity across Earth Engine and Cloud - In 2024, Google Earth Engine made significant advancements in platform management, cloud integration, and core functionality. New features include a centralized Overview page in the Cloud Console, improved Vertex AI integration, support for VPC Service Controls, and performance enhancements for zonal statistics exports and Python v1. These updates enable users to leverage advanced tools, stronger security, and seamless integrations to improve sustainability and climate resilience.

BigDAG: A Simple Tool for Managing BigQuery Workflows - BigDAG is a Python tool that helps manage data workflows in BigQuery. It automatically detects dependencies between SQL scripts, provides a command-line interface for deployment, and supports dry runs and recreation of datasets.

10 Optimisation Tips to Reduce Your Mighty BigQuery Cost - 10 ways to reduce your eye watering BigQuery bills.

Intelligible advanced SQL queries with pipe syntax - The pipe syntax for SQL queries offers a more straightforward and readable way to write complex queries. It preserves the semantic order of operations and reduces code duplication. With the pipe operator |>, users can easily add or remove steps in their queries, such as filtering, selecting, aggregating, and ordering data. This new syntax enhances the interactive SQL workflow, making it easier to explore and analyze data.

Latest Updates on Google Data Analytics (November 2024) - The highlights of the updates on BigQuery, Looker Studio, Google Analytics (GA) & Google Tag Manager (GTM). By Alexander Junke.

Google Cloud and Swift pioneer advanced AI and federated learning tech to help combat payments fraud - Google Cloud and Swift are collaborating to develop advanced AI and federated learning technologies to combat payments fraud. Federated learning enables secure collaboration and knowledge sharing without compromising data privacy, fostering a safer financial ecosystem.

How Vertex AI’s vector search helps unlock high-performance gen AI apps - Vertex AI’s vector search helps developers build lightning-fast, scalable, and cost-effective AI applications. It enables rapid development and deployment of high-performance search applications, fully customized to meet specific business needs. Vector search is crucial for applications that demand incredible flexibility, scale, and speed, such as e-commerce, media, and other consumer-facing web services.

How Fullstory uncovers user insights with Vertex AI serving Gemini 1.5 Pro - Fullstory, a leading behavioral data analytics platform, uses Vertex AI serving Gemini 1.5 Pro to strengthen its autocapture technology, Fullcapture. Fullcapture goes beyond traditional autocapture capabilities to give users a deeper understanding of their customer data by capturing every user interaction in detail and enabling revisiting of any aspect of user behavior as needed.

Toyota shifts into overdrive: Developing an AI platform for enhanced manufacturing efficiency - Toyota has developed an innovative AI Platform using Google Cloud's AI Infrastructure, empowering factory workers to create and deploy machine learning models across key use cases. The platform combines on-premises infrastructure and cloud computing for agile development, cost-effectiveness, and scalability.

Building product recommendation bot using Gemini — Part 2— Voice, Gemini 2.0 announcement - In this part we cover giving our bot a voice input (using Speech-to-Text or LLM) and voice output (using Text-to-Speech or new Gemini 2.0).

Rapid UI prototyping for AgentBuilder Agents with Google Mesop and Cloud Run - This article explores two options for building a user interface (UI) for your AI prototypes using Agent Builder Agents.

Slides, Videos, Audio

Kubernetes Podcast - #243 Kubernetes v1.32 Penelope, with Frederico Muñoz.

Security Podcast - #202 Beyond Tiered SOCs: Detection as Code and the Rise of Response Engineering.

Releases

Resource Manager - Using IAM attributes in custom organization policies is generally available. You can use the iam.managed.preventPrivilegedBasicRolesForDefaultServiceAccounts managed organization policy constraint to prevent default service accounts from being granted the Editor (roles/editor) or Owner (roles/owner) roles.

Cloud Memorystore - Memorystore for Redis is supported by Database Center.

Memorystore for Redis Cluster - AOF and RDB persistence are Generally Available. Cross-region replication is now Generally Available on Memorystore for Redis Cluster. Memorystore for Redis Cluster is supported by Database Center.

Cloud Monitoring - Reporting of the "pending" status of the Ops Agent on the Cloud Monitoring VM Instances dashboard has been refined to include additional states. You can now override the validation that checks for metric existence when you create a PromQL-based alerting policy. Text widgets can now link to sections of a dashboard and they can render variables.

Policy Intelligence - You can use Policy Simulator for deny policies to simulate changes to deny policies before you apply them.

reCAPTCHA Enterprise - reCAPTCHA Mobile SDK v18.7.0-beta01 is now available for iOS.

Cloud Run - The CPU allocation setting has been renamed to Billing in the Google Cloud console for Cloud Run services. You can now create custom organization policies for Serverless VPC Access connectors and apply them to projects, folders, or organizations (GA).

Security Command Center - AI summaries of attack paths are disabled in Security Command Center Effective December 13, 2024, the preview of Gemini AI-generated summaries of Security Command Center attack paths is discontinued. When activating the Security Command Center Enterprise tier, you now have the option to connect Security Command Center to an existing Google Security Operations instance or provision a new instance.

Service Mesh - 1.21.x & Managed Cloud Service Mesh & 1.22.x & 1.23.x. As part of the Per-cluster entitlement to GKE Enterprise, a GKE cluster needs to have its cluster_tier set to ENTERPRISE in order for that cluster to be considered GKE Enterprise.

SAP Solutions - New SAP HANA certification: 24 TB and 32TB X4 bare metal machine types for OLAP workloads SAP has certified the Compute Engine x4-megamem-1440-metal and x4-megamem-1920-metal machine types for use with SAP HANA OLAP workloads in scale-out configurations with up to 8 nodes.

Cloud Spanner - Spanner now supports IDENTITY columns.

Cloud SQL Postgres - Cloud SQL now offers notifications for maintenance that's either begun or completed.

VPC Service Controls - VPC Service Controls feature: Support for using identity groups in the ingress and egress rules to allow access to resources protected by a service perimeter is generally available.

Virtual Private Cloud - Private Service Connect service connectivity automation lets you automate connectivity to supported Google service instances that are located in a different project, folder, or organization than the service consumer (custom scope). Private Service Connect port mapping is available in General Availability.

AlloyDB - AlloyDB System insights offers a unified, customizable database monitoring dashboard that includes predefined metrics and other Google Cloud metrics. The Perform a vector search tutorial describes how to set up and perform a vector search in AlloyDB for PostgreSQL.

Anthos Config Management - Config Controller now uses the following versions of its included products: Config Connector v1.125.0, release notes.

Apigee Integrated Portal - On December 10, 2024, we released a new version of the Apigee integrated portal. Bug ID Description 381086551 Fixed an issue that caused the page list view to fail for some portals with large numbers of pages.

Apigee X - On December 10, 2024, we released an updated version of Apigee (1-14-0-apigee-2). Bug ID Description 357880539 Resolved issue with missing span in the Apigee UI for distributed trace.

AppEngine Standard - Custom organization policies for Serverless VPC Access connectors are now generally available, and can be applied to projects, folders, or organizations.

Application Integration - Integration templates (Preview) Save time and effort building integrations with integration templates. Generate and view OpenAPI Specification You can now generate and view the OpenAPI Specification for any published integration that uses API triggers. API trigger input and output variables You can now set request and response payloads for an API trigger using trigger specific input and output variables.

Cloud Architecture Center - Infrastructure for a RAG-capable generative AI application using Vertex AI and AlloyDB: Added more design alternatives. Infrastructure for a RAG-capable generative AI application using GKE: Added design alternatives. Deploy automated malware scanning for files uploaded to Cloud Storage: Added the Deploy using the Terraform CLI section. (New guide) Stream logs from Google Cloud to Datadog: Provides an architecture to send log event data from across your Google Cloud ecosystem to Datadog Log Management.

Cloud Asset Inventory - The following resource types are now publicly available through the analyze policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning).

BigQuery - Regional endpoints, which help you run your workloads in compliance with data residency and data sovereignty requirements, are now generally available (GA). You can now discover, procure, and commercialize your Analytics Hub listings on Google Cloud Marketplace to share data offerings at scale. You can now replicate a dataset from the source region to one or more other regions with cross-region dataset replication. BigQuery Managed Disaster Recovery provides managed failover and redundant compute capacity for business-critical workloads. You can now create remote models in BigQuery ML based on the gemini-2.0-flash-exp model in Vertex AI.

Bigtable - Bigtable is now supported by Database Center, which is in Preview. You can now enable row-affinity routing to let Bigtable automatically ensure that single-row requests for a given row are routed to the same cluster. You can now use the Google Cloud console to create and manage authorized views of your Bigtable tables. You can now select a row in a Bigtable Studio query results table to view formatted row data.

Chronicle - Google SecOps has updated the list of supported default parsers.

Chronicle Security Operations - Google SecOps has updated the list of supported default parsers.

Chronicle SOAR - Release 6.3.27 is now in General Availability. Release 6.3.28 is currently in Preview.

Colab - Gemini in Colab Enterprise, which is a product in the Gemini for Google Cloud portfolio, now includes error fixing in Preview.

Cloud Composer - In January 2025, we plan to release Cloud Composer 2 and Cloud Composer 3 versions that will always use the environment's service account for performing PyPI packages installations. (Cloud Composer 3) It is now possible to upgrade an environment if the [sentry]sentry_on Airflow configuration option is set to true. Cloud Composer no longer adds any missing IAM permissions to the Cloud Storage bucket when it is used to create an environment with a custom environment's bucket. The COMPOSER_AGENT_BUILD_SERVICE_ACCOUNT environment variable is changed to reserved. Increased allowed timeouts when detecting tasks stuck in the "queued" state during the Airflow worker liveness check. (Cloud Composer 2) Airflow worker liveness check configuration was changed to be consistent with the configuration used in Cloud Composer 3. New Airflow builds are available in Cloud Composer 3: composer-3-airflow-2.10.2-build.4 (default) composer-3-airflow-2.9.3-build.11. New images are available in Cloud Composer 2: composer-2.10.1-airflow-2.10.2 (default) composer-2.10.1-airflow-2.9.3. Cloud Composer version 2.5.3 has reached its end of support period. Fixed the issue in the environment's component responsible for uploading the logs of Airflow components to Cloud Logging.

Compute Engine - C3 bare metal instances are available in the following additional zones: c3-highcpu-192-metal: asia-southeast1-a and c, europe-west4-c, us-east1-d, us-east4-c, us-east5-a, us-west1-a and b c3-standard-192-metal: europe-west1-b and c, europe-west4-b and c, us-east1-d, us-east4-a, us-west1-a and b c3-highmem-192-metal: europe-west4-c, us-east4-a and c, us-west1-a and b. Generally available: Instance flexibility in a managed instance group (MIG) lets you configure multiple machine types in the group. Fixed the issue causing incorrect detection of CPU load on T2D machine series VMs in managed instance groups (MIGs).

Config Connector - Config Connector version 1.126.0 is now available. Config Connector system management CRDs ControllerReconciler and NamespacedControllerReconciler are promoted to Beta. New Beta resources (direct reconciler) BigQueryTransferConfig Manage the metadata needed to perform a BigQuery data transfer. Use BigQueryConnectionConnection to provide the IAM Service Account IAMPolicyMember Added spec.memberFrom.bigQueryConnectionConnectionRef See an example on IAMPolicyMember use BigqueryConectionConnection "cloudSQL" IAMPartialPolicy Added spec.memberFrom.bigQueryConnectionConnectionRef. New Alpha Resources Add new resource WorkstationConfig.

Data Catalog - You can now transition your Data Catalog content and usage to Dataplex Catalog.

Database Migration Service - Database Migration Service now supports MySQL minor version 8.0.40 for homogeneous MySQL migrations.

Dataplex - You can now transition your Data Catalog content and usage to Dataplex Catalog.

Dataproc - New Dataproc on Compute Engine subminor image versions: 2.0.126-debian10, 2.0.126-rocky8, 2.0.126-ubuntu18 2.1.74-debian11, 2.1.74-rocky8, 2.1.74-ubuntu20, 2.1.74-ubuntu20-arm 2.2.40-debian12, 2.2.40-rocky9, 2.2.40-ubuntu22. Dataproc on Compute Engine: Updated Dataproc Metastore (DPMS) gRPC proxy image version to v.

Datastream - Datastream now supports binary log reader as a CDC method for Oracle sources.

Cloud Firestore - Firestore is supported by Database Center.

Gemini - AI summaries of attack paths are disabled in Security Command Center Effective December 13, 2024, the preview of Gemini AI-generated summaries of Security Command Center attack paths is discontinued. Customer-managed encryption keys (CMEK) are now integrated with code customization for Gemini Code Assist. The maximum number of repositories that can be indexed for Gemini Code Assist code customization has increased to 20,000. VS Code Gemini Code Assist is now a separate extension from Cloud Code. You can now adjust the cloudcode.duetAI.recitation.maxCitedLength setting in the settings UI for VS Code Gemini Code Assist. Gemini Code Assist for IntelliJ now supports code transformation.

Cloud Healthcare API - v1beta1 & v1 & v1alpha2. A new release is available.

IAM - Using IAM attributes in custom organization policies is generally available. You can use the iam.managed.preventPrivilegedBasicRolesForDefaultServiceAccounts managed organization policy constraint to prevent default service accounts from being granted the Editor (roles/editor) or Owner (roles/owner) roles.

Infrastructure Manager - You can use Infrastructure Manager (Infra Manager) to preview, create, manage, and delete deployments in Google Cloud Console. Infra Manager is available in the following regions: Asia Pacific asia-northeast1 asia-northeast2 asia-east2 asia-south1 asia-south2 asia-southeast1 asia-southeast2 australia-southeast1 australia-southeast2 Europe europe-north1 europe-central2 europe-west3 europe-west4 europe-west6 Americas us-east1 us-east4 us-west1 us-west2 us-west3 us-west4 northamerica-northeast2 southamerica-west1 For more information, see Locations.

Google Kubernetes Engine - GKE now provides insights and recommendations that help you identify and amend clusters running a minor version that reached end of standard support, clusters with nodes in violation of version skew policy, and clusters without a maintenance window to achieve reliable operations, up-to-date security posture and supportability. The C4A machine family is generally available in the following versions: Standard clusters in version 1.28.13-gke.1024000, 1.29.8-gke.1057000, 1.30.4-gke.1213000 or later. GKE cluster versions have been updated. Starting with GKE version 1.33, clusters running cgroupv1 will automatically be upgraded to cgroupv2 unless you opt out first. (2024-R47) Version updates GKE cluster versions have been updated.

GKE new features - GKE now provides insights and recommendations that help you identify and amend clusters running a minor version that reached end of standard support, clusters with nodes in violation of version skew policy, and clusters without a maintenance window to achieve reliable operations, up-to-date security posture and supportability. The C4A machine family is generally available in the following versions: Standard clusters in version 1.28.13-gke.1024000, 1.29.8-gke.1057000, 1.30.4-gke.1213000 or later.

Cloud Logging - Reporting of the "pending" status of the Ops Agent on the Cloud Monitoring VM Instances dashboard has been refined to include additional states.