Welcome to issue #430 December 23rd, 2024

News

Calling all devs: Code the future of baseball with Google Cloud and MLB - Google Cloud and MLB are hosting a hackathon where developers can use Gemini AI models, real MLB datasets, and Google Cloud Platform to create innovative baseball fan experiences. Participants can compete in five challenges, with prizes including cash, Google Cloud credits, and the chance to present their work at Google Cloud Next '25 and the 2025 MLB® All-Star Game®. Submissions are due by February 4, 2025.

Database Center: Now with support for Bigtable, Firestore, and Memorystore - Google Cloud’s Database Center now supports Bigtable, Memorystore, and Firestore databases. With Database Center, you can gain a comprehensive view of your entire database fleet, proactively de-risk your fleet with intelligent recommendations, and optimize your fleet with AI-powered assistance.

The Year in Google Cloud - 2024 - In 2024, Google Cloud made significant advancements in AI, with topics like generative AI, distributed system design, and AI-powered applications dominating the charts. The year also saw the introduction of new products and services, including Google Axion Processors, Trillium TPUs, and Cloud Run functions.

Find sensitive data faster (but safely) with Google Distributed Cloud’s gen AI search solution - Google Distributed Cloud (GDC) now offers AI-optimized servers with NVIDIA H100 GPUs and a ready-to-deploy on-prem conversational search solution based on the Gemma 2 LLM. This solution combines traditional search with generative AI and allows organizations to bring retrieval-augmented generation (RAG) to their on-premises environment for multimodal and multilingual natural-language search experiences.

Google Cloud's commitment to responsible AI is now ISO/IEC certified - Google Cloud has achieved an accredited ISO/IEC 42001:2023 certification for its AI management system, demonstrating its commitment to developing and deploying AI responsibly. This certification offers enhanced trust and transparency, compliance support, risk management, and access to innovative and responsible AI services.

Reach beyond the IDE with tools for Gemini Code Assist - Gemini Code Assist, a tool that aids developer workflows, introduces new features to enhance the development experience. These tools enable developers to retrieve information and perform actions from various engineering systems, such as summarizing comments from Jira issues or finding the last person who merged changes in Git. By integrating with partners like Atlassian, GitHub, GitLab, Google Docs, Sentry, and Snyk, Gemini Code Assist provides real-time access to data and insights, fostering uninterrupted flow, data-driven decisions, and a comprehensive toolkit for developers.

Looker now available in the AWS Marketplace, bringing AI for BI to multi-cloud environments - Looker, Google Cloud’s complete AI for BI platform, is now available on the AWS Marketplace, allowing AWS customers to benefit from Looker’s powerful analytics and reporting capabilities in their environment. This simplifies procurement for businesses that use AWS cloud and databases such as Amazon Redshift, empowering them to quickly and easily integrate Looker’s rich AI-driven visualizations and insights into their existing infrastructure.

AI Playground: Where learning and innovation converge in the heart of London - The Google Cloud AI Playground in Shoreditch, London, is set to launch in Q1 2025 as a dynamic hub for businesses and individuals to explore and learn about AI. It will feature interactive demos of Google's Gemini model family, hands-on workshops and hackathons led by experts, and opportunities to connect with the AI community. The AI Playground aims to address the growing need for AI skills development and empower individuals and businesses to harness the transformative power of AI.

The Cyber Threat Intelligence Program Design Playbook is now available - The CTI Program Design Playbook, a training track from Mandiant Academy, is now available. This on-demand training is designed for professionals who actively defend networks and provides essential knowledge and skills to design, build, operate, and optimize a cyber threat intelligence (CTI) program from the ground up.

Become Among the First Certified Chrome Enterprise Administrators - Google is offering a free Professional Chrome Enterprise Administrator certification exam until March 2025. The exam validates expertise in managing Chrome Enterprise browser environments using Chrome Enterprise Core.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

How Google Cloud can help customers achieve compliance with NIS2 - Google Cloud offers a range of tools and resources to help customers meet their NIS2 compliance goals, including risk management, incident handling, business continuity, and minimum security requirements. Customers can take advantage of Google Cloud's strong inherited controls, such as encryption and multi-factor authentication, as well as its extensive compliance offerings and easy access to documentation.

How Virgin Media O2 uses Privileged Access Manager to achieve principle of least privilege - Virgin Media O2 uses Google Cloud's Privileged Access Manager (PAM) to enforce the principle of least privilege, ensuring that users only have the access they need to perform their tasks. PAM automates provisioning and permissions management based on roles and responsibilities, providing just-in-time, time-bound, and approval-based access. This approach reduces identity sprawl, improves compliance, and boosts productivity.

Using Cilium and GKE Dataplane V2? Be sure to check out Hubble for observability - Hubble is a powerful observability tool that provides deep visibility into your network packet flow with added Kubernetes context. It includes Hubble Relay, a service that collects network telemetry data, and Hubble CLI, a command-line interface tool for live traffic information within the cluster. With Hubble, you can effectively troubleshoot your Kubernetes environment by tracing requests, identifying DNS issues, detecting network policy mishaps, and optimizing network performance.

Migrate your applications to Kubernetes using Istio - How Istio can help you to migrate your applications to Kubernetes.

Streamlining Teams Credential Management with Passbolt on GKE: An Open-Source and Collaborative Solution - Passbolt, an open-source password manager, provides a secure and collaborative platform for teams to manage credentials. It ensures sensitive data protection, seamless team collaboration, and compliance with best practices. This blog explores Passbolt's capabilities and demonstrates its deployment on Google Kubernetes Engine (GKE) for a secure and scalable solution.

Securing Grafana on Kubernetes with GCP IAP, Gateway API, and Terraform - A step-by-step guide to secure your Grafana deployment on Kubernetes using GCP IAP, Gateway API, and Terraform.

App Development, Serverless, Databases, DevOps

Google Cloud and SAP: Powering AI with enterprise data - Google Cloud and SAP have a long-standing partnership, offering RISE with SAP on Google Cloud to accelerate innovation and reduce costs.

Spanner in 2024: A year of innovation - Spanner, Google's always-on, virtually unlimited database, had a year of innovation in 2024. It introduced multi-model capabilities, including graph, full-text search, and vector search, allowing for interoperability between different data models without downtime. Spanner also improved its price-performance, offering increased storage and throughput at no additional cost, and introduced Spanner editions for intuitive, tier-based pricing. Additionally, it launched the Cassandra to Spanner Proxy Adapter, making it easier to migrate Cassandra workloads to Spanner.

What’s new for Google Cloud databases — December 2024 - Google Cloud databases have seen significant developments and customer successes. Product updates span Cloud SQL, AlloyDB for PostgreSQL, Memorystore, Spanner, Bigtable, Firestore, and Datastore.

How Memorystore helps FanCode stream 2X more live sports - FanCode, India's premier sports destination, migrated from AWS to Google Cloud and adopted Memorystore for Redis Cluster to enhance its content delivery capabilities. This fully integrated, scalable backend infrastructure supports real-time features and high-traffic live events, enabling FanCode to stream 2x more live events annually with consistent performance and reliability.

Achieve peak SAP S/4HANA performance with Compute Engine X4 machines - Google Cloud's X4 machine types are purpose-built for demanding SAP HANA workloads, delivering strong performance, scalability, and reliability. X4 machines are available in various memory configurations and vCPU cores, with "standard sizing" SAP certification for both SAP HANA OLTP and OLAP.

Designing and Implementing Apigee Infrastructure and Proxies on GCP: A Detailed Case Study - Explore a real-time use case of Apigee setup, proxy configurations, and optimizing GCP API infra.

Exploring the Quirks of GCP’s Metadata Server - The GCP metadata server provides instance-specific data such as metadata, service account credentials, and configuration details. It's not accessible over the public internet for security reasons. To explore the metadata server, a Docker-based web application called gcpmetadataexplorer is created. It demonstrates various requests and responses from the server, including metadata, JSON, recursive content, and special endpoints for access and identity tokens.

AlloyDB Omni and local models on GKE - This guide demonstrates how to deploy a local AI model and AlloyDB Omni to the same Kubernetes cluster and integrate them. It covers setting up a GKE cluster, installing cert-manager, deploying AlloyDB Omni with AI integration enabled, deploying a local model using Hugging Face, registering the model in AlloyDB Omni, and testing the setup.

Big Data, Analytics, ML&AI

Optimizing RAG retrieval: Test, tune, succeed - Retrieval-augmented generation (RAG) supercharges large language models (LLMs) by connecting them to real-time, proprietary, and specialized data. However, RAG systems can be difficult to master and can lead to silent failures if not thoroughly evaluated. This blog post provides a series of best practices to identify issues within your RAG system and fix them with a transparent, automated evaluation framework.

Cloud Pub/Sub 2024 highlights: Native integrations, sharing and more - Google Cloud Pub/Sub, a messaging service, introduced new features and enhancements in 2024 to meet modern streaming workloads' demands. These include simplified data ingestion from AWS Kinesis Data Streams and Cloud Storage, enhanced streaming analytics with Apache Flink integration, and improved stream sharing and export capabilities through Analytics Hub and BigQuery continuous queries.

Faster results with Short query optimized mode in BigQuery - BigQuery's new Short query optimized mode significantly improves query response times for small datasets. It skips the creation of an asynchronous job, resulting in faster execution. This mode is especially beneficial for users who frequently query small datasets.

Unlocking PDFs for RAG with Markdown and Gemini - How to convert PDF docs into Markdown to improve RAG workflows.

Human in the loop and Google Search with Langgraph - End to end demonstration of how to use Langgraph to create a chatbot able to handover concerning questions to human experts.

Taming a Llama with Rotten Tomatoes: A Vertex AI Fine-Tuning Saga - Fine-tuning Meta's open-source large language model Llama 2, on the Rotten Tomatoes dataset for sentiment analysis utilizing Google Cloud's Vertex AI platform and employed techniques like 8-bit quantization and LoRA (Low-Rank Adaptation) to optimize the training process.

Various

Google Cloud Associate Data Practitioner Certification: Exam Prep Guide - Passing Google Cloud Associate Data Practitioner certification exam.

Slides, Videos, Audio

The 10 top Kubernetes Podcast episodes of 2024

Kubernetes Podcast - #244 2024 Recap, with Abdel Sghiouar, Kaslin Fields and Mofi Rahman.

Security Podcast - #203 Cloud Shared Responsibility: Beyond the Blame Game with Rich Mogull.

Releases

Agent Assist - Pub/Sub intermediate transcription is available in preview. Agent Assist offers a native UI Connector with Genesys Cloud to integrate with voice conversations.

AlloyDB - AlloyDB lets you monitor the following additional monitoring metrics through the Cloud Monitoring dashboard. The instance/postgres/ultrafastcache_hitrate and node/postgres/ultrafastcache_hitrate metrics help in identifying any performance issue due to caching on instances or individual nodes. The node/postgres/backends_by_state, node/postgres/backends, node/postgres/wait_count, and node/postgres/wait_time metrics help in tracking node health. You can use an API to export data from AlloyDB clusters. This feature is generally available (GA).

Anthos clusters on AWS - You can now launch clusters with the following Kubernetes versions.

Google Distributed Cloud Bare Metal - 1.31. Release 1.31.0-gke.889 Google Distributed Cloud for bare metal 1.31.0-gke.889 is now available for download. Version 1.28 end of life: In accordance with the Version Support Policy, version 1.28 (all patch releases) of Google Distributed Cloud for bare metal has reached its end of life and is no longer supported. Functionality changes: Added support for configuring the GKE Identity Service to enforce a minimum transport layer security (TLS) version of 1.2 for HTTPS connections. Fixes: Fixed an issue where the registry mirror reachability check fails for a single unreachable registry mirror. Known issues: For information about the latest known issues, see Google Distributed Cloud for bare metal known issues in the Troubleshooting section.

Anthos clusters on bare metal - 1.31. Release 1.31.0-gke.889 Google Distributed Cloud for bare metal 1.31.0-gke.889 is now available for download. Version 1.28 end of life: In accordance with the Version Support Policy, version 1.28 (all patch releases) of Google Distributed Cloud for bare metal has reached its end of life and is no longer supported. Functionality changes: Added support for configuring the GKE Identity Service to enforce a minimum transport layer security (TLS) version of 1.2 for HTTPS connections. Fixes: Fixed an issue where the registry mirror reachability check fails for a single unreachable registry mirror. Known issues: For information about the latest known issues, see Google Distributed Cloud for bare metal known issues in the Troubleshooting section.

GDCV for VMware - Google Distributed Cloud (software only) for VMware 1.31.0-gke.889 is now available for download. Announcing an early look at two preview features: A new architecture called advanced clusters. Upgrade changes: Dataplane V2 is required for all user clusters. The following issues are fixed in 1.31.0-gke.889: Fixed the issue that additional manual steps are needed after disabling always-on secrets encryption with gkectl update cluster. A lot of high-severity container vulnerabilities are fixed in 1.31.0-gke.889, for detailed list check release page.

Apigee X - On December 20, 2024 we released an updated version of Apigee. Support for environment-level client IP address resolution This release introduces the ability to specify, per environment, how to capture the client IP address on API requests from the X-Forwarded-For header. On December 19, 2024, we released an updated version of Apigee (1-14-0-apigee-3) for trial organizations only. Bug ID Description N/A Updates to security infrastructure and libraries. On December 17, 2024, we released a new version of Apigee. With this release, the maximum number of apps per AppGroup is increased from 500 to 30,000.

Apigee Hybrid - v1.14.0. hybrid v1.14.0 On December 16, 2024 we released an updated version of the Apigee hybrid software, v1.14.0. Enhanced Proxy Limits in Hybrid environments Starting in version v1.14, new Apigee hybrid organizations can be provisioned with the ability to deploy more than 50 proxies per environment enabled. Cassandra credential rotation Starting in version v1.14, you can rotate Cassandra credentials in Kubernetes secrets. Enable and disable metrics-based scaling with customAutoscaling.enabled Starting in version v1.14, you can enable and disable metrics-based auto-scaling with the customAutoscaling.enabled configuration property. New analytics and debug data pipeline for hybrid orgs Starting with version 1.14, all newly created Apigee hybrid orgs created can use a new data pipeline to collect analytics and debug data and allow various runtime components to write data directly to our control plane. Forward Proxy allowlist access Starting in version v1.14, forward proxies pass through access to allowlisted URLs. Guardrails checks to ensure backups before upgrade Starting in version 1.14 new guardrails checks have been added to ensure a backup is enabled and has been made before proceeding with an upgrade. Bug ID Description 382323427 Added a guardrails check that requires backup to be enabled for Apigee Hybrid upgrades. Bug ID Description N/A Security fixes for apigee-redis.

App Hub - App Hub supports resources from Cloud Run services in Preview.

Application Integration - Dynamic Backend Authentication support for Connectors Application Integration now supports dynamic backend authentication for connectors. You can now build and publish custom connectors for Google Cloud Marketplace.

BigQuery - The Sovereign Controls for EU control package now supports BigQuery Data Transfer Service. You can now manage data canvases, data preparations, notebooks, saved queries, and workflows in Dataplex. You can now search for and view the metadata of data canvases, data preparations, notebooks, saved queries, and workflows in the Dataplex console. You can now use the Google Cloud Code extension for VS Code to work with BigQuery datasets and notebooks in your VS Code environment.

Bigtable - You can now enable 2x node scaling when you create a new Bigtable cluster. The Preview of Bigtable automated backup has been expanded to let you configure the backup retention period in automated backup policies, and the default is now seven days. You can use Organization Policy Service custom constraints to manage specific operations on Bigtable resources.

Billing - Simulate scenarios in FinOps hub to maximize your savings from resource-based CUDs In the FinOps hub, we added support for resource-based CUD recommendations as a starting point to simulate various usage scenarios, and customize the recommendation to purchase a CUD that maximizes your savings.

Capacity Planner - Preview: You can create future reservation requests for VMs of a single machine type using the Google Cloud console.

Chronicle - Google SecOps has updated the list of supported default parsers. Looker dashboard updates The following changes have been made to the Looker dashboards in Google SecOps: All dashboards have been moved to the ingestion_metrics_connector explore.

Chronicle Security Operations - Google SecOps has updated the list of supported default parsers. Looker dashboard updates The following changes have been made to the Looker dashboards in Google SecOps: All dashboards have been moved to the ingestion_metrics_connector explore.

Cloud Composer - Cloud Composer 3 is now available in Mexico (northamerica-south1). Cloud Composer 3 is now generally available (GA): All Airflow builds starting from airflow-2.9.3-build.11 and airflow-2.10.2-build.4 are supported at the GA level.

Compute Engine - Preview: You can create instances that use only IPv6 IP addresses. The A3 Edge accelerator-optimized machine type is no longer available in Turin, Italy: europe-west12-b.

Data Fusion - The Cloud Data Fusion version 6.10.1.2 patch revision is generally available (GA). Cloud Data Fusion supports the CMEK organization policy.

Dataform - You can now manage Dataform repositories in Dataplex. You can now search for and view the metadata of Dataform repositories in the Dataplex console.

Dialogflow - Dialogflow CX (Conversational Agents): You can now set either a partial match or a full match to banned phrases.

Document AI - v1. Property description is now Generally Available (GA) as part of the custom extractor in both the Document AI section of the Google Cloud console and the API, with additional support for parent entities in hierarchies.

Cloud Endpoints - Version 1.0.15 of the endpoints-management-java Java library is available.

IAM - Principal access boundary policies are generally available.

Integration Connectors - You can now build and publish custom connectors for Google Cloud Marketplace.

Google Kubernetes Engine - GKE cluster versions have been updated. 1.32 is now available in the Rapid channel Kubernetes 1.32 is now available in the Rapid channel. New features Dynamic Resource Allocation graduated to beta, and enables efficient scheduling of advanced device driver resources (because this is a beta API, using it in GKE clusters requires opt-in). Deprecated in Kubernetes 1.32 The following Beta versions of graduated APIs were deprecated in 1.29 and removed in 1.32 in favor of newer versions: flowcontrol.apiserver.k8s.io/v1beta3 FlowSchema, PriorityLevelConfiguration deprecated since 1.29, will no longer be served in 1.32, instead, use flowcontrol.apiserver.k8s.io/v1, available since 1.29. Cloud DNS additive VPC scope is now generally available on GKE clusters running version 1.28.3-gke.1430000 or later. Trillium, our sixth-generation TPU, is now generally available.

GKE new features - Cloud DNS additive VPC scope is now generally available on GKE clusters running version 1.28.3-gke.1430000 or later. Trillium, our sixth-generation TPU, is now generally available.

GKE - (2024-R49) Version updates Note: Your clusters might not have these versions available.

Google Kubernetes Engine Rapid - (2024-R49) Version updates Note: Your clusters might not have these versions available. 1.32 is now available in the Rapid channel Kubernetes 1.32 is now available in the Rapid channel.

Google Kubernetes Engine Regular - (2024-R49) Version updates There are no new releases in the Regular channel.

Google Kubernetes Engine Stable - (2024-R49) Version updates There are no new releases in the Stable channel.

Cloud Logging - Cloud Logging adds support for the northamerica-south1 region. You can now create custom roles that let you create and manage Log Scopes.

NetApp - Google Cloud NetApp Volumes now lets you test if an Active Directory policy is properly connected to the Active Directory service using the Google Cloud console. Google Cloud NetApp Volumes now supports Kerberos for large capacity volumes.

Network Connectivity Center - IPv6 route exchange is available in public preview.

Cloud Router - Standard best path selection mode is now generally available.

Policy Intelligence - The Organization Policy recommender generates insights and organization policy recommendations to restrict the creation and upload of service account keys. You can use Policy Simulator for principal access boundary policies to simulate changes to principal access boundary policies before you apply them.

Cloud PubSub - Documentation is now available to help you troubleshoot Pub/Sub issues by using audit logs. Documentation is now available to help you choose between Pub/Sub and Google Cloud Managed Service for Apache Kafka.

reCAPTCHA Enterprise - Configuring allowlists for IP addresses is now available in GA.

Resource Manager - You can use custom constraints with Organization Policy to provide more granular control over specific fields for some Secure Source Manager resources. The Organization Policy recommender generates insights and organization policy recommendations to restrict the creation and upload of service account keys. You can use the iam.managed.allowedPolicyMembers managed organization policy constraint to implement domain restricted sharing. You can use Organization Policy Service custom constraints to manage specific operations on Bigtable resources. Cloud Load Balancing resources now let you use custom constraints to define your own restrictions on Google Cloud services.

Cloud Run - New finer-grained predefined IAM roles are available for Cloud Run: Cloud Run Service Invoker, Cloud Run Jobs Executor, Cloud Run Jobs Executor With Overrides. Service-level minimum instances are now set using the --min command line flag, --service-min-instances remains available as an alias to --min.

Secure Source Manager - You can use custom constraints with Organization Policy to provide more granular control over specific fields for some Secure Source Manager resources.

Security Command Center - Install new version of the Security Command Center Enterprise use case The installation and configuration of a new version of the SCC Enterprise - Cloud Orchestration & Remediation use case in the Security Operations console is required for the toxic combination functionality of Security Command Center Enterprise. Security Health Analytics now supports new resource types for creating custom modules. Vulnerability Assessment for AWS now supports scanning container images in Elastic Container Registry (ECR). For Security Command Center Enterprise customers, the Sensitive Data Protection discovery service is now automatically enabled during the Enterprise activation process. Security Command Center can now produce Cloud Infrastructure Entitlement Management (CIEM) misconfiguration findings for federated identities that are connected to your AWS environment through the AWS IAM Identity Center. Detector for Container Threat Detection released to General Availability Container Threat Detection, a built-in service available in Security Command Center Premium and Enterprise, has launched three new detectors to General Availability: Execution: Container Escape: Detects when a process inside a container tries to break out of its isolation and interact with the host system or other containers.

Service Mesh - Managed Cloud Service Mesh. Advanced load balancing for managed Cloud Service Mesh (TD) is now available in preview. 1.23.x. 1.23.4-asm.1 is now available for in-cluster Cloud Service Mesh. 1.22.x. 1.22.7-asm.1 is now available for in-cluster Cloud Service Mesh. 1.21.x. 1.21.5-asm.17 is now available for in-cluster Cloud Service Mesh. Managed Cloud Service Mesh. Upgrading the gRPC client may cause excessive streams to Traffic Director. 1.21.x & 1.22.x & 1.23.x. Single Cluster Gateway for Mesh is now generally available. 1.23.x. Routing traffic between Cloud Service Mesh workloads and Cloud Run Services is now available in preview.

Cloud SQL Postgres - You can now enable query insights for Cloud SQL Enterprise Plus edition.

Cloud SQL SQL Server - You can use the following observability dashboards in Cloud SQL for SQL Server to monitor, analyze, and diagnose issues with your instances, databases, and queries: System insights Query insights Both of these dashboards are available to you in the Google Cloud Console.

Cloud Storage Transfer - When providing a URL list to transfer files, you can now host the list itself in an access-controlled Cloud Storage bucket.

Cloud TPU - This Release Note announces General Availability of Trillium AKA v6e.

Traffic Director - Managed Cloud Service Mesh. Advanced load balancing for managed Cloud Service Mesh (TD) is now available in preview. 1.23.x. 1.23.4-asm.1 is now available for in-cluster Cloud Service Mesh. 1.22.x. 1.22.7-asm.1 is now available for in-cluster Cloud Service Mesh. 1.21.x. 1.21.5-asm.17 is now available for in-cluster Cloud Service Mesh. Managed Cloud Service Mesh. Upgrading the gRPC client may cause excessive streams to Traffic Director. 1.21.x & 1.22.x & 1.23.x. Single Cluster Gateway for Mesh is now generally available. 1.23.x. Routing traffic between Cloud Service Mesh workloads and Cloud Run Services is now available in preview.

Vertex AI - Vector Search hybrid search and sparse embeddings are generally available (GA) Vector Search hybrid search and sparse embeddings are generally available (GA). You can copy tuned Gemini 1.5 Pro 002 and Gemini 1.5 Flash 002 adapter models across projects.

Cloud Vision API - Safe Search model update We will be updating the SAFE_SEARCH_DETECTION feature model to improve quality.

Virtual Private Cloud - IPv6-only subnets and instances are available in Preview.