Welcome to issue #378 December 25th, 2023

News

Introducing automated credential discovery to help secure your cloud environment - To help organizations improve their security, today we are launching — at no cost — a secret discovery tool in our Sensitive Data Protection offering that can find and monitor for plaintext credentials stored in your environment variables.

Protect your business-critical Google Maps Platform workloads with Enhanced Support - Protect your business-critical Google Maps Platform workloads with Enhanced Support.

Google recognized as a Leader and positioned furthest in vision among all vendors evaluated in the 2023 Gartner Magic Quadrant for Cloud Database Management Systems

The year in Google Cloud: Top news of 2023

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Cloud CISO Perspectives: Our 2024 Cybersecurity Forecast report

Personalized Service Health: Early Warning System for Disruptive Events Impacting Your Google Cloud Services - Google Cloud's Personalized Service Health (PSH) is a valuable service that lets you identify Google Cloud service disruptions relevant to your projects so you can manage and respond to them efficiently. With PSH, you can proactively identify and address potential issues before they cause a significant impact on your operations.

Protecting Secrets in Kubernetes with Google Secret Manager and the Secrets Store CSI Driver - Using the Secrets Store CSI driver for Kubernetes.

Democratizing AI: How GKE Makes Machine Learning Accessible - A curated list of many of the new features that were released on GKE especially useful for Machine Learning, Artificial Intelligence, and Large Language Models.

Configuring the KubernetesExecutor to Hum at Etsy - Migrating Airflow to Kubernetes.

App Development, Serverless, Databases, DevOps

Migrating from Cassandra to Bigtable at Latin America’s largest streaming service - Today we hear from Grupo Globo, the largest media group in Latin America, which operates the Globoplay streaming service. This post outlines their migration from Apache Cassandra to Bigtable and learnings along the way.

NetApp Cloud Volumes ONTAP Flash Cache improves cloud EDA workflows in Google Cloud

Best practices for consuming public Docker Hub content - In this blog, we outline some best practices that your teams can follow in order to reduce ecurity and reliability risks within your CI/CD pipeline.

Simplifying Log Management - How to Export and Analyze Logs in Looker Studio for Better Insights.

An Overview of Cloud Run Jobs and Prefect - A brief overview of Cloud Run Jobs and Prefect.

Connecting to Filestore from a different VPC - Filestore is a Google Cloud product for managed NFS — and as usual, connecting to it in a not-so-standard scenario can be tricky.

Functional testing with testcontainers - In this article, we explore how the TestContainers library simplifies functional testing by managing external dependencies effectively. We’ll focus on using TestContainers with Google Cloud Platform’s (GCP) Firestore and PubSub services in a .NET environment.

Visualize and Inspect Workflows Executions - Workflows recently added some deeper introspection capability: you can now view the history of execution steps. From the Google Cloud console, you can see the lists of steps, and see the logical flow between them.

Cross-Project Cloud SQL Connection with Private Service Connect and Terraform - In this article, we’ll explore how to establish a cross-project Cloud SQL connection using Private Service Connect.

Big Data, Analytics, ML&AI

How Charlotte Tilbury Beauty uses Google Cloud to respond to customer data requests

Dataflow and Vertex AI: Scalable and efficient model serving - Streaming predictions on Dataflow using Vertex AI.

Virgin Media O2 (VMO2) analyzes billions of records at sub-millisecond latencies with Memorystore for Redis - Three years ago, VMO2 set out to modernize its data platforms, moving away from legacy on-premises platforms into a unified data platform built on Google Cloud. This migration to cloud included multiple Hadoop-based systems, data warehouses, and operational data stores.

Using COUNTIF() in BigQuery - A brief explanation of COUNTIF function in BigQuery.

Building a Powerful Recommandation System with TensorFlow and BigQuery ML in Almost 5 Mins - Unlocking the Power of Text Similarity: Building a System with TensorFlow and BigQuery ML.

Nuclera aims to accelerate drug discovery with Google DeepMind AlphaFold2 on Vertex AI - Nuclera, a UK and US-based biotechnology company, is collaborating with Google Cloud to serve the life science community, marrying Nuclera’s rapid protein access benchtop system with Google DeepMind’s pioneering protein structure prediction tool, AlphaFold2 (served on Google Cloud’s Vertex AI machine learning platform.

Zeotap builds marketer’s AI companion with Vertex AI - This blog shows how Zeotap is leveraging Google's generative AI prowess to enable marketers to derive even more value from their customer data by creating a CDP that is easy to use yet robust, drive deeper insights and marketing success.

Insights, clustering models and visualizations made easy with Duet AI - Leverage Duet AI as your data science partner and embark on a journey of discovery as you unlock the future of data exploration.

Getting started with Duet AI on Google Cloud and VS Code - Exploring Google’s Duet AI on VS Code and GCP console.

Getting Started with Gemini AI API via Google Cloud Code Application Templates - Testing Gemini API via an Application Template in VS Code.

Gemini Function Calling - A promising feature of the Gemini large language model released recently by Google DeepMind, is the support for function calls. It’s a way to supplement the model, by letting it know external functions or APIs can be called.

Various

Passing 10x GCP certifications: A return on experience (Part 2: Why) - A list the reasons why it makes sense to pass one or multiple GCP certifications.

Explain and customize cloud networking with Duet AI - How Duet AI can help network engineers for example to interconnect cloud networks with on-prem network structures.

Slides, Videos, Audio

Security Podcast - #153 Kevin Mandia on Cloud Breaches: New Threat Actors, Old Mistakes, and Lessons for All.

Releases

AlloyDB - AlloyDB Omni version 15.4.0 is now available. The AlloyDB Omni Kubernetes Operator version 0.3.0 is now available in Preview, and includes bug fixes and improvements to the operator. Automated and continuous backups are now incremental backups.

Anthos clusters on VMware - GKE on VMware, formerly Anthos clusters on VMware, is a component of Google Distributed Cloud Virtual, software that brings Google Kubernetes Engine (GKE) to on-premises data centers. New features in GKE on VMware 1.28.0-gke.651: Preview: Support for max surge configuration for node pool rolling updates. Breaking change in GKE on VMware 1.28.0-gke.651: Cloud Monitoring now requires projects to enable the kubernetesmetadata.googleapis.com API and grant the kubernetesmetadata.publisher IAM role to the logging-monitoring service account. Version changes in GKE on VMware 1.28.0-gke.651: Bumped etcd to version v3.4.27-0-gke.1. The following issues are fixed in 1.28.0-gke.651: Fixed an issue where disable_bundled_ingress failed user cluster load balancer validation. Anthos clusters on VMware 1.16.4-gke.37 is now available. The following issues are fixed in 1.16.4-gke.37: Fixed a warning in the storage preflight check.

AppEngine Admin API - A warning message now appears before you publish a container image to a public repository.

Google Cloud Armor - Google Cloud Armor integration with reCAPTCHA Enterprise for mobile devices is now in General Availability.

Cloud Asset Inventory - The following resource types are now publicly available through the ExportAssets, ListAssets, BatchGetAssetsHistory, QueryAssets, Feed, and Search (SearchAllResources, SearchAllIamPolicies) APIs. The following resource types are now publicly available through the analyze policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning).

Batch - You can use Image streaming to enable Batch jobs to initialize without waiting for a container image to finish downloading. Logs from Batch jobs created after December 19, 2023 will no longer use the general-purpose generic_task monitored resource type and instead use the new batch.googleapis.com/Job monitored resource type.

BigQuery - Operational Health administrative resource charts are now in preview.

Chronicle - When you create a custom measure in a dashboard, you can't use the list, percentile, and percentile_distinct functions.

Confidential VM - Confidential Space. Confidential Space.

Compute Engine - Generally available: The accelerator-optimized A3 machine type is now available on Compute Engine. Generally available: You can rename an existing VM using the Google Cloud console, gcloud CLI, and REST.

Dataflow - Dataflow now supports data sampling for pipeline exceptions.

Dataproc Serverless - New Dataproc Serverless for Spark runtime versions: 1.1.43 2.0.51 2.1.30 2.2.0-RC3.

Dataproc - New Dataproc on Compute Engine subminor image versions: 2.0.87-debian10, 2.0.87-rocky8, 2.0.87-ubuntu18 2.1.35-debian11, 2.1.35-rocky8, 2.1.35-ubuntu20, 2.1.35-ubuntu20-arm 2.2.1-debian12, 2.2.1-rocky9, 2.2.1-ubuntu22.

Datastore - Index scans in Key Visualizer are now supported at the General Availability (GA) level.

Cloud Deploy - You can now define custom target types and deploy to targets of those types, in preview. You can now access sample custom targets, including the following: Terraform Infrastructure Manager GitOps Vertex AI Helm.

Dialogflow - Vertex AI Conversation: You can now select the generative model for data store agents. Dialogflow CX now provides a new integration for Workplace from Meta.

Document AI - Custom Extractor supports fine tuning (Preview) so that you can customize foundation model results for user specific documents. Custom Extractor with genAI is now available in the EU and northamerica-northeast1 regions. You can now demo genAI-powered extraction results within Custom Extractor along with output from other Document AI products such as OCR, Form Parser, and ID processing.

Error Reporting - The Error Reporting API is now generally available.

Eventarc - Eventarc is available in the europe-west10 (Berlin, Germany) region.

Cloud Firestore - Index scans in Key Visualizer are now supported at the General Availability (GA) level.

Cloud Functions - A warning message now appears before you publish a container image to a public repository.

Google Kubernetes Engine - You can now modify the vm.max_map_count Linux kernel attribute for nodes in a GKE Standard cluster node pool using the node system configuration. The GKE NEG controller now supports IPv6 endpoints with GKE version 1.28.4-gke.1083000 and later. All newly created Google Kubernetes Engine (GKE) Autopilot clusters starting with 1.27.4-gke.900 will automatically collect and send metrics from the kube-state-metrics package to Managed Service for Prometheus.

Cloud Monitoring - Observability for Google Kubernetes Engine: The collection of kube state metrics is enabled by default for new GKE Autopilot clusters, starting with version 1.27.4-gke.900. On your custom dashboards, you can add section widgets that create a table of contents for your dashboard. On your custom dashboards, you can group widgets into a single-view widget or into a collapsible group.

reCAPTCHA Enterprise - reCAPTCHA Enterprise for WAF and Google Cloud Armor integration is now available for mobile applications in GA.

Resource Manager - The dry-run feature for Organization Policy is now in General Availability.

Cloud Run - For revisions that have CPU allocation set to CPU always on, instance autoscaling now occurs for CPU activity even outside of incoming requests.

Secret Manager - Generally available: Key Access Justifications is now generally available with Secret Manager.

SAP Solutions - New SAP certification for operating systems For use with SAP HANA on Google Cloud, SAP has certified the operating system Red Hat Enterprise Linux (RHEL) for SAP 9.2.

Cloud Spanner - Cloud Spanner now supports partition queries whose query plans don't contain any distributed unions. The number of mutations per commit that Cloud Spanner supports has increased from 40,000 to 80,000.

Cloud SQL MySQL - You can now specify the SSL mode of your Cloud SQL for MySQL instances. Cloud SQL for MySQL has completed the upgrade to MySQL 5.7.44. Cloud SQL for MySQL now supports the following flags for MySQL 8.0: innodb_buffer_pool_dump_now innodb_buffer_pool_load_abort innodb_buffer_pool_load_now For more information about these flags, see supported flags. Cloud SQL Enterprise Plus edition now supports the following regions: europe-west8 (Milan) europe-west12 (Turin) us-west4 (Las Vegas).

Cloud SQL Postgres - Cloud SQL Enterprise Plus edition now supports the following regions: europe-west8 (Milan) europe-west12 (Turin) us-west4 (Las Vegas).

Transfer Appliance - 4.0. Transfer Appliance has introduced GA support for the data export appliance in the US.

Vertex AI - Model Garden updates: Support for hyperparameter tuning and customized datasets for OpenLLaMA models using the dataset format used by supervised tuning in Vertex AI.

VMware Engine - Google Cloud VMware Engine now supports additional Terraform resources for automating private cloud, cluster, and network management.

Virtual Private Cloud - You can use Packet Mirroring to collect IPv6 traffic.