Welcome to issue #404 June 24th, 2024

News

Bringing file system optimizations to Cloud Storage with hierarchical namespace - Cloud Storage hierarchical namespace (HNS) is a new bucket creation option that provides optimizations for folder structure, resources, and operations. HNS buckets have an optimized storage layout that delivers higher initial bucket queries per second (QPS) and support a number of new APIs oriented towards applications optimized for file-oriented storage, such as Hadoop ecosystem tooling or AI/ML workloads.

Simplify historical data tracking in BigQuery with Datastream's append-only CDC - Datastream's append-only mode simplifies change data capture by preserving every change as a new row in your target BigQuery table. It offers cost efficiency, improved data accuracy, and real-time insights. With append-only mode, businesses can maintain a historical record of changes, track data modifications, and gain deeper insights from their data.

Normalize billing data across clouds with new Looker template and BigQuery views - Google Cloud introduces a new Looker template and updates the BigQuery view to support FOCUS v1.0 GA, a unifying specification for cloud billing data. The Looker template allows users to visualize open billing data in Looker, while the updated BigQuery view transforms data towards the FOCUS v1.0 GA format.

Google is a Leader in the 2024 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms - Google has been recognized as a Leader in the 2024 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms. The company's Vertex AI platform provides a unified AI experience for building, deploying, and managing any type of AI/ML model. Vertex AI offers a wide range of features, including pre-trained models, infrastructure, and tools for data governance and privacy.

Announcing Anthropic’s Claude 3.5 Sonnet on Vertex AI, providing more choice for enterprises - Anthropic's newly released model, Claude 3.5 Sonnet, is now generally available on Google Cloud's Vertex AI platform. With advanced capabilities in reasoning, knowledge, math, and coding, it can power various applications, including coding assistance, customer support, data analysis, and visual processing. Enterprises can leverage Vertex AI's enterprise-grade infrastructure, tooling, and security to build and deploy production-grade generative AI applications.

Enhance the security of your DialogFlow CX chatbots with Apigee - Apigee, a full lifecycle API management platform, can enhance the security of Dialogflow CX chatbots by providing centralized authentication, controlled access, API key management, and threat protection. By using Apigee as an intermediary, organizations can securely integrate their chatbots with websites without exposing sensitive credentials. This post provides a step-by-step guide on how to configure Apigee to improve security for conversational agents.

What’s new with Google Cloud databases — June 2024 edition - Google Cloud has released several updates to its database services, including vector search support across AlloyDB, Spanner, Cloud SQL, Memorystore for Redis, and Firestore. Gemini in Databases, an AI assistant, simplifies database management and development.

CSPs reach further and monetize smarter with network API digital marketplace syndication - The telecommunications industry is undergoing a major transformation driven by digitization and the need for more agile and efficient ways to deliver services. One key trend is the emergence of network APIs to incubate innovation and drive new revenues through ecosystems. Google Cloud proposes a marketplace syndication approach to reach and target developers where they are, by onboarding network APIs to an aggregator platform and then syndicating them to different marketplaces. A digital API marketplace can enable operators to package and sell network APIs, providing benefits such as a catalog of APIs, discovery, billing, and lifecycle management.

IPv6 Hybrid Connectivity: More ways to connect on-prem IPv6 workloads to Google Cloud - Google Cloud has expanded its IPv6 Hybrid Connectivity portfolio with IPv6 BGP sessions, Partner Interconnect IPv6, and IPv6-only HA-VPN. These new capabilities simplify route management, maximize compatibility with on-premises networking equipment, and enhance IPv6 connectivity options for enterprises.

Exploring Google Cloud networking enhancements for generative AI applications - Google Cloud offers new networking capabilities to optimize traffic for generative AI applications. These capabilities include Cross-Cloud Network for accelerated AI training and inference, Model as a Service Endpoint for secure and reliable access to AI models, custom AI-aware load balancing for minimized inference latency, optimized traffic distribution for AI inference applications, and Service Extensions for enhanced gen AI serving.

How Confidential Accelerators can boost AI workload security - Google Cloud has announced two new Confidential Computing offerings to protect the privacy and confidentiality of AI/ML workloads: Confidential VMs powered by NVIDIA H100 Tensor Core GPUs with HGX protected PCIe, and Confidential VMs with Intel Advanced Matrix Extensions (Intel AMX) support.

---

Your cloud, simplified

DoiT delivers technology and cloud expertise to buy, optimize, and manage Google Cloud with ease. Access a global team of cloud experts with decades of experience in cloud architecture, Kubernetes, machine learning, and much more – all on call for you. Learn More

---

Infrastructure, Networking, Security, Kubernetes

Network Performance Decoded: Benchmarking TCP and UDP bulk flows - Google Cloud's networking team shares best practices for maximizing network performance and efficiency. The first set of whitepapers covers benchmarking Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) bulk flows. These whitepapers provide valuable insights into optimizing network performance, regardless of whether deployments are on Google Cloud, cross-cloud, on-premises, or with other cloud providers. Additionally, an updated whitepaper on Measuring cloud network performance with PerfKit Benchmarker is available, offering a comprehensive toolkit for reproducible and efficient network performance benchmarking.

Cloaked and Covert: Uncovering UNC3886 Espionage Operations

6 deployment archetypes for your reliability, cost, operational, and latency needs - Google Cloud deployment archetypes guide provides an architectural model to build cloud topologies that meet the business and technical requirements of an application. It explores six deployment archetypes: zonal, regional, multi-regional, global, hybrid, and multicloud, presenting use cases and design considerations for each.

TLS and mTLS connection with GCP Application Load Balancer - The GCP Application Load Balancer can be configured to use mutual TLS (mTLS) to verify the identity of clients that connect to it. This helps to ensure that only trusted clients communicate with the load balancer’s backend applications. To implement mTLS with GCP Application Load Balancer, several resources need to be configured, including Server TLS policies, Trust Config, and Trust Store.

Service Mesh Uncharted: Google Cloud Service Mesh (aka ASM) — Part 1 - Google Cloud Service Mesh (ASM) is a premium managed service mesh offering under GKE Enterprise. It provides effortless setup and management, instant observability, enhanced security, and traffic control. ASM has three major deployment methods: Managed Service Mesh (recommended), In-Cluster Control Plane for GKE with Istio APIs (self-managed service mesh), and Hybrid Service Mesh. This blog covers the Managed Service Mesh deployment method, including prerequisites, setup instructions, and application deployment.

How I Deployed Palo Alto NGFW VM Series on GCP - Palo Alto VM-Series NGFW can be deployed on Google Cloud Platform (GCP) to secure workloads, prevent inbound and outbound threats, and provide centralized control through Panorama hosted on-prem. The deployment involves creating Compute Engine instances with specific network interfaces for management, untrusted, and trusted traffic.

Wildcard certificates with Ingress Gateway - Using the same TLS certificate for multiple gateways in Istio Ingress Gateway can cause 404 errors in browsers that utilize HTTP/2 connection reuse. To resolve this issue, configure a single wildcard Gateway instead of multiple gateways and bind both VirtualServices to this Gateway. This ensures that requests to different hosts with the same wildcard certificate are handled correctly and 404 errors are avoided.

Google Cloud Networking: Hybrid Connectivity with Hub and Spoke Topology - This article provides a step-by-step guide to implement a hybrid network architecture using Google Cloud's hub-and-spoke topology. It covers setting up projects, VPC networks, firewall rules, VPC network peering, HA VPN connection, DNS managed zones, custom DNS server, DNS forwarding, and Cloud NAT. The goal is to enable seamless connectivity between on-premise infrastructure and cloud resources, ensuring secure access to various cloud and on-premise environments.

Leveling up FinOps: 5 cost management innovations from FinOps X 2024 - Google Cloud is introducing several FinOps enhancements to improve cost management. These include making open cloud billing data a reality, enabling users to speak in the language of business, expanding the definition of cost to include carbon, modeling what an efficient cloud looks like in near real-time, and sending actionable alerts instead of noise.

Cloud CISO Perspectives: 3 promising AI use cases for cybersecurity - Google Cloud CISO Phil Venables discusses three promising use cases for AI in cybersecurity: malware analysis, boosting security operations (SecOps) teams, and scaling security solutions. AI can automate malware analysis, generate summarized reports, and identify indicators of compromise. It can assist SecOps teams by enabling natural language queries, generating AI-generated summaries, and helping onboard new team members. AI can also improve fuzzing coverage, help patch vulnerabilities, and provide a sequence of stacked benefits to enhance productivity.

Tips for troubleshooting Google Cloud Load Balancing backends - Google Cloud Load Balancing backends can sometimes fail to turn green and healthy, indicating unhealthy backends. This article provides troubleshooting tips to help resolve these issues, including checking connectivity between health check ranges and the load balancer front end IP, examining health check logs, verifying backend VM configuration, and reading the Google Cloud documentation for a comprehensive understanding of health checks.

App Development, Serverless, Databases, DevOps

Beginner guides: Deploy Backend Applications to Cloud Run - This article provides a step-by-step guide on how to effortlessly deploy your Node.js REST API as container images to Google Cloud Run using Artifact Registry. It covers building Docker images, pushing them to Artifact Registry, granting Docker access to the registry, deploying the container to Cloud Run, and managing application versions. By following this guide, you can simplify the deployment process and focus on writing code without the hassle of managing infrastructure.

Archive your SAP Data to Google’s Cloud Storage - Integrating SAP ArchiveLink with Google Cloud Storage for Seamless Data Archival.

How Altice Labs enhanced API management with Google's Application Integration - Altice Labs enhanced API management using Google's Application Integration to streamline communication with fragmented systems, ensure precise control and protection of APIs, and orchestrate complex integration flows. Application Integration helped Altice Labs manage complexity by automating business processes and connecting any application with point-and-click simplicity.

Troubleshooting Cloud SQL/AlloyDB Creation Error: “Failed to create a subnetwork” - This article discusses an error that can occur when creating a Cloud SQL or AlloyDB instance due to the exhaustion of the allocated address space for Private Service Access (PSA).

From notebook to Cloud Run service in 10 minutes: applied to Gemini Function Calling - What if you see some code you really like in a Jupyter notebook? And you want to turn it into an app? Let’s learn how in this blog post.

Big Data, Analytics, ML&AI

Harnessing Google Cloud for Ericsson's Cognitive Network Solutions - Ericsson's Cognitive Software, powered by AI, optimizes network performance by identifying and resolving anomalies in cellular networks. The Cell Anomaly Detector, showcased at Mobile World Congress 2024, leverages Google Cloud's Vertex AI and BigQuery for advanced hyperscale cloud services.

Updates around Dataform assertions - New features and improvements to enhance the productivity of implementing tests.

Understanding the GA4 BigQuery Export Schema and Structure - A qualitative investigation into one of the weirdest data structures ever forced upon millions of innocent, unsuspecting analysts.

Replicating INDX-like tables with BigQuery Connector for SAP - BigQuery Connector for SAP enables near real-time replication of SAP data directly into BigQuery for real-time analytics with AI or machine learning. INDX-like tables in SAP are cluster tables designed for storing temporary or persistent data in a packed binary format, often compressed. However, replicating INDX-like tables to BigQuery using the connector results in unreadable data due to binary storage. To overcome this limitation, a designated target structure with specific fields and an assignment for an end of block event-related rule can be implemented to replicate readable data to BigQuery.

How to choose between dbt clone and dbt defer. And how we clone for all contributors. - This blog post discusses the challenges of using production data in development environments for dbt projects and explores two approaches offered by dbt to address these challenges: defer and clone.

How to build and execute a Dataflow Flex Template (Java) - A guide to building and executing a basic Dataflow Flex template in Java. It includes prerequisites, steps for building the template, and executing it.

Boost your log analysis with BigQuery vector search and LLMs - BigQuery vector search enables semantic similarity search in BigQuery. It can be used to analyze logs and asset metadata stored in BigQuery tables. For example, SRE or incident response teams can search for semantically similar logs to validate anomalies. These relevant search results can provide more complete context for large language models (LLMs) to identify relationships, patterns, and nuances. This can accelerate user workflows, from threat detection and investigation to network forensics, business insights, and application troubleshooting.

How dida automates sales processes with mathematics and machine learning - Enpal, a German greentech company, partnered with dida, an AI solutions provider, to automate part of its solar panels sales process using Google Cloud. dida leveraged Google Maps Platform, Cloud Storage, Compute Engine, and TensorBoard to build a custom AI solution that reduced the time required to generate quotes for prospective customers by 87.5%, from 120 minutes to just 15 minutes.

How Amdocs is revolutionizing telecom customer experience with AI - Amdocs and Google Cloud have partnered to deliver new AI services for communications service providers (CSPs). The Amdocs AI & Data Platform leverages Google Cloud's advanced data capabilities to unify real-time data collection and transform it into standardized business insights. This enables CSPs to enhance customer experiences, optimize operational efficiency, and stay competitive.

Automating Insights and Dashboard Report-Level Narratives using Vertex AI Gemini 1.5-Flash - This blog post explores how to automate insights and dashboard report-level narratives using Vertex AI Gemini 1.5-Flash LLM, Looker Studio, and Google BigQuery BQML. By leveraging these tools, businesses can streamline the process of extracting, analyzing, and presenting data, reducing the time it takes to understand and respond to changes in the business environment.

Quizaic — A Generative AI Case Study - Part 3— Prompting and Image Generation.

Slides, Videos, Audio

Security Podcast - #177 Cloud Incident Confessions: Top 5 Mistakes Leading to Breaches from Mandiant.

GCP Life Podcast - #67 In this episode we discuss; Google and OCI, Privileged Access Manager, Gitlab & GCP, Meta & Twiggy, Medibank EDR, Optus Fine, Apple Intelligence, Elon Musk and Apple, Apple & GCP.

Releases

GDCV for VMware - A vulnerability (CVE-2024-26584) was discovered in the Linux kernel that can lead to a privilege escalation on Container-Optimized OS nodes.

Apigee Advanced API Security - On June 17, 2024 we released an updated version of Advanced API Security. Shadow API Discovery, which is in preview, no longer requires separate creation of P4SA permissions in order to enable the functionality.

Apigee X - On June 20, 2024, we released an updated version of Apigee. This release includes a change in the user experience of selecting a physical location for control plane hosting when provisioning a Subscription or Pay-as-you-go Apigee organization with data regionalization enabled. Update Pay-as-you-go environment types using the Apigee UI in the Google Cloud console Apigee Pay-as-you-go customers can modify the type of an existing environment using the Apigee UI in the Cloud console.

Apigee Hybrid - On June 17, 2024 we released an updated version of the Apigee hybrid software, 1.10.5. For fixed vulnerabilities check release page.

AppEngine Flexible Ruby - Ruby 3.3 is now available in preview.

AppEngine Standard Ruby - Ruby 3.3 is now available in preview.

Assured Workloads for Goverment - During the Regional Controls Public Preview, the ComplianceRegime enum value has changed from FREE_REGIONS to REGIONAL_CONTROLS.

Batch - Documentation has been added to explain how to view resource metrics for your jobs in Cloud Monitoring. You can configure a job to automatically install the Ops Agent, which provides additional resource metrics in Cloud Monitoring.

BigQuery - Additional collation support for the NULLIF conditional expression has been added. You can now perform supervised tuning on a BigQuery ML remote model based on a gemini-1.0-pro-002 model. Global rate limits on BigQuery Omni connection creation and use have replaced the regional limits on AWS and Azure connections.

Bigtable - The Python client library for Bigtable now offers an asynchronous API for use with asynchronous applications.

Chronicle Security Operations - Google SecOps now integrates with Access Transparency. Google SecOps now supports data RBAC. lastAlertStatusChangeTime is added to the response of the GetRule Detection Engine API.

Chronicle SOAR - Release 6.3.7 is now in General Availability. Release 6.3.8 is currently in Preview. When running an imported playbook with an assigned user that doesn't exist, the playbook stops working when it gets to manual actions. Entity properties not showing in the platform if the key name contains the time string (ID #51599403).

Colab - You can now use customer-managed encryption keys (CMEK) to protect runtimes in Colab Enterprise.

Cloud Composer - ​​We are thrilled to announce the Public Preview launch of the new generation of Cloud Composer, Cloud Composer 3.

Confidential VM - Support for AMD SEV-SNP on Confidential VM instances is now generally available.

Compute Engine - Preemptible allocation quotas also apply to some temporary GPU VMs. The issue related to creating C2 sole tenant nodes with more than 60 CPUs. Generally available: You can now use the Require OS Config organization policy constraint to automatically enable VM Manager for all new VMs in your organization, folder, or project.

Confidential VM - Support for AMD SEV-SNP on Confidential VM instances is now generally available.

Contact Center AI Platform - Web SDK 2.21 is released For more information, see Web SDK changelog.

Database Migration Service - In Database Migration Service for heterogeneous Oracle migrations, you can now use the Promote action directly on the migration job details page to finalize your migration process.

Dataflow - Dataflow SQL is deprecated.

Dataform - You can now inspect past manual compilation results of a selected release configuration.

Dataproc Serverless - Dataproc Serverless for Spark: To fix compatibility with open table formats (Apache Iceberg, Apache Hudi and Delta Lake) the ANTLR version will be downgraded from 4.13.1 to 4.9.3 in Dataproc Serverless for Spark runtime versions 1.2 and 2.2 on June 26, 2024. Dataproc Serverless for Spark: Spark runtime version 2.2 will become the default Dataproc Serverless for Spark runtime version on August 1, 2024. New Dataproc Serverless for Spark runtime versions: 1.1.66 1.2.10 2.0.74 2.2.10.

Datastream - Datastream now supports the change tables CDC method for SQL Server sources. Datastream now supports the append-only write mode when ingesting data to BigQuery.

Deep Learning Containers - M122 release TensorFlow 2.16 container images are now available.

Deep Learning VM - M122 release Updated Nvidia drivers to version 550.90.07 to fix vulnerabilities.

Cloud Domains - You can migrate your Google Domains DNS settings and export your domain and email forwarding configurations if you use Google Domains as your DNS provider.

Cloud Functions - Cloud Functions has added support for a new runtime, Ruby 3.3, at the Preview release level. You can now enable execution ID in the logs for 2nd gen Python functions that use functions-framework >= 3.7.0 and 2nd gen Node.js functions that use functions-framework >= 3.4.0 by setting the runtime environment variable LOG_EXECUTION_ID to true.

Google Kubernetes Engine - The following vulnerabilities were discovered in the Linux kernel that can lead to a privilege escalation on Container-Optimized OS nodes: CVE-2024-26584 For more information, see the GCP-2024-036 security bulletin. (2024-R20) Version updates GKE cluster versions have been updated.

GKE - (2024-R20) Version updates The following versions are now available: 1.26.15-gke.1191000 1.26.15-gke.1300000 1.26.15-gke.1436000 1.27.14-gke.1093000 1.28.10-gke.1141000 1.29.5-gke.1192000 The following node versions are now available: 1.26.15-gke.1436000 1.27.14-gke.1093000 1.28.10-gke.1141000 1.29.5-gke.1192000 The following versions are no longer available: 1.28.7-gke.1026001 1.28.8-gke.1095000 Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.25 to version 1.26.15-gke.1320000 with this release.

Google Kubernetes Engine Rapid - (2024-R20) Version updates The following versions are now available in the Rapid channel: 1.26.15-gke.1436000 1.27.14-gke.1093000 1.28.10-gke.1141000 1.29.5-gke.1192000 1.30.1-gke.1500000 The following versions are no longer available in the Rapid channel: 1.26.15-gke.1381000 1.27.14-gke.1022000 1.28.10-gke.1058000 1.29.5-gke.1060000 Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.25 to version 1.26.15-gke.1390000 with this release.

Google Kubernetes Engine Regular - (2024-R20) Version updates The following versions are now available in the Regular channel: 1.26.15-gke.1381000 1.27.14-gke.1022000 1.28.10-gke.1058000 1.29.5-gke.1060000 The following versions are no longer available in the Regular channel: 1.26.15-gke.1090000 1.27.13-gke.1166000 1.28.9-gke.1209000 Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.25 to version 1.26.15-gke.1320000 with this release.

Google Kubernetes Engine Stable - (2024-R20) Version updates The following versions are now available in the Stable channel: 1.26.15-gke.1191000 1.26.15-gke.1300000 1.27.13-gke.1166000 1.28.9-gke.1209000 The following versions are no longer available in the Stable channel: 1.28.7-gke.1026001 1.28.8-gke.1095000 Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.28 to version 1.28.9-gke.1000000 with this release.

Marketplace Partners - You can now create custom private offers with flexible payment options, including a duration of up to 5 years, with an annual ratable commit drawdown schedule, if applicable.

Cloud Monitoring - In the Monitoring API, you can now configure documentation links for your notifications.

Network Connectivity Center - Include export filters is now available in public preview.

Cloud VPN - Cloud VPN lets you connect two VPC networks in different regions by using HA VPN gateways.

Pub/Sub Lite - Pub/Sub Lite is deprecated.

reCAPTCHA Enterprise - reCAPTCHA Enterprise Mobile SDK v18.6.0-beta01 is now available for Android.

Security Command Center - The Security Command Center Assets page will require new permissions On or after July 11, 2024, a new Identity and Access Management (IAM) permission will be required to view the Assets page in Google Cloud console.

Cloud Source Repositories - Effective June 17, 2024, Cloud Source Repositories isn't available to new customers.

Cloud Spanner - Named schemas is now generally available. Generated columns no longer require the STORED attribute.

Cloud SQL MySQL - You can now use the gcloud sql instances describe command or the SQL Admin API to retrieve a list of database versions that are available to your MySQL instance for upgrade.

Cloud SQL Postgres - You can now use the in-place major version upgrade feature to upgrade your Cloud SQL for PostgreSQL instance to PostgreSQL 16. You can now use the gcloud sql instances describe command or the SQL Admin API to retrieve a list of database versions that are available to your PostgreSQL instance for upgrade.

Cloud SQL SQL Server - You can now use the gcloud sql instances describe command or the SQL Admin API to retrieve a list of database versions that are available to your SQL Server instance for upgrade.

Cloud Storage - Hierarchical namespace for Cloud Storage buckets is now available in Preview. Renaming a folder in a bucket with hierarchical namespace enabled using command line is not supported. Cloud Storage FUSE now offers list caching, which is a cache for directory and file list, or ls, responses that improves list operation speeds.

Vertex AI - Starting on September 15, 2024, you can only customize classification, entity extraction, and sentiment analysis objectives by moving to Vertex AI Gemini prompts and tuning.

Vertex AI Workbench - The M122 release of Vertex AI Workbench user-managed notebooks includes the following: Updated Nvidia drivers to version 550.90.07 to fix vulnerabilities. The M122 release of Vertex AI Workbench instances includes the following: Updated Nvidia drivers to version 550.90.07 to fix vulnerabilities.

Virtual Private Cloud - Private Service Connect backends support using a cross-regional internal Application Load Balancer to access global Google API targets.

Workflows - Support for a Vertex AI API connector is available in Preview.

Workstation - Cloud Workstations is available in the australia-southeast2 region (Melbourne, Australia).