Welcome to issue #370 October 30th, 2023

News

Connecting the South Pacific with new subsea cables - Announcements of plans for the South Pacific Connect initiative, which will deliver two new transpacific subsea cables — Honomoana and Tabua — to help increase the reliability and resilience of digital connectivity in the Pacific.

GKE Stateful High Availability (HA) Controller now in public preview - The Stateful HA Operator is a fully automated solution that reduces the toil of customizing your application to meet its availability needs.

Meet Gen AI Navigator: Your personalized guide to adopting generative AI - Gen AI Navigator is a practical tool designed to help businesses of any size or scale realize the true potential of Gen AI in the year ahead.

Kubeflow Pipelines v2: Making ML pipelines easier, faster, and more scalable - Release of Kubeflow Pipelines V2 and what's new.

What’s new with Cloud Firewall Standard - A General Availability of the fully qualified domain name (FQDN) feature for Cloud Firewall.

Save up to 52 percent on your database compute costs with AlloyDB committed use discounts - By committing to a consistent amount of compute usage for a one- or three-year period, you can get deep discounts: a 25% discount for a one-year commitment and a 52% discount for a three-year commitment.

Empowering allergy sufferers with our Pollen API, now generally available - By leveraging our now generally available Pollen API, pharmaceutical companies can cultivate personalized experiences that are incredibly valuable for their allergy medication customers. By delivering hyperlocal pollen levels, detailed plant information, and heatmap visualizations of pollen levels, they can tailor treatment options and services to meet individual exposure and pollen sensitivity.

Introducing the new Places API with access to EV, accessibility features, and more - New features in Text Search, Place Details and Photos, and Nearby Search–which now includes EV charging–make it easier for developers to surface helpful information about the world in the products they build.

Build immersive maps at scale with Photorealistic 3D, 2D, and Street View Tiles, now in GA - Photorealistic 3D Tiles, 2D Tiles, and Street View Tiles are general available.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Building core strength: New technical papers on infrastructure security - Building on the principles laid out in Building Secure and Reliable Systems, we are excited to announce a new series of technical whitepapers on infrastructure security.

Provision cross-region internal Application LB with automatic failover, health checks and geo-routing - An example of using cross-regional Application Load Balancer on workloads in GKE.

Elite performance in demand-based downscaling: The power of workload autoscaling - This blog outlines the significance of workload autoscaling and outlines steps that you as a developer or platform admin can follow to harness its benefits.

Empowering all to be safer with AI this Cybersecurity Awareness Month - In acknowledgment of Cybersecurity Awareness Month, now in its 20th year, we recently shared our progress across a number of security efforts, and announced a few new technologies that help us keep more people safe online than anyone else.

Estimating the Cost of Kubernetes Deployment on GKE Autopilot - Predicting the Cost of Your Kubernetes Deployment Prior to Cloud Migration.

Lower GCP Cost: Schedule GKE Cluster to Start and Stop - Reducing non-production GKE cluster by starting/stopping regularly with Cloud Scheduler.

The new Kubernetes Gateway API with Istio and Anthos Service Mesh (ASM) - This blog posts explains Kubernetes Gateway in the context of Istio, Anthos Service Mesh (ASM) and GKE.

Proactively Detect Network Misconfigurations in Google Cloud with Network Analyzer - Use Network Intellignece and Google Cloud Network Analyzer to proactively detect network misconfigurations.

Pulumi after years of Terraform - Is it worth changing IaC tool?

App Development, Serverless, Databases, DevOps

Success through culture: why embracing failure encourages better software delivery - Things break. That’s life. When things don’t go as planned, it’s what happens next that’s important.

How NCR Voyix reduced cost and complexity with Log Analytics from Cloud Logging

How to build a conversational AI experience using generative AI to improve employee productivity - Using Google Cloud generative AI features in Dialogflow, you can create a lifelike conversational AI agent that empowers employees to retrieve the most relevant information from internal or external knowledge bases.

Understanding Alloy DB connectors - AlloyDB connectors provide an automated mTLS connection with Cloud IAM integration for your application and they are available in library and binary form.

Why Google Cloud Firestore is a Game-Changer? - An overview of Cloud Firestore.

How to use Apigee Standard + Identity Platform to expose and secure your APIs with OAuth in Google Cloud - In this tutorial we will use Apigee & Identity Platform to secure a Cloud Run API with OAuth.

Apigee-X Network : Part 1 — Fundamentals - A deep dive into the fundamentals of Apigee X’s network design.

Big Data, Analytics, ML&AI

Interactive and conversational search with Google Cloud and Elasticsearch - A new search experience for retailers using generative AI with Vertex AI and ElasticSearch. This enhanced interface offers users an interactive, conversational experience that summarizes pertinent data and tailors responses based on each customer's unique needs, all while drawing upon the retailer's public and internal knowledge.

Shining a light in the dark: Measuring global internet shutdowns - Censored Planet Observatory’s goal: Make network censorship data universally accessible and useful.

What's new in Dataflow: Intelligence, developer experience, performance and more - An overview of Dataflow’s key new capabilities.

Meeting Security Requirements for Dataflow pipelines — Part 3/3 - This blog post is part of a set of articles providing an in-depth analysis of GCP’s security practices to deploy your Apache Beam pipeline on Cloud Dataflow.

Quick way to learn the basics of Apache Beam Programming - Coding exercises to learn Beam concepts in Python.

Serving open-source large language models efficiently on Vertex AI Model Garden - An updated LLM-efficient serving solution that improves serving throughput in Vertex AI.

iGEM, GCP, and Neo4j: Where Synthetic Biology Meets Artificial Intelligence - Reorganize the iGEM Parts Registry for synthetic biologists.

How to Avoid Five Common Mistakes in Google BigQuery / SQL - While working with BigQuery for years, I observed 5 issues that are commonly made, even by experienced Data Scientists.

Dataplex — Data Lineage for Spark Applications | Data Governance | Part — 4 - This is in continuation of the Data lineage Part 4.0 blog post focussing on the Data Lineage for the Spark Applications.

Using Large Language Models at Scale with Google BigQuery, Dataform and Vertex AI - This article goes through setting up a data pipeline using only SQL to run reviews through a large language model (LLM) to identify the sentiment of the said review.

Various

The overwhelmed person’s guide to Google Cloud: week of Oct 23 - A weekly curation of the most helpful blogs, exciting new features, and useful events coming out of Google Cloud.

Releases

Data Fusion - Cloud Data Fusion supports patch revisions. The Cloud Data Fusion version 6.7.3.1 patch revision is generally available (GA).

Dataflow - The Cloud Spanner to BigQuery template for batch pipelines is generally available (GA).

Dataproc Metastore - Dataproc Metastore is now available in the me-central2 (Dammam) region.

Dataproc Serverless - Announcing the General Availability (GA) release of Dataproc Serverless GPU accelerators. New Dataproc Serverless for Spark runtime versions: 1.1.36 2.0.44 2.1.23. Dataproc now collects the dataproc.googleapis.com/job/yarn/vcore_seconds and dataproc.googleapis.com/job/yarn/memory_seconds job-level resource attribution metrics to track YARN application vcore and memory usage during job execution. Dataproc now collects a dataproc.googleapis.com/node/yarn/nodemanager/health health metric to track the health of individual YARN node managers running on VMs.

Dataproc - Dataproc now collects the dataproc.googleapis.com/job/yarn/vcore_seconds and dataproc.googleapis.com/job/yarn/memory_seconds job-level resource attribution metrics to track YARN application vcore and memory usage during job execution. Dataproc now collects a dataproc.googleapis.com/node/yarn/nodemanager/health health metric to track the health of individual YARN node managers running on VMs.

Cloud Deploy - Cloud Deploy now uses Skaffold 2.8 as the default Skaffold version for all target types. You can now deploy Cloud Run jobs, in addition to Cloud Run services.

Dialogflow - On the week of October 30, 2023, auto speech adaptation (CX, ES) will be updated for non-English agents. Dialogflow CX has added the following prebuilt components: Account balance Account statement Alphanumeric collection Arrange appointment Authentication Credit card enrollment Credit card statement Disputes Financial services steering Lock card Lost and stolen card Make a payment Open account Transaction history.

Google Kubernetes Engine - The Cloud Storage FUSE CSI driver now enforces injected sidecar containers to follow the Restricted Pod security standard.

Cloud Logging - Ops Agent version 2.43.0 introduces support for Compute Engine Arm VMs that are running SLES 15 and OpenSUSE Leap 15.

Cloud Monitoring - Ops Agent version 2.43.0 introduces support for Compute Engine Arm VMs that are running SLES 15 and OpenSUSE Leap 15. You can configure your synthetic monitors to collect log data and trace data for your outbound HTTP requests when you use the generic template.

SAP Solutions - Cloud Storage Backint agent for SAP HANA version 1.0.31 Version 1.0.31 of the Cloud Storage Backint agent for SAP HANA is available.

Cloud Spanner - Cloud Spanner now supports FULL JOIN with USING in PostgreSQL-dialect databases. Cloud Spanner PostgreSQL now supports the SELECT DISTINCT statement.

Cloud SQL Postgres - You can now specify the SSL mode of your Cloud SQL instances, which gives you more accurate SSL encryption.

Cloud Storage - Turbo replication performance monitoring in the Google Cloud console has been moved and expanded. Managed folders are now available in Preview.

Cloud Text-to-Speech - Styles are now supported in Neural2 voices through SSML. Studio voices now support 5,000 bytes of either text or SSML input per synthesis request. Long Audio Synthesis now supports Studio voices. Long Audio Synthesis now supports SSML inputs.

Anthos Config Management - Config Controller now uses the following versions of its included products: Config Connector v1.110.0, release notes Anthos Config Management v1.16.1, release notes.

Apigee X - On October 24, 2023, we released an updated version of Apigee (1-11-0-apigee-7). With this release, the HeaderName element is available as a child element of Authentication. Bug ID Description 294293907 Fixed issue with Google authentication for gRPC-based target servers.

Artifact Registry - Artifact Registry remote repositories are now generally available. Artifact Registry virtual repositories are now generally available.

Batch - Documentation has been added to explain how to run dsub pipelines on Batch.

BigQuery - Custom data masking now supports an expanded list of functions, including SHA hash functions with salt.

Billing - Control access to single-project budgets If you are a billing account-level user and are creating a budget for a single project, you can now prevent project users such as Project Owners and Project Editors from making changes to the budget. Budgets for project users is now Generally Available Project users such as Project Owners, Project Editors, and Project Viewers in Google Cloud can now create budgets and stay on top of their cloud costs, without needing additional permissions to access Cloud Billing accounts.

Certificate Authority Service - v1. Certificate Authority Service is now available in the following region: europe-west10 For more information, see Certificate Authority Service locations.

Access Transparency - Access Transparency supports Vertex AI Workbench instances in the GA stage.

Compute Engine - Preview: Hyperdisk Balanced is now available in preview with H3 VMs. Preview: Project zonal metadata is custom project metadata that you can set exclusively for VMs in a specific zone in a project.

Config Connector - Config Connector version 1.111.0 is now available. Added support for ContainerAttachedCluster (v1beta1) resource. Added support for AlloyDBCluster (v1beta1) resource. Added support for AlloyDBInstance (v1beta1) resource. Added support for AlloyDBBackup (v1beta1) resource. Added name validation for ValidatingWebhookConfigurationCustomization and MutatingWebhookConfigurationCustomization CRDs. Added validation for duplicate webhooks in spec.webhooks list of the customizable ControllerResource and NamespacedControllerResource CRDs. Added errors on invalid webhook names into status of ValidatingWebhookConfigurationCustomization and MutatingWebhookConfigurationCustomization custom resources. Fixed an reconciliation issue in ComputeManagedSSLCert resource. Fixed issue of the retrieved maxWorkers in DataflowFlexTemplateJob resource. Graduated ValidatingWebhookConfigurationCustomization, MutatingWebhookConfigurationCustomization, ControllerResource and NamespacedControllerResource CRDs to v1beta1. Fixed an issue in ComputeForwardingRule resource when used with PSC. Resource AlloyDBCluster(v1beta1): Added spec.networkConfig field. Resource ComputeSubnetwork(v1beta1): Added status.internalIpv6Prefix field. Resource ComputeTargetHTTPSProxy(v1beta1): Added spec.serverTlsPolicyRef field. Resource ContainerCluster(v1beta1): Added spec.nodeConfig.fastSocket field. Resource ContainerNodePool(v1beta1): Added spec.nodeConfig.fastSocket field. Resource NetworkConnectivitySpoke(v1beta1): Added spec.linkedVPCNetwork field. Resource RunJob(v1beta1): Added spec.template.template.vpcAccess.networkInterfaces field. Resource RunService(v1beta1): Added spec.template.vpcAccess.networkInterfaces field. Resource SecretManagerSecretVersion(v1beta1): Added spec.isSecretDataBase64 field.