Welcome to issue #369 October 23rd, 2023

News

Customize load balancers for unique application needs with Service Extensions callouts - Service Extension callouts on Google Cloud Application Load Balancers, that were announced at Google Next ‘23, are now available in public preview. Service Extensions empower users to quickly and easily customize the data plane of Google Cloud Networking products.

Google Cloud and E-ISAC team up to advance security in the electricity industry - To enhance our commitment for this sector, today we are announcing a new partnership with the Electricity Information Sharing and Analysis Center (E-ISAC). Google Cloud is proud to be the first leading cloud provider to join the E-ISAC Vendor Affiliate Program.

Vertex AI adds Mistral AI model for powerful and flexible AI solutions - Mistral AI is teaming up with Google Cloud to natively integrate their cutting-edge AI model within Vertex AI.

Introducing Actions and Alerts in Advanced API Security - Actions and Alerts enhance Advanced API Security capabilities by reducing the time between threat detection and resolution through automation, minimizing the potential impact, and making your API security approach more proactive.

Google opens Falcon, a reliable low-latency hardware transport, to the ecosystem - As a hardware-assisted transport layer, Falcon is designed to be reliable, high performance, and low latency and leverages production-proven technologies including Carousel, Snap, Swift, PLB, and CSIG.

C3D VMs with 4th Gen AMD EPYC Processors now GA, with 45 percent performance increase - A general availability of the C3D machine series powered by 4th Generation AMD EPYC™ Processors (code-named Genoa) to Google Compute Engine (GCE) and Google Kubernetes Engine (GKE) customers.

Google Cloud (Apigee) named a Leader in the 2023 Gartner® Magic Quadrant™ for API Management

Google Public Sector brings citizen-centric technology to government

Improve Kubernetes cost and reliability with the new Policy Controller policy bundle - Google Kubernetes Engine (GKE) Policy Controller lets you enforce fully programmable policies for your clusters, where a policy bundle is a pre-built set of constraints that Google Cloud creates and maintains.

Google Public Sector and Carahsoft launch ATO Accelerator for ISV Partners - Google Public Sector and Carahsoft are launching an Authority to Operate (ATO) accelerator to support independent software vendors (ISVs) in their journey to achieve compliance authorizations, such as FedRAMP, DoD Impact Levels, and StateRAMP programs, to serve government agencies.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

How we’ll build sustainable, scalable, secure infrastructure for an AI-driven future

Stanford launches platform for high-risk data on Google Cloud - Stanford Research Computing Center (SRCC) teamed up with Stanford’s School of Medicine and Google Cloud to fund, design, and launch Carina, a customizable high-risk data platform for Stanford researchers.

Cloud CISO Perspectives: How boards can help cyber-crisis communications

Google Cloud C3D Review: Record-breaking performance with EPYC Genoa - Benchmarks for Compute Engine C3D instances that are using AMD Genoa CPUs.

Visualize Cloud DNS public zone queries using log-based metrics and Cloud Monitoring - This blog post will show you how to create a log-based metric using Cloud DNS public zone logs data.

Migrating an app from AWS to Google Cloud: A startup’s point of view - In this post AJ Ross, CTO at Fluxon, discusses the key considerations and comparative technologies for startups looking to move from AWS to Google Cloud.

Argo Events: How to Synchronize GCP Secret Manager with Kubernetes Secrets - This article explains how to use Argo events to synchronize secrets managed in the Google Cloud Platform Secret Manager with Kubernetes secrets.

Ansible External Secure Access & Dynamic Inventory on Google Cloud Platform - An example on how to use Google Cloud IAP with Ansible for conf management.

App Development, Serverless, Databases, DevOps

Digitalparking delivers dependable, secure parking services with streamlined IT - An overview of how Digitalparking migrated from on-prem to GCP.

Making of a Jump Start Solution: Generative AI for document summarization - A deeper look at one of the Jump Start Solutions - Generative AI Document Summarization.

ABAP SDK for Google Cloud enables Google Workspace APIs to be consumed from SAP applications - This article explains how to consume Google Workspace APIs natively in SAP applications.

Best practices for SAP performance benchmarking

Windows Server 2012 is welcome on Google Cloud, even after End of Support - In this blog post, we discuss your options for running a Windows Server 2012 instance on Google Cloud, even though it has reached EOS.

Running GCP PubSub emulator on a local Docker environment - A short list of things that need to be covered to run a Google Cloud Platform Pub/Sub emulator in a Docker container.

Understanding Direct VPC Egress for Cloud Run - This post describes how to set up a Cloud Run service connecting to a VPC resource using both Cloud Serverless VPC Access and the Direct VPC access.

Big Data, Analytics, ML&AI

Stanford center makes comprehensive COVID dataset available on Google Cloud - Stanford’s Center for Population Health Sciences is hosting COVID datasets on GCP.

Getting started with Feast on Google Cloud - This post is the second in a short series of blog posts about Feast on Google Cloud. In this post, we’ll provide an introductory tutorial for building an ecommerce reranking model for product recommendations using Feast on Google Cloud.

Evaluating tenancy strategies for Cloud Composer - This guide compares the pros and cons of different tenancy strategies for Cloud Composer.

Sharing Datasets across organizations with BigQuery Analytics Hub - This blog provides a detailed guide on setting up Analytics Hub for sharing datasets across organizations using Google Cloud, including essential steps and considerations.

GCP Cost Optimization: stop using Dataflow and use Pub/Sub subscriptions - Reduce costs from streaming pipelines by switching to Pub/Sub subscriptions.

Dataplex — Data Catalog | Auto Discovery and Metadata Harvesting | Part — 3.1 - This part of the Dataplex blog series focussed on the Data Catalog.

Dataplex — Data Catalog | Tagging — Operations and Business Metadata| Part — 3.2 - This part of the Dataplex blog series focussed on the Data Catalog.

Dataplex —Data Lineage | Data Governance | Part — 4 - An overview of Dataplex Lineage.

Breaking data silos with BigQuery Omni cross-cloud joins - This article explains how to use BigLake and BigQuery Omni to join data across BigQuery and S3 buckets without copying over data from S3 into GCP.

How Google attacks Apache Hive Data Warehouse - After Snowflake Google is now also aiming at Apache Hive.

Pay attention to this when UNNESTing in BigQuery - Here is a common confusion that can be encountered when working with ARRAYs in BigQuery.

Using BigQuery hashing functions - Today I want to highlight hash functions and showcase how I use them in BigQuery.

How generative AI fits into the entire software development lifecycle - A day in the life of a Google Cloud developer using Google AI assistance to get from idea to production.

Rapidly build an application in Gradio power by a Generative AI Agent - In this blog, we will discuss how to use Gradio, an open source frontend framework, with Vertex AI Conversation.

InstaDeep’s scalable reinforcement learning on Cloud TPU - n this article, we dive into the scaling capabilities of Cloud TPUs and their transformative impact on Reinforcement Learning workloads for both research and industry.

How GenAI saved days of development — an SQL query parser for BigQuery usage statistics - A 2.5 hours effort to create an SQL parser for usage statistics collection by using GenAI instead of days of coding.

Using Vertex AI Search to create a search widget within our web app based on our PDF files - Quickly build search app through VertexAI Search.

Slides, Videos, Audio

Security Podcast - #144 LLMs: A Double-Edged Sword for Cloud Security? Weighing the Benefits and Risks of Large Language Models.

Releases

Anthos Config Management - 1.16.2. Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 3e66ee2). The constraint template library includes a new template: K8sAvoidUseOfSystemMastersGroup. The constraint template library includes a new template: K8sPSPWindowsHostProcess. Policy Controller bundles have been updated to the following versions: asm-policy-v0.0.1: 202309.0, cis-k8s-v1.5.1: 202309.0, cost-reliability-v2023: 202309.0, nist-sp-800-190: 202309.0, nist-sp-800-53-r5: 202309.0, nsa-cisa-k8s-v1.2: 202309.0, pci-dss-v3.2.1: 202309.0, policy-essentials-v2022: 202309.0, psp-v2022: 202309.0, pss-baseline-v2022: 202309.0, pss-restricted-v2022: 202309.0. Updated the Open Telemetry image from 0.54.0 to 0.86.0 to address security vulnerabilities. Fixed a recurring transient error in the RootSync and RepoSync API.

Anthos clusters on VMware - Anthos clusters on VMware 1.16.2-gke.28 is now available. The following issue is fixed in 1.16.2-gke.28: Fixed the known issue where a non-HA Controlplane V2 cluster is stuck at node deletion until it timesout. Anthos clusters on VMware 1.14.9-gke.21 is now available. The following issues are fixed in 1.14.9-gke.21: Fixed the known issue where a non-HA Controlplane V2 cluster is stuck at node deletion until it timesout.

Apigee X - On October 19, 2023, we released an updated version of Apigee. Looker Studio Integration This release includes the public preview of Looker Studio Integration, which connects Apigee data to Google's Looker Studio.

AppEngine Standard Python3 - Python 3.12 is now available in preview.

Google Cloud Armor - Cloud Armor for regional HTTP(S) load balancers is now Generally Available.

Batch - Documentation has been added to explain how to colocate the VMs for a job by using a compact placement policy. Documentation has been added to explain how to securely reference sensitive data in a job by using Secret Manager secrets for encryption.

BigQuery - Stored procedures for Apache Spark are now available without enrollment. The BigQuery migration assessment is now available for Apache Hive in preview. You can now use DLP functions to support encryption and decryption between BigQuery and DLP, using AES-SIV.

Cloud Composer - Airflow 2.6.3 is available in Cloud Composer images. Airflow 2.6.3 consolidates the logic for handling tasks that are stuck in the queued state: The [kubernetes]worker_pods_pending_timeout, [celery]stalled_task_timeout, and [celery]task_adoption_timeout Airflow configuration options are deprecated and merged into the [scheduler]task_queued_timeout option. Cloud Composer 2.4.6 images are available: composer-2.4.6-airflow-2.6.3 composer-2.4.6-airflow-2.5.3 (default) composer-2.4.6-airflow-2.4.3.

Compute Engine - Generally available: c3d-standard, c3d-highmem, c3d-highcpu, and c3d-standard-lssd machine types for general-purpose C3D VMs are generally available.

Data Fusion - The Cloud Data Fusion SAP SLT No RFC Replication plugin version 0.11.0 is available in the Hub in Cloud Data Fusion enterprise edition versions 6.8.0 and later.

Dataproc Metastore - Dataproc Metastore now supports multi-regional configurations.

Datastream - Support for the PostgreSQL ARRAY data type is now added in Datastream.

Cloud Domains - On September 7, 2023 Squarespace acquired all domain registrations and related customer accounts from Google Domains.

Error Reporting - You can now view error groups on your custom dashboards.

Eventarc - Eventarc support for internal HTTP endpoints as event destinations is available in Preview.

Cloud Filestore - Filestore enterprise singleshare backups for GKE are now generally available.

Cloud Firestore - For documents with many fields that don't require indexing, you can now add collection-level index exemptions on all fields in a collection group. The sum() and average() aggregation functions are now available.

Cloud Functions - Cloud Functions now supports the Python 3.12 runtime at the Preview release level.

Google Kubernetes Engine - GKE Infrastructure Dashboards and Metrics Packages are now generally available for both GKE Autopilot and Standard clusters with control plane version 1.27.2-gke.1200 and later. New Autopilot clusters created with versions 1.24.17-gke.2146000, 1.25.14-gke.1466000, and 1.26.9-gke.1494000 or later are now provisioned with e2-small default nodes, which are removed immediately after cluster creation. You can now use the GKE API to apply Resource Manager tags to your GKE resources. Compute resources can now be reserved in advance for use with GKE. (2023-R22) Version updates GKE cluster versions have been updated. Filestore Enterprise now supports backups on GKE, allowing you to make reliable copies of your data to be stored for later use.

GKE - (2023-R22) Version updates There are no version updates for 2023-R21.

Google Kubernetes Engine Rapid - (2023-R22) Version updates There are no version updates for 2023-R21.

Load Balancing - Service Extensions callouts are available for Google Cloud Application Load Balancers, excluding Classic.

Cloud Logging - You can now create log buckets in the us-west8 region.

Cloud Monitoring - You can now view error groups on your custom dashboards.

Cloud PubSub - Public preview: Pub/Sub BigQuery subscriptions now support BigQuery change data capture.

Security Command Center - Container Threat Detection, a built-in service of Security Command Center Premium, has launched a new detector, Unexpected Child Shell, in Preview.

Service Mesh - Managed Anthos Service Mesh. Managed Anthos Service Mesh 1.17 is rolling out in the rapid channel.

SAP Solutions - New SAP NetWeaver certification: C3D series of general-purpose machine types For use with SAP NetWeaver, SAP has certified the Compute Engine general-purpose machine types c3d-standard and c3d-highmem.

Cloud Spanner - Query Optimizer version 6 is generally available, and is the default optimizer version.

Cloud SQL MySQL - Cloud SQL supports InnoDB page compression for MySQL 5.7 and MySQL 8.0 and later.

Cloud SQL SQL Server - You can now import transaction log backups.

Cloud Storage - Changes to the Autoclass feature that were announced on July 17, 2023 begin taking effect today.

Cloud Tasks - Support for Customer Managed Encryption Keys (CMEK) is now available for Cloud Tasks.

Cloud Text-to-Speech - The Long Audio Synthesis API now supports the following languages: English, Spanish, French, German, Japanese, Hindi, Italian, Korean, Portuguese, Thai, Vietnamese, Danish, Filipino. There is no longer billing differentiation for Cloud Text-to-Speech Offline Custom Voice API calls.

Vertex AI - New Vertex AI Vector Search Console Vector Search has launched a console experience in Google Cloud for creating and deploying indexes, now available in Preview. Vertex AI Vector Search Improvements Vector Search has improved the initial index creation process for smaller indexes (<100MB), reducing time to build from about 1 hour to about 5 mins.