Welcome to issue #373 November 20th, 2023

News

Architecture Diagramming Tool - Architecture Diagramming Tool now generates Terraform for your sketches.

FinOps Sketchnotes: Introducing Cloud FinOps - Over the next several months, we’ll be featuring Sketch Notes like the one above designed to demystify Cloud FinOps and Google Cloud Platform services.

Introducing BigQuery cross-region replication: enhanced geo-redundancy for your data - Cross-region dataset replication allows you to easily replicate any dataset, including ongoing changes, across cloud regions.

Announcing enhancements to effective route views in Google Cloud console - Route tables: The unsung heroes of network routing.

Looker Studio brings powerful explorations, fresher data and faster filtering - Introducing new ways for analysts to provide business users with options to explore data and self-serve business decisions, expanding ways all our users can analyze and explore data.

Cloud Deploy adds pipeline automation and Cloud Run Jobs support - Cloud Deploy now also supports continuous deployment, the end-to-end automation of continuous delivery.

Memorystore for Redis Cluster is GA and provides up to 60 times more throughput and microseconds latency - With Memorystore for Redis Cluster, you get a fully-managed and fully open-source software (OSS) compatible Redis Cluster offering with zero downtime scaling (in or out), providing up to 60 times more throughput than Memorystore for Redis, with microseconds latency.

Use the edit functionality for easy management of CDF pipelines - When you edit a pipeline you've already deployed, you don't have to duplicate the pipeline and implement a versioning strategy across multiple pipelines. Instead, you edit a single pipeline and the versions are tracked for you.

Google researchers discover 'Reptar,’ a new CPU vulnerability - Today, we’re detailing the findings of Reptar (CVE-2023-23583), a new CPU vulnerability that impacts several Intel desktop, mobile, and server CPUs.

What’s new in Assured Workloads: Japan regions, move analysis capability - Expanded Assured Workloads availability to our Japan regions, A new analysis tool that can identify potential policy violations that need to be addressed before migrating resources to Assured Workloads.

Accelerating generative AI around the world with new data residency guarantees - Data residency guarantees continue GCP commitment to provide AI services with enterprise-grade scale, safety, security and privacy.

Early Registration Now Open for Google Cloud Next ’24 (April 9-11) in Las Vegas - Google Cloud Next is heading to Las Vegas in 2024 from April 9th to 11th.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Protecting your remote workforce with context-aware data loss rules and URL filtering

Creating a SaaS Platform on Google Kubernetes Engine - This post goes over the fundamentals of deciding what architecture to choose when building a SaaS platform on Google Kubernetes Engine.

Cloud CISO Perspectives: Why ISACs are valuable security partners - Google Cloud has announced multiple collaborations with sector-specific information sharing and analysis centers over the past 18 months, and it this post it will be discussed why these ISACs are valuable partners for Google Cloud and our industry.

Demystifying Cloud Pricing - Shedding some light on the cloud pricing models that apply to all major cloud service providers (CSPs), not just Google Cloud, and how you can use these insights to understand more about how you’re using your cloud platform.

How to choose the correct load balancer type - An overview of Cloud Load Balancer types on GCP.

Migrating CI/CD from Kubernetes to Compute Engine: a journey of cost efficiency and reliability - Transitioning from Kubernetes with Docker-in-Docker to Compute Engines with Instance Group (and autoscaling) led to significant savings and enhanced reliability.

Create tailored guardrails with custom organization policy constraints in GCP - Use custom organization policy constraints to implement customizable control.

FinOps Cost Management using Terraform Cloud Run Tasks - Learn how to manage Google Cloud costs using a Terraform Cloud Custom Run Task to dynamically control infrastructure as code deployments.

Evaluating EKM Performance Impact: Part 1 — Performance Test on GCP with Thales DPOD Service - This study directly compares the performance impact of using conventional KMS keys against external encryption keys in database operations.

Enhancing Cybersecurity with Security Command Center’s Attack Path Simulation and Attack Path - Protecting your Google Cloud Infrastructure just got easier with Security Command Center’s latest feature — Attack Path Simulations.

App Development, Serverless, Databases, DevOps

Building a cross-cloud architecture for Spanner - This blog explores how to use Google Cloud’s Cross-Cloud Interconnect to achieve latency, security, availability, etc.

A Cloud Run service was slow, here’s how we fixed it - Improving code in Cloud Run app.

Increase Compute Engine VM performance with custom queues - Maximizing the network performance on Compute Engine VMs by assigning custom queues per virtual network interface card (vNIC).

Be a Regional Persistent Disk monitoring superhero: How to know when you’re at RPO=0

How Cloud SQL helped Build Beyond with Gramercy Tech & Vatom

The portability and familiarity of PostgreSQL with the scale and reliability of Spanner - Looking under the hood at how the PostgreSQL interface for Spanner works and what that means for compatibility with PostgreSQL.

Automation is here for Cloud Deploy. - Here’s the final missing piece for complete progressive delivery capability in Google Cloud.

Introducing Konsol : Taking Firebase Console experience to the next level on Android devices - Explaining Konsol Android app which includes features Cloud Firestore, Test Lab, Firebase Project Management, Cloud Storage & FCM.

Introducing a new Eventarc destination — internal HTTP endpoint in a VPC network - Eventarc has added support (in public preview) for delivering events to internal HTTP endpoints in a Virtual Private Cloud (VPC) network.

Big Data, Analytics, ML&AI

Unlocking the power of semi-structured data with the JSON Type in BigQuery - This post explores the architectural concepts that power BigQuery’s support for semi-structured JSON.

Generative AI use cases to inspire your Startup or Small Business

Top five ways generative AI can drive value in capital markets

The four building blocks of responsible generative AI in banking

Build a Pose Generator with Firebase and Vertex AI Imagen API - Generate AI-powered images with Firebase and Imagen!

GA4 Sessionization and Traffic Source handling in BigQuery - Answering questions like which acquisition channel the user came from, how to handle multiple sources within the same session.

Duet AI Review: My Perception and Use Cases — Part 2 - This article is Part 2 of a two-part review on Duet AI.

Slides, Videos, Audio

Kubernetes Podcast - #211 etcd, with Marek Siarkowicz and Wenjia Zhang.

Security Podcast - #148 Decoding SaaS Security: Demystifying Breaches, Vulnerabilities, and Vendor Responsibilities.

Releases

AlloyDB - IAM authentication for AlloyDB is generally available (GA). You can now restrict an OAuth 2.0 access token so that it works only for AlloyDB authentication. You can now configure the AlloyDB Auth Proxy to automatically authenticate IAM-based database logins. AlloyDB Omni version 15.2.2 is available. The AlloyDB Omni Kubernetes Operator version 0.2.0 is available in Preview.

Anthos Config Management - 1.16.3. Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: a1f01f4 ). Policy Controller bundles have been updated to the following versions: asm-policy-v0.0.1: 202310.0, cis-k8s-v1.5.1: 202310.0, cost-reliability-v2023: 202310.0-preview, nist-sp-800-190: 202310.0, nist-sp-800-53-r5: 202310.0, nsa-cisa-k8s-v1.2: 202310.0, pci-dss-v3.2.1: 202310.0, policy-essentials-v2022: 202310.0, psp-v2022: 202310.0, pss-baseline-v2022: 202310.0, pss-restricted-v2022: 202310.0. The constraint template library's K8sPSPAllowedUsers, K8sPSPAllowPrivilegeEscalationContainer, K8sPSPAutomountServiceAccountTokenPod, K8sPSPCapabilities, K8sPSPFlexVolumes, K8sPSPForbiddenSysctls, K8sPSPFSGroup, K8sPSPHostFilesystem, K8sPSPHostNamespace, K8sPSPHostNetworkingPorts, K8sPSPPrivilegedContainer, K8sPSPProcMount, K8sPSPReadOnlyRootFilesystem, K8sPSPSELinuxV2, K8sPSPVolumeTypes, and K8sRequiredProbes no longer raise violations during updates of existing objects for immutable fields. Updated the Open Telemetry image from 0.86.0 to 0.87.0 to address security vulnerabilities.

Anthos clusters on Azure - The following vulnerabilities were discovered in the Linux kernel that can lead to a privilege escalation on Container-Optimized OS and Ubuntu nodes.

Anthos clusters on VMware - The following issues are fixed in 1.16.3-gke.45: Fixed a Cilium issue causing egress NAT to erroneously break long-lived connections. Anthos clusters on VMware 1.16.3-gke.45 is now available. The Prometheus and Grafana add-ons field, loadBalancer.vips.addonsVIP, is deprecated. The following vulnerabilities were discovered in the Linux kernel that can lead to a privilege escalation on Container-Optimized OS and Ubuntu nodes.

Assured Workloads for Goverment - The IL4 compliance program now supports the following products: Cloud DNS, Cloud Interconnect, Cloud Monitoring, Cloud Router, Cloud SQL, Cloud VPN, Pub/Sub.

BigQuery ML - The following BigQuery ML features for Vertex AI large language models (LLMs) are now generally available (GA): The SQL syntax for remote models has been updated to provide access to all text generation and text embedding LLMs (for example, text-bison-32k and textembedding-gecko-multilingual) and also to provide support for different LLM versions. Region support for text-bison* LLM models has been expanded. The following BigQuery ML point-in-time lookup functions are now generally available (GA). The following AI features in BigQuery are now in preview: The ability to process documents from BigQuery, and the ability to transcribe audio files from BigQuery object tables.

BigQuery - You can now see query performance insights about partition skew.

Chronicle - Multiple supported default parsers have changed, check release page for more information.

Cloud Composer - Starting December 1, 2023, in the europe-central2, northamerica-northeast1, us-west1, and us-west2 regions it will be possible to create new Cloud Composer 1 environments only in projects that already have Cloud Composer 1 environments. All Cloud Composer environment's GKE clusters are set up with maintenance exclusions for the following periods: From November 20, 2023 to November 29, 2023 (already configured) From December 20, 2023 to January 2, 2024 (will be configured in December) For more information, see Maintenance exclusions.

Compute Engine - Preview: When creating or modifying an on-demand reservation, you can configure reservations to be automatically deleted at a specific date and time.

Data Fusion - You can apply a patch revision version when you create a new Cloud Data Fusion instance by adding the optional --patch_revision argument to the gcloud beta data-fusion instances create command. You can update the patch revision version of an instance by adding the optional --patch_revision argument to the gcloud beta data-fusion instances update command.

Dataflow - Dataflow supports NVIDIA® L4 and NVIDIA® A100 80 GB GPU types. The Cloud Spanner to Vertex AI Vector Search template is generally available (GA). Dataflow jobs now scale to 4,000 worker VMs.

Dataproc - You can use CMEK (Customer Managed Encrytion Keys) with encrypted Dataproc cluster data, incuding persistent disk data, job arguments and queries submitted with Dataproc jobs, and cluster data saved in the cluster Dataproc staging bucket.

Deep Learning Containers - M113 release Miscellaneous bug fixes and improvements in Python 3.10 container images.

Cloud Deploy - You can now configure alerts for Cloud Deploy release render failures. Cloud Deploy now supports delivery pipeline automation, including automated release promotion and automated rollout phase advancement, in preview.

Eventarc - Eventarc is available in the me-central2 (Dammam, Kingdom of Saudi Arabia) region.

Google Kubernetes Engine - (2023-R24) Version updates GKE cluster versions have been updated. You can now run workloads on L4 GPUs in Autopilot clusters that use GKE version 1.28.3-gke.1203000 and later. Dynamic Workload Scheduler support on GKE through the Provisioning Request API launched in Preview in version 1.28. The following vulnerabilities were discovered in the Linux kernel that can lead to a privilege escalation on Container-Optimized OS and Ubuntu nodes.

GKE - (2023-R24) Version updates The following control plane and node versions are now available: 1.24.17-gke.2230000 1.24.17-gke.2266000 1.25.15-gke.1083000 1.25.15-gke.1115000 1.26.10-gke.1073000 1.26.10-gke.1101000 1.27.7-gke.1088000 1.27.7-gke.1121000 The following control plane versions are no longer available: 1.24.17-gke.2198000 1.24.17-gke.2211000 1.25.15-gke.1033000 1.25.15-gke.1049000 1.26.10-gke.1024000 1.26.10-gke.1038000 1.27.7-gke.1038000 1.27.7-gke.1056000.

Google Kubernetes Engine Rapid - (2023-R24) Version updates The following versions are now available in the Rapid channel: 1.24.17-gke.2230000 1.24.17-gke.2266000 1.25.15-gke.1083000 1.25.15-gke.1115000 1.26.10-gke.1073000 1.26.10-gke.1101000 1.27.7-gke.1088000 1.27.7-gke.1121000 1.28.3-gke.1118000 1.28.3-gke.1203000 The following versions are no longer available in the Rapid channel: 1.24.17-gke.2198000 1.24.17-gke.2211000 1.25.15-gke.1033000 1.25.15-gke.1049000 1.26.10-gke.1024000 1.26.10-gke.1038000 1.27.7-gke.1038000 1.27.7-gke.1056000 1.28.3-gke.1098000 Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.23 to version 1.24.17-gke.2230000 with this release.

Cloud Monitoring - Observability for Google Kubernetes Engine: The Observability tab for a GKE cluster adds a dashboard for GPU metrics. A new query interface for creating charts is now in Public Preview.

Network Connectivity Center - The Advanced Data Networking (ADN) traffic is accounted only for large-sized flows (approximately >20 Kbps) that cross VPC boundaries.

reCAPTCHA Enterprise - reCAPTCHA Enterprise Mobile SDK v18.4.0 is now available for iOS.

Cloud Run - Cancelling a currently running job execution is now at general availability (GA). Deploying sidecar containers to your Cloud Run service is now at general availability (GA).

Service Mesh - 1.19.x. 1.19.3-asm.4 is now available for in-cluster Anthos Service Mesh. 1.18.x. 1.18.5-asm.2 is now available for in-cluster Anthos Service Mesh. 1.17.x. 1.17.8-asm.4 is now available for in-cluster Anthos Service Mesh. 1.16.x. 1.16.7-asm.14 is now available for in-cluster Anthos Service Mesh.

SAP Solutions - Version 2.7 of Google Cloud's Agent for SAP is generally available (GA).

Cloud Spanner - Cloud Spanner now supports automatic cleanup of long running transactions (in Preview). Cloud Spanner now supports Hibernate ORM 6.3 in GoogleSQL Hibernate dialect. Cloud Spanner now provides an integration workflow with Vertex AI Vector Search to enable vector similarity search on data stored in Spanner. Managed autoscaling for compute capacity on Cloud Spanner instances is now in preview.

Cloud SQL MySQL - The demote API is now available. Cloud SQL for MySQL now supports minor version 8.0.35.

Cloud SQL Postgres - The demote API is now available.

Cloud SQL SQL Server - Cloud SQL supports the bulk insert functionality of SQL Server for importing data.

Cloud Storage - New bandwidth quotas are now in effect.

Cloud TPU - Cloud TPU now supports TensorFlow 2.15.0, which adds support for PJRT.

Vertex AI - Vertex AI Feature Store The following features of the new and improved Vertex AI Feature Store are now generally available (GA): Feature Registry: Register your feature data sources in BigQuery by creating feature groups and features, Cloud Bigtable online serving: Serve features from one or more BigQuery data sources. Numerical filtering available in Vertex AI Vector Search With Vector Search you can restrict results by "filtering" your index results.

VMware Engine - Google Cloud console experience for VMware Engine: You can use the Google Cloud console to manage your VMware Engine environments without opening another tab. VMware Engine network: Further simplification of the networking architecture and experience in VMware Engine removes the need for private service networking. Integrated networking: Private cloud deployment is now just one simple step. Advanced VPC Peering: Virtual Private Cloud network peerings define network connectivity between VMware Engine networks, Google VPCs, and other services. Increase to the default VPC Peer count: Any standard VMware Engine network now supports 25 VPC Peers by default. Integrated Cloud DNS for workloads (DNS Bindings): Bi-directional Cloud DNS capabilities that enable DNS resolution for VMware Engine workloads, delivering enterprise needs in a simplified and more streamlined manner. DNS Server IP: Workloads within your private cloud can now use native Cloud DNS for DNS resolution. Management DNS for private clouds: Automatic Management DNS Peering is now Automatic Management DNS for Private Clouds. External access rules: Control access to external IP addresses. (Legacy Networks) DNS forwarding rules: Allows configuration of management appliance DNS resolution for private clouds attached to legacy VMware Engine networks. ESXi (NSX-T Distributed Log Forwarding): You can now configure both ESXi logs, including NSX-T Distributed Firewall (DFW) Logs, to a remote syslog server. Finer-grained access controls for additional resources: VMware Engine provides finer-grained, per-action access controls for actions performed on new resources added. Additional Google Cloud CLI and VMware Engine API Endpoints: More capabilities delivered using VMware Engine API and Google Cloud CLI enables you to programmatically manage VMware Engine environments, including VMware Engine API and Google Cloud CLI functions for managing the new networking model, network peering, external access rules and external IP service, consumer DNS, and more. DNS Profiles: Existing DNS Profiles will be migrated to each private cloud in which the DNS Profile was assigned. Firewall Tables: Existing firewall tables and rules have been migrated to external access rules. Elevate privilege option is no longer available. Announced August 10, 2022: Removed ability to manage point-to-site (P2S) VPN gateways for projects with existing P2S VPN gateways.

Workflows - Support for a Kubernetes API connector is available in Preview.