Welcome to issue #358 August 7th, 2023

News

What's new in Bigtable observability - Learn about new tools and metrics for Cloud Bigtable including query stats, high-granularity metrics, and table stats.

Introducing Personalized Service Health: Upleveling incident response communications - Personalized Service Health sends custom granular alerts about Google Cloud service disruptions, and integrates with incident management tooling.

Announcing public binaries for Spack on Google Cloud - Introducing a new Google-hosted build cache for Spack, backed by GCS, designed to significantly reduce software installation time in HPC environments.

Simplify data lake pipelines with new Pub/Sub Cloud Storage subscriptions - With new Pub/Sub Cloud Storage subscriptions, you can write raw streaming data into Cloud Storage without having to perform any transformations.

Introducing predictable cost options for Cloud Data Loss Prevention - Cloud DLP now offers a new pricing model for the discovery service, allowing you to choose the option that best fits your needs.

Unlock real-time observability for Vertex AI with Datadog - Datadog’s observability solution can now monitor, analyze and optimize ML model performance in production for Vertex AI.

---

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Proactively manage your subnet IP address allocation with Network Analyzer - Explore how to better understand IP address utilization, with Network Analyzer, using the new IP utilization summary insight.

Cloud CISO Perspectives: Late July 2023 - Threat actors know that our health systems and the people they treat are vulnerable to cyber threats. Guest author Taylor Lehmann discusses the existential threat that healthcare faces, and how cloud can help.

Granting Temporary Access in Google Cloud - This blog post explains how to set temporary access for a GCP project via conditional IAM.

Why you should integrate Chronicle SIEM with your GCP environment - This post explores several powerful features you can use with Chronicle SIEM when you link your tenant to a GCP Project.

How to set hard payment/spending/cost limits for Google Cloud Platform projects - This article explains how to configure and test a hard spending limit for the Google Cloud Platform.

Cloud Custodian for Compliance As Code and Auto-Remediation on Google Cloud Platform (GCP) - Using Cloud Custodian to implement your own Compliance as Code and Auto-Remediation strategy on Google Cloud.

A Deep Dive into the Google Cloud DNS - An in-depth overview of Cloud DNS.

Media CDN : Configuring failover with path rewrites - This blog post explains how to set backup origin during failover for Media CDN.

Enhanced Security for Egress Web Traffic: Secure Web Proxy on Google Cloud Platform - This article shows how to set up a Secure Web Proxy gateway in a Hub and Spoke network topology and guides you on verifying the setup in a client virtual machine.

App Development, Serverless, Databases, DevOps

Building internet-scale event-driven applications with Cloud Spanner change streams - Explore Cloud Spanner change stream and see how they can help you build your event driven apps.

A definitive guide to the Database Observability with Cloud SQL: Part 1 - Utilize familiar PostgreSQL tools such as pgBadger and pg_stat_statements for database observability.

DevOps Awards winner Kakao Mobility on balancing speed and stability - Kakao Mobility improved the developer experience and service reliability with DORA principles and by adopting microservices and Anthos Service Mesh.

DevOps Awards winner Decathlon Digital on “aligning to accelerate”

Simplifying creation of Go applications on Google Cloud - We’ve released four gonew templates to help developers bootstrap Go applications for several common use cases using Google Cloud services.

Working with Incoming Data in Cloud Workflows - Build a workflow on Google Cloud using pub/sub, Eventarc, and Workflows.

Building Scalable Serverless Applications: Hosting .NET Minimal APIs with Google Cloud Functions - An example of creating and deploying a Cloud Function in .NET.

My Cloud Workstation productivity setup - Learn how a Cloud Workstation image can be customized to include oh-my-zsh, k9s, custom tools and anything else developers dream of.

Big Data, Analytics, ML&AI

GCP — Create Custom Bigquery Linage using DataCatalog Python API — Big Data Processing - Create Custom Data Linage for your Bigquery tables using Dataplex data linage Python client.

Introducing BQFlow ETL - BQFlow is a Python library that moves data between Google APIs and BigQuery with minimal overhead and configuration.

BigQuery partitioning - by time-unit column or by ingestion time - What’s the best way to partition time-series data in BigQuery? By time-unit column or by ingestion time? Daily or hourly? It depends, keep reading to learn trade-offs, pitfalls, and other traps.

How to run inference workloads from a Dataflow Java pipeline - Learn how to run ML inference using Google Cloud Dataflow Java, Go or Python.

How to build and execute AI use cases at the edge - Google Distributed Cloud enables customers to run AI and other modern workloads in the edge locations and data centers.

Predicting and mitigating weather risk to your business with BigQuery and Weather Source - Built on BigQuery, Weather Source makes weather analytics simple and accessible so organizations can understand how weather impacts their business.

Govern Vertex AI Generative AI / Large Language Model Access With Quotas - Roleless access control.

SQL-only LLM for text generation using Vertex AI model in BigQuery - Learn how to use a Large Language Model on a source-code dataset as large as 2.8 million open source GitHub repositories for code summarization and more.

Deploy your custom Knowledge Base Assistant powered by Vertex AI and Pinecone - Creating a Knowledge Base Assistant with GCP products.

Vertex AI Vizier for fewer repetitions of costly ML training - This blog post explains how to reduce cost for ML training and other optimization processes: the Black Box optimization workflow with Google Vertex AI Vizier.

Various

IT pros’ top 5 challenges, ranked — and how Google Cloud Next can help you slay them - Google Cloud Next ‘23 will be a tremendous event for architects and IT pros — here are the ‘must see’ items during the show.

5 reasons why DevOps, IT Ops, Platform Engineers, and SREs should attend Google Cloud Next - Google Cloud Next is a must-attend event for DevOps, IT Ops, Platform Engineers, and SREs.

Slides, Videos, Audio

Security Podcast - #132 EP132 Chaos Engineering for Security: How to Improve Software Resilience with Kelly Shortridge.

GCP Life Podcast - #45 “Why is it flat?” – In this episode we discuss; Pixel Watch Mini Review, Austpost Safety, Deploy Parameters, Cleanup Policies, GCP Profit, APRA, HWL Ebsworth, Zenbleed Expoloit, SA Government AI Trial, EU and AI regulation.

Releases

Anthos clusters on bare metal - 1.13. Release 1.13.10 Anthos clusters on bare metal 1.13.0 is now available for download. Functionality changes: Upgraded local volume provisioner to v2.5.0. Fixes: Fixed an issue where the apiserver could become unresponsive during a cluster upgrade for clusters with a single control plane node. Fixes: The following container image security vulnerabilities have been fixed: CVE-2009-5155 CVE-2015-8985 CVE-2019-9511 CVE-2019-9513 CVE-2020-11080 CVE-2020-13844 CVE-2022-23524 CVE-2022-23525 CVE-2022-23526 CVE-2022-36055 CVE-2023-0464 CVE-2023-0465 CVE-2023-0466 CVE-2023-1667 CVE-2023-2283 CVE-2023-2454 CVE-2023-2455 CVE-2023-2650. Known issues: For information about the latest known issues, see Anthos clusters on bare metal known issues in the Troubleshooting section.

Anthos clusters on Azure - You can now launch clusters with the following Kubernetes versions. Anthos Multi-Cloud is disabling the unauthenticated kubelet read-only port 10255 for node pools in our next minor release with Kubernetes v1.27.

Apigee X - On August 3, 2023, we released an updated version of Apigee X. Previously, Advanced API Security scores didn't evaluate proxies calling shared flows via flow hooks and the FlowCallout policy in the proxy.

BigQuery - BigQuery now supports using manifest files for external tables. With Analytics Hub, you can now track usage metrics of your shared datasets. Cloud console updates: The following features are now available in preview: On the Welcome page, in the Recently accessed section, you can view your 10 most recently accessed resources. You can now GRANT or REVOKE access to materialized views with a SQL statement. BigQuery Omni is now available in the AWS - US West (Oregon) (aws-us-west-2) and the AWS - Europe (Ireland) (aws-eu-west-1) regions. BigQuery Storage Write API multiplexing is now generally available (GA).

Billing - Cloud Billing Pricing API now available in preview You can now get list prices, as well as account-specific custom prices and discounts for Google Cloud stock keeping units (SKUs) with the Pricing API (preview).

Cloud Composer - It is now possible to enable and disable High Resilience mode for an existing environment.

Compute Engine - Preview: You can create C3 VMs with Local SSD attached using new machine types (-lssd). Preview: H3 VMs, designed for compute-intensive high performance computing (HPC) workloads, are now in preview.

Config Connector - Config Connector version 1.107.0 is now available. Added support for customization on cnrm-webhook-manager pod replicas. Optimized HPA rule for cnrm-webhook-manager with targetCPUUtilizationPercentage adjusted from 90 to 70. Added GOMEMLIMIT environment variable (default value is set to 110MiB) to the webhook container in cnrm-webhook-manager. Resource DataflowFlexTemplateJob(v1beta1): Added spec.additionalExperiments field. Resource BigQueryDataset(v1beta1): Added spec.access.routine field.

Dataplex - Dataplex is available in the following regions: Los Angeles (us-west2) Salt Lake City (us-west3) Las Vegas (us-west4) Columbus (us-east5) Santiago (southamerica-west1) Finland (europe-north1) Warsaw (europe-central2) Madrid (europe-southwest1) Paris (europe-west9) Jakarta (asia-southeast2) For more information, see Locations and Pricing.

Dialogflow - Dialogflow CX has added a new prebuilt component for feedback.

Cloud Data Loss Prevention - You can enrich your manually curated metadata in Dataplex with insights gathered from Sensitive Data Protection data profiles.

Document AI - v1. Launched the following Document AI Workbench features: Create and train models programmatically with more public APIs, including: DatasetSchema APIs: UpdateDatasetSchema, GetDatasetSchema.

Cloud Functions - The following new regions are now available: europe-southwest1, europe-west8, europe-west9, us-south1, and us-east5.

Google Kubernetes Engine - GKE Autopilot supports the creation of certificate signing request (CSR) objects on new clusters with version 1.27 or later, as long as those CSRs do not conflict with system components identities, and Google-managed IAM service accounts. You can now run workloads on A100 80GB GPUs in Autopilot clusters that use GKE version 1.27 and later.

Live Stream API - Slates are now supported.

Cloud Logging - Ops Agent version 2.36.0 introduces support for Compute Engine Arm VMs that are running Debian 11 (Bullseye).

Migrate for Compute Engine 4.8 - 5.0. Preview: Migrate to Virtual Machines lets you migrate disks from source virtual machine (VM) instances to Persistent Disk volumes on Google Cloud.

Cloud Monitoring - Ops Agent version 2.36.0 introduces support for Compute Engine Arm VMs that are running Debian 11 (Bullseye). You can now snooze an alerting policy from an Incident details page related to the alerting policy.

AutoML Natural Language - This legacy version of AutoML Natural Language is deprecated and will no longer be available on Google Cloud after December 31, 2023.

Retail Recommendations AI - Retail API: View search performance tiers and performance upgrade requirements Text query search and browse search have different performance tiers that increasingly improve your search results.

Cloud Run - Accessing a service that's prohibited by the Internal or Internal and Cloud Load Balancing ingress setting now results in a 404 rather than 403 error code.

Security Command Center - Attack exposure scores and attack paths released to General Availability The attack path simulation feature that generates attack exposure scores and attack paths for findings that expose your high-value resources is now released to General Availability. AI-generated summaries of the simulated attack paths for Security Command Center findings are released to Preview. The Security Health Analytics detector NETWORK_POLICY_DISABLED now recognizes network policies that are implemented by using GKE Dataplane V2.

Anthos Service Mesh - 1.18.x. 1.18.2-asm.0 is now available for in-cluster Anthos Service Mesh. Anthos Service Mesh now supports setting up a mesh containing multiple Anthos clusters on AWS. 1.15.x. Google has ended support for in-cluster Anthos Service Mesh 1.15 following the official policy.

SAP Solutions - New SAP HANA certifications: C3 series of general-purpose machine types For running SAP HANA OLAP and OLTP scale-up workloads, SAP has certified the Compute Engine general-purpose machine types c3-standard and c3-highmem. New SAP NetWeaver certifications: C3 series of general-purpose machine types For use with SAP NetWeaver, SAP has certified the Compute Engine general-purpose machine types c3-standard and c3-highmem.

Cloud SQL MySQL - Cloud SQL now supports multiple categories of API rate quotas. The Cloud SQL Node.js Language Connector is now available for public preview. You can now migrate large MySQL databases from external sources to Cloud SQL for MySQL faster using Database Migration Service.

Cloud Storage - You can now use locational endpoints to perform operations in a manner that's compliant with International Traffic in Arms Regulations (ITAR).

Vertex AI - Prebuilt containers to perform custom training with TensorFlow 2.12 are now generally available (GA). Updated prebuilt images for Tensorflow 2.11 are now available. Vertex AI Tensorboard pricing has changed from a per-user monthly license of $300 per month to $10 GiB per month for storage of your logs.

VPC Service Controls - Preview stage support for the following integration: Certificate Manager.

Virtual Private Cloud - Connection reconciliation is available in General Availability.

AI Platform Pipelines - This legacy version of AI Platform Pipelines is deprecated and will no longer be available on Google Cloud after July 31, 2024.

AI Platform Prediction - This legacy version of AI Platform Prediction is deprecated and will no longer be available on Google Cloud after January 31, 2025.

AI Platform Training - This legacy version of AI Platform Training is deprecated and will no longer be available on Google Cloud after January 31, 2025.