Welcome to issue #360 August 21st, 2023

News

Optimizing your cloud costs is even easier with Committed Use Discount recommendations - New Committed Use Discount (CUD) recommendations can help you find new cost-saving opportunities for resource-based and Flexible CUDs.

Automate your data warehouse migration to BigQuery with new data migration tool - The new, automated open-source data migration tool moves Teradata, Hive, Redshift, and Oracle data warehouses from on-premises to BigQuery.

Expanding the Google Cloud network observability partner ecosystem - We’re enhancing Google Cloud network observability in collaboration with our network performance monitoring (NPM) partners.

Cloud Deploy adds deploy hooks for easier rollouts - Use pre- and post-deployment hooks to run custom actions immediately before and after deployment as part of a CI/CD pipeline in Cloud Deploy.

Expanding our data processing commitments for Google Cloud and Google Workspace - At Google Cloud, we are committed to meeting our customers’ data processing and security needs. We are pleased to announce the next version of the Cloud Data Processing Addendum.

Increase your availability and maximize your Bigtable spend with committed use discounts - With new Cloud Bigtable committed use discounts, you can get up to a 40% discount on Bigtable compute capacity.

Announcing Direct VPC egress for Cloud Run: better performance and lower costs - Now with direct VPC egress, you can send traffic from Cloud Run services and jobs directly to a VPC without needing to proxy through a VPC connector.

---

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Building the next generation of zero trust with Sentima - Sentima uses advanced AI and machine learning tools on Google Cloud to develop a new generation of Zero Trust cybersecurity solutions.

Cloud CISO Perspectives: Early August 2023 - Cloud CISO Phil Venables explains why he considers himself a short-term pessimist, long-term optimist when it comes to cybersecurity.

Modular GKE Ingress with the Gateway API - Learn how to deploy a Gateway API on GKE to expose your application securely. This takes you through the steps to deploy gateway, HTTPRoute, and GCPGatewayPolicy.

How to set up observability for a multi-tenant GKE solution - Learn how to set up a GKE multi-tenant solution for observability using Log Router, and setting up a sink to route a tenant’s logs to their dedicated GCP project.

Getting started with Ray on Google Kubernetes Engine - Ray provides a simple API for building distributed, parallelized applications, especially for deep learning applications.

GCP - Cloud NAT Rules - An overview of Cloud NAT rules.

Breaking Free: Insourcing NFS PVCs into Kubernetes and Saying Goodbye to External Solutions - Explore a unique solution regarding hosting NFS servers in-house without relying on NAS services like Filestore.

Improve your Kubernetes security posture, with the Pod Security Admission (PSA) - This article describes how you can easily use Pod Security Admission controller to improve your Kubernetes security posture.

Connectivity Test with Network Intelligence Center in Google Cloud - What is GCP Network Intelligence Center?

App Development, Serverless, Databases, DevOps

Streaming Cloud Spanner changes with Apache Kafka at scale - Learn how to use the recently launched Cloud Spanner Change Streams Kafka connector to stream Cloud Spanner data into Kafka in a flexible and scalable way!

How YouTube uses Bigtable to power one of the world’s largest streaming services - YouTube uses Bigtable to store user activity for personalization, record metrics, and power reporting dashboards and analytics to name a few use cases.

tab32 elevates the healthcare and dental industry with Google Cloud - tab32 built its dental practice management platform on Google Cloud for secure data access, reduced costs, continuous care and improved interoperability.

Manage FHIR Data from Android App with Open Health Stack and Google Cloud - Build secure, scalable, compliant and data-driven healthcare mobile apps with Android-FHIR SDK, OHS and Google Cloud Healthcare API.

DevOps Award winner Moloco on ‘accelerating DevOps with machine learning’ - Moloco leverages the DORA four keys to validate the ML infrastructure it built on Google Cloud.

DevOps Awards winner ANZ on “going beyond the four keys” - The ANZx team knew that it needed to bring together relevant deployment data if it ever really wanted to fine-tune its software delivery performance.

Cl/CD for Gitlab repositories with Cloud Build repositories Gen2 - New capabilities of Cloud Build repositories (2nd gen). Connect with Gitlab repos and manage with Terraform.

Self-Service API consumer onboarding for Cloud Run with sidecar containers and Apigee - The newly added API management capability is transparent for the primary application container within the Cloud Run service.

How four companies are boosting Web3 innovation with Blockchain Node Engine - Four Blockchain Node Engine customers talk about how it freed them from worrying about how to run and scale their blockchain infrastructure.

Instacart migrates to Memorystore and sees a 23 percent reduction in latency and costs - By upgrading to Memorystore from self-managed memcached, Instacart reduced maintenance time, simplified operating procedures, and reduced costs.

Firestore Tips #1 & #2: How to Play God in Firestore, and Firestore vs. SQL - A basic overview of Firestore and comparison with SQL databases.

Firestore Tips 3–5: Properties any Document Needs & Handling Concurrent Document Updates - Securely updating keys and incrementing numbers “atomically”.

Python Logs Aren’t Code. They’re A Communication Tool. - Embrace logs as communication plus 3 non-negotiables you must include for a functional, transparent data pipeline.

How to create a SLO for Cloud Run programatically

Integrate Google Cloud Healthcare API with API Gateway for SMART-ON-FHIR - Cloud Healthcare API integration with API Gateway.

GenAI based Chatbot using Google Chat, Dialogflow, GPT and Google Cloud Platform - Creating a Google Chatbot that recommends a recipe based on a list of ingredients.

Big Data, Analytics, ML&AI

Using machine learning to help ZSL & Network Rail monitor and improve biodiversity near British railways - With British wildlife declining at unprecedented rates, ZSL used Vertex AI and Looker Studio to identify and map species along Network Rail’s estate.

Understanding Driver Pools in Dataproc - Learn about driver pools in Dataproc — a mechanism to scale application concurrency in Dataproc clusters.

No more waiting on hold: How Google Cloud Contact Center AI improves Sales Development operations - Since adopting Contact Center AI in 2021, Google Cloud call center agents have become 56% more efficient thanks to CCAI.

Get your BigQuery production sample, all self-serving - BigQuery, PubSub, Scheduler, Monitoring and more work together to make your DevOps processes easier with sampling data.

Unlock the power of geospatial analysis and visualization with BigQuery and Tableau - When it comes to geospatial data, BigQuery can help you store and analyze it, while Tableau can provide powerful visualization capabilities.

Implementing MLOps tools and processes for supply chain science at Wayfair - Wayfair migrated to Kubeflow-based Vertex AI Pipelines for orchestration while leveraging the internal tooling it built on top of Vertex AI.

Flexibility in Vizier’s Black Box Optimization - This article describes the ways to configure Vizier to accelerate your convergence to the best possible model.

How to Use Google’s PaLM 2 API with Python - Customize and integrate Google’s LLM in your application.

Various

Why security professionals should attend Google Cloud Next ‘23 - Get a preview of Google Cloud has lined up for security professionals at Next ‘23.

New Partner Delivery Excellence portfolio: Helping Partners deliver customer value - The new Partner Delivery Excellence portfolio is a rich set of resources to help Google Cloud Partners deliver high-quality Google Cloud projects.

Public sector sessions you won’t want to miss at Next ‘23!

How one telecom giant increases advertising conversions and saves millions annually - Virgin Media O2 is improving conversions and saving millions of dollars thanks to personalized advertising powered by Google Cloud and Zeotap.

Tips to enhance your prompt-engineering abilities - Prompts should be clear, concise, and informative to get the desired output from large language models (LLMs) and other generative foundation models.

Slides, Videos, Audio

Security Podcast - #134 How to Prioritize UX and Security in the Cloud: UX as a Security Capability.

Releases

Vertex AI - The Vertex AI Matching Engine public endpoint is now generally available (GA).

VPC Service Controls - Preview stage support for the following integration: Backup and DR Service.

Virtual Private Cloud - Private Service Connect backends with published service targets can be added to cross-region Application Load Balancers. VLAN attachments for Cloud Interconnect that have Dataplane v1 can access Private Service Connect endpoints from hybrid networks. Private Service Connect endpoints that have global access enabled can access published services that are based on the following load balancer configurations: Internal Application Load Balancer with global access enabled Regional internal proxy Network Load Balancer with global access enabled.

AlloyDB - Instance machine-type configuration has added an option for 96 vCPUs and 768 GB of RAM per node. AlloyDB now offers committed use discounts (CUDs) that provide 25% or 52% discount on AlloyDB's compute resources in exchange for your commitment, and you can continuously use them for one- or three-year terms, respectively.

Anthos clusters on bare metal - 1.15. Release 1.15.4 Anthos clusters on bare metal 1.15.4 is now available for download. Functionality changes: Audit logs are compressed on the wire for Cloud Audit Logs consumption, reducing egress bandwidth by approximately 60%. Fixes: Fixed an issue for clusters configured with manual load balancing where CA rotation reported that there were no (0) control plane nodes. Fixes: The following container image security vulnerabilities have been fixed: High-severity container vulnerabilities: CVE-2017-11468 CVE-2023-0464 CVE-2023-2650 Medium-severity container vulnerabilities: CVE-2019-9511 CVE-2019-9513 CVE-2020-11080 CVE-2020-13844 CVE-2022-23524 CVE-2022-23525 CVE-2022-23526 CVE-2022-36055 CVE-2023-0465 CVE-2023-0466 Low-severity container vulnerabilities: CVE-2015-8985 CVE-2009-5155. Known issues: For information about the latest known issues, see Anthos clusters on bare metal known issues in the Troubleshooting section.

Anthos clusters on VMware - Anthos clusters on VMware 1.14.7-gke.42 is now available. Upgraded VMware vSphere Container Storage Plug-in from 2.7.0 to 2.7.2. The following issues are fixed in 1.14.7-gke.42: Fixed a known issue that admin SSH public key has error after admin cluster upgrade or update. The following vulnerabilities are fixed in 1.14.7-gke.42: High-severity container vulnerabilities: CVE-2023-2828 CVE-2023-3138 Container-optimized OS vulnerabilities: CVE-2023-3090 CVE-2023-35001 CVE-2023-3567 CVE-2023-31248 CVE-2023-27534 CVE-2023-24329 CVE-2023-28840 Ubuntu vulnerabilities: CVE-2023-35788 Windows vulnerabilities: CVE-2022-41723 CVE-2022-41725.

Apigee X - On August 15, 2023, we released an updated version of Apigee X (1-11-0-apigee-1). Bug ID Description 155498623 XPaths in maskconfigs now mask values with special characters. Bug ID Description 281112632, 294892189 Security fix for apigee-runtime. On August 14, 2023, we released an updated version of Apigee X. This release includes a major redesign of the Advanced API Security scores page in the Apigee UI in Cloud console.

AppEngine Flexible Go - Go 1.21 is now available in preview.

AppEngine Standard Go - Go 1.21 is now available in preview.

Artifact Registry - v1. Artifact Registry remote repositories for OS packages are now in Preview.

Batch - Batch is enforcing a 60-day retention policy for all finished (failed or succeeded) jobs: Any existing jobs that have finished before August 17, 2023 are automatically deleted 60 days after, on October 16, 2023.

BigQuery - You can now replicate a dataset from the source region to one or more other regions with cross-region dataset replication. Starting September 15, 2023, prices will apply for network egress from a BigQuery Google Cloud region to another Google Cloud region on the same continent and between different continents.

Bigtable - You can now create a copy of a Cloud Bigtable backup and store it in any project or region where you have a Bigtable instance.

Cloud Build - Cloud Build now generates provenance attestations that meet Supply-chain Levels for Software Artifacts (SLSA) v1.0 specifications.

Carbon Footprint - Updated carbon model to version 9.

Certificate Manager - Certificate Manager now supports integration with regional external Application Load Balancers and regional internal Application Load Balancers.

Chronicle - Chronicle has updated Rules Engine's YARA-L 2.0 language to support float literals. Enhancements to strings.concat and strings.coalesce strings.concat has been updated to take an unlimited number of arguments. Added a new argument get_validation_report to fetch the validation report for a parser or a parser extension.

Access Transparency - Access Transparency logs are enhanced with an eventID that signifies the incident that resulted in the access by Google personnel.

Cloud Composer - Fixed an issue where changing the environment size didn't modify the Redis component size properly. Fixed an issue where data lineage for BigQueryInsertJobOperator was reported to the incorrect target object in cases where Airflow tasks were running in parallel. The apache-airflow-providers-google package is upgraded to version 10.5.0 in images with Airflow 2.5.3 and 2.4.3. Cloud Composer 2.4.1 images are available: composer-2.4.1-airflow-2.5.3 (default) composer-2.4.1-airflow-2.4.3. Cloud Composer versions 2.0.24, 2.0.23, 1.19.7, and 1.19.6 have reached their end of full support period.

Compute Engine - Generally available: The Ops Agent (version 2.38.0 and later) now supports the automatic tracking of GPU usage metrics reported from the NVIDIA Management Library (NVML) for Linux virtual machine instances that have attached NVIDIA GPUs.

Dataflow - Dataflow cost monitoring is generally available (GA).

Dataproc - New Dataproc on Compute Engine subminor image versions: 2.0.74-debian10, 2.0.74-rocky8, 2.0.74-ubuntu18 2.1.22-debian11, 2.1.22-rocky8, 2.1.22-ubuntu20, 2.1.22-ubuntu20-arm. New Dataproc Serverless for Spark runtime versions: 1.1.28 2.0.36 2.1.15. Backported the patches for HIVE-20618 in the new Dataproc on Compute Engine 2.0 and 2.1 images.

Dialogflow - Dialogflow CX now supports past, future, and partial date and time values for ambiguous end-user inputs matched to @sys.date, @sys.date-time, and @sys.time system entities. Dialogflow CX route groups can now be flow-level or agent-level.

Cloud Networking Products - You can now select a cross-region internal Application Load Balancer as a health checked target for DNS routing policies.

Eventarc - Eventarc support for direct events from Cloud IoT is retired from August 16, 2023.

Cloud Functions - Cloud Functions now supports pre-deployment testing in the Google Cloud console at the Preview release level. Cloud Functions now supports the Go 1.21 runtime at the Preview release level.

Cloud Healthcare API - v1. Pub/Sub notifications for DICOM store bulk import is generally available (GA). v1. The following conditional FHIR methods are generally available (GA) using the FHIR identifier search parameter: The fhir.create FHIR standard conditional create fhir.conditionalDelete fhir.conditionalPatch fhir.conditionalUpdate.

IAM - For Credential Access Boundaries, removed the requirement to enable uniform bucket-level access for your Cloud Storage bucket.

Google Kubernetes Engine - You can now easily identify clusters that use deprecated Kubernetes APIs removed in versions 1.25, 1.26, and 1.27. GKE Infrastructure Dashboards and Metrics Packages are now available for both GKE Autopilot and Standard clusters with control plane version 1.27.2-gke.1200 and later. You can now troubleshoot issues with CPU limit utilization and Memory limit utilization of containers running in GKE by using the new "interactive playbook" dashboards in Cloud Monitoring.

Load Balancing - Regional Application Load Balancers and regional proxy Network Load Balancers now support load balancing traffic to external backends outside Google Cloud. Cloud Load Balancing introduces the cross-region internal Application Load Balancer.

Cloud Logging - The behavior of simple text searches as changed. Version 2.37.0 of the Ops Agent introduces runtime health checks for errors in the configuration of logging pipelines and in parsing logs.

Cloud Memorystore - Added support for RDB Snapshots for the M5 capacity tier.

Migrate for Compute Engine 4.8 - 5.0. Preview: Migrate to Virtual Machines supports the migration of VMs running Amazon Linux 2 to Google Cloud as part of a preview program.

Cloud Monitoring - Cloud Monitoring is ending support for the ingestion of AWS CloudWatch metrics in AWS connector projects. The improved dashboard building experience is now GA: Improved the performance Simplified the layout and expanded the configurable settings Improved the widget drag and drop experience Enhanced the text widget Unified the chart-configuration experience between dashboards and the Metrics Explorer For more information, see Add charts and tables to a custom dashboard. Version 2.38.0 of the Ops Agent provides GA support for NVIDIA GPU metrics, including metrics reported from the NVIDIA Management Library (NVML) and the Data Center GPU Manager (DCGM). Observability for Google Kubernetes Engine: You can now enable a curated set of kube state metrics from the Observability tab for your GKE cluster. You can now troubleshoot GKE issues involving CPU and memory utilization by using the new "interactive playbook" dashboards in Cloud Monitoring. Version 2.37.0 of the Ops Agent introduces runtime health checks for errors in the configuration of logging pipelines and in parsing logs. Version 2.37.0 of the Ops Agent introduces GA support for an OpenTelemetry Protocol (OTLP) receiver.

Network Intelligence Center - You can now view allow rules that are less likely to be active based on usage patterns and adaptive analysis.

reCAPTCHA Enterprise - reCAPTCHA Enterprise Mobile SDK v18.3.0 is now available for Android. reCAPTCHA Enterprise Mobile SDK v18.3.0 is now available for iOS.

Cloud Run - You can now send traffic directly to a VPC network with no Serverless VPC Access connector required (Preview).

Cloud Scheduler - VPC Service Controls support for Cloud Scheduler jobs with the following targets is now in GA: Cloud Functions Cloud Run Dataflow API Data Pipelines To learn more, see the documentation on how to secure cron jobs with VPC Service Controls. Support for Cloud Scheduler to call the following resources internally is in GA: Cloud Functions Cloud Run.

Security Command Center - New assets experience released to General Availability The Security Command Center Assets page in the Cloud console is now powered by Cloud Asset Inventory.

SAP Solutions - PD Async Replication support For cross-region active-passive disaster recovery for SAP HANA and SAP NetWeaver, Persistent Disk Asynchronous Replication (PD Async Replication) is supported. Google Cloud's Agent for SAP version 2.5 Version 2.5 of Google Cloud's Agent for SAP is generally available (GA).

Cloud Spanner - Cloud Spanner Data Boost is now available in all regions. Cloud Spanner now lets you check the progress on long-running operations, such as backups, restores, and schema updates.

Cloud SQL MySQL - Cloud SQL Enterprise Plus edition now supports four new regions: europe-central2 (Warsaw) europe-west9 (Paris) southamerica-east1 (San Paulo) us-west1 (Oregon). Cloud SQL for MySQL now supports minor version 8.0.34. You can now re-encrypt an existing Cloud SQL CMEK-enabled primary instance or replica with a new primary key version. You can now retain up to 35 days of retention logs for your Cloud SQL for MySQL Enterprise Plus edition instances when using point-in-time recovery. Cloud SQL for MySQL now supports storage of point-in-time recovery logs in Cloud storage.

Cloud SQL Postgres - Cloud SQL Enterprise Plus edition now supports four new regions: europe-central2 (Warsaw) europe-west9 (Paris) southamerica-east1 (San Paulo) us-west1 (Oregon). You can now re-encrypt an existing Cloud SQL CMEK-enabled primary instance or replica with a new primary key version.

Cloud Trace - Version 2.37.0 of the Ops Agent introduces GA support for an OpenTelemetry Protocol (OTLP) receiver.