Welcome to issue #356 July 24th, 2023

News

Firestore extends triggering support to include Datastore Mode - Firestore adds triggering support for Datastore Mode, through an integration with Eventarc.

Introducing Application Integration: Connect your applications visually, without code - Google Cloud IPaaS to help you automate business processes by connecting any application with visual point-and-click configurations, not code.

Google Cloud and CyberGRX collaborate to help scale and accelerate cloud assessments - Google Cloud and CyberGRX collaborate to help scale and accelerate risk assessments and due diligence services.

Access to Looker data models from Power BI now generally available - Power BI users can now access centrally defined metrics and data relationships from Looker’s semantic layer through Power BI Desktop.

Google Cloud expands availability of enterprise-ready generative AI - New foundation models are GA in Google Cloud’s Vertex AI, adding to an enterprise-ready platform for generative AI.

Conversational AI on Gen App Builder unlocks generative AI-powered chatbots and virtual agents - Improve customer service with generative AI powered chatbots and virtual agents.

Document AI introduces powerful new Custom Document Splitter to automate document processing - With Document AI Workbench’s latest GA feature, Custom Document Splitter, train state of the art machine learning models to classify and split multiple documents in a single file to automate processes.

Ten Sessions Not to Miss about Google Cloud at VMware Explore 2023 Las Vegas - A list of sessions involving Google Cloud VMware Engine on VMware Explore 2023 Las Vegas, August 21-24.

---

Expert support, on demand. Imagine having a direct line to over 150 senior cloud architects for any cloud-related question or issue you encounter. With thousands of cloud questions and issues resolved, DoiT is your gateway to world-class cloud expertise. All from a Google Cloud Partner. Learn more

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Cloud CISO Perspectives: Early July 2023 - Google Cloud CISO Phil Venables talks with Royal Hansen, vice president of Privacy, Safety, and Security Engineering at Google, about red teaming AI and the Aspen Security Forum.

Three Private Service Connect patterns - Networking basics - Explore three Private Service Connect patterns for private communication between consumer and producer networks.

How to Protect Your Statefulset Data with Backup for GKE - In this article we will go through the Backup For GKE components and how to backup volumes from a MySQL StatefulSet deployed in a GKE cluster.

GCP Workload Identity Federation for GitHub Actions: A Really Effortless Setup - This article describes how to set GCP Workload Identity Federation for GitHub Actions using Terraform.

FinOps Challenges in Google Cloud Platform - This article discusses the challenges posed by financial operations in Google Cloud Platform (GCP) and provides strategies for managing expenses effectively.

Beauty of routing in GCP — how to achieve VPC transitivity - Tips to solve VPC transitivity issues.

Massive detection of unused service accounts on Google Cloud - In this guide, discover how to detect and manage unused service accounts in Google Cloud organizations: mastering best practices of GCP.

App Development, Serverless, Databases, DevOps

DevOps Awards winner Improbable on “unleashing the full power of the cloud” - Gaming provider Improbable’s use of CI/CD as a service helped earn it a 2022 DevOps Award for unleashing the full power of the cloud.

DevOps Awards winner Uber on building engineering sustainability - Uber’s Sustainability Suite has helped it save an estimated hundreds of thousands of kilograms of CO2 per year, and earned it a 2022 DevOps Award.

DevOps Awards winner Boa Vista Services on securing the software supply chain - Boa Vista Serviços’ internal Stargate Project to improve code quality and security earned it a 2022 DevOps Award for Securing the Software Supply Chain.

DevOps Awards winner Deutsche Bank on paving the ‘DevOps Runway’ - To reduce costs and maximize efficiency, Deutsche Bank took a DevOps approach to redesigning its system architecture while streamlining applications.

Using Workforce Identity Federation with API-based web applications - Here’s how to configure an example Javascript web application hosted in Google Cloud to call Google Cloud APIs after being authenticated with an Azure AD using Workforce Identity Federation.

Six essential tips for automating API delivery with CI/CD pipelines - Incorporate Apigee into your CI/CD to automate deployments, promote consistency, and monitor performance.

From MySQL to NoSQL: Bitly’s big move to Bigtable - Bitly, the link & QR Code management platform, migrated to Bigtable and increased scalability, availability, and speed.

Streamlining GCP Budget Alerts: Introducing a New Micro Service for Slack Integration - Application that posts Budget alerts to Slack channel.

How Google Cloud Workstations saved my demo… (and my bacon) - Google’s cloud based developer environments are a winner for road warrior devs — as well as those who employ them.

Verify Columnar Engine Usage with AlloyDB Operational Views - A closer look at Views associated with Columnar Engine.

Google Cloud Synthetic Monitoring Tutorial - This tutorial covers step by step instructions on how you can get started with Google Cloud Synthetic Monitoring, that has just been….

The big picture: How Google Photos scaled rapidly on Spanner - Google Photos stores more than 4 trillion photos for over a billion users and uses Spanner to ensure easy access and robust privacy for its users.

Big Data, Analytics, ML&AI

Vodafone: A DevOps approach to AI/ML through cloud-native CI/CD pipelines - How Vodafone improved the performance of its ML pipelines by using DevOps principles of automation, code mirroring and CI/CD.

Put your data to use across organizations with a Data Sharing and Analytics Platform - Working with a large UK Public Sector organization, we’re developing a large hybrid and multi-cloud data sharing and analysis capability.

How to Fix Missing Dates for Time Series Analysis - Learn how to use TVFs in BigQuery to effortlessly generate date ranges for your time series analysis.

A Guide to Using Window Functions - Create running totals, moving averages, and rankings with ease in BigQuery.

Modern data pipeline building with BigQuery Dataform — Part 2: Incremental Tables - As ELT is becoming more and more popular, BigQuery Dataform enables modern SQL based transformations within BigQuery’s environment.

ST_Transform in BigQuery — Project Any Coordinates to Lng/Lat and Back - BigQuery only supports WGS84 geodesic coordinates, but my data is in another coordinate system. How can I convert it?

How to use custom holidays for time-series forecasting in BigQuery ML - With custom holiday modeling features, BigQuery users can build more powerful and accurate time-series forecasting models using BigQuery ML.

How to use advance feature engineering to preprocess data in BigQuery ML - How to preprocess data using BigQuery ML so you can get better insights and models.

Generative AI - Image Generation using Vertex AI Imagen - Examples of using Imagen, a text-to-image diffusion model developed by Google AI.

A Guide to Tuning Language Foundation Models in Google Cloud Generative AI Studio - Achieve better results by tuning foundation model with Generative AI Studio.

Various

Build your cloud career with our summer learning list of no-cost training - This list of top no-cost courses and labs from Google Cloud Skills Boost can help you on your cloud career path.

Google Cloud Professional Cloud Network Engineer (PCNE) Certification Tips and Preparation - Tips for passing GCP Network certification exam.

The Path to Proficiency: My Google Cloud Professional Machine Learning Engineer Certification… - Share experience of passing the Google Cloud Professional Machine Learning Engineer.

Slides, Videos, Audio

Security Podcast - #130 Cloud is Secure: Are you Using It Securely - True or False? And What about SaaS?

Releases

Access Approval - Access Approval supports AlloyDB for PostgreSQL in the Preview stage.

AlloyDB - AlloyDB support for Data Residency is generally available (GA). AlloyDB now supports setting up resource locations policies that can be used to constrain the location of new in-scope resources.

Anthos Config Management - Config Controller now uses the following versions of its included products: Config Connector v1.106.0, release notes Anthos Config Management v1.15.2, release notes.

Anthos clusters on bare metal - 1.15. Release 1.15.3 Anthos clusters on bare metal 1.15.3 is now available for download. Fixes: Fixed an issue where the apiserver could become responsive during a cluster upgrade for clusters with a single control plane node. The following container image security vulnerabilities have been fixed: CVE-2022-3821 CVE-2022-4415 CVE-2022-29458 CVE-2023-1667 CVE-2023-2283 CVE-2023-2454 CVE-2023-2455. Known issues: For information about the latest known issues, see Anthos clusters on bare metal known issues in the Troubleshooting section.

Anthos clusters on VMware - Anthos clusters on VMware 1.13.10-gke.42 is now available. Upgraded VMware vSphere Container Storage Plug-in from 2.6.2 to 2.7.2. The following issues are fixed in 1.13.10-gke.42: Fixed an issue that CPv1 stackdriver operator has --is-kubeception-less=true specified by mistake. The following vulnerabilities are fixed in 1.13.10-gke.42: High-severity container vulnerabilities: CVE-2023-3138 CVE-2023-2828 CVE-2023-2454 CVE-2023-0215 CVE-2023-0286 CVE-2023-0361 CVE-2022-4450 CVE-2023-1999 Container-optimized OS vulnerabilities: CVE-2023-28642 Ubuntu vulnerabilities: CVE-2023-1380 CVE-2023-2612 CVE-2023-30456 CVE-2023-31436 CVE-2023-32233 Windows vulnerabilities: CVE-2022-41723 CVE-2022-41725.

Apigee X - On July 21, 2023, we released an updated version of Apigee X. The Advanced API Security Abuse detection Incident details page now displays unique IP addresses, even if more than one incident corresponds to the same IP address. On July 20, 2023, we released an updated version of Apigee X (1-10-0-apigee-6). Bug ID Description 290943249 Fixed latency issue between Istio and runtime container. Bug ID Description 290709899 Security fix for apigee-runtime.

Google Cloud Armor - Cloud Armor supports parsing of the GraphQL content-type in public preview. Cloud Armor allows you to filter using custom rules or apply Adaptive Protection based on originating client IP addresses in public preview.

Cloud Asset Inventory - The following resource types are now publicly available through the analyze policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning).

Batch - VPC Service Controls support for Batch is generally available (GA). Documentation has been added for Cloud Life Sciences users to explain how to migrate to Batch.

BigQuery ML - Multivariate time series forecasting with the ARIMA_PLUS_XREG model in BigQuery ML is now generally available (GA). BigQuery ML has introduced new Explainable AI capabilities for better model explainability: You can now use the ML.EXPLAIN_FORECAST function with ARIMA_PLUS_XREG models. BigQuery can now use search indexes to optimize some queries that contain the equal operator (=), IN operator, LIKE operator, or STARTS_WITH function to compare string literals with indexed data. Primary and foreign key table constraints are now generally available. The google.cloud.bigquery.storage.v1beta2 API package for BigQueryWrite operations is deprecated and will be removed on July 17, 2024.

BigTable - Cloud Bigtable change streams are now generally available (GA).

Cloud Build - The e2-medium machine type is now supported as a custom machine type that you can specify in your cloudbuild.yaml build configuration file. Cloud Build repositories (2nd gen) is now generally available.

Key Access Justifications - Access Approval supports AlloyDB for PostgreSQL in the Preview stage.

Data Fusion - Workforce identity federation is generally available (GA) in Cloud Data Fusion. The Cloud Data Fusion SAP ODP plugin supports extracting data through CDS views. Apache Hadoop MapReduce is deprecated in Cloud Data Fusion versions 6.7.0 and later (CDAP-18913). In the SAP Ariba plugin 1.2.1, a Token Endpoint field that takes an authentication URL has been added to the plugin properties. In Cloud Data Fusion versions 6.7 and later, SAP Ariba plugin version 1.2.1 fixes the issue causing the following error after entering authentication credentials: CDF_ARIBA_01501 - Failed to call given Ariba service.

Dataproc Serverless - New Dataproc Serverless for Spark runtime versions: 1.1.24 2.0.32 2.1.11.

Dataproc - New Dataproc on Compute Engine image versions, which includes a 2.1.18-ubuntu20-arm image that supports ARM machine types: 2.0.70-debian10, 2.0.70-rocky8, 2.0.70-ubuntu18 2.1.18-debian11, 2.1.18-rocky8, 2.1.18-ubuntu20, 2.1.18-ubuntu20-arm. Fixed a race condition in Spark startup that could lead to nodes failing to initialize when using premium disk tier.

Cloud Deploy - Cloud Deploy has completed Google Cloud data residency requirements. Cloud Deploy now provides the ability to pass deploy parameters to your manifests, per delivery pipeline, per target, and per release (in preview).

Dialogflow - Dialogflow CX has launched three new generative AI features.

Document AI - v1.4. The following Form Parser (pretrained-form-parser-v2.0-2022-11-10) features are Generally Available (GA): General field extraction: You can extract 11 different types of entities from documents Enhanced checkbox detection Internationalization (i18n) support that covers over 200 languages Upgraded key-value pair (KVP) detection model. Form parser v2.1 (pretrained-form-parser-v2.1-2023-06-26) is in Public Preview, which uses our native PDF text extraction model on PDF documents. The Form Parser features has the following limitations: Checkbox doesn't support radio buttons and might not reliably parse all selection marks or keyless checkboxes. v1. The Custom Document Splitter (CDS) within Document AI Workbench is now Generally Available (GA) for production use cases to split and classify multiple documents within a single file.

Cloud Firestore - The project usage monitoring page has moved to a new URL under the existing project usage page.

Google Kubernetes Engine - Update to the Issue release note published on July 19, 2023 We investigated this issue and are rolling back the --no-enable-insecure-kubelet-readonly-port flag in the gcloud CLI. In GKE version 1.25 and later, there is a bug fix in the Ingress Controller to unset the Cloud Armor Ingress Security Policy when removed from the BackendConfig. There's a known issue causing the gcloud CLI to crash when you run the command to disable the insecure kubelet read-only port, as described in Stop using the insecure kubelet read-only port in GKE clusters. Starting in GKE version 1.27 and gke-metrics-agent version 2.0.0, the memory request and limit of gke-metrics-agent will increase by an extra 60MiB.

Cloud Life Sciences - Cloud Life Sciences is deprecated.

Cloud Logging - Support for routing your logs through the Log Router of another Google Cloud project is now Generally Available (GA). The pricing language for Cloud Logging has changed; however, the free allotments and the rates haven't changed. You can now upgrade log buckets in most regions to use Log Analytics.

Marketplace Partners - We've redesigned the Private Offers experience to improve offer creation and management, including the following changes: You can fill in information in any order, instead of having to enter it all at once.

Memorystore for Memcached - Added support for Creating a Memcached instance that uses a specific IP address range.

Cloud Memorystore - Redis version 7.0 is now Generally Available for Memorystore for Redis.

Cloud Monitoring - We made improvements to the dashboard building experience: Improved the performance Simplified the layout and expanded the configurable settings Improved the widget drag and drop experience Enhanced the text widget Unified the chart-configuration experience between dashboards and the Metrics Explorer For more information, see Add charts and tables to a custom dashboard. You can now create synthetic monitors, which let you continuously test the availability, consistency, and performance of your services and application web pages and APIs, by using automated script based tests.

Cloud Router - The Cloud Router custom learned routes feature is Generally Available (GA).

Network Intelligence Center - Connectivity Tests now includes a feature that verifies connectivity from a VM to a Private Service Connect endpoint. Connectivity Tests now includes a feature that verifies connectivity from a VM or an IP address to a load balancer.

Cloud PubSub - Payload unwrapping for push subscriptions is now available. A weekly digest of client library updates from across the Cloud SDK.

Anthos Service Mesh - 1.15.x. 1.15.7-asm.21 is now available for in-cluster Anthos Service Mesh. 1.16.x. 1.16.6-asm.3 is now available for in-cluster Anthos Service Mesh. 1.17.x. 1.17.4-asm.2 is now available for in-cluster Anthos Service Mesh.

SAP Solutions - Google Cloud's Agent for SAP version 2.3 Version 2.3 of Google Cloud's Agent for SAP is generally available (GA). Google Cloud's Agent for SAP version 2.1 Version 2.1 of Google Cloud's Agent for SAP is generally available (GA).

Cloud Spanner - Spanner supports cascading deletes for foreign keys.

Cloud SQL - Cloud SQL now supports default maintenance windows for your instances. You can now enable query insights for multiple instances at a time.

Cloud Storage - The gcloud storage command-line tool has changed some of the metadata it returns for buckets and objects, as well as changed the format of some metadata names it returns. gcloud storage GA release 1.3 is now available. Beginning October 16, 2023, the Autoclass feature will change its storage class transition behavior. Beginning October 16, 2023, the Autoclass feature and the matchesStorageClass condition for Object Lifecycle Management will be incompatible. Beginning October 16, 2023, the following pricing changes apply to buckets that use the Autoclass feature: Each storage class transition from Coldline or Archive storage to Standard storage will change from being free to being charged as a Class A operation at the Standard storage rate.

Cloud TPU - Cloud TPU now supports TensorFlow 2.12.1.

Cloud Trace - When viewing a span, you can now also view the linked spans.

Vertex AI - Vertex AI Workbench instances are now available in Preview. Model tuning updates for text-bison: Upgraded tuning pipeline now offers more efficient tuning and better performance on text-bison. Imagen on Vertex AI now offers the following Generally Available (GA) features: Image generation (text-to-image generation)* Image editing* Image visual captioning Visual Question Answering (VQA) * Restricted access feature. Imagen now supports human face generation for the following features: Image generation (text-to-image generation)* Image editing* * Restricted access feature. The Vertex AI PaLM API has added support for the following languages: Spanish (es) Korean (ko) Hindi (hi) Chinese (zh) For the complete list of supported languages, see Supported languages.

VPC Service Controls - General availability support for the following integration: Batch.

Virtual Private Cloud - All service attachments, including those created before March 1, 2023, consume one NAT IP address for each connected endpoint or backend. You can publish a service that is hosted on an internal passthrough Network Load Balancer that forwards traffic on all ports (--ports=all).