Welcome to issue #350 June 12th, 2023

News

Announcing Dataform in GA: Develop, version control, and deploy SQL pipelines in BigQuery - Dataform, now GA, lets data engineers build SQL pipelines in BigQuery while following best practices like Git, CI/CD, and code lifecycle management.

Announcing general availability of Cloud Firewall threat intelligence and geo-location features - Four new Cloud Firewall features are now generally available, including threat intelligence, geo-location objects, address groups, and local IP ranges.

Announcing first Google Cloud OSCAL package submission for DoD Impact Level 5 - Google Cloud has successfully submitted the complete OSCAL package for Department of Defense Impact Level 5 to eMASS, a major milestone.

Highlight your generative AI skills by earning the new no-cost skill badge - Explore no-cost generative AI from Google Cloud and earn a new skill badge to show your knowledge.

New Cryptomining Protection Program offers $1 million for costly cryptomining attacks - Google Cloud now offers our Security Command Center Premium customers $1 million of protection against cryptomining attacks. Here’s how you can get it.

Generative AI support on Vertex AI is now generally available - Google Cloud announces Generative AI support on Vertex AI generally available.

Improving search experiences with Enterprise Search on Gen App Builder - How organizations can create combine generative AI and enterprise search with Generative AI App Builder.

Your guide to sessions, learning paths, and more at Google Cloud Security Summit 2023 - Get ready for Google Cloud Security Summit 2023 with this detailed look at the sessions and speakers.

Building ML workflows in BigQuery the easy way, without code - A flexible, automated analytic workflow tool that integrates with BigQueryML allows no-code forecasting and model training.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Strengthening Network Security: Fully Qualified Domain Name (FQDN) egress Filtering with GCP Firewall Policy - This blog post goes through the process of implementing FQDN egress filtering in GCP using FQDN objects in the firewall policy rules.

Managing carbon emissions across clouds: a practical example - Read how Thoughtworks uses a combination of Google Cloud products to measure carbon emissions and track sustainability goals.

Designing Multi-regional Internal Load Balancing with Google Cloud iLB + Cloud DNS - Learn about Google Cloud internal Load balancers and advanced features like multi-regional access and DNS failover.

How to easily migrate your on-premises firewall rules to Cloud Firewall policies - Migrating an on-prem firewall or firewall appliance configuration to Google Cloud can be daunting. It requires thinking about security and segmentation with a different mindset.

Ten ways troubleshooting GKE apps is now easier in Cloud Logging, part 1 - There are several new Cloud Logging features that let find logs that matter and resolve issues more quickly, especially for GKE environments.

Google Cloud: Set Up Cloud DNS Using Terraform - This article explains how to use Terraform to set up a domain with Cloud DNS in the Google Cloud Platform.

Fight The Hidden Cost of Regional Kubernetes Clusters — Cross Zonal Egress — Part 1 - Prioritizing intra-zonal communication in GKE cluster.

Google Cloud Cross-Cloud Interconnect: Almost everything you need to know - A brief overview of Cloud Cross-Cloud Interconnect.

Your First Step in Google Cloud Platform : Building a Landing Zone with Terraform - In this article are unpacked key elements of constructing an efficient Landing Zone within Google Cloud Platform.

Data Security in Google Cloud series — Part 1: Physical data encryption of sensitive data using Cloud DLP and KMS - Using Data Loss Prevention to Inspect and Redact sensitive data and then use KMS-generated symmetric encryption key to encrypt the sensitive data detected by DLP.

App Development, Serverless, Databases, DevOps

Three Lisps on GCP Cloud Functions - I have always been a fan of the power of functional programming. I use it whenever I can, and I find it to be extremely powerful….

From Receipts to Riches: Save Money w/ Google Cloud & Supermarket Bills - Part 2 - Upload receipts and invoices, analyze with Document AI and classify into categories to understand where the dollars are going.

Computer vision made easy: Vision AI on Spring Boot and Java - Build a Vision AI application using Spring Boot and Java that extracts text from images, detects the local language and translates to English.

The Modernization Imperative: Shifting left is for suckers. Shift down instead - Instead of developers “shifting left,” they need to “shift down” and push more workloads down onto the platforms they’re already using.

Migration to Google Cloud part 1 — Migrating dependencies to Artifact Registry - Migrating Maven artifacts to the Artifact registry.

Simplify Access Management with Google Cloud Workforce Identity Federation — Part 2 Okta SSO Integration - Integration of Workforce Identity Federation and Okta for SSO.

Essential SQL for exploring AlloyDB Columnar Engine. - Sharing the SQL script frequently used in work with AlloyDB, specifically leveraging its Columnar engine.

Big Data, Analytics, ML&AI

Data Time Machine: Unearthing History with BigQuery’s Time Travel Feature - A tutorial on how to use time travel in BigQuery.

Improve query performance and optimize costs in BigQuery using the anti-pattern recognition tool - A new BigQuery anti-pattern recognition tool scans SQL queries to identify anti-patterns, and provides optimization recommendations.

Document AI: Understanding invoices to passports and beyond - Use cloud AI tools to extract key data points from receipts, passports, and all manner of documents.

Helping businesses with generative AI - Explore how Google Cloud is helping organizations innovate with generative AI.

Predict, personalize, and wow your customers with better analytics and AI - Learn how analytics and AI can provide a true 360 view of the customer and help create better financial services experiences.

Productionizing SQL-based workflows in Google Cloud - How to set up a repeatable and scalable ELT pipeline in Google Cloud using Dataform and Cloud Build.

Google Cloud Dataflow — data pipelines with Apache Beam and Apache Hop - This post explains how to run Apache Beam pipelines in Apache Hop on Google Cloud.

Deploying Google Cloud Dataproc with Terraform — what, why and how - A brief overview of Dataproc and how it can be deployed with Terraform.

Enable Federated Access to AlloyDB Columnar from BigQuery. - This blog post explains how to enable querying columnar store data on AlloyDB from BigQuery.

BigQuery: Migration to Standard Edition from On-Demand in 5 Steps - In this guide, we’ll walk through the process of figuring out whether and how to transition from the ‘On-Demand’ edition to the ‘Standard'.

Generative AI - Document Retrieval and Question Answering with LLMs - Apply LLMs to your domain-specific data.

Getting Started with BigQuery ML: A Practical Tutorial for Beginners - A walkthrough to get started with BigQuery ML and demonstrating practical applications.

Generative AI Learning Path Notes — Part 1 - An overview of Generative AI Learning Path in Google Cloud Skills with notes.

Various

Apply now for the new Middle East & Africa startup accelerator on climate change - Last chance to apply for Google for Startups accelerator program focused on climate change in Middle East and Africa.

Slides, Videos, Audio

Kubernetes Podcast - #202 KubeCon EU 2023.

Security Podcast - #124 Safe Browsing: Lessons from How Google Secures Five Billion Devices at Low False Positive Rates.

Releases

Access Approval - Access Approval supports Anthos Identity Service in the Preview stage.

AlloyDB - AlloyDB for PostgreSQL is now available in the following regions: us-east5 (Columbus) us-south1 (Dallas) For more information, see AlloyDB Locations.

Anthos Config Management - 1.15.1. The Anthos Config Management SKU and Anthos Policy Controller SKU are end-of-sale.

Anthos clusters on Azure - Security bulletin A new vulnerability (CVE-2023-2878) has been discovered in the secrets-store-csi-driver where an actor with access to the driver logs could observe service account tokens. You can now launch clusters with the following Kubernetes versions: 1.24.13-gke.500 1.25.8-gke.500 1.26.4-gke.2200. This release fixes the following vulnerability: CVE-2023-1872. Known issues: For information about the latest known issues, see Known issues for Anthos clusters on Azure. Security bulletin A new vulnerability (CVE-2023-1872) has been discovered in the Linux kernel that can lead to a privilege escalation to root on the node.

Anthos clusters on VMware - Security bulletin A new vulnerability (CVE-2023-2878) has been discovered in the secrets-store-csi-driver where an actor with access to the driver logs could observe service account tokens. Known issue If you create a version 1.13.8 or version 1.14.4 admin cluster, or upgrade an admin cluster to version 1.13.8 or 1.14.4, the kind cluster pulls the following container images from docker.io: docker.io/kindest/kindnetd docker.io/kindest/local-path-provisioner docker.io/kindest/local-path-helper If docker.io isn't accessible from your admin workstation, the admin cluster creation or upgrade fails to bring up the kind cluster. Security bulletin A new vulnerability (CVE-2023-1872) has been discovered in the Linux kernel that can lead to a privilege escalation to root on the node.

Apigee X - On June 9, 2023 we released an updated version of Apigee X. Bug ID Description 286452898 Previously, the Apigee Analytics topk query parameter, which returns the top k results for a query, always returned the results in descending order, even when the order parameter was ASC.

Cloud Asset Inventory - The following resource types are now publicly available through the Export APIs (ExportAssets, ListAssets, and BatchGetAssetsHistory), Feed API, and Search APIs (SearchAllResources, SearchAllIamPolicies).

Batch - Batch is available in the following regions: asia-east2 (Hong Kong) europe-central2 (Warsaw) us-south1 (Dallas) us-west2 (Los Angeles) us-west3 (Salt Lake City) us-west4 (Las Vegas) For more information, see Locations.

Billing - Use folders and organizations in budgets: When you set up budgets for your Cloud Billing account, you can set the budget's scope to one or more folders or organizations that are linked to your account, in addition to the current options for specific projects and labels.

Cloud Build - When you enable the Cloud Build API in a project, Cloud Build automatically creates a default service account to execute builds on your behalf.

Chronicle - The following changes are available in the Unified Data Model. Chronicle now links to a customer-supplied Google Cloud Project to integrate more closely with Google Cloud services, such as Cloud IAM, Cloud Monitoring, and Cloud Audit Logs.

Key Access Justifications - Access Approval supports Anthos Identity Service in the Preview stage.

Access Transparency - Access Transparency supports Memorystore for Redis in the GA stage.

Cloud Composer - Cloud Composer 2.3.0 release started on June 9, 2023. An environment can now run two Airflow triggerers. The apache-airflow-providers-google package in images with Airflow 2.5.1 and 2.4.3 was upgraded to version 2023.6.6+composer. (Cloud Composer 2) Fixed an issue where it was not possible to delete a Cloud Composer environment if the environment's service account was already deleted. The google-cloud-asset package is added to images with Airflow 2.5.1 and 2.4.3. Cloud Composer 2.3.0 images are available: composer-2.3.0-airflow-2.5.1 (default) composer-2.3.0-airflow-2.4.3. Cloud Composer versions 2.0.16, 2.0.15, 1.18.12, and 1.18.11, have reached their end of full support period.

Confidential VM - Confidential Space. Ports can now be opened for ingress network traffic when using Confidential Space image version 230600 and above.

Compute Engine - Generally available: Hyperdisk Throughput provides cost-effective and throughput-oriented block storage with dynamically configurable capacity and throughput. Generally available: NVIDIA A100 80GB GPUs are now available in the following additional regions and zones: Iowa, North America: us-central1-a For more information about using GPUs on Compute Engine, see GPU platforms. For MIGs that have T2D machine series VMs, autoscaling based on CPU utilization doesn't work as expected. Generally available: Accelerator-optimized (G2) machine types with attached NVIDIA® L4 GPUs are generally available in the following regions and zones: Singapore, APAC: asia-southeast1-b Netherlands, Europe: europe-west4-a,b,c Iowa, North America: us-central1-a,b South Carolina, North America: us-east1-b,d Virginia, North America: us-east4-a Oregon, North America: us-west1-a,b.

Data Fusion - Cloud Data Fusion version 6.8.3 is generally available (GA). Cloud Data Fusion 6.8.3 supports the ability to configure Java options for a pipeline run by setting the system.program.jvm.opts runtime argument (CDAP-20381). Cloud Data Fusion 6.8.3 supports upgrades in the Pipeline Post-run Action (Pipeline Alerts) plugins during the pipeline upgrade process (CDAP-20567). Fixed in 6.8.3: Fixed an issue where the event publish feature did not work with RBAC-enabled instances (CDAP-20375). Zendesk plugins version 1.2.0 is available in the Cloud Data Fusion Hub.

Datastream - The maximum event size that Datastream supports is now increased.

Cloud Deploy - As of June 6, 2023, Google Cloud Deploy is ready to support HIPAA compliance.

Cloud Functions - You can now create and then configure Serverless VPC Access connector for your function directly from the Create form in the Google Cloud console at the Preview release level.

Google Kubernetes Engine - New Autopilot clusters that run GKE version 1.25.5-gke.1000 and later automatically use Image streaming to pull eligible images. In addition to the existing egress network policy GKE already supports, you can now control the egress traffic of your Pods by using a network policy that matches a fully-qualified domain name or a regular expression. The PD CSI Driver will be automatically enabled on upgrades to 1.25, for clusters with the add-on disabled. (2023-R12) Version updates GKE cluster versions have been updated. A new vulnerability (CVE-2023-1872) has been discovered in the Linux kernel that can lead to a privilege escalation to root on the node. A new vulnerability (CVE-2023-2878) has been discovered in the secrets-store-csi-driver where an actor with access to the driver logs could observe service account tokens. (2023-R11) Version updates GKE cluster versions have been updated.

GKE - (2023-R12) Version updates The following control plane versions are now available: 1.23.17-gke.6800 1.24.14-gke.1200 1.25.10-gke.1200 1.26.5-gke.1200 The following node versions are now available: 1.23.17-gke.6800 1.24.14-gke.1200 1.25.10-gke.1200 1.26.5-gke.1200 Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.25 to version 1.25.8-gke.1000 with this release. (2023-R11) Version updates The following control plane and node versions are now available: 1.22.17-gke.11400 1.23.17-gke.5600 1.24.13-gke.2500 1.25.9-gke.2300 1.26.4-gke.1400 The following control plane versions are no longer available: 1.22.17-gke.7500 1.22.17-gke.9400 1.23.17-gke.1700 1.24.10-gke.2300 1.25.7-gke.1000 1.25.9-gke.400 1.26.2-gke.1000 Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.21 to version 1.22.17-gke.8000 with this release.

Google Kubernetes Engine Rapid - (2023-R12) Version updates The following versions are now available in the Rapid channel: 1.23.17-gke.6800 1.24.14-gke.1200 1.25.10-gke.1200 1.26.5-gke.1200 1.27.2-gke.1200 The following versions are no longer available in the Rapid channel: 1.22.17-gke.8000 1.23.17-gke.2000 1.23.17-gke.3600 1.24.13-gke.2500 1.25.8-gke.1000 1.26.4-gke.500 1.26.4-gke.1400 1.27.1-gke.400 Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to version 1.22.17-gke.11400 with this release. (2023-R11) Version updates The following versions are now available in the Rapid channel: 1.22.17-gke.11400 1.23.17-gke.5600 1.24.13-gke.2500 1.25.9-gke.2300 1.26.4-gke.1400 The following versions are no longer available in the Rapid channel: 1.22.17-gke.9400 1.23.17-gke.1700 1.24.13-gke.500 1.25.9-gke.400 Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to version 1.22.17-gke.8000 with this release.

Google Kubernetes Engine Regular - (2023-R12) Version updates Version 1.24.13-gke.2500 is now available in the Regular channel. (2023-R11) Version updates The following versions are now available in the Regular channel: 1.22.17-gke.8000 1.23.17-gke.2000 1.24.12-gke.1000 1.25.8-gke.1000 1.26.3-gke.1000 The following versions are no longer available in the Regular channel: 1.22.17-gke.7500 1.23.17-gke.1700 1.24.11-gke.1000 1.26.2-gke.1000 Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.21 to version 1.22.17-gke.8000 with this release.

Google Kubernetes Engine Stable - (2023-R12) Version updates Version 1.25.8-gke.1000 is now the default version in the Stable channel. (2023-R11) Version updates Version 1.24.12-gke.500 is now the default version in the Stable channel.

Load Balancing - The global external HTTP(S) load balancer now supports a configurable client HTTP Keepalive Timeout.

Cloud Logging - You can now configure CMEK and a default storage location for individual folders, in addition to organizations.

Cloud Memorystore - Access Transparency is now Generally Available for Memorystore for Redis.

Migrate for Compute Engine 4.8 - 5.0. Generally available: The Estimated cut-over time field is now generally available.

Cloud Monitoring - Google Cloud Managed Service for Prometheus can now ingest exemplars attached to histogram metrics. A new version of Managed Service for Prometheus is now available.

Cloud Run - You can now create and then configure a Serverless VPC Access connector for your service or job directly from the Create form in the Google Cloud console.

Security Command Center - Usage-based pricing for organization-level activations of Security Command Center You can now use usage-based pricing instead of a fixed-price subscription to activate Security Command Center Premium tier at the organization level. Security Command Center Cryptomining Protection Program The Security Command Center Cryptomining Protection Program is launched to General Availability.

SAP Solutions - Google Cloud's Agent for SAP version 2.0 Version 2.0 of Google Cloud's Agent for SAP is generally available (GA). Google Cloud's monitoring agent for SAP HANA is deprecated, and is replaced by the SAP HANA monitoring metrics collection feature of version 2.0 of Google Cloud's Agent for SAP.

Cloud Spanner - In both the GoogleSQL and PostgreSQL dialects, adds support for the IF NOT EXISTS clause in CREATE TABLE, CREATE INDEX, and ALTER TABLE ADD COLUMN, along with IF EXISTS for DROP TABLE and DROP INDEX. Fine-grained access control is now available for PostgreSQL-dialect databases. A monthly digest of client library updates from across the Cloud SDK.

Cloud SQL SQL Server - You can now import and export differential database backups.

Cloud Storage Transfer - We discovered a security vulnerability in the Storage Transfer Service agent container.

Cloud TPU - You can now view historical logs of maintenance events on your TPU in system event audit logs.

Vertex AI - PaLM Text and Embeddings APIs, and Generative AI Studio The Generative AI support on Vertex AI is now generally available (GA). Vertex AI Model Garden The Vertex AI Model Garden is now generally available (GA). Vertex AI Codey APIs The Vertex AI Codey APIs are now in Preview.