Welcome to issue #385 February 12th, 2024

News

8 new ways to bridge the gap to geospatial analysis with Earth Engine - Improvements and integrations released in the past few months to make it easier to use Earth Engine.

Streamline digital commerce with the Integrated Commerce Network from Kin + Carta - The Integrated Commerce Network, is a pre-integrated digital commerce solution from a curated group of our digital commerce ISV partners and delivered by systems integrator partner Kin + Carta.

Simplify DORA compliance with Google Cloud's updated contracts

Exploring Cloud SQL Enterprise Plus edition for your MySQL and PostgreSQL workloads

Firestore Multiple Databases is now generally available - General availability of Firestore Multiple Databases, which lets you manage multiple Firestore databases within a single Google Cloud project, enhancing data separation, security, resource management, and cost tracking.

How Arvato Systems makes 3D picture production easier, faster and cheaper with Google Cloud

---

You need your people to be productive from anywhere and on any device, but you can’t sacrifice security. Cameyo’s Virtual App Delivery (VAD) platform - which is integrated with and can run in Google Cloud - enables you to deliver ALL your apps to any device with Zero Trust security. And GCP Weekly readers get a free 30-day business trial. Sign up today here

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Five key things to consider when building a cloud FinOps team - This blog post delves into five key factors to consider when building a successful cloud FinOps team.

Start your cloud networking journey today - An overview of core networking services in Google Cloud.

Faster startup times for Kubernetes workloads with Kube Startup CPU Boost - This article describes Kubernetes resource management for Pods and containers.

FastAPI on GKE with GPU: A Step-by-Step Guide - To run a FastAPI application that requires GPU support on Google Cloud Run, which itself does not natively support GPUs, you would need to integrate it with Google Kubernetes Engine (GKE), which supports GPUs.

GKE Multi-Cluster Services (MCS): Feels like magic — at first - Dissecting the building blocks of Multi-Cluster Services in GKE.

Google Cybersecurity Action Team Threat Horizons Report #9 Is Out! - This is blog post is inspired by the Threat Horizons Report.

App Development, Serverless, Databases, DevOps

UM-Bridge: leveraging Kubernetes for scalable Uncertainty Quantification in the cloud

Salk Institute scientists scale brain research on Google Cloud with SkyPilot - SkyPilot is a framework for running LLMs, AI, and batch jobs on any cloud, offering maximum cost savings, highest GPU availability, and managed execution.

Intelligencia AI speeds up deployment time with Cloud SQL

Streaming Data from Client to Cloud Run Server using HTTP/1 - Creating a Cloud Run service using a Node.js server that logs incoming data chunks.

FastAPI CloudRun Starter: Serveless Backend Deployment - FastAPI CloudRun Starter repository for efficient and streamlined backend deployment.

Google Cloud Adoption: Site Reliability Engineering (SRE), and Best Practices for SLI / SLO/ SLA - The best practices of Site Reliability Engineering.

Cut Container Startup Time for Better Performance and Costs — Part1 - Optimization tactics across infrastructure, container design, and even code-level tweaks to speed up container startups.

Telemetry at Greenbids - Using OpenTelemetry to gather business related metrics.

Big Data, Analytics, ML&AI

Leveraging streaming analytics for actionable insights with gen AI and Dataflow - In this blog post, we showcase how to get real-time LLM insights in an easy and scalable way using Dataflow.

Medical Text Processing on Google Cloud

No GPU? No problem. localllm lets you develop gen AI apps on local CPUs - In this post, we introduce you to a novel solution that allows developers to harness the power of LLMs locally on CPU and memory, right within Cloud Workstations, Google Cloud’s fully managed development environment.

BigQuery Methods For Re-Creating Pandas’ Top EDA Functions - In this guide, we’ll explore how to re-create key Pandas functions used for EDA such as describe and corr in BigQuery.

How to avoid circular dependencies in Dataform - Handling circular dependencies in Dataform.

Data Sync Between Google BigQuery and Google Sheet - As a data engineer, one of the frequent tasks you might encounter is syncing data from a Google Sheet to a Google BigQuery table.

Analyzing customer feedback at scale — from Google Sheets to BigQuery! - This post walks you through analyzing sentiment at scale without writing a single line of code.

Reducing BigQuery Costs by 100–200x with dbt Incremental Models - Reducing costs for dbt models in BigQuery.

Google launches Entity Resolution for BigQuery - An Introduction to Entity Resolution — How to share Data more easily.

Apache Beam schemas and Cloud Dataflow updates - Use @SchemaFieldNumber with your Java pipelines in Apache Beam to make sure that schemas are always backwards compatible.

What’s new with Looker 2024? - Summarizing the Looker 2024 roadmap.

Generative AI Comes to Looker via Vertex AI and BigQuery BQML - An overview of Looker's roadmap for 2024.

Cheap VertexAI data syncing using GCS - Learn how to sync and persist VertexAI notebooks user’s data using GCS.

Enterprise MLOps with Google Cloud Vertex AI part 3: CI/CD - This is the third article in our series on MLOps processes.

Slides, Videos, Audio

Kubernetes Podcast - #218 Kubernetes stale reads, with Madhav Jivrajani.

GCP FinOps Hub: The Key to Mastering Your Finances on Google Cloud - this tutorial, covers everything how to use GCP FinOps, cost management and how to setup Direct VPC egress to reduce Cloud Run costs. Learn how to optimize your spending and stay within your budget with our step-by-step guide.

Releases

AlloyDB - You can now use public IP with the AlloyDB Language Connectors (Preview) to connect to your cluster.

Anthos clusters on Azure - You can now launch clusters with the following Kubernetes versions: 1.28.5-gke.100, 1.27.9-gke.100, 1.26.12-gke.100.

Apigee X - On February 8, 2024 we released an updated version of the Apigee APIs. API support for update operations on KeyValueMap entries Starting with this release, the Apigee APIs support update operations for KeyValueMap entries.

Bare Metal Solution - You can now view the details of the OS of your Bare Metal Solution server.

BigQuery - Custom data masking is now generally available (GA). BigQuery now offers entity resolution. You can now view query plans to see details of SQL pushdowns in federated queries. Billing for Spark stored procedures begins on March 12, 2024.

Chronicle - Chronicle requires a minimum Transport Layer Security (TLS) version of 1.2 to maintain security compliance. When the data ingestion rate for a tenant reaches a certain threshold, Chronicle controls the rate of ingestion for new data feeds to prevent a source with a high ingestion rate from affecting the ingestion rate of another data source.

Cloud Composer - Starting January 20, 2024, in the asia-southeast1, us-west3, and us-west4 regions it is possible to create new Cloud Composer 1 environments only in projects that already have Cloud Composer 1 environments. The enabling and disabling functionality for the Logs in Cloud Logging only feature was temporarily rolled back. Cloud Composer 2.6.0 release started on February 5, 2024. Data lineage is now generally available (GA) in Cloud Composer 2. Data lineage integration is now enabled by default in newly created environments with Cloud Composer version 2.1.2 and later, if Data Lineage API is enabled in the environment's project. Python 3.11 is available in environments with Airflow 2.6.3: New environments with Airflow 2.6.3 use Python 3.11. Airflow worker memory requirements in Python 3.11 are 10% higher compared to workers in Python 3.8. (Available without upgrading) The default worker_concurrency in Airflow 2.6.3 and later versions is now calculated using a different formula. Connections to the Redis environment component are now additionally secured with a password. The apache-airflow-providers-google package is upgraded to version 10.13.1 in images with Airflow 2.6.3. Cloud Composer 2.6.0 images are available: composer-2.6.0-airflow-2.6.3 (default) composer-2.6.0-airflow-2.5.3. Cloud Composer versions 2.1.5 and 1.20.5 have reached their end of full support period. Cloud Composer 2.6.0 is a version with an extended upgrade timeline.

Compute Engine - Generally available: Hyperdisk Throughput is available with the following VMs: A3 C3 C3D G2 H3 M3 Hyperdisk Throughput support for Z3 VMs is also available in Preview. When you purchase a resource-based commitment for GPUs, Local SSD disks, or both, you can attach any of your existing on-demand or auto-created future reservations to that commitment.

Dataproc Serverless - New Dataproc Serverless for Spark runtime versions: 1.1.49 2.0.57 2.1.36 2.2.0-RC9. Dataproc Serverless for Spark: Backported patch for HADOOP-18652.

Dataproc - New Dataproc on Compute Engine subminor image versions: 2.0.92-debian10, 2.0.92-rocky8, 2.0.92-ubuntu18 2.1.40-debian11, 2.1.40-rocky8, 2.1.40-ubuntu20, 2.1.40-ubuntu20-arm 2.2.6-debian12, 2.2.6-rocky9, 2.2.6-ubuntu22. Dataproc on Compute Engine: Backported patch for HADOOP-18652. Dataproc on Compute Engine Ranger Cloud Storage enhancement: Enabled downscoping Added caching of tokens in local cache Both settings are configurable and can be enabled by customers.

Deep Learning VM - M116 release Added the CUDA version to the TensorFlow 2.15 image family name, for this release and future releases.

Cloud Deploy - Cloud Deploy now uses Skaffold 2.10 as the default Skaffold version for all target types. Google Cloud Deploy is now available in the following regions: me-central1 (Doha) me-central2 (Dammam) europe-west12 (Turin) europe-west10 (Berlin).

Google Kubernetes Engine - The following GKE versions fix a memory leak issue with the Google Cloud Storage FUSE CSI driver DaemonSet Pod: 1.25.16-gke.1360000 and later 1.26.13-gke.1052000 and later 1.27.10-gke.1055000 and later 1.28.6-gke.1095000 and later 1.29.1-gke.1425000 and later. (2024-R03) Version updates GKE cluster versions have been updated. Clusters on control plane versions 1.26.6-gke.1900 and later might encounter intermittent connection establishment failures.

Cloud Logging - You can now create log buckets in the africa-south1 region. Log buckets in the following regions can now be upgraded to use Log Analytics: europe-west10 For more information, see Supported regions.

reCAPTCHA Enterprise - reCAPTCHA Enterprise Mobile SDK v18.4.1 is now available for iOS.

Resource Manager - You can use the Google Cloud console with Policy Simulator for Organization Policy to test organization policies. With the secure-by-default organization policy enforcements, insecure posture is addressed with a bundle of organization policies that are enforced at the time of creation of an organization resource.

Security Command Center - Exports of compliance reports will require new permissions On or after March 15, 2024, a new Identity and Access Management (IAM) permission will be required to export a compliance report from the Google Cloud console. New security posture service released to General Availability The new security posture service is released to General Availability. Mandiant analyst CVE ratings added to vulnerability findings The addition of CVE information, including ratings of the vulnerability by Mandiant Threat Intelligence analysts, to the details of Security Command Center vulnerability findings is released to Preview. Improvements to compliance standards support now available Improvements to the Security Command Center Compliance page in the Google Cloud console are released to General Availability. Prioritize high-value resources automatically by data sensitivity The optional integration of the Sensitive Data Protection discovery feature with the Security Command Center attack path simulation feature is released to Preview. Attack exposure scores informed by Mandiant Threat Intelligence The inclusion of CVE exploitability ratings in the calculation of attack exposure scores for vulnerability findings is released to Preview. High-value resources now include attack exposure scores The calculation of attack exposure scores for high-value resources by the Security Command Center Attack Path Simulations feature is released to Preview.

Service Mesh - 1.17.x. Google has ended support for in-cluster Anthos Service Mesh 1.17 following the official policy. 1.17.8-asm.20 is now available for in-cluster Anthos Service Mesh. 1.20.x. 1.20.3-asm.4 is now available for in-cluster Anthos Service Mesh. 1.19.x. 1.19.7-asm.3 is now available for in-cluster Anthos Service Mesh. 1.18.x. 1.18.7-asm.4 is now available for in-cluster Anthos Service Mesh.

Cloud Spanner - Made changes to the information schema to improve the accuracy of data type reporting.

Cloud SQL MySQL - Cloud SQL now automatically updates your read replicas when you perform self-service maintenance on the primary instance. Cloud SQL now supports near-zero downtime planned maintenance on HA-enabled Cloud SQL Enterprise Plus instances with all combinations of public IP connectivity. Cloud SQL for MySQL now supports minor version 8.0.36.

Cloud SQL Postgres - Cloud SQL now automatically updates your read replicas when you perform self-service maintenance on the primary instance. Cloud SQL now supports near-zero downtime planned maintenance on HA-enabled Cloud SQL Enterprise Plus instances with all combinations of public IP connectivity. The rollout of the following extensions and flags is underway: Extensions autoinc (version 1.0): provides functions for incrementing fields automatically.

Cloud SQL SQL Server - Cloud SQL now automatically updates your read replicas when you perform self-service maintenance on the primary instance.

Transcoder API - v1. You can now convert the input video in a transcoding job to a supported high dynamic range (HDR) format.

Vertex AI - M116 release The M116 release of Vertex AI Workbench user-managed notebooks includes the following: Updated custom container user-managed notebooks to use NVIDIA driver version 535.104.05. The M116 release of Vertex AI Workbench managed notebooks includes the following: Fixed a bug (present in versions M113 through M115) that prevented new local kernels from being usable. M116 release The M116 release of Vertex AI Workbench instances includes the following: Removed the Cloud Storage browser in the left side pane in favor of the existing Mount shared storage button. The following models have been added to Model Garden: Stable Diffusion XL LCM: The Latent Consistency Model (LCM) enhances text-to-image generation in Latent Diffusion Models by enabling faster and high-quality image creation with fewer steps. The following changes have been made to Model Garden: Added one-click tuning button, and dedicated deployment, tuning, quantization, and evaluation notebooks for Llama 2. Query an index from the Vector Search console Vector Search has launched an improved console experience for querying both private and public deployed indexes, now available in Preview.

Virtual Private Cloud - Support for IPv6 extension headers is available in General Availability.