Google Cloud Weekly - GCP Newsletter #391

Welcome to issue #391 March 25th, 2024

News

AI Official Blog Partners Vertex AI

Anthropic’s Claude 3 Sonnet and Claude 3 Haiku are now generally available on Vertex AI - Claude 3 Sonnet and Claude 3 Haiku are generally available to all customers on Vertex AI.

BigQuery Official Blog Partners

Combine data across BigQuery and Salesforce Data Cloud securely with zero ETL - Bidirectional data sharing between BigQuery and Salesforce Data Cloud is now generally available.

BigQuery Official Blog Serverless Spark

Unify analytics with Spark procedures in BigQuery, now generally available

BigQuery Official Blog

Introducing new BigQuery features to simplify time-series data analysis - New features simplify writing queries that perform two of the most common time series operations: windowing and gap filling.

Cloud Dataflow Official Blog

At least once Streaming: Save up to 70% for Streaming ETL workloads - Introducing at-least-once streaming mode and comparison with exactly-once processing for streaming jobs.

Google Kubernetes Engine Official Blog

Take control of GKE scaling with new quota monitoring

Cloud Run Official Blog Serverless

Introducing Cloud Run volume mounts: connect your app to Cloud Storage or NFS - With volume mounts, mounting a volume in a Cloud Run service or job is a single command. You can mount a Cloud Storage bucket or an NFS share, like a Cloud Filestore instance.

Compute Engine HPC Official Blog

Rocky Linux 8 and CentOS 7 versions of HPC VM image now generally available - With these HPC VM images, it's easy to build an HPC-ready VM instances.

Active Assist Data Studio FinOps Official Blog Terraform

Unlock cloud savings with new Looker Studio Dashboard and Terraform scripts - An OSS Recommendations Dashboard to view cost savings recommendations for optimizing Google Cloud resources.

NetApp Official Blog VMware Engine

Google Cloud VMware Engine supercharged with Google Cloud NetApp Volumes - By combining the simplicity, performance, and advanced data management capabilities of NetApp Volumes with the reliability and operational efficiency of Google Cloud VMware Engine, organizations can achieve agility and cost savings.

Application Integration NoSQL Official Blog Partners

Leveraging Couchbase connector and Application Integration in the Google ecosystem - Application Integration now supports Couchbase connector, to empower users to efficiently manage Couchbase NoSQL databases within their integration flows.

Official Blog Security

Introducing stronger default Org Policies for our customers - With the release of secure-by-default organization resources, potentially insecure postures and outcomes are addressed with a bundle of organization policies that are enforced as soon as a new organization resource is created.

Google Kubernetes Engine GPU Official Blog

Automatic driver installation simplifies using NVIDIA GPUs in GKE - GKE can now automatically install NVIDIA GPU drivers, making it easier for customers to take advantage of GPUs.

AI Official Blog

Google named a Leader in The Forrester Wave: AI Infrastructure Solutions, Q1 2024

Sponsor

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

AI Google Kubernetes Engine Official Blog

How to secure Ray on Google Kubernetes Engine - Sharing security insights and hardening techniques for training AI/ML workloads on Ray framework.

GCP Experience Official Blog

How Deutsche Bank uses Google Distributed Cloud for its Autobahn FX real-time trading platform

GCP Experience Official Blog VPC Service Controls

How Commerzbank safeguards its data with VPC Service Controls

Cloud Armor Official Blog

How to improve resilience to DDoS attacks with Cloud Armor Advanced rate limiting capabilities - Google Cloud Armor's always-on Layer 3 and Layer 4 DDoS defense, Web Application Firewall (WAF), Adaptive Protection, bot management, threat intelligence, and rate-limiting capabilities can help enterprises build a comprehensive DDoS mitigation strategy.

Assured workloads Official Blog Public Sector

How to set compliance controls for your Google Cloud Organization

FinOps

Enter the FinOps Universe: The Team-Up of Finance and Cloud Tech - Some thoughts on how to organize FinOps for your organization.

Google Kubernetes Engine Stackdriver

Controll K8s logging Cost on GCP - How to cut down 80% on k8s logging on cloud?

Private Service Connect Security

Accessing Google APIs via Private Service Connect and Private Google Access - Using PSC and PGA to for accessing Google APIs privately.

Google Kubernetes Engine Istio Networking

How to install and use Istio Ambient Mesh on GKE - A complete guide on how to install Istio in Ambient Mesh mode on Google Kubernetes.

App Development, Serverless, Databases, DevOps

Cloud SQL GCP Experience Official Blog Partners

NetRise elevates their user experience with Google Cloud

AlloyDB Cloud Spanner Official Blog

Choosing a suitable database for your startup: An overview of AlloyDB and Spanner

Cloud Memorystore GCP Experience Official Blog

Statsig supports up to 7.5 million QPS with Memorystore for Redis Cluster

Cloud Load Balancing Cloud Run

How to connect your Google load balancing with Google Cloud Run - This article explains how to connect Cloud Run to Load Balancer.

Artifact Registry DevOps GitHub Workload Identity Federation

Push code with GitHub Actions to Google Cloud’s Artifact Registry - Push code from GitHub to Google Cloud Artifact Registry (without using service account keys) with Workload Identity Federation.

Cloud Build Cloud Run Data Science Machine Learning Python

Deploy A Production-Ready Streamlit App with Cloud Run and Cloud Build - How to ship containerized applications on a serverless architecture and over a CICD pipeline.

Cloud Build DevOps Docker

Modernizing cloudbuild.yaml for Container Builds - Minimizing square brackets in Cloud Build yaml file.

DevOps Google Cloud Platform

How to verify your Google App

Big Data, Analytics, ML&AI

BigQuery GCP Experience Official Blog Partners

Built with BigQuery: How Pendo Data Sync maximizes ROI on your data

GCP Experience Official Blog Partners

How Palo Alto Networks uses BigQuery ML to automate resource classification

GCP Experience Official Blog Partners

How RealTruck drives data reliability and business growth with Masthead and BigQuery

AI Google Kubernetes Engine Official Blog

Why GKE for your Ray AI workloads? Portability, scalability, manageability, cost - This blog post explains the numerous benefits that running Ray on GKE brings to the table — scalability, cost-efficiency, fault tolerance, isolation, and portability, to name a few — and resources on how to get started.

AI Google Kubernetes Engine Official Blog

Advanced scheduling for AI/ML with Ray and Kueue - In this blog dives into how KubeRay and Kueue work together to orchestrate advanced scheduling for Ray applications on GKE.

AI Google Kubernetes Engine GPU Official Blog Partners

Accelerate your generative AI journey with NVIDIA NeMo framework on GKE - This blog post shows how generative AI models can be adapted to your use cases by demonstrating how to train models on Google Kubernetes Engine (GKE) using NVIDIA accelerated computing and NVIDIA NeMo framework.

BigQuery Generative AI LLM

In-Place LLM Insights: BigQuery & Gemini for Structured & Unstructured Data Analytics - Introduction.

Neo4j Vertex AI

Building Knowledge Graphs from Scratch Using Neo4j and Vertex AI

BigQuery Billing FinOps Official Blog

BigQuery customers save up to 54% in TCO compared to alternative cloud data platforms - A report from TechTarget’s Enterprise Strategy Group.

BigQuery

DELETE + INSERT vs MERGE in BigQuery - How do you merge changes from staging tables into target tables in BigQuery?

BigQuery dbt

Table Snapshots: BigQuery vs dbt - How do you know which one to use?

BigQuery

RANGE data type in BigQuery - Explanation of new BigQuery RANGE datatype.

Various

Google Cloud Platform Official Blog

The future of infrastructure modernization: how Google Cloud Innovators are embracing the cloud

Slides, Videos, Audio

Kubernetes Podcast - #221 Creating Envoy, with Matt Klein.

Security Podcast - #164 Quantum Computing: Understanding the (very serious) Threat and Post-Quantum Cryptography.

 

Releases

Chronicle - Chronicle has added a new rule set to Cloud Threat Detections , called Serverless Threats, that detects activity associated with potential compromise or abuse of server-less resources in Google Cloud, such as Cloud Run and Cloud Functions. Chronicle now supports direct ingestion and parsing of reCAPTCHA Enterprise logs from Google Cloud. There is no longer a limit on the number of feeds you can create for the same log type in Feed Management.

Cloud Composer - The Logs in Cloud Logging only feature is gradually rolled out to all regions: New Cloud Composer environments now save Airflow task logs only in Cloud Logging by default. Cloud Composer 2.6.5 release started on March 19, 2024. Airflow 2.7.3 is available in Cloud Composer images. Fixed an issue where past Airflow task instances could be marked as failed in some cases. Fixed an issue where Airflow task logs for the first try of a task might not be visible in Airflow UI. BigQueryInsertJobOperator now correctly handles ephemeral tables created with tableDefinitions. In BigQueryInsertJobOperator, fixed the handling of parsing errors during Lineage emission when the query is too long or deeply nested. The apache-airflow-providers-google package is upgraded to version 10.16.0 in images with Airflow 2.6.3, and images with Airflow 2.7.3 have this version. The apache-airflow-providers-cncf-kubernetes package was upgraded to version 8.0.1 in images with Airflow 2.6.3, and images with Airflow 2.7.3 have this version. Cloud Composer 2.6.5 images are available: composer-2.6.5-airflow-2.7.3 composer-2.6.5-airflow-2.6.3 (default) composer-2.6.5-airflow-2.5.3. Cloud Composer versions 2.1.10, 2.1.9, 1.20.10, and 1.20.9 have reached their end of full support period.

Compute Engine - Generally available: Disaster recovery with Persistent Disk Async Replication has been expanded to allow you to replicate data on a disk in one region to any other region within the same continent. Generally available: In a managed instance group (MIG), you can set metadata and labels for all VMs in the group without the need to create a new instance template. Generally available: In a managed instance group (MIG), you can turn off repairs to inspect failed and unhealthy VMs, to implement your own repair logic, or to monitor the application health without triggering repairs by MIG. Generally available: The organization-wide patch status dashboard and organization-wide OS policy compliance reports in VM Manager are now generally available.

Container Registry - Container Registry is scheduled to be shut down and superseded by Artifact Registry on March 18, 2025.

Dataproc Serverless - Announcing the Preview release of Dataproc Serverless for Spark 1.2 runtime: Spark 3.5.0 BigQuery Spark Connector 0.35.1 Cloud Storage Connector 3.0.0 Conda 23.11 Java 17 Python 3.12 R 4.3 Scala 2.12. New Dataproc Serverless for Spark runtime versions: 1.1.55 1.2.0-RC1 2.0.63 2.1.42 2.2.0-RC15. Dataproc Serverless for Spark: Upgraded Spark RAPIDS plugin to version 24.2.0 in the latest runtimes.

Dataproc - New Dataproc on Compute Engine subminor image versions: 2.0.96-debian10, 2.0.96-rocky8, 2.0.96-ubuntu18 2.1.44-debian11, 2.1.44-rocky8, 2.1.44-ubuntu20, 2.1.44-ubuntu20-arm 2.2.10-debian12, 2.2.10-rocky9, 2.2.10-ubuntu22.

Deep Learning Containers - M118 release Pytorch 2.1.0 with CUDA 12.1 and Python 3.10 container images are now available.

Deep Learning VM - M118 release Restored legacy gpu image families for TensorFlow 2.12 through 2.14, and for PyTorch 2.0.

Cloud Data Loss Prevention - The discovery and inspection services, which support BigQuery, now support tables that contain columns with INTERVAL, RANGE, RANGE, and RANGEdata types.

Eventarc - Eventarc support for creating triggers for direct events from Network Services is generally available (GA).

Google Kubernetes Engine - (2024-R08) Version updates GKE cluster versions have been updated. Cilium cluster-wide network policies are now generally available with the following GKE versions: 1.28.6-gke.1095000 or later 1.29.1-gke.1016000 or later You can now control your GKE workloads' ingress and egress traffic cluster-wide, without being bound to a namespace for your network policies. Workloads running on GKE clusters with COS-based nodes may experience DNS resolution issues.

Cloud Monitoring - You can now use Duet AI for Developers to help you create a synthetic monitor.

reCAPTCHA Enterprise - reCAPTCHA Enterprise platform logs are now available in Chronicle. reCAPTCHA Enterprise Mobile SDK v18.5.0-beta02 is now available for Android and iOS.

Cloud Run - Direct VPC egress (Preview) is now available in the following additional regions: africa-south1 asia-south1 asia-southeast2 australia-southeast2 europe-central2 europe-west2 europe-west6 europe-west8 europe-west9 europe-west10 me-central1 me-central2 southamerica-west1 us-east5 us-west2 us-west3 us-west8. Cloud Run services can now connect to a Firestore database using integrations (Preview). Cloud Run services can now connect to Vertex AI to access generative AI models using integrations (Preview). You can now mount an NFS file share as a volume for Cloud Run services and jobs.

Cloud Spanner - Leader-aware routing now dynamically routes read-write transactions to the leader region in Spanner multi-region instances, reducing latency and improving performance. Statistics for active partitioned data manipulation language (DML) queries are now generally available.

Cloud SQL - Cloud SQL Enterprise Plus edition now supports the me-central2 (Dammam) region.

Cloud Storage - You can now use the GCS FUSE file cache feature, a client-based read cache that lets repeat file reads to be served from a faster cache storage of your choice.

Vertex AI - Vector Search heuristics-based compaction Vector Search uses heuristics-based metrics assess whether to trigger compaction. The M118 release of Vertex AI Workbench user-managed notebooks includes the following: Pytorch 2.1.0 with CUDA 12.1 and Python 3.10 user-managed notebooks instances are now available. The M118 release of Vertex AI Workbench managed notebooks includes the following: Updated Nvidia drivers to R535, which fixed a bug where the latest PyTorch 2.0 kernel didn't work due to outdated drivers. The M118 release of Vertex AI Workbench instances includes the following: Updated Nvidia drivers to R535.

VPC Service Controls - Beta stage support for the following integration: App Hub.

AlloyDB - Updated the default major version of PostgreSQL compatibility for new AlloyDB clusters to PostgreSQL 15. The Quotas documentation is updated to include additional guidance on setting the maximum number of concurrent connections for your database instance size.

Anthos Config Management - Config Controller now uses the following versions of its included products: Config Connector v1.113.0, release notes Anthos Config Management v1.17.2, release notes.

Anthos Config Management - 1.17.3. The constraint template library includes a new template: K8sPSSRunAsNonRoot. Policy Controller bundles have been updated to the following versions: cis-gke-v1.4.0: 202402.0-preview, nist-sp-800-190: 202402.0, nist-sp-800-53-r5: 202402.0, pci-dss-v3.2.1: 202402.0, pss-baseline-v2022: 202402.0, pss-restricted-v2022: 202402.0. Fixed a regression introduced in 1.16.0 that limits the length of the Secret name referenced in the spec.git.secretRef.name field of the RootSync object. Fixed a regression introduced in 1.17.0 that caused Config Sync to sometimes fail to pull the latest commit from a Git branch by upgrading git-sync (Config Sync dependency for pulling from git) from v4.1.0 to v4.2.1.

Anthos clusters on bare metal - 1.28. Release 1.28.300-gke.131 GKE on Bare Metal 1.28.300-gke.131 is now available for download. Functionality changes: Updated preflight checks to add a check for networking kernel modules. Fixes: Fixed an issue with configuring a proxy for your cluster that required you to manually set HTTPS_PROXY and NO_PROXY environment variables on the admin workstation. The following container image security vulnerabilities have been fixed in 1.28.300-gke.131: High-severity container vulnerabilities: CVE-2022-28948 CVE-2023-29499 Medium-severity container vulnerabilities: CVE-2023-3446 CVE-2023-3817 CVE-2023-32611 CVE-2023-32665 CVE-2023-49290 CVE-2024-21664 GHSA-2c7c-3mj9-8fqh Low-severity container vulnerabilities: CVE-2021-25743 CVE-2023-2975. Known issues: For information about the latest known issues, see GKE on Bare Metal known issues in the Troubleshooting section. 1.15. Release 1.15.11 GKE on Bare Metal 1.15.11 is now available for download. The following container image security vulnerabilities have been fixed in 1.15.11: Medium-severity container vulnerabilities: CVE-2023-46218 CVE-2023-49290 CVE-2024-21664 GHSA-2c7c-3mj9-8fqh Low-severity container vulnerabilities: CVE-2021-25743. Known issues: For information about the latest known issues, see GKE on Bare Metal known issues in the Troubleshooting section.

Anthos clusters on VMware - GKE on VMware 1.28.300-gke.123 is now available. Increased the default memory limit for node-exporter. The following issues are fixed in 1.28.300-gke.123: Fixed the issue where the admin cluster backup did a retry on non-idempotent operations.

Artifact Registry - Effective March 22, 2024, Artifact Registry npm repositories enforce not including uppercase letters in package names in order to match npmjs naming rules. Fixed the issue causing images copied to Artifact Registry from Container Registry with the automatic migration tool to fail to propagate their creation time to Artifact Registry.

Cloud Asset Inventory - The following resource types are now publicly available through the ExportAssets, ListAssets, BatchGetAssetsHistory, QueryAssets, Feed, and Search (SearchAllResources, SearchAllIamPolicies) APIs.

Bare Metal Solution - You can now create and manage VRFs for the networks in your Bare Metal Solution environment.

BigQuery - The March 20, 2024 release notes announced the preview for user-defined aggregate functions, but user-defined aggregate functions are not yet supported. You can now add Salesforce Data Cloud data to BigQuery. Incremental materialized views now support LEFT OUTER JOIN and UNION ALL. You can use the CREATE AGGREGATE FUNCTION statement to create user-defined aggregate functions. The maximum notebook size has been increased from 10 MB to 20 MB. You can now view lists of all saved queries and all notebooks in your project. You can now create and run Spark stored procedures that are written in Python, Java, and Scala. The minimum duration between scheduled queries has been reduced from 15 minutes to 5 minutes. You can now undelete a dataset that is within your time travel window to recover it to the state that it was in when it was deleted. These BigQuery features are now generally available (GA): Text analysis configuration options for the following: CREATE SEARCH INDEX DDL Existing LOG_ANALYZER and new PATTERN_ANALYZER analyzers, which are used in various functions, including SEARCH The TEXT_ANALYZE function.

BigQuery ML - The following advanced processing functions: ML.BAG_OF_WORDS ML.TF_IDF BAG_OF_WORDS TF_IDF COSINE_DISTANCE EUCLIDEAN_DISTANCE EDIT_DISTANCE. You can now perform hierarchical forecasts in BigQuery ML time series models, which let you aggregate and roll up values for all time series in the model.

BigTable - You can now view Bigtable cost data with instance granularity in the Google Cloud Billing detailed export to BigQuery. You can now create daily backups of your Bigtable table by enabling automated backup.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko

To make sure you keep getting these emails, please add zdenko@gcpweekly.com to your address book or whitelist us. Want out of the loop? Unsubscribe. Our postal address: Třebanická 183, Prague, Prague 14300

Older messages

GCP Newsletter #390

Monday, March 18, 2024

Welcome to issue #390 March 18th, 2024 News Cloud Dataflow Official Blog Save up to 40 percent with Dataflow streaming committed use discounts - Today, we are announcing the general availability of

GCP Newsletter #389

Monday, March 11, 2024

Welcome to issue #389 March 11th, 2024 News AI Google Kubernetes Engine Official Blog Running AI on fully managed GKE, now with new compute options, pricing and resource reservations Google Kubernetes

GCP Newsletter #388

Monday, March 4, 2024

​ Welcome to issue #388 March 4th, 2024 News BigQuery Official Blog Vertex AI Unleash the power of generative AI with BigQuery and Vertex AI - New innovations for BigQuery and Vertex AI. AlloyDB

GCP Newsletter #387

Monday, February 26, 2024

​ Welcome to issue #387 February 26th, 2024 News Google Distributed Cloud Edge Official Blog Introducing Google Distributed Cloud for retail and manufacturing Google Maps Platform Official Blog Next

GCP Newsletter #386

Monday, February 19, 2024

Welcome to issue #386 February 19th, 2024 News BigQuery Official Blog Introducing vector search in BigQuery - The public preview of vector search in BigQuery, which enables vector similarity search on

You Might Also Like

AI search engine startup Perplexity eyes a $3B valuation

Tuesday, April 23, 2024

Plus: It's Tesla earnings day and AWS wants to host your AI models View this email online in your browser By Cody Corrall Tuesday, April 23, 2024 Welcome back to TechCrunch PM. Today we have big

🎞️ We Tried 3D Printing a Photo — You'll Love This Secret Samsung Galaxy Bluetooth Feature

Tuesday, April 23, 2024

Also: Transferring Your Phone Number to a New Carrier, and More! How-To Geek Logo April 23, 2024 📩 Get expert reviews, the hottest deals, how-to's, breaking news, and more delivered directly to

You're invited – product sense, prioritization, careers

Tuesday, April 23, 2024

Product Sense Product Sense Wednesday, May 1st @ 01:00 PM EST Learn how to identify opportunities, assess risks, and make informed decisions that lead to successful product innovations by better

CTRL-C, Exceptions, Ruff Speed-up, and More

Tuesday, April 23, 2024

Asyncio Handle Control-C (SIGINT) #626 – APRIL 23, 2024 VIEW IN BROWSER The PyCoder's Weekly Logo Asyncio Handle Control-C (SIGINT) When the user presses CTRL-C on the keyboard, the OS raises an

Writing Contests Just Landed On Product Hunt 🔥

Tuesday, April 23, 2024

Upvote us to keep the $$$ coming! 👍 ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌ ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌ ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌ ͏ ‌  ͏ ‌  ͏ ‌  ͏ ‌ 

Daily Coding Problem: Problem #1421 [Hard]

Tuesday, April 23, 2024

Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Uber. Given an array of integers, return a new array such that each element at index i

Ranked | The Top 10 EV Battery Manufacturers 🔋

Tuesday, April 23, 2024

Asia dominates this ranking of the world's largest EV battery manufacturers in 2023. See which battery makers feature in the top 10. View Online | Subscribe Presented by: EnergyX's

Bringing PGO to the build pipeline

Tuesday, April 23, 2024

Plus how Go grew at Google, cmp.Or, and ways to visualize makefiles, Go binaries, and live Go processes. | #​504 — April 23, 2024 Unsub | Web Version Together with Three Dots Labs Go Weekly How Dolt

Noonification: Leetcode: Two-sum an Intuitive Approach

Tuesday, April 23, 2024

Top Tech Content sent at Noon! Get Algolia: AI Search that understands How are you, @newsletterest1? 🪐 What's happening in tech this week: The Noonification by HackerNoon has got you covered with

The best AI chatbot for coding

Tuesday, April 23, 2024

9 video gadget must-haves; 6 things Linux should borrow from MacOS -- ZDNET ZDNET Tech Today - US April 23, 2024 placeholder Can Meta AI code? I tested it against Llama, Gemini and ChatGPT - it wasn