Google Cloud Weekly - GCP Newsletter #391
Welcome to issue #391 March 25th, 2024NewsAI Official Blog Partners Vertex AIAnthropic’s Claude 3 Sonnet and Claude 3 Haiku are now generally available on Vertex AI - Claude 3 Sonnet and Claude 3 Haiku are generally available to all customers on Vertex AI. BigQuery Official Blog PartnersCombine data across BigQuery and Salesforce Data Cloud securely with zero ETL - Bidirectional data sharing between BigQuery and Salesforce Data Cloud is now generally available. BigQuery Official Blog Serverless SparkUnify analytics with Spark procedures in BigQuery, now generally available BigQuery Official BlogIntroducing new BigQuery features to simplify time-series data analysis - New features simplify writing queries that perform two of the most common time series operations: windowing and gap filling. Cloud Dataflow Official BlogAt least once Streaming: Save up to 70% for Streaming ETL workloads - Introducing at-least-once streaming mode and comparison with exactly-once processing for streaming jobs. Google Kubernetes Engine Official BlogTake control of GKE scaling with new quota monitoring Cloud Run Official Blog ServerlessIntroducing Cloud Run volume mounts: connect your app to Cloud Storage or NFS - With volume mounts, mounting a volume in a Cloud Run service or job is a single command. You can mount a Cloud Storage bucket or an NFS share, like a Cloud Filestore instance. Compute Engine HPC Official BlogRocky Linux 8 and CentOS 7 versions of HPC VM image now generally available - With these HPC VM images, it's easy to build an HPC-ready VM instances. Active Assist Data Studio FinOps Official Blog TerraformUnlock cloud savings with new Looker Studio Dashboard and Terraform scripts - An OSS Recommendations Dashboard to view cost savings recommendations for optimizing Google Cloud resources. NetApp Official Blog VMware EngineGoogle Cloud VMware Engine supercharged with Google Cloud NetApp Volumes - By combining the simplicity, performance, and advanced data management capabilities of NetApp Volumes with the reliability and operational efficiency of Google Cloud VMware Engine, organizations can achieve agility and cost savings. Application Integration NoSQL Official Blog PartnersLeveraging Couchbase connector and Application Integration in the Google ecosystem - Application Integration now supports Couchbase connector, to empower users to efficiently manage Couchbase NoSQL databases within their integration flows. Official Blog SecurityIntroducing stronger default Org Policies for our customers - With the release of secure-by-default organization resources, potentially insecure postures and outcomes are addressed with a bundle of organization policies that are enforced as soon as a new organization resource is created. Google Kubernetes Engine GPU Official BlogAutomatic driver installation simplifies using NVIDIA GPUs in GKE - GKE can now automatically install NVIDIA GPU drivers, making it easier for customers to take advantage of GPUs. AI Official BlogGoogle named a Leader in The Forrester Wave: AI Infrastructure Solutions, Q1 2024 SponsorArticles, TutorialsInfrastructure, Networking, Security, KubernetesAI Google Kubernetes Engine Official BlogHow to secure Ray on Google Kubernetes Engine - Sharing security insights and hardening techniques for training AI/ML workloads on Ray framework. GCP Experience Official BlogHow Deutsche Bank uses Google Distributed Cloud for its Autobahn FX real-time trading platform GCP Experience Official Blog VPC Service ControlsHow Commerzbank safeguards its data with VPC Service Controls Cloud Armor Official BlogHow to improve resilience to DDoS attacks with Cloud Armor Advanced rate limiting capabilities - Google Cloud Armor's always-on Layer 3 and Layer 4 DDoS defense, Web Application Firewall (WAF), Adaptive Protection, bot management, threat intelligence, and rate-limiting capabilities can help enterprises build a comprehensive DDoS mitigation strategy. Assured workloads Official Blog Public SectorHow to set compliance controls for your Google Cloud Organization FinOpsEnter the FinOps Universe: The Team-Up of Finance and Cloud Tech - Some thoughts on how to organize FinOps for your organization. Google Kubernetes Engine StackdriverControll K8s logging Cost on GCP - How to cut down 80% on k8s logging on cloud? Private Service Connect SecurityAccessing Google APIs via Private Service Connect and Private Google Access - Using PSC and PGA to for accessing Google APIs privately. Google Kubernetes Engine Istio NetworkingHow to install and use Istio Ambient Mesh on GKE - A complete guide on how to install Istio in Ambient Mesh mode on Google Kubernetes. App Development, Serverless, Databases, DevOpsCloud SQL GCP Experience Official Blog PartnersNetRise elevates their user experience with Google Cloud AlloyDB Cloud Spanner Official BlogChoosing a suitable database for your startup: An overview of AlloyDB and Spanner Cloud Memorystore GCP Experience Official BlogStatsig supports up to 7.5 million QPS with Memorystore for Redis Cluster Cloud Load Balancing Cloud RunHow to connect your Google load balancing with Google Cloud Run - This article explains how to connect Cloud Run to Load Balancer. Artifact Registry DevOps GitHub Workload Identity FederationPush code with GitHub Actions to Google Cloud’s Artifact Registry - Push code from GitHub to Google Cloud Artifact Registry (without using service account keys) with Workload Identity Federation. Cloud Build Cloud Run Data Science Machine Learning PythonDeploy A Production-Ready Streamlit App with Cloud Run and Cloud Build - How to ship containerized applications on a serverless architecture and over a CICD pipeline. Cloud Build DevOps DockerModernizing cloudbuild.yaml for Container Builds - Minimizing square brackets in Cloud Build yaml file. DevOps Google Cloud PlatformBig Data, Analytics, ML&AIBigQuery GCP Experience Official Blog PartnersBuilt with BigQuery: How Pendo Data Sync maximizes ROI on your data GCP Experience Official Blog PartnersHow Palo Alto Networks uses BigQuery ML to automate resource classification GCP Experience Official Blog PartnersHow RealTruck drives data reliability and business growth with Masthead and BigQuery AI Google Kubernetes Engine Official BlogWhy GKE for your Ray AI workloads? Portability, scalability, manageability, cost - This blog post explains the numerous benefits that running Ray on GKE brings to the table — scalability, cost-efficiency, fault tolerance, isolation, and portability, to name a few — and resources on how to get started. AI Google Kubernetes Engine Official BlogAdvanced scheduling for AI/ML with Ray and Kueue - In this blog dives into how KubeRay and Kueue work together to orchestrate advanced scheduling for Ray applications on GKE. AI Google Kubernetes Engine GPU Official Blog PartnersAccelerate your generative AI journey with NVIDIA NeMo framework on GKE - This blog post shows how generative AI models can be adapted to your use cases by demonstrating how to train models on Google Kubernetes Engine (GKE) using NVIDIA accelerated computing and NVIDIA NeMo framework. BigQuery Generative AI LLMIn-Place LLM Insights: BigQuery & Gemini for Structured & Unstructured Data Analytics - Introduction. Neo4j Vertex AIBuilding Knowledge Graphs from Scratch Using Neo4j and Vertex AI BigQuery Billing FinOps Official BlogBigQuery customers save up to 54% in TCO compared to alternative cloud data platforms - A report from TechTarget’s Enterprise Strategy Group. BigQueryDELETE + INSERT vs MERGE in BigQuery - How do you merge changes from staging tables into target tables in BigQuery? BigQuery dbtTable Snapshots: BigQuery vs dbt - How do you know which one to use? BigQueryRANGE data type in BigQuery - Explanation of new BigQuery RANGE datatype. VariousGoogle Cloud Platform Official BlogThe future of infrastructure modernization: how Google Cloud Innovators are embracing the cloud Slides, Videos, AudioKubernetes Podcast - #221 Creating Envoy, with Matt Klein. Security Podcast - #164 Quantum Computing: Understanding the (very serious) Threat and Post-Quantum Cryptography. ReleasesChronicle - Chronicle has added a new rule set to Cloud Threat Detections , called Serverless Threats, that detects activity associated with potential compromise or abuse of server-less resources in Google Cloud, such as Cloud Run and Cloud Functions. Chronicle now supports direct ingestion and parsing of reCAPTCHA Enterprise logs from Google Cloud. There is no longer a limit on the number of feeds you can create for the same log type in Feed Management. Cloud Composer - The Logs in Cloud Logging only feature is gradually rolled out to all regions: New Cloud Composer environments now save Airflow task logs only in Cloud Logging by default. Cloud Composer 2.6.5 release started on March 19, 2024. Airflow 2.7.3 is available in Cloud Composer images. Fixed an issue where past Airflow task instances could be marked as failed in some cases. Fixed an issue where Airflow task logs for the first try of a task might not be visible in Airflow UI. BigQueryInsertJobOperator now correctly handles ephemeral tables created with tableDefinitions. In BigQueryInsertJobOperator, fixed the handling of parsing errors during Lineage emission when the query is too long or deeply nested. The apache-airflow-providers-google package is upgraded to version 10.16.0 in images with Airflow 2.6.3, and images with Airflow 2.7.3 have this version. The apache-airflow-providers-cncf-kubernetes package was upgraded to version 8.0.1 in images with Airflow 2.6.3, and images with Airflow 2.7.3 have this version. Cloud Composer 2.6.5 images are available: composer-2.6.5-airflow-2.7.3 composer-2.6.5-airflow-2.6.3 (default) composer-2.6.5-airflow-2.5.3. Cloud Composer versions 2.1.10, 2.1.9, 1.20.10, and 1.20.9 have reached their end of full support period. Compute Engine - Generally available: Disaster recovery with Persistent Disk Async Replication has been expanded to allow you to replicate data on a disk in one region to any other region within the same continent. Generally available: In a managed instance group (MIG), you can set metadata and labels for all VMs in the group without the need to create a new instance template. Generally available: In a managed instance group (MIG), you can turn off repairs to inspect failed and unhealthy VMs, to implement your own repair logic, or to monitor the application health without triggering repairs by MIG. Generally available: The organization-wide patch status dashboard and organization-wide OS policy compliance reports in VM Manager are now generally available. Container Registry - Container Registry is scheduled to be shut down and superseded by Artifact Registry on March 18, 2025. Dataproc Serverless - Announcing the Preview release of Dataproc Serverless for Spark 1.2 runtime: Spark 3.5.0 BigQuery Spark Connector 0.35.1 Cloud Storage Connector 3.0.0 Conda 23.11 Java 17 Python 3.12 R 4.3 Scala 2.12. New Dataproc Serverless for Spark runtime versions: 1.1.55 1.2.0-RC1 2.0.63 2.1.42 2.2.0-RC15. Dataproc Serverless for Spark: Upgraded Spark RAPIDS plugin to version 24.2.0 in the latest runtimes. Dataproc - New Dataproc on Compute Engine subminor image versions: 2.0.96-debian10, 2.0.96-rocky8, 2.0.96-ubuntu18 2.1.44-debian11, 2.1.44-rocky8, 2.1.44-ubuntu20, 2.1.44-ubuntu20-arm 2.2.10-debian12, 2.2.10-rocky9, 2.2.10-ubuntu22. Deep Learning Containers - M118 release Pytorch 2.1.0 with CUDA 12.1 and Python 3.10 container images are now available. Deep Learning VM - M118 release Restored legacy gpu image families for TensorFlow 2.12 through 2.14, and for PyTorch 2.0. Cloud Data Loss Prevention - The discovery and inspection services, which support BigQuery, now support tables that contain columns with INTERVAL, RANGE Eventarc - Eventarc support for creating triggers for direct events from Network Services is generally available (GA). Google Kubernetes Engine - (2024-R08) Version updates GKE cluster versions have been updated. Cilium cluster-wide network policies are now generally available with the following GKE versions: 1.28.6-gke.1095000 or later 1.29.1-gke.1016000 or later You can now control your GKE workloads' ingress and egress traffic cluster-wide, without being bound to a namespace for your network policies. Workloads running on GKE clusters with COS-based nodes may experience DNS resolution issues. Cloud Monitoring - You can now use Duet AI for Developers to help you create a synthetic monitor. reCAPTCHA Enterprise - reCAPTCHA Enterprise platform logs are now available in Chronicle. reCAPTCHA Enterprise Mobile SDK v18.5.0-beta02 is now available for Android and iOS. Cloud Run - Direct VPC egress (Preview) is now available in the following additional regions: africa-south1 asia-south1 asia-southeast2 australia-southeast2 europe-central2 europe-west2 europe-west6 europe-west8 europe-west9 europe-west10 me-central1 me-central2 southamerica-west1 us-east5 us-west2 us-west3 us-west8. Cloud Run services can now connect to a Firestore database using integrations (Preview). Cloud Run services can now connect to Vertex AI to access generative AI models using integrations (Preview). You can now mount an NFS file share as a volume for Cloud Run services and jobs. Cloud Spanner - Leader-aware routing now dynamically routes read-write transactions to the leader region in Spanner multi-region instances, reducing latency and improving performance. Statistics for active partitioned data manipulation language (DML) queries are now generally available. Cloud SQL - Cloud SQL Enterprise Plus edition now supports the me-central2 (Dammam) region. Cloud Storage - You can now use the GCS FUSE file cache feature, a client-based read cache that lets repeat file reads to be served from a faster cache storage of your choice. Vertex AI - Vector Search heuristics-based compaction Vector Search uses heuristics-based metrics assess whether to trigger compaction. The M118 release of Vertex AI Workbench user-managed notebooks includes the following: Pytorch 2.1.0 with CUDA 12.1 and Python 3.10 user-managed notebooks instances are now available. The M118 release of Vertex AI Workbench managed notebooks includes the following: Updated Nvidia drivers to R535, which fixed a bug where the latest PyTorch 2.0 kernel didn't work due to outdated drivers. The M118 release of Vertex AI Workbench instances includes the following: Updated Nvidia drivers to R535. VPC Service Controls - Beta stage support for the following integration: App Hub. AlloyDB - Updated the default major version of PostgreSQL compatibility for new AlloyDB clusters to PostgreSQL 15. The Quotas documentation is updated to include additional guidance on setting the maximum number of concurrent connections for your database instance size. Anthos Config Management - Config Controller now uses the following versions of its included products: Config Connector v1.113.0, release notes Anthos Config Management v1.17.2, release notes. Anthos Config Management - 1.17.3. The constraint template library includes a new template: K8sPSSRunAsNonRoot. Policy Controller bundles have been updated to the following versions: cis-gke-v1.4.0: 202402.0-preview, nist-sp-800-190: 202402.0, nist-sp-800-53-r5: 202402.0, pci-dss-v3.2.1: 202402.0, pss-baseline-v2022: 202402.0, pss-restricted-v2022: 202402.0. Fixed a regression introduced in 1.16.0 that limits the length of the Secret name referenced in the spec.git.secretRef.name field of the RootSync object. Fixed a regression introduced in 1.17.0 that caused Config Sync to sometimes fail to pull the latest commit from a Git branch by upgrading git-sync (Config Sync dependency for pulling from git) from v4.1.0 to v4.2.1. Anthos clusters on bare metal - 1.28. Release 1.28.300-gke.131 GKE on Bare Metal 1.28.300-gke.131 is now available for download. Functionality changes: Updated preflight checks to add a check for networking kernel modules. Fixes: Fixed an issue with configuring a proxy for your cluster that required you to manually set HTTPS_PROXY and NO_PROXY environment variables on the admin workstation. The following container image security vulnerabilities have been fixed in 1.28.300-gke.131: High-severity container vulnerabilities: CVE-2022-28948 CVE-2023-29499 Medium-severity container vulnerabilities: CVE-2023-3446 CVE-2023-3817 CVE-2023-32611 CVE-2023-32665 CVE-2023-49290 CVE-2024-21664 GHSA-2c7c-3mj9-8fqh Low-severity container vulnerabilities: CVE-2021-25743 CVE-2023-2975. Known issues: For information about the latest known issues, see GKE on Bare Metal known issues in the Troubleshooting section. 1.15. Release 1.15.11 GKE on Bare Metal 1.15.11 is now available for download. The following container image security vulnerabilities have been fixed in 1.15.11: Medium-severity container vulnerabilities: CVE-2023-46218 CVE-2023-49290 CVE-2024-21664 GHSA-2c7c-3mj9-8fqh Low-severity container vulnerabilities: CVE-2021-25743. Known issues: For information about the latest known issues, see GKE on Bare Metal known issues in the Troubleshooting section. Anthos clusters on VMware - GKE on VMware 1.28.300-gke.123 is now available. Increased the default memory limit for node-exporter. The following issues are fixed in 1.28.300-gke.123: Fixed the issue where the admin cluster backup did a retry on non-idempotent operations. Artifact Registry - Effective March 22, 2024, Artifact Registry npm repositories enforce not including uppercase letters in package names in order to match npmjs naming rules. Fixed the issue causing images copied to Artifact Registry from Container Registry with the automatic migration tool to fail to propagate their creation time to Artifact Registry. Cloud Asset Inventory - The following resource types are now publicly available through the ExportAssets, ListAssets, BatchGetAssetsHistory, QueryAssets, Feed, and Search (SearchAllResources, SearchAllIamPolicies) APIs. Bare Metal Solution - You can now create and manage VRFs for the networks in your Bare Metal Solution environment. BigQuery - The March 20, 2024 release notes announced the preview for user-defined aggregate functions, but user-defined aggregate functions are not yet supported. You can now add Salesforce Data Cloud data to BigQuery. Incremental materialized views now support LEFT OUTER JOIN and UNION ALL. You can use the CREATE AGGREGATE FUNCTION statement to create user-defined aggregate functions. The maximum notebook size has been increased from 10 MB to 20 MB. You can now view lists of all saved queries and all notebooks in your project. You can now create and run Spark stored procedures that are written in Python, Java, and Scala. The minimum duration between scheduled queries has been reduced from 15 minutes to 5 minutes. You can now undelete a dataset that is within your time travel window to recover it to the state that it was in when it was deleted. These BigQuery features are now generally available (GA): Text analysis configuration options for the following: CREATE SEARCH INDEX DDL Existing LOG_ANALYZER and new PATTERN_ANALYZER analyzers, which are used in various functions, including SEARCH The TEXT_ANALYZE function. BigQuery ML - The following advanced processing functions: ML.BAG_OF_WORDS ML.TF_IDF BAG_OF_WORDS TF_IDF COSINE_DISTANCE EUCLIDEAN_DISTANCE EDIT_DISTANCE. You can now perform hierarchical forecasts in BigQuery ML time series models, which let you aggregate and roll up values for all time series in the model. BigTable - You can now view Bigtable cost data with instance granularity in the Google Cloud Billing detailed export to BigQuery. You can now create daily backups of your Bigtable table by enabling automated backup. |
If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com
Have a great week,
Zdenko
Older messages
GCP Newsletter #390
Monday, March 18, 2024
Welcome to issue #390 March 18th, 2024 News Cloud Dataflow Official Blog Save up to 40 percent with Dataflow streaming committed use discounts - Today, we are announcing the general availability of
GCP Newsletter #389
Monday, March 11, 2024
Welcome to issue #389 March 11th, 2024 News AI Google Kubernetes Engine Official Blog Running AI on fully managed GKE, now with new compute options, pricing and resource reservations Google Kubernetes
GCP Newsletter #388
Monday, March 4, 2024
Welcome to issue #388 March 4th, 2024 News BigQuery Official Blog Vertex AI Unleash the power of generative AI with BigQuery and Vertex AI - New innovations for BigQuery and Vertex AI. AlloyDB
GCP Newsletter #387
Monday, February 26, 2024
Welcome to issue #387 February 26th, 2024 News Google Distributed Cloud Edge Official Blog Introducing Google Distributed Cloud for retail and manufacturing Google Maps Platform Official Blog Next
GCP Newsletter #386
Monday, February 19, 2024
Welcome to issue #386 February 19th, 2024 News BigQuery Official Blog Introducing vector search in BigQuery - The public preview of vector search in BigQuery, which enables vector similarity search on
You Might Also Like
Humanoid robots for factories and for homes - Sync #483
Saturday, September 7, 2024
Plus: new rumours about OpenAI's new models; DeepMind AlphaProteo; DIY, pirated medicine; OpenAI and Anthropic to share models with US government; and more! ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
🐧 Maybe You Shouldn't Try Linux — What to Know About Apple HomeKit
Saturday, September 7, 2024
Also: I Don't Want a Half-Life:3 At This Point, and More! How-To Geek Logo September 7, 2024 Did You Know In the mid-1990s, a microbiologist named Raul Cano successfully revived a 45 million-year-
Daily Coding Problem: Problem #1549 [Easy]
Saturday, September 7, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Google. In linear algebra, a Toeplitz matrix is one in which the elements on any given
Ranked | Revenue of Top Sports Teams in North America, by League 🏆
Saturday, September 7, 2024
From the Dallas Cowboys to the New York Yankees, we compare the highest-earning sports teams as demand for professional sports climbs higher. View Online | Subscribe Presented by: OANDA FEATURED STORY
⚙️ The ultimate guide to intelligent automation
Saturday, September 7, 2024
Understand how to effectively use AI for complex tasks.
Inside the Secrets of Physical Penetration Testing
Saturday, September 7, 2024
Top Tech Content sent at Noon! A dev conference with discussions, workshops, and 1:1 feedback sessions Read this email in your browser How are you, @newsletterest1? 🪐 What's happening in tech today
🐍 New Python tutorials on Real Python
Saturday, September 7, 2024
Hey there, There's always something going on over at realpython.com as far as Python tutorials go. Here's what you may have missed this past week: Quiz: Generate Images With DALL·E and the
North Korean Threat Actors Deploy COVERTCATCH Malware via LinkedIn Job Scams
Saturday, September 7, 2024
THN Daily Updates Newsletter cover Backups: The Key to Cybersecurity How Much Cybersecurity is Enough? Recovery + Resistance = Resilience Download Now Sponsored LATEST NEWS Sep 7, 2024 North Korean
Content Neverending
Saturday, September 7, 2024
What do you know about how Flickr started? ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
📧 Refit in .NET: Building Robust API Clients in C#
Saturday, September 7, 2024
Refit in .NET: Building Robust API Clients in C# Read on: my website / Read time: 8 minutes The .NET Weekly is brought to you by: The State of Designer-Developer Collaboration 2024 Survey Take