Welcome to issue #389 March 11th, 2024

News

Running AI on fully managed GKE, now with new compute options, pricing and resource reservations

GKE provides fully managed Kubernetes support for Elastic Cloud

Announcing Anthropic’s Claude 3 models in Google Cloud Vertex AI - Claude 3 — Anthropic’s new family of state-of-the-art models — will be generally available in Vertex AI Model Garden over the coming weeks.

Confluent brings real-time capabilities to Google Cloud generative AI

Unify customer and partner data with the new entity resolution framework in BigQuery - The BigQuery entity resolution framework allows you to integrate with the identity provider of your choice using standard SQL queries.

Personalized Service Health now in the Google Cloud mobile app

Unveiling Public Sector at Next '24: A Look at Our Gov and Edu Sessions

---

Imagine having a direct line to over 150 senior cloud architects for any cloud-related question or issue you encounter. With thousands of cloud questions and issues resolved, DoiT is your gateway to world-class cloud expertise.

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Build an enterprise developer platform on GKE for fast, reliable application delivery - This blog post describes how Google Kubernetes Engine (GKE) enables you to build a secure, scalable internal developer platform in Google Cloud for fast and reliable application delivery.

Deploying a Gemini Chatbot with Memory on a GKE Backend and Dialogflow

Move-in ready Kubernetes security with GKE Autopilot

Regional vs. zonal GKE clusters: making the right choice for your workloads - In this blog post, we delve into the characteristics and considerations associated with Regiona vs. zonal GKE cluster type to guide you toward an informed decision.

App Development, Serverless, Databases, DevOps

Build generative AI applications with similarity search in Cloud SQL for MySQL

Memorystore for Redis vector search and LangChain integrations for gen AI - An example of how to combine Memorystore for Redis with LangChain to create a chatbot that answers questions about movies.

Build generative AI and similarity search applications at virtually unlimited scale with Spanner - This blog post discusses how vector search can enhance gen AI applications, and how Spanner’s underlying architecture supports extremely large-scale vector search deployments.

Domain-specific AI apps: A three-step design pattern for specializing LLMs - This article embarks on a journey through the key advantages of domain-specific LLMs.

Creating Spring-based gRPC microservices managed by Prometheus and Grafana - This blog post explores integrating gRPC with Spring-based microservices.

Extending your tools, adding matrices to Google Cloud build - Join my journey in overcoming copy-paste chaos and easily enhancing out-of-box tools!

DZ BANK unlocks 70% toil savings and 90% cost savings with a Cloud Run-first approach - DZ BANK shares how migrating to Google Cloud resulted in spectacular efficiency gains and cost savings.

How Glance improves database operations with Spanner - How Glance migrated from Azure Cosmos DB to Cloud Spanner.

Build supercharged gen AI applications with LangChain and Google Cloud databases

Big Data, Analytics, ML&AI

How to Create an RFM Model in BigQuery - Learn what an RFM model is, how to create one, and how to segment on the results.

Dataplex Data Catalog API Default Activation on March 4th 2024 - TL;DR Google is enabling a selection of APIs by default for all Google BigQuery users starting March 4th, 2024, including its Dataplex API amongst others. This could impact your Google Cloud spend if left unchecked.

BigQuery Connector for SAP: Handle table structure changes like a pro! - Part 1 - The BigQuery Connector for SAP offered by Google Cloud is a tool to connect SAP SLT with BigQuery and achieve near real time data-replication.

BigQuery Connector for SAP: Handle table structure changes like a pro! - Part 2

Apply GenAI on Dataset in Data Mesh with HandOns experiment (GCP BigQuery) - Apply GenAI on Dataset in Data Mesh with HandOns experiment (GCP BigQuery).

Tips and tricks to control and reduce your BigQuery bill - 7 tips to reduce BigQuery billing.

Optimising incremental Dataform Pipelines - Combining incremental tables with partition pruning to save cost in BigQuery.

Dividends from data: Building a lean data stack for a Series C Fintech - PrimaryBid is a regulated capital markets technology platform connecting public companies to their communities during fundraisings, in this article they described their data stack.

Benchmarking your Financial Performance using Looker and Google BigQuery

Intelligent Document Discovery with Vertex AI Search - This post explores the capabilities of Vertex AI Search, specifically, website search — a standout feature within Google Cloud’s Vertex AI platform.

eBay accelerates its AI-driven recommendation engine with Vertex AI - By implementing Google Cloud Vertex AI and Vector Search, Ebay accelerated engineering velocity and generated more advanced models to deliver better results to their customers.

Various

Google Cloud Next 2024 : new location, new guide - Google Cloud Next 2024 in Las Vegas: Your Ultimate Guide & Top Tips for Cloud enthusiasts.

Serverless Toronto Meetup - All in AI with Googler Mark Ryan and LlamaIndex Creator Jerry Liu.

Slides, Videos, Audio

Kubernetes Podcast - #220 Falco Graduation, with Mike Coleman.

Security Podcast - #162 IAM in the Cloud: What it Means to Do It 'Right' with Kat Traxler.

Releases

AlloyDB - You can now use Key Access Justifications when working with external CMEK keys.

Anthos clusters on bare metal - 1.28. Release 1.28.200-gke.118 GKE on Bare Metal 1.28.200-gke.118 is now available for download. Fixes: Fixed an issue where upgrades are blocked because cluster-operator can't delete stale, failing preflight check resources. Fixes: The following container image security vulnerabilities have been fixed in 1.28.200-gke.118: Critical container vulnerabilities: CVE-2023-25775 High-severity container vulnerabilities: CVE-2023-5717 CVE-2023-6931 CVE-2023-6932 CVE-2023-35827 CVE-2023-51780 CVE-2023-51781 CVE-2023-51782 CVE-2024-21626 Medium-severity container vulnerabilities: CVE-2023-6004 CVE-2023-6121 CVE-2023-6918 CVE-2021-44879 CVE-2023-34324 CVE-2023-45863 CVE-2023-46343 Low-severity container vulnerabilities: CVE-2023-5178 CVE-2023-5197 CVE-2023-6531 CVE-2023-6817 CVE-2023-46813 CVE-2023-46862 CVE-2024-0193 CVE-2024-0641. Known issues: For information about the latest known issues, see GKE on Bare Metal known issues in the Troubleshooting section. 1.15. Release 1.15.10 GKE on Bare Metal 1.15.10 is now available for download. GKE on Bare Metal version 1.15.10 and later has been qualified on and supports Red Hat Enterprise Linux (RHEL) version 8.9. Fixes: The following container image security vulnerabilities have been fixed in 1.15.10: High-severity container vulnerabilities: CVE-2023-29499 Medium-severity container vulnerabilities: CVE-2023-6004 CVE-2023-6918 CVE-2023-32611 CVE-2023-32665 CVE-2024-0553. Known issues: For information about the latest known issues, see GKE on Bare Metal known issues in the Troubleshooting section.

Anthos clusters on Azure - You can now launch clusters with the following Kubernetes versions.

Cloud Asset Inventory - The following resource types are now publicly available through the ExportAssets, ListAssets, BatchGetAssetsHistory, QueryAssets, Feed, and Search (SearchAllResources, SearchAllIamPolicies) APIs.

BigQuery - The INFORMATION_SCHEMA.WRITE_API_TIMELINE* views, containing per minute aggregated BigQuery Storage Write API ingestion statistics, are now generally available (GA). Duet AI in BigQuery can now assist with Python code generation and code completion. You can now selectively refresh the metadata cache for BigLake tables by using the BQ.REFRESH_EXTERNAL_METADATA_CACHE system procedure.

Cloud Build - Cloud Build repositories (2nd gen) now supports integration with Bitbucket Cloud and Bitbucket Data Center.

Channel Services - v1. Partners selling Workspace can now use the BigQuery Export (Rebilling) feature to programmatically access their Workspace billing data.

Cloud Composer - Cloud Composer 2.6.3 release was rolled back.

Dataflow - Streaming jobs created after March 7, 2024 automatically encrypt all user data with customer-managed encryption keys (CMEK).

Dataproc Metastore - Dataproc Metastore now supports scheduled backups.

Dataproc Serverless - New Dataproc Serverless for Spark runtime versions: 1.1.53 2.0.61 2.1.40 2.2.0-RC13. Dataproc Serverless for Spark: Upgraded Cloud Storage connector to 2.2.20 version in the latest 1.1, 2.0, and 2.1 runtimes. Dataproc Serverless for Spark: Extended Spark metrics collected for a batch now include executor:resultSize, executor:shuffleBytesWritten, and executor:shuffleTotalBytesRead.

Dataproc - New Dataproc Serverless for Spark runtime versions: 1.1.53 2.0.61 2.1.40 2.2.0-RC13. Dataproc Serverless for Spark: Upgraded Cloud Storage connector to 2.2.20 version in the latest 1.1, 2.0, and 2.1 runtimes. Dataproc Serverless for Spark: Extended Spark metrics collected for a batch now include executor:resultSize, executor:shuffleBytesWritten, and executor:shuffleTotalBytesRead.

Datastore - You can now use the Firestore emulator to test Firestore in Datastore mode behavior.

Cloud Deploy - Cloud Deploy support for custom targets is now generally available.

Dialogflow - Vertex AI Conversation now supports the Dialogflow CX Messenger integration for preview. Vertex AI Conversation generative agent creation with playbooks is now available for public preview.

Cloud Data Loss Prevention - A new detection model is available for the LOCATION infoType detector.

Eventarc - Eventarc support for creating triggers for direct Batch events is generally available (GA).

Cloud Functions - Cloud Functions (1st gen) now supports custom service accounts for Cloud Build, at the Preview release level.

IAM - To improve performance, we've removed the ability to expand abbreviated permissions in the predefined roles table.

Google Kubernetes Engine - For GKE versions later than 1.29.1-gke.1760000, the NEG, Ingress, L4 internal load balancer with subsetting, and L4 RBS controllers will skip processing the nodes missing the topology.kubernetes.io/zone label until the zone information is ready. Managed ASM installation and node scaling fails on GKE Autopilot clusters on versions between 1.28.6-gke.1095000 and 1.28.7-gke.1025000 and on versions between 1.29.1-gke.1016000 and 1.29.1-gke.1781000. You can now preload data or container images in new nodes to get fast workload deployment and auto scaling. The following vulnerabilities were discovered in the Linux kernel that can lead to a privilege escalation on Container-Optimized OS and Ubuntu nodes: CVE-2023-3611 For more information, see the GCP-2024-017 security bulletin. Starting in GKE 1.29.2-gke.1035000, you can configure Identity-Aware Proxy (IAP) with Google Managed OAuth Client for load balancers configured through GKE Ingress. (2024-R07) Version updates GKE cluster versions have been updated. (2024-R06) Version updates GKE cluster versions have been updated. NVIDIA H100 (80 GB) GPUs are now available in GKE Autopilot mode in versions 1.28.6-gke.1369000 or later, and 1.29.1-gke.1575000 or later. GPU workloads running in Autopilot mode can now be configured using the Accelerator Compute Class.

Cloud Logging - Log buckets in the africa-south1 region can now be upgraded to use Log Analytics.

Marketplace Partners - When you create a new private offer, or replace an existing private offer, you select a payment frequency for how your customer is charged.

Migrate for Compute Engine 4.8 - 5.0. Preview: Migrate to Virtual Machines lets you import a virtual disk image to a Compute Engine image. Generally available: You can now use Customer-Managed Encryption Keys (CMEK) in Migrate to Virtual Machines to do the following: Protect data stored by Migrate to Virtual Machines during the migration process.

Cloud Monitoring - You can display events, such as the crash of a GKE pod, on your dashboards.

Network Intelligence Center - Network Analyzer now includes an insight that gives a summary of the IP address utilization of all the Private Service Access ranges.

Cloud Run - You can now apply minimum instance configuration at the service level (in Preview).

Secret Manager - Preview: Secret Manager add-on for Google Kubernetes Engine (GKE) is now available in preview.

Service Mesh - Managed Anthos Service Mesh. Managed Anthos Service Mesh 1.18 has completed its rollout in the rapid channel.

SAP Solutions - Generally available: Guided Deployment Automation in Workload Manager for SAP The Guided Deployment Automation tool in Workload Manager is generally available (GA).

Cloud Spanner - Spanner now supports a new client library interface.

Cloud SQL MySQL - You can now upgrade network architecture for your HA-enabled instances in a region, even if you have multiple instances in your network and region.

Cloud SQL Postgres - You can now upgrade network architecture for your HA-enabled instances in a region, even if you have multiple instances in your network and region. The new maintenance version listed in the February 7th entry for PostgreSQL extensions, flags, minor versions, extension versions, and plugin versions is [PostgreSQL version].R20240130.00_05.

Cloud SQL SQL Server - You can now upgrade your Cloud SQL instances to use the new network architecture to get additional capabilities not available in the old network architecture.

Vertex AI - Create an empty index with Vector Search You can create an empty index in Vector Search for batch and for streaming. Vertex AI Prediction You can now use A3 machine types to serve predictions.

VMware Engine - Beginning mid-March 2024, the VMware Engine operations team will upgrade VMware components to newer versions.

VPC Service Controls - General availability support for the following integration: Storage Insights.

Virtual Private Cloud - Internal ranges are available in General Availability.

Workflows - An issue that allowed jumps in or out of parallel branches, parallel loops, and for loops is resolved.