Welcome to issue #402 June 10th, 2024

News

What’s new in PostgreSQL 16: New features available in Cloud SQL today - PostgreSQL 16 is now available in Cloud SQL, bringing new features for deeper insights into database operations and enhanced usability. Improvements include observability enhancements such as the pg_stat_io view for Input/Output (IO) behavior, last_seq_scan and last_idx_scan columns in pg_stat_*_tables for tracking sequential and index scans, and n_tup_newpage_upd column in pg_stat_*_tables for monitoring updates.

Introducing GKE Compliance: Maintain clusters and workloads against industry standards - GKE Compliance is a new feature in Google Kubernetes Engine (GKE) Enterprise that helps customers maintain compliance with industry standards, benchmarks, and control frameworks. It provides centralized compliance reporting, in-depth compliance reports, and the ability to manage compliance at scale. GKE Compliance is fully managed by Google and included in the GKE Enterprise license at no additional cost.

Boost developer productivity with new pipeline validation capabilities in Dataflow - Dataflow pipeline validation is now generally available. It performs dozens of checks to ensure that your batch or streaming job is error-free and can run successfully.

Announcing BigQuery Encrypt and Decrypt function compatibility with Sensitive Data Protection - BigQuery now offers seamless integration with Sensitive Data Protection, supporting native SQL functions that enable interoperable deterministic encryption and decryption between BigQuery and Sensitive Data Protection. This integration simplifies the protection of sensitive data in various scenarios, including securing PII, healthcare records, and financial data, and facilitating secure data sharing with external parties. Enhanced performance optimizations leverage BigQuery's distributed architecture to accelerate token-based encryption and decryption tasks, significantly improving the overall performance of these operations.

Google is a Leader in The Forrester Wave™: AI Foundation Models for Language, Q2 2024 - Google is recognized as a Leader in The Forrester Wave™: AI Foundation Models for Language, Q2 2024, receiving the highest scores in Current Offering and Strategy categories.

Introducing Google Cloud NetApp Volumes Flex volumes, auto-tiering, and more - Google Cloud NetApp Volumes, a fully managed file storage service, now offers the Flex storage service level, enabling smaller storage pools and volumes, and zonal or regional storage options.

All Google Cloud courses and labs are now available at no cost through Innovators - Google Cloud Innovators, a free developer program, now offers members 35 monthly learning credits for courses and hands-on labs on Google Cloud Skills Boost. These credits renew monthly, allowing members to continuously learn and earn skill badges.

In an AI era, forge a path to enduring business innovation with Google Cloud and SAP - Google Cloud and SAP are collaborating to provide a comprehensive, open, and multi-cloud platform that enables businesses to get the most out of their data with analytics and generative AI. New advancements include an AI-powered solution for resilient supply chains, an integrated analytics offering with SAP Datasphere, the introduction of Google Cloud's memory-optimized X4 instances, regional expansions for RISE with SAP, SAP BTP, and SAP Datasphere, and multiple Gemini updates.

---

Your cloud, simplified

DoiT delivers technology and cloud expertise to buy, optimize, and manage Google Cloud with ease. Access a global team of cloud experts with decades of experience in cloud architecture, Kubernetes, machine learning, and much more – all on call for you. Learn More

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

How Philips Hue uses Google Cloud Certificate Authority Service to enable Matter integration - Philips Hue, a global leader in lighting and smart home solutions, uses Google Cloud Certificate Authority Service (CAS) to enable secure communication and connection for its IoT devices. By leveraging Google Cloud CAS, Philips Hue benefits from increased agility and scalability, optimized costs, improved security and reliability, and enhanced innovation. This collaboration accelerates innovation in the IoT sector and establishes cloud-based Certificate Authorities as the preferred approach for IoT vendors.

Ransomware Rebounds: Extortion Threat Surges in 2023, Attackers Rely on Publicly Available and Legitimate Tools

5 more myths about platform engineering: how it’s built, what it does, and what it doesn’t - Platform engineering is a new approach to managing IT infrastructure and software development that aims to streamline the software development process by providing developers with self-service tools and platforms, abstracting away complex infrastructure details, and automating repetitive tasks. It's not a one-size-fits-all solution and requires a tailored approach to meet your organization's specific needs. Start small with a minimal viable platform, prioritize high-value tasks, and iterate based on feedback to build a platform that truly delivers value to your developers and organization.

Phishing for Gold: Cyber Threats Facing the 2024 Paris Olympics - The 2024 Paris Olympics face an elevated risk of cyber threats, including cyber espionage, disruptive and destructive operations, financially-motivated activity, hacktivism, and information operations.

App Development, Serverless, Databases, DevOps

Aviator scales developer collaboration with Google Cloud - By using Google Cloud products like Kubernetes Engine, Managed Service for Prometheus, and Cloud SQL, Aviator streamlines code reviews, accelerates review cycles, simplifies merging, and generates release notes automatically.

Empowered development: GitLab on Google Cloud for streamlined delivery and enhanced security - Modernize your software delivery with an integrated solution that increases speed, security, and scalability.

Enhance business continuity for your on-premises SQL Server instance with Cloud SQL for SQL Server - Cloud SQL for SQL Server can be used as a disaster recovery (DR) instance for on-premises SQL Server instances. This can be achieved by setting up replication of regular backups to Cloud SQL for SQL Server.

Building PDF Open Source Services with Angular & GCP — Part 1 - This article introduces the project of building open-source PDF services using Angular, Firestore, Cloud Storage, and CloudRun.

Building PDF Open Source Services with Angular & GCP — Deploy services to Cloud Run - Second part of PDF service project.

Sending GCP Alerts to ServiceNow - This blog outlines the process of integrating Google Cloud Monitoring (GCM) alerts with ServiceNow, enabling automatic incident creation in ServiceNow when GCM alerts are triggered.

Sending Alerts to MS Teams from GCP Monitoring - This blog post explains how to integrate Google Cloud Platform (GCP) alerts with Microsoft Teams, enabling real-time notifications and streamlined collaboration.

Deploying Next.js on Firebase App Hosting - Next.js deployment on Firebase App Hosting has evolved from Cloud Functions to Cloud Run, supporting container-based deployments. Firebase App Hosting now offers a user-friendly GUI in the management console for deployments, similar to Vercel. Automatic framework detection and build processes are handled by firebase-framework-tools, enabling seamless deployment of Next.js applications.

Multiple Cloud Build Triggers in One Github Repo - This blog post outlines the usage of one repo for multiple Cloud Functions and creating a trigger for each one.

Big Data, Analytics, ML&AI

Chugai Pharmaceutical: Accelerating drug discovery through AI, machine learning and data analysis - Chugai Pharmaceutical, a leading pharmaceutical company, is accelerating drug discovery through AI, machine learning, and data analysis. The company is leveraging BigQuery for data analysis and app development, and plans to expand the use of Google Cloud to its subsidiary Tech Kobo for mobile application development and mBaaS using Firebase.

Helping marketers access data warehousing with Supermetrics Storage - Supermetrics Storage centralizes cross-channel marketing data in BigQuery, enabling faster and more efficient data access and analysis without the need for technical expertise. It offers data ownership, a single source of truth, faster report loading, and access to historical data for advanced analytics. Built with BigQuery, Supermetrics Storage provides speed, scale, and security, reducing the need for technical skills and allowing marketers to focus on impactful reporting.

Hands on with Gemini models in BigQuery: Decoding sentiment in customer reviews - Gemini, Google's most advanced multimodal model, can be used in BigQuery for sentiment analysis of customer reviews. With ML.GENERATE_TEXT function, you can directly utilize powerful large language models from Vertex AI within your SQL queries to analyze text in a BigQuery table. This allows you to perform sophisticated text generation and analysis tasks on data stored in BigQuery without needing to move data or write complex code outside the BigQuery environment. You can also identify recurring themes in the reviews using techniques like topic modeling and keyword extraction.

Getting started with retrieval augmented generation on BigQuery with LangChain - The blog demonstrates how to build a simple RAG pipeline using BigQuery and LangChain, and highlights the benefits of using BigQuery Vector Search, which is optimized for large-scale analytical workloads and offers features like scalability, serverless operation, and fine-grained access control.

How to integrate Gemini and Sheets with BigQuery - Gemini, Google's advanced multimodal model, can now be integrated into Google Sheets, enabling users to enhance their productivity with AI-powered features like summarizing sheets and creating formulas. This integration can be achieved directly through Sheets using the urlFetchApp function, or through BigQuery, which serves as a broker between Apps Script and Gemini. With BigQuery, users can create menu items linked to Apps Script functions, allowing them to query Gemini with selected prompts and place the responses in designated columns. This integration empowers users to leverage Gemini's capabilities for various tasks and save time on custom assignments.

6 Easy Ways How to Upload Data into Google Cloud BigQuery - Discover 6 powerful ways to move data into BigQuery: manual upload, Cloud Storage, DataFlow, Pub/Sub, Data Transfer Service, and APIs/SDKs.

Productionise genAI directly in dbt - Using Vertex AI in DBT.

Observability with GCP Cloud Asset Inventory and Looker Studio - Creating an automated pipeline to export data to BigQuery and visualise it with Looker Studio.

Building a Smart Retail Shopping Assistant PART 1 - A Deep Dive into Knowledge-Driven Conversational AI (Part 2 coming.

How We Halved Our Data Pipeline Costs Using Applied AI - A digital marketing agency built a data pipeline using Google Cloud Platform to ingest data from various marketing APIs into BigQuery for analysis. To optimize processing costs, they applied a hill climbing algorithm with a logarithmic probability function to each job, finding the optimal chunk size for faster and cheaper processing. This resulted in a 50% reduction in processing costs within a month of deployment.

Exploring Google Cloud Reasoning Engine - Exploring Reasoning Engine and Gemini multimodal model to have a rich and new experience for the user.

Quizaic — A Generative AI Case Study - Quizaic is a demo application that uses generative AI to create high-quality trivia quizzes and manage the interactive quiz-playing experience. The app is built using Google Cloud Platform, AI, Flutter, Machine Learning, and LLM.

Slides, Videos, Audio

Kubernetes Podcast - #227 10th Anniversary Special with Tim Hockin and Kelsey Hightower.

Security Podcast - #175 Meet Crystal Lister: From Public Sector to Google Cloud Security and Threat Horizons.

GCP Life Podcast - #66 In this episode we discuss; Google I/O Extended, Google undersea cable, Malaysian Data Centre, Finland Data Centre, Telstra Cuts 2800 Jobs, Google Cuts 100 Jobs, MS Cuts Jobs, TikTok Hack, MS Security, Optus Deloitte Report, Medibank Legal Case, NVIDIA Revenue, NVIDIA’s AI Demand, GCP Gen AI Ops, Parliament AI Drafted Inacuracies.

Releases

Anthos clusters on bare metal - 1.28. Release 1.28.600-gke.163 Google Distributed Cloud for bare metal 1.28.600-gke.163 is now available for download. Functionality changes: Updated preflight checks add a check for networking kernel modules (ip_tables or np_tables) and remove the iptables package check. Fixed an issue where the kubelet doesn't honor shortened, 1-second grace period for pod deletion during eviction-based draining. The following container image security vulnerabilities have been fixed in 1.28.600-gke.163: High-severity container vulnerabilities: CVE-2023-39323 CVE-2023-45285 Medium-severity container vulnerabilities: CVE-2023-39318 CVE-2023-39319 CVE-2024-2961 CVE-2023-39326 CVE-2024-28182 Low-severity container vulnerabilities: CVE-2023-5363 CVE-2023-6246 CVE-2023-6779 CVE-2023-6780. Known issues: For information about the latest known issues, see Google Distributed Cloud for bare metal known issues in the Troubleshooting section.

GDCV for VMware - Google Distributed Cloud for VMware 1.16.9-gke.40 is now available for download. The following vulnerabilities are fixed in 1.16.9-gke.40: Container-optimized OS vulnerabilities: CVE-2024-26907 CVE-2024-26882 CVE-2023-50387 CVE-2017-18342 CVE-2024-26883 CVE-2023-29007 CVE-2024-0743 Ubuntu vulnerabilities: CVE-2023-52435 CVE-2023-52486 CVE-2023-52489 CVE-2023-52491 CVE-2023-52492 CVE-2023-52493 CVE-2023-52494 CVE-2023-52498 CVE-2023-52583 CVE-2023-52587 CVE-2023-52588 CVE-2023-52594 CVE-2023-52595 CVE-2023-52597 CVE-2023-52598 CVE-2023-52599 CVE-2023-52601 CVE-2023-52602 CVE-2023-52604 CVE-2023-52606 CVE-2023-52607 CVE-2023-52608 CVE-2023-52614 CVE-2023-52615 CVE-2023-52616 CVE-2023-52617 CVE-2023-52618 CVE-2023-52619 CVE-2023-52622 CVE-2023-52623 CVE-2023-52627 CVE-2023-52631 CVE-2023-52633 CVE-2023-52635 CVE-2023-52637 CVE-2023-52638 CVE-2023-52642 CVE-2023-52643 CVE-2024-1151 CVE-2024-2201 CVE-2024-23849 CVE-2024-26592 CVE-2024-26593 CVE-2024-26594 CVE-2024-26600 CVE-2024-26602 CVE-2024-26606 CVE-2024-26608 CVE-2024-26610 CVE-2024-26614 CVE-2024-26615 CVE-2024-26625 CVE-2024-26627 CVE-2024-26635 CVE-2024-26636 CVE-2024-26640 CVE-2024-26641 CVE-2024-26644 CVE-2024-26645 CVE-2024-26660 CVE-2024-26663 CVE-2024-26664 CVE-2024-26665 CVE-2024-26668 CVE-2024-26671 CVE-2024-26673 CVE-2024-26675 CVE-2024-26676 CVE-2024-26679 CVE-2024-26684 CVE-2024-26685 CVE-2024-26689 CVE-2024-26695 CVE-2024-26696 CVE-2024-26697 CVE-2024-26698 CVE-2024-26702 CVE-2024-26704 CVE-2024-26707 CVE-2024-26712 CVE-2024-26715 CVE-2024-26717 CVE-2024-26720 CVE-2024-26722 CVE-2024-26808 CVE-2024-26825 CVE-2024-26826 CVE-2024-26829 CVE-2024-26910 CVE-2024-26916 CVE-2024-26920 CVE-2023-24023 CVE-2023-52600 CVE-2023-52603 CVE-2024-26581.

Cloud Architecture Center - Infrastructure for a RAG-capable generative AI application using Vertex AI: Added a design alternative that uses Vertex AI Vector Search for the vector store and semantic search components in the architecture. (New guide: 1 of 4) Cross-Cloud Network for distributed applications: Provides an overview about how you can design Cross-Cloud Network for distributed applications. (New guide 2 of 4) Network segmentation and connectivity for distributed applications in Cross-Cloud Network: Describes how to design the network segmentation structure and connectivity of Cross-Cloud Network for distributed applications. (New guide 3 of 4) Service networking for distributed applications in Cross-Cloud Network: Describes how to design Cross-Cloud Network service networking for distributed applications. (New guide 4 of 4) Network security for distributed applications in Cross-Cloud Network: Describes how to design Cross-Cloud Network security for distributed applications.

BigQuery - The BigQuery ML ML.GENERATE_EMBEDDING function now supports the output_dimensionality argument for text-embedding and text-multilingual-embedding models. Analytics Hub data egress controls are now generally available (GA). The slot recommender for editions analyzes historical usage data to recommend optimal capacity purchasing for edition and on-demand workloads.

Billing - You can now view granular cost data for more Google Cloud services You can now view granular Cloud Logging log bucket cost data in the Google Cloud Billing detailed export.

Chronicle SOAR - Release 6.3.5 is now in General Availability. Release 6.3.6 is currently in Preview. Change Alert Priority action does not update the case priority (ID #00277602).

Compute Engine - You can't provision C2 sole tenant nodes with 60 vCPUs. You can now order and request quota for X4 bare metal instances.

Config Connector - Config Connector version 1.118.2 is now available. LoggingLogMetric Change .spec.projectRef.kind from required to be optional.

Data Fusion - Cloud Data Fusion supports annotating resources with tags in Preview.

Database Migration Service - Database Migration Service for heterogeneous Oracle migrations now features additional logging information that can help you better monitor the health and progress of your migration jobs. Database Migration Service for heterogeneous Oracle migrations can now skip foreign keys and triggers, so dropping them from the destination database is no longer required.

Dataflow - Iceberg read/write support is available through the new Managed I/O Java API.

Dataproc Serverless - New Dataproc Serverless for Spark runtime versions: 1.1.64 1.2.8 2.0.72 2.2.8. Dataproc on Compute Engine: Update restartable job error messages to include job IDs. Dataproc Serverless for Spark: Automatically apply goog-dataproc-session-id, goog-dataproc-session-uuid and goog-dataproc-location labels for a session resource.

Dataproc - New Dataproc on Compute Engine subminor image versions: 2.0.105-debian10, 2.0.105-rocky8, 2.0.105-ubuntu18 2.1.53-debian11, 2.1.53-rocky8, 2.1.53-ubuntu20, 2.1.53-ubuntu20-arm 2.2.19-debian12, 2.2.19-rocky9, 2.2.19-ubuntu22. Dataproc on Compute Engine: When creating a cluster with the latest Dataproc on Compute Engine image versions, the secondary worker boot disk type now defaults to the primary worker boot disk type, which is pd-standard if the primary worker boot disk type is not specified.

Dialogflow - Dialogflow CX now offers custom webhook templates for integration with Salesforce. Data store agents: You can now run self-service evaluation which will assess the quality of your data store agent and recommend changes. All generative features: It was announced previously that the text-bison-001 model will be deprecated. Data store agents: The gemini-1.5-flash generative model is now available for selection in the console.

Cloud Data Loss Prevention - From May 27 through June 7, 2024, a bug caused Sensitive Data Protection to sometimes inaccurately populate integer fields as null instead of zero for findings written to BigQuery. The KAZAKHSTAN_PASSPORT infoType detector is available in all regions.

Document AI - Layout Parser in Document AI is generally available.

Backup for GKE - Backup for GKE introduces new policies for handling namespaced resources conflict during restoration that are compatible with GitOps tools. Backup for GKE now supports specifying the restore order when you create or update a restore plan. Backup for GKE now allows configuration of volume data restore policies bound to specific volume types and overridden for specific volumes. Starting June 24, 2024, Backup for GKE will gradually roll out the Backup-Side Restore Validation feature to help ensure that backups are restorable. Backup for GKE now provides enhanced granularity in resource selection during the restore creation process.

Google Kubernetes Engine - Fully managed cAdvisor/Kubelet metrics are now available on GKE clusters running version 1.29.3-gke.1093000 or later. Updated 2024-R13 release notes to indicate that control planes and nodes with auto-upgrade enabled in the Regular channel were not upgraded from version 1.28 to version 1.29.1-gke.1589018. (2024-R18) Version updates GKE cluster versions have been updated. Updated 2024-R03 release notes to indicate that control planes and nodes with auto-upgrade enabled in the Stable channel were upgraded from version 1.27 to version 1.27.7-gke.1121002, not 1.28.3-gke.1203001 as previously stated.

GKE new features - Fully managed cAdvisor/Kubelet metrics are now available on GKE clusters running version 1.29.3-gke.1093000 or later.

GKE - (2024-R18) Version updates Version 1.29.4-gke.1043002 is now the default version.

Google Kubernetes Engine Rapid - (2024-R18) Version updates The following versions are now available in the Rapid channel: 1.26.15-gke.1390000 1.27.14-gke.1042000 1.28.10-gke.1075000 1.29.5-gke.1091000 1.30.1-gke.1261000 The following versions are no longer available in the Rapid channel: 1.26.15-gke.1372001 1.30.1-gke.1139000.

Google Kubernetes Engine Regular - Updated 2024-R13 release notes to indicate that control planes and nodes with auto-upgrade enabled in the Regular channel were not upgraded from version 1.28 to version 1.29.1-gke.1589018. (2024-R18) Version updates Version 1.29.4-gke.1043002 is now the default version in the Regular channel.

Google Kubernetes Engine Stable - (2024-R18) Version updates Version 1.27.11-gke.1062004 is now the default version in the Stable channel. Updated 2024-R03 release notes to indicate that control planes and nodes with auto-upgrade enabled in the Stable channel were upgraded from version 1.27 to version 1.27.7-gke.1121002, not 1.28.3-gke.1203001 as previously stated.

Load Balancing - Bring your own IP lets you bring your own public IPv6 addresses to Google Cloud.

Media CDN - By default, Media CDN proxies only GET, HEAD, and OPTIONS methods to your origin and filters out the methods that can modify your origin.

Migration Center - Generally available: Migration Center lets you download the detailed pricing report as CSV files that include both servers and databases assets. Preview: Detailed description of CloudSQL shape recommendation calculation is available in the detailed pricing report in CSV format.

Cloud Monitoring - You can now pin your event type selections for custom dashboards.

Resource Manager - Cloud Data Fusion supports annotating resources with tags in Preview.

Security Command Center - Vulnerability Assessment for AWS service released to General Availability The Vulnerability Assessment for AWS service, a built-in service of the Enterprise tier of Security Command Center, is released to General Availability.

Sensitive Data Protection - From May 27 through June 7, 2024, a bug caused Sensitive Data Protection to sometimes inaccurately populate integer fields as null instead of zero for findings written to BigQuery. The KAZAKHSTAN_PASSPORT infoType detector is available in all regions.

Service Mesh - 1.21.x. 1.21.3-asm.3 is now available for in-cluster Cloud Service Mesh. The following 3 changes break backwards compatibility in 1.21. 1.18.x. 1.18.7-asm.26 is now available for in-cluster Cloud Service Mesh. 1.19.x. 1.19.10-asm.6 is now available for in-cluster Cloud Service Mesh. 1.20.x. 1.20.7-asm.2 is now available for in-cluster Cloud Service Mesh.

SAP Solutions - New SAP certifications: X4 series of memory-optimized bare metal machine types For use with SAP HANA scale-up (OLAP and OLTP) and SAP NetWeaver workloads, SAP has certified the following Compute Engine memory-optimized bare metal machine types: x4-megamem-1440-metal and x4-megamem-1920-metal.

Cloud Spanner - Query Optimizer version 7 is generally available.

Cloud SQL MySQL - You can now use the gcloud sql instances describe command or the SQL Admin API to retrieve a list of database versions that are available to your MySQL instance for upgrade.

Cloud SQL Postgres - PostgreSQL version 16 is now generally available. You can now use the gcloud sql instances describe command or the SQL Admin API to retrieve a list of database versions that are available to your PostgreSQL instance for upgrade.

Cloud SQL SQL Server - You can now use the gcloud sql instances describe command or the SQL Admin API to retrieve a list of database versions that are available to your SQL Server instance for upgrade.

Cloud Storage - Cloud Storage now offers a new pre-defined dual region, EUROPE-WEST2 (London) and EUROPE-WEST1 (Belgium).

Vertex AI Workbench - You can now create a Vertex AI Workbench instance based on a custom container. v2. You can now use Workforce Identity Federation with Vertex AI Workbench instances in Preview.

Virtual Private Cloud - Support for IPv6 static routes with a next hop instance identified by address (next-hop-address) is available in Preview. Bring your own IP lets you bring your own public IPv6 addresses to Google Cloud.

Agent Assist - Agent Assist now offers a native UI Connector with Genesys Cloud to integrate with Chat conversations. Agent Assist now offers a native UI Connector with Twilio Flex to integrate with chat conversations.