Welcome to issue #427 December 2nd, 2024

News

Australia Connect initiative delivers new digital pathways for the Indo-Pacific - Google Cloud announces the Australia Connect initiative to enhance digital connectivity in Australia and the Indo-Pacific region. The Bosun subsea cable will connect Darwin, Australia to Christmas Island, with an interlink cable connecting Melbourne, Perth, and Christmas Island.

Securing AI: Advancing the national security mission - Google Public Sector is committed to supporting agencies with secure AI solutions and AI-powered security tools. A recent study commissioned by Google found that internal cybersecurity protection is the top AI use case for federal agencies, with 62% identifying strengthening cybersecurity as a key driver for implementing AI. Join Google Public Sector Summit On-Demand on December 3, 2024, to explore how AI can be used to enhance national security while upholding safety and responsibility standards.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Cloud CISO Perspectives: To end ransomware scourge, start with more reporting — not blocking cyber-insurance - In this month's Cloud CISO Perspectives, Monica Shokrai and Kimberly Goody discuss the role of cyber-insurance in combating ransomware. They argue that mandatory reporting of ransomware payments could be more effective than banning cyber-insurance coverage for ransomware. They also highlight the importance of adopting secure by design and default technologies, such as those developed by Google Cloud, to reduce the risk of ransomware attacks.

Building a Kubernetes Client for Google Kubernetes Engine (GKE) in Python - This blog post introduces a method for creating a Kubernetes client for GKE in Python.

GCP IAM Security — Enhancing Privileged Account Management with PAM - GCP-PAM, introduced in 2024, enhances privileged account management by implementing the principle of least privilege, providing temporary permissions, maintaining audit trails, and integrating with GCP services. It offers a user-friendly interface for administrators to create entitlements, set conditions, and approve permission requests, while providing requesters with a simple process to request temporary permissions.

Implementing Cosign Image Validation in GKE - This article explores how to implement Cosign image signing and validation in Google Kubernetes Engine.

A Simple Guide to Google Cloud Committed Use Discounts - Learn how to leverage CUDs for maximum savings on your compute engine.

How to Set Up HashiCorp Vault in Kubernetes with GCS and GCP KMS: A Complete Guide - HashiCorp Vault is a secure secrets management tool that stores sensitive information like passwords and API keys. This guide shows you how to set up Vault in Kubernetes using Google Cloud Platform (GCP) for secure storage and management of secrets. By combining Vault and GCP, you can enhance the security, scalability, and professionalism of your infrastructure, regardless of your organization's size.

App Development, Serverless, Databases, DevOps

Optimizing Costs with GitHub Actions self-hosted runner: Dynamically Starting and Stopping GCP VMs - Optimizing costs for GCP VM-based GitHub Actions self-hosted runner by dynamically starting and stopping VM machine in GitHub Actions.

GCP Bucket Names as Subdomain Names: A Practical Guide - Google Cloud Storage allows you to name buckets in ways that can align with domain naming conventions, including the use of subdomain-like names. This feature enables developers to integrate storage buckets seamlessly with applications, especially those requiring custom domain routing. By mapping a bucket name to a subdomain, you can access files using simplified URLs, maintain brand consistency, improve SEO, and simplify API integration.

How to programmatically access IAP protected apps on Google Cloud - This article explains how to programmatically access Identity-Aware Proxy (IAP) protected applications on Google Cloud.

Tracing with Langtrace and Gemini - Langtrace is an open-source observability tool that helps you improve your Large Language Model (LLM) apps by collecting and analyzing traces. It has an SDK to collect traces from LLM APIs, Vector Databases, and LLM-based Frameworks. The traces are OpenTelemetry compatible and can be exported to Langtrace or any other observability stack.

Serverless Automation for GCP Project Liens - Serverless Automation for GCP Project Liens is a guide on how to set up an automated way to add a project lien to every project in your organization in order to prevent accidental project deletion.

Consolidating Next.js Logging: From Winston to Google Cloud - This article will walk you through centralizing your Next.js logs in the cloud using Google Cloud Logging and Winston.

Creating and Deploying a Google Cloud Run Service Using Artifact Registry and GitHub Actions

Big Data, Analytics, ML&AI

Dynamic Query Execution for Sharded Tables in BigQuery - Using EXECUTE IMMEDIATE to clean up rows from multiple sharded tables in BigQuery.

BigQuery Vector Search: A Practitioner’s Guide - Technical article for BigQuery users and administrators who are responsible for managing and optimizing vector search indexes and queries.

BigQuery : Tables, Views and Stored Procedures referenced in a Routine - How to find all tables, views, stored procedures etc. which are being used in a BigQuery Routine.

Introducing BigQuery Workflows - BigQuery Workflows, a new feature in Google Cloud's BigQuery platform, simplifies data management by automating tasks like running SQL queries or notebooks in sequence. With its visual interface, built-in scheduling, and centralized monitoring, it offers an easy-to-use solution for managing data pipelines.

Text-to-SQL with Gemini and BigQuery: Using LlamaIndex to Simplify Dynamic Prompt Generation - This article demonstrates how to build a text-to-SQL application using LlamaIndex, Gemini, and Google BigQuery. It addresses common challenges like handling dynamic business context, multiple tables, and dynamic prompts. Real-world applications include business intelligence dashboards, customer support tools, data exploration, and data engineering.

Funnel Performance Analysis with BigQuery and Looker Studio - Are we losing our customers without even knowing about it?

Integrate External Data Sources into Vertex AI Agent Builder - This tutorial will guide you through integrating external data sources, specifically APIs, into Vertex AI Agent Builder.

Upgrade your Vector Search Efficiency and Recall with ScaNN Index! - ScaNN, a state-of-the-art approximate nearest neighbor search algorithm, offers a powerful solution for optimizing vector search performance. ScaNN excels at handling large-scale datasets and complex queries, making it particularly well-suited for retail use cases. AlloyDB, a fully managed relational database service, offers seamless integration with ScaNN through the alloydb_scann extension.

Slides, Videos, Audio

Kubernetes Podcast - #242 KubeCon NA 2024.

Security Podcast - #200 Zero Touch Prod, Security Rings, and Foundational Services: How Google Does Workload Security.

Releases

Sensitive Data Protection - The PHONE_NUMBER infoType functionality that was previously only available by setting InfoType.version to latest or stable is now also used when InfoType.version is set to legacy.

Cloud Spanner - Default backup schedules are now available and automatically enabled for all new instances.

Cloud SQL MySQL - You can now create instances with both private services access and Private Service Connect enabled for them.

Cloud SQL Postgres - You can now create instances with both private services access and Private Service Connect enabled for them.

Cloud Composer - In December 2024, Google will remove the following previously deprecated Airflow operators from the apache-airflow-providers-google package: DataPipelineHook, CreateDataPipelineOperator, RunDataPipelineOperator, AutoMLDatasetLink, AutoMLDatasetListLink, AutoMLModelLink, AutoMLModelTrainLink, AutoMLModelPredictLink.

Compute Engine - Preview: Use the disk performance status metric to monitor the health of your Hyperdisk or Persistent Disk volumes.

Data Fusion - The Cloud SQL MySQL plugins version 1.11.5 is available in Cloud Data Fusion versions 6.10.0 and later.

Cloud Data Loss Prevention - The PHONE_NUMBER infoType functionality that was previously only available by setting InfoType.version to latest or stable is now also used when InfoType.version is set to legacy.

Anti Money Laundering AI - A new major engine version is available for Retail and Commercial lines of business, within the v4 tuning version.

Integration Connectors - The following connectors are now generally available (GA): Monday.com PayPal Slack To view the list of all the GA connectors, see Connectors in GA.

GKE new features - Cloud TPU Trillium (v6e) machine types are now in public preview for Autopilot clusters running version 1.31.2-gke.1384000 or later. Cluster autoscaler and node auto-provisioning support the C4 machine family in GKE version 1.28.15-gke.1159000, 1.29.10-gke.1227000 or later.

Google Kubernetes Engine Stable - (2024-R46) Version updates There are no new releases in the Stable channel.

Cloud Run - You can now set a task timeout up to 168 hours (7 days) for Cloud Run jobs.