Welcome to issue #400 May 27th, 2024

400 - nice round milestone. My wish and hope is that this newsletter is a useful resource of information for you about Google Cloud.

News

Understand the change in Cloud Monitoring service discovery and how to adapt - Cloud Monitoring has changed the way services are defined. Now, all services in the Services Overview dashboard must be explicitly created. To simplify this, a list of candidates based on auto-discovered services is provided when defining a new service in the console UI. Auto-detected services come with predefined SLIs for availability and latency, while custom services require explicit definition of these SLIs.

Create ecommerce experiences with commercetools and Google Cloud Application Integration - Google Cloud’s Application Integration offers a commercetools connector to integrate commercetools data with Google Cloud services like BigQuery and Vertex AI. The connector simplifies integrations, enhances data flow, and accelerates innovation by reducing the technical overhead of building custom integrations. Use cases include transferring real-time data from commercetools to BigQuery, generating on-demand AI-driven insights with Vertex AI, and achieving ecommerce personalization by integrating commercetools with customer data platforms and recommendation engines.

More FedRAMP High authorized services are now available in Assured Workloads - Google Cloud has expanded its FedRAMP High authorized cloud services, adding over 100 new services. These include Vertex AI services, Cloud Build and Run, Cloud Filestore, and security controls like VPC Service Controls and Cloud Armor.

Google Cloud Achieves FedRAMP High Authorization on 100+ Additional Services - AI, security, and analytics services among many others now mission-ready for FedRAMP High workloads.

Achieve higher performance and lower query cost for BigQuery integer or timestamp lookups - BigQuery numeric search indexes enable optimized lookups on INT64 and TIMESTAMP data types, improving performance and reducing query costs for account IDs, transaction IDs, and log timestamps.

Unlock the business impact of gen AI with new Generative AI Ops services - Google Cloud Consulting has launched Generative AI Ops services to help customers move their generative AI workloads from proof-of-concept into production. These services include prompt engineering, design, and optimization; performance and system evaluation; model optimization and continuous tuning; monitoring and observability; and business integration and testing.

Improving connectivity and accelerating economic growth across Africa with new investments - The Umoja subsea cable will directly connect Africa with Australia, improving connectivity and resilience.

Cloud SQL: Rapid prototyping of AI-powered apps with Vertex AI - The Google-ml-intergration extension for Cloud SQL for PostgreSQL allows developers to easily generate vectors from textual data, perform efficient search over a large corpus of vectors, and fetch real-time predictions directly within their SQL queries.

Maximize performance and optimize spend with Compute Engine’s latest VMs, N4 and C4 - Google Cloud has announced two new general-purpose VM series, C4 and N4, powered by the latest 5th generation Intel Xeon processors and Google's Titanium system. C4 is designed for demanding workloads that benefit from industry-leading performance, while N4 offers price-performance gains and flexible configurations for cost optimization. N4 is generally available now, while C4 is available in preview for Compute Engine and Google Kubernetes Engine (GKE). Together, C4 and N4 provide tailored solutions for all general-purpose workloads, enabling businesses to lower costs without compromising on performance or workload-specific requirements.

Google Cloud Research Innovators launch fourth cohort to drive innovation - The Google Cloud Research Innovators program has selected 23 new participants for its fourth cohort. These researchers will collaborate with Google experts to accelerate their groundbreaking projects using AI and cloud computing technology. They will receive access to Google Cloud technology, cloud credits, networking opportunities, and a community of support. The new Research Innovators will focus on solving real-world problems, such as using neuroscience and machine learning to better understand attention, improving cloud computing performance, and using AI and ML analytics to gain insights from seismic data.

---

Tired of cloud spending surprises?

Don't let cloud overspending sneak up on you! With Revolgy Anomaly Detection you'll be the first to know about any unusual charges on Google Cloud. Learn how to act before the invoice arrives.

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Our clean energy progress in Japan - Google has announced two new solar power purchase agreements in Japan, bringing the company closer to its goal of running on 24/7 carbon-free energy on every grid where it operates by 2030. These agreements will add a combined 60 megawatts of new solar energy capacity to the Japanese grid, supporting Google's data centers in the region and aligning with Japan's clean energy ambitions. The projects are expected to be fully operational within four years and underscore Google's commitment to invest nearly $690 million into sustainable infrastructure in Japan.

Enabling modern manufacturing outcomes with AI, edge, and modern infrastructure - Google Distributed Cloud for manufacturing leverages AI, edge computing, and modern infrastructure to enhance operational efficiency, product quality, and safety standards. It offers benefits such as reduced scrap, enhanced safety practices, accelerated insights, and improved sustainability. By embracing edge computing with Google Distributed Cloud, manufacturers gain an essential tool to address the complex challenges of the dynamic manufacturing industry.

IOC Extinction? China-Nexus Cyber Espionage Actors Use ORB Networks to Raise Cost on Defenders - China-nexus cyber espionage actors are increasingly using ORB networks to conduct espionage operations, making it more difficult for defenders to detect and attribute attacks. ORB networks are made up of compromised devices, such as routers and IoT devices, that are used to relay traffic and obfuscate the source of attacks. This trend is challenging traditional defense strategies that rely on blocking adversary infrastructure, as ORB networks are constantly evolving and difficult to track.

Holes in Your Bitbucket: Why Your CI/CD Pipeline Is Leaking Secrets - Bitbucket Secured Variables can be leaked in your pipeline and expose you to security breaches. To protect your secrets, store them in a dedicated secrets manager, closely review Bitbucket artifact objects, and deploy code scanning throughout the full lifecycle of your pipeline.

Reusing the same address space for multiple GKE clusters in a single project - This blog post presents an architecture that leverages Private Service Connect to hide the GKE Cluster ranges, but connects the networks together using a multi-nic VM that functions as a network appliance router. This keeps the GKE cluster networks hidden but connected, allowing the reuse of the address space for multiple clusters.

Sharing details on a recent incident impacting one of our customers - Google Cloud experienced an incident that impacted one customer's use of Google Cloud VMware Engine (GCVE) in a single cloud region. The incident was caused by an inadvertent misconfiguration during deployment using an internal tool, leading to the automatic deletion of the customer's GCVE Private Cloud after a system-assigned 1-year period. Google Cloud has since taken steps to prevent such incidents from happening again, including deprecating the internal tool and reviewing all GCVE deployments. The customer's data backups stored in Google Cloud Storage were not affected and assisted in the rapid restoration of services.

Protecting Your Google Cloud Environment: Managing Service Account Key Exposure - Google Cloud is implementing a crucial security measure on June 16, 2024, to protect your organization from the risks of exposed service account keys. By default, this policy will proactively disable any service account keys identified as being publicly exposed.

Implementing Privileged Access Managed in GCP with Terraform - Short guide on PAM implementation using Terraform.

Introducing Google Privileged Access Manager - Enabling self-service for just-in-time access to GCP IAM Roles.

App Development, Serverless, Databases, DevOps

How the Nerds at Nerdery do great work with ChromeOS - Why digital consultancy Nerdery chose ChromeOS for its Nerds.

Why this senior care provider relies on ChromeOS - Alden Network, a senior care provider with nearly 50 locations in Illinois and Wisconsin, chose ChromeOS devices for their 1,000 clinicians who need secure access to patient records on the go.

ChromeOS: The secret ingredient for piping hot Domino’s pizza - From order to oven: How ChromeOS streamlines Domino’s delivery process.

How Facturation.net migrated Azure VMs to Google Cloud in one weekend - Facturation.net, a leading Canadian provider of cloud-based invoicing and billing software, migrated its production virtual machines (VMs) from Azure to Google Cloud in just one weekend using Migrate to Virtual Machines (M2VM).

Releasing Artifact Registry assets across Organizations and Projects with serverless - This article presents a serverless approach to automatically copy Artifact Registry or Container Registry images across different projects and organizations. It uses Cloud Pub/Sub, Cloud Run, and EventArc to trigger the image copy process when changes are made to the source Artifact Registry.

Getting Started with Gemini Code Assist - Google Cloud’s always-on Coding Assistant.

Using Google Cloud Firestore with Django's ORM - Django Google Cloud Connectors provides database backends for both Datastore and Firestore.

What I wish I knew when I started with Google Cloud ????☁️ - 10X your productivity and enjoy the awesome features that Google Cloud has to offer; for newbies and seasoned developers.

Big Data, Analytics, ML&AI

UPS leverages BigQuery and Striim for AI-secured package delivery - UPS Capital leverages Google’s Data Cloud and AI technologies to safeguard packages from porch piracy. UPS Capital’s customers have reduced losses by 35% with DeliveryDefense by redirecting shipments to a safer location or adding an adult signature in response to alerts about high-risk deliveries.

More flexibility for your Dataflow jobs with new controls for latency versus cost - Dataflow Streaming Engine users can now choose between lower peak latency or lower streaming costs for their workloads by adjusting the autoscaling utilization hint value. The autoscaling hint value can be set to a higher or lower value using a Dataflow service option. Dataflow’s autoscaling UI provides insights on when it’s worth adjusting the autoscaling behavior and additional dashboards and metrics to monitor the impact of changes.

Supercharge BigQuery with BigFunctions - Framework to build a governed catalog of powerful BigQuery functions.

Migration from Native Tables to External Tables in BigQuery - An Analysis and Decision Report.

Google Data Cloud innovations for continuous real-time intelligence - Google Cloud offers innovations for continuous real-time intelligence, enabling organizations to harness real-time analytics and make informed decisions. With Dataflow, BigQuery, and Apache Kafka for BigQuery, enterprises can leverage streaming infrastructure for visibility, predictions, and activation. Customers like Spotify, Puma, Compass, and Tyson Foods have achieved significant business impact using Google Cloud's data, AI, and real-time solutions.

How partners can augment solution development with gen AI - Google Cloud partners can use generative AI (gen AI) to enhance their DevOps practices and accelerate solution development. The gen AI solution integrates with existing workflows and provides tools for code generation, review, and deployment. Partners can use this solution to streamline software delivery operations, improve code quality, and reduce errors. The reference architecture includes components such as Vertex AI, Cloud Storage, BigQuery, and security and observability services.

Unlocking enhanced LLM capabilities with RAG in BigQuery - Now you can build smarter AI applications from right inside your data warehouse.

From data chaos to data clarity: How Tamr's Data Products leverage Google generative AI - Tamr's Data Products leverage Google's generative AI to deliver accurate entity resolution and golden record creation at scale. These turnkey solutions improve data quality using ML-based mastering models, data cleaning, and standardization services. With Google's Gemini model, Tamr can extract structured data from text fields and perform flexible classification tasks without complex ETL pipelines or extensive ML model development. This partnership simplifies data management, accelerates time-to-value, and enhances data-driven insights for businesses.

MLOps end-to-end system on Google Cloud Platform (II): Our solution in detail - A detailed review of ML solution, designed from scratch on GCP using Vertex AI.

Architectural Blueprints for RAG Automation: Advanced Document Understanding using Vertex AI Search - This post deep dives into using Vertex AI Search to streamline the creation and evaluation of retrieval-augmented generation (RAG) pipelines for advanced document question answering.

Unlocking Multimodal AI with Google Gemini, Embeddings, Vertex Search, and RAG: A Practical Guide with BigQuery - Google's latest AI innovations, including Gemini, embeddings, Vertex Search, and Retrieval Augmented Generation (RAG), are revolutionizing how we interact with and extract insights from data. By leveraging these concepts with BigQuery, users can unlock powerful AI capabilities such as image tagging, vector search, and retrieval augmented generation. This enables enhanced image discovery, improved user experience, efficient scalability, and the generation of creative ideas and insights. The combination of these technologies opens up a world of possibilities for building recommendation systems, question-answering bots, and interactive multimodal experiences.

Various

How to learn networking on Google Cloud — step by step guide - Roadmap to learning networking in Google Cloud. This guide simplifies finding the right resources, no matter your starting point.

How I became Google Cloud Champion Innovator in less than 1 year - Is a Google Cloud Champion Title in Your Reach? One-Year Journey Shows You How.

Slides, Videos, Audio

GCP Life Podcast - #65 - In this episode we discuss; Victoria digital drivers licence, Digital ID Bill, GDG Melbounre Meetup, Trillium TPU, Firebass App Hosting, Firestore-Eventarc, FinOps Hub, Federal Gov. IT Projects, Qld Quantum Computers, Google Threat Intelligence, MediSecure Breach, Federal Gov Ransomware, FirstMac Data Leak, Vertex AI Announcements, Gemini Context Window, Chat GPT-4o, Gemini vs Chat GPT-4o.

Releases

AlloyDB - Query federation between BigQuery and AlloyDB is now available in Preview.

Application Integration - The TIBCO EMS trigger is now available in preview. Application Integration is now available in Milan (europe-west8). Terraform support You can now use Terraform to provision new regions and create authentication profiles.

Google Cloud Armor - Cloud Armor supports Layer 7 filtering in globally scoped edge security policies for Media CDN in Preview. Cloud Armor now supports regional internal Application Load Balancers in public preview.

Artifact Registry - Cleanup policies for Artifact Registry are Generally Available (GA).

Backup and DR Service - Backup and DR Service 11.0.11.323 is now available to update your backup/recovery appliance. Backup and DR Service supports migrating from manual protection to the new dynamic protection using tags. Backup and DR Service now supports auto patch updates. If the management console and backup/recovery appliance connectivity is not established for more than 6 hours, contact customer support to resolve the issue.

Bare Metal Solution - You can now order Performance SSD storage for your Bare Metal Solution.

BigQuery - In BigQuery ML univariate time series models, the FORECAST_LIMIT_LOWER_BOUND and FORECAST_LIMIT_UPPER_BOUND parameters now work with the TIME_SERIES_ID_COL parameter. BigQuery ML now offers the following Generative AI features: Grounding and safety attributes when you use Vertex AI Gemini models with the ML.GENERATE_TEXT function: Use the ground_with_google_search argument to perform grounding. The interactive SQL translator, the translation API, and the batch SQL translator features let you translate the following SQL dialects into GoogleSQL: IBM DB2 SQL Greenplum SQL SQLite These features are in preview. You can now query data in AlloyDB using a federated query. The following Generative AI features are now in preview: Creating remote models based on the Vertex AI gemini-1.5-pro foundation model. You can now use a search index to optimize lookups on the INT64 and TIMESTAMP data types. You can use DLP functions to support encryption and decryption between BigQuery and Sensitive Data Protection, using AES-SIV.

Chronicle - Enhanced the existing curated detections for AWS rule sets in the Cloud Threats category to add 40 new detections.

Chronicle Security Operations - Enhanced the existing curated detections for AWS rule sets in the Cloud Threats category to add 40 new detections.

Chronicle SOAR - Release 6.3.3 is now in General Availability. Release 6.3.4 is currently in Preview. Unable to edit case comments via API (ID #49966652). Unable to create or import advanced reports for certain Looker users (ID #00265303). Error when trying to add a user to Google SecOps SOAR. Event details search option in alert tab stops working (ID #00287518). SOAR filtering not working due to unsupported commas in names. Unable to re-run the playbooks (ID #00282282). Google SecOps SOAR fails to return API keys (ID #50630848).

Data Fusion - Cloud Data Fusion version 6.10.1 is generally available (GA). Creating a private instance with Private Service Connect is GA in Cloud Data Fusion version 6.10.1. Per Namespace Service Accounts are GA in Cloud Data Fusion version 6.10.1. Syncing multiple pipelines from a namespace is GA in Cloud Data Fusion version 6.10.1, For more information, see Sync Cloud Data Fusion pipelines with a remote repository. Changed in Cloud Data Fusion 6.10.1: Source Control Management supports Bitbucket and Gitlab. Fixed in Cloud Data Fusion 6.10.1: Fixed an issue causing runtime arguments of pipeline triggers to not propagate to downstream pipelines (CDAP-20947). Cloud Data Fusion version 6.10.1 has a known issue in the Cloud Storage plugin causing pipelines to intermittently fail if the plugin contains a * regex pattern and uses Dataproc 2.0.

Database Migration Service - Database Migration Service now supports migrations to MySQL minor version 8.0.36.

Dataproc Serverless - New Dataproc Serverless for Spark runtime versions: 1.1.62 1.2.6 2.0.70 2.1.49 2.2.6. Upgraded Spark BigQuery connector to version 0.36.2 in the latest 1.2 and 2.2 Dataproc Serverless for Spark runtime versions.

Dialogflow - Vertex AI Agents: OpenAPI tools now support private network access. Vertex AI Agents: OpenAPI tool authentication now supports Bearer Token. Dialogflow CX: VPC Service Controls now support Cloud Functions and Cloud Run.

Cloud Data Loss Prevention - The TRADE_UNION infoType detector is available in all regions.

Cloud Filestore - The regional service tier is now generally available.

Integration Connectors - The TIBCO EMS connector is available in preview. Integration Connectors is now available in Milan (europe-west8). The following connectors are now generally available (GA): Apache Cassandra Apache CouchDB Kintone To view the list of all the GA connectors, see Connectors in GA.

Networking Interconnect - Partner Interconnect support for dual-stack IPv4 and IPv6 is now generally available.

Google Kubernetes Engine - GKE now provides insights and recommendations to create a backup plan for unprotected clusters that have existed for more than 7 days. The C4 machine family is available in Public Preview for Standard clusters running GKE version 1.29.2-gke.1521000 and later. The GKE Container Security API is now enabled automatically when GKE Enterprise is enabled on a project. (2024-R16) Version updates GKE cluster versions have been updated.

GKE new features - GKE now provides insights and recommendations to create a backup plan for unprotected clusters that have existed for more than 7 days. The C4 machine family is available in Public Preview for Standard clusters running GKE version 1.29.2-gke.1521000 and later.

GKE - (2024-R16) Version updates The following versions are now available: 1.26.15-gke.1243000 1.26.15-gke.1360000 1.27.11-gke.1062004 1.27.14-gke.1011000 1.28.10-gke.1012000 1.29.4-gke.1043001 1.29.4-gke.1670000 1.29.5-gke.1010000 The following node versions are now available: 1.26.15-gke.1360000 1.27.11-gke.1062004 1.27.14-gke.1011000 1.28.10-gke.1012000 1.29.4-gke.1043001 1.29.5-gke.1010000 The following versions are no longer available: 1.26.15-gke.1191000 1.27.11-gke.1062000 1.28.9-gke.1250000 1.29.1-gke.1589018 1.29.3-gke.1282005 1.29.4-gke.1043000 1.29.4-gke.1447000 1.29.4-gke.1447001 1.29.4-gke.1542000.

Google Kubernetes Engine Rapid - (2024-R16) Version updates Version 1.30.0-gke.1167000 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - (2024-R16) Version updates The following versions are now available in the Regular channel: 1.26.15-gke.1243000 1.27.13-gke.1070000 1.28.9-gke.1069000 1.29.4-gke.1043001 The following versions are no longer available in the Regular channel: 1.26.15-gke.1191000 1.28.7-gke.1026000.

Google Kubernetes Engine Stable - (2024-R16) Version updates Version 1.27.11-gke.1062004 is now available in the Stable channel.

Load Balancing - Global external Application Load Balancers and global external proxy Network Load Balancers can now load balance IPv6 traffic.

Media CDN - Dual-token authentication is Generally Available. You can use the globally scoped edge security policies of Cloud Armor for Layer 7 filtering.

Migration Center - Preview: A new version of Migration Center discovery client, 6.2.0, is available in Preview. The Migration Center discovery client CLI (mcdc CLI) supports the offline assessment of migrating VMs running on VMWare vSphere to VMs running on Google Distributed Cloud (GDC) air-gapped. The following are open known issues with the Migration Center discovery client 6.2.0: The maximum number of scanned servers is limited to a total of 2,000 servers.

Cloud Interconnect - Partner Interconnect support for dual-stack IPv4 and IPv6 is now generally available.

Cloud Router - Cloud Router supports BGP route policies in Public Preview. Cloud Router support for IPv6 BGP sessions is generally available.

Cloud Run - Uptime checks can now be configured and viewed directly within the Cloud Run "metrics" page.

Sensitive Data Protection - The TRADE_UNION infoType detector is available in all regions.

Service Mesh - 1.18.x & 1.19.x & 1.20.x. Anthos Service Mesh and Traffic Director have converged into a single, unified product: Cloud Service Mesh. If you're using the Istio APIs with the Traffic Director control plane implementation, disabling multi-cluster load balancing is not supported.

SAP Solutions - Google Cloud's Agent for SAP version 3.3 Version 3.3 of Google Cloud's Agent for SAP is generally available (GA).

Cloud SQL Postgres - Monitoring active queries in Cloud SQL for PostgreSQL, which is part of the Gemini in Databases Preview, is temporarily unavailable.

VMware Engine - We are introducing changes to deleting a private cloud; specifically, when you delete a private cloud, your billing will stop immediately but the private cloud deletion can take up to 24 hours. All new VMware Engine private clouds now deploy with the following: VMware vSphere version 7.0 Update 3 NSX-T version 3.2.3.1 Existing private clouds will be upgraded in May and June 2024.

Workload Manager - Preview: You can now define organizational best practices for your workloads using custom rules written in the Rego policy language.