Welcome to issue #362 September 4th, 2023

News

All 161 things we announced at Google Cloud Next ‘23 – a recap - Recap of product, customer and partner announcements from Google Cloud Next ‘23 conference.

Day 2 at Next ‘23: Developer love, infrastructure spotlights, and more - Here are a few of the highlights from day two of Google Cloud Next ‘23.

What’s new in Google Cloud’s workload-optimized infrastructure - Additions to Google Cloud’s workload-optimized infrastructure including the C3A, C3D, and new SAP workload support, let your workloads thrive.

Welcome to Google Cloud Next ’23 - At Google Cloud Next ’23, we announced how we’re helping every business, government, and user benefit from generative AI and leading cloud technologies.

Shifting down: a new way to cloud for developers - Learn about a new way to cloud for developers that favors “shifting down” over “shifting left,” for a cloud experience that is easy, fast, and secure.

Titanium: A robust foundation for workload-optimized cloud computing - Titanium, a system of purpose-built, custom silicon and multiple tiers of offloads, delivers workloads performance, reliability, and security.

Vertex AI extends enterprise-ready generative AI development with new models, tooling - Google Cloud’s Vertex AI brings tools and 100 AI foundation models — including Clade 2 and Llama 2 — to its one-stop shop for building generative AI apps.

Build data-driven applications with Cloud SQL and validated partner solutions - With the public launch of Google Cloud Ready - Cloud SQL, customers can discover trusted partner products that work well with Cloud SQL.

Supercharging Vertex AI with Colab Enterprise and MLOps for generative AI - Vertex AI is a one-stop shop for machine learning development with features like the newly-announced Colab Enterprise.

What’s new and what’s next with Google Cloud databases - Announcing a series of new features and capabilities to help organizations innovate faster with Google Cloud databases.

Reimagine data analytics for the era of AI

Run data and AI anywhere with new Google Distributed Cloud services - Enhancements to our Google Distributed Cloud (GDC) portfolio help you run your workloads — including generative AI — anywhere.

Vertex AI Search and Conversation is now generally available - Vertex AI Search and Conversation is now generally available. Build and deploy search engines and chatbots quickly and responsibly.

Expanding our AI-optimized infrastructure portfolio: Introducing Cloud TPU v5e and announcing A3 GA - The new Cloud TPU v5e is the most cost-efficient, versatile, and scalable Cloud TPU to date, and the A3 Supercomputer is now generally available to power your large-scale AI models.

Introducing Mandiant Hunt for Chronicle to help you uncover hidden threats in real-time - Announced today at Next ‘23, Mandiant Hunt integrates frontline intelligence and expertise into Chronicle Security Operations to search for undetected attacks.

Expanding Duet AI, an AI-powered collaborator, across Google Cloud - Google Cloud expanded Duet AI preview to include AI assistance capabilities across more products and services to serve cloud users of all types.

Announcing AlloyDB AI for building generative AI applications with PostgreSQL - Announcing a series of new features and capabilities to help developers build enterprise generative AI apps with Google Cloud PostgreSQL databases.

New AI capabilities that can help address your security challenges - At Next’23, we are supercharging security with Duet AI, as well as bringing innovation and enhancements across our security operations and cloud platforms.

Introducing the next evolution of container platforms - Our container platform continues to evolve with GKE Enterprise, support for AI-optimized hardware, and the addition of Duet AI for GKE in preview.

Introducing Cross-Cloud Network: a new way to network - New innovations from Google Cloud Networking for Cloud Next ‘23.

Announcing the Pollen API: Providing actionable info about airborne pollen levels worldwide - Today we’re announcing our new Pollen API so businesses can share this relevant information with their users to help increase brand loyalty and engagement.

Powering the future with our new Solar API - Today we launched our Solar API, within a new suite of environment APIs from Google Maps Platform. The Solar API surfaces important building, rooftop, shading, and panel configuration data covering over 320 million buildings in 40 countries, with the goal of helping accelerate the transition to clean, renewable, solar energy.

Introducing the Air Quality API: Promoting resilience to a changing climate - Today we announced the general availability of our Air Quality API, as part of a new suite of environment APIs available from Google Maps Platform. The Air Quality API gives you access to robust and reliable air quality data, lets you visualize pollution via heatmaps, and lets you share in-depth pollutant details and recommendations.

Going beyond the map: Introducing Environment APIs - Today at Google Cloud Next, we introduced a new suite of Environment APIs: Solar, Air Quality, and Pollen. These APIs help developers better understand an ever-changing world so they can create new sustainability tools, share actionable insights, and encourage people to adapt to new environmental realities.

Built with BigQuery: The new era of CDPs Built on BigQuery - Enterprises are leveraging cloud investments to supercharge marketing efforts by building a Composable CDP with BigQuery and Hightouch.

Introducing Duet AI in Apigee API Management and Application Integration - With Duet AI in Apigee API Management and Application Integration, developers can easily build APIs, integration flows, and extensions using LLMs.

What’s new with Bigtable at Next ‘23? - Learn about the latest Bigtable capabilities. From HTAP, to new cost savings announcements, plus ways to integrate with Google Cloud and open-source.

Cloud TPU v5e accelerates large-scale AI inference - Designed to be efficient, scalable, and versatile, the new Cloud TPU v5e delivers high-throughput and low-latency inference performance.

Migration on your mind? Here’s the biggest migration news from Next ‘23 - To help you on your migration journey, we made several announcements at Google Cloud Next to help you migrate to the cloud quickly and easily.

How supply chain organizations can use data and AI to drive efficiency and optimization - How supply chain organizations use Google’s Data and AI Cloud to drive efficiency and optimization.

Industry Value Networks for accelerated transformation, innovation and outcomes - Industry Value Network (IVN) offerings help customers accelerate time to value and reduce risk with partner solutions pre-integrated on Google Cloud.

Accelerate your cloud transformation with Delivery Navigator - Delivery Navigator brings together a library of transformation methods with project-management tool integration and telemetry.

Harnessing the power of PaLM in BigQuery - By integrating Vertex AI foundation models in BigQuery, you can analyze unstructured data from right inside BigQuery.

Cloud Spanner Data Boost: Analyze operational data isolated from transactional workloads - Cloud Spanner Data Boost lets you run analytical queries or batch processing jobs without affecting the underlying transactional workload.

Simplifying best practices at scale with auto-generated keys in Cloud Spanner - Cloud Spanner auto-generated UUID and sequence keys provide best practices using the SQL developers already know.

The future of Looker is powered by AI - Duet AI in Looker includes direct integrations to conversational AI and large foundation models that will change the way you work with data.

Announcing BigQuery Studio — a collaborative analytics workspace to accelerate data-to-AI workflows - BigQuery Studio lets analytics practitioners use SQL, Python, Spark or natural language directly within BigQuery, to streamline analytics workflows.

BigQuery data clean rooms now available in preview - BigQuery data clean rooms offer security and privacy controls to help teams conduct meaningful analyses while protecting the underlying data.

Announcing Memorystore for Redis Cluster: Up to 60x more throughput, with microseconds latencies - Memorystore for Redis Cluster is available in preview. Clusters support TB of data, 10s of millions of operations per second at microsecond latencies.

Build new gen AI capabilities with AlloyDB Omni, now in preview - The public preview of AlloyDB Omni includes AlloyDB AI, adds compatibility with PostgreSQL 15, and several management enhancements.

Google Cloud deepens investment to activate partner ecosystem around generative AI - Google Cloud partners can take advantage of new initiatives to accelerate their gen AI go to market motions.

Accelerating sustainability with cloud AI - At Google Cloud Next, we expanded our suite of AI-powered solutions to help customers sustainably measure, optimize, and transform their operations.

---

Simplify Cloud Costs, Amplify Engineering Excellence IImagine having a direct line to over 150 senior cloud architects for any cloud-related question or issue you encounter. With thousands of cloud questions and issues resolved, DoiT is your gateway to world-class cloud expertise. All from a Google Cloud Partner. Learn more. Learn more

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Cloud CISO Perspectives: Late August 2023 - At Next ‘23, guest columnist Sunil Potti explains Google Cloud’s vision for how AI can help achieve stronger security outcomes.

Cloud TPU support in GKE under the hood - Support for both Cloud TPU v4 and Cloud TPU v5e on GKE is now GA, letting you do large-scale AI workload orchestration on optimized infrastructure.

Secure access to GCP services in GitLab Pipelines with Workload Identity Federation - This blog post explores how to set up Workload Identity Federation in GCP and securely access the GCP services using short-lived tokens in Gitlab pipelines.

Authenticate to Google Cloud from Terraform Cloud using workload identity - Secure your connection between Google Cloud and Terraform Cloud avoiding the exposure of your service account credentials.

Github OIDC Integration with GCP - Workload Identity Federation - Setting up Workload Identity Federation in Github.

App Development, Serverless, Databases, DevOps

What’s new in Oracle to PostgreSQL migrations with Database Migration Service - Now GA, Oracle-to-PostgreSQL migrations in Database Migration Service help migrate data, schema and convert PL/SQL code to PostgreSQL-compatible SQL.

Dialogflow and Service Directory - Using Dialogflow Webhook within VPC.

GCP Alerts on Microsoft Teams - Posting Alert notifications to Microsoft Teams.

Big Data, Analytics, ML&AI

How to scale AI training to up to tens of thousands of Cloud TPU chips with Multislice - Learn how new Cloud TPU Multislice functionality can enable 2x higher scale than alternate accelerators, with 2x higher performance/dollar and near-linear scaling out-of-the-box.

5 Useful Tips To Master Vertex AI Model Registry (With Code Examples) - Useful tips to work with Vertex AI Model registry.

Understanding the pricing for Vertex AI Text Bison Foundation Model - Explanation of the pricing for Text Bison Foundation Model in Vertex AI.

Powering Q&A Chatbots with AlloyDB and GenAI Models - This blog post shows how to use AlloyDB’s pgvector extension to store vector embeddings of a PDF file into AlloyDB for PostgreSQL and also will create a small Q&A chatbot on top of it.

Various

Google Cloud Innovators and Community join forces to boost your success - Together, the Google Cloud Innovators program and the Google Cloud Community are partnering to create a one-stop shop to help you learn cloud.

Extending our open, flexible ecosystem for AI at Next ‘23 - Google Cloud partners are launching products, services, and solutions that are infused with Google Cloud generative AI and run our infrastructure.

Staying Up-to-Date with GCP: The Customizable Release Notes Solution - Stay informed with GCP Release Notes at your schedule and your preferred product with this simple deployment.

Streamlining Data Quality Control with a Centralized Service

Slides, Videos, Audio

Security Podcast - #136 Next 2023 Special: Building AI-powered Security Tools - How Do We Do It?

Releases

Access Approval - Access Approval supports Anthos Identity Service in the GA stage.

Anthos clusters on VMware - Anthos clusters on VMware 1.15.4-gke.37 is now available. The following issues are fixed in 1.15.4-gke.37: Fixed a known issue where incorrect log rotation configuration for fluent-bit caused low disk space on the Seesaw VM. The following vulnerabilities are fixed in 1.15.4-gke.37: High-severity container vulnerabilities: CVE-2022-32190 CVE-2023-29002 CVE-2023-38403 Container-Optimized OS vulnerabilities: CVE-2015-8863 CVE-2016-4074 CVE-2018-14647 CVE-2018-1000802 CVE-2022-28737 CVE-2023-3611 CVE-2023-3776 CVE-2023-3777 CVE-2023-29491 CVE-2023-38408.

Artifact Registry - Artifact Analysis automatic scanning for Python and Node.js (npm) vulnerabilities in container images is now generally available. Artifact Analysis now offers support for Vulnerability Exploitability eXchange (VEX). Artifact Analysis now offers the ability to export a consolidated software bill of materials (SBOM) for scanned containers in Artifact Registry.

Cloud Asset Inventory - The following resource types are now publicly available through the ExportAssets, ListAssets, BatchGetAssetsHistory, QueryAssets, and Feed APIs.

Assured Workloads for Goverment - The Japan Regions compliance program is now in Preview.

Bare Metal Solution - You can now view and respond to maintenance events through Google Cloud console.

Batch - Documentation has been added to explain the VM OS environment for Batch.

BigQuery ML - You can now use EXPORT DATA statements to directly export BigQuery data to Bigtable. Data clean rooms is now in preview. Duet AI in BigQuery, an AI-powered collaborator in Google Cloud, can help you complete, generate, and explain SQL queries. BigQuery Studio is now in preview. BigQuery DataFrames is now in preview. The following Generative AI features are now generally available (GA) in BigQuery ML: Creating a remote model based on the Vertex AI large language model (LLM) text-bison.

BigQuery - You can now use EXPORT DATA statements to directly export BigQuery data to Bigtable. Data clean rooms is now in preview. Duet AI in BigQuery, an AI-powered collaborator in Google Cloud, can help you complete, generate, and explain SQL queries. BigQuery Studio is now in preview. BigQuery DataFrames is now in preview. The following Generative AI features are now generally available (GA) in BigQuery ML: Creating a remote model based on the Vertex AI large language model (LLM) text-bison.

BigTable - You can now use EXPORT DATA statements to directly export BigQuery data to Cloud Bigtable. Cloud Bigtable is available in the europe-west10 (Berlin) region.

Cloud Build - Cloud Build's free tier now offers the e2-standard-2 machine type as the new default machine type. You can now configure Cloud Build to generate Google-signed identity tokens for user-specified service accounts during a build.

Chronicle - You can now use Cloud Monitoring to customize and receive notifications about ingestion health metrics.

Key Access Justifications - v1. Access Approval supports Anthos Identity Service in the GA stage.

Access Transparency - Access Transparency supports GKE Connect and GKE Hub in the GA stage.

Compute Engine - Generally available: NVIDIA L4 GPUs are now available in the following additional regions and zones: APAC Taiwan (asia-east1-a,c) Tokyo (asia-northeast1-a,c) Singapore(asia-southeast1-c) Europe Belgium (europe-west1-c) London (europe-west2-a,b) North America Northern virginia (us-east4-c) Dalles (us-west1-c) For more information about using GPUs on Compute Engine, see GPU platforms. Preview: You can use future reservations to reserve resources at a specific date and time in the near future.

Dataproc Serverless - Announcing the Preview release of Dataproc Serverless for Spark Interactive sessions and the Dataproc Jupyter Plugin.

Datastore - The sum() and avg() aggregation functions are now available for Firestore in Datastore mode.

Dialogflow - Dialogflow CX generative features are now GA and publicly available: Generative AI agents Generators Generative fallback Generative personalization. Dialogflow CX answer feedback is now GA, but access is limited and must be requested. Dialogflow CX Messenger has been redesigned and is now GA.

Eventarc - Eventarc is available in the me-central2 (Dammam, Kingdom of Saudi Arabia) region. Eventarc support for creating triggers for direct events from the following sources is generally available (GA): AlloyDB for PostgreSQL API Gateway Apigee Registry Backup for GKE BeyondCorp Certificate Manager Cloud Data Fusion Cloud Dataplex Cloud Functions Cloud Memorystore for Memcached Database Migration Dataflow Dataproc Metastore Datastream Firebase Alerts Firebase Remote Config Firebase Test Lab GKE Hub Google Cloud Memorystore for Redis Network Connectivity Network Management Transcoder User-managed notebooks (Notebooks) Vision AI VM Migration Workflows.

Cloud Functions - Cloud Functions (2nd gen) is now the default choice in the Google Cloud console user interface.

Google Kubernetes Engine - Starting on September 1, 2023, a new automation policy is in effect where GKE automatically starts a credential rotation within 30 days of your cluster's certificate authority expiry date to ensure that your cluster doesn't have a complete outage. For GKE nodes running version 1.25 and later, when Image streaming is enabled, the mv command and renameat2 system call might fail on symlink files in container images with the error message "No such device or address". GKE now supports the ability to create nodes and workloads with multiple network interfaces. Your clusters can now perform operations, such as node auto-provisioning or version upgrades, on multiple node pools in parallel. You can now create Cloud Tensor Processing Unit (TPU) nodes in GKE to run AI workloads, from training to inference models. You can now sequence the rollout of cluster upgrades across fleets or across scopes. Version 4.80.0 of the Terraform provider for Google Cloud fixes an issue observed when deploying new GKE Autopilot clusters that would destroy and re-create the cluster due to a change in the dns_config state.

Cloud Logging - Summarizing log entries with Duet AI assistance is now available in Preview.

reCAPTCHA Enterprise - reCAPTCHA Enterprise for WAF and Google Cloud Armor integration is now available for mobile applications in Public Preview.

Cloud Spanner - A monthly digest of client library updates from across the Cloud SDK. Spanner Studio includes Duet AI (in Preview), an AI-powered collaborator in Google Cloud that accelerates SQL development by helping you write SQL statements.

Cloud SQL Postgres - The pgvector extension, version 0.4.2 is generally available. You can now set password policies for local database users of Cloud SQL for PostgreSQL instances.

Cloud TPU - You can now create Cloud Tensor Processing Unit (TPU) nodes in Google Kubernetes Engine (GKE) to run AI workloads, from training to inference models.

Vertex AI - Pricing update The pricing for text-bison has been reduced to $0.0005 per 1,000 input and output characters. Experiment management: Google Cloud console now supports visualization of your model's performance changes over steps during training, and shows advanced run comparisons. Colab Enterprise is now available in Preview. Imagen on Vertex AI now offers the following Generally Available (GA) features: Subject model tuning (standard)* Style model tuning* * Restricted access feature. Stream responses from Generative AI models Generative AI model streaming support is now Generally Available (GA). New Generative AI support on Vertex AI models and expanded language support Generative AI support on Vertex AI has been updated to include new language model candidates (latest models), language models that support input and output tokens up to 32k, and more supported languages. Model tuning for the text-bison model is now Generally Available (GA) Tuning the text-bison model with supervised fine-tuning (SFT) is now Generally Available (GA). Model tuning for the chat-bison model is now available in Preview You can now use supervised fine-tuning to tune the chat-bison model. New embedding model now available in Preview Generative AI support on Vertex AI users can now create embeddings using a new model trained on a wide range of non-English languages in (Preview). Reinforcement learning from human feedback (RLHF) tuning for text-bison The Generative AI text generation foundation model (text-bison) now supports RLHF tuning. Vertex AI Codey APIs language support Vertex AI Codey APIs now support additional programming languages. Vertex AI Codey APIs now support supervised fine-tuning (SFT) The code chat (codechat-bison) and code generation (code-bison) Vertex AI Codey APIs models now support supervised fine-tuning (SFT). Metrics-based model evaluation You can evaluate the performance of foundation models and tuned models against an evaluation dataset for classification, summarization, question answering, and general text generation. Vertex AI Vector Search is the new product name for Vertex AI Matching Engine. Vertex AI Model Registry Models and Vertex AI Datasets are now synced to Dataplex's Data Catalog. CountToken API now available in Preview The CountToken API is now available in (Preview). The Vertex AI Pipelines Template Gallery is now generally available (GA). Tabular Workflow for Forecasting is available in Preview.