Welcome to issue #364 September 18th, 2023

News

Introducing Infrastructure Manager: Provision Google Cloud resources with Terraform - Infrastructure Manager uses Terraform to provision and manage Google Cloud resources using an integrated Infrastructure as Code (IaC) approach.

Introducing dynamic topic destinations in Pub/Sub using Dataflow - Dynamic destinations for Pub/Sub topics in Dataflow allow you to use a single publisher client to dictate which messages go to which topics.

Expanding your Bigtable architecture with change streams - Learn how to use change streams for Bigtable and how to incorporate change data capture into your system design.

BigQuery's user-friendly SQL: Elevating analytics, data quality, and security - BigQuery’s new SQL capabilities deliver improved analytics flexibility, data quality and security.

Document AI Workbench is now powered by generative AI to structure document data faster - With Custom Extractor and Summarizer, Document AI Workbench is now powered by generative AI to structure document data faster.

Generative AI on Google Cloud: New training content, from introductory to advanced - New generative AI training options are available for both a non-technical audience (introductory)and for more advanced technical practitioners.

Helping you deliver high-performance, cost-efficient AI inference at scale with GPUs and TPUs - Based on the results of MLPerf™ v3.1 Inference Closed, Google Cloud GPU and TPU offerings deliver exceptional performance per dollar for AI inference.

VMware scales multi-cloud security with Workforce Identity Federation - VMware was able to scale its multi-cloud security with Workforce Identity Federation. Here’s how.

Photorealistic 3D Tiles now in Preview with updated daily quotas - In May 2023, we announced the experimental release of Photorealistic 3D Tiles to power immersive experiences. Now Photorealistic 3D Tiles is available in Preview through the Map Tiles API. To help developers do more with 3D Tiles, we’ve made updates to the quota and terms, added support for OAuth, and provided additional guidance on our attribution policies. We’ve also released 2D Map Tiles and Street View Tiles in Preview for your non-JavaScript environments.

Boosting Subsea Cables with Multi-Core Fiber Technology - Google and NEC are collaborating to adopt multi-core fiber (MCF) technology for the first time in the submarine cable industry.

---

Simplify Cloud Costs, Amplify Engineering Excellence Imagine having a direct line to over 150 senior cloud architects for any cloud-related question or issue you encounter. With thousands of cloud questions and issues resolved, DoiT is your gateway to world-class cloud expertise. All from a Google Cloud Partner. Learn more. Learn more

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Deploy, secure, and monitor streaming service with Media CDN - Learn how to deploy, secure and monitor a streaming media service with Cloud Media CDN.

Improving launch time of Stable Diffusion on Google Kubernetes Engine (GKE) by 4X - Explore the use of GKE to run a fast scaling stable diffusion application container.

Cloud CISO Perspectives: Early September 2023 - Google Cloud CISO Phil Venables talks about this year’s mWISE, and why it’s important to have a vendor-sponsored but vendor-agnostic conference.

Programmatically switch off the nodes in a GKE cluster with Cloud Scheduler - Resize GKE node pools via Cloud Scheduler to save save the cost when cluster is not utilized.

Google Kubernetes Engine Troubleshooting Made Simple with Interactive Playbooks - Using GKE interactive playbooks for troubleshooting guidance for common issues.

FinOps: A Google Cloud 2023 Perspective - An introduction to the GCP FinOps framework.

Optimizing Costs of Google Kubernetes Engine - How we decreased 75% of our production GKE cluster costs.

Ten ways troubleshooting GKE apps is now easier in Cloud Logging, part 2 - Learn about five new features in Cloud Logging that make it easier to troubleshoot an application on GKE, part 2.

App Development, Serverless, Databases, DevOps

Light the way ahead: Platform Engineering, Golden Paths, and the power of self-service - Build templated and well-integrated code and capabilities for rapid project development.

How Compute Engine future reservations can help you get the capacity you need - With future reservations, you can easily request Compute Engine capacity in advance of a planned or expected scale-up event.

SAP MaxDB High Availability (HA) solution using Google Cloud Native storage - Regional Persistent Disk to provide a highly available and scalable storage solution for SAP MaxDB.

Recap of what’s new with Cloud Spanner (at Google Cloud Next ’23) - Cloud Spanner now offers Data Boost and auto-generated keys, plus integration with Duet AI for AI-powered assistance in Google Cloud.

Exploring GCP’s Multi-Writer Persistent Disks: A Guide to Building a Shared Filesystem - Exploring new feature for SSD disks, usage as a multi-writer persisten disk.

Big Data, Analytics, ML&AI

Fine tune autoscaling for your Dataflow Streaming pipelines - Dataflow’s new in-flight job option update feature lets Streaming Engine users to adjust min/max number of workers at runtime.

How to optimize your existing queries with search indexes - Learn about some added SQL operators and functions to help you optimize and improve your search experience in BigQuery.

Applying Generative AI to product design with BigQuery DataFrames - Generative AI is a powerful tool for accelerating the branding process for new products or compounds.

Reducing BigQuery physical storage cost with new billing model - As customers scale data warehousing in the cloud with BigQuery, cost optimization becomes crucial for scalable data storage in the cloud.

Controlling our Data Platform Costs at BlaBlaCar - Tips and Feedback from Blablacar’s journey towards cutting costs in the data platform.

What is Optical Character Recognition? OCR explained by Google - A history of the optical image recognition (OCR) and a guide to how OCR intersects with large language models (LLMs).

Slides, Videos, Audio

Security Podcast - #138 Terraform for Security Teams: How to Use IaC to Secure the Cloud.

Releases

AlloyDB - Maintenance operations on highly available primary instances now occur with less than one second of downtime for most workloads. AlloyDB for PostgreSQL is now available in the following regions: europe-west12 (Turin) me-central1 (Doha) For more information, see AlloyDB Locations. AlloyDB now lets you use and manage data-encryption keys residing outside of Google Cloud using Cloud External Key Manager.

Anthos clusters on VMware - A standalone tool that you run before upgrading an admin or user cluster is now available.

Apigee X - On September 11, 2023, we released an updated version of Apigee X (1-11-0-apigee-3). Bug ID Description 296296456 Implemented fix to ensure that continueOnError is honored in the SpikeArest policy. Bug ID Description 296506425, 295936113, 295925991, 295688738, 296110120, 281112632 Security fix for apigee-runtime.

AppEngine Standard Go - Go 1.21 is now generally available.

Artifact Registry - Artifact Registry now supports HTTP access to Apt repositories.

Cloud Asset Inventory - The following resource types are now publicly available through the Search APIs (SearchAllResources, SearchAllIamPolicies). The following resource types are now publicly available through the analyze policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning).

Assured Workloads for Goverment - v1. The ability to analyze a project for compliance before migrating it to an Assured Workloads folder is now generally available. The IL5 compliance program now supports the following products.

Bare Metal Solution - You can now view the status of your server activity in the Google Cloud console. Observability for storage volumes is generally available (GA).

BigQuery - You can now create a federated dataset in BigQuery that federates to an existing database in AWS Glue. The array subscript operator now returns a value in an array directly by index. The struct subscript operator has been added.

BigTable - You can now choose not to specify the number of nodes when creating a Cloud Bigtable cluster using the gcloud CLI.

CDN - The Cloud CDN private origin authentication capability for Amazon Simple Storage Service (Amazon S3) and compatible object stores is now Generally Available.

Chronicle - You can now change the type of a Chronicle reference list. You can now create and manage forwarder configurations using the Chronicle user interface and also through the Chronicle Forwarder Management API. Chronicle can now directly ingest the following log types from Google Cloud.

Cloud Composer - (Cloud Composer 2 only) In the next two weeks we plan to introduce a 1 vCPU maximum limit for Airflow Triggerers in Cloud Composer. (Available without an upgrade) Fixed the issue with reaching the "Write requests per day" quota limit when executing Airflow CLI commands in Cloud Composer 2. Cloud Composer 2 is now available in Doha (me-central1).

Compute Engine - Generally available: The Red Hat Knowledgebase provides you with access to articles, solutions, product documentation, and community discussions for Red Hat products. Generally available: You can create C3-standard VMs with Local SSD attached using new machine types, for example c3-standard-44-lssd. You can manage future reservations using the Google Cloud console.

Dataflow - Dataflow now supports the Tau T2A Arm machine series as a worker machine type.

Dataproc Serverless - The dataproc.diagnostics.enabled property is now avaiable to enable running diagnostics on Dataproc Serverless for Spark.

Dataproc - New Dataproc on Compute Engine subminor image versions: 2.0.76-debian10, 2.0.76-rocky8, 2.0.76-ubuntu18 2.1.24-debian11, 2.1.24-rocky8, 2.1.24-ubuntu20, 2.1.24-ubuntu20-arm. Scala has been upgraded to version 2.12.18 and Apache Tez has been upgraded to version 0.10.2 in Dataproc on Compute Engine 2.1 images. Announcing the Private Preview release of the Dataproc on Compute Engine Flink Jobs resource.

Dialogflow - Improvements to Prebuilt Components: Name collection: fixed issue with names containing multiple special characters Credit card expiration date collection: updated SSML in agent responses Date of birth (DOB) collection: fixed issue with entities for specific pronunciations of months and days. Dialogflow CX has added the FILTER system function. Dialogflow CX flow export now exports flows recursively.

Cloud Data Loss Prevention - Since August 12, 2023, there has been an increase in latency issues with inspection jobs and data profile generation.

Error Reporting - Error Reporting now supports Cloud Run Jobs as a monitored resource type.

Cloud Firestore - The Google Cloud console now supports a usage dashboard for each database.

IAM - You can now configure IAM workforce identity federation using the Google Cloud console.

Google Kubernetes Engine - You can now use node auto-provisioning for TPU slices. Compute Engine persistent disk CSI Drivers deployed on clusters running version 1.26 and later now support filesystem size expansion during restoration from a snapshot or a clone when the PVC data source is larger than the original volume.

Cloud Logging - Log buckets in the following regions can now be upgraded to use Log Analytics: us-east5 us-south1 me-central1 europe-west12 For more information, see Supported regions.

Migrate for Compute Engine 4.8 - 5.0. Preview: Migrate to Virtual Machines from an Azure source now open to all users.

Cloud Monitoring - Ops Agent version 2.40.0 introduces support for Compute Engine Arm VMs that are running RHEL 9 or Rocky Linux 9.

Cloud NAT - In some cases, it is possible to create multiple NAT gateway configurations that include the same subnet.

Security Command Center - Security Command Center now supports CIS Google Cloud Computing Foundations Benchmark v2.0.0.

SAP Solutions - New SAP HANA certification: Hyperdisk Extreme usage with N2 machine types For running SAP HANA on Google Cloud, SAP has certified using Hyperdisk Extreme with N2 machine types that have 80 vCPUs or more. New OS certifications for SAP HANA For use with SAP HANA on Google Cloud, SAP has certified the following operating systems: Red Hat Enterprise Linux (RHEL) 8.8 for SAP SUSE Linux Enterprise Server (SLES) 15 SP5 for SAP For more information, see Certified operating systems for SAP HANA.

Cloud Spanner - You can now create definer's rights views in Cloud Spanner. Query Optimizer version 6 is generally available.

Cloud Storage - Manifest files are now available in Storage Insights. The following dual-region pairs are now generally available: Montréal and Toronto, Canada Sydney and Melbourne, Australia.

VPC Service Controls - Preview stage support for the following integration: Serial console.

Virtual Private Cloud - Policy-based routing is available in General Availibility. If you've used Google provider for Terraform versions earlier than 4.76.0 to create Private Service Connect service attachments, do not upgrade to versions 4.76.0 through 4.81.x.