Welcome to issue #321 November 21st, 2022

News

Simplify and automate cost optimization with Autoclass for Cloud Storage - Autoclass simplifies data lifecycle management and reduces TCO for Cloud Storage by auto-transitioning data to the best storage class for your workloads.

Introducing Cloud Armor features to help improve efficacy: advanced rule tuning and auto deploy - With Cloud Armor’s latest update, customers can customize WAF rules with tuning options that help reduce the number of false positive alerts that may be generated for particularly sensitive applications.

Introducing IAM Deny, a simple way to harden your security posture at scale - Our latest new capability for Google Cloud IAM is IAM Deny, which can help create more effective security guardrails.

Introducing new, faster search and investigative experience in Chronicle Security Operations - New features for Chronicle can enable security teams to more rapidly hunt, investigate, and respond to threats.

Cloud Storage gets better system observability with customizable monitoring dashboards - In-context dashboards for Google cloud storage, customizable, create alert, view logs for better storage system insights at project level and bucket level.

Deploy Google Cloud in a few clicks with RAD Lab UI - Today, we’re announcing RAD Lab UI--a user interface on top of RAD Lab to make it even more accessible for non-technical users to deploy targeted workloads to Google Cloud with just a few clicks.

SAP Build Process Automation is better with Google Document AI and Google Workspace - SAP integrates Google Cloud Document AI & Google Workspace with SAP Build Process Automation removing inefficiencies that transform workflows.

What can you build with the new Google Cloud developer subscription? - Hear from developers who have a Google Cloud Skills Boost annual subscription and how they have used their credits, and certification voucher.

Apigee named a leader in the 2022 Gartner® Magic Quadrant™ for Full Life Cycle API Management - Learn more about the Gartner MQ assessment process and why Google Cloud Apigee is rated a leader for the 7th time in a row.

How the year’s final Google Cloud Security Talks will ready you for security and cloud success in 2023 - The series’ final installment of the year will equip you with the ability to trust nothing and detect everything, helping to advance both your cloud and security operations transformation.

Google Public Sector and Battelle NEON expand data access to advance ecological research - Google Cloud AI and ML will help researchers better access and use one of the world’s largest ecological observation facilities and sensor networks.

Google Public Sector announces continuity-of-operations offering for government entities under cyberattack - Google offers Continuity of Operations via Google Workspace for its collaboration and continuity needs, ensuring continued effective and secure work in the event of an attack.

Google Cloud Community Learning & Certification Hub Newsletter - Quarterly Learning Hub Newsletter to keep you up to date on everything happening with Google Cloud learning, training, and certifications.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Using the Open Source Insights dataset to analyze the security and compliance of your dependencies - In this blog, we’ll cover several ways your team can use the Open Source Insights dataset, which scans millions of open-source packages, creates dependency graphs, and annotates it with metadata.

Automated Cleanup of Unused Google Cloud Projects - Use Remora to automate the Unattended Project Recommender for discovery and cleanup of unused projects in your Google Cloud organization.

Composite availability: calculating the overall availability of cloud infrastructure - Understand how to calculate the composite reliability of your cloud infrastructure to help design Cloud architectures with an optimal SLA.

Securing tomorrow today: Why Google now protects its internal communications from quantum threats - Google has enabled post-quantum cryptography for its internal communication protocol. Here’s why.

Kubernetes and Worker Queues - Learn how to use Tekton for orchestrating an OSS license scan workflow on Kubernetes.

A Crash Course in Google Kubernetes Engine (GKE) - An overview of Kubernetes concepts and straightforward demonstration.

Does the internet need sunscreen? No, submarine cables are protected from solar storms - With more solar storms on the horizon, Google scientists evaluated whether subsea cables are susceptible to large-scale geomagnetic disturbances.

App Development, Serverless, Databases, DevOps

Increase the reliability of your Cloud Run service with health checks - Cloud Run Healthchecks provides you with a mechanism to identify if your application containers are ready to serve traffic and remediate the faulty containers.

How to migrate on-premises Active Directory users to Google Cloud Managed Microsoft AD - Google Cloud now can help on-prem Active Directory users migrate to Managed Microsoft AD. Here’s how to get it done.

Gregorian Pi - Singing of a Constant - As a follow-up to Emma's 100 Trillion digits of Pi calculation, we demonstrate another way to interact with Pi.

Making your notifications event-based using GCP Pub/Sub - Making alerts from microservices event-based — how they should be!

How Kiwi.com is accelerating travel with Cloud SQL - Travel company Kiwi uses automation and managed services from Google Cloud and Cloud SQL to avoid work that would take 10 employees to complete.

6 ways to optimize your API management costs with Apigee - APIs are a critical driver in transition to cloud and digitalization. However in the current economy, optimizing costs is of paramount importance. Use Apigee to minimize your API management costs and maximize value.

Addressing your API use cases: Choosing between Apigee, API Gateway, and Cloud Endpoints - Google Cloud has multiple solutions for API use cases: Apigee API Management, API Gateway, and Cloud Endpoints, and each has its sweet spot.

Understanding transactional locking in Cloud Spanner - Explore different types of locks in Cloud Spanner and discuss some common cases of transaction locking in Cloud Spanner.

Why should game companies choose Cloud Spanner to power their games - Why game companies should choose Spanner, Google’s fully-managed relational database, to scale their database service and what challenges and pain points our technologies help address.

Big Data, Analytics, ML&AI

How The FA is moving the goal posts with a data cloud approach in Qatar - Google Cloud is helping The FA prepare for a tournament like no other.

BigQuery helps Soundtrack Your Brand hit the high notes without breaking a sweat - BigQuery performance helps Soundtrack Your Brand transform big data into music to offer businesses personalized music recommendations.

How to prevent OOMs while streaming data to GCS via Apache Beam/Dataflow? - Tips to debug Out Of Memory errors when running Beam pipeline on Cloud Dataflow.

Built with BigQuery: How Connected-Stories leverages Google Data Cloud and AI/ML for creating personalized Ad Experiences - Connected-Stories NEXT is an end-to-end creative management platform built on Google Cloud to develop, serve, and optimize interactive video and display ads that scale across any channel.

Hidden gems of BigQuery — Part 2 — Change tracking - New amazing features like DEFAULT values and Change Tracking are discussed.

Automating self-service tech support with Tensorflow - Speed up access to self-service tech support content with AI that can better parse and surface the right articles for the right queries.

How to create robust data pipeline for BigQuery and Google Cloud? - Kestra is an open-source data orchestration and scheduling platform and has an entire range of plugins for Google Cloud.

GCP — BigQuery — Data Security at rest — Series Menu - 5-part blog discussion on the different security controls for Bigquery.

How to run a large scale ML workflow on Dataflow ML for autonomous driving - Developing autonomous driving technology is a battle with data. In this blog, we will walk through how Dataflow ML can be used in autonomous driving development.

BigQuery’s schema auto-detection does not work perfectly like we want it to, so we build our own. - Python library to automatically generate BigQuery schema based on the input file.

BigQuery, Airbyte, Intercom, and another story of Google Cloud Platform cost optimization - It is always a good practice to check the costs if you are trying a new tool connected to BigQuery. Here is a case of how you can do that.

Using AI to increase asset utilization and production uptime for manufacturers - How to use the cloud and AI to increase asset utilization and production uptime in manufacturing.

Easily integrate machine learning models into applications with Vertex AI integration for Cloud Spanner - Add Vertex AI prediction models to your Spanner application using familiar SQL in minutes to solve near real time problems like campaign management, fraud detection and toxic player detection.

Document AI Warehouse - Document AI Warehouse is a Google product for ingestion and management of documents. This article walks us through getting an environment….

Making Better Coffee using Artificial Intelligence - How to quickly build a Latte Art Detector on Google Cloud using Vertex AI, some Python and Cloud Run (almost no coding).

ML Experiment Tracking with Vertex AI - Experimenting with machine learning models can get messy.

Various

"Ask for help, and give thanks" — Hal Cohen's journey through two kidney transplants, life’s challenges - Googler Hal Cohen has experienced two kidney transplants and a series of health complications. Here’s how resiliency and gratitude has guided him.

Multicloud Mindset: Thinking about open source and security in a multicloud world - We’re excited to introduce our Multicloud Mindset series, featuring engaging, live conversations about multicloud topics that takes place on Twitter Spaces.

Toronto Serverless Meetup - Online at 22nd November 2022 - Google Cloud Next '22 Recap with Guillaume Blaquiere.

Slides, Videos, Audio

GCP Podcast - #328 Database Migration Service with Shachar Guz, Inna Weiner, and Gabe Weiss.

Security Podcast - #97 Special: Coordinated Release of Detection Rules for CobaltStike Abuse.

Releases

Agent Assist - Agent Assist has launched backend modules as a GA feature. The Agent Assist Console is now GA. Agent Assist now supports sentiment analysis of voice data as a private Preview feature. Agent Assist now supports CCAI Transcription as a GA feature.

Anthos Config Management - VPC Service Controls now support Config Controller. Config Controller now uses the following versions of its included products: Anthos Config Management v1.13.1, release notes Config Connector v1.96.0, release notes.

Anthos clusters on bare metal - 1.11. Release 1.11.8 Anthos clusters on bare metal 1.11.8 is now available for download. Fixes: The following container image security vulnerabilities have been fixed: CVE-2019-25013 CVE-2020-16156 CVE-2021-3326 CVE-2021-3999 CVE-2021-4037 CVE-2021-33574 CVE-2021-35942 CVE-2022-1184 CVE-2022-1586 CVE-2022-1587 CVE-2022-2663 CVE-2022-3061 CVE-2022-3116 CVE-2022-3176 CVE-2022-3303 CVE-2022-3586 CVE-2022-3621 CVE-2022-3646 CVE-2022-3649 CVE-2022-20421 CVE-2022-23218 CVE-2022-23219 CVE-2022-33745 CVE-2022-33746 CVE-2022-33748 CVE-2022-37434 CVE-2022-39188 CVE-2022-40307 CVE-2022-42309 CVE-2022-42310 CVE-2022-42311 CVE-2022-42312 CVE-2022-42313 CVE-2022-42314 CVE-2022-42315 CVE-2022-42316 CVE-2022-42317 CVE-2022-42318 CVE-2022-42319 CVE-2022-42320 CVE-2022-42321 CVE-2022-42322 CVE-2022-42323 CVE-2022-42324 CVE-2022-42325 CVE-2022-42326 CVE-2022-43750. Known issues: For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Anthos clusters on VMware - Anthos clusters on VMware 1.13.2-gke.26 is now available. Fixed a validation error where the GKE Hub membership is not found when using a gcloud version that is not bundled with the admin workstation.

Assured Workloads for Goverment - The Impact Level 4 (IL4) compliance regime is now generally available. The Israel Regions and Support compliance regime is now in Preview.

BigQuery - Object tables are now in preview. Metadata caching is now in preview. The slot estimator helps you manage slot capacity based on historical performance metrics.

BigTable - Cloud Bigtable now lets you retrieve metadata about a table, giving you greater observability when troubleshooting.

Cloud Build - You can now configure Cloud Build to continue executing a build even if specified steps fail.

Chronicle - You can collect Splunk CIM logs by using the Chronicle forwarder and Splunk default parser. UDM Search UDM Search is a new Chronicle search feature which enables you to find UDM events within your Chronicle instance. Reference Lists Google has made enhancements to the Chronicle reference lists feature, it now enables you to perform more complex matching beyond exact string matches.

Cloud Composer - All Composer environment GKE clusters are set up with maintenance exclusions for the period between November 18, 2022 and November 30, 2022.

Compute Engine - Preview: You can limit the runtime of a VM to automatically stop or delete it when a time limit is reached. Generally available: You can double the default size limit for a managed instance group (MIG): Zonal MIGs support up to 2,000 VMs and regional MIGs support up to 4,000 VMs. Generally available: Use the new distribution shape ANY SINGLE ZONE in a regional managed instance group (MIG) to automatically select a single zone that has available resources within your quota. Balanced persistent disks and SSD persistent disks now offer baseline IOPS and throughput performance.

Config Connector - Config Connector version 1.97.0 is now available. Added spec.gcRules to BigtableGCPolicy (Issues #624, #542, #482, #345, #300). Added spec.load.jsonExtension to BigQueryJob. Added spec.externalDataConfiguration.avroOptions to BigQueryTable. Added spec.compressionMode to ComputeBackendBucket. Added spec.compressionMode to ComputeBackendService. Added spec.advancedOptionsConfig.jsonCustomConfig to ComputeSecurityPolicy. Added spec.managementConfig.fullManagementConfig to ConfigControllerInstance. Added spec.nodeConfig.guestAccelerator[].gpuSharingConfig and spec.notificationConfig.pubsub.filter to ContainerCluster. Added spec.nodeConfig.guestAccelerator[].gpuSharingConfig to ContainerNodePool. Added spec.config.dataprocMetricConfig, spec.config.gceClusterConfig.confidentialInstanceConfig, spec.config.gceClusterConfig.shieldedInstanceConfig, spec.config.masterConfig.diskConfig.localSsdInterface, spec.config.metastoreConfig.dataprocMetastoreServiceRef, spec.config.secondaryWorkerConfig.diskConfig.localSsdInterface, spec.config.securityConfig, spec.config.workerConfig.diskConfig.localSsdInterface and spec.virtualClusterConfig to DataprocCluster. Added spec.cloudLoggingConfig to DNSManagedZone. Added spec.persistenceConfig to RedisInstance. Added status.version to SecretManagerSecretVersion. Added spec.maintenanceVersion and status.availableMaintenanceVersions to SQLInstance. Added spec.passwordPolicy to SQLUser. Added spec.customPlacementConfig to StorageBucket. Added spec.notificationConfig to StorageTransferJob (Issue #303). Moved SQLUser output-only field sqlServerUserDetails from spec to status. Added support for DLPJobTrigger resource.

Contact Center AI Insights - Topic modeling is now a GA feature.

Dataproc - Added support for Dataproc to attach to a gRPC Dataproc Metastore in any region. Secure Boot, Virtual trusted platform module (vTPM), and Integrity monitoring Shielded VM features are enabled by default for Dataproc on Compute Engine clusters that use 2.1 preview images. Nodemanagers in DECOMMISSIONING, NEW, and SHUTDOWN state are now included in the /cluster/yarn/nodemanagers metric. Dataproc Serverless for Spark now shows the subminor runtime version used in the runtimeConfig.version field,. Fixed a bug that caused a Dataproc cluster with a Dataproc Metastore service to fail the creation process, if the cluster was in the same network but different subnetworks. Dataproc Serverless for Spark now now uses runtime version 1.0.23 and 2.0.3. New sub-minor versions of Dataproc images: 1.5.77-debian10, 1.5.77-rocky8, 1.5.77-ubuntu18, 2.0.51-debian10, 2.0.51-rocky8, 2.0.51-ubuntu18, preview 2.1.0-RC4-debian11, preview 2.1.0-RC4-rocky8, preview 2.1.0-RC4-ubuntu20. Downgraded google-auth-oauthlib Python package to fix gcsfs Python package for 2.0 and 2.1 images. Backported HIVE-17317 in the latest 2.0 and 2.1 images. Dataproc Serverless for Spark runtime version 1.0.23 and 2.0.3 downgrades google-auth-oauthlib Python package to fix gcsfs Python package. Upgraded Apache Commons Text to 1.10.0 for Knox in 1.5 images, and for Spark, Pig, Knox in 2.0 images, addressing CVE-2022-42889. Dataproc Serverless for Spark runtime version 1.0.23 and 2.0.3 adds PyMongo Python library.

Dialogflow - Dialogflow CX agents can now be exported to JSON.

Document AI - v1beta3. The Identity Document Proofing Processor is now available in Public Preview. The Identity Document Proofing Processor is designed to help predict the validity of ID documents with four different signals: is_identity_document detection: predict whether an image contains a recognized identity document.

Cloud Filestore - Filestore Backups for High Scale and Enterprise tier instances is available in Preview. Filestore Multishares for GKE is now generally available.

Cloud Functions - Cloud Functions container runtimes have been patched against CVE-2022-3786 and CVE-2022-3602.

Networking Interconnect - Dedicated Interconnect support is available in the following colocation facilities: Telecom Italia Sparkle Milano Data Center, Milan For more information, see the Locations table.

Google Kubernetes Engine - GKE Autopilot clusters support compact placement policies in version 1.25 and later. GKE Autopilot clusters support signaling to GKE that a particular node is problematic in version 1.24 and later.

Load Balancing - Internal HTTP(S) load balancers and internal TCP proxy load balancers now support global access.

Marketplace - You can download private offers as PDFs.

Cloud Interconnect - Dedicated Interconnect support is available in the following colocation facilities: Telecom Italia Sparkle Milano Data Center, Milan For more information, see the Locations table.

Policy Intelligence - Policy Analyzer now offers organization policy analysis.

Cloud PubSub - The Kafka Connector library for Pub/Sub and Pub/Sub Lite is now generally available. BigQuery subscriptions now support the JSON type for all string fields, including data and attributes.

Pub/Sub Lite - The Kafka Connector library for Pub/Sub and Pub/Sub Lite is now generally available.

Resource Manager - Policy Analyzer now offers organization policy analysis.

Cloud Run - Logs from Cloud Run services can now be tailed or viewed in a command-line friendly format using gcloud beta run services logs tail and gcloud beta run services logs read.

Security Command Center - Event Threat Detection, a built-in service of Security Command Center Premium, has launched the Initial Access: Excessive Permission Denied Actions rule to Preview. The files attribute was added to the Finding object of the Security Command Center API.

Cloud Spanner - Time to live (TTL) is now supported in PostgreSQL-dialect databases. Added support for the JSONB data type in the Cloud Spanner PostgreSQL dialect.

Cloud Storage - The following regions are now generally available for dual-region storage: Mumbai (asia-south1) Delhi (asia-south2) Columbus (us-east5) Dallas (us-south1) Las Vegas (us-west4). Turbo replication is now available for all dual-region combinations.

Cloud Tasks - Support for internal ingress from Cloud Tasks to Cloud Run and Cloud Functions is now at General Availability.

Traffic Director - The backendServices API reference documentation now reflects that the outlierDetection object supports gRPC clients.

Transcoder API - The output color space of transcoded videos matches the input color space.

Cloud Translation - For online document translations, you can increase the page limit for native PDF documents to 300 pages.

VMware Engine - Starting November 17, 2022, newly created private clouds will utilize IP address layout (IP Plan) version 2.0 subnet allocations. Stretched private clouds are now available in the europe-west3 (Frankfurt) region. You can now use the gcloud command-line tool or the API to manage VMWare Engine networks, network policies, and private clouds.

Virtual Private Cloud - Preview: Connectivity to Private Service Connect endpoints used to access a managed service is supported over VLAN attachments for Cloud Interconnect. Preview: Private Service Connect endpoints with consumer HTTP(S) controls now support accessing regional Google APIs and managed services using the following load balancers: Regional internal HTTP(S) load balancer Regional external HTTP(S) load balancer.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko