Welcome to issue #273 December 20th, 2021

News

Cloud Tasks: Now available in 23 GCP Regions - Launch announcement for Google Cloud Tasks service availability in 23 new GCP Regions.

Policy Troubleshooter for BeyondCorp Enterprise is now GA! - Easily troubleshoot and unblock user access issues with Policy Troubleshooter for BeyondCorp Enterprise, now generally available.

Google Cloud enables the National Cancer Institute's Institute for Systems Biology-Cancer Gateway in the Cloud to support breast cancer research with fast and secure data sharing - Google’s open, cloud-based data and analytics enable the National Cancer Institute’s Institute for Systems Biology-Cancer Gateway in the Cloud (ISB-CGC) to securely and safely share up-to-the-minute research.

2021 Gartner® Magic Quadrant™ for Cloud Database Management Systems recognizes Google as a Leader - Unified capabilities for transactional and analytical use cases highlighted, as well as progress in security, elasticity, advanced analytics, and more.

Forrester names Google Cloud a leader in AI Infrastructure - Google Cloud named a Leader in The Forrester Wave™: AI Infrastructure, Q4 2021 report.

Google expands learning paths to help researchers get on the cloud - Academic researchers live in a world of constant change, as research processes and tools continue to get more powerful, but also more complex. With research tools advancing on a daily basis, academic and nonprofit research teams face an ongoing need for learning and re-training. But researchers tell us that with their heavy workloads they don’t have time to keep up with new technologies: they want to focus on moving their research forward.

Easy as Pie Serverless Hackathon - Learn Google Cloud serverless products.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Top Google Cloud infrastructure blogs of 2021 - Top blog posts of 2021 — Google Cloud infrastructure.

Google Cloud managed compute platforms: Top 10 blog posts of 2021 - Blogs about managed compute platforms were some of the best performing content on the Google Cloud blog in 2021.

Four security trends for ‘22—and what to do about them - Here are four cloud security trends that organizations and practitioners should be planning for in 2022—and what they should do about them.

Use your favorite DevOps and security solutions with GKE Autopilot out of the box - GKE Autopilot’s architecture makes it easy to run popular third-party tools—with minimal configuration and no side-car proxies.

Compliance Engineering - Continuous Compliance GCP case studies - Here are 3 real GCP controls framework technical examples for regulated FSI Google Cloud customers to help maintain security and compliance postures.

Optimize your system design using Architecture Framework Principles - System design part of Architecture Framework has been updated to version 2.0, enabling better security, compliance, reliability, operations, and cost- and performance-optimization.

Small footprint, big impact: running cloud-connected Kubernetes at the edge - This blog post summarizes the new edge profile for Anthos on bare metal. It also introduces the new guide for installing Anthos on bare metal on the edge and managing it using Anthos Config Management.

Google Cloud recommendations for investigating and responding to the Apache “Log4j 2” vulnerability - Google Cloud recommendations for investigating and responding to Apache Log4j 2 vulnerability (CVE-2021-44228).

Investigating the usage of GCP Service Accounts - Three GCP services to help you to investigate Google Cloud Service Account usage and mitigate against unintended consequences during key rotation.

In case you missed it: Google Cloud Security Talks, Zero Trust Edition - Check out Google Cloud Security Talks sessions on-demand for the latest zero trust thought leadership and product updates. Learn more about Google’s BeyondCorp and BeyondProd approaches and how you can protect your users and critical information.

Improving the speed and security of your cloud deployments - Highlights the direct experiences of users as they adapt, adopt, and deploy the security foundations blueprint in their cloud environments.

IAM Conditions: For a limited time only? - this article explains how you can define and enforce conditional, attribute-based access control for Google Cloud resources.

IPAM Autopilot for GCP VPCs - IPAM Autopilot is a tool for IP address management in GCP across multiple VPCs.

Tutorial: Managing Serverless GCP Load Balancers Dynamically with Terraform - This tutorial will go step by step into the different pieces of the Terraform script required to deploy and maintain a dynamic serverless Load Balancer.

How to secure your Apigee X data with VPC Service Control - The article provides the list of steps necessary to secure your Apigee X deployment’s data with GCP VPC Service Controls.

App Development, Serverless, Databases, DevOps

How to investigate high tail latency when using Cloud Spanner - This article summarizes common issues about high tail latency and provides useful tips so that Cloud Spanner users can mitigate those issues.

How to keep sensitive data out of your chatbots - Dialogflow tools and settings can help protect sensitive and identifiable information from being exposed to your chatbots or chat agents, preserving privacy and improving data security.

Serverless Telegram bot with Kotlin, Firebase and Google Cloud Functions - Step by step creation of serverless Telegram bot using Goolge Cloud Functions together with Firebase Realtime Database.

Big Data, Analytics, ML&AI

How to migrate an on-premises data warehouse to BigQuery on Google Cloud - Checkout how Independence Health Group is addressing their enterprise data warehouse (EDW) migration, and learn how to plan your own migration.

Better data for better AI: new speech datasets and benchmarks for data - Good data is essential to building good ML apps. Learn about a new and useful public dataset from Google Cloud and benchmarks to drive better datasets.

Get a single one CSV file with BigQuery export - Cloud Workflow helps to solve complex problem in a nicely manner. The composition of several files in a single one is a great example!

BigQuery — Is clustering more efficient than partitioning…? Yes ! - Comparison of query performance for partitioned and clustered table in BigQuery.

Pipeline in the cloud - Scheduling an automatic Dataflow Pipeline that extracts and cleans data in the cloud.

Setup a slim CI for dbt with BigQuery and Docker - A Slim CI is a lightweight version of a CI in which we only want to run and test what is relevant. Let’s see how to use one for dbt.

Cloud Composer upgrade - Performing Cloud Composer upgrade from Airflow 1.x to 2.x.

Find anything blazingly fast with Google's vector search technology - How do YouTube, Google Search, and Google Play instantly find what you want in the vast sea of web content? Try the demo and find out. Hint: it’s vector search.

Build your own custom funnels using BigQuery and Data Studio with Google Analytics - Using BigQuery and Google Data Studio, you can create a custom user journey funnel of your web or mobile app users.

Various

Machine learning, Google Kubernetes Engine, and more: 10 free training offers to take advantage of before 2022 - Free Google Cloud training on machine learning, Google Kubernetes Engine, how to get started with Google Cloud, and more.

Why 2021 was an electrifying year for 24/7 carbon-free energy - Recapping Google’s progress in 2021 toward running on 24/7 carbon-free energy by 2030 — and decarbonizing the electricity system as a whole.

How to become a certified cloud professional - Sharing some tips with you on gaining hands-on experience with Google Cloud.

Preparing for Google Cloud Professional Cloud Architect exam — new curriculum [December 2021] - Thoughts on the changes and advice when retaking the Cloud Architect exam with the new syllabus.

Slides, Videos, Audio

GCP Podcast - #288 2021 Year End Wrap Up.

Releases

Anthos clusters on AWS - Anthos Clusters on AWS aws-1.10.0-gke.5 (previous generation) is now available. This release supports creating instances in the c5a, c5ad, i3en, m5a, m5ad, r5a, r5ad, and t3a families. Kubernetes 1.18 is no longer supported. This release fixes the following security issues: CVE-2021-3733 CVE-2021-3737 CVE-2021-3711. This release fixes an earlier issue with 1.21 clusters that use both OIDC and an HTTP proxy. To install Anthos Service Mesh, follow the steps in Connecting to your cluster before starting your Anthos Service Mesh installation. You no longer need the ServiceUsageViewer role to install Anthos clusters on AWS.

Anthos clusters on bare metal - 1.8. Release 1.8.7 Anthos clusters on bare metal 1.8.7 is now available for download. Fixes: The following container image security vulnerability has been fixed: CVE-2020-21913. Known issues: For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

BI Engine - BigQuery BI Engine SQL interface is now generally availabile.

BigQuery - The row-level security feature now supports administrator access to historical data for tables with row-level access policies.

Cloud Composer - Cloud Composer 2 is now generally available (GA). Private Service Connect support is available in Preview for Cloud Composer 2. Authorized networks support is available in Preview. For the latest updates of the potential impact of the open-source Apache Log4j 2 vulnerability on Google Cloud products, see the Apache Log4j 2 Vulnerability page. Cloud Composer 2.0.0 images are available: composer-2.0.0-airflow-2.1.4 composer-2.0.0-airflow-2.0.2.

BigTable - A Cloud Bigtable instance can now have clusters in up to 8 regions.

Chronicle - Role-based access control (RBAC) Role-based access control (RBAC) enables you to tailor access to Chronicle features based on an employee's role in the organization.

Compute Engine - Preview: Compute-optimized C2D machine types are now available in preview. Accelerator-optimized (A2) machine types with gVNIC are currently experiencing a known issue. Generally available: When rolling out configuration or application updates to a stateful or stateless managed instance group, use the minimum and most disruptive allowed actions to control disruption to your workload. Public preview: You can use the gcloud tool or API to configure stateful IP addresses in a managed instance group. Generally available: You can now share reservations of Compute Engine zonal resources between multiple projects. You can now save copies of all charts from the Observability tab on Compute Engine's VM instance details page to one of your custom dashboards.

Config Connector - Config Connector 1.69.0 is now available. Added support for VPCAccessConnector resource. Added support for ComputePacketMirroring resource. Added support for PrivateCACAPool resource. Added support for IAMWorkloadIdentityPool resource. Added support for IAMWorkloadIdentityPoolProvider resource. Added support for CloudIdentityMembership resource. Rollout support for state-into-spec: absent to ContainerCluster resource (Issue #576). Add billgProject flag in ConfigConnectorContext to specify a quota project to send along with user_project_override header, used for all requests sent from Config Connector. Fixed the issues in config-connector export that the exported YAML now include zero primitives to match the Google Cloud resource live state. Fixed the issues in ContainerCluster with creating autopilot clusters.

Database Migration Service - Database Migration Service now supports creating Cloud SQL for MySQL, Cloud SQL for PostgreSQL, and Cloud SQL for SQL Server instances with customer-managed encryption keys (CMEK) enabled.

Dataproc Metastore - An Apache Log4j 2 vulnerability. that impacted Dataproc clusters has been addressed (see Recreate and update a cluster, which provides guidance to Dataproc users). Dataproc Metastore users do not need to take any action; the fix applied by Dataproc Metastore is sufficient to address the issue.

Dataproc - Dataproc has released the following sub-minor image versions to address an Apache Log4j 2 vulnerability (also see Create a cluster and Recreate and update a cluster for more information). Dataproc has released the following sub-minor image versions to address an Apache Log4j 2 vulnerability (also see Create a cluster and Recreate and update a cluster for more information): 1.4.77-debian10, 1.4.77-ubuntu18, 1.5.52-centos8, 1.5.52-debian10, 1.5.52-ubuntu18, 2.0.26-centos8, 2.0.26-debian10, 2.0.26-ubuntu18. Upgraded log4j version to 2.16.0. Dataproc has added new images, listed in this release note, to address an Apache Log4j 2 vulnerability. New sub-minor versions of Dataproc images: 1.4.76-debian10, 1.4.76-ubuntu18, 1.5.51-centos8, 1.5.51-debian10, 1.5.51-ubuntu18, 2.0.25-centos8, 2.0.25-debian10, 2.0.25-ubuntu18. HIVE-21040: msck does unnecessary file listing at last level of directory tree. Fixed executor log links on Spark History Server Web UI for running and completed applications. Fixed a bug where driver log links on PHS Web UI stop working once the job cluster is deleted. YARN-8990: Fixed a Fairscheduler race condition. SPARK-7768: Make user-defined type (UDT) API public. SPARK-35817: Queries against wide Avro tables can be slow.

Datastream - Datastream now supports customer-managed encryption keys (CMEK).

Dialogflow - Dialogflow CX auto sync for agent collaboration is now GA (generally available). Dialogflow CX change history is now GA (generally available). The Dialogflow CX simulator now allows you to specify flow versions when interacting with the simulator. Dialogflow CX now supports the asia-southeast1 (Jurong West, Singapore) and europe-west3 (Frankfurt, Germany) regions.

Cloud Data Loss Prevention - The ICCID_NUMBER infoType detector is available in all regions.

Document AI - v1beta3 & v1. New Lending Processors (Preview) The following new processors are now available in limited preview: 1040 Schedule D Parser HOA Statement Parser HUD-92900B Parser Retirement/Investment Statement Parser SSA-89 Parser VBA26-0551 Parser. New Versions of Lending Processors We have launched new versions of the following lending processors.

Eventarc - Eventarc for Cloud Run for Anthos is now available in Preview. A dedicated user interface is now available in Preview.

Cloud Filestore - The following Filestore features are now generally available (GA): Enterprise tier Customer-managed encryption key support Snapshots Private services access support. Filestore snapshots now support reverting to a snapshot.

Google Kubernetes Engine - File capability CAP_NET_BIND_SERVICE required by metrics-server to bind privileged port 443 is dropped in clusters that enable PodSecurityPolicy and use the Ubuntu with Docker container runtime in node pools.

Load Balancing - Internal TCP/UDP Load Balancing now allows you to configure a connection tracking policy for the load balancer's backend service.

Migrate for Compute Engine - V.4.11.9 Security updates available. V.4.11.8 Security updates available.

KF - 2.7. Added buildDisableIstioSidecar configuration feature. Added buildPodResources configuration feature. Added controllerCACerts configuration feature. Added buildRetentionCount configuration feature. Added V3 Google stack as build option. Added V3 kf-v2-to-v3-shim stack as build option. Fixed an issue that could prevent SIGTERM from reaching an app. Fixed an issue that caused extra reconciliation loops and logs. Improved CLI performance. Improved subresource API server resilience. Updated Config Connect to v1.66.0. Updated Tekton to v0.29.0. Support for Anthos Service Mesh (ASM) v1.11+, which recommends ingress gateways be outside of the istio-system namespace. Changed build ImagePullPolicy default from always download to prefer cached. Improved Workload Identity reliability.

Cloud Monitoring - The Slack notification channel for alerting is now generally available (GA). The Pub/Sub notification channel for alerting is now generally available (GA).

Cloud Run - The ability to configure Cloud Run services to have CPU allocated for the entire lifetime of container instances is now at general availability (GA).

Security Command Center - Event Threat Detection, a built-in service of Security Command Center, launched the Persistence: New API Method rule to Preview.

Service Directory - Configuring an external TCP/UDP load balancer in Service Directory is available in Preview.

Anthos Service Mesh - 1.12.x. 1.12.0-asm.4 is now available. Fixed a compatibility issue in the previous release between GKE 1.22, the Anthos Service Mesh Certificate Authority (Mesh CA), and Certificate Authority Service (CA Service). Managed Anthos Service Mesh. Managed Anthos Service Mesh now supports Locality Load Balancing and Consistent Hash Load Balancing in the regular and rapid channels. 1.10.x & 1.11.x & 1.12.x. Anthos Service Mesh now supports Locality Load Balancing and Consistent Hash Load Balancing. 1.9.x. This release note was updated on December 16, 2021. 1.7.x & 1.8.x & 1.9.x. Anthos Service Mesh 1.7-1.9 are no longer supported. Managed Anthos Service Mesh. Managed Anthos Service Mesh now supports VPC Service Controls (VPC-SC) as a preview feature in the rapid channel. 1.11.x. 1.11.5-asm.3 is now available.

SAP Solutions - Filestore Enterprise for SAP systems The Enterprise tier of Filestore (Filestore Enterprise) is now generally available (GA) as a file sharing solution for SAP systems on Google Cloud. Monitoring agent for SAP HANA, version 2.2 Version 2.2 of the monitoring agent for SAP HANA is now available. Google Cloud monitoring agent for SAP NetWeaver, version 2.1 Version 2.1 of the monitoring agent for SAP NetWeaver is now available. Backint agent for SAP HANA version 1.0.15 Version 1.0.15 of the Google Cloud Backint agent for SAP HANA is now available.

Cloud SQL MySQL - You can now see the database minor version when viewing information about an instance. You can now set or upgrade your minor version for Cloud SQL for MySQL 8.0. Cloud SQL now supports MySQL 8.0.26. Cloud SQL for MySQL now supports point-in-time recovery using a timestamp. Cloud SQL for MySQL now supports database auditing.

Cloud SQL Postgres - You can now see the database minor version when viewing information about an instance. PostgreSQL version 14 is now generally available.

Cloud SQL SQL Server - You can now see the database minor version when viewing information about an instance. A new feature enables more flexibility for integrating Cloud SQL for SQL Server with Managed Service for Microsoft Active Directory.

Cloud Storage Transfer - Integration with AWS Security Token Service is now generally available (GA) for Storage Transfer Service. Creating and managing data transfers with the gcloud command-line tool is now available in Preview.

Cloud Storage - Public Access Prevention is now in GA. orgpolicy.policy.get permission is now included in certain Cloud Storage IAM roles.

Cloud Tasks - You can now create Cloud Tasks queues in multiple GCP Regions around the world.

Traffic Director - Traffic Director authorization policies are generally available for gRPC deployments with Java and Go clients. Control plane observability is now in Preview.

VMware Engine - Added ability to forward syslog messages of a desired severity (like Error or Warning) to Cloud Logging from NSX-T.

Virtual Private Cloud - When you create a custom mode VPC network, you can select predefined firewall rules which address common use cases for connectivity to instances. Accessing published services using a Private Service Connect endpoint from on-premises hosts that are connected to a VPC network using Cloud VPN is now available in General Availability. Connectivity from on-premises hosts to a Private Service Connect endpoint that is used to access managed services now correctly establishes for all service attachment configurations.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko