Welcome to issue #324 December 12th, 2022

News

Announcing Google Cloud support for Impact Level 5 (IL5) workloads - Google Cloud now has an IL5 provisional authorization, which unlocks all unclassified workloads and civilian public sector workloads.

4 new features of Active Assist to help automate idle resource management - 4 new features to help automate your idle resource management through Active Assist’s Unattended Project Recommender.

Pub/Sub Group Kafka Connector now GA: a drop-in solution for data movement - The open source Pub/Sub Group Kafka Connector makes it much simpler and more cost-effective to transmit data between Kafka and Google Cloud.

---

Meet DoiT

We are committed to solving both essential and advanced cloud challenges. That's why we provide intelligent technology that simplifies and automates cloud use, alongside expert consultancy and unlimited technical support - all at no extra cost to our customers.

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Five steps to help make your software supply chain more secure - From our new report on supply chain security vulnerabilities, CISO Phil Venables offers five tips on how Google Cloud can help secure your software.

Trust Update: December 2022 - The breadth of Google Cloud compliance work spans the globe and many business sectors. Here’s an update on what we’ve accomplished so far in 2022.

Implementing IAM access control as code with HashiCorp Terraform - Understanding IAM and using Terraform for more than just infrastructure as code we can implement account access controls.

Let’s Encrypt Wildcard Cert Auto-renewal on GKE - This article demonstrates how to create a wildcard certificate using Let’s encrypt and configure auto-renewal of a wildcard certificate using cert-manager on GKE ingress.

Project migration between GCP organizations - Necessary steps to move projects from one organization to another in GCP.

Traffic Based Horizontal Pod Autoscaler for GKE Clusters - Using the Gateway API Controller for traffic-based autoscaling.

Google Kubernetes Engine Ingress configuration for production environments - Setting up a single GKE Ingress to your Google Cloud GKE cluster simplifies overhead and is a simple configuration.

Google Cloud Tech Nibble: Securing the SSH Button with IAP - Securing the SSH button with Identity Aware Proxy rather than opening ssh on the firewall.

App Development, Serverless, Databases, DevOps

HEDIS on FHIR to improve quality of patient care - Enable providers to deliver personalized, contextual, and timely patient care by making care gaps and treatment recommendations accessible to clinicians in real time at the point of care.

How to reduce microservices complexity with Apigee and Anthos Service Mesh - When moving from a monolithic to microservices architecture, using a combination of API management and service mesh can streamline the process.

The business value of Cloud SQL: how companies speed up deployments, lower costs and boost agility - Migrating your databases to Cloud SQL can lower costs, boost agility, and speed up deployments, with a 3-year ROI of 246%. Get details in this IDC report.

Workflows patterns and best practices - Part 3 - In this series of blog posts, we summarize Workflows and service orchestrations patterns and point to relevant content.

Migrate Your Business from GCP IoT Core 04｜VPC Network Peering and Transfer Data to GCP - This tutorial will demonstrate the VPC Network Peering between EMQX Cloud and GCP to help you with data transmission.

Set up a Google Chat alert with Google Cloud - This article shows how to display an alert from Google Cloud in Google Chat.

Using Slack to automate manual approvals in Google Cloud Build - Workaround solution to automate Google Cloud Build’s manual approvals using slack to avoid infra abruption or wrong production deployments.

Big Data, Analytics, ML&AI

How NTUC created a centralized Data Portal for frictionless access to data across business lines - Using Google Cloud tools, NTUC integrated data across its sprawling enterprise ecosystem in a centralized Data Portal platform.

How to build comprehensive customer financial profiles with Elastic Cloud and Google Cloud - Google Cloud and Elastic Cloud reference architecture for financial transaction search.

How StreamNative facilitates integrated use of Apache Pulsar through Google Cloud - StreamNative, a company founded by the original developers of Apache Pulsar and Apache BookKeeper, is partnering with Google Cloud to build a streaming platform on open source technologies.

Moneyball for the Front Office - Learn how the Boston Red Sox and other sports teams are engaging fans by bringing powerful analytics to the front office.

Best practices of Dataproc Persistent History Server - The challenge with ephemeral clusters and serverless Spark is that you will lose the application logs when the cluster machines are deleted after the job. Persistent History Server (PHS) enables you to monitor Spark applications running on different ephemeral clusters or serverless Spark.

Performance considerations for loading data into BigQuery - Performance considerations for loading data into BigQuery for various file types.

Getting started with Looker Studio Pro - Using Looker Studio Pro gives you dedicated support, and helps with challenges like seeing who uses which reports and managing access to content.

Automatically Drop Removed dbt Models in BigQuery - DBT snippet to remove old tables and views in BigQuery.

Extend BigQuery NLP armory with Stemmers - Implement English, Spanish and Greek stemmers using Javascript UDFs.

BigQuery Efficiency | How I Reduced My Table Size by -35.5%, and Rows by -93.1% - Learn how to use ARRAYs and STRUCTs to dramatically reduce storage and query costs.

Interpreting BigQuery BI Engine Metrics - An overview of BI Engine.

Solution : Not able to connect to a Private Cloud SQL Instance from a Private Data Fusion Instance ? - Why Cloud Data Fusion with Private IP “does not” connect with Cloud SQL instance with Private IP despite using the same VPC network?

Various

Increasing digital equity in partnership with Los Angeles County - Los Angeles County Internal Services Department (ISD) hosted the inaugural “2022 Tech Empowerment Day.” As part of the Delete the Divide initiative on October 6, over 3,200 middle and high-school students and educators across 58 local schools convened at the SoFi Stadium where they were introduced to the latest STEM technologies and innovations.

Google Maps Platform Hackathon Winners Announced - In September, we launched the Google Maps Platform hackathon on Devpost. We were blown away by the creativity of the projects submitted. Over 2,600 participants spanning 100+ countries joined to build something they’re passionate about and created unique experiences using Google Maps Platform.

12 no-cost ways to learn Google Cloud over the holidays - Upskill, boost your resume and stand out to employers with no-cost, in-demand cloud training skills.

Slides, Videos, Audio

GCP Podcast - #329 Active Assist and Resource Lifecycle Management with Sharon Fang and Michael Sudakovitch.

Security Podcast - #100 2022 Accelerate State of DevOps Report and Software Supply Chain Security.

GCP Life Poscast - #29 In this episode we discuss; Partner All-Stars program, Medibank Update, Paul Dearlove, IAM Deny, ASX Drop Blockchain, Bruno Aziza, Chaos Engineering.

Releases

AlloyDB - AlloyDB cross-region replication replicates your primary cluster's data and resources.

Anthos Config Management - 1.14.0. Config Sync now ignores validating and applying any resource configuration that has the annotation config.kubernetes.io/local-config with any value except for "false", instead of ignoring only when the value is "true". The following five metrics are removed because these metrics aren't needed for monitoring system performance or health: rendering_count skip_rendering_count resource_override_count git_sync_depth_override_count no_ssl_verify_count For information on current metrics, see Monitor Config Sync. The first edition of the Config Sync Service Level Indicators (SLIs) is published. The constraint template library includes a new template: K8sBlockAllIngress. The constraint template library includes a new template: K8sBlockCreationWithDefaultServiceAccount. The constraint template library includes a new template: K8sBlockObjectsOfType. The constraint template library includes a new template: K8sEnforceCloudArmorBackendConfig. The constraint template library includes a new template: K8sEnforceConfigManagement. The constraint template library includes a new template: K8sRequireDaemonsets. The constraint template library includes a new template: K8sRequireDefaultDenyEgressPolicy. The constraint template library includes a new template: K8sRequireValidRangesForNetworks. The constraint template library includes a new template: K8sRestrictRbacSubjects. The following enhancements are made to Config Sync metrics: Enhanced the histogram distribution bounds for the parser_duration_seconds and apply_duration_seconds metrics to support longer durations. Added resource tags to all Config Sync metrics to identify the source component. Fixed a known compatibility issue in Config Sync that was announced in Anthos Config Management 1.13.1 affecting Autopilot on GKE 1.23 and later. Various reliability and stability improvements to Config Sync.

Anthos clusters on bare metal - 1.12. Release 1.12.5 Anthos clusters on bare metal 1.12.5 is now available for download. Fixes: The following container image security vulnerabilities have been fixed: CVE-2019-25013 CVE-2021-3326 CVE-2021-3999 CVE-2021-4037 CVE-2021-33574 CVE-2021-35942 CVE-2022-1184 CVE-2022-1586 CVE-2022-1587 CVE-2022-2663 CVE-2022-3061 CVE-2022-3176 CVE-2022-3303 CVE-2022-3586 CVE-2022-3621 CVE-2022-3646 CVE-2022-3649 CVE-2022-20421 CVE-2022-23218 CVE-2022-23219 CVE-2022-32221 CVE-2022-33745 CVE-2022-33746 CVE-2022-33748 CVE-2022-34903 CVE-2022-37434 CVE-2022-39188 CVE-2022-40307 CVE-2022-42309 CVE-2022-42310 CVE-2022-42311 CVE-2022-42312 CVE-2022-42313 CVE-2022-42314 CVE-2022-42315 CVE-2022-42316 CVE-2022-42317 CVE-2022-42318 CVE-2022-42319 CVE-2022-42320 CVE-2022-42321 CVE-2022-42322 CVE-2022-42323 CVE-2022-42324 CVE-2022-42325 CVE-2022-42326 CVE-2022-43680 CVE-2022-43750. Known issues: For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Anthos clusters on VMware - Anthos clusters on VMware 1.11.6-gke.18 is now available. Fixed OOM events associated with monitoring-operator- Pods by increasing memory limit to 1 GB.

Apigee X - On December 8, we released an updated version of Apigee X. GA release of Simplified Onboarding for Apigee X (Pay-as-you-go) in the Google Cloud console.

AppEngine Standard Go - The Go 1.18 and Go 1.19 runtimes for App Engine standard environment are now available in Preview.

AppEngine Standard NodeJS - The Node.js 18 runtime for App Engine standard environment is now available in Preview.

Cloud Asset Inventory - Preview: You can now query asset metadata via the Cloud Asset Inventory API or the Cloud console, without needing to export the data to a BigQuery table first.

BeyondCorp Enterprise - BeyondCorp Enterprise integration with Microsoft Intune is generally available (GA).

BigQuery - The demo query guide helps you query a public dataset from Google Trends and is now in preview.

BigTable - Cloud Bigtable now lets you restore from a backup to a different project. The ability to configure deletion protection for a Cloud Bigtable table is now generally available (GA). You can now retrieve information about a Cloud Bigtable query to help you evaluate the query's performance. A new suite of client-side metrics for the Cloud Bigtable client for Java is generally available (GA) in versions 2.16.0 and later.

Billing - Preview: Get estimated costs in the Google Cloud console You can now estimate the cost of Compute Engine and Cloud Storage workloads in the Google Cloud console. View expiring commitments and automatically renew resource-based commitments with the Committed use discount dashboard In the Committed use discount dashboard, you can now see subscription expiration notifications for commitments that are expiring within the next 30 days.

Channel Services - More options are now available when you create and manage repricing configurations for your customers. Rebilling data exported to BigQuery now includes the columns: CustomerRepricingConfigName, ChannelPartnerRepricingName, and Tags. v1. The Partner Sales Console Dashboard and the Reporting API now use the rebilled values for Google Cloud and Maps customer costs.

Chronicle - The following changes were made to UDM Search. The following supported default parsers have changed.

Cloud Composer - Cloud Composer 1.20.1 and 2.1.1 release started on December 6, 2022. (Cloud Composer 2) Environment snapshots and Scheduled snapshots are now generally available (GA) for Cloud Composer 2 versions 2.1.1 and later. The following versions for Cloud Composer 1.20.1 and 2.1.1 are available: composer-1.20.1-airflow-1.10.15 (default) composer-1.20.1-airflow-2.2.5 composer-1.20.1-airflow-2.3.4 composer-2.1.1-airflow-2.2.5 composer-2.1.1-airflow-2.3.4 (default). (Available without upgrading) Allowed custom secondary IP range for pods is now narrower. (Cloud Composer 2) The Composer Local Development CLI tool is now available to help streamline testing and developing using local Airflow environments with Composer 2.

Compute Engine - Generally available: You can merge your active hardware resource commitments into one larger commitment to track and manage them as a single entity.

Data Fusion - Cloud Data Fusion is available in the following region: me-west1 For more information, see Locations and Pricing. Cloud Data Fusion version 6.8.0 is in Preview. Features in 6.8.0: Replication from Oracle to BigQuery using Datastream is generally available (GA). In Cloud Data Fusion 6.8.0, Reference name is no longer mandatory for the following plugins: BigQuery Source BigQuery Sink Dataplex Source Dataplex Sink Spanner Sink GCS Sink For these plugins, their unique identifiers in lineage are generated based on their configuration properties. Changes in 6.8.0: For Replication jobs with an Oracle (by Datastream) source, ensured data consistency when multiple CDC events are generated with the same timestamp, by ordering events reliably. Fixed in 6.8.0: For custom Dataproc compute profiles, fixed the issue causing the wrong Cloud Storage bucket to be used to stage data. Upgrading the Cloud Data Fusion version for Replication jobs is broken.

Dataproc Metastore - v1beta1. Dataproc Metastore administrator interface is available in preview.

Dataproc - Added the dataproc.googleapis.com/job/state metric to track the status of Dataproc Jobs states (such as, RUNNING or PENDING). Dataproc Serverless for Spark runtime version 2.0 will become the default Dataproc Serverless for Spark runtime version on January 24, 2023 (instead of December 13, 2022, as previously announced).

Deep Learning VM - M101 Release TensorFlow patch version upgrades: From 2.8.3 to 2.8.4.

Cloud Networking Products - Cloud DNS per resource IAM permissions are available in GA.

Error Reporting - Use the Error Reporting page's new resource filter to filter error groups by resource type.

Google Kubernetes Engine - (2022-R27) Version updates GKE cluster versions have been updated.

GKE - (2022-R27) Version updates Version 1.24.5-gke.600 is now the default version.

Google Kubernetes Engine Rapid - (2022-R27) Version updates Version 1.25.3-gke.800 is now the default version in the Rapid channel The following versions are now available in the Rapid channel: 1.22.16-gke.1300 1.23.14-gke.401 1.24.8-gke.401 1.25.4-gke.1600 The following versions are no longer available in the Rapid channel: 1.21.14-gke.7100 1.21.14-gke.9500 1.22.14-gke.300 1.23.12-gke.100 1.24.4-gke.800 1.25.2-gke.1700 Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to version 1.21.14-gke.8500 with this release.

Google Kubernetes Engine Regular - (2022-R27) Version updates Version 1.24.5-gke.600 is now the default version in the Regular channel The following versions are now available in the Regular channel: 1.22.15-gke.2500 1.23.13-gke.900 1.24.7-gke.900 The following versions are no longer available in the Regular channel: 1.22.12-gke.2300 1.23.12-gke.100 Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.21 to version 1.22.13-gke.1000 with this release.

Google Kubernetes Engine Stable - (2022-R27) Version updates Version 1.23.11-gke.300 is now the default version in the Stable channel The following versions are now available in the Stable channel: 1.21.14-gke.8500 1.22.15-gke.2500 1.23.13-gke.900 1.24.7-gke.900 The following versions are no longer available in the Stable channel: 1.21.14-gke.3000 1.22.12-gke.2300 Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.20 to version 1.21.14-gke.4300 with this release.

Load Balancing - Currently, health check probes for hybrid NEGs originate from Google's centralized health checking mechanism.

Cloud Monitoring - For public and private uptime checks, a new create flow is available in Public Preview.

Network Intelligence Center - Performance Dashboard now shows latency metrics between VMs and Internet endpoints: In the Project performance view, Performance Dashboard shows latency between VMs across all Google Cloud regions and Internet endpoints.

reCAPTCHA Enterprise - You can now enable the email verification feature of MFA from the Google Cloud console.

Cloud Run - Healthcheck probes are now at general availability (GA). Cloud Run support for a new second generation execution environment is now at generally availability (GA). Cloud Run support for network file systems such as NFS, NDB, 9P, CIFS/Samba, and Ceph, as well as Cloud Filestore and Cloud Storage FUSE, is now at general availability (GA.).

Security Command Center - The Malicious URL Observed detector of Container Threat Detection, a built-in service of Security Command Center Premium, is now generally available. Sensitive Actions Service, a built-in service of Security Command Center Premium, is now generally available. The kernelRootkit attribute was added to the Finding object of the Security Command Center API.

Cloud Spanner - We identified an issue in how we calculate the Total Database Storage metric in multi-regional Spanner instances. New SQL syntax, RETURNING in the PostgreSQL dialect and THEN RETURN in Google Standard SQL, selects and returns data from rows that were just updated as part of a DML statement.

Cloud Storage Transfer - Storage Transfer Service offers Preview support for event-driven transfers - serverless, real-time replication from AWS S3 to Cloud Storage, and between Cloud Storage buckets.

Vertex AI - M101 Release The M101 release of Vertex AI Workbench includes the following: TensorFlow patch version upgrades: From 2.8.3 to 2.8.4. The Pipeline Templates feature is now generally available (GA).

Cloud Vision API Product Search - Product Search legacy category migration In 90 days, the legacy categories "apparel", "homegoods", and "toys" will be upgraded.

VMware Engine - In order to support new features in the future, Google Cloud VMware Engine will convert the resource names for private clouds to a standardized format that is more consistent with Google Cloud.

VPC Service Controls - Preview stage support for the following integrations: Cloud IDS Document AI Warehouse.

Workflows - A list.prepend function is available to support creating a copy of a list with a new element added to the beginning.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko