Welcome to issue #310 September 5th, 2022

News

Introducing Pay-as-you-go pricing for Apigee API Management - Access Apigee API management with no upfront commitment while maintaining flexibility to manage costs.

Introducing Vertical Autoscaling in streaming Dataflow Prime jobs - Introducing Vertical Autoscaling in Dataflow Prime to automatically scale workers memory.

Introducing on-demand backup, schema extension support for Google Cloud’s Managed Microsoft AD - Schema extension and on-demand backup/restore are now available with Managed Microsoft AD.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Data governance building blocks on Google Cloud for financial services - Financial services firms have special governance requirements for their data, and there are several Google Cloud tools that can help.

Cloud Wisdom Weekly: 5 ways to reduce costs with containers - Understand the core features you should expect of container services, including specific advice for GKE and Cloud Run.

Building a Fleet of GKE clusters with ArgoCD - How to build a Fleet of GKE clusters using ArgoCD, the Fleet api, and Fleet features like Anthos Service Mesh and Multi Cluster Ingress.

Skaffold for Local Java App Development - Example of using Skaffold for local development of Java applications.

Cloud CISO Perspectives: August 2022 - Google Cloud CISO Phil Venables shares his thoughts on bug bounties, along with the latest security updates from the Google Cybersecurity Action Team.

GKE with CertManager - Using cert-manager add-on with GKE.

GKE with gRPC and ingress-nginx - This article details how to secure mixed HTTP and gRPC (HTTP/2) web traffic with a single ingress controller on GKE.

Personalized recommendations with customized recommender - Recommendations on Google Cloud are super useful and powerful, but sometimes to generic and not aligned with your use cases, up to now!

App Development, Serverless, Databases, DevOps

Develop for Compute Engine in your IDE with Cloud Code - See the status of Compute Engine VM instances, connect via SSH, and upload application files for debugging and development from your IDE using Cloud Code.

Spanner on a modern columnar storage engine - Google’s planet-scale database, Spanner, was migrated to a modern columnar storage engine with many critical services running on top uninterrupted.

Google Cloud — VM Manager - A brief overview on how to enable and set VM Manager.

Why EMQX Is Your Best Google Cloud IoT Core Alternative - An overview of EMQX, Cloud IoT alternative.

Running a serverless, Geospatial, Python app in Google Cloud - Deploying Geo web app on Cloud Run.

Optimise Cloud logging in Google Cloud with logging query language - Is your cloud logging bill more than what you had estimated? Let’s learn a way to filter them out and reduce your monthly consumption.

Big Data, Analytics, ML&AI

Integrating ML models into production pipelines with Dataflow - A deep dive into Dataflow’s integration with Apache Beam's machine learning prediction and inference transform for infusing models into data pipelines.

Understand end-to-end latency for Oracle to BigQuery replication with Datastream and Dataflow - The goal of this article is to explain how we can control end-to-end latency when replicating data from Oracle to BigQuery using Datastream and Dataflow.

Into Google Cloud Dataflow auto-scaling: Max Number of Workers - What does “max number or workers” mean? And how it can affect your Google cloud bill.

Find nearby cafes on Looker using Google Maps API and BigQuery Remote Functions - Running a little snippet of python from a Looker dashboard.

Data quality in Google Cloud BigQuery and Data Lake using Great Expectations. - Integrating Great Expectations (open-source data quality framework) with BigQuery.

Differences between Numbering Functions in BigQuery using SQL - Learn how to use rank, dense rank, row number, cumulative distribution, percentiles rank, quartiles, percentiles, and more.

BigQuery Slot Squeezes - Useful BigQuery Job metrics.

End-to-End DBT project in Google Cloud Platform (Part 2) - Packaging DBT project and deploying it on Google Cloud Platform.

Continuous training with BigQuery ML and Vertex AI - A simple continuous training system which includes the new Cloud Function v2 and the Vertex AI Pipeline Notification in preview.

Various

Meet Ashish Vengsarkar, tackling some of Google Cloud’s most complex challenges by day and building New York Times crosswords in his free time - A director in Google Cloud's optical networking group, Ashish Vengsarkar and his team handle the creation and running of one of the world's largest computer networking systems.

Sorry, Google, I’m going rogue! - Experience with validating app on GCP.

Slides, Videos, Audio

GCP Podcast - #318 GKE Turns 7 with Tim Hockin.

Security Podcast - #81 Demystify Data Sovereignty and Sovereign Cloud Secrets at Google Cloud.

Releases

Anthos clusters on bare metal - 1.11. Release 1.11.5 Anthos clusters on bare metal 1.11.5 is now available for download. Fixes: Increased the default storage size limit of etcd to 6 GiB. Known issues: For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Anthos clusters on Azure - You can now launch clusters with the following Kubernetes versions: 1.21.14-gke.2900 1.22.12-gke.1100 1.23.9-gke.800. This release fixes the following vulnerabilities: CVE-2022-29901 CVE-2022-28693 CVE-2022-29900 CVE-2022-23825 CVE-2022-34903 CVE-2022-32208 CVE-2022-32206 CVE-2022-2097 CVE-2022-31030 CVE-2021-4209.

Google Cloud Armor - The Google Cloud Armor custom rules language now supports URL, URL Unicode and utf-8 decoding.

BigQuery ML - The Random Forest model is now generally available (GA).

BigQuery - Customer-managed encryption keys are now integrated with CMEK organization policies. The slot recommender creates recommendations for customers using on-demand billing and is now generally available (GA). A weekly digest of client library updates from across the Cloud SDK. Community contributed UDFs are now generally available in the bigquery-utils GitHub repository and the bigquery-public-data.persistent_udfs public dataset. Cloud console updates: In the query editor, when you select a function signature from the autocomplete list, you can remove the parameter names quickly by pressing the Backspace or Delete key.

Billing - Non-USD accounts can now view more cost information with the Commitment Analysis Dashboard Customers with non-USD billing accounts now have additional information (including local currency cost data) available to them within the Committed use discount dashboard and have been granted access to the Commitments Analysis Dashboard. View all on-demand spend applicable for committed use discounts (CUDs) You can now view all CUD eligible usage without any active commitments within the Commitment Analysis Dashboard for further cost optimization opportunities.

Chronicle - The following supported default parsers have changed. The following changes are available in the Unified Data Model: The ip_location field was added to Noun type.

Cloud Composer - The apache-airflow-providers-google package was upgraded to 2022.8.23+composer. Cloud Composer now generates an error message if an environment label matches internal environment labels used by Cloud Composer. (Available without upgrading) It is no longer possible to set environment variables with names http_proxy and https_proxy. (Cloud Composer 1) Fixed a transient issue that caused environment creation operations to fail with the Couldn't bring up ['composer-fluentd-daemon', 'airflow-worker'] in time message. Cloud Composer 1.19.8 and 2.0.25 images are available: composer-1.19.8-airflow-1.10.15 (default) composer-1.19.8-airflow-2.1.4 composer-1.19.8-airflow-2.2.5 composer-2.0.25-airflow-2.1.4 composer-2.0.25-airflow-2.2.5. Cloud Composer versions 1.16.15 and 1.17.0.preview.11 have reached their end of full support period.

Compute Engine - The following changes have been introduced to how your resource usage is calculated to determine applicable sustained use discounts: Usage will be calculated on an hourly basis instead of a per microsecond basis.

Datastream - You can now set the number of maximum concurrent CDC tasks for a stream using the Datastream API.

Cloud Data Loss Prevention - The PERSON_NAME infoType detection model that was previously only accessible by setting InfoType.version to latest has been promoted to be the default detection model for this infoType.

Cloud Filestore - Filestore is now available in Madrid, Spain (europe-southwest1 region). Filestore is now available in Paris, France (europe-west9 region). Filestore is now available in Milan, Italy (europe-west8 region).

Google Kubernetes Engine - (2022-R21) Version updates GKE cluster versions have been updated.

GKE - (2022-R21) Version updates The following control plane and node versions are now available: 1.21.14-gke.4300 1.22.12-gke.2300 1.23.9-gke.2100 1.24.3-gke.2100 The following control plane versions are no longer available: 1.21.12-gke.2200 1.23.7-gke.1400 1.23.8-gke.400 Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.20 to version 1.21.14-gke.700 with this release.

Google Kubernetes Engine Rapid - (2022-R21) Version updates The following versions are now available in the Rapid channel: 1.21.14-gke.4300 1.22.12-gke.2300 1.23.9-gke.2100 1.24.3-gke.2100 Version 1.24.3-gke.200 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - (2022-R21) Version updates The following versions are now available in the Regular channel: 1.21.14-gke.2700 1.22.12-gke.500 1.23.8-gke.1900 The following versions are no longer available in the Regular channel: 1.21.14-gke.700 1.23.7-gke.1400 Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to version 1.21.14-gke.2100 with this release.

Google Kubernetes Engine Stable - (2022-R21) Version updates The following versions are now available in the Stable channel: 1.21.14-gke.2700 1.22.12-gke.500 1.23.8-gke.1900 Version 1.21.14-gke.700 is now the default version in the Stable channel.

Load Balancing - External TCP and SSL proxy load balancers now allow you to specify a forwarding rule with a global anycast IP address and any port from 1-65535.

Cloud Monitoring - You can now collect additional MongoDB metrics from the Ops Agent, starting with version 2.19.0. You can now manage Monitoring-specific roles by using the Cloud Monitoring pages in the Google Cloud console.

Cloud Interconnect - Dedicated Interconnect support is available in the following colocation facilities: Telecom Italia Cebrosa Campus, Turin Telecom Italia Moncalieri Campus, Turin For more information, see the Locations table.

Cloud Run - Terraform samples are now available in many of the Cloud Run tutorials and guides, such as: Static outbound IP address, Pub/Sub with Cloud Run Authenticating service to service Rollbacks & Rollouts Request Timeouts and many other pages.

SAP Solutions - Monitoring agent for SAP HANA version 2.7 Version 2.7 of the monitoring agent for SAP HANA is now available.

Cloud Storage Transfer - Storage Transfer Service now offers preview support for multipart uploads for transfers originating from a file system, if the destination or intermediate bucket uses the Standard storage class.

Cloud Text-to-Speech - Text-to-Speech now offers these new voices: cloud-ta-IN-Wavenet-C cloud-ta-IN-Standard-C cloud-ta-IN-Wavenet-D cloud-ta-IN-Standard-D.

Cloud Translation - Cloud Translation - Advanced (v3) support for a multi-regional US endpoint is now Generally Available (GA).

Workflows - Added support to deploy a workflow using a cross-project service account through the Google Cloud console.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko