Welcome to issue #210 October 5th, 2020

News

Turn any Dataflow pipeline into a reusable template - Flex Templates allow you to create templates from any Dataflow pipeline with additional flexibility to decide who can run jobs, where to run the jobs, and what steps to take based on input and output parameters.

PyTorch / XLA now generally available on Cloud TPUs - PyTorch is now GA on Google Cloud TPUs.

Easily view your old queries with Cloud Logging recent queries - Cloud Logging now presents you with 30 days of queries, making it easier to find the information you need in your logs.

Cloud Firestore now supports not equal queries - Firestore now supports the not equals != and not-in query operators.

Google Cloud Functions On .NET (Public Alpha) - via https://www.reddit.com/r/googlecloud/comments/j4n449/net_support_is_coming_to_google_cloud_functions/.

Google named a leader in the 2020 Gartner Magic Quadrant for Full Life Cycle API Management - For the fifth year in a row, Google Cloud (Apigee) has been named a leader in Gartner’s Full Life Cycle API Management Magic Quadrant.

Announcing Google Cloud Public Sector Summit, a free global digital event: Dec. 8-9 - Our first-ever Google Cloud Public Sector Summit, Dec. 8-9, will bring together customers across the globe to share lessons and discuss the future of digital service.

Firebase Summit 2020: A Two Day Virtual Event - 27-28 October at 9:30AM PST each day.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Google Cloud migration made easy - Migration to cloud is the first step to digital transformation because it offers a quick, simple path to cost savings and enhanced flexibility. In this article we’ll focus on migration of on-premise or public cloud hosted infrastructure into Google Cloud.

A deeper dive into Confidential GKE Nodes—now available in preview - Confidential GKE Nodes, now in preview, encrypt the memory of your nodes and the workloads that run on top of them.

How to alert on SLOs - How to use SLO error budget alerts in Monitoring.

How to track active users in Google Cloud Platform - Using log sinks in BigQuery to track GCP users in an organization.

Load Balancer in Google Cloud Platform - A brief introduction to Load Balancing.

Assume an AWS Role from a Google Cloud without using IAM keys - How to establish a secure authentication from GCP to AWS resources without security keys.

Connect a GKE Cluster with MongoDB Atlas through Cloud NAT - This article shows how to connect pods of a GKE cluster with MongoDB Atlas by using Cloud NAT.

Securing SSH to Google Compute Instance with Identity Aware Proxy - Securing the ssh connection to compute engine instance as well as centralizing the access control by the use of Identity-Aware Proxy.

How DNS Works and How to Create Your Own Nameservers - In this article, you'll learn how DNS resolution works, what are the types of DNS servers, and in the end, you will learn how to create and use custom nameserver addresses with your domain.

App Development, Serverless, Databases, DevOps

Why you should add Google Analytics to your Crashlytics integration - Combining Crashlytics with Google Analytics to simplify keeping track of all the crashes users experience.

5 Best Firebase and Firestore Courses for Frontend Developers - The best online courses to learn Firebase from popular online platforms.

Understand production performance with Cloud Profiler history view - Cloud Profiler’s new history view feature lets you view the performance of a single function or a group of functions over time.

Troubleshooting your apps with Cloud Logging just got a lot easier - Learn how to use the Logs Explorer feature in Cloud Logging to troubleshoot your applications.

Running express on Google Cloud - Deploying and running NestJS web app on App Engine.

Improve App Engine Startup Times through Warmup Requests - Using warm-up requests to reduce startup times for App Engine app.

Deploy a Serverless RESTFUL API using Google Cloud Run and Cloud Firestore - Creating and deploying a basic CRUD web application in Python on Cloud Run.

Big Data, Analytics, ML&AI

BigQuery Explained: Working with Joins, Nested & Repeated Data - Dive into joins with BigQuery, optimizing join patterns, and know about nested and repeated fields for denormalizing data.

Leveraging BigQuery with Google Analytics Data - Analyzing Google Analytics with BigQuery.

Scheduling BigQuery Slots - Reliably adding and removing short term slot capacity on a schedule.

How To Download A BigQuery Result Into a CSV File Using Kotlin - Downloading data from BigQuery in Kotlin.

Become a BigQuery Power User - 5 tips to help you put this beast to work.

BigQuery REST API call using JWT - This post looks at calling BigQuery through REST API and authenticating with JWT tokens.

Operationalizing BigQuery ML through Cloud Build and Looker - Implementing MLOps with BigQuery ML, Cloud Build and Looker.

Serving your Machine Learning model in Google Cloud Platform with Python, Docker & Kubernetes - Learn how to serve a ML model from scratch by following the suggested end-to-end architecture.

Dead lettering in Google Cloud Pub/Sub - Overview and usage of Dead letter topics in Pub/Sub.

Our Journey to BigQuery from Hadoop - This article describes data migration from Hadoop HDFS to Google Bigquery.

PyTorch / XLA is now Generally Available on Google Cloud TPUs - PyTorch / XLA, a package that lets PyTorch connect to Cloud TPUs and use TPU cores as devices, is now generally available.

Various

Helping European businesses grow and digitally transform in the cloud - From subsea cables to new regions to new customers, 2020 was an extraordinary year in EMEA.

SAP on Google Cloud: 2 analyst studies reveal quantifiable business benefits - From uptime and infrastructure to efficiency and productivity—both Forrester and IDC identified major benefits to companies that have made the move to Google Cloud.

Google Professional Cloud Architect Exam Guide - Everything you need to know about GCP Cloud Architect exam.

How to prepare for Google Cloud certified Professional Data Engineer exam - A review of courses to prepare for the Data Engineer certification exam.

Slides, Videos, Audio

GCP Podcast - #238 ASML with Arnaud Hubaux.

Kubernetes Podcast - #123 Leader Election, with Mike Danese.

The Firebase Podcast - 5 things I wish I knew about Firebase as a beginner - David East has worked on Firebase since it was a very small startup. In this podcast he'll tell you 5 things he wishes he knew when he first started.

Releases

BigQuery - BigQuery table-level access control is now generally available.

BigQuery ML - Time series models now let you change DATA_FREQUENCY from the default value (AUTO_FREQUENCY) when forecasting multiple time series using TIME_SERIES_ID_COL.

Billing - Budget alert email notifications: Cloud Billing Budgets functionality has been updated to offer more control over recipients of alert emails. Committed use discounts (CUDs) are now available to purchase for Cloud VMware Engine.

CDN - Added a new tutorial for delivering HTTP and HTTPS content over the same hostname when using Cloud CDN.

Cloud Composer - New versions of Cloud Composer images: composer-1.12.1-airflow-1.10.6, composer-1.12.1-airflow-1.10.9, and composer-1.12.1-airflow-1.10.10. In-cluster image build logs will now appear in Monitoring under separate log names build-log-*, instead of in the Composer Agent logs. Fixed an issue that caused bq load commands for files larger than 100MB to fail with the a RedirectMissingLocation error.

Compute Engine - N2D machine types are now available in all three zones of us-east1-b,c,d in Moncks Corner, South Carolina. N2D machine types are available in The Dalles, Oregon, the us-west1-b zone.

Config Connector - Added Cloud IAM support for ComputeImage. Fixed an issue where an IAMPolicy cannot be deleted when the externally referenced resource does not exist. Fixed an infinite diff condition on spec.minMasterVersion.

Cloud Dataflow - Dataflow now supports Flex Templates in GA. You can now use a network tags parameter to add network tags to all worker VMs that execute a particular Dataflow job.

Data Fusion - This release is in parallel with the CDAP 6.2.2 release. Cloud Data Fusion now supports autoscaling Dataproc clusters. Cloud Data Fusion now displays the number of pending preview runs, if any, before the current run. Improved performance for skewed joins by including Distribution in the Joiner plugin settings. Wrangler now supports BigQuery views and materialized views.

Dataproc - Launched Dataproc integration with Compute Engine sole-tenant nodes, which allows users to create a cluster in a sole-tenant node group. Creating clusters and instantiating workflow requests that succeed even when the requester did not have ActAs permission on the service account now generate a warning field in the audit log request. New sub-minor versions of Dataproc images: 1.3.70-debian10, 1.3.70-ubuntu18, 1.4.41-debian10, 1.4.41-ubuntu18, 1.5.16-debian10, 1.5.16-ubuntu18, 2.0.0-RC12-debian10, and 2.0.0-RC12-ubuntu18. All supported images Upgraded Conscrypt to the 2.5.1 version. Image 1.5 Upgraded Delta Lake to the 0.6.1 version. Image 1.5 and Image 2.0 Preview Upgraded Cloud Storage connector to the 2.1.5 version. Image 2.0 preview The Anaconda optional component is no longer available or required when using the Jupyter optional component (Miniconda is installed and activated by default). YARN-9607: Auto-configuring rollover-size of IFile format for non-appendable filesystems.

Cloud Firestore - Firestore now supports the not equals != and not-in query operators.

Istio on GKE - 1.4.x. There is a known issue with the upgrade from GKE 1.16 to 1.17 versions lower than 1.17.9-gke.6300 (R30 or earlier).

Load Balancing - Added a new tutorial for delivering HTTP and HTTPS content over the same hostname when using Cloud CDN.

Cloud Memorystore - Added Cloud Console UI support for creating Redis instances with Shared VPC using private services access.

Cloud Profiler - Cloud Profiler history view is available in beta.

Cloud Run - Cloud Run now supports server-side HTTP and gRPC streaming.

Cloud Spanner - The NUMERIC data type is now generally available.

Cloud Storage - Tokyo+Osaka dual-region (asia1) launched.

Cloud Talent Solution - Cloud Talent Solution's consumption pricing and quota limits will change effective January 4, 2021.

Traffic Director - Automated Envoy deployment for Google Compute Engine VMs is now in General Availability.

Service Mesh - 1.6.x & 1.4.x & 1.5.x. 1.6.11-asm.1, 1.5.10-asm.2, and 1.4.10-asm.18 Fixes the security issue, ISTIO-SECURITY-2020-010, with the same fixes as Istio 1.6.11.

Anthos GKE on AWS - Anthos GKE on-AWS 1.5.0-gke.6 is now available and clusters run on 1.16.15-gke.700 and v1.17.9-gke.2800. Workload identity (preview) lets you bind Kubernetes service accounts to AWS IAM accounts with specific permissions. You can now route traffic from the GKE on AWS management service and Connect through an HTTP/HTTPS proxy. Improved installation experience This version enables installation and upgrade by using any Google Cloud–authenticated service account. When creating multiple multiple management clusters, users may have seen name collisions with S3 bucket.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko