Welcome to issue #236 April 5th, 2021

News

Analyze your GKE and GCE logging usage data easier with new dashboards - Download open source JSON dashboards for Cloud Monitoring that help you analyze logging volumes, logs-based metrics and logs exports across multiple projects.

Migrate your MySQL and PostgreSQL databases using Database Migration Service, now GA - Check out how to migrate your on-premises databases to the cloud with Database Migration Service, now generally available for PostgreSQL and MySQL.

Data Lake management just got easier with Dataproc Metastore GA - Today, we are excited to announce the general availability of Dataproc Metastore. A fully managed, serverless technical metadata repository based on the Apache Hive metastore.

Delivering high-performing global APIs with Apigee X and Cloud CDN - Google Cloud’s Apigee X and Cloud CDN help enterprises deliver more performant APIs and expand their digital business ecosystems.

Cloud SQL for SQL Server—now with Active Directory authentication - Check out the latest from Cloud SQL: Windows authentication is now supported, so you can use this Active Directory integration easily for your databases.

Loading Google Maps Platform JavaScript in Modern Web Applications - Two open source libraries that make it easier for you to use the Maps JavaScript API in modern web apps.

This bears repeating: Introducing the Echo subsea cable - The new Echo subsea cable will run from the U.S. to Singapore with a stop-over in Guam, and with plans to land in Indonesia.

New this month: Data champions, a key analyst report & features you’re going to want to check out! - Sudhir Hasbe, senior director of product management, shares the latest on how customers and industries are using Google Cloud smart analytics to grow and innovate.

Data Cloud Summit - May 26, 2021 - Join us to learn how leading companies are powering innovation with our data solutions. Attend sessions, demos, and live Q&As to discover how data can help you make smarter business decisions and solve your organization’s most complex challenges.

Free AI and machine learning training for fraud detection, chatbots, and more - These no-cost training opportunities can help you gain the latest AI and machine learning skills from Google Cloud.

Community driven: official Google Cloud Certification Training - Free courses for GCP Certifications.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

New whitepaper: Scaling certificate management with Certificate Authority Service - This whitepaper explains how organizations can more easily manage devices with Google Cloud’s Certificate Authority Service.

How reCAPTCHA Enterprise protected customers during the holidays - Google Cloud reduced online fraud for customers over the 2020 holiday season thanks to reCAPTCHA Enterprise.

How Google Cloud can help the Public Sector embrace zero trust - Helping governments reduce their risk from cyber attacks with a zero trust approach to security.

Devices and zero trust - In a zero trust environment, every device has to earn trust in order to be granted access. Learn how it works!

A story about GKE, zonal nodes and stateful set scaling - When using zonal node pools in GKE you have to be really careful when using StatefulSets without preparations.

SRE Public Resources for GCP Customers - A list of articles, videos and courses related to SRE.

Collect Kubernetes Logs and Store to Google Cloud Storage (GCS) with Grafana Loki - Setting up monitoring using Grafana for GKE.

Multicluster Istio paired with Vault: How to do this? - How to build a secure, production-ready CA for provisioning certificates and keys for all your Istio workloads in the mesh.

App Development, Serverless, Databases, DevOps

GraphQL: Building a consistent approach for the API consumer - Compare two leading ways to interact with APIs: REST and GraphQL; and how they can be managed with Apigee API management.

Preparing PostgreSQL for migration with Database Migration Service - Everything you need in one place to prepare you PostgreSQL instance and databases for migration using Google Cloud’s Database Migration Service.

Demystifying Cloud Spanner multi-region configurations - Cloud Spanner is a strongly consistent, highly scalable, relational database. It powers billion-user products every month. In order to provide high availability and geographic locality, Cloud Spanner creates multiple copies (aka replicas) of your data and then stores these replicas across different geographical locations. This blog post discusses what this entails.

Take a tour of best practices for Cloud Bigtable performance and cost optimization - Get a look at how you can optimize performance in cost for NoSQL, key-value cloud database service Cloud Bigtable, designed for low latency and high scalability.

Connect A Custom Domain to Firebase Hosting - Google Firebase Hosting is available for free, under their free plan Spark, which provides a good amount of generous limits.

How to develop and test with Google Cloud Datastore running locally? - Using Cloud Datastore emulator as Docker image.

CloudCover | How to successfully migrate data from DynamoDB to Spanner and Bigtable - Migrating data streaming from AWS to GCP.

Big Data, Analytics, ML&AI

Creating an experience management (XM) data warehouse with survey responses - Google Cloud and Trifacta have partnered to create a design pattern to easily shape, display, and use survey data.

Speeding up small queries in BigQuery with BI Engine - A quick and easy way to speed up small queries in BigQuery is to use BI Engine. Now, it is available to any application, not just dashboards.

Unlock geospatial insights with Data Studio and BigQuery GIS - Learn how to create arbitrary-polygon choropleth maps in Data Studio with BigQuery GIS and Google Maps.

Troubleshoot BigQuery performance with these dashboards - A few key dashboards can help monitor and optimize BigQuery performance to keep your apps running reliably.

Spring forward with BigQuery user-friendly SQL - The newest set of user-friendly SQL features in BigQuery are designed to enable you to load and query more data with greater precision, allow users to evolve their data rapidly as your needs change, and lower your query and storage costs.

How to efficiently manage your BigQuery spend - Discussion on choosing correctly BigQuery billing model.

A short look at BigQuery’s brand new Materialized Views - Materialized Views are now available in BigQuery. Let me give you an overview of the feature and introduce you to its do’s and don’ts.

BigQuery: User-Defined Functions to supercharge your SQL - An overview of User Defined Functions in BigQuery.

Real-Time data delivery at scale with BigQuery - Using BigQuery Authorized Views to cut storage and processing costs.

How BigQuery helps scale and automate insights for baseball fans - An in depth look at how BigQuery features and functionality can create insights from data for baseball fans.

How sweet it is: Using Cloud AI to whip up new treats with Mars Maltesers - Mars uses Google Cloud AI to invent a tasty new cake that includes maltesers and marmite!

Serving FastAI models with Google Cloud AI Platform - Deployment and the serving of FastAI models at scale on AI Platform.

Tutorial: VS Code with Google Cloud AI Platform as a backend - In this tutorial, we’ll setup VS Code, your local environment, and an AI Platform Notebook. This gives you a few benefits, including using your local git credentials, local IDE extensions, and a smoother user experience.

A comparative analysis between BigQuery, Redshift, and Snowflake - A complete analysis of the three cloud databases. The post goes into a performance, usability, and functional comparisons.

Part VI: Exploring COVID-19 virus variants to predict if new vaccine might need to be developed in future - Setting up a bioinformatics pipeline to simulate virus mutation on GCP.

Various

Why Google Cloud is the ideal platform for Block.one and other DLT companies - We’re sharing more on why Google Cloud is uniquely positioned to be an excellent partner for Block.one and other distributed ledger technology (DLT) companies.

University of Missouri expands data analytics with Google Cloud - Hear from the University of Missouri on how they expanded their student data analytics with Google Cloud.

How FFN accelerated their migration to a fully managed database in the cloud - See how Freedom Financial Network migrated a terabyte of data in just hours to fully managed Cloud SQL database service as part of creating a microservices architecture.

Auto Trader: Charting the road from Oracle to PostgreSQL - See how auto marketplace Auto Trader moved from Oracle databases to managed PostgreSQL from Google Cloud, along with GKE, to add agility and innovate faster.

Delivery Hero turns to Google Cloud BigQuery to improve data accessibility and sharing - Online food delivery network Delivery Hero wants to improve data access between its different business units, with the aim of advancing its machine learning capabilities.

How Cloud Technology Can Help Support Economic Recovery - As new COVID-19 relief dollars flow into state and local budgets, agency leaders can embrace cloud technology to help deliver critical services to the community.

Slides, Videos, Audio

GCP Podcast - #253 Data Governance with Jessi Ashdown and Uri Gilad.

Kubernetes Podcast - #144 Weaveworks (part 1), with Alexis Richardson.

GoDevNet: Go webapp serverless options in GCP - Valentin Deleplace

Releases

AI Platform - Deep Learning VMs - M66 Release PyTorch 1.8 support in deep learning environments (Deep Learning VM Image and Deep Learning Containers) is available.

AppEngine Standard Java - Updated Java SDK to version 1.9.88.

Cloud Asset Inventory - New resource types are now available.

BigQuery - BigQuery standard SQL now supports the ALTER TABLE DROP COLUMN. The maximum length has been increased from 128 characters to 300 characters for the following BigQuery fields: table column names, column alias names, and user-defined function names. BigQuery standard SQL now supports the following statements for creating, configuring, and deleting datasets: CREATE SCHEMA ALTER SCHEMA DROP SCHEMA These statements are generally available (GA). BigQuery standard SQL now supports the TABLESAMPLE operator, which lets you query random subsets of data from large BigQuery tables. BigQuery standard SQL now supports the following JSON functions: JSON_EXTRACT_STRING_ARRAY JSON_QUERY_ARRAY JSON_VALUE_ARRAY These statements are generally available (GA). INFORMATION_SCHEMA views for table partitions are now available. The INFORMATION_SCHEMA.TABLES view now includes a DDL column that can be used to recreate the table. Support for the BigNumeric type in BigQuery standard SQL is now generally available (GA).

Billing - Effective April 1, 2021, for customers in India: Due to new Reserve Bank of India (RBI) regulations, your bank might begin declining automatic card charges for recurring payments for your Google Cloud usage.

CDN - Cloud CDN now treats HTTP responses with a valid, future date in the Expires header as cacheable, even if those responses do not have a Cache-Control: public directive. Cloud CDN now treats the no-cache Cache-Control directive in a response as per RFC 7234 and allows these responses to be cached, provided that they are validated every time before being reused.

Cloud Composer - New versions of Cloud Composer images: composer-1.15.2-airflow-1.10.14 (default) composer-1.15.2-airflow-1.10.12 composer-1.15.2-airflow-1.10.10. Irrelevant warnings about asynchronous DAG loading parameters no longer show up in the Airflow logs. Corrected the validation of custom Cloud SQL and Airflow web server IP ranges that are specified during the environment creation. Fixed an Airflow web UI bug that caused the DAG Tree View page to crash in rare cases.

Compute Engine - Memory-optimized machines are now available in the following regions and zones: M1 ultramem (Jakarta ) asia-southeast2-a,c M1 ultramem (Osaka) asia-northeast2-a M1 ultramem, M2 ultramem and M2 megamem (Osaka) asia-northeast2-b M2 ultramem and M2 megamem (Osaka) asia-northeast2-c See VM instance pricing for details. Preview: You can now configure your VM to shutdown automatically when you revoke the Cloud KMS key protecting a persistent disk attached to the VM.

Data Fusion - Cloud Data Fusion version 6.4.0 is now available. Features in 6.4.0: GA: You can now ingest data from SAP tables with the SAP Table Batch Source plugin. Changed in version 6.4.0: Behavior change: When you validate a plugin, macros get resolved with preferences. Fixed in 6.4.0 (for more information, see the CDAP release note): Fixed Bigtable batch source plugin. FTP Batch Source (system plugin for data pipelines) FTP Batch Source version 3.0.0 is backward compatible, except that it uses a different artifact.

Dataproc - Dataproc support of Dataproc Metastore services is now available in GA.

Dialogflow - The legacy analytics page has been removed from the Dialogflow ES console.

Error Reporting - Service Errors is now available Generally Available (GA).

IAM - Policy Simulator is now generally available.

KMS - Cloud EKM now supports Dataflow Appliance and Pub/Sub.

Google Kubernetes Engine - (2021-R10) Version updates GKE cluster versions have been updated.

Google Kubernetes Engine Rapid - Version 1.19.8-gke.1600 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - Version 1.18.16-gke.302 is now the default version in the Regular channel.

Google Kubernetes Engine Stable - (2021-R10) Version updates Version 1.17.17-gke.2800 is now the default version in the Stable channel.

Load Balancing - External TCP/UDP Network Load Balancing is now supported with backend services.

Cloud Logging - Logs Views are now Generally Available (GA).

Cloud Run for Anthos - The free trial for Cloud Run for Anthos on Google Cloud has been extended and is now available until September 30, 2021. Known issue: Clusters that are upgraded to version 0.20.0-gke.6 might recieve the following error when you update the cluster's configmap: Error from server (InternalError): error when replacing "/tmp/file.yaml": Internal error occurred: failed calling webhook "config.webhook.istio.networking.internal.knative.dev": the server rejected our request for an unknown reason To resolve the error, you must run the following command to remove the validatingwebhookconfiguration configuration that is no longer supported in 0.20.0: kubectl delete validatingwebhookconfiguration config.webhook.istio.networking.internal.knative.dev After removing the unsupported configuration, you can proceed with updating your cluster's configmap.

Cloud Spanner - You can now use Customer-Managed Encryption Keys (CMEK) to protect databases in Cloud Spanner. You can now optionally specify the priority of data requests.

Cloud SQL Postgres - Cloud SQL for PostgreSQL flags are now generally available.

Cloud SQL SQL Server - You can integrate Cloud SQL for SQL Server with Managed Service for Microsoft Active Directory.

Cloud Storage - Cloud CDN, external HTTP(S) Load Balancing, and Cloud Storage services use BoringSSL, and are not affected by the recent OpenSSL security advisory that relates to CA certificate checks (CVE-2021-3450) and TLS renegotiation (CVE-2021-3449).

Deep Learning VM - M66 Release PyTorch 1.8 support in deep learning environments (Deep Learning VM Image and Deep Learning Containers) is available.

Dialogflow Enterprise - The legacy analytics page has been removed from the Dialogflow ES console.

Service Mesh - 1.9.x. 1.9.2-asm.1 is now available. Anthos Service Mesh user authentication is now available as a public preview feature on installations of 1.9. 1.6.x & 1.4.x & 1.5.x. The Anthos Service Mesh Topology (beta) page in Cloud Console won't display properly if unsupported versions, including versions earlier than Anthos Service Mesh 1.6.8, are installed on your clusters or if you have disabled the Canonical Service controller in clusters in your project.

Secret Manager - Secret Manager now has a Best Practices guide. Secret Manager Event Notifications is generally available. Secret Manager Expiration is generally available. Secret Manager Rotation is generally available.

Memorystore for Memcached - v1. Added new Memorystore for Memcached region: Warsaw (europe-central2).

Document AI - v1. Lending DocAI General Availability (GA) released Lending DocAI is now General Availability. Lending DocAI processors added The following Lending DocAI processors are now available: Pay stub parser Bank statement parser 1040 Schedule C parser 1099-DIV parser 1099-G parser 1099-INT parser. v1. Document AI General availability (GA) released Document AI is now General Availability (GA).

Anthos GKE on AWS - An issue has been discovered with Anthos clusters on AWS 1.7.0. Anthos clusters on AWS 1.7.0-gke.11 is now available. Anthos clusters on AWS now supports Kubernetes 1.19. Anthos clusters on AWS now supports exporting logs and metrics from an Anthos clusters on AWS user cluster to Cloud Logging and Cloud Monitoring. Anthos clusters on AWS now supports CMK encryption for component volumes. Workload identity in user clusters is now generally available. Anthos clusters on AWS now supports gp3 EBS volume types.

Anthos Service Mesh 1.5 - 1.9.x. 1.9.2-asm.1 is now available. Anthos Service Mesh user authentication is now available as a public preview feature on installations of 1.9. 1.6.x & 1.4.x & 1.5.x. The Anthos Service Mesh Topology (beta) page in Cloud Console won't display properly if unsupported versions, including versions earlier than Anthos Service Mesh 1.6.8, are installed on your clusters or if you have disabled the Canonical Service controller in clusters in your project.

Dataproc Metastore - v1. The GA release of Dataproc Metastore is now available. Dataproc Metastore imports and exports now support Avro storage format. Dataproc Metastore supports backing up and restoring service metadata and configuration. You can enable sync from a Dataproc Metastore service to Data Catalog. Dataproc Metastore supports asynchronous background tasks through the Canary release channel.

Transcoder API - v1beta1. Beta stage support for VPC Service Controls.

Anthos Service Mesh 1.5 - 1.9.x. 1.9.2-asm.1 is now available. Anthos Service Mesh user authentication is now available as a public preview feature on installations of 1.9. 1.6.x & 1.4.x & 1.5.x. The Anthos Service Mesh Topology (beta) page in Cloud Console won't display properly if unsupported versions, including versions earlier than Anthos Service Mesh 1.6.8, are installed on your clusters or if you have disabled the Canonical Service controller in clusters in your project.

AI Platform Unified - AI Platform (Unified) is now available in General Availability (GA). AI Platform (Unified) has added support for the following regions for custom model training, as well as batch and online prediction for custom-trained models: us-west1 (Oregon) us-east1 (South Carolina) us-east4 (N.

VMware Engine - The Google Cloud Business Associate Agreement (BAA) now also covers Google Cloud VMware Engine. Restructured documentation to better group content and improve workflow discoverability.

Anthos Service Mesh - 1.9.x. 1.9.2-asm.1 is now available. Anthos Service Mesh user authentication is now available as a public preview feature on installations of 1.9. 1.6.x & 1.4.x & 1.5.x. The Anthos Service Mesh Topology (beta) page in Cloud Console won't display properly if unsupported versions, including versions earlier than Anthos Service Mesh 1.6.8, are installed on your clusters or if you have disabled the Canonical Service controller in clusters in your project.

Anthos clusters on AWS - An issue has been discovered with Anthos clusters on AWS 1.7.0. Anthos clusters on AWS 1.7.0-gke.11 is now available. Anthos clusters on AWS now supports Kubernetes 1.19. Anthos clusters on AWS now supports exporting logs and metrics from an Anthos clusters on AWS user cluster to Cloud Logging and Cloud Monitoring. Anthos clusters on AWS now supports CMK encryption for component volumes. Workload identity in user clusters is now generally available. Anthos clusters on AWS now supports gp3 EBS volume types.

Database Migration Service - Database Migration Service makes it easier for you to "lift and shift" your MySQL and PostgreSQL workloads into Cloud SQL.

Anthos Service Mesh - 1.9.x. 1.9.2-asm.1 is now available. Anthos Service Mesh user authentication is now available as a public preview feature on installations of 1.9. 1.6.x & 1.4.x & 1.5.x. The Anthos Service Mesh Topology (beta) page in Cloud Console won't display properly if unsupported versions, including versions earlier than Anthos Service Mesh 1.6.8, are installed on your clusters or if you have disabled the Canonical Service controller in clusters in your project.

Cloud Run for Anthos - The free trial for Cloud Run for Anthos on Google Cloud has been extended and is now available until September 30, 2021. Known issue: Clusters that are upgraded to version 0.20.0-gke.6 might recieve the following error when you update the cluster's configmap: Error from server (InternalError): error when replacing "/tmp/file.yaml": Internal error occurred: failed calling webhook "config.webhook.istio.networking.internal.knative.dev": the server rejected our request for an unknown reason To resolve the error, you must run the following command to remove the validatingwebhookconfiguration configuration that is no longer supported in 0.20.0: kubectl delete validatingwebhookconfiguration config.webhook.istio.networking.internal.knative.dev After removing the unsupported configuration, you can proceed with updating your cluster's configmap.

SAP Solutions - The Cloud Data Fusion plugin for SAP is now available. The Deployment Manager template that Google Cloud provides to automate the deployment of Linux high-availability clusters for SAP HANA now supports Red Hat Enterprise Linux (RHEL) images.

Transfer Appliance - Transfer Appliance version 2.2 is deprecated and replaced by Transfer Appliance version 4.0. Transfer Appliance version 4.0 is now available to order in Singapore.

GKE - (2021-R10) Version updates Version 1.17.17-gke.2800 is now the default version.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko