Welcome to issue #258 September 6th, 2021

News

More than just relational data at scale with Spanner’s new JSON data type - Achieve better agility and data quality without losing functionality when managing semi-structured data in Cloud Spanner with new JSON data type.

Cloud SQL for PostgreSQL now supports Linux Huge Pages - Cloud SQL for PostgreSQL now supports Linux Huge Pages.

New Cloud Functions min instances reduces serverless cold starts - Setting ‘min instances’ on your Cloud Functions applications translates to lower startup times.

Monitoring made simple for Google Cloud VMware Engine and Google Cloud operations suite - Learn how we simplified monitoring for Google Cloud VMware Engine and Google Cloud operations suite.

Save money and time with automated VM management and suspend/resume - A new guide helps you understand how to control costs for your cloud-based VMs.

SAP HANA: A solution to memory error impacts - Introducing Memory Poisoning Recovery (MPR) from Google Cloud, minimizing the impact of memory errors for SAP HANA customers.

Google Cloud improves Healthcare Interoperability on FHIR - Google Cloud improves healthcare interoperability using the FHIR schema using Healthcare Data Engine, BigQuery, and Looker.

Begin your headless commerce journey with Google Cloud and commercetools - Learn the most common Google Cloud architecture components that can integrate with commercetools API to build a scalable headless commerce solution.

Google invests 1 billion euros in Germany's digital future - Google is supporting Germany's transition to a digital and sustainable economy, investing 1 billion euros in digital infrastructure and clean energy.

Announcing the Government & Education Summit, Nov 3-4, 2021 - The pandemic has served as a catalyst for new ideas and creative solutions to long-standing global issues, including climate change, public health, and resource assistance. We’ve seen all levels of government and education leverage cloud technology to meet these challenges with a fervor and determination not seen since the industrial revolution. We can’t wait to bring those stories to you at the 2021 Google Cloud Government and Education Summit.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Traffic Director explained! - If your application is deployed in a microservices architecture then you are likely familiar with the networking challenges that come with it. Traffic Director helps you run microservices in a global service mesh.

What is Network Intelligence Center? - Network Intelligence Center provides a single console for managing Google Cloud network observability, monitoring, and troubleshooting.

How Blibli manages updates for stateful and stateless applications on GKE - Learn from Blibli, an Indonesian Ecommerce provider, on how to manage updates to your applications on Google Kubernetes Engine (GKE).

Improve gRPC service availability and efficiency with Traffic Director - Make your proxyless gRPC services with Traffic Director more reliable and efficient with the new capabilities: Retry and Session Affinity.

Google Cloud VMware Engine, PowerCLI and BigQuery Analytics - Exporting data from Google Cloud billing data into a BigQuery dataset is relatively straight-forward. However, exporting data from GCVE into BigQuery requires PowerShell scripting. The following blog details steps to extract data from GCVE and load it into BigQuery for reporting and analysis.

Tutorial: Cost Spike Alerting for Google Cloud Platform - Build a solution that would alert daily on any large percentage cost increase or decrease from the previous day on a per service basis.

Using GCP Cloud Asset Inventory Export to keep track of your GCP resources over time - Google Cloud Asset Inventory is a great service that allows you to view, monitor, and analyze your GCP assets, giving you the option to export a snapshot of your entire inventory at any point in time (up to 35 days backward).

Estimate your GKE costs early in the development cycle using GitLab - This tutorial demonstrates the best practice of shifting Google Kubernetes Engine (GKE) cost visibility to your development team using GitLab. Creating awareness of costs early in the development process helps you to avoid surprises in your Google Cloud bill.

App Development, Serverless, Databases, DevOps

Protecting Healthcare data with DLP: A guide for getting started - Use Google Cloud DLP to protect healthcare data without reducing its ability to help patients get better care.

Modernizing your serverless applications - Introducing the "Serverless Migration Station" series of videos & codelabs with sample code to help existing users modernize their serverless apps.

SAP on Google Cloud: Set up your network for a seamless deployment - The network deployment for your SAP systems has a tremendous impact on its availability and performance. Here are some options with Google Cloud.

Migration success with Operational Health Reviews from Google Cloud’s Professional Service Organization - Google Cloud PSO helps customers maximize their migrations and deployments.

Deploying a Cloud Spanner-based Node.js application - How to deploy a Node.js application on Cloud Spanner, and cover a few important Cloud Spanner concepts along the way.

Reduce latency with Cloud Spanner multi-region leader placement - Some examples of how you can use configurable leader placement to improve application latency when using Cloud Spanner.

Why API-first is the best approach to application integration - Why API-first approaches are essential a successful integration strategy.

Choosing server-less option on GCP - Discussion about server-less options in GCP: Google Cloud Functions, App Engine Standard, App Engine Flex, Cloud Run, Kubernetes Engine.

Google Cloud Build — under the hood - Investigating security on Cloud Build.

Solving for “live streaming” - Building a production-grade architecture with Google Cloud - Building video streaming solution on Google Cloud.

Django on Google Cloud Run - This tutorial explains the creation of a fully containerized Django app and deploying it to Cloud Run with CI/CD, credentials management, and static file hosting.

Big Data, Analytics, ML&AI

BigQuery Admin reference guide: Recap - Looking for a path to jumpstart your BigQuery architecture and administration knowledge? This series walks you through the BigQuery fundamentals.

How to load Salesforce data into BigQuery using a code-free approach powered by Cloud Data Fusion - Move Salesforce data into BigQuery using an intuitive drag-and-drop solution based on pre-built connectors, and the self-service model of a code-free data integration service provided by Cloud Data Fusion.

What are the newest datasets in Google Cloud? - Want to know about the latest datasets from Google Cloud? Find information here in one handy location. Check back regularly as we update this blog with new datasets and best practices so you can enrich your analytics or ML initiatives.

Export Amazon Aurora to BigQuery using BigQuery Data Transfer Service - Loading AWS Aurora snapshots from S3 to BigQuery, using Data Transfer Service.

Using dataform to improve data quality in BigQuery - Get inspired on ways to use Dataform to monitor and improve your data quality!

Back to the future of the Datawarehouse Episode 2/3 - An overview of security issues for BigQuery and other data-related services on Google Cloud.

6 Best Practices for Managing Data Access to BigQuery - What to know in terms of security when setting up a data environment in BigQuery.

Sparkling Vertex AI Pipeline - How to deploy a simple Spark ML pipeline using Vertex AI Pipelines.

MLOps with Tensorflow Extended (TFX) and Tensorflow Decision Forest (TF-DF) (Part 2) - Example of deploying ML pipeline to Kubeflow.

Various

ManTech and Google Cloud open joint facility to expedite government adoption of cloud technologies - Transitioning from legacy infrastructure to the cloud, mitigating security risk, and enabling secure collaboration for a hybrid workforce are challenges many agencies face today. To meet the need for faster industry-wide cloud adoption, Google Cloud is partnering with ManTech, a company that deeply understands the unique needs of the U.S. government mission.

TIM Group chooses Cloud SQL over Oracle to power its billing apps - Italy’s largest telecom provider used Cloud SQL for PostgreSQL to automate billing and credit systems.

Journey from System Engineer to Google Cloud Engineer - Passing 3 GCP certification exams in 6 months.

Slides, Videos, Audio

GCP Podcast - #274 BigQuery Admin Reference Guides with Leigha Jarett.

Kubernetes Podcast - #161 Unicron, with Daniel Megyesi.

Security podcast - #29 - Future of EDR: Is It Reason-able to Suggest XDR?

Releases

Access Approval - v1. Speaker ID is supported by Access Approval in Preview stage.

Vertex AI - You can now use a pre-built container to perform custom training with TensorFlow 2.6 and PyTorch 1.9.

AI Platform Prediction - Runtime version 2.6 is now available.

Anthos clusters on AWS - Anthos clusters on AWS aws-1.8.2-gke.2 is now available. The supported versions also offer the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on AWS 1.8.

GKE on-prem 1.7 - Anthos clusters on VMware 1.7.3-gke.6 is now available. Fixes: Fixed the Ubuntu user password expiration issue. Anthos clusters on VMware 1.8.2-gke.11 is now available. Starting from version 1.8.2, Anthos clusters on VMware uses cert-manager instead of Istio Citadel for issuing TLS certificates used by metrics endpoints. Fixes: Fixed the Ubuntu user password expiration issue. HPA with custom metrics doesn't work in version 1.8.2 due to the migration from Istio to cert-manager for the monitoring pipeline.

Anthos GKE on AWS - Anthos clusters on AWS aws-1.8.2-gke.2 is now available. The supported versions also offer the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on AWS 1.8.

Cloud Asset Inventory - New resource types are now available.

BigQuery - An updated version of ODBC driver for BigQuery is now available that includes enhancements. Exporting table data in Parquet format is now generally available (GA).

Cloud Build - Users can now use build triggers in projects in the VPC Service Controls perimeter.

Cloud Composer - Cloud Composer 1.16.16 release started on August 30, 2021. Airflow 2.1.2 is available in Cloud Composer images. (Airflow 2) Cloud Composer now supports the stable Airflow REST API. (Airflow 2) Cloud Composer now uses a custom authentication backend for authentication in the stable Airflow REST API. New versions of Cloud Composer images: composer-1.16.16-airflow-1.10.12 composer-1.16.16-airflow-1.10.14 composer-1.16.16-airflow-1.10.15 (default) composer-1.17.0-preview.12-airflow-2.0.2 composer-1.17.0-preview.12-airflow-2.1.1 composer-1.17.0-preview.12-airflow-2.1.2.

Compute Engine - Generally available: When deleting VMs from a managed instance group, you can flag the operation to continue even if some instances were already deleted or if other instance validation errors occur.

Data Fusion - Preview: Cloud Data Fusion version 6.5.0 is now available. Features in 6.5.0: Preview: Cloud Data Fusion now supports role-based access control (RBAC). Changes in 6.5.0: In version 6.5.0, Spark 3 is the new default engine used when using Cloud Data Fusion Preview and when running pipelines on Dataproc clusters. Fixed in 6.5.0 preview version (for more information, see the CDAP release note): Fixed an issue in Replication that caused jobs to fail if more than 1000 tables were selected for replication. You can create connections for Database, MySQL, Oracle, PostgreSQL, and SQL Server sources, but the plugin properties do not include Use Connection.

Cloud Dataflow - Dataflow Prime is now available in Preview.

Dataproc Metastore - v1. Hive version 3.1.2 will become the default Dataproc Metastore service creation version in 1 week on September 6, 2021.

Dataproc - New sub-minor versions of Dataproc images: 1.4.70-debian10, 1.4.70-ubuntu18, 1.5.45-centos8, 1.5.45-debian10, 1.5.45-ubuntu18, 2.0.19-centos8, 2.0.19-debian10, 2.0.19-ubuntu18. Backported SPARK-34295: Added a new spark.yarn.kerberos.renewal.excludeHadoopFileSystemsconfiguration option. Image 2.0: OOZIE-3599: Upgraded Jetty version to 9.4.

Datastore - Added DATA_READ and DATA_WRITE Data Access audit logs.

Dialogflow - New Dialogflow CX agent roles are introduced for granular control of agent resources.

Cloud Firestore - Added DATA_READ and DATA_WRITE Data Access audit logs.

Cloud Functions - Cloud Functions has added support for a new runtime, Go 1.16, at the Preview release level. Cloud Functions adds support for setting a minimum number of instances, available at the Preview release level.

Google Kubernetes Engine - Multi-Instance GPU on GKE is is now generally available. GKE Autoscaling profiles are now generally available.

Cloud Logging - You can now collect nginx metrics and logs from the Ops Agent, starting with version 2.1.0.

Cloud Monitoring - Cloud Monitoring now lets you configure how long Monitoring waits to close an incident when observations stop arriving. You can now collect nginx metrics and logs from the Ops Agent, starting with version 2.1.0. The VM Instances page features enhanced scorecards for VM health.

Network Connectivity Center - You can now create Router appliance spokes by using the Google Cloud Console.

Cloud Spanner - The R2DBC driver for Cloud Spanner is available in Preview. In the Cloud Console, a database's Query page now supports multiple query tabs so you no longer have to clear one query to create and run another. Added support for changing the leader region location of a Cloud Spanner database. Added support for the JSON data type.

Cloud SQL MySQL - Cloud SQL for MySQL now supports using a custom import to set up replication from large external databases.

Cloud SQL Postgres - Cloud SQL for PostgreSQL support for pglogical, native logical replication, wal2json and test_decoding is now generally available. Cloud SQL for PostgreSQL support for Automatic IAM database authentication is now generally available.

Traffic Director - Traffic Director deployed with proxyless gRPC can now use the advanced traffic management features retry and session affinity.

Transcoder API - v1. All client library code samples updated to v1 of the API. Delete operations for jobs and job templates now return a 404 resource not found error if the specified resource name does not exist. Encryption support (and its associated documentation) is temporarily unavailable.

Vertex AI - You can now use a pre-built container to perform custom training with TensorFlow 2.6 and PyTorch 1.9.

Cloud VPN - Added Terraform examples to automate HA VPN gateway creation: HA VPN gateway between Google Cloud networks HA VPN gateway to a peer VPN gateway.

Workflows - v1. Support for iterating over a sequence of numbers or through a collection of data is generally available (GA).

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko