Welcome to issue #190 May 18th, 2020

News

Google Cloud and NVIDIA’s enhanced partnership accelerates computing workloads - We’re announcing our support for the new NVIDIA Ampere architecture and the NVIDIA A100 Tensor Core GPU.

Announcing Google Cloud VMware Engine: Accelerating your cloud journey - VMware Engine is a fully managed offering that lets enterprises run existing virtual machines on Google Cloud infrastructure.

New WAF capabilities in Cloud Armor for on-prem and cloud workloads - New web application firewall capabilities are now available in Cloud Armor for workloads on-premise or in the cloud.

DataStax brings Apache Cassandra as a service to Google Cloud - Open source database Apache Cassandra is now available as a service on Google Cloud for massive scalability.

Live with Google Cloud Developer Relations - Join Google Cloud Developer Relations in online discussions and livestreams, as we explore everything cloud -- from hybrid workflows and serverless to big data and machine learning.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Get to know Google Cloud with our new Architecture Framework - The new Architecture Framework provides and overview of Google Cloud services and deployment principles.

Anthos in depth: Transforming your legacy Java applications - How to modernize legacy Java applications with Anthos.

Using logging for your apps running on Kubernetes Engine - How to use Cloud Logging with Google Kubernetes Engine.

Load balancing TCP services in Multiple GKE clusters (different regions) - Routing a TCP traffic to GKE clusters sitting in different regions.

Container Load Balancing on Google Kubernetes Engine - Overview of container native load balancing.

There and Back Again — Scaling Multi-Tenant Kubernetes Cluster(s) - A story about using GKE.

The case of the missing DNS packets: a Google Cloud support story - A Google Cloud Technical Solution Engineer shares a particularly thorny support case that they recently solved. Along the way, they’ll show you the information gathered in the course of troubleshooting, and how they reasoned their way through to a resolution.

Edge Security with Cloud Armor - Tutorial on how to set up Cloud Armor to secure web app on GKE.

GCP — Secure Bastion - Using Identity Aware Proxy to expose SSH and TCP services over the Internet in a secure manner.

App Development, Serverless, Databases, DevOps

Building an Async Event-driven Slack Bot on GCP for Engineering Support - Using serverless products on GCP to create Slack Bot.

How to Set Up a Deployment Pipeline on GCP with Cloud Build, Container Registry and Cloud Run - Automatically building and deploying containers into Cloud Run when changes get pushed to your Git repositories.

Cloud Firestore geo queries - A voyage through the options for querying Cloud Firestore by geographic proximity or map bounds.

Deploying Python Application in Google Cloud (GCE, GAE, GKE & Cloud Run) — Part 1 - Group of articles which describes several ways of deploying Python web application on GCP products.

Lesson Learned from my first GCP APP Engine Deployment with Node.js - Issues when deploying NodeJS app for the first time on App Engine.

How To Connect From Cloud Functions To The Private IP Address Of Cloud SQL In Google Cloud - Creating infrastructure with Terraform and connecting to Cloud Function to private Cloud SQL instance.

Launch checklist for Cloud Spanner - This launch checklist provides a list of considerations that need to be made prior to launching a production application on Cloud Spanner. It is not intended to be exhaustive, but serves to highlight areas that can have a large impact on production performance.

Big Data, Analytics, ML&AI

Run Dataflow Jobs in a Shared VPC without Regional Endpoints on GCP - Configuring Dataflow jobs to use Shared VPC.

Get Started with Airflow + Google Cloud Platform + Docker - Setting up Airflow with Docker compose to connect to GCP.

Airflow on GCP (May 2020) - This is a complete guide to install Apache Airflow on a Google Cloud Platform Virtual Machine from scratch.

Cloud Firestore on Beam with Java - Creating custom transformation in Java to upload data to Cloud Firestore.

CI/CD your BigQuery infrastructure with Google Cloud Build - Using Cloud Build for BigQuery CI/CD.

Using BigQuery Flex Slots to run machine learning workloads more efficiently - Create Flex Slots, run query, then delete the flex slots.

Guidelines for cost optimisation of chargeable Data Studio dashboards using BigQuery - Few approaches on how to optimize costs when using data from BigQuery in Data Studio.

Sync Oracle to BigQuery With Golden Gate BigQuery Adapter - In this article, we are going to see how to sync oracle data to BigQuery with the golden gate bigquery adapter with the initial load.

Import SQL Server data in BigQuery - A list of four approaches for a one-off data dumps from a RDBMS like SQL Server to BigQuery.

Google Cloud Data Catalog and Tableau integration - Leverage Data Catalog to discover & annotate Tableau assets.

Google Cloud Data Catalog — Integrate Your On-Prem RDBMS Metadata - Code samples with a practical approach on how to ingest metadata from on-premise Relational Databases into Google Cloud Data Catalog.

Setup Kubeflow Cluster in a Shared VPC on Google Cloud Platform - This post tries to describe the steps you need to follow to set up a Kubeflow using a Shared VPC through the command line.

Democratizing image classification - Creating ML infrastructure on Kubernetes for image classification.

How to build a custom Sentiment Analysis model with Google AutoML Natural Language - Creating a custom Sentiment Analysis model on Cloud AutoML Natural Language.

MLOps series #1 : Batch scoring with Mlflow Model (Mleap flavor) on Google Cloud Platform - Deploying ML model from Databrics cluster to Cloud Dataproc.

Streaming analytics on Google Cloud for regulated industries. - This blog demonstrates how a streaming analytics pipeline on Google Cloud using PubSub, Apache Beam (on Dataflow runner), Cloud Storage, and BigQuery can be executed in a single region and protected end to end using Customer-Managed Encryption key (CMEK).

YouTube-8M on AI Platform - The article goes through how to use AI Platform to train, evaluate and run predictions for the Youtube-8M dataset.

Various

Cloud cost optimization: principles for lasting success - Learn organizational principles that can help you run your cloud environment efficiently and cost effectively.

Automate Inventory of Google Cloud Resources - To visualize or policy enforcement, or meet compliance auditing requirement.

GPU shortage in all regions - It seems like there's a mass shortage of all GPUs.

How Khan Academy Successfully Handled 2.5x Traffic in a Week

Slides, Videos, Audio

GCP Podcast - #220 Strise with Marit Rødevand.

Kubernetes Podcast - #103 CSI: Storage, with Saad Ali.

Okay, Kelsey tell me how....? - If you could get 15 minutes with Kelsey Hightower, what would you ask? Well you’re in luck! This week Kelsey will share bright ideas for staying afloat and outpacing your competition. This session is customized and focused on your experience right now.

Releases

AI Platform - Deep Learning VMs - M47 release Fixed an OS login issue under single user mode for a user external to an organization.

AppEngine Standard Python - Updated Python SDK to version 1.9.91.

BigQuery - Updated versions of Magnitude Simba JDBC drivers have been released. Updated versions of Magnitude Simba ODBC drivers have been released.

Cloud Composer - New versions of Cloud Composer images: composer-1.10.3-airflow-1.10.2, composer-1.10.3-airflow-1.10.3 and composer-1.10.3-airflow-1.10.6. Resource quota limits have been updated, allowing environment administrators to set quotas with more granularity. The machine type of the Airflow web server will now be preserved during Composer environment updates, including cases like new PyPi module installations, or adding new environment variables. Composer version 1.6.1 has been deprecated.

Compute Engine - Automatically manage the size of sole-tenant node groups with the sole-tenant node group autoscaler. You can identify idle persistent disk resources by using idle persistent disk recommendations.

Config Connector - fix ContainerNodePool version upgrade scenario. increase the cpu/memory request for webhook and recorder. Miscellaneous bug fixes and improvement.

Google Kubernetes Engine - Container Threat Detection is now available in Beta. GKE cluster versions have been updated. The default version for new clusters is now 1.14.10-gke.36. The following Kubernetes versions are now available for new clusters and for opt-in master upgrades and node upgrades for existing clusters. No channel Note: Your clusters might not have these versions available. The COS image for GKE 1.16 clusters is now cos-77-12371-251-0. The COS image for GKE 1.17 clusters is now cos-81-12871-69-0. 1.14.10-gke.27. 1.14.10-gke31. 1.14.10-gke.32. 1.14.10-gke.34. We expect the following changes in the coming weeks. Google Kubernetes Engine will gradually upgrade clusters in the Regular channel to GKE 1.16 beginning in an upcoming release.

Google Kubernetes Engine Rapid - 1.17.5-gke.0 is now available in the Rapid release channel.

Google Kubernetes Engine Stable - 1.14.10-gke.36 is now available in the Stable release channel.

Cloud Logging - You can now use regular expressions to query your logs data and create filters.

Cloud Memorystore - Released support for VPC Service Controls for Memorystore for Redis.

Cloud Profiler - The Cloud Profiler Python agent is now generally available.

Cloud Run - Cloud Run (fully managed) now supports connecting to a VPC network with Serverless VPC Access, in beta.

Security Command Center - Security Command Center Premium and Standard tiers are now available. The Security Command Center Premium tier includes: Security Health Analytics Web Security Scanner managed scans Event Threat Detection Container Threat Detection Learn more about the Security Command Center Premium tier. The Event Threat Detection API will be deprecated in the coming months. Container Threat Detection currently supports the following Kubernetes Engine versions on the Regular and Rapid channels: >= 1.15.9-gke.12 >= 1.16.5-gke.2 >= 1.17 In a future update, Container Threat Detection will support version 1.14 and the Stable channel.

Cloud SQL Postgres - PostgreSQL 9.6 minor version is upgraded to 9.6.16. Cloud SQL has expanded support for PostgreSQL extensions.

Cloud TPU - Cloud TPU currently supports TensorFlow version 1.15.2.

VPC Service Controls - Beta stage support for the following integration: Memorystore for Redis.

AI Platform Prediction - AI Platform Prediction now supports the following regions for batch prediction, in addition to those that were already supported: northamerica-northeast1 (Montréal) southamerica-east1 (São Paulo) australia-southeast1 (Sydney) See the full list of available regions.

Deep Learning VM - M47 release Fixed an OS login issue under single user mode for a user external to an organization.

Service Mesh - 1.4.9-asm.1 Fixes the security issue, CVE-2020-10739, with the same fixes as OSS Istio 1.4.9.

AI Platform Training - AI Platform Training now supports the following regions, in addition to those that were already supported: northamerica-northeast1 (Montréal) southamerica-east1 (São Paulo) australia-southeast1 (Sydney) GPUs are available for training in each of the new regions: NVIDIA Tesla P4 GPUs are available in northamerica-northeast1.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko