News

Anthos—driving business agility and efficiency - Anthos updates, including GA for AWS support.

App modernization with Migrate for Anthos: now supporting day-two ops - Migrate for Anthos accelerates the process of migrating applications to containers.

Back up on demand, emulate and develop with ease — new Spanner features - Cloud database service Spanner adds backup-restore feature plus new developer features, like local emulator, query optimizer versioning, and more.

Modernize Enterprise Networking with Cisco SD-WAN and Google Cloud - Cisco SD-WAN Cloud Hub with Google Cloud brings a new set of capabilities to our customers to simplify enterprise networking and advance security capabilities, while helping IT teams minimize operational costs and meet application service-level objectives.

Keep your teams working safely with BeyondCorp Remote Access - Enabling remote access to internal apps with a simpler and more secure approach without a remote-access VPN.

Beam summit - Digial Summit June 15-19, 2020.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Best practices for optimizing your cloud costs - Following these best practices will help optimize your cloud costs to the needs of your business, so you can get through these unpredictable times.

Security blueprint: PCI on GKE - The PCI on GKE blueprint contains a set of Terraform configurations and scripts that demonstrate how to bootstrap a PCI environment in Google Cloud.

Kubernetes Engine (GKE) multi-cluster life cycle management series - Five-part series on multi-cluster GKE architecture.

Deploy Springboot to GKE from scratch in 7 minutes - Deploying Springboot on Google Kubernetes Engine.

How to Secure Kubernetes the Easy Way - How to use Terraform and Kubeadm to bootstrap and secure your Kubernetes cluster.

How to Scale Kubernetes Applications Using Custom Metrics - Scale your containers using custom Stackdriver metrics that are important to your business.

Google Anthos : First Look - A brief overview of Anthos.

Automating the Deployment of Infrastructure Using Terraform on GCP - Example of creating Compute Engine instances using Terraform.

Capacity Management with Load Balancing - Comparing regional and global load balancing.

ChatOps for Production Access Control - Using IAM Conditions with Cloud Functions and Slack for access control.

Migrating a Monolith to Google Kubernetes Engine (GKE) — Data migration - Different approaches for data migration when migrating a monolith application to microservices.

22 Google Cloud Regions and 67 Zones Equals Endless Possibilities - Overview of Google Cloud regions and zones and important things related to them.

App Development, Serverless, Databases, DevOps

Updating Cloud IoT Core Config with Google Cloud functions (in Go) - How to write a Google Cloud Function to update your IoT devices’ configurations on Cloud IoT Core.

Migrating a shell script deployed Cloud Run service to use Terraform - Converting Cloud Run deployment shell script to Terraform deployment.

Big Data, Analytics, ML&AI

Building Firewatch Australia, Part 1 - Data Processing - A serverless pipeline to load data about Australian bushfires into Cloud Firestore.

Building a Simple ETL Pipeline with Python and Google Cloud Platform - Extracting data from an FTP server using Google Cloud Functions.

Creation of an ETL in Google Cloud Platform for automated reporting - Learn how to create your own serverless and fully scalable ETL for automated reporting using PyTrends as an example.

Cloud Data Fusion — Import custom plugins and integrate AWS Aurora with GCS - Using Data Fusion plugin to connect to AWS Aurora.

Apache Spark & Google Cloud DataProc - The article goes through a process of setting Dataproc cluster and executing batch Spark job which stores results in BigQuery.

Ingest Data from Google Cloud Dataflow to BigQuery — Without the Headaches (Part II) - Handling BigQuery schema changes in a Dataflow job.

Optimize BigQuery costs with Flex Slots - Explanation of BigQuery Flex Slots and how to provision via Python library.

Introduction to Arrays in BigQuery - Tutorial on working with arrays in BigQuery.

How to UNPIVOT multiple columns into tidy pairs with SQL and BigQuery - This post is for anyone dealing with time series in CSVs with one new column for each day.

Empowering Apple Mobility Trends Reports with BigQuery and Data Studio - Analyzing Apple's mobility data using BigQuery and Data Studio.

Training a recommendation model for Google Analytics data using BigQuery ML - This article demonstrates how to implement a WALS (matrix factorization) model to do collaborative filtering to recommend newspaper articles using BigQuery ML.

Using BigQuery (and BigQuery ML) from Kubeflow Pipelines - Creating a Kubeflow pipeline which creates BigQuery ML model.

On-demand small batch predictions with Cloud Run and Embedded-tf - Going over possibilities to serve large number of Tensorflow models on GCP.

Slides, Videos, Audio

GCP Podcast - #217 Cost Optimization with Justin Lerma and Pathik Sharma.

Kubernetes Podcast - #100 Kubernetes Community Redux, with Paris Pittman.

Adventures in the Cloud with Yufeng Guo - Using Data Studio with BigQuery, Serverless ML with Cloud Run, BigQuery ML and K-means.

Releases

Anthos Config Management - 1.3.1. Anthos Config Management images are now included in the Google-provided system images for Binary Authorization. Policy Agent now allows configuration of namespaces that will bypass the admission controller. You can now exempt Namespaces from Policy Controller enforcement. Earlier versions of Anthos Config Management relied on APIs that have been deprecated in Kubernetes v1.16. The Anthos Config Management Syncer pod now reports when it detects that it is fighting with another process over a resource. Anthos Config Management no longer allows managing resources in unmanaged Namespaces. If you define a CRD with an integer field that has min/max values, Anthos Config Management will be unable to update the CRD. Anthos Config Management no longer overwrites undeclared labels and annotations on Namespaces. This release includes several performance and memory improvements.

Anthos - Anthos 1.3.1 is now available.

AppEngine Standard Python - App Engine is now available in the us-west4 region (Las Vegas, NV).

BigQuery ML - BigQuery ML now supports exporting BigQuery ML models to Cloud Storage and using them for online prediction.

BigTable - Cloud Bigtable is now available in the us-west4 (Las Vegas) region.

Billing - For customers with self-serve/online Cloud Billing accounts, you can now find your Cloud Billing documents in the Documents page of the Cloud Billing console. Budget alerts: new budget filters are now available.

Cloud Dataflow - Dataflow is now able to use workers in zones in the us-west4 region (Las Vegas).

Data Fusion - Cloud Data Fusion version 6.1.2 is now available. Reduced preview startup by 60%. Fixed a bug that caused errors when Wrangler's parse-as-csv with header was used when reading multiple small files. Fixed a bug that caused zombie processes when using the Remote Hadoop Provisioner. Fixed a bug that caused DBSource plugin to fail in preview mode. Fixed a race condition that caused a failure when running a Spark program.

Dataproc - Dataproc is now available in the us-west4 region (Las Vegas).

Datastore - Support for us-west4 region (Las Vegas).

Dialogflow - In May 2020, the Facebook Messenger integration will be updated, and you may notice slight changes related to fulfillment. Beta launch of one-click integrations with two telephony partners: AudioCodes SignalWire.

Cloud Filestore - Filestore is available in the us-west4 (Las Vegas) region.

Cloud Firestore - Support for us-west4 region (Las Vegas).

GKE on Prem - Preflight check in gkeadm for access to the Cloud Storage bucket that holds the admin workstation OVA. Preflight check for internet access includes additional URL www.googleapis.com. Preflight check for test VM DNS availability. Preflight check for test VM NTP availability. Preflight check for test VM F5 access. Before downloading and creating VM templates from OVAs, GKE on-prem checks if the VM template already exists in vCenter. Rename gkeadm’s automatically created service accounts. OVA download displays download progress. gkeadm prepopulates bundlepath in the seed config on the admin workstation. Fix for Docker failed DNS resolution on admin workstation at startup. Admin workstation provisioned by gkeadm uses thin disk provisioning. Improved user cluster Istio ingress gateway reliability. Ubuntu image is upgraded to include newest packages. Update the vCenter credentials for your clusters using the preview command gkectl update credentials vsphere. The gkeadm configuration file, admin-ws-config.yaml, accepts paths that are prefixed with ~/ for the Certificate Authority (CA) certificate. Test VMs wait until the network is ready before starting preflight checks. Improve the error message in preflight check failure for F5 BIG-IP. Skip VIP check in preflight check in manual load balancing mode. Upgraded Calico to version 3.8.8 to fix several security vulnerabilities. Upgraded F5 BIG-IP Controller Docker image to version 1.14.0 to fix a security vulnerability. Fixed gkeadm admin workstation gcloud proxy username and password configuration. Fixed the bug that was preventing gkectl check-config from automatically using the proxy that you set in your configuration file when running the full set of preflight validation checks with any GKE on-prem download image. Fixed an admin workstation upgrade failure when the upgrade process was unable to retrieve SSH keys, which would cause a Golang segmentation fault.

Cloud Healthcare API - Code samples are available for the Cloud Healthcare API v1 in Go, Java, Node.js, and Python.

KMS - Cloud KMS and Cloud EKM resources are available in the us-west4 region.

Google Kubernetes Engine - The ability to create new GKE clusters or update existing GKE clusters with node pools running Windows Server is now generally available. Master global access for private clusters is now available in beta. A known kernel bug in Linux 4.19 may cause softlockup when running eBPF workloads. The region us-west4 in Las Vegas is now available.

Load Balancing - External HTTP(S) load balancers now support header-based routing and query parameter-based routing. Internal TCP/UDP Load Balancing with failover groups is available in General Availability.

Cloud Memorystore - Added new Memorystore for Redis region: Las Vegas (us-west4).

Cloud Profiler - The Cloud Profiler Node.js agent is now generally available. The Cloud Profiler Node.js agent now supports release 12 of Node.js. The Cloud Profiler Node.js agent no longer supports release 8 of Node.js.

Cloud PubSub - Pub/Sub is now available in the us-west4 region (Las Vegas). Dead-letter topics for Pub/Sub are now available at the General Availability release level.

Cloud Spanner - Cloud Spanner regional instances can now be created in Las Vegas (us-west4).

Cloud SQL MySQL - Support for us-west4 region (Las Vegas).

Cloud SQL Postgres - Support for us-west4 region (Las Vegas).

Cloud SQL SQL Server - Support for us-west4 region (Las Vegas).

Cloud Storage - Las Vegas region (us-west4) launched.

Cloud TPU - Cloud TPUs and Cloud TPU Pods now support PyTorch 1.5 via the PyTorch/XLA integration.

Virtual Private Cloud - Private Google Access for on-premises hosts permits on-premises hosts to send traffic from any internal IP addresses, not just RFC 1918 addresses. For auto mode VPC networks, added a new subnet 10.182.0.0/20 for the Las Vegas us-west4 region. Packet Mirroring pricing will come into effect from June 20, 2020.

Cloud VPN - Cloud VPN is now available in region us-west4 (Las Vegas, Nevada, USA).

AI Platform Prediction - Visualization settings for AI Explanations are now available.

Cloud VPN - Cloud VPN is now available in region us-west4 (Las Vegas, Nevada, USA).

Anthos GKE deployed on-prem - Preflight check in gkeadm for access to the Cloud Storage bucket that holds the admin workstation OVA. Preflight check for internet access includes additional URL www.googleapis.com. Preflight check for test VM DNS availability. Preflight check for test VM NTP availability. Preflight check for test VM F5 access. Before downloading and creating VM templates from OVAs, GKE on-prem checks if the VM template already exists in vCenter. Rename gkeadm’s automatically created service accounts. OVA download displays download progress. gkeadm prepopulates bundlepath in the seed config on the admin workstation. Fix for Docker failed DNS resolution on admin workstation at startup. Admin workstation provisioned by gkeadm uses thin disk provisioning. Improved user cluster Istio ingress gateway reliability. Ubuntu image is upgraded to include newest packages. Update the vCenter credentials for your clusters using the preview command gkectl update credentials vsphere. The gkeadm configuration file, admin-ws-config.yaml, accepts paths that are prefixed with ~/ for the Certificate Authority (CA) certificate. Test VMs wait until the network is ready before starting preflight checks. Improve the error message in preflight check failure for F5 BIG-IP. Skip VIP check in preflight check in manual load balancing mode. Upgraded Calico to version 3.8.8 to fix several security vulnerabilities. Upgraded F5 BIG-IP Controller Docker image to version 1.14.0 to fix a security vulnerability. Fixed gkeadm admin workstation gcloud proxy username and password configuration. Fixed the bug that was preventing gkectl check-config from automatically using the proxy that you set in your configuration file when running the full set of preflight validation checks with any GKE on-prem download image. Fixed an admin workstation upgrade failure when the upgrade process was unable to retrieve SSH keys, which would cause a Golang segmentation fault.

Dialogflow Enterprise - In May 2020, the Facebook Messenger integration will be updated, and you may notice slight changes related to fulfillment. Beta launch of one-click integrations with two telephony partners: AudioCodes SignalWire.

Anthos GKE on-prem - Preflight check in gkeadm for access to the Cloud Storage bucket that holds the admin workstation OVA. Preflight check for internet access includes additional URL www.googleapis.com. Preflight check for test VM DNS availability. Preflight check for test VM NTP availability. Preflight check for test VM F5 access. Before downloading and creating VM templates from OVAs, GKE on-prem checks if the VM template already exists in vCenter. Rename gkeadm’s automatically created service accounts. OVA download displays download progress. gkeadm prepopulates bundlepath in the seed config on the admin workstation. Fix for Docker failed DNS resolution on admin workstation at startup. Admin workstation provisioned by gkeadm uses thin disk provisioning. Improved user cluster Istio ingress gateway reliability. Ubuntu image is upgraded to include newest packages. Update the vCenter credentials for your clusters using the preview command gkectl update credentials vsphere. The gkeadm configuration file, admin-ws-config.yaml, accepts paths that are prefixed with ~/ for the Certificate Authority (CA) certificate. Test VMs wait until the network is ready before starting preflight checks. Improve the error message in preflight check failure for F5 BIG-IP. Skip VIP check in preflight check in manual load balancing mode. Upgraded Calico to version 3.8.8 to fix several security vulnerabilities. Upgraded F5 BIG-IP Controller Docker image to version 1.14.0 to fix a security vulnerability. Fixed gkeadm admin workstation gcloud proxy username and password configuration. Fixed the bug that was preventing gkectl check-config from automatically using the proxy that you set in your configuration file when running the full set of preflight validation checks with any GKE on-prem download image. Fixed an admin workstation upgrade failure when the upgrade process was unable to retrieve SSH keys, which would cause a Golang segmentation fault.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko