Welcome to issue #221 December 21st, 2020

News

Introducing a single command to build and deploy to Cloud Run - Now you can use Google Cloud Buildpacks to automatically convert your application code into a container image and deploy it to Cloud Run.

Introducing a new dashboard creation experience in Cloud Monitoring - Cloud Monitoring now lets you build more advanced custom dashboards.

Extensions for connectivity and new data types now available in Cloud SQL for PostgreSQL - Check out new extensions and data types in open source database PostgreSQL, and why they matter when you’re using our Cloud SQL managed service.

Introducing the new Google Maps Platform Public Status Dashboard - The Public Status Dashboard provides status information on the products that are generally available and covered by the Google Maps Platform SLA.

Dataproc Metastore: Fully managed Hive metastore now in public preview - Dataproc Metastore lets you use your Apache Hive metastore to simplify technical metadata management when you’re building a data lake on Google Cloud.

Automatically right-size Spanner instances with the new Autoscaler - New Autoscaler lets you scale Spanner instances up and down easily to optimize costs and usage based on utilization.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Protecting your Kubernetes deployments with Policy Controller - In November, the Kubernetes project disclosed a vulnerability which every Kubernetes administrator or adopter should be aware of. The vulnerability, known as CVE-2020-8554, stems from default permissions allowing users to create objects that could act as a “Man in the Middle” and therefore potentially intercept sensitive data.

Migrating apps to containers? Why Migrate for Anthos is your best bet - Learn how Migrate for Anthos speeds up and improves the process of migrating apps from VMs to containers.

Where is my GKE master? - Different ways to access your GKE cluster's control plane.

Google cloud -Public GKE cluster’s egress traffic via Cloud NAT for IP whitelisting - Rerouting GKE egress traffic via Cloud NAT.

GCP IAM Authentication and Authorization 101 - Using IAM and RBAC in GKE cluster.

GCP IAM roles explained - When to use basic vs predefined vs custom roles.

Google Cloud Authentication by Example - Different ways to authenticate to Google Cloud on workstation.

Billing reports and Saved Cost Views - Try out Saved Cost Views to create and share custom views from the Google Cloud billing reports page.

App Development, Serverless, Databases, DevOps

Book - Building Serverless Applications with Google Cloud Run

How I design 12-factor cloud native app on GCP. - Discussion and implementing 12 factor app on GCP.

Basic CI/CD on Google Cloud Platform using Cloud Build - A walk through a simple web-app CI/CD pipeline set up on Google Cloud Platform.

Cloud Functions to Cloud Run - Migrating Cloud Function to Cloud Run.

The serverless gambit: Building ChessMsgs.com on Cloud Run - Building a web-based Chess service completely serverless (and stateless!) utilizing multiple open source projects and Node/Express. Deployed on Cloud Run using Cloud Build.

Deploying to serverless platforms with GitHub Actions - GitHub Actions aim to reduce operational overhead by abstracting away test infrastructure and creating a developer-centric CI/CD system. Learn how you can deploy Google Cloud serverless options using integrated GitHub Actions.

Our way of dealing with more than 2 billion records in the SQL database - Improving performance on a big MySQL table with GCP products.

Day #8 with Cloud Workflows: calling an HTTP endpoint - Continuation of Workflows series. This week also part #9 and #10 on the same website.

Big Data, Analytics, ML&AI

Create a secure and code-free data pipeline in minutes using Cloud Data Fusion - Learn how to set up a secure, no-code data pipeline and see how you can move data easily and anonymize it in your cloud data warehouse.

How to compute a growth rate in BigQuery using SQL - Analyzing Google Analytics public dataset with BigQuery to obtain various data.

Investigate BigQuery slowness with Python and INFORMATION_SCHEMA - Investigation of slot resource contention in BigQuery jobs.

Create a real time Dashboard on covid-19 in France with GCP - Using public API to create Covid dashboard in Data Studio.

Monitor your BigQuery Data Warehouse Dataprep Pipeline with Data Studio - Building a simple, flexible monitoring solution for Cloud Dataprep jobs, with Cloud Functions, BigQuery and Data Studio.

Most popular public datasets to enrich your BigQuery analyses - Free public datasets from Google Cloud, available to help you get started easily with big data analytics in BigQuery and Cloud Storage.

How to automatically scale your machine learning predictions - How to automatically scale machine learning predictions with Google Cloud AI Platform Prediction Service.

Baking recipes made by AI - In this post, we’ll show you how to build an explainable machine learning model that analyzes baking recipes, and we’ll even use it to come up with our own, new recipes—no data science expertise required.

A machine learning pipeline with TensorFlow Estimators and Google Cloud Platform - TensorFlow on GCP — a way to industrialise complex machine learning pipelines.

Customer segmentation with BigQuery ML: unsupervised learning without Python! - Using standard SQL to perform customer clusterization based on clickstream behavior.

Getting started with Machine Learning on GCP — Part 2: Making data clean and usable - Creating Beam/Dataflow pipeline in Jupyter Notebook.

Various

Continuing our long-standing commitment to Kubernetes and CNCF - With $3 million per year in Google Cloud credits, CNCF can continue to host, develop and test the Kubernetes open-source project.

Rodan + Fields achieve business continuity for retail workloads with SAP on Google Cloud - Learn how Rodan + Fields designed and implemented a cloud-native, automated resilience strategy for their SAP workloads on Google Cloud.

Our heads are in the clouds at the Museum of Borgmon Modern Art - Google’s SRE team sees the strangest things in their alerting system, now gathered into the Museum of Modern Borgmon Art.

Slides, Videos, Audio

Kubernetes Podcast - #132 Akri, with Kate Goldenring.

Running Kotlin on Google Cloud

Intro to Kubeflow Pipelines on Google Cloud Platform - This is an introductory look into deploying and creating Kubeflow Pipelines on Google Cloud's AI Platform.

Releases

AI Platform - Deep Learning VMs - M60 release Added TF 2.4.0 Deep Learning VM Images Added TF 2.4.0 Deep Learning Containers.

Google Cloud Armor - Google Cloud Armor Managed Protection Plus tier is now available in public preview. Third-party named IP address lists are now in general availability. The following new WAF rules have been added in general availability: Method enforcement Scanner detection Protocol attack PHP injection attack Session fixation.

BigQuery Transfer - BigQuery Data Transfer Service is now fully integrated with VPC Service Controls, and can be protected using a service perimeter.

BigTable - Key Visualizer diagnostic messages are visible to all Cloud Bigtable customers.

Billing - Start using the Reports page and Cost Table in the Cloud Console for product-level cost details or subaccounts Beginning with your January 2021 invoice or statement (available in February 2021), to simplify the format, we are removing all cost details from your invoice and statement documents, including product-level costs and costs by subaccounts (for Resellers). Recommendations for Compute Engine committed use discounts are now Generally Available. Cloud Billing Reports page now allows you to save your report views.

CDN - Cache modes, TTL overrides and custom response headers are now supported on backend buckets and backend services, and are now Generally Available.

Cloud Build - Users can now create manual triggers to run builds at a specified time.

Cloud Composer - Preview: A new Logs tab has been added to the Environment details page. New versions of Cloud Composer images: composer-1.13.3-airflow-1.10.9, composer-1.13.3-airflow-1.10.10, and composer-1.13.3-airflow-1.10.12. Composer will now fail faster when the network settings in Private IP environments prohibit the download of publicly stored Python packages. Composer Agent error messages are now more descriptive.

Compute Engine - The m1-node-96-1433 sole-tenant node type is now Generally Available. Compute-optimized (C2) machines are now available in Montréal, in all three zones , northamerica-northeast1-a,b,c. Preview: Accelerator-optimized (A2) machine types are now available in the following three regions: Iowa, North America: us-central1-a,c Netherlands, Europe: europe-west4-a,b Singapore, APAC: asia-southeast1-c. Preview: NVIDIA® A100 GPUs are now available in the following three regions: Iowa, North America: us-central1-a,c Netherlands, Europe: europe-west4-a,b Singapore, APAC: asia-southeast1-c For more information, see GPUs on Compute Engine.

Dataproc - Announcing the General Availability (GA) release of CentOS 8 based 1.5 images. New sub-minor versions of Dataproc images: 1.3.79-debian10, 1.3.79-ubuntu18, 1.4.50-debian10, 1.4.50-ubuntu18, 1.5.25-centos8, 1.5.25-debian10, 1.5.25-ubuntu18, 2.0.0-RC21-debian10, and 2.0.0-RC21-ubuntu18. Image 2.0 preview: Changed the default value of Spark SQL property spark.sql.autoBroadcastJoinThreshold to 0.75% of executor memory. Image 1.4-1.5: Fixed a NullPointerException in a primary worker shuffle when the BypassMergeSortShuffleWriter is used when some output partitions are empty. Images 1.5-2.0 preview: Fixed ZOOKEEPER-1936: Server exits when unable to create data directory due to race condition. Fixed a bug where Dataproc agent logs had separate entries for exception stack trace in StackDriver. Announcing the Beta release of the Dataproc cluster Stop/Start. Announcing the General Availability (GA) release of the Dataproc Workflow Timeout feature, which allows users to set a timeout on their graph of jobs and automatically cancel their workflow after a specified period.

Dialogflow - GA (general availability) launch of Dialogflow CX. CX Regionalization expanded to multiple regions globally. CX Analytics for agent activity statistics. CX Prebuilt agents for common agent use cases. CX Customer-managed encryption keys (CMEK) to manage your own Dialogflow data encryption keys. CX Security settings to control data redaction and data retention. CX DTMF input for telephony partner integrations. CX Parameter redaction to redact end-user parameter data from logs.

IAM - You can now attach service accounts to resources in other projects. You can now use Cloud Monitoring to check when your service accounts and service account keys were used.

Cloud Logging - Logs regionalization is now generally available.

Cloud Memorystore - Added support for TLS encryption on Memorystore for Redis.

Cloud Monitoring - The dashboard editor that lets you create and edit all dashboard widget types, including gauges, scorecards, and text boxes, is now Generally Available.

Cloud NAT - The ability to enable or disable Endpoint-Independent Mapping for your gateway is available in General Availability.

Cloud Run - Cloud Run container instances can now process up to 250 concurrent requests, see Configuring maximum concurrency.

Cloud Spanner - A new multi-region instance configuration is now available in Europe - eur6 (Netherlands/Frankfurt/Zurich). A new multi-region instance configuration is now available in North America - nam12 (Iowa/Northern Virginia/Oregon/Oklahoma).

Cloud SQL MySQL - In Cloud SQL for MySQL, parallel replication is generally available for improving replication performance.

Cloud SQL Postgres - Cloud SQL has expanded support for PostgreSQL extensions. The following PostgreSQL minor versions have been upgraded: PostgreSQL 9.6.18 is upgraded to 9.6.19. Cloud SQL for PostgreSQL now supports the effective_cache_size flag.

Cloud Tasks - The relationship between your queues and your App Engine app has changed.

Cloud TPU - Cloud TPU now supports Shared VPC Shared VPC allows an organization to connect resources from multiple projects to a common VPC network to communicate with each other securely and efficiently using internal IPs from that network.

Cloud Vision API - OCR On-Prem General Availability (GA) release OCR On-Prem is now generally available for approved customers.

Virtual Private Cloud - Access to Google APIs and services using Private Service Connect is now available in Preview. DNS peering for private services access is now available in General Availability. The ability to connect VM interfaces other than nic0 to a Shared VPC is now available in Preview.

VPC Service Controls - Preview support for the following integration: Dataproc Metastore.

AI Platform Prediction - You can now configure AI Platform Prediction to automatically scale prediction nodes for model versions that use GPUs for online prediction.

Deep Learning VM - M60 release Added TF 2.4.0 Deep Learning VM Images Added TF 2.4.0 Deep Learning Containers.

Dialogflow Enterprise - GA (general availability) launch of Dialogflow CX. CX Regionalization expanded to multiple regions globally. CX Analytics for agent activity statistics. CX Prebuilt agents for common agent use cases. CX Customer-managed encryption keys (CMEK) to manage your own Dialogflow data encryption keys. CX Security settings to control data redaction and data retention. CX DTMF input for telephony partner integrations. CX Parameter redaction to redact end-user parameter data from logs.

Service Mesh - 1.8.x. 1.8.1-asm.5 is now available. Multi-cluster support for GKE on-prem Beta Anthos Service Mesh now supports multi-cluster meshes when running on GKE on-prem. New flags for the install_asm script The install_asm script was enhanced to provide you with more granular control over the changes that the script makes on your project and GKE on Google Cloud cluster. Third-party add-ons removed from all profiles The Prometheus, Grafana, and Kiali add-ons were removed from all Anthos Service Mesh profiles. Anthos Service Mesh 1.8 isn't supported on Anthos attached clusters and GKE on AWS Anthos Service Mesh 1.8 currently isn't supported on Anthos attached clusters (Microsoft AKS and Amazon EKS) and GKE on AWS (Amazon EC2). Reduced permissions required for installation The permissions required for installation have been scaled back.

Anthos GKE on AWS - GKE on AWS 1.6.0-gke.3 is now available. GKE on AWS now supports Kubernetes 1.18. GKE on AWS now supports mounting AWS Elastic File System file systems without having to install a driver. You can now specify an AWS KMS alias in your anthos-gke.yaml instead of a KMS ARN. You can now use custom DNS hostnames in your VPC by setting enableDnsHostnames to false. Cluster state synchronizations between the management service and S3 now use HTTPS.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko