Welcome to issue #228 February 8th, 2021

News

Search and browse Google Cloud code samples - New features in documentation to provide quick and easy ways to search and browse all code samples that are available for Google Cloud.

The time for digital excellence is here—Introducing Apigee X - Apigee X, the new version of Google Cloud’s API management platform, helps enterprises accelerate from digital transformation to digital excellence.

Introducing real-time data integration for BigQuery with Cloud Data Fusion - See how Cloud Data Fusion makes it easy to replicate and integrate data in real-time, low-latency pipelines from cloud databases into cloud data warehouse BigQuery.

The Dunant subsea cable, connecting the US and mainland Europe, is ready for service - The Dunant submarine cable system, crossing the Atlantic Ocean between Virginia Beach in the U.S. and Saint-Hilaire-de-Riez on the French Atlantic coast, has been deployed and tested and is now ready for service.

Introducing HPC VM images—pre-tuned for optimal performance - Google Cloud’s first pre-configured HPC VM image is a CentOS 7-based image optimized for tightly-coupled MPI workloads.

Cloud Firestore for Games is now in Beta! - Cloud Firestore for Games is now publicly available in beta for C++ and Unity developers.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

What are my hybrid and multicloud deployment options with Anthos? - Anthos is a managed application platform that extends Google Cloud services and engineering practices to your environments so you can modernize apps faster and establish operational consistency across them. Anthos offers multiple deployment options to choose from, depending on where your infrastructure and applications live today. In this article we are outlining the Anthos deployment options.

Give app teams autonomy over their DNS records with Cloud DNS peering - This solution based on Cloud DNS peering allows application owners to manage their own DNS records, even in a Shared VPC environment.

The cloud trust paradox: 3 scenarios where keeping encryption keys off the cloud may be necessary - Although rare, there are sometimes situations where encryption keys should be stored off the cloud. Here are three to consider.

Set up Anthos Service Mesh for multiple GKE clusters using Terraform - Learn how to federate multiple GKE clusters with Anthos Service Mesh 1.7 and 1.8, creating a managed Istio configuration in a single VPC.

Limiting public IPs on Google Cloud - Learn how to restrict public IP creation for Google Cloud resources, including VMs, load balancers, and VPN gateways using organization policies.

Deploying OPA on a GKE cluster - Deploying OPA (Open Policy Agent) on a GKE cluster and set some policies to allow user to deploy images only from a specified registry.

BackendConfig — SessionAffinity — GKE Workloads - BackendConfig — SessionAffinity — GKE Ingress with NEG.

Exploring GKE Private Cluster with CloudSQL through Cloud SQL Proxy - Exploring GKE Private Cluster secure connection CloudSQL through Cloud SQL Proxy.

Centrally Managing Artifact Registry Container Image Vulnerabilities on Google Cloud: Part One - Utilizing Pub/Sub and Cloud Functions to store project level container image vulnerabilities in a centralized service or location, concretely Cloud Storage.

Assess the security of Google Kubernetes Engine (GKE) with InSpec for GCP - You can validate the security posture of your GKE clusters using Chef InSpec™ by assessing their compliance against the Center for Internet Security (CIS) 1.1.0 benchmark for GKE.

How to allow access to Google Services (APIs) if deny all egress rule is created? - Using Cloud Function to update netblocks in firewall rules.

Simulating DNS ALIAS/ANAME records on Google Cloud DNS - Using Cloud DNS with Heroku.

De-identifying Sensitive Information in Salesforce with Google Cloud Data Loss Prevention - How to move from detection to prevention — de-identifying sensitive data at scale.

Iris 3: Automatic labeling for cost control - Major improvements for your favorite Google Cloud resource labeller — including PubSub and CloudSQL support.

Budgets: Setting up a Budget - Overview of budgets on GCP and how to set them.

App Development, Serverless, Databases, DevOps

How to optimize your network for live video on Google Cloud - The response to COVID-19 has required broadcasters to come up with creative and ad-hoc solutions to keep shows on the air. This post provides our recommendations for using Google Cloud networking solutions for live TV operations.

Content Centric Software-as-a-Service Solution using Google Cloud Platform - Lessons learned about how to build a multi-tenant, content-centric SaaS solution utilizing Google Cloud native features with a focus on performance, cost, operations, and security.

The Misadventures of One Cloud Function - Setting a service account for multiple Cloud Functions in GCP project using Terraform.

Triggering (Almost) any Event in Cloud Run using Eventarc - Use CloudRun and Eventarc to monitor and automate actions on your GCP projects.

Creating AI Web Apps using TensorFlow, Google Cloud Platform, and Firebase - From the Sandbox to the Cloud: How to Easily Deploy your Machine Learning Models into Production Web Applications.

Google Cloud Scheduler: serveless CRON jobs with Python - How to I use Google Cloud Scheduler and Python to create a scheduled job such as a bot that posts everyday on Twitter ‘Have a nice day’.

How to Build a Commerce Chatbot with DialogFlow CX & commercetools (Step By Step) - Guide to Building a Chatbot that Performs 3 Simple Functions.

You don’t need a Dockerfile to build a Go Container - Using buildpacks to build Go applications into containers.

How I spend $500 per day because of a misconfiguration - Sharing so that others don't do similar mistake.

Day #13 with Cloud Workflows: Logging with Cloud Logging - Writing entries into Cloud Logging from Workflows.

Day #14 with Cloud Workflows: Subworkflows - Example of using sub workflows in Workflows.

Big Data, Analytics, ML&AI

GeoBeam - Use DataFlow to move shp and gdb files to BigQuery GIS, an also process geotiffs in flight and write results to BigQuery.

Composer, Dataflow and Private IP addresses - Invoking Dataflow jobs with private IP from Composer (Airflow).

Inner workings of Harness’s Cloud Billing Data Ingestion Pipeline - Data pipeline to consume billing data on GCP.

Setting Up Data Quality Monitoring For Cloud Dataprep Pipelines - A simple, flexible, comprehensive Data Quality monitoring tool for Cloud Dataprep pipelines with Cloud Function, BigQuery, and Data Studio.

A Simple Way to Query Table Metadata in Google BigQuery - Effortless approach to determine what is in the BigQuery dataset and which tables are useful for analysis with INFORMATION_SCHEMA and TABLES.

How to calculate Real Active Users. What are the numbers? - A complete SQL guide for marketers and machine learning engineers. MAU, DAU and WAU, Firebase and BigQuery example with Data Studio template.

Can machine learning make you a better athlete? - How to use computer vision, posture tracking, and math to improve your tennis serve, soccer kick, and more.

Google Cloud AI leaders share tips for getting started with AI - Google Cloud AI leaders share their perspective on how businesses can get started in AI.

Continuous model evaluation with BigQuery ML, Stored Procedures, and Cloud Scheduler - Continuous evaluation—the process of ensuring a production machine learning model is still performing well on new data—is an essential part in any ML workflow. Performing continuous evaluation can help you catch model drift, a phenomenon that occurs when the data used to train your model no longer reflects the current environment.

BigQuery Anomaly Detection using Kmeans Clustering from BQ ML - Find rogue transactions the smart way using BigQuery ML.

Using BigQuery Arrays to Analyze my Netflix Viewing History - How to do Advanced Data Cleaning in SQL.

How to develop with PyTorch at lightning speed - Increase your productivity using PyTorch Lightning, a popular wrapper, and Google Cloud Platform.

How to Translate and Dub Videos with Machine Learning - Use speech-to-text, translation, and text-to-speech to automatically translate and dub videos.

Various

Avoid cost overruns: How to manage your quotas programmatically - Managing and monitoring the quotas you set on cloud resources can help you avoid unexpected costs.

A primer on Cloud Bigtable cost optimization - Check out how to understand resources that contribute to costs and how to think about cost optimization for the Cloud Bigtable database.

Google Cloud is a sad mess - Sharing some pains of using GCP.

7 Free Google Cloud DevOps Engineer Certification Courses - A collection of free Udemy and Coursera courses to learn Cloud Computing and DevOps and prepare for Google Cloud DevOps Engineer certification.

10 Best & Free Google Cloud Professional Architect Certification Courses & Practice Tests - A collection of free and Best online courses and practice tests from Udemy and Coursera to prepare for the Google Cloud Professional Cloud Architect certification.

Slides, Videos, Audio

GCP Podcast - #245 Botcopy with Dustin Dye and Alex Seegers.

Kubernetes Podcast - #136 Backstage, with Lee Mills and Matt Clarke.

Releases

AppEngine Standard Java - Removed data logging in the deprecated endpoints library. Updated Java SDK to version 1.9.85.

BigQuery - Updated version of Magnitude Simba JDBC driver includes bug and security fixes and enhancements for additional DDL keywords and dynamic SQL.

Billing - PayPal now available as a form of payment in many countries If you have an online, auto-pay Cloud Billing account, you might be able to add PayPal as a form of payment on that account. Invoices are now simpler, providing only your cost totals. Cloud Billing Budgets now shows your historic cost trends when you're planning your budget.

Cloud Build - You can now configure Cloud Build to access secrets from Secret Manager via environment variables.

Cloud Composer - Timeouts for environment upgrade operations are increased. Fix for CVE-2021-3156 vulnerability. On a failed environment upgrade operation, the created CloudSQL database is now correctly rolled back. Create and update operations for environments no longer fail if your account doesn't have the serviceusage.services.get permission. Fixed SQL operation conflicts that were occurring during environment upgrade operations. Upgrade operations that might have resulted in a semi-upgraded environment state when the operation timeout was reached are now correctly rolled back and errors are reported. You can now enable and disable RBAC in environments with installed custom PyPI packages. New versions of Cloud Composer images: composer-1.14.2-airflow-1.10.14 composer-1.14.2-airflow-1.10.12 (default) composer-1.14.2-airflow-1.10.10.

Compute Engine - Generally Available: Sole-tenant nodes now support GPUs and local SSDs. Generally Available: Specify when maintenance begins on VMs in a sole-tenant node group. NVIDIA® T4 GPUs are now available in the following additional regions and zones: Jakarta, Indonesia, APAC: asia-southeast2-a,b For more information about using GPUs on Compute Engine, see GPUs on Compute Engine. Preview: You can now use schedule-based autoscaling from the Google Cloud Console. N2D machine types are now available in London, zone europe-west2-c. You can now create instances with up to 24 local SSD partitions for 9 TB of local SSD space using N1, N2, and N2D machine types. Preview: You can now create virtual machines for high performance computing (HPC) workloads using the HPC VM image.

Container Registry - On-Demand Scanning is available in Preview.

Data Fusion - Preview: You can now replicate data continuously and in real time from operational data stores, such as SQL Server and MySQL, into BigQuery.

Dataproc - Dataproc 2.0 image version will become a default Dataproc image version in 3 weeks on February 22, 2021.

Dialogflow - Several new Dialogflow CX prebuilt agents have been launched. Dialogflow CX conditions now support sentiment.

Cloud Functions - There is now a security level feature for HTTP functions that controls whether the function's URL supports HTTPS only, or both HTTP and HTTPS.

Cloud Healthcare API - v1. The Quickstart using curl or Windows PowerShell has been updated with additional information on storing and viewing DICOM, FHIR, and HL7v2 data.

Load Balancing - Identity-Aware Proxy (IAP) is supported with Internal HTTP(S) Load Balancing. For internal TCP/UDP load balancers, you can create multiple forwarding rules with the same IP address.

Anthos Migrate - On February 01, 2021, we released Migrate for Anthos 1.6.1. Released a fix, rolling out gradually and taking full effect 2/5/2021, for a migctl setup installation that fails on a GKE cluster when the automatically created bucket already exists. Released a fix, rolling out gradually and taking full effect 2/5/2021, for a migctl crash when kubectl is not in PATH.

Cloud Run for Anthos - Cloud Run for Anthos on Google Cloud version 0.19.0-gke.1 is now available for the following GKE minor versions: 1.18 1.19 1.20.

Cloud Scheduler - Two new headers, X-CloudScheduler-JobName and X-CloudScheduler-ScheduleTime, have been added to the default headers for AppEngineHttpTarget and HttpTarget.

Security Command Center - Security Command Center's v1 API now includes a Severity field for Findings. Event Threat Detection, a built-in service of Security Command Center Premium, has launched previews for two new detectors. Documentation New documentation includes guides on enabling real-time email and chat notifications and investigating and responding to threats.

Dialogflow Enterprise - Several new Dialogflow CX prebuilt agents have been launched. Dialogflow CX conditions now support sentiment.

Service Mesh - 1.8.x. 1.8.2-asm.2 is now available. The install_asm script lets you reinstall the same version You can use the install_asm script when you need to reinstall the same Anthos Service Mesh version to change the control plane configuration.

AI Platform Training - You can now use E2, N2, and C2 machine types for training.

Artifact Registry - On-Demand Scanning is available in Preview.

Anthos GKE on AWS - GKE on AWS 1.6.1-gke.2 is now available. Snapshots now collect AWS EFS logs from user cluster nodes. Bug fixes and performance improvements.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko