Welcome to issue #212 October 19th, 2020

News

What’s happening in BigQuery: Time unit partitioning, Table ACLs and more - Check out new cloud data warehouse features from BigQuery, such as time unit partitioning, table ACLs, and expanded metadata access.

Smile with new user-friendly SQL capabilities in BigQuery - New commands, functions, scripting, and SQL language improvements in BigQuery.

Prevent planned downtime during the holiday shopping season with Cloud SQL - New maintenance deny periods for Cloud SQL let you choose when downtime occurs for database maintenance--especially useful for retailers during shopping season.

New Dataproc optional components support Apache Flink and Docker - Run native Apache Spark and Hadoop clusters on Dataproc fast and cost-effectively. New optional components for Docker and Flink available.

Cache is king: Announcing lower pricing for Cloud CDN - We’ve simplified and lowered pricing for Cloud CDN by reducing cache fill charges.

Democratizing Zero Trust with an expanded BeyondCorp Alliance - Last year, we announced our BeyondCorp Alliance with partners that share our Zero Trust vision, and seamlessly extend our platforms by adding key functionality and intelligence. Today, we’re announcing new partners to this alliance.

Learn at no cost how to get insights from your data, regardless of your analytics experience - How Google Cloud helps IT practitioners of all skill levels, from beginners to experts, to build their data analytics skills.

And then there were two: Simplifying our product launch stages - Google Cloud now has just two launch stages: Preview and General Availability.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Kubernetes on GCP: Simplicity vs. Flexiblity - Exploring different ways of running Kubernetes on Google Cloud Platform (GCP).

Exponential growth in DDoS attack volumes - How Google prepares for and protects against the largest volumetric DDoS attacks.

Cloud Code makes YAML easy for hundreds of popular Kubernetes CRDs - Cloud Code makes working with Kubernetes YAML easy thanks to expanded support for CRDs.

Rapid cloud foundation buildout and workload deployment using Terraform - Learn how to deploy a microservices app to Google Cloud with Terraform and the Cloud Foundation Toolkit.

Patching Windows VMs with GCP’s VM Manager - Using OS patch management for Compute Engine to update instances running Windows.

Service Account Credentials API: A solution to different issues - To avoid service account key file isn’t easy. Hopefully, a poorly known API can help you in several use cases.

Google Config Connector — My way to CNCF - Google config connector helps users manage GCP resources in a Kubernetes-style way.

Google Config Sync — My choice for CI/CD - If you’re looking for a CI/CD tool fits GCP well, Config Sync could be the one.

Continuous compliance testing using InSpec on Google Cloud Platform - Integrating compliance testing as part of an automated infrastructure pipeline and visually evidencing the results in real time.

A Safety Net for Terraform in Google Cloud Build - Handling timeout in Cloud Build when using Terraform.

App Development, Serverless, Databases, DevOps

Using OAuth 2.0 to authenticate server Applications against Google CServices - Authenticating Service Accounts with JWT in Powershell and C#.

Scaling Cloud Spanner Instances - Best practices for scaling up and down Cloud Spanner.

How to Setup a Complete Development Environment in the Cloud using Google Cloud Shell & VSCode - Set up your own development environment in the cloud using Google Cloud Shell.

Google Firebase Authentication Vulnerability - The use case of a brute attack in email/password Firebase Auth.

GCP Cloud NAT + Golang’s HTTP client = - Why did it stop after 64 request?

Four reasons that Google Cloud Run is better than traditional FaaS offerings - What are the advantages of Cloud Run in comparison with other FaaS.

container-instance-metadata-server - The container-instance-metadata-server emulates the Cloud Run container instance metadata server for a given service account and user supplied metadata.

Big Data, Analytics, ML&AI

Best practises for KubernetesPodOperator in Cloud Composer - Examples and best practices on using KubernetesPodOperator in Cloud Composer.

BigQuery explained: Blog series - Find links to all posts in the BigQuery Explained series.

Explore Public Datasets with Google BigQuery and DataStudio - Exploring and Reporting Massive Datasets Right Inside Your Web-browser — With an example of COVID-19 Dataset.

New options for BigQuery GIS geospatial data ingestion - Introducing ST_GeogFromText and ST_GeogFromGeoJson functions to convert geospatial data in BigQuery.

Inject your SpringBoot app data in Google BigQuery - Inserting data into BigQuery from Java SpringBoot application.

How to create and deploy a model card in the cloud with Scikit-Learn - How to create and deploy a model card in the cloud with Scikit-Learn.

Explore & Visualize 200+ Years of Global Temperature Using Apache Spark, BigQuery, and Google Data Studio - Visualize observable changes in global temperature using NOAA’s historical weather data.

How To Test GCP Dataflow Pipeline - An Example with Java SDK and Apache Beam Programming Model.

Tezos Public Finance Dataset Integrated into Google BigQuery - Tezos dataset is integrated into Google BigQuery!

How to parse forms using Google Cloud Document AI - A step-by-step guide to extracting structured data from paper forms.

How to create a concise image representation using machine learning - Designing and training an autoencoder on HRRR images in Keras.

How we use Supermetrics & Google BigQuery at Supermetrics - Description of how Supemetrics is using BigQuery.

Various

How To Pass Google Cloud Professional Data Engineer Exam without IT background. - Passing Data Engineer certification exam with non-IT background.

How to pass a GCP certification? Cloud Architect & Data Engineer edition - Preparation for Cloud Architect and Data Engineer exams.

Slides, Videos, Audio

GCP Podcast - #240 reCAPTCHA Enterprise with Kelly Anderson + Spring ML Potholes with Eric Clark.

Kubernetes Podcast - #125 Okteto, with Ramiro Berrelleza.

Where Serverless meets Containers - Kelsey Hightower on Cloud Run - Caught in the Serverless vs. Kubernetes debate? Then, you’re in for a treat because at the last #ServerlessTO meetup, Kelsey explained how Serverless and Containers Technologies are converging – not diverging.

Releases

Compute Engine - Support for 1500 MTU in VPC networks is now Generally available. Compute-optimized (C2) machine types are now available in the following regions and zones: Finland: europe-north1-a,b,c Seoul: asia-northeast3-a,b,c See VM-instance-pricing for details. N2 machine types are now available in the following four regions and zones: Las Vegas: us-west4-a,b,c Montréal: northamerica-northeast1-a,b,c Finland: europe-north1-a Hong Kong: asia-east2-a,b,c For pricing details, see VM instance pricing.

Config Connector - Support export sub-command in the config-connector CLI. Add support for the AccessContextManagerServicePerimeter resource. Add support for Folder-level IAM Audit Configs. Fix deadLetterTopicRef in the PubSubSubscription resource (Issue #281).

Dataproc - Announcing the GA (General Availability) release of the Dataproc - Docker Optional Component and the Dataproc - Flink Optional Component. New sub-minor versions of Dataproc images: 1.3.72-debian10, 1.3.72-ubuntu18, 1.4.43-debian10, 1.4.43-ubuntu18, 1.5.18-debian10, 1.5.18-ubuntu18, 2.0.0-RC14-debian10, and 2.0.0-RC14-ubuntu18.

Cloud Functions - In runtimes that use buildpacks you can now configure aspects of your build by setting build configuration variables.

IAM - Credential Access Boundaries are now generally available. If a role binding in an IAM policy refers to a deleted member (for example, deleted:user:tamika@example.com?uid=123456789012345678901), you can now add role bindings for a newly created member with the same name (in this case, user:tamika@example.com).

Identity Platform - Sign in with Apple is now supported.

Cloud Logging - We've renamed the Logs Viewer (Preview) to the Logs Explorer. Cloud Logging has stopped populating the following two logs-based metrics related to exclusions: logging.googleapis.com/excluded_log_entry_count logging.googleapis.com/excluded_byte_count.

Resource Manager - You can now customize who receives notifications from GCP with Essential Contacts.

Cloud Run for Anthos - Cloud Run for Anthos on Google Cloud version 0.17.2-gke.1 is now available for the following GKE minor version: 1.16 Fixes the security issue, ISTIO-SECURITY-2020-010 for Cloud Run for Anthos on Google Cloud clusters running on 1.15+ k8s version.

Cloud Run - You can now specify a minimum number of container instances to be kept warm and ready to serve requests, for services requiring reduced latency and fewer cold starts. You can now control egress traffic from a service and route all outbound requests to your VPC network. You can now allocate 4 vCPUs to container instances of Cloud Run services.

Cloud Spanner - A new multi-region instance configuration is now available in North America - nam9 (North Virginia/Iowa/South Carolina/Oregon). CHECK constraints is now generally available, allowing you to define a boolean expression on the columns of a table and require that all rows in the table satisfy the expression. Generated columns support is now generally available, allowing you to define columns that are computed from other columns in a row.

Cloud SQL MySQL - Cloud SQL now offers "deny maintenance periods".

Cloud SQL Postgres - Cloud SQL for PostgreSQL now offers IAM database authentication to help you better monitor and manage access for users and service accounts to databases. Cloud SQL now offers "deny maintenance periods". Database auditing in Cloud SQL for PostgreSQL is available through the open-source pgAudit extension.

Cloud SQL SQL Server - Cloud SQL now offers "deny maintenance periods".

Cloud Storage Transfer - Obtaining the status of the latest transfer operation is in Preview.

Cloud Talent Solution - Cloud Talent Solution has launched the v4 version of the API. As of today Cloud Talent Solution versions v3, v3p1beta1, and v4beta1 are deprecated. When using orderBy to order job search results by distance_from from the search location, equidistant jobs from the center of the search location will be tie-broken based on each job's relevance to the search keywords. When using the EmploymentType as part of HistogramQuery, facet counts for CONTRACTOR no longer also include facet counts for CONTRACT_TO_HIRE. HistogramQuery facet counts no longer differ from the number of jobs returned when filtering search results by a given facet. CTS has made improvements to the handling of accented characters and gendered terms in job titles and search keywords.

Cloud Vision API - LABEL_DETECTION model upgrade The LABEL_DETECTION model will undergo an upgrade over the next 90 days to a newer version.

Virtual Private Cloud - Support for 1500 MTU in VPC networks is now available in General Availability.

Migrate for Compute Engine 4.8 - 4.11. Support added for migration of VMs from vSphere configured with CSM firmware type setting.

Cloud VPN - Classic VPN partial deprecation Starting on October 31, 2021, you will no longer be able to do the following: Create new Classic VPN tunnels using static routing (route based or policy based) that connect to another Classic VPN gateway Create new Classic VPN tunnels using static routing (route based or policy based) that connect a Google Cloud Virtual Private Cloud (VPC) network to another cloud provider's network Create new Classic VPN tunnels using dynamic routing (all configurations) You can continue to create the following types of connections and get support for them: VPN tunnels using static routing from Classic VPN gateways to on-premises VPN gateways and from on-premises VPN gateways to Classic VPN gateways VPN tunnels using static routing from a Classic VPN gateway to and from a Compute Engine virtual machine (VM) acting as a VPN gateway Although Google will not proactively disable existing connections on the deprecation date, deprecated Classic VPN configurations will no longer receive regular updates or maintenance.

Service Mesh - 1.4.x. 1.4.10-asm.19 is now available. You can now allow an experimental feature to exceed 4GB of memory usage.

Document AI - v1beta3. Document AI Preview released The following beta and preview features are available in API version v1beta3: General processors: Document OCR (Optical Character Recognition), form parser, and document splitter.

Anthos GKE on AWS - GKE on AWS 1.5.0 supports volume snapshots.

BigQuery - Dynamic SQL is now generally available (GA). BigQuery standard SQL now supports the following new functions. BigQuery now supports the following new statements. BigQuery standard SQL now supports DATE arithmetics operators. The following INFORMATION_SCHEMA views are now generally available (GA). BigQuery now supports Unicode table names. Queries can now have duplicate column names.

BigTable - A tutorial is now available that demonstrates how to send a Cloud Bigtable read request using a Cloud Functions HTTP(S) request. The steps to create a new Cloud Bigtable instance and edit an existing instance have been streamlined and improved in the Google Cloud Console.

Billing - Discount sharing for committed use discounts is now Generally Available.

Cloud Composer - New versions of Cloud Composer images: composer-1.12.3-airflow-1.10.6, composer-1.12.3-airflow-1.10.9, and composer-1.12.3-airflow-1.10.10. Cloud Build logs from the tenant project are now published in the Composer logs. Setting or updating the machine type of the Airflow web server or Cloud SQL instance in Composer versions that don't support this feature (older than composer-1.7.2) will now return an error instead of failing silently.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko