Welcome to issue #289 April 11th, 2022

News

BigLake: unifying data lakes and data warehouses across clouds - BigLake unifies data warehouses and data lakes into a consistent format for faster data analytics across Google Cloud and open source formats and engines.

Announcing Serverless Spark components for Vertex AI Pipelines - You can use Vertex AI Pipelines to automate ML workflows in conjunction with Dataproc for running serverless Spark workloads!

Securely exchange data and analytics assets at scale with Analytics Hub, now available in Preview - Efficiently and securely exchange valuable data and analytics assets across organizational boundaries with Analytics Hub. Start your free trial today.

Database observability for developers: announcing Cloud SQL Insights for MySQL (in preview) - Learn how Cloud SQL Insights for MySQL provides database observability for developers.

Now in preview, BigQuery search features provide a simple way to pinpoint unique elements in data of any size - Now in preview, BigQuery search features provide a simple way to pinpoint unique elements in data of any size.

Introducing Serverless Spark for interactive development - Integrations with Vertex AI, BigQuery, and Dataproc for GKE make Serverless Spark ready for interactive development.

How to migrate from Apache HBase to Cloud Bigtable with Live Migrations - Making it easier to migrate from HBase to Bigtable—a fully managed service that is compatible with the open source HBase API.

Bringing together the best of both sides of BI with Looker and Data Studio - Get the self-serve speed you need with the certainty of central BI by integrating Looker and Data Studio.

Boost the power of your transactional data with Cloud Spanner change streams - Change streams track changes in your Spanner database and integrate this data with other systems for analytics, event triggering, and compliance.

Modernize your Oracle workloads to PostgreSQL with Database Migration Service, now in preview - Learn how to modernize your Oracle workloads to PostgreSQL with Database Migration Service on Google Cloud, now in preview.

Accelerate your move to the cloud with the new Database Migration Program - Database Migration Program is a new and stress free approach to migrating self managed databases, whether on premises or in the cloud, to industry-leading, managed database services on Google Cloud.

Announcing BigQuery and BigQuery ML operators for Vertex AI Pipelines - Announcing the official release of new BigQuery and BigQueryML components for Vertex AI Pipelines that help make it easier to operationalize BigQuery and BQML jobs in a Vertex AI Pipeline. It also provides an end-to-end example of using Dataflow, BigQuery and BigQuery ML components to predict the topic label of text documents using BQML and Dataflow.

Deliver exception messages through Slack and Webhooks for fast resolution - Error Reporting captures and collates application exceptions from your logs. Now use Slack and webhooks for alert notifications.

Reduce scaling costs by up to 50% in Cloud Spanner with doubled provisioned storage - Spanner has doubled the storage-limit for Spanner instances from 2TB to 4TB per node to help optimize the cost of storage intensive workloads.

Introducing Topaz — the first subsea cable to connect Canada and Asia - The Topaz subsea cable is the first fiber cable to connect Canada and Asia, and will provide better resiliency and lower latency for Google services.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Meet Canadian compliance requirements with Protected B Landing Zone - Google Cloud has published a Terraform-based Infrastructure-as-Code (IaC) template to help customers to ensure the foundational settings, policies, and folder structures are correctly configured in alignment with the Annex 4A - Profile 1 (PBMM and ITSG-33).

Google's subsea fiber optics, explained - Stephanie Wong goes behind the scenes to show you how Google predicts network demand to build its fiber optic network and support Google Cloud.

IaC with GitHub Actions for Google Cloud Platform - Automate resource provisioning in Google Cloud Platform by using Terraform and GitHub actions.

GCP integration with PagerDuty using Terraform - This article will show you, how Storytel 2022 went from a basic setup with a single global on-call team to a Full Service Ownership setup.

An Intermediate-Level Deep Dive into GKE Workload Identity - GKE Workload Identity is very useful, but I didn’t have a concrete idea of exactly how Pods on GKE get Google Cloud authorization, so I look.

Workload Identity Federation for On-Premise Workloads with SPIFFE - Learn how you can combine SPIFFE and Google Workload Identity Federation to securely access Google Cloud APIs from On-Premise.

Google Cloud Anthos Series: Anthos Multi-cloud & Hybrid story - Google Cloud Anthos Series (Deploy Anthos on AWS) : Part 9.

Container-native Load Balancing and How it helped us - Implementing container-native load balancing using zonal network endpoint groups in GKE.

App Development, Serverless, Databases, DevOps

A migration story part 2: How NCR & Opus built better availability and resilience for card management in the cloud - Learn how NCR and Opus combined domain expertise with Google Cloud’s differentiating capabilities to deliver an improved card management solution in the cloud.

How to manage data on Cloud VMs: A conversation - Why VMs are some of Google’s most trusted and reliable offerings, and how VMs benefit companies operating at scale in the cloud.

Google Cloud’s key investment areas to accelerate your database transformation - This blog focuses on the 6 key database investment areas that help you accelerate your digital transformation journey.

Deploy a coloring page generator in minutes with Cloud Run - In this post, you'll see how to create an image processing service and make it available online using minimal resources.

Are you using Cloud Functions for event based processing? - Using Apache Airflow as an alternative for Cloud Functions event processing.

Log-based Alerting in GCP - An overview of Log based alerting.

Workflows state management with Firestore - Using Firestore to save intermediate results between steps in Cloud Workflow.

Help! How do I undelete my deleted Google Cloud SQL instance? - Preventing and handling unwanted deletion of Cloud SQL instance.

Big Data, Analytics, ML&AI

Limitless Data. All Workloads. For Everyone - Read about the newest innovations in data cloud announced at Google Cloud’s Data Cloud Summit.

Meet Google’s unified data and AI offering - By bridging data and AI, data analysts can take advantage of user-friendly ML tools and data scientists get the most out of data. Built-in MLOps ensure AI work is ready for production.

Build a secure data warehouse with the new security blueprint - Introducing our new security blueprint that helps enterprises build a secure data warehouse.

Dataproc Serverless for Spark on GCP - A brief overview of Serverless Spark and how it compares to similar products on Google Cloud.

Enabling data science on Google Cloud Platform at Adevinta - Empowering data scientists to develop an end-to-end machine learning platform on Google Cloud Platform.

RepConnect Monitoring - Setting monitoring for data ingestion pipeline.

Know More About Your App Users Through BigQuery - A more customized approach to event analytics beyond Firebase and Google Analytics.

Data Apps: From Local to Live in 10 Minutes - This post explains how the Talabat Machine Learning Ops team built this simple yet elegant pipeline that brings their Machine Learning models and analyses live in a few minutes with the least possible effort required by Data Scientists.

Price optimization notebook for apparel retail using Google Vertex AI - We demonstrate how to build a custom demand forecasting and price optimization solution on top of Vertex AI, Google Cloud’s data science platform.

MLOps in BigQuery ML with Vertex AI Model Registry - Vertex AI Model Registry enables new capabilities within BigQuery ML. Learn how to streamline model management and easily deploy models for real time predictions.

Various

Train your organization on Google Cloud Skills Boost - To help more than 40 million people build cloud skills, Google Cloud has launched new enterprise level features on Google Cloud Skills Boost.

Journey from Qwiklabs to Google Cloud Skills Boost

World Health Day 2022 - Every year, World Health Day is celebrated on 7th of April which focuses on contemporary health issues that require immediate attention….

Slides, Videos, Audio

GCP Podcast - #299 Apache Beam with Kenneth Knowles and Pablo Estrada.

Kubernetes Podcast - #175 Shipa, with Bruno Andrade.

Security Podcast - #59 Zero Trust: So Easy Even a Government Can Do It?

Releases

Anthos clusters on AWS - A security vulnerability, CVE-2022-0847, has been discovered in the Linux kernel version 5.8 and later that can potentially escalate container privileges to root. Anthos Clusters on AWS aws-1.11.0-gke.1 (previous generation) is now available. This version adds support for Kubernetes 1.22 clusters. The workload identity webhook is not supported on cluster versions 1.22 and later. To use an Application Load Balancer (ALBs) with version 1.22 clusters, you need to upgrade your ALB controller configuration. This release includes fixes for the following CVEs: CVE-2021-22600 CVE-2022-23648 CVE-2021-3177 CVE-2019-15165 CVE-2021-3521 CVE-2021-4122. This version no longer supports creation or maintenance of Kubernetes 1.18 clusters.

Anthos - Anthos component releases for March, 2022 Anthos clusters on VMware: March 3, 2022: 1.10.2 patch release March 15, 2022: 1.8.8 patch release March 24, 2022: 1.9.5 patch release Anthos clusters on bare metal: March 14, 2022: 1.8.9 patch release March 23, 2022: 1.9.6 patch release March 31, 2022: 1.10.3 patch release March 31, 2022: 1.11.0 quarterly minor release Anthos clusters on AWS: March 21, 2022: region announcement Anthos clusters on Azure: March 21, 2022: region announcement Anthos Config Management: March 22, 2022: 1.11.0 quarterly minor release Anthos Service Mesh: March 2, 2022: 1.12.4 patch release March 10, 2022: 1.10.6 & 1.11.8 & 1.12.5 patch release March 30, 2022: 1.13.1 patch release Connect: N/A Cloud Run for Anthos: N/A Migrate for Anthos and GKE: March 28, 2022: 1.11.0 quarterly minor release Cloud Logging: March 11, 2022: release updates March 21, 2022: release updates March 25, 2022: release updates Cloud Monitoring: March 4, 2022: release updates March 11, 2022: release updates March 28: release updates.

AppEngine Standard Java - Updated the Java SDK to version 1.9.96 to also support Java 17 (preview).

BigQuery - BigLake is now available in Preview. BigQuery now supports the creation of search indexes and a SEARCH function. Analytics Hub is now available in Preview. BigQuery Omni now supports cross-cloud transfer.

BigTable - The Cloud Bigtable HBase replication library lets you replicate data from HBase to Bigtable.

Cloud Composer - Snapshots are available in Preview. (Airflow 2) The max_connections limit in databases is increased. When syncing objects in an environment bucket to Airflow components, Cloud Composer now reports problems with file names. Airflow 2.2.3 packages are updated: added keyrings.google-artifactregistry-auth, updated apache-airflow-providers-google to version 6.6.0. When an environment is created, Cloud Composer validates if a policy that prohibits PSC connections is set on the project. Improved the validation when installing PyPI packages. (Cloud Composer 1) The Airflow web server is restarted when values of [core]log_filename_template and [core]logging_level Airflow configuration options change. (Airflow 2) Fixed the logging level of some messages generated by Airflow workers. Improved the error message generated when Cloud Composer Service Agent does not have enough permissions to create an environment. Improved the error message generated when a PyPI package installation times out. Improved the handling of insufficient quota errors during the environment creation. Improved the handling of deployment errors generated when creating or updating an environment. Fixed the severity level for Airflow Worker Set Controller logs. (Cloud Composer 2) Fixed an issue which caused failures when creating environments with Private Service Connect in a Shared VPC configuration. Cloud Composer 1.18.5 and 2.0.9 images are available: composer-2.0.9-airflow-2.2.3 composer-2.0.9-airflow-2.1.4 composer-1.18.5-airflow-2.2.3 composer-1.18.5-airflow-2.1.4 composer-1.18.5-airflow-1.10.15 (default). Cloud Composer 1.15.2 has reached its end of full support period.

Compute Engine - Generally available: You can now set the number of threads per core on a VM. Generally available: You can now autoscale both regional and zonal managed instance groups based on a Cloud Monitoring metric that provides an aggregated value for the group.

Config Connector - Config Connector version 1.80.0 is now available. Added support for ApigeeOrganization resource. Added support for NetworkServicesTLSRoute resource. Added spec.destination.loggingLogBucketRef to LoggingLogSink.

Data Catalog - Public Preview: Data Catalog is integrated with Analytics Hub, enabling you to work with linked datasets.

Data Catalog Resources - v1. Public Preview: Data Catalog is integrated with Analytics Hub, enabling you to work with linked datasets.

Database Migration Service - Database Migration Service now supports migrating Oracle workloads into Cloud SQL for PostgreSQL.

Dialogflow Enterprise - Dialogflow CX now supports Access Transparency logging of Google personnel access to Dialogflow data. Dialogflow CX now provides a setting to lock agent editing. Dialogflow CX system functions have new list-related functions: APPEND, GET, and REMOVE.

Dialogflow - Dialogflow CX now supports Access Transparency logging of Google personnel access to Dialogflow data. Dialogflow CX now provides a setting to lock agent editing. Dialogflow CX system functions have new list-related functions: APPEND, GET, and REMOVE.

Document AI - v1. New Version of Lending W2 Processor We have released a new Release Candidate version of the W2 Processor.

Eventarc - Eventarc is now available in the asia-south2 (Delhi, India) region.

Google Kubernetes Engine - A security vulnerability, CVE-2022-0847, has been discovered in the Linux kernel version 5.8 and later that can potentially escalate container privileges to root.

Cloud Logging - You can now add indexed LogEntry fields to your Cloud Logging buckets to make querying your logs data faster.

Retail Recommendations AI - Retail Search is generally available. There are new data use terms for access and use of customer data for Recommendations AI and Retail Search.

Security Command Center - The iamBindings[] and nextSteps attributes were added to the Finding object.

SAP Solutions - Backint agent for SAP HANA version 1.0.17 Version 1.0.17 of the Google Cloud Backint agent for SAP HANA is now available. Google Cloud monitoring agent for SAP NetWeaver, version 2.2 Version 2.2 of the monitoring agent for SAP NetWeaver is now available.

Cloud Spanner - Cloud Spanner now allows you to export a subset of your database tables to Google Cloud Storage as Avro files.

Cloud Storage - With dual-region storage, users can now specify two regions within the same continent to create a dual-region of their choosing.

Vertex AI - Vertex AI Model Registry is available in Preview. Vertex AI Workbench is generally available (GA). The Vertex AI Workbench managed notebooks executor is generally available (GA).

VMware Engine - VMware Engine nodes are now available in the following additional zone: Ashburn, Northern Virginia, North America (us-east4-b).

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko