Welcome to issue #330 January 23rd, 2023

News

Log Analytics in Cloud Logging is now GA - Cloud Logging’s Log Analytics, with advanced search, as well as aggregation and transformation of all log data types, is now generally available.

Get migrating in 2023: join our upcoming fireside chat featuring Forrester - The State of Public Cloud 2022, a study by Forrester Research and commissioned by Google Cloud, to help organizations understand how to get their cloud migration projects right in 2023 and beyond.

---

Claudie is multi-cloud managed Kubernetes

Claudie sets new standards for multi-cloud and hybrid-cloud operability, disaster recovery and high availability.

• Capable of placing each nodepool into a different cloud provider.
• Enables seamless workload migration between cloud providers while taking full production load.
• Operated via Infrastructure-as-Code and therefore compatible with GitOps driven cluster provisioning.
• Makes clusters resilient to a full cloud provider outage and breaks the cloud provider lock-in.
• Enables a combination of cloud resources for the best cost efficiency.

100% upstream Kubernetes compliant and open-source. Engineered by Berops.

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Reliability and SRE in the 2022 State of DevOps Report - Learn more about the connection between SRE, DevOps and reliability.

Traffic Management through Gateway API in GKE - Running Gateway API in GKE to achieve advance traffic management.

Using IPv6 Unique Local Addresses for private connectivity in Google Cloud - Support for IPv6 Unique Local Addresses, or ULA, in Google Cloud helps large enterprises running out of private IPv4 addresses.

Creating a Helm repository in two ways: using Google Cloud Artifact and Github Pages - In this article we will walk through the workflow of using Helm repo and two ways to set it up.

Canary deployments using Kubernetes Gateway API, Flagger and Google Cloud Deploy - Now you can accelerate your app releases, gradually shift traffic to new releases, enable canary release strategies and allow A/B testing of your app.

App Development, Serverless, Databases, DevOps

Answering the Top-25 most asked Google Cloud questions of 2022 - We’re giving descriptive and explanatory answers to the Top-25 Most Asked Questions around Google Cloud!

A journey from App Engine to Cloud Run: Adopting containers and reducing infrastructure costs by 70 percent - A Google App Engine customer consolidated its workloads on Cloud Run, keeping with serverless while adding containers and event-driven architecture.

Become a GCP Master: Get Comfortable with the gcloud CLI - Become a GCP power user by learning how to use the gcloud CLI for almost any tasks you need to achieve in GCP.

Managing Dialogflow CX Agents with Terraform - Manage Dialogflow CX agents and their components with Terraform. Create a Dialogflow CX agent with one command in your Google Cloud account.

How does Bigtable work/Best practices - An overview of Cloud Bigtable.

How to Set GOOGLE_APPLICATION_CREDENTIALS in Python - Configuring Application Default Credentials and fixing oauth2client.client.ApplicationDefaultCredentialsError.

Wildthing, I think I love you… - How to set up a Google Cloud wildcard SSL certificate.

Building an automated serverless deployment pipeline with Cloud Build - A high level overview of a pipeline that will build, push, and deploy a serverless application from a GitHub repository to Cloud Run.

A reference architecture for transforming insurance claims processing with Google Cloud - Through this Google Cloud reference architecture, organizations that process Insurance claims can dramatically enhance the policyholder experience.

GCP — Workflow Integrates with Cloud Build for DR Orchestration - Implementing a custom Disaster Recovery solution.

EventSync: the event-driven management missing piece. - Event driven has a missing piece. You can produce, consume, duplicates (fan-out) events, but what about aggregation (fan-in)?

Big Data, Analytics, ML&AI

How to deploy Tink for BigQuery encryption on-prem and in the cloud - Google Cloud customers who want app-level encryption in hybrid cloud data warehouses can encrypt and decrypt that data outside BigQuery. Here’s how to do that securely.

Built with BigQuery: How to accelerate data-centric AI development with Google Cloud and Snorkel AI - Built with BigQuery: How to Accelerate Data-Centric AI development with Google Cloud and Snorkel AI.

Run data science workloads without creating more data silos - Build a data lake solution enabling data science teams to run workloads without creating new data silos, no matter how complex your organization is.

Google Cloud Dataplex Data Lineage, BigQuery & dbt: It Just Works! - For those of us who have been in love with dbt and also, working in the awesome world of Google Cloud (GC) — dbt’s out-of-the-box Data….

Simple tips to make datasets cleaner using dbt and BigQuery - Two tips that can help you make datasets little more organized.

BigQuery Ingestion-Time Partitioning and Partition Copy With dbt - Combining ingestion-time partitioning and partition copy is a great way to achieve great performance for your models.

A step-by-step guide to Row and Columns Access policies in BigQuery - Example of using access policies in BigQuery.

Code Conversion Chronicles: A Journey from Oracle to BigQuery. - Issues when migrating from Oracle to BigQuery.

An Intro to Bigquery Geospatial Functions - A case study of using BigQuery geospatial functions.

How to calculate a correlation matrix on Big Data using Google BigQuery - An approach with code to efficiently calculate the correlation matrix for tables that have both many rows and a high number of columns.

Optimize Cloud Composer via Better Airflow DAGs - Think of Cloud Composer as the engine and the Apache Airflow DAGs as the fuel you provide. This guide suggests a variety of ways to improve your Airflow DAGs and keep your Cloud Composer environment running as efficiently as possible.

Built with BigQuery: How Tamr delivers Master Data Management at scale and what this means for a data product strategy - Built with BigQuery: How Tamr delivers Master Data Management at scale and what this means for a data product strategy.

How to do multivariate time series forecasting in BigQuery ML - Multivariate time series forecasting allows BigQuery users to use external covariate along with target metric for forecasting.

All you need to know to get started with Vertex AI Pipelines - Sharing experience using Vertex AI Pipelines in a project running in production.

Step-by-Step Guide to Accelerate ML Jobs with Rapids, Dask, Vertex and GPUs - An example of training 4 million of rows of data with the xgboost on Vertex AI.

Slides, Videos, Audio

Security Podcast - #104 CISO Walks Into the Cloud: And The Magic Starts to Happen!

Releases

Apigee X - On January 19, we released an updated version of Apigee X (1-9-0-apigee-19). The VerifyAPIKey policy and the VerifyAccessToken action of the OAuth2 policy now support CacheExpiryInSeconds. Bug ID Description 217173784 The HMAC.policy-name.error variable is populated for HMAC failing policies.

AppEngine Standard Go - The Go 1.18 and Go 1.19 runtimes for App Engine standard environment are now generally available.

AppEngine Standard NodeJS - The Node.js 18 runtime for App Engine standard environment is now generally available.

Batch - Batch is available in the following regions: northamerica-northeast1 (Montréal) europe-west2 (London) For more information, see Locations.

BigQuery ML - More than 20 BigQuery ML components for Vertex AI Managed Pipelines are now generally available. You can now use the TRANSFORM clause to train models which you can then export in the Tensorflow SavedModel format. The following functions have been added for BigQuery ML: ML.ROBUST_SCALER ML.NORMALIZER ML.ONE_HOT_ENCODER ML.IMPUTER ML.MAX_ABS_SCALER ML.LABEL_ENCODER These features are now available in preview. BigQuery ML support for multivariate time-series forecasting with the ARIMA_PLUS_XREG model is now available in preview. Sparse input support in BigQuery ML model training is now generally available (GA).

BigQuery - You can now attach Resource Manager tags to datasets, which let you conditionally apply Identity and Access Management (IAM) policies to your resources.

BigTable - Client-side metrics are now available to customers who use the Cloud Bigtable HBase client for Java version 2.6.4 or later.

Billing - Committed Use Discounts: View your usage data at an hourly granularity in the CUD Analysis dashboard You can now analyze the effectiveness of your spend-based or resource-based committed use discounts at an hourly granularity.

Cloud Build - Users can now configure private pools to consume fewer IP addresses within their peered Virtual Private Clouds (VPCs). gcloud commands for creating and managing build triggers are now generally available.

CDN - Cloud CDN supports private origin authentication for Amazon Simple Storage Service (Amazon S3) and compatible object stores.

Chronicle - ListCuratedRules and ListCuratedRuleDetections Two new methods are now available for the Detection Engine API. The following changes are available in the Unified Data Model: A new field, called source_labels, was added to EntityMetadata. Chronicle has released a set of ingestion scripts, written in Python, that can be deployed as Cloud Functions. The following supported default parsers have changed.

Compute Engine - You can now use the Observability tab on the Compute Engine VM instances page to see the five virtual machines consuming the most of a resource. Preview: When a managed instance group (MIG) repairs a failed or an unhealthy VM, you can apply the latest instance template and per-instance configuration to recreate the VM instead of applying the configuration originally used to create the VM.

Database Migration Service - Database Migration Service support for migrating Oracle workloads into Cloud SQL for PostgreSQL is now in Public Preview.

Dataproc Metastore - v1. Dataproc Metastore is available in the following regions: asia-east2 (Hong Kong), europe-central2 (Warsaw), europe-north1 (Finland), and us-west4 (Las Vagas).

Cloud Deploy - Google Cloud Deploy now supports Skaffold version 2.0.

Dialogflow - Dialogflow CX now provides channel-specific responses. Dialogflow CX now provides custom payload templates. Migrating from Dialogflow ES to Dialogflow CX is recommended for complex agents, and migrating can significantly improve your agent design and end-user experience.

Cloud Functions - The ability to optionally configure 2nd gen functions with user-specified concurrency and vCPU is now available in preview. Cloud Functions now supports the Node.js 18 runtime at the General Availability release level.

Networking Interconnect - Dedicated Interconnect support is available in the following colocation facilities: Telefónica Nabiax Alcala datacentre, Madrid Telecom Italia Cebrosa Campus, Turin For more information, see the Locations table. Dataplane v2 for Cloud Interconnect is fully available for customers using Dedicated Interconnect or Partner Interconnect in the following regions: us-west2 (Los Angeles) us-west4 (Las Vegas) All new VLAN attachments that you create in these regions are automatically provisioned on Dataplane v2.

Google Kubernetes Engine - You can now attach ephemeral and block devices backed by Local NVMe SSDs during GKE node pool creation, using the Ephemeral Storage Local SSD API and the Local NVMe SSD Block API respectively, with node version 1.25.3-gke.1800 or later.

Load Balancing - Internal TCP/UDP load balancers can now be configured to handle private IPv6 traffic within your VPC.

Cloud Logging - You can now save your SQL queries, and you can set custom retention periods on log buckets upgraded to use Log Analytics.

Migrate for Compute Engine 4.8 - 5.0. Generally available: Migrate to Virtual Machines from an AWS source lets you migrate AWS EC2 instances to Compute Engine.

Cloud Monitoring - You can now use the Observability tab on the Compute Engine VM instances page to see the five virtual machines consuming the most of a resource.

Cloud Interconnect - Dedicated Interconnect support is available in the following colocation facilities: Telefónica Nabiax Alcala datacentre, Madrid Telecom Italia Cebrosa Campus, Turin For more information, see the Locations table. Dataplane v2 for Cloud Interconnect is fully available for customers using Dedicated Interconnect or Partner Interconnect in the following regions: us-west2 (Los Angeles) us-west4 (Las Vegas) All new VLAN attachments that you create in these regions are automatically provisioned on Dataplane v2.

Service Mesh - 1.14.x. 1.14.6-asm.4 is now available for in-cluster Anthos Service Mesh. 1.15.x. 1.15.4-asm.2 is now available for in-cluster Anthos Service Mesh. 1.13.x. 1.13.9-asm.9 is now available for in-cluster Anthos Service Mesh. Managed Anthos Service Mesh. Service mesh cloud gateway is now available as a preview feature for managed Anthos Service Mesh in the rapid release channel.

Anthos Service Mesh - 1.14.x. 1.14.6-asm.4 is now available for in-cluster Anthos Service Mesh. 1.15.x. 1.15.4-asm.2 is now available for in-cluster Anthos Service Mesh. 1.13.x. 1.13.9-asm.9 is now available for in-cluster Anthos Service Mesh. Managed Anthos Service Mesh. Service mesh cloud gateway is now available as a preview feature for managed Anthos Service Mesh in the rapid release channel.

Cloud SQL MySQL - Cloud SQL now supports viewing an audit log for an automated backup of an instance to verify whether the backup is completed successfully. Cloud SQL for MySQL now supports using the lower_case_table_names flag for MySQL 8.0.

Cloud SQL Postgres - Cloud SQL now supports viewing an audit log for an automated backup of an instance to verify whether the backup is completed successfully.

Cloud SQL SQL Server - Cloud SQL for SQL Server enables you to use point-in-time recovery. Cloud SQL now supports viewing an audit log for an automated backup of an instance to verify whether the backup is completed successfully.

Cloud Storage Transfer - Temporary pricing for Storage Transfer Service has expired; refer to Pricing for current charges.

Cloud Storage - Some gsutil commands that use the -m global flag, such as cp and rsync, are currently failing if the requester does not have the storage.buckets.get permission. gcloud storage GA release 1.2 is now available.

Tensorflow Enterprise - Starting with version 2.9, TensorFlow Enterprise releases are supported for one year.

Vertex AI - Vertex AI Explainability When uploading TensorFlow 2 models, the ExplanationMetadata field is now optional, making it easier to configure your model for explainability.

VMware Engine - If you are running VMware Engine Horizon (VDI) on Google Cloud VMware Engine, you may encounter errors after changing your private cloud resource naming to meet the standards for Google Cloud CLI and VMware Engine API.

VPC Service Controls - Preview stage support for the following integration: Essential Contacts.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko