Welcome to issue #296 May 30th, 2022

News

Cloud on Spain’s terms: New Google Cloud region in Madrid now open - The new Madrid region (europe-southwest1) provides low-latency, highly available services with international security and data protection standards.

The new Google Cloud region in Columbus, Ohio is open - Google Cloud’s Columbus, Ohio region is now open, bringing a second region to the midwest, for a total of 33 regions across the globe.

Introducing Network Analyzer: One stop shop to detect service and network issues - Introducing Network Analyzer, the newest module for Google Cloud’s Network Intelligence Center.

Change streams for Cloud Spanner: now generally available - Cloud Spanner change streams are now generally available. With change streams, you can capture and stream out changes from your Cloud Spanner databases in near real-time.

Accelerate your High Performance Computing journey with new Google Cloud HPC Toolkit - We introduce the Cloud HPC Toolkit, an open source tool enabling users to easily and quickly create HPC clusters based upon proven best practices.

Take the 2022 Accelerate State of DevOps Survey - The 2022 State of DevOps report will be security focussed - take the survey to shape the future of DevOps and make your voice heard.

Google is named a Leader in the 2022 Gartner® Magic Quadrant™for Cloud AI Developer Services report - Google is named a Leader in the 2022 Gartner® Magic Quadrant™for Cloud AI Developer Services report.

Introducing high-performance Confidential Computing with N2D and C2D VMs - We’re excited to announce that Confidential Computing is now available on the latest general-purpose N2D and compute-optimized C2D Virtual Machines.

Announcing Quick Start Widget to integrate API key generation into your onboarding - Today we’re announcing Quick Start Widget to help developers make it easier for their users to get started with Google Maps Platform, generate an API key, and embed maps in their websites and apps.

Introducing GKE cost estimator, built right into the Google Cloud console - The new GKE cost estimator lets you assess your compute running costs right from the Google Cloud console.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Discover our new edge concepts at Hannover Messe that bring smart factories to life - Intel and Google Cloud demonstrate edge-to-cloud technology at Hannover Messe.

Assembling and managing distributed applications using Google Cloud Networking solutions - At the 2022 Networking Spotlight, Google Cloud and customers Twitter, Snapchat and MongoDB discuss new app connectivity and delivery architectures.

How Google Cloud monitors its Quality Management System - Google Cloud shares its quality management principles that help deliver high quality products and services to customers.

Setting up Config Connector with Terraform & Helm - This article explains how to set Config Connector with Terraform.

Trendyol GCP Network Transformation - Experience in setting and using GCP VPC network.

ExternalDNS with GKE & Cloud DNS - This tutorial demonstrates how to setup and configure ExternalDNS on GKE and Cloud DNS.

Oops! Deleted the Terraform state file. Imported the resource but now it wants to recreate it! - Handling a case when someone by accident deletes the Terraform state file.

App Development, Serverless, Databases, DevOps

AlloyDB for PostgreSQL under the hood: Columnar engine - In this technical deep dive, we take a look at the columnar engine that delivers industry leading query performance for AlloyDB for PostgreSQL.

Running a virtual event globally, how Gramercy Tech leveraged serverless technology - Gramercy Tech runs Eventfinity platform on Google Cloud Run Serverless Containers across multiple regions, lowering operational effort and improving availability.

Migrate databases to Google Cloud VMware Engine (GCVE) - Processes and tools used to migrate databases to Google Cloud VMware Engine (GCVE).

Application Rationalization through Google Cloud’s CAMP Framework - Application Rationalization through CAST Highlight (automated source code scan with business context) and mFit (VM workload assessment & automated migration).

Enterprise DevOps Guidebook - Chapter 1 - Learn more about how to implement DORA best practices with our DevOps Enterprise Guidebook.

Eliminate hotspots in Cloud Bigtable - Learn how hotspots can impact the performance of your Cloud Bigtable database. Debugging hot tablets can reduce P99 latencies and increase throughput.

Monitoring transaction ID utilization using Cloud SQL for PostgreSQL metrics - Using Cloud SQL PostgreSQL metrics to monitor transaction ID utilization and vacuum activity, and prevent transaction ID wraparound.

What is new with Google Cloud and C++ - The latest news regarding C++ progress in GCP.

How to create a Static Outbound IP for Google Cloud Functions using Terraform - Setting the static IP address for Cloud Functions via Terraform.

How to Use Logging Query Language to Analyze GCP Logs Data in Python - How to use Google Cloud’s Logs API and Logging Query Language in Python to obtain real-time data on active GCP instances.

Big Data, Analytics, ML&AI

Get value from data quickly with Informatica Data Loader for BigQuery - With Informatica’s Data Loader on Google Cloud, accelerate data uploads and keep data flowing to get insights and answers faster.

Built with BigQuery: Gain instant access to comprehensive B2B data in BigQuery with ZoomInfo - Built with BigQuery shows how ZoomInfo helps its customers incorporate market intelligence into other internal data sets.

Google Cloud Dataplex -Part 1-Lakes, Zones, Assets and discovery - Intro to Dataplex.

Fast export large database tables — using GCP Serverless Dataproc (Spark) - Import large tables from MySQL (any JDBC) to Cloud Spanner (or GCS).

Analyze and plot 5.5M records in 20s with BigQuery and Ploomber - Develop scalable pipelines on Google Cloud using open-source software.

Connect Oracle to BigQuery using DB Link - Background: I searched for an easy, end-to-end, working guide to configure Oracle Database Gateway with use of Simba BigQuery ODBC driver….

Data Masking with Tokenization using Google Cloud DLP and Google Cloud Dataflow - How to automate data masking using Google Cloud DLP and Google Cloud Dataflow.

BigQuery Remote Functions, Cloud Functions 2.0, and Plus Codes Revisited - Using BigQuery remote Cloud Function to convert Geo coordinates to Plus Code.

Training Deep Learning-based recommender models of 100 trillion parameters over Google Cloud - Training a recommender model of 100 trillions parameters on Google Cloud, using GKE to orchestrate C2 VMs, M2 VMs and A100 GPUs.

Notified team gets smart on MLOps through Advanced Solutions Lab for Machine Learning - Notified worked with Google Cloud and the Advanced Solutions Lab to train their team on Machine Learning Operations (MLOps).

Vertex AI Pipelines vs. Cloud Composer for Orchestration - A personal opinion based on workshops with many different customers.

How to reduce your ML model inference costs on Google Cloud - Reducing your costs at least by half can be so easy, 4 steps I regularly recommend to companies.

Various

K8s in the Cloud - With a whole universe of things to learn, it’s important to start with the essentials. Let’s get to know why Kubernetes is usable, scalable….

Why IT leaders choose Google Cloud certification for their teams - Why IT leaders should choose Google Cloud training and certification to increase staff tenure, improve productivity for their teams, satisfy customers, and speed up digital transformation.

Google Cloud Data Heroes Series: Meet Antonio, a Data Engineer from Lima, Peru - Google Cloud continues their Data Hero series with a profile on Antonio C., a data engineer, teacher, writer, and enthusiast on GCP.

Qwiklabs Trivia May- Last Week! - When it comes to trivia, there are several benefits- Increased brain health, improved cognitive skills and reduced stress. And with….

Slides, Videos, Audio

GCP Podcast - #305 GKE Release Channels with Kobi Magnezi and Abdelfettah Sghiouar.

Security Podcast - #66 EP66 Is This Binary Legit? How Google Uses Binary Authorization and Code Provenance.

GCP Life - #15 - In this episode we discuss; Google Bankrupt, Anthony McMahon, Open Source Service, Zero Trust, Web3, GCP TPU, Terraform vet, Cloud SQL.

SRE Podcast - #8 - Incident Management with Adrienne Walcer.

Releases

Access Approval - Access Approval lets you know if the notification emails for access requests don't get delivered to you because you provided an incorrect email address while setting up the notification configurations.

Anthos clusters on bare metal - 1.11. Release 1.11.2 Anthos clusters on bare metal 1.11.2 is now available for download. Starting with Anthos clusters on bare metal release 1.11.2, you can enable or disable Anthos VM Runtime by updating the VMRuntime custom resource only. If you have Anthos VM Runtime enabled for your Anthos clusters on bare metal, you must disable it before upgrading clusters to version 1.11.2 or higher. Starting with Anthos clusters on bare metal release 1.11.2, the Anthos VM Runtime API version has changed from v1alpha1 to v1. Functionality changes: The containerd runtime has been upgraded to 1.5.11-gke.0 to address CVE-2022-24769 Added a preflight check that disallows Ubuntu 18.04 distributions with 4.15.x Linux kernels. Fixes: Fixed cluster custom resource status reporting for pending reconciliations. Known issues: For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Anthos clusters on VMware - Anthos clusters on VMware 1.11.1-gke.53 is now available. Fixed for v1.11.1 Fixed the known issue where v1.11.0 user clusters cannot be created with a v1.10.x admin cluster. Anthos clusters on VMware 1.10.4-gke.32 is now available. Fixed for v1.10.4 Fixed the following vulnerabilities High-severity CVEs CVE-2022-1271 CVE-2021-4160 CVE-2022-27666 CVE-2018-25032 CVE-2022-1055 CVE-2022-23219 CVE-2022-23218 CVE-2021-3999 CVE-2018-25032 RBAC fixes anetd Changed to use kubelet kubeconfig to only allow the anetd to update its own node resource, and the pod resources that are running on the node.

AppEngine Flexible - You can now disable external ephemeral IP addresses for App Engine Flex services.

AppEngine Standard Java - Updated the Java SDK to version 1.9.97.

Google Cloud Armor - Google Cloud Armor integration with reCAPTCHA Enterprise is now in General Availability.

Artifact Registry - Apt and Yum repositories are now generally available. Artifact Registry is now available in the us-east5 region (Columbus, United States).

Cloud Asset Inventory - The following resource types are now publicly available through the Analyze Policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning): Cloud KMS cloudkms.googleapis.com/EkmConnection Cloud Run run.googleapis.com/Job run.googleapis.com/Execution.

BigQuery - You can now load data into BigQuery using Informatica Data Loader. Metrics for query/statement_scanned_bytes and query/statement_scanned_bytes_billed are no longer delayed for 6 hours in order to smooth reporting over the duration of the job.

Cloud Build - Users can now receive build status notifications in Google Chat via a Google Chat notifier.

Chronicle - The following supported default parsers have changed, listed by product name and ingestion label: Apache Hadoop (HADOOP) Suricata IDS (SURICATA_IDS) GCP Compute (GCP_COMPUTE) Elastic Audit Beats (ELASTIC_AUDITBEAT) Cloudflare (CLOUDFLARE) Proofpoint On Demand (PROOFPOINT_ON_DEMAND) FortiGate (FORTINET_FIREWALL) CSV Custom IOC (CSV_CUSTOM_IOC) CrowdStrike Falcon (CS_EDR) Cisco Firepower NGFW (CISCO_FIREPOWER_FIREWALL) CIS Albert Alerts (CIS_ALBERT_ALERT) SonicWall (SONIC_FIREWALL) Okta User Context (OKTA_USER_CONTEXT) Elastic Windows Event Log Beats (ELASTIC_WINLOGBEAT) Check Point (CHECKPOINT_FIREWALL) Barracuda Email (BARRACUDA_EMAIL) Microsoft Azure Activity (AZURE_ACTIVITY) Carbon Black App Control (CB_APP_CONTROL) OpenSSH (OPENSSH) OneLogin (ONELOGIN_SSO) Office 365 (OFFICE_365) FireEye NX (FIREEYE_NX) ExtraHop RevealX (EXTRAHOP) Cisco Umbrella DNS (UMBRELLA_DNS) Kaspersky AV (KASPERSKY_AV) IBM Guardium (GUARDIUM) F5 ASM (F5_ASM) Cisco Email Security (CISCO_EMAIL_SECURITY) Workspace Activities (WORKSPACE_ACTIVITY) Forcepoint Proxy (FORCEPOINT_WEBPROXY) Azure AD Organizational Context (AZURE_AD_CONTEXT) Tanium Stream (TANIUM_TH) Apache (APACHE) For details about the changes in each parser, see Supported default parsers.

Key Access Justifications - Access Approval lets you know if the notification emails for access requests don't get delivered to you because you provided an incorrect email address while setting up the notification configurations.

Cloud Composer - Cloud Composer 1.18.10 and 2.0.14 release started on May 26, 2022. (Airflow 2) If your DAGs use the google-ads package version 14.0.0 or earlier, please upgrade your environment to Cloud Composer version 2.0.14 so that your environment uses Google Ads API v10. Added new database metrics: a metric that shows the total limit of database connections, and a metric for the number of active database connections. (Airflow 1) The google-cloud-bigquery package is upgraded from 1.28.0 to 2.13.0. (Airflow 2) Updates for the apache-airflow-providers-google package: Breaking changes: Upgrade to support Google Ads v10 (#22965) Features: [FEATURE] google provider - BigQueryInsertJobOperator log query (#23648) [FEATURE] google provider - split GkeStartPodOperator execute (#23518) Add exportContext.offload flag to CLOUD_SQL_EXPORT_VALIDATION. (Cloud Composer 2) Several false error log messages are no longer generated after an environment is created. (Cloud Composer 2) Fixed a problem where the Airflow web server becomes unavailable after all PyPI packages are uninstalled from an environment. Fixed a problem where Cloud Composer always reported an error when checking for connectivity to the PyPI repository during PyPI package installation in Private IP environments. Cloud Composer 1.18.10 and 2.0.14 images are available: composer-1.18.10-airflow-1.10.15 (default) composer-1.18.10-airflow-2.1.4 composer-1.18.10-airflow-2.2.3 composer-1.18.10-airflow-2.2.5 composer-2.0.14-airflow-2.1.4 composer-2.0.14-airflow-2.2.3 composer-2.0.14-airflow-2.2.5. Cloud Composer versions 1.16.5 and 1.17.0.preview.1 have reached their end of full support period. Private Service Connect support in Cloud Composer 2 is now generally available (GA). Privately used public IP addresses are now generally available (GA).

Compute Engine - Preview: You can now use the SSH troubleshooting tool from the Cloud console to help you determine the cause of failed SSH connections. Generally available: Columbus, Ohio, USA us-east5-a,b,c has launched with E2, N2, and N2D VMs in all three zones.

Config Connector - Config Connector version 1.86.0 is now available. Added support for ComputeRegionNetworkEndpointGroup resource. Added spec.serviceDirectoryRegistrations field to ComputeForwardingRule. Fixed issue where webhooks were unintentionally returning 500 errors when rejecting immutable field changes.

Data Catalog - v1. Data Catalog is now available in Santiago (southamerica-west1).

Data Catalog Resources - v1. Data Catalog is now available in Santiago (southamerica-west1).

Dataflow - Dataflow is now available in Columbus (us-east5).

Dataproc - New sub-minor versions of Dataproc images: 1.5.66-debian10, 1.5.66-ubuntu18, 1.5.66-rocky8 2.0.40-debian10, 2.0.40-ubuntu18, 2.0.40-rocky8. Upgraded Spark to 3.1.3 in Dataproc image version 2.0. Fixed a bug where job was not being marked as terminated after master node reboot. Fixed a bug where Flink was not able to run on HA clusters. Backported the fix for HIVE-20514 to Hive 2.3 in Dataproc image version 1.5. Fixed a bug with HDFS directories initialization when core:fs.defaultFS is set to an external HDFS.

Dialogflow Enterprise - Dialogflow CX now supports version-specific webhooks. Dialogflow CX now supports fine-grained webhook errors for built-in events.

Dialogflow - Dialogflow CX now supports version-specific webhooks. Dialogflow CX now supports fine-grained webhook errors for built-in events.

Cloud Functions - We have updated the documentation to clarify that to get the updates and security patches for runtimes and their dependencies, you need to deploy a function.

Cloud Healthcare API - A release was made. Updates may include general performance improvements, bug fixes, and updates to the API reference documentation.

KMS - Cloud KMS is available in the following region: us-east5 For more information, see Cloud KMS locations.

Google Kubernetes Engine - (2022-R13) Version updates GKE cluster versions have been updated. 1.24 is now available in the Rapid channel Kubernetes 1.24 is now available in the Rapid channel. New API versions storage.k8s.io/v1 CSIStorageCapacity. Notable changes GKE does not support node images that use Docker as the runtime in GKE version 1.24 and later. Deprecated API versions These APIs are still served in version 1.24 but are in a deprecation period: PodSecurityPolicy policy/v1beta1 PodSecurityPolicy Deprecated in 1.21 with removal targeted for version 1.25. Nodes on version 1.24.0-gke.1000 with more than 80GB of memory will fail to start successfully due to a known bug, which will be resolved in future 1.24 versions. You can now easily assess the running cost implications at cluster creation time. GKE clusters that run control plane versions 1.21 or later and node versions 1.16 or earlier might experience: Readiness check failures. The us-east5 region in Columbus, Ohio is now available.

GKE - (2022-R13) Version updates Version 1.22.8-gke.201 is now the default version.

Google Kubernetes Engine Rapid - (2022-R13) Version updates Version 1.23.5-gke.2400 is now the default version in the Rapid channel. 1.24 is now available in the Rapid channel Kubernetes 1.24 is now available in the Rapid channel. New API versions storage.k8s.io/v1 CSIStorageCapacity. Notable changes GKE does not support node images that use Docker as the runtime in GKE version 1.24 and later. Deprecated API versions These APIs are still served in version 1.24 but are in a deprecation period: PodSecurityPolicy policy/v1beta1 PodSecurityPolicy Deprecated in 1.21 with removal targeted for version 1.25. Nodes on version 1.24.0-gke.1000 with more than 80GB of memory will fail to start successfully due to a known bug, which will be resolved in future 1.24 versions.

Google Kubernetes Engine Regular - (2022-R13) Version updates Version 1.22.8-gke.201 is now the default version in the Regular channel.

Google Kubernetes Engine Stable - (2022-R13) Version updates Version 1.21.11-gke.1100 is now the default version in the Stable channel.

Load Balancing - Regional external and regional internal HTTP(S) load balancers now support regional SSL policies.

Cloud Logging - Support has been removed for two previously deprecated system metrics: logging.googleapis.com/excluded_log_entry_count and logging.googleapis.com/excluded_byte_count. You can now collect IIS logs and additional metrics from the Ops Agent, starting with versions 2.14.0 (logs) and 2.15.0 (additional metrics). You can now collect Varnish logs and metrics from the Ops Agent, starting with versions 2.16.0 (logs) and 2.15.0 (metrics). You can now collect Active Directory Domain Services logs and metrics from the Ops Agent, starting with version 2.15.0. You can now collect Jetty logs from the Ops Agent, starting with version 2.16.0.

Memorystore for Memcached - Added new Memorystore for Memcached region: Columbus (us-east5).

Cloud Memorystore - Added new Memorystore for Redis region: Milan (europe-west8).

Cloud Monitoring - You can now configure an uptime check to validate a specific JSONpath. A new version of Managed Service for Prometheus is now available. You can now collect IIS logs and additional metrics from the Ops Agent, starting with versions 2.14.0 (logs) and 2.15.0 (additional metrics). You can now collect Varnish logs and metrics from the Ops Agent, starting with versions 2.16.0 (logs) and 2.15.0 (metrics). You can now collect Active Directory Domain Services logs and metrics from the Ops Agent, starting with version 2.15.0.

Cloud VPN - Cloud VPN is available in region us-east5 (Columbus, US).

Cloud PubSub - Pub/Sub is now available in us-east5 (Columbus, Ohio).

Pub/Sub Lite - The Kafka Shim Java client library for Pub/Sub Lite is now GA.

Cloud Run - The following new region is now available: us-east5.

Security Command Center - The compliances, exfiltration, and processes attributes were added to the Finding object.

SAP Solutions - Google Cloud monitoring agent for SAP NetWeaver version 2.3 Version 2.3 of the Google Cloud monitoring agent for SAP NetWeaver is now available. Monitoring agent for SAP HANA version 2.4 Version 2.4 of the monitoring agent for SAP HANA is now available.

Cloud Spanner - Cloud Spanner change streams capture and stream out inserts, updates, and deletes in near real-time—useful for analytics, archiving, and triggering downstream application workflows.

Cloud SQL - Support for us-east5 (Columbus).

Cloud Storage - Cloud Storage is now available in Columbus, Ohio (us-east5 region). JSON copy requests and XML copy requests now return a permanent error on timeouts for objects larger than 2.5 GiB and a retryable error on timeouts for objects smaller than 2.5 GiB.

Cloud TPU - Cloud TPU now supports Tensorflow 2.8.2 and 2.9.1.

Virtual Private Cloud - For auto mode VPC networks, added a new subnet 10.202.0.0/20 for the Columbus us-east5 region.

Cloud VPN - Cloud VPN is available in region us-east5 (Columbus, US).

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko