Welcome to issue #305 August 1st, 2022

News

No pipelines needed. Stream data with Pub/Sub direct to BigQuery - We are introducing a new type of Pub/Sub subscription that writes directly from Pub/Sub to BigQuery. This new ELT path will be able to simplify your event-driven architecture.

Meet the new Professional Cloud Database Engineer certification - Google Cloud launches a new Professional certification.

Introducing password policies for Cloud SQL for PostgreSQL and MySQL local users - New password validation for Cloud SQL for PostgreSQL and MySQL local users simplifies password management and can help better secure databases.

Unify data lakes and warehouses with BigLake, now generally available - BigLake, a storage engine that extends innovations in BigQuery storage to open file formats running on cloud object stores, is generally available.

New Google Cloud Marketplace Private Offers features to help our partners grow - Learn how new Google Cloud Marketplace Private Offers features help our ISV partners make better negotiated deals with our mutual customers.

Improve responsiveness with session affinity on Cloud Run - We launched session affinity for Cloud Run services. Use session affinity to improve responsiveness of services that store local state on containers.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

5 ways a SOAR solution improves SOC analyst onboarding - Security analysts are in short supply, so when you do acquire good talent, you want to ramp them up successfully. A SOAR solution can help smooth onboarding.

Achieving Autonomic Security Operations: Why metrics matter (but not how you think) - Metrics can be a vital asset - or a terrible failure - for keeping organizations safe. Follow these tips to ensure security teams are tracking what truly matters.

Data Intensive Applications with GKE and MariaDB SkySQL - Running stateless and stateful applications with GKE and MariaDB for simplified operations.

How Google Cloud can help stop credential stuffing attacks - By using a layered approach with Google Cloud Armor, customers can limit and often prevent credential stuffing attacks.

Google Cloud Global External HTTP(S) Load Balancer - Deep Dive - This blog looks at the Global External HTTP(S) Load Balancer and connects the dot as to how it works. This is focused on the two modes which are Global External HTTP(S) Load Balancer and Global External HTTP(S) Load Balancer (classic).

Cloud CISO Perspectives: July 2022 - Google Cloud CISO Phil Venables shares his thoughts on the important role and challenges of including cybersecurity in the boardroom, along with the latest security updates from the Google Cybersecurity Action Team.

Cloud IAM Google Cloud - Identity and access management: Authorization on Google Cloud.

How to introduce more empathy into security operations - The call for empathy is growing louder in cybersecurity, yet it remains largely overlooked. Here is how infosec practitioners can practice empathy.

Identity & Access management: Authentication with Cloud Identity - Identity & Access management: Authentication with Cloud Identity.

GSuite domain takeover through delegation

Google Cloud — Free Vulnerability Scanning with Security Command Center - Using free vulnerability scanning in Security Command Center.

App Development, Serverless, Databases, DevOps

Databases on Google Cloud Part 4: Query, Index, CRUD and Crush your Java app with Firestore APIs - In this “A Guide to Databases on Google Cloud part 2 - Options at a glance”, We will look into setting up Firestore, creating complex queries and indexes, making the database calls for standard Create, Read, Update, and Delete (CRUD) operations using Firestore APIs on a Java Spring Boot application deployed on Cloud Run without using a Dockerfile.

Dash on GCP — Part 3 - Deploy the dashboard to Cloud Run.

Firebase JS v9 — embrace pipe() and curry() with the new API - Firebase changed its Javascript API with the v9 version to make a more modular approach.

Docker for amd64, arm64 or armv7 using Cloud Build - This article will provide you all you need to build docker containers across multiple platforms.

TeamCity on Google Cloud - Deploying TeamCity using Cloud Deployment Manager.

Big Data, Analytics, ML&AI

Top 5 Takeaways from Google Cloud’s Data Engineer Spotlight - Google Cloud Data Hero’s Top Five Takeaways from the Data Engineer Spotlight.

Cloud Composer at Deutsche Bank: workload automation for financial services - Deutsche Bank’s use of Cloud Composer service is a showcase of workload automation in the financial services sector.

Managing the Looker ecosystem at scale with SRE and DevOps practices - Following DevOps and SRE best practices can help organizations bring order to distributed Looker environments.

Running Nextflow on Google Batch - Using Batch to analyze RNA sequencing data.

Google rolls out BigLake and integrates Analytics Hub and BigQueryML - How Google makes its Data Platform more powerful with 3 awesome Updates.

BigQuery: Snapshot dataset with Cloud Workflow - Data are precious and you can’t lost them. One solution: backup them. But how to do that on a whole dataset with BigQuery?

6 BigQuery SQL Functions Every User Should Know - Check if your database has them too.

Two (completely different) types of dbt incremental models in BigQuery - Partition-based loading or tracking the history of your downstream model with incremental loads.

Use R to train and deploy machine learning models on Vertex AI - How to train and deploy a machine learning model with R on Vertex AI.

How Cohere is accelerating language model training with Google Cloud TPUs - Google Cloud and Cohere discuss how Cohere’s new framework deployed on Cloud TPU v4 Pods helps accelerate large language model training.

Various

Get to know the top 3 teams of the Google Cloud Hackathon Singapore - On 10th April 2022, Google Cloud launched the first Singapore Google Cloud Hackathon, where startup teams were tasked to build solutions to create innovative solutions.

Notes from my Professional Cloud Database Engineer beta certification exam - Preparing and passing the Cloud Database Engineer certification exam.

Slides, Videos, Audio

GCP Podcast - #313 Arm Servers on GCP with Jon Masters and Emma Haruka Iwao.

Security Podcast - #76 Powering Secure SaaS … But Not with CASB? Cloud Detection and Response?

Releases

Anthos clusters on VMware - Anthos clusters on VMware 1.11.2-gke.53 is now available. Fixed a known issue in which the cluster backup feature affected the inclusion of always-on secrets encryption keys in the backup.

BigQuery - You can now create BigQuery subscriptions in Pub/Sub to write messages directly to an existing BigQuery table. Inverse trigonometric SQL functions are now generally available (GA). BigLake is now generally available (GA). The new Migrate section in the BigQuery documentation helps you migrate to BigQuery.

Chronicle - The following changes are available in the Unified Data Model: Added the MUTEX value to the EntityMetadata.EntityType enumerated type.

Cloud Composer - Cloud Composer 1.19.5 and 2.0.22 release started on July 28, 2022. (Available without upgrading) Fixed a problem where DAG import errors were not displayed on the Environment details page, if the error messages did not have a creation time set. Cloud Composer 1.19.5 and 2.0.22 images are available: composer-1.19.5-airflow-1.10.15 (default) composer-1.19.5-airflow-2.1.4 composer-1.19.5-airflow-2.2.5 composer-2.0.22-airflow-2.1.4 composer-2.0.22-airflow-2.2.5. Cloud Composer versions 1.16.11 and 1.17.0.preview.7 have reached their end of full support period.

Compute Engine - The quota limits displayed in the Cloud console might be incorrect in the us-east5 region. Preview: You can now merge or split your existing hardware resource commitments to create new upsized or downsized commitments. Generally available: Use the Cloud console, the gcloud tool, or the API to configure a VM to shut down when a Cloud KMS key is revoked. Generally available: When you create VMs in bulk, you can now use the following new values with the TARGET_SHAPE flag: ANY: Use this value to place VMs in zones to maximize unused zonal reservations.

Config Connector - Config Connector version 1.90.0 is now available. Fixed issue where spec.layer7DdosDefenseConfig field in ComputeSecurityPolicy was not being reflected onto underlying resource. Added support for ServiceDirectoryEndpoint resource. Added support for the DLPStoredInfoType resource. Added support for state-into-spec: absent to MonitoringAlertPolicy. Added spec.iap.oauth2ClientIdRef field to ComputeBackendService. Added spec.egressPolicies.egressTo.externalResources field to AccessContextManagerServicePerimeters,. Added spec.externalDataConfiguration.connectionId field to BigQueryTable. Added spec.includeBuildLogs field to CloudBuildTrigger. Added spec.cacheKeyPolicy.cdnPolicy.includeNamedCookies field to ComputeBackendService. Added spec.enableUlaInternalIpv6 and spec.internalIpv6Range fields to ComputeNetwork. Added spec.maxPortsPerVm field to ComputeRouterNats. Added spec.advancedOptionsConfig field to ComputeSecurityPolicy. Added spec.sslPolicyRef field to ComputeTargetHTTPSProxy. Added spec.monitoringConfig.managedPrometheus field to ContainerCluster. Added spec.sqlServerUserDetails field to SQLUser. Added spec.schemaSettings field to PubSubTopic. Added status.pscConnectionId and status.pscConnectionStatus fields to ComputeForwardingRule. Added status.creationTime and status.managedZoneId fields to DNSManagedZones. Added support for "reconcile resource immediately once its dependency is ready" feature for ComputeTargetPool, ComputeNetworkEndpointGroup, NetworkServicesGRPCRoute, NetworkServicesTLSRoute.

Data Catalog - The UI for dataset entry detail pages now includes a section that lets you see what entries are included in that dataset.

Cloud Deploy - You can now have Google Cloud Deploy generate a skaffold.yaml configuration file for you when you create a release, based on a single Kubernetes manifest which you provide. You can now view and compare Kubernetes and Skaffold confguration files for releases, using Google Cloud Console.

Document AI - v1beta3 & v1. New Release Candidate (RC) versions for PDAI Invoice and Expense processors - July 2022 We have launched new RC versions of Invoice parser and Expense parser on Jul 15, 2022.

Eventarc - Eventarc is available in the following regions: us-east5 (Columbus, Ohio, North America) us-south1 (Dallas, Texas, North America).

Cloud Healthcare API - Updates may include general performance improvements, bug fixes, and updates to the API reference documentation.

Google Kubernetes Engine - GKE node system configuration now supports setting the cgroup mode to use the cgroupv2 resource management subsystem. (2022-R18) Version updates GKE cluster versions have been updated.

Google Kubernetes Engine Rapid - (2022-R18) Version updates The following versions are now available in the Rapid channel: 1.21.14-gke.2100 1.22.12-gke.300 1.23.8-gke.1900 1.24.2-gke.1900 Version 1.23.8-gke.400 is now the default version in the Rapid channel The following versions are no longer available in the Rapid channel: 1.21.13-gke.900 1.22.9-gke.2000 1.23.6-gke.1700 1.24.1-gke.1800 Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.19 to version 1.20.15-gke.9900 with this release.

Google Kubernetes Engine Regular - (2022-R18) Version updates The following versions are now available in the Regular channel: 1.23.7-gke.1400 Version 1.22.10-gke.600 is now the default version in the Regular channel The following versions are no longer available in the Regular channel: 1.23.5-gke.1501 Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.19 to version 1.20.15-gke.9900 with this release.

Google Kubernetes Engine Stable - (2022-R18) Version updates The following versions are now available in the Stable channel: 1.20.15-gke.9900 1.21.13-gke.900 1.22.10-gke.600 1.23.7-gke.1400 Version 1.21.12-gke.1700 is now the default version in the Stable channel The following versions are no longer available in the Stable channel: 1.20.15-gke.8700 1.21.12-gke.1500 1.22.8-gke.200 1.23.6-gke.2200 Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.19 to version 1.20.15-gke.9900 with this release.

Load Balancing - Cloud Load Balancing introduces the internal regional TCP proxy load balancer.

Cloud Logging - You can now collect Couchbase logs and metrics from the Ops Agent, starting with version 2.18.2.

Cloud Monitoring - You can now add table widgets to custom dashboards that let you limit the number of table rows, display only those rows with the highest, or lowest values, and that display a visual indicator of the value as compared to the range of possible values. You can now add user-defined labels to public and private Uptime checks. You can now configure the acceptable response codes for public and private HTTP Uptime checks. You can now collect Aerospike metrics from the Ops Agent, starting with version 2.18.2. You can now collect Couchbase logs and metrics from the Ops Agent, starting with version 2.18.2. You can now collect Vault metrics from the Ops Agent, starting with version 2.18.2.

Cloud PubSub - You can now create BigQuery subscriptions in Pub/Sub to write messages directly to an existing BigQuery table.

Anthos Service Mesh - Managed Anthos Service Mesh. Version 1.14 is now available for managed Anthos Service Mesh and is rolling out to the Rapid Release Channel.

SAP Solutions - Cloud Storage Backint agent for SAP HANA version 1.0.21 Version 1.0.21 of the Cloud Storage Backint agent for SAP HANA is now available.

Cloud Spanner - Query Optimizer version 5 is generally available.

Cloud SQL Postgres - For PostgreSQL versions 9.6 to 13, the [PostgreSQL version].R20220710.01_00 maintenance version caused a behavior change for configuration parameters: Session-level configuration parameters with dashes (-) cannot be set. The following PostgreSQL minor versions and extension versions are now available: 14.3 is upgraded to 14.4. Added information about checking the LC_COLLATE value for your databases before performing a major version upgrade of the databases for your Cloud SQL for PostgreSQL instance.

Cloud Storage - Configurable dual-region storage is generally available (GA).

Vertex AI - We now offer Preview support for Custom prediction routines (CPR).

VMware Engine - Resource creation of named objects now enforce naming requirements that match other Google Cloud products like Compute Engine.

VPC Service Controls - General availability for the following integration: BigQuery Reservation API.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko