Welcome to issue #343 April 24th, 2023

News

Private Service Connect: Now hybrid and global - Private Service Connect global access now in preview, allows PSC endpoints to be accessed by clients from any region in Google Cloud.

Learn by doing: Introducing Google Cloud Jump Start Solutions - Jump Start Solutions are pre-built sample applications and infrastructure best-practices that you can deploy within a few clicks in your own Google Cloud account.

Earth Engine Partner Initiative expands, creating sustainable transformation opportunities

Bringing our world-class expertise together under Google Cloud Consulting - The Google Cloud Consulting portfolio provides a unified services capability, bringing together offerings, across multiple specializations, into a single place.

Google named a Leader in the 2023 Forrester Wave: Data Management for Analytics

---

Meet DoiT

The true promise of the cloud with ease, not cost. DoiT provides technology and cloud expertise to reduce cloud costs and boost engineer productivity. All from a Google Cloud Partner. Learn more

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Running Infrastructure-as-Code with the least privilege possible - Google service account impersonation lets you run your terraform code and manage resources without overly broad access.

How Chronicle can help advance security product development and overcome data lake challenges

Migrate from PSP policies to Policy Bundle - Policy Controller enables the enforcement of programmable policies for Anthos clusters.

Framing up FinOps: How to optimize your cloud costs on Google Cloud - Recommendations from several Google Cloud cost-optimization experts.

Understanding Nasuni’s new ransomware protection service on Google Cloud - At its core, the Nasuni File Data Platform can help organizations shift capacity off local file storage hardware and into cost-effective object storage.

I Hate IAM: but I need it desperately - Identity and Access management doesn't have to be painful.

Deep-dive: Kubernetes Network Policy in GKE - This post will help you understand Network Policies in detail, the use cases, and the various Network Policy configurations to reduce the attack vector of microservices hosted in Kubernetes clusters.

Reading Mandiant M-Trends 2023 - The famous Mandiant 2023 M-Trends (NOT G-Trends, mind you…) report is out, and here are some of the things found to be surprising and some not surprising.

App Development, Serverless, Databases, DevOps

Ingesting IoT Data From EMQX Enterprise to GCP Pub/Sub - This article will introduce how to ingest IoT data from EMQX Enterprise to GCP Pub/Sub service to help you complete the migration.

Migrating Devices from GCP IoT Core to EMQX Enterprise - This blog will demonstrate how to connect your devices on GCP IoT Core to the EMQX Enterprise we’ve deployed already.

Four organizations working toward a more sustainable future with Google Maps Platform - In honor of Earth Day, we’re sharing a few organizations that are using Google Maps Platform to educate, inspire, conserve, and take action.

Configuring your Compute Engine VMs with Tier_1 high-bandwidth networking - In this blogpost, we look at Compute Engine’s Tier_1 bandwidth tier, how to configure it, and some things to watch out for.

Take control of your supply chain with Artifact Registry remote and virtual repositories - Getting started using remote and virtual repositories in Artifact registry.

Manage Chat Notifications for Cloud Build pipeline executions - It is possible to setup notifications for all of the cloud build pipeline execution to Google Chat, do you know how?

Deploy and Handle GCP CloudTasks with Goblet in minutes. - This blog post explains how Goblet deploys CloudTaskQueues and provides the user with an simple code interface to enqueue and handle CloudTasks.

Cloud Translation - Language detection and translation.

Intel Sapphire Rapids on Google Compute Engine C3 — MPI & Storage Performance Evaluation - Over a month ago I wrote about Intel Sapphire Rapids leveraged by the GCE C3 instances at Google Cloud in the context of SPEC CPU® 2017.

Write Google Cloud function in Kotlin - An example of writing and deploying Cloud Function.

How to Leverage R and Google Cloud Storage for Scalable Analytics: Examples and Use Cases - A comprehensive guide to integrating R with GCP Cloud Storage for efficient data management.

Big Data, Analytics, ML&AI

How to Submit Spark Serverless Jobs, Manage Quota and Capture Errors - Today Dataproc Serverless is the modernest way to run your spark jobs in GCP. It lets you get out of the cluster boundaries, giving the….

Unleashing the Power of Geospatial Data with DBSCAN Clustering in BigQuery - One of the most powerful tools for analyzing geospatial data is DBSCAN clustering, which can be used to identify patterns and relationships….

How a green energy provider used Dataplex for its data governance and quality

Fine-Tuning BigQuery Costs: Best Practices and Advanced Techniques - Exploring a range of strategies and techniques for optimizing your usage and minimizing expenses.

Track your cloud costs with BigQuery for easy analysis - With Billing export to Cloud Storage going away, you can easily track costs using BigQuery direct reporting.

Built with BigQuery: How Mercari US turned BigQuery into an ML-powered customer-growth machine

Optimize Google Cloud BigQuery and Control Cost - I unknowingly blew $3,000 in 6 hours on one query in Google Cloud BigQuery. Here’s why and 3 easy steps to optimize the cost.

Streamlining with DocAI can yield cost savings and help accelerate missions - Document AI for the government empowers organizations to leverage Google machine learning to classify, split, and extract information to speed processing and analysis.

Recommending news articles using Vertex AI Matching Engine - How to ingest, train and deploy an AI model to better match articles and text artifacts based on a customer action.

Scaling deep retrieval with TensorFlow Recommenders and Vertex AI Matching Engine - In this blog, we demonstrate how to build a playlist recommendation system by implementing an end-to-end candidate retrieval workflow from scratch with Vertex AI.

Various

50 Years of the Internet: Celebrating the vision of Vint Cerf and Bob Kahn and Exploring the Future of Connectivity and Innovation - This year marks the 50th anniversary of Dr. Vinton Cerf’s and Dr. Robert Kahn’s original work on the Internet and nearly the 50th anniversary of their 1974 paper on internet architecture, important milestones that provide us with an opportunity to reflect on the significance of this work, and the impact it has had on the world over the past five decades.

Boost your cloud skills — play The Arcade with Google Cloud to earn points and prizes

Slides, Videos, Audio

Security Podcast - #117 Can a Small Team Adopt an Engineering-Centric Approach to Cybersecurity?

GCP Life Podcast - #37 In this episode we discuss; LTT Hack, TikTok Ban, 3CX Hack, Latitude Hack, Crown Hack, Service NSW Hacks, Chrome Bugs, Security Frameworks, Pwn2Own, Aussie Sackings, Open Letter To Open AI, Vicuna, ChatGPT In Italy, Gen App Builder, Hugging GPT, Auto GPT, Baby GPT, Chat GPT Plugins, Reflexions Paper, Any Yet It Understands.

Releases

Anthos Config Management - 1.14.3. Fixed an issue where users are seeing an inaccurate PENDING instead of INSTALLED state on Policy Controller status.

Anthos clusters on bare metal - 1.14. Release 1.14.4 Anthos clusters on bare metal 1.14.4 is now available for download. Fixes: The following container image security vulnerabilities have been fixed: CVE-2021-3711 CVE-2021-3712 CVE-2021-40528 CVE-2022-23824 CVE-2022-42331 CVE-2022-42332 CVE-2022-42333 CVE-2022-42334 CVE-2022-48303 CVE-2023-0361 CVE-2023-23916. Known issues: For information about the latest known issues, see Anthos clusters on bare metal known issues in the Troubleshooting section.

Apigee X - On April 20, 2023 we released an updated version of Apigee. This release contains a new Advanced API Security Detected Traffic view, which displays information about API traffic originating from detected bots. On April 17, 2023, we released an updated version of Apigee X (1-9-0-apigee-25). Bug ID Description N/A Upgraded infrastructure and libraries.

AppEngine Flexible Java - Java 11 and 17 are now generally available.

AppEngine Standard Go - If you use the local development server to simulate an App Engine app in production, you must now run dev_appserver.py with Python 3 and set the CLOUDSDK_DEVAPPSERVER_PYTHON environment variable in your shell to the path of your Python 2 interpreter.

AppEngine Standard Java - Updated the Java SDK to version 2.0.13.

AppEngine Standard - If you use the local development server to simulate an App Engine app in production, you must now run dev_appserver.py with Python 3 and set the CLOUDSDK_DEVAPPSERVER_PYTHON environment variable in your shell to the path of your Python 2 interpreter.

Assured Workloads for Goverment - The FedRAMP Moderate compliance regime now supports the following products. The following compliance regimes now support the list of products below: Australia Regions with Assured Support Canada Regions and Support Canada Protected B Israel Regions and Support US Regions and Support The following products are now supported.

BigQuery - Updates to preferred tables for existing BI engine reservations now take up to ten seconds to propagate, down from five minutes.

BigTable - The Cloud Bigtable documentation has been updated to include guidance on deleting data.

Billing - Cloud Billing Reports and Cost Breakdown report now support CSV downloads For Cloud Billing Reports and Cost Breakdown reports, we have added the ability to download the data in the report table to a comma-separated values (CSV) file.

Certificate Manager - Certificate Manager now supports Mutual TLS (mTLS) authentication. The Certificate Authority Service integration feature is now generally available.

Chronicle - Chronicle enhanced the detection engine so that all rules have a value set to the $risk_score variable. Chronicle released the following additional data enrichment and precomputed analytic capabilities that can provide additional context during an investigation: Enriched entities with WHOIS data.

Cloud Composer - (Composer 2 only) Cloud Composer is now available in Taiwan (asia-east1), Jakarta (asia-southeast2), and Netherlands (europe-west4). Java Runtime in Airflow workers and schedulers is updated from version 11 to version 17. The apache-airflow-providers-google package in images with Airflow 2.3.4 and 2.4.3 was upgraded to 2023.4.13+composer. (Airflow 2.4.3 only) In environments with enabled data lineage integration, removed unnecessary warnings about deprecated operators that appeared in Airflow task logs. The Google Display and Video 360 API v.1.1 is deprecated and its sunset date is April 27, 2023. The Google Campaign Manager API v3.5 API is deprecated and its sunset date is May 1, 2023. Cloud Composer 2.1.13 images are available: composer-2.1.13-airflow-2.4.3 (default) composer-2.1.13-airflow-2.3.4. Cloud Composer versions 2.0.10 and 1.18.6, have reached their end of full support period.

Database Migration Service - Database Migration Service now supports Oracle multi-tenant (CDB/PDB) architecture.

Dataflow - Dataflow ML now supports the Automatic Model Refresh feature, which lets you update your machine learning model without stopping your Apache Beam pipeline.

Dataproc - Announcing Dataproc General Availability (GA) support for CMEK organization policy.

Datastream - Datastream now supports Oracle multi-tenant (CDB/PDB) architecture.

Cloud Data Loss Prevention - You can assign a sensitivity level to a built-in or custom infoType.

Eventarc - Support for creating triggers for direct events from Cloud Firestore is available in Preview.

Cloud Firestore - Eventarc events and Firestore events for Cloud Functions (2nd gen) now available in Preview.

Cloud Functions - There is a change in retry policy for 1st gen functions that use Pub/Sub subscriptions. Cloud Functions (2nd gen) has added support at the Preview release level for accepting requests from the Shared VPC network that a function is connected to, including when Ingress is configured as "Internal" or "Internal and Cloud Load Balancing.".

Cloud Healthcare API - The Cloud Healthcare API offers single-region support in the northamerica-northeast2 (Toronto, Canada) region.

KMS - Cloud HSM resources are now available in the following regions: europe-west12 me-central1 For information about which locations are supported by Cloud KMS, Cloud HSM, and Cloud EKM, see Cloud KMS locations.

Google Kubernetes Engine - (2023-R09) Version updates GKE cluster versions have been updated.

GKE - (2023-R09) Version updates Version 1.25.7-gke.1000 is now the default version.

Google Kubernetes Engine Rapid - (2023-R09) Version updates Version 1.26.3-gke.400 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - (2023-R09) Version updates Version 1.25.7-gke.1000 is now the default version in the Regular channel.

Google Kubernetes Engine Stable - (2023-R09) Version updates Version 1.24.11-gke.1000 is now available in the Stable channel.

Load Balancing - Typically with HTTPS communication, the authentication works only one way: the client verifies the identity of the server. Global external HTTP(S) load balancers now support proxying traffic to external backends outside Google Cloud.

Cloud Logging - You can now configure Log Analytics on Cloud Logging buckets and BigQuery linked datasets by using the following Terraform modules: google_logging_project_bucket_config google_logging_linked_dataset.

Recommender - New Service limit (quota) recommender is now available in Preview.

Resource Manager - You can now create dry-run organization policies using the Google Cloud console. You can now create tags that are children of projects as well as organization resources.

Cloud Run - Cloud Run integrations (Preview) are now available in europe-west1. Session affinity for Cloud Run service revisions is now at general availability (GA). A Cloud Run service revision will now accept requests from the Shared VPC network that it is connected to, including when Ingress is configured as "Internal" or "Internal and Cloud Load Balancing." (Preview).

Service Mesh - Managed Anthos Service Mesh. 1.14.6-asm.11 is now available for managed Anthos Service Mesh. 1.15.7-asm.1 is now available for managed Anthos Service Mesh. The Service dashboard now displays telemetry from external mesh services that have a canonical service label in the regular release channel. 1.14.x. In-cluster Anthos Service Mesh 1.14 is no longer supported. Managed Anthos Service Mesh. Enabling mesh.googleapis.com automatically enables trafficdirector.googleapis.com, networkservices.googleapis.com, and networksecurity.googleapis.com.

Anthos Service Mesh - Managed Anthos Service Mesh. 1.14.6-asm.11 is now available for managed Anthos Service Mesh. 1.15.7-asm.1 is now available for managed Anthos Service Mesh. The Service dashboard now displays telemetry from external mesh services that have a canonical service label in the regular release channel. 1.14.x. In-cluster Anthos Service Mesh 1.14 is no longer supported. Managed Anthos Service Mesh. Enabling mesh.googleapis.com automatically enables trafficdirector.googleapis.com, networkservices.googleapis.com, and networksecurity.googleapis.com.

SAP Solutions - Google Cloud's Agent for SAP version 1.3 Version 1.3 of the Google Cloud's Agent for SAP is now available.

Cloud SQL MySQL - Cloud SQL for MySQL now supports 40+ new database flags.

Cloud Storage - The Storage Insights inventory reports feature is now generally available. On July 17, 2023, the gcloud storage command-line tool will change some of the metadata it returns for buckets and objects, as well as change the format of some metadata names it returns.

Cloud Tasks - You can now create tasks by sending an HTTP request to your queue. For tasks that have HTTP targets (as opposed to App Engine targets), you can now set routing for tasks at the queue level.

Virtual Private Cloud - Private Service Connect backends support using an internal regional TCP proxy load balancer to access published services. Private Service Connect endpoints for published services can be configured with global access.