Welcome to issue #301 July 4th, 2022

News

Introducing Google Public Sector - Announcing Google Public Sector, a new Google division that will focus on helping U.S. public sector institutions accelerate their digital transformations.

Introducing Earth Engine for governments and businesses - Google Earth Engine available to businesses and governments worldwide as an enterprise-grade service through Google Cloud.

Bonjour Paris: New Google Cloud region in France is now open - The Google Cloud region in Paris, France gives users low-latency access to services while meeting compliance, privacy, and digital sovereignty needs.

Announcing Apigee Advanced API Security for Google Cloud - The new Advanced API Security is a paid add-on to Apigee’s API management solution that enables developers to further strengthen their API security by more easily detecting and mitigating security threats.

IP Masquerading and eBPF are now in GKE Autopilot - GKE Autopilot has new networking GA features: Egress NAT policy for IP masquerading and the eBPF-based Dataplane V2 for Network Policy and Network Policy Logging.

Introducing Query Insights for Cloud Spanner: troubleshoot performance issues with pre-built dashboards - Spanner’s ‘Query insights’ - a new tool that makes it easy to debug query performance issues.

Now in preview, BigQuery BI Engine Preferred Tables - Launching preferred tables in preview for BigQuery BI Engine, enabling customers to achieve predictable performance and optimized use of their BigQuery investments.

Announcing general availability of Cloud Armor’s new edge security policies, and support for proxy load balancers - Google Cloud expands its scope of DDoS and web application firewall protection with new edge security policies and proxy load balancers.

Introducing new Cloud Armor features including rate limiting, adaptive protection, and bot defense - Cloud Armor strengthens its already formidable defenses with new features to counter advanced L7 attacks and block malicious bots.

More support for structured logs in new version of Go logging library - Learn how to enrich your Go applications observability using Google logging client.

Jumpstart your location experiences with new integrations from across Google - Google Maps Platform developers at small businesses and large companies alike have access to new features and integrations with other Google products to enhance their end-user experiences and back-end operations.

Google Cloud announces new products, partners and programs to accelerate sustainable transformations - In advance of the Google Cloud Sustainability Summit, we announced new programs and tools to help drive sustainable digital transformation.

Billing info is at your fingertips in the latest Cloud Console mobile app - You can check your bill, credit usage and forecasts in the latest version of the Cloud Console mobile app.

Earn Google Cloud swag when you complete the #LearnToEarn challenge - Earn swag with the Google Cloud #LearnToEarn challenge.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Announcing MITRE ATT&CK mappings for Google Cloud security capabilities - Google Cloud now supports improved, threat-informed defenses by mapping our native security capabilities to MITRE ATT&CK.

What GKE users need to know about Kubernetes' new service account tokens - Learn about Kubernetes' new tokens that arrived in Kubernetes 1.21.

CISO Perspectives: June 2022 - Google Cloud CISO Phil Venables shares his thoughts on the RSA Conference and the latest security updates from the Google Cybersecurity Action Team.

Upload public keys for GCP service accounts with terraform - This article shows how to manage IAM Service Account Keys by manually generating an RSA key pair and deploying it with Terraform to GCP SA.

Network Visibility : Understanding ‘GCP to Internet’ Latency - Getting 'Google Cloud to Internet endpoint' round trip numbers for workloads hosted on Google Cloud.

Schedule GKE scale down to Zero Nodes - Resize GKE using cloud function and cloud scheduler.

App Development, Serverless, Databases, DevOps

Cloud Monitoring metrics, now in Managed Service for Prometheus - Query over 1,000 free Google Cloud metrics using PromQL. You can now view your Cloud Monitoring metrics alongside your Prometheus metrics.

Secure Supply Chain on Google Cloud - A Sketchnote about Secure Supply Chain on Google Cloud.

Incorporating quota regression detection into your release pipeline - Check quotas across cloud environments before promoting images to prevent outages due to inconsistent API quota limits.

How to migrate a group of individual instances to a stateful MIG using Python script - Requirements and examples of python script usage for automated and safe instance migration between configurations.

Change Streams in Cloud Spanner | Replication to BigQuery - Replicating data from Cloud Spanner to BigQuery.

Spring Cloud GCP — Cloud Spanner - An intro using Cloud Spanner with Java Spring framework.

This might be the cleanest way I’ve seen parallel processing done in a (serverless) workflow engine - Examples of using parallel processing in Cloud Workflows.

Password Policies with CloudSQL for PostgreSQL. - This blog will walkthrough on all available options as policies that can be applied at instance level and cases it won’t be enforced.

It’s business time: One good use of the GCS Custom-Time field - An overview and usage of Custom-Time field in Cloud Storage objects.

Trigger gsutil with Watchman - Sync local files to Cloud Storage bucket.

Big Data, Analytics, ML&AI

Twitter: gaining insights from Tweets with an API for Google Cloud - The Twitter API toolkit for Google Cloud enables data processing automation at the click of a button because it provisions the underlying infrastructure it needs to work.

Cloud TPU v4 records fastest training times on five MLPerf 2.0 benchmarks - Cloud TPU v4 ML supercomputers set performance records on five MLPerf 2.0 benchmarks.

Enterprise Data Integration with Data Fusion - What is Cloud Data Fusion? A Fully managed, cloud-native enterprise data integration at any scale.

Real-time IoT data from Mapify to BigQuery - Step-by-step guide on how to build a solution in which Mapify handles real-time IoT sensor data and streams it to BigQuery.

Tell Me, BigQuery: What is Trending on Google? - Exploring and enriching the international BigQuery Google Trends dataset with Looker.

How to Split and Sample a Dataset in BigQuery Using SQL - Easily segment your data into training, validation, and test sets.

Trick: Almost-Free Jupyter Notebooks on the Cloud! - A cheaper alternative to Vertex AI Workbench managed notebooks.

A General Framework for Machine Learning Pipelines on GCP - A blueprint for developing Vertex AI pipelines.

Various

Have You Yet..? - And so the much anticipated Learn to Earn Cloud Data Challenge 2022 is almost here!

Slides, Videos, Audio

GCP Podcast - #310 Disaster Recovery with Cody Ault and Jo-Anne Bourne.

Kubernetes Podcast - #183 Consulting, with Steve Wade.

Security Podcast - #72 What Does Good Detection and Response Look Like in the Cloud? Insights from Expel MDR.

Releases

Anthos Config Management - 1.12.0. Shell access is disabled by default in the Config Sync hydration-controller container. Policy Controller now supports Cloud Monitoring. Anthos Config Management is now compatible with GKE Autopilot with some cluster requirements. Config Sync supports syncing configurations stored as OCI images in Google Artifact Registry or Container Registry as a preview feature. Added a field spec.override.reconcileTimeout in RootSync and RepoSync, for configuring the threshold for how long to wait for resources in an apply group to reconcile before giving up. The constraint template library includes a new template: K8sRequiredResources. The template library's K8sProhibitRoleWildcardAccess template now supports regular expression matching of clusterRole names by using the new regexMatch field. The template library's K8sNoExternalServices template supports a new field: cloudPlatform. Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 206bbe9). This release includes several Config Sync performance improvements: Config Sync reconciler now watches resources for status updates instead of polling, leading to faster, more responsive, and more efficient detection of object failure and reconciliation. Config Sync removed resource limits from reconciler-manager, reconciler, and git-importer to make them burstable. Config Sync increased resource limits of admission-webhook (cpu: 1, memory: 2Gi) and otel-agent (cpu: 1, memory: 1Gi). Fixed the issue causing the Config Sync webhook deployment to report readiness prematurely by adding a readiness probe that waits for the TLS certificate to be generated and injected by the sidecar. Fixed the issue causing git-importer to wipe out non-blocking validation errors before retrying. Fixed the issue causing reconciler to throw an error when deleting an object that was already deleted. Fixed two issues when resources are marked unmanaged using the configmanagement.gke.io/managed: disabled annotation: If a repo contains unmanaged resources on the initial sync, the sync fails.

Anthos clusters on bare metal - 1.12. Release 1.12.0 Anthos clusters on bare metal 1.12.0 is now available for download. The dockershim component in Kubernetes enables cluster nodes to use the Docker Engine container runtime. Improved cluster lifecycle functionalities: Upgraded Anthos clusters on bare metal to use Kubernetes version 1.23. Known issues: For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Google Cloud Armor - Google Cloud Armor now supports TCP Proxy load balancers and SSL proxy load balancers in General Availability For more information, see the security policy overview. Advanced network DDoS protection is now available for network load balancers, protocol forwarding, and VMs with public IP addresses in public preview.

BeyondCorp Enterprise - The BeyondCorp Enterprise client connector is now generally available.

BigQuery - An updated version of JDBC driver for BigQuery is now available. You can now set default configurations at a project or organization level. You can now set the view field in the tables.get() API method to indicate which table information is returned. Previously, all BigQuery BI Engine projects had a maximum reservation size per project per location limit of 100 GB.

BigTable - You are now able to configure the storage utilization target for a cluster when you use autoscaling for Cloud Bigtable. Cloud Bigtable now gives you the option to undelete a table for up to seven days from the time of deletion using the gcloud CLI.

Billing - Preview: View your Google Kubernetes Engine (GKE) costs in Cloud Billing reports and cost data export to BigQuery You can view your GKE costs by cluster, namespace, and pod labels in the Detailed cost export, and the built-in reports in the Google Cloud console. We've added new features to view your billing information and cost estimates in the Google Cloud Console mobile app. Attribution for your committed use discounts (CUDs) now appears at the same time as eligible usage.

Cloud Build - Regional support for default pools and build triggers is now generally available.

Chronicle - Chronicle Forwarder configuration on Linux has been updated to include two separate configuration files.

Cloud Composer - Cloud Composer supports Per-folder Roles Registration.

Compute Engine - Generally available: You can now create shared reservations of Compute Engine zonal resources using the Google Cloud Console. GA: You can now use the SSH troubleshooting tool from the Cloud console to help you determine the cause of failed SSH connections.

Dataproc Metastore - v1. Metadata federation is generally available (GA). Private Service Connect for Dataproc Metastore is generally available (GA).

Dataproc - New sub-minor versions of Dataproc images: 1.5.71-debian10, 1.5.71-rocky8, 1.5.71-ubuntu18 2.0.45-debian10, 2.0.45-rocky8, 2.0.45-ubuntu18. For 1.5 images and the 2.0.45-ubuntu18 image, backported the upstream fix for KNOX-1997.

Datastore - Not-equal (!=), IN, and NOT_IN query filters now available in all client libraries: Java Python PHP Node.js C# Go Ruby Note: This feature is available for Firestore in Datastore mode.

Cloud Deploy - Google Cloud Deploy is now available in the following regions: asia-east2 (Hong Kong) europe-west2 (London) europe-west3 (Frankfurt) us-east4 (N. The ability to deploy to Anthos user clusters is now generally available.

Eventarc - Eventarc support for Customer-Managed Encryption Keys (CMEK) using Google Cloud CLI is available in Preview. Eventarc is available in this region: europe-southwest1 (Madrid, Spain). Support for Firebase Realtime Database is in Preview.

Cloud Filestore - Filestore High Scale SSD tier is generally available (GA).

Cloud Functions - Cloud Functions now supports Python 3.10 at the General Availability release level. Cloud Functions now supports Java 17 at the General Availability release level.

IAM - In June 2022, IAM had an issue that resulted in excess usage metrics for service accounts and service account keys when any of the following actions were performed: Listing service account keys Getting a service account key Disabling a service account key Enabling a service account key Each time you took any of these actions, Cloud Monitoring recorded an authentication usage metric for the parent service account, and for each of its service account keys, regardless of whether you used the service account or its keys to authenticate.

Identity Platform - Identity Platform Web v9 modular SDK is now available at the GA stage.

KMS - Customers enrolled in Key Access Justifications will now see justifications listed in Cloud Audit Logs for Cloud KMS.

Google Kubernetes Engine - (2022-R16) Version updates GKE cluster versions have been updated. A new vulnerability (CVE-2022-1786) has been discovered in the Linux kernel versions 5.10 and 5.11. GKE Cost Allocation has been released for public preview. You can now give multiple containers time-shared access to the full compute resources of a single NVIDIA GPU accelerator.

GKE - (2022-R16) Version updates Control plane and node version 1.24.1-gke.1800 is now available.

Google Kubernetes Engine Rapid - (2022-R16) Version updates Version 1.23.7-gke.1400 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - (2022-R16) Version updates The following versions are now available in the Regular channel: 1.20.15-gke.8700 1.21.12-gke.2200 The following versions are no longer available in the Regular channel: 1.20.15-gke.8200 1.21.12-gke.1500 Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to version 1.21.12-gke.1700 with this release.

Google Kubernetes Engine Stable - (2022-R16) Version updates Version 1.21.12-gke.1500 is now the default version in the Stable channel.

Cloud Logging - You can now collect Apache Flink logs from the Ops Agent, starting with version 2.17.0. A weekly digest of client library updates from across the Cloud SDK. The Cloud Logging agent version 1-18 for Windows is now available.

Managed Microsoft AD - Support for schema extensions in Managed Microsoft AD is available for Preview.

Cloud Monitoring - Managed Service for Prometheus: You can now query Cloud Monitoring metrics by using PromQL. The new experience for creating metric-based alerting policies by using the Google Cloud console is now Generally Available.

Cloud VPN - Cloud VPN no longer checks a peer's IKE identity.

reCAPTCHA Enterprise - You can now enable platform logging for reCAPTCHA Enterprise API calls.

Security Command Center - The contacts and indicator.signatures attributes were added to the Finding object.

Cloud Spanner - The ANALYZE DDL command allows administrators to manually update the query statistics package that the optimizer uses to build query execution plans. Query insights is now generally available.

Cloud SQL - A second June maintenance changelog is now available. Cloud SQL for MySQL supports in-place major version upgrades in Preview.

Cloud Storage Transfer - Expanded overwrite options are new generally available (GA). Metadata preservation options are now generally available (GA).

Cloud Storage - Object Lifecycle Management now supports new conditions and a new action. The XML API now supports setting a default Cloud KMS key on a bucket when creating the bucket.

Transfer Appliance - Transfer Appliance now supports monitoring of the amount of data stored on your appliance, and whether online transfer is enabled, through Cloud Monitoring.

Vertex AI - Vertex AI Forecasting is available in GA.

VPC Service Controls - Support to add individual VPC networks to a perimeter is now available in Preview.

Cloud VPN - Cloud VPN no longer checks a peer's IKE identity.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko