Welcome to issue #312 September 19th, 2022

News

Introducing Datastream for BigQuery - Serverless, seamless, and low-latency replication from relational databases directly to BigQuery, enabling real-time insights on operational data.

Try out Cloud Spanner databases at no cost with new free trial instances - Create a 90-day Spanner free trial instance with 10GB storage at no cost. Try Spanner free.

Introducing fine-grained access control for Cloud Spanner: A new way to protect your data in Spanner - Fine-grained access control for Cloud Spanner, now available in preview, provides table and column level protection for Spanner databases.

Golang’s database/sql driver support for Cloud Spanner is now Generally Available - Enabling Golang application developers to use Google Cloud Spanner as a database provider via database/sql driver support.

Introducing more ways to protect corporate applications with BeyondCorp Enterprise - Customers who use Microsoft Intune for device management can now integrate signal information for app access with BeyondCorp Enterprise and Workspace.

Announcing Advanced Polylines for the Maps SDKs for Android - As part of our ongoing efforts to enable developers to further style and customize the Google basemap, today we’re announcing the launch of advanced polyline styling for the Maps SDK for Android.

Announcing Pub/Sub metrics dashboards for improved observability - New Pub/Sub metrics dashboards are easily accessible from Google Cloud Console, helping you build better solutions with Pub/Sub.

Google + Mandiant: Transforming Security Operations and Incident Response - Google announced the completion of its acquisition of Mandiant, Inc. (NASDAQ: MNDT). Mandiant will join Google Cloud and retain the Mandiant brand.

Register for Google Cloud Next - Register now for Google Cloud Next ‘22, coming live to a city near you, as well as online and on demand.

Join us at the Google Government Summit in Washington D.C. - We’re excited to welcome the government community in person for a full day of networking, best practices and learning from peers to help accelerate the mission of government.

Sign up for the Google Cloud Fly Cup Challenge - Learn more about how to participate in the Google Cloud Fly Cup, brought to you in partnership with The Drone Racing League.

Test your skills in the Google Maps Platform Hackathon - Calling developers, innovators, data scientists, designers, educators, and enthusiasts - build and showcase your idea using Google Maps Platform.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Using Kubernetes Autoscaling to Optimise Availability and Cost - Here at Untienots, we run most of our workloads in Kubernetes which allows us to benefit from autoscaling to balance availability and cost.

GKE with NGINX Service Mesh - Implement NGINX Service Mesh with strict MutualTLS.

Using GCP Media CDN with private AWS storage buckets - This blog focuses on how Media CDN supports AWS Signature Version 4 to connect to private S3 buckets.

CI/GitOps with Helm, GitHub Actions, GitHub Container Registry and Config Sync - Since Anthos Config Management 1.13.0, Config Sync supports syncing Helm charts from private OCI registries. This article will demonstrate how you can package and push a Helm chart to GitHub Container Registry with GitHub actions (using PAT token), and then how you can deploy a Helm chart with Config Sync.

Not So Secure Default Setup of Google Cloud Platform - Have you ever wondered about the several configurations you see in your project that don’t seem configured by you?

App Development, Serverless, Databases, DevOps

Architecting for database encryption on Google Cloud - In this post, we provide a guide on how to accelerate your design considerations and decision making when securely migrating or building databases with the various encryption options supported on Google Cloud platform.

Introducing a modern cloud native approach for running Moodle at scale - Deploying Moodle on Google Cloud.

Come for the sample app, stay for the main course: Cloud Spanner free trial instances - Cloud Spanner now offers free trial instances with sample data and guided tutorials to try the fully managed relational database.

Migrate your most demanding enterprise PostgreSQL databases to AlloyDB for PostgreSQL with Database Migration Service - Database migration service now supports easy-to-use, secure, and serverless migration to AlloyDB for PostgreSQL.

Latest database innovations for transforming the customer experience - Google Cloud adds Spanner free trial instances and fine-grained access control, Datastream for BigQuery and PostgreSQL, database migrations to AlloyDB.

PostgreSQL on Google Cloud — AlloyDB, Cloud SQL, or Cloud Spanner - AlloyDB, Cloud SQL, or Cloud Spanner — what would meet your needs?

GitOps your service orchestrations - This blog post describes how to set up a simple Git-driven development, testing, and deployment pipeline for Workflows using Cloud Build.

How Einride scaled with serverless and re-architected the freight industry - Einride, a Swedish freight mobility company, is partnering with Google Cloud to reimagine the freight industry as we know it.

Implementing observability for always-on Ecommerce experience - Google Cloud provides extensive tools for logging and monitoring including support for open source platforms such as Prometheus and Grafana.

Accelerate migration from self-managed object storage to Cloud Storage - New Storage features accelerate migration from self-managed object storage to Cloud Storage.

Building a secure CI/CD pipeline using Google Cloud built-in services - Build a secure CI/CD pipeline using Google Cloud's built-in services using Cloud Build, Cloud Deploy, Artifact Registry, Binary Authorization and GKE.

Why I love Go - Learn all the reasons David Yach, industry veteran and Director of Engineering at Google Cloud, loves to use Go for software development.

How Google scales ad personalization with Bigtable - Learn how Google leverages Bigtable’s scale, performance and feature set to drive large-scale user modeling for ad personalization.

Long-term data protection with Persistent Disk Archive Snapshots - Persistent Disk Archive Snapshots provide a low-cost option for data protection that requires long-term retention, such as helping to support compliance, audit, or disaster recovery.

How to implement OS Login on Google Compute Engine - Configuring OS Login on Google Compute Engine.

Monitoring website changes with Workflows, Cloud Functions and SendGrid - Creating a simple system that detects web page changes.

Big Data, Analytics, ML&AI

How Let’s Enhance uses NVIDIA AI and GKE to power AI-based photo editing - Let’s Enhance uses AI to beautify images. GKE provides auto-provisioning, autoscaling and simplicity, while GPUs provide superior processing power.

Databases on Google Cloud Part 6: BigQuery and No-code SQL-only ML - An overview of BigQuery.

An automated way to handle failures in a streaming data pipeline - How to replay failed elements from an ingestion data pipeline based on a rule engine?

How to Create Remote Functions in BigQuery - Use Remote Functions to process and augment BigQuery data in unlimited ways.

Cloud Wisdom Weekly: 4 ways AI/ML boosts innovation and reduces costs - Whether ML models into production or injecting AI into operations, tech companies and startups want to do more with AI— and these tips can help.

Take your ML models from prototype to production with Vertex AI - Learn how to build, train, scale, and deploy machine learning models on Google Cloud with Vertex AI.

Recommendation Systems with Deep Learning - Digging deeper into how you can build a Recommendation System that uses Deep Learning instead of Matrix Multiplication.

Various

Drive digital transformation, get Cloud Digital Leader certified - Discover Google Cloud Digital Leader no-cost training and certification discount, and attend free webinar on September 15, 2022.

Google Cloud Partners With BNB Chain to Provide Web 3 Startup Infrastructure - The Internet has developed over the long haul with each phase introducing significant human civilization-defining innovations. Web3 is one….

Slides, Videos, Audio

GCP Podcast - #319 Storage Spotlight with Sean Derrington and Nishant Kohli.

Security Podcast - #83 What Does reCAPTCHA Actually Do and How Does It Do it? Product Manager Explains.

Releases

Anthos Config Management - 1.13.0. On October 31, 2022, we will introduce a new billing SKU for Policy Controller which is a part of Anthos Config Management. Anthos Config Management release 1.10 is no longer supported with the release of Anthos Config Management 1.13. The Config Sync feature to sync configurations stored as OCI images in Google Artifact Registry or Container Registry is generally available (GA). Config Sync is open sourced. Config Sync supports syncing from private Helm repositories (including OCI-based ones) as a preview feature. The Google Cloud Console now shows the sync status for all syncs in clusters registered to fleet. Config Sync now supports user-provided CA certificates for verifying HTTPS connections to Git servers. The constraint template library includes a new template: K8sStorageClass. The contraint template library's K8sEmptyDirHasSizeLimit template now supports regular expression matching of exempt volume names by using the new exemptVolumesRegex parameter. The contraint template library's K8sMemoryRequestEqualsLimit template now supports regular expression matching of exempt container names by using the new exemptContainersRegex parameter. Increased the resource-group-controller container memory request to 200Mi to avoid OOMKilled on Autopilot clusters. Stopped the nomos command line tool from panicing when nomos status is unable to fetch ResourceGroups. Fixed an issue that could cause accidental pruning when API resource discovery requests failed. Added optimizations to the reconciler and reconciler-manager to reduce API requests by avoiding unnecessary resource object updates. Config Sync now handles empty initContainers fields in managed configs correctly.

Google Cloud Armor - The following four libinjection signatures have been added to the sqli-v33-stable and xss-v33-stable rules: owasp-crs-v030301-id942100-sqli: SQL Injection Attack Detected via libinjection owasp-crs-v030301-id942101-sqli: SQL Injection Attack Detected via libinjection owasp-crs-v030301-id941100-xss: XSS Attack Detected via libinjection owasp-crs-v030301-id941101-xss: XSS Attack Detected via libinjection. Advanced rule tuning features for preconfigured WAF rules is now available in public preview.

Artifact Registry - tArtifact Registry is now available in the me-west1 region (Tel Aviv, Israel).

BigQuery ML - BigQuery ML is now available in the Madrid (europe-southwest1), Milan (europe-west8), and Paris (europe-southwest1) regions.

BigQuery - BigQuery Omni now supports the following quota and limit: The quota for total query result sizes for a project is now 1 TB per day. The BigQuery Data Transfer Service for Google Ads now supports the new Google Ads API. In addition to standard rounding, BigQuery now supports the rounding mode ROUND_HALF_EVEN for parameterized NUMERIC or BIGNUMERIC columns. BigQuery is now available in the Madrid (europe-southwest1), Milan (europe-west8), and Paris (europe-southwest1) regions. The Merge is coming! You may experience disruptions in the Ethereum public datasets in BigQuery. The is_case_insensitive schema option, which allows you to make a dataset and its table names case-insensitive, is now in preview. JDBC driver update, release 1.3.0 1001 You can now configure the connector to authenticate the connection using an external account (workforce or workload identity federation). ODBC driver update, release 2.5.0 1001 You can now configure the connector to authenticate the connection using an external account (workforce or workload identity federation), with limited support, using Azure AD and Okta identity providers. In Cloud Monitoring, you can view metrics for quota usage and limits of the Storage Write API's concurrent connections and throughput quotas.

BigTable - Cloud Bigtable is available in the me-west1 (Tel Aviv) region. A weekly digest of client library updates from across the Cloud SDK.

Billing - You can now estimate the cost of your workloads using the Cost Estimation API (Preview).

Cloud Build - Users can now use Cloud Build's GitHub Issues notifier to create issues in their GitHub repository in response to build events. Users can now use Cloud Console to configure a trigger to send build logs to GitHub or GitHub Enterprise. gcloud support for manual triggers is now available. Cloud Build now supports Supply chain Levels for Software Artifacts (SLSA) level 3 assurance.

Cloud Composer - Cloud Composer 1.19.9 and 2.0.26 release started on September 12, 2022. Encryption with customer-managed encryption keys (CMEK) now applies to the persistent disk of the environment's Redis queue. (Available without upgrading) Fixed a problem where the termination grace period for Airflow worker Pods in Cloud Composer 2 was set to 30 seconds (from 3600 seconds) after updating an environment. (Available without upgrading) When an environment is deleted, Cloud Composer automatically deletes the persistent disk of the environment's Redis queue. Set the logging level of Airflow's DeprecationWarning messages to Warning. (Available without upgrading) Cloud Composer now makes several attempts to create an environment when the Cloud Composer connection subnetwork is locked by another operation. Fixed a potential race condition in Airflow workers that could cause new tasks to be executed on a worker that is scheduled to be scaled down. Adjusted CPU limits for the FluentD environment component (responsible for uploading task logs to Cloud Logging), to avoid potential problems that might result in missing logs in Cloud Logging. The apache-airflow-providers-google package was upgraded to 2022.8.26+composer. Cloud Composer 1.19.9 and 2.0.26 images are available: composer-1.19.9-airflow-1.10.15 (default) composer-1.19.9-airflow-2.1.4 composer-1.19.9-airflow-2.2.5 composer-2.0.26-airflow-2.1.4 composer-2.0.26-airflow-2.2.5. Cloud Composer versions 1.17.0.preview.12, and 2.0.0-preview.0 have reached their end of full support period.

Compute Engine - Generally available: A new machine type for the memory-optimized-machine family called m2-hypermem-416 with 416 vCPUs and 8832 GB of memory. Generally available: NVIDIA® T4 GPUs are now available in the following region and zones in Middle East: Tel Aviv, Israel: me-west1-b,c. Generally available: Tel Aviv, Israel, Middle East me-west1-a,b,c has launched with E2 and N2 VMs available in all three zones, and M1 VMs in zones a and c.

Config Connector - Config Connector version 1.94.0 is now available. Added spec.memberFrom.sqlInstanceRef field to IAMPolicyMember (Issue #689). Added spec.bindings[].members[].memberFrom.sqlInstanceRef field to IAMPartialPolicy (Issue #689). Removed the validation on spec.cluster.numNodes > 0 in BigtableInstance (Issue #673). Added support for major version upgrades to SQLInstance (spec.databaseVersion is now mutable). Added spec.nodeConfig.reservationAffinity to ContainerCluster. Added spec.nodePoolAutoConfig to ContainerCluster. Added spec.nodeConfig.reservationAffinity to ContainerNodePool. Extended support for value absent in state-into-spec annotation to most Config Connector resources.

Database Migration Service - Database Migration Service now supports migrating PostgreSQL workloads into AlloyDB for PostgreSQL.

Dataflow - Dataflow is now available in Tel Aviv (me-west1).

Dataproc - Dataproc Serverless for Spark now uses runtime version 1.0.17 and 2.0.0-RC2.

Datastream - Datastream now supports BigQuery as a destination. Datastream now supports PostgreSQL as source. We have introduced changes to Datastream pricing.

Cloud Deploy - Google Cloud Deploy is now available in the following regions: asia-northeast2 (Osaka) asia-south1 (Mumbai) asia-south2 (Delhi) asia-southeast1 (Singapore) asia-southeast2 (Jakarta) australia-southeast2 (Melbourne) europe-central2 (Warsaw) europe-north1 (Finland) europe-west4 (Netherlands) europe-west6 (Zurich) northamerica-northeast2 (Toronto) southamerica-west1 (Santiago) us-west3 (Salt Lake City) us-west4 (Las Vegas). Deploying your application to Cloud Run is now supported in preview.

Document AI - Schema support for checkboxes and nested entitites Customers using Document AI Workbench, and processors for Purchase Order (PO), Invoice, or Expense, now have access to a new schema.

Eventarc - Eventarc support for direct events from Cloud IoT is available in Preview.

Cloud Functions - Cloud Functions has added support for a new runtime, .NET Core 6.0, at the Preview release level.

KMS - Cloud KMS is available in the following region: me-west1 For more information, see Cloud KMS locations.

Google Kubernetes Engine - CVE-2022-2068 has been patched in the Filestore CSI driver for GKE clusters running version 1.23 or later. Starting from GKE version 1.25 and gke-metrics-agent version 1.0.0, we increase the memory request and limit of gke-metrics-agent to 100 MiB. 1.25 is now available in the Rapid channel Kubernetes 1.25 is now available in the Rapid channel. Notable changes Support for the deprecated quobyte and storageOS volume types is removed in 1.25. Deprecated API versions These APIs are still served in version 1.25 but are in a deprecation period: The following Beta versions of graduated APIs will be removed in 1.26 in favor of newer versions: flowcontrol.apiserver.k8s.io/v1beta1 FlowSchema, PriorityLevelConfiguration deprecated since 1.23 use flowcontrol.apiserver.k8s.io/v1beta2 instead, available since 1.23 autoscaling/v2beta2 HorizontalPodAutoscaler deprecated since 1.23 use autoscaling/v2 instead, available since 1.23 (or autoscaling/v1) The following Beta versions of graduated APIs will be removed in 1.27 in favor of newer versions: storage.k8s.io/v1beta1 CSIStorageCapacity, deprecated since 1.24. Removed API versions PodSecurityPolicy policy/v1beta1 PodSecurityPolicy Deprecated in 1.21 and removed in 1.25. (2022-R22) Version updates GKE cluster versions have been updated. The me-west1 region in Tel Aviv, Israel is now available. On GKE Standard clusters using control plane version 1.24.2-gke.300 or later, you can configure the cluster and node pools to deploy an alternative version of the Logging agent designed to maximize logging throughput.

GKE - (2022-R22) Version updates Version 1.22.12-gke.300 is now the default version.

Google Kubernetes Engine Rapid - 1.25 is now available in the Rapid channel Kubernetes 1.25 is now available in the Rapid channel. Notable changes Support for the deprecated quobyte and storageOS volume types is removed in 1.25. Deprecated API versions These APIs are still served in version 1.25 but are in a deprecation period: The following Beta versions of graduated APIs will be removed in 1.26 in favor of newer versions: flowcontrol.apiserver.k8s.io/v1beta1 FlowSchema, PriorityLevelConfiguration deprecated since 1.23 use flowcontrol.apiserver.k8s.io/v1beta2 instead, available since 1.23 autoscaling/v2beta2 HorizontalPodAutoscaler deprecated since 1.23 use autoscaling/v2 instead, available since 1.23 (or autoscaling/v1) The following Beta versions of graduated APIs will be removed in 1.27 in favor of newer versions: storage.k8s.io/v1beta1 CSIStorageCapacity, deprecated since 1.24. Removed API versions PodSecurityPolicy policy/v1beta1 PodSecurityPolicy Deprecated in 1.21 and removed in 1.25. (2022-R22) Version updates Version 1.24.3-gke.900 is now the default version in the Rapid channel The following versions are now available in the Rapid channel: 1.21.14-gke.5300 1.22.13-gke.1000 1.23.10-gke.1000 1.24.4-gke.800 1.25.0-gke.1100 The following versions are no longer available in the Rapid channel: 1.20.15-gke.13700 1.21.14-gke.3000 1.22.12-gke.1200 1.23.9-gke.2100 1.24.3-gke.200 Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to version 1.21.14-gke.4300 with this release.

Google Kubernetes Engine Regular - (2022-R22) Version updates Version 1.22.12-gke.300 is now the default version in the Regular channel.

Google Kubernetes Engine Stable - (2022-R22) Version updates Version 1.1.22.12-gke.500 is now the default version in the Stable channel.

Cloud Logging - Support for adding custom indexed LogEntry fields to your Cloud Logging buckets is now Generally Available.

Marketplace Partners - You can now provide Private Offers with more granular discounts on specific usage metrics to your customers by enabling metric discounts.

Memorystore for Memcached - Added new Memorystore for Memcached region: Tel Aviv (me-west1).

Cloud Monitoring - You can now configure public uptime checks to send ICMP pings as part of the check.

Cloud VPN - Cloud VPN is now available in region me-west1 (Tel Aviv, Israel).

Cloud PubSub - BigQuery subscriptions support writing string fields in a Pub/Sub message to TIMESTAMP, DATETIME, DATE, or TIME columns in a BigQuery table. Pub/Sub is now available in me-west1 (Tel Aviv, Israel).

Pub/Sub Lite - Pub/Sub Lite is now available in Tel Aviv (me-west1).

Retail Recommendations AI - Bulk importing of historical Google Analytics 4 user events with BigQuery is generally available.

Cloud Run - Startup CPU boost for revisions is now available to provide additional CPU during container instance startup time. The following new region is now available: me-west1. You can now deploy Cloud Run services from Cloud Deploy (Preview). Startup healthcheck probes are now available (Preview).

Secret Manager - Cloud Secret Manager is now available in the following region: me-west1 For more information, see Secret Manager locations.

Cloud Spanner - Fine grained access control for Spanner is now available in public preview. The Spanner Golang database/sql driver is now generally available. You can create Cloud Spanner regional instances in Tel Aviv, Israel (me-west1).

Cloud SQL Postgres - Cloud SQL read replicas now follow the maintenance settings for the primary instance, including the maintenance window, rescheduling, and the deny maintenance period. Support for me-west1 (Tel Aviv). Cascading Replicas is now generally available for Cloud SQL.

Cloud Storage - Cloud Storage is now available in Tel Aviv, Israel (me-west1 region). gcloud storage is now in GA gcloud storage provides faster uploading and downloading performance when compared to the gsutil command line tool.

Cloud TPU - Cloud TPU now supports Tensorflow 2.10.0.

Transfer Appliance - Users can now review data that successfully transferred and failed to transfer in log files that auto-generate after a transfer is completed.

Virtual Private Cloud - For auto mode VPC networks, added a new subnet 10.208.0.0/20 for the Tel Aviv me-west1 region.

Cloud VPN - Cloud VPN is now available in region me-west1 (Tel Aviv, Israel).

Workflows - Execution results include the current or final step of the workflow execution.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko