Welcome to issue #345 May 8th, 2023

News

Introducing Active Assist recommendations for service limits (quotas) - Active Assist’s service limit (quota) recommender provides actionable and automatic recommendations to review quotas that have high utilization.

Introducing Organization Restrictions, a new way to keep threat actors out - Now you can restrict access to only-authorized Google Cloud organizations by using Organization Restrictions.

Managing fine-grained access at Spanner scale - Protect mission-critical Spanner databases with fine-grained access permissions.

New asset query simplifies asset inventory management in Security Command Center - Security Command Center users can now perform SQL-like queries to get detailed information on where assets are located and how they are configured.

Extending Zero Trust access to multi-cloud applications - Google Cloud now makes it even easier to add Zero Trust security to applications in multi-cloud environments. Here’s how it works.

3 new ways to authorize users to your private workloads on Cloud Run - Identity Aware Proxy, Regional Internal Load Balancer, and Shared VPC Ingress for Cloud Run offer new design patterns for internal apps.

Introducing Query plan samples for Cloud Spanner: get performance insights from query execution plans - Cloud Spanner’s new Query plan samples let developers and DBAs visualize query execution plans for historical queries.

Introducing Workload Manager: Maximize reliability and performance by automating best practices - Google Cloud Workload Manager helps you run workloads against best practices, to improve reliability, performance, and overall system quality.

Running ML models now easier with new Dataflow ML innovations on Apache Beam - Dataflows ML features extended with Automatic Model Refresh, TensorFlowHub integration and new supported framework provided by Apache Beam.

Announcing Cloud Storage FUSE and GKE CSI driver for AI/ML workloads - Now in Preview, Cloud Storage FUSE CSI driver lets you access objects in buckets as files mounted as a local file system in GKE.

---

Meet DoiT

The true promise of the cloud with ease, not cost. DoiT provides technology and cloud expertise to reduce cloud costs and boost engineer productivity. All from a Google Cloud Partner. Learn more

---

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

When one becomes two: Resource hierarchy strategies for divested organization - Google Cloud resource hierarchy considerations to ensure a smooth separation and maintain business continuity when breaking apart a company.

A recovering CIO’s perspective on cloud migrations and our revamped Rapid Migration Program, RaMP - The revamped Rapid Migration Program, or RaMP, codifies migration best practices learned in over 10 years of helping customers move to the cloud.

Backup for GKE - concepts - Part 1 - how it works - Automate your kubernetes backup and restore for better resiliency and disaster response.

Increasing Resiliency with Load Balancers - You can customize Cloud Load Balancing to handle the reliability, failover and sharding that you need.

Internet-facing application delivery: Networking Architecture - Explore multiple cloud and hybrid architectures for securely hosting your internet-accessible cloud applications.

Framing up FinOps: All about Google Cloud billing tools - In this recap of the latest Framing up FinOps podcast, learn how cloud billing tools can help inform and optimize your Cloud FinOps practice.

Minimize Cloud Outage Risk By Proactively Monitoring Your Quotas - Protect your cloud investment from unplanned outages and costs by leveraging GCP’s Quota Monitoring Solution (QMS).

Service Account Impersonation in Google Cloud - This post explains how to use short-lived keys and service account impersonation to avoid service account key generation for CLI and Terraform usage.

Github Action CI/CD to deploy applications on GKE using Workload Identity Federation. - This blog demonstrates creating a Github Action CI/CD to push images to Google Container Registry and deploy applications in GKE using Workload identity federation.

Configuring ArgoCD on GKE with Ingress and GitHub SSO - Configuring ArgoCD with Ingress (GKE) integrated with GitHub SSO to authenticate users.

App Development, Serverless, Databases, DevOps

BBC: Keeping up with a busy news day with an end-to-end serverless architecture - The BBC built its log-processing infrastructure on Google Cloud serverless tools including Cloud Run and BigQuery.

Effingo: the internal Google copy service moving data at scale - Google uses its Effingo data copy service to move data at global scale for data replication, durability, and latency purposes.

Using Cloud Spanner to handle high throughput writes - Four approaches to a database that needs many writes or reads, such as social media view counts.

From receipts to riches: Save money w/ Google Cloud & supermarket bills - Part 1 - Doc AI, Cloud Functions, BigQuery, Datastore, Storage and Logging work together to help you understand where your money goes.

Buffer HTTP requests with Cloud Tasks - Use the new BufferTask API to make integration easier for arbitrary HTTP backends.

Evaluating the true cost or TCO of a database — and how Cloud Spanner compares - Cloud Spanner databases offer high performance at lower costs by providing a fully managed experience with unlimited scalability and high availability.

Scalable electronic trading on Google Cloud: A business case with BidFX - BidFX Liquidity Provision Analytics (“LPA”) offers skew detection, execution time optimization, pricing comparison, and top of book analysis.

Right way to alert on aggregated logs in Google Cloud - Setting destination project for log-based alerts.

Automated Canary Deployment with Post-Deployment Verification on GCP CloudRun using Google Cloud Deploy for continuous delivery - This blog post explores the Canary Deployment strategies offered by Google Cloud Deploy, including their strengths, limitations, and optimal use cases.

Evaluating your GCP resource realtime - How to build a service to validate GCP resource from CAI Feed.

Big Data, Analytics, ML&AI

All data cloud, all the time: Recapping the Google Data Cloud & AI Summit - This year’s Google Data Cloud & AI Summit featured new product announcements, customer stories, and countless learning opportunities.

Jumpstart Your BigQuery Remote Function development today - Use remote functions to handle DLP, unstructured data analysis and security or compliance constraints inside your BigQuery dataset.

ELT Batch pipeline with Cloud Storage, BigQuery orchestrated by Airflow/Composer - The goal of this article is showing a real world use case for ELT batch pipeline, with Cloud Storage, BigQuery, Apache Airflow and Cloud Composer.

BigQuery — keep fresh data while avoiding large-scale mutations - Avoid merge or join and use deduplication and clone in large dataset updates.

Seeing the World: Vertex AI Vision Developer Toolkit - In this blog, we show how developers can build computer vision applications with Vertex AI Vision.

Google Cloud and Equinix: Building Excellence in ML Operations (MLOps) - Improving architecture, governance, error correction and cost optimization with smarter ML Ops.

How OPPO enhances AI capabilities on mobile devices with Google Vertex AI - Google Cloud helps OPPO to develop AI models with Google Vertex AI NAS.

BigQuery ML and Looker: Meeting the Predictive Analytics Challenge - Integrating Machine Learning models at scale into data visualizations and dashboards with BigQuery ML and Looker.

Various

Google Cloud Partners with Code of Support to help veterans & caregivers build cloud computing skills - Code of Support Foundation (COSF) announces a new partnership with Google Cloud to provide complimentary Google Cloud Skills Boost access to 500 veterans and caregivers.

Meet our Data Champions: Credit Karma’s Scott Wong on doing 60 billion model predictions per day - Credit Karma uses Google services like Cloud Bigtable and BigQuery to power financial recommendations for nearly 130 million members.

Slides, Videos, Audio

Security Podcast - #119 RSA 2023 - What We Saw, What We Learned, and What We're Excited About.

Releases

AlloyDB - AlloyDB Omni version alloydb-omni-0.2.0-preview-postgresql-14.4 is available.

Anthos Config Management - 1.15.0. The spec.git fields of the ConfigManagement object are deprecated and are scheduled for shut down on or after May 15, 2024. 1.15.0. The constraint template library's K8sEnforceConfigManagement template adds new requireDriftPrevention and requireRootSync parameters, which requires enabling referential constraints. The constraint template library includes a new template: K8sContainerEphemeralStorageLimit. The constraint template library includes a new template: K8sDisallowedRepos. The constraint template library includes a new template: K8sRestrictNfsUrls. Added new metric labels: commit and type. Added a --name flag to nomos status to support filtering status by RootSync or RepoSync names. Changed error message ResourceFightWarning to ResourceFightError so that resource fighting conflict can be exposed as errors in nomos status and RootSync/RepoSync status. Upgraded bundled Kustomize version from v4.5.2 to v5.0.1. Upgraded bundled Helm version from v3.6.3 to v3.11.2. Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: effa347).

Anthos clusters on Azure - You can now launch clusters with the following Kubernetes versions: 1.24.11-gke.1000 1.25.7-gke.1000 1.26.2-gke.1001. Updated OS image to Ubuntu 22.04. Kubernetes 1.26.2 will incorrectly apply the default StorageClass to PersistentVolumeClaims which have the deprecated annotation volume.beta.kubernetes.io/storage-class. This release fixes the following vulnerability: CVE-2023-0461.

Anthos clusters on VMware - Anthos clusters on VMware 1.15.0-gke.581 is now available. Preview: Support for vSphere 8.0 Preview: Support for VM-Host affinity for user cluster node pools Preview: Support for High availability control plane for admin clusters Preview: Support for system metrics collection using Google Cloud Managed Service for Prometheus Preview: You can now filter application logs by namespace, Pod labels and content regex. CSI migration for the vSphere storage driver is enabled by default. Admin cluster update operations are now managed by an admin cluster controller. Deprecations Support for gkeadm on MAC and Windows is deprecated. Fixed the false error message generated by the cluster autoscaler about a missing ClusterRoleBinding. Fixed the following vulnerabilities: Critical container vulnerabilities: CVE-2022-32221 CVE-2022-47629 CVE-2021-46848 CVE-2022-41903 CVE-2022-23521 High-severity container vulnerabilities: CVE-2022-3094 CVE-2023-23916 CVE-2022-42898 CVE-2021-3449 CVE-2023-26604 CVE-2023-23946 CVE-2022-39260 CVE-2022-3970 CVE-2022-23218 CVE-2022-23219 CVE-2021-3999 CVE-2019-25013 CVE-2021-33574 Container-optimized OS vulnerabilities: CVE-2023-28466 CVE-2023-0461 CVE-2020-17437 CVE-2022-32149 CVE-2022-40320 CVE-2019-18276 CVE-2022-40304 Ubuntu vulnerabilities: CVE-2022-4203 CVE-2022-4304 CVE-2022-4450 CVE-2023-0215 CVE-2023-0216 CVE-2023-0217 CVE-2023-0286 CVE-2023-0401 CVE-2022-28321 CVE-2022-3328. Known issues: You might see a false error message about vCenter.dataDisk. Anthos clusters on VMware 1.14.4-gke.54 is now available. Added admin cluster CA certificate validation to the admin cluster upgrade preflight check. Fixed an issue where the Connect Agent continued using the older image after registry credential update. Fixed the following vulnerabilities: High-severity container vulnerabilities: CVE-2023-26604 CVE-2023-0361 CVE-2022-29154 Container-optimized OS vulnerabilities: CVE-2023-0386 CVE-2023-23916 CVE-2023-0464 CVE-2023-27561 CVE-2022-40320 CVE-2023-1652 CVE-2023-28466.

AppEngine Standard - Memory limits for second-generation runtimes have been increased to better support the growing memory utilization of many newer runtimes.

BigQuery - The INSERT INTO SELECT statement now lets you filter data from files in Amazon S3 and Azure Blob Storage and append it into BigQuery tables. You can now use configuration YAML files to transform SQL code when you translate SQL queries from your source database. The table clones feature of BigQuery is now generally available (GA). You can now add descriptions to the columns of a view. If you use query queues, then you can set the interactive and batch queue timeouts in your default configuration.

Chronicle - Chronicle made the following changes to the detection engine rules and YARA-L language: Expanded support for arithmetic operations. Exclusions for Curated Detections You can now configure exclusions to more finely tune the results of the Curated Detections provided by the Google Cloud Threat Intelligence (GCTI) team. The following supported default parsers have changed. UDM Search Pivot Table The UDM Search Pivot Table enables you to further analyze your UDM search results, giving you the following capabilities: Group search results by up to five UDM fields.

Database Migration Service - Database Migration Service now supports faster migrations from PostgreSQL source databases to a destination Cloud SQL for PostgreSQL instance.

Deep Learning Containers - M108 release Miscellaneous software updates.

Deep Learning VM - M108 release The image name common-container-experimental was changed to common-container.

Cloud Deploy - You can now perform deployment verification in the same cluster where your application is running (GKE and Anthos only).

Dialogflow - Dialogflow CX now provides the ADD_DATE system function.

Cloud Data Loss Prevention - The discovery service can now generate the following observation finding types in Security Command Center: Data sensitivity Data risk These findings provide the calculated sensitivity and data risk levels of the BigQuery tables that you profile.

Google Kubernetes Engine - In GKE version 1.26, for VPC peering-based private clusters that were created after 2020-08, the Konnectivity service will be initialized but not used. The managed Cloud Storage FUSE CSI driver for GKE is now available in Preview in GKE versions 1.26.3 and later. We're working on automatically enabling the PD CSI Driver on upgrades to 1.25, for clusters with the add-on disabled.

Cloud Monitoring - Observability for Google Kubernetes Engine: You can now enable GKE control plane metrics from the Observability tab for your GKE cluster.

reCAPTCHA Enterprise - Users can now see how reCAPTCHA Enterprise works on the Google Cloud console. reCAPTCHA Enterprise Mobile SDK v18.2.0 is now available for iOS.

Cloud Run - CPU allocation recommender now automatically recommends CPU allocation changes based on traffic received by your Cloud Run service over the past month.

Security Command Center - An issue that affected the display of the counts of controls for certain CIS Google Cloud Platform Benchmark (CIS Benchmark) reports in the Google Cloud console has been fixed.

Service Mesh - Managed Anthos Service Mesh. The managed data plane is enabled on by default in the regular and rapid channels.

Anthos Service Mesh - Managed Anthos Service Mesh. The managed data plane is enabled on by default in the regular and rapid channels.

SAP Solutions - Version 1.5 of the Google Cloud's Agent for SAP is now available.

Cloud Spanner - Cloud Spanner now supports new query capabilities for PostgreSQL dialect databases: Set operations (such as UNION and INTERSECT) with ORDER BY, LIMIT, or OFFSET, or in subqueries Parameterized LIMIT and OFFSET operations Statement hints for configuring the query optimizer (such as optimizer_version and optimizer_statistics_package). Cloud Spanner sampled query plans are now available in Preview.

Cloud SQL Postgres - Fast migration for Cloud SQL is now available.

Cloud SQL SQL Server - You can now disable simultaneous multithreading (SMT) while creating or editing instances and read replicas.

Vertex AI - M108 release The M108 release of Vertex AI Workbench user-managed notebooks includes the following: Miscellaneous software updates.

VMware Engine - After installing Windows Server 2022 update KB5022842 (OS Build 20348.1547), guest OS can not boot up when virtual machine(s) is configured with secure boot enabled.

Workflows - The Cloud Workflows service agent has the ability to consume quota and billing for a project through the serviceusage.services.use permission.