Welcome to issue #298 June 13th, 2022

News

The new Google Cloud region in Dallas, Texas is now open - Google Cloud’s Dallas, Texas region is now open, bringing the first region to Texas, for a total of 34 regions across the globe.

Accelerating BigQuery migrations with automated SQL translation - Readable translations of legacy SQL queries across a wide breadth of data warehouses with just a push of a button.

Accelerate the deployment of ML in production with Vertex AI - Google Cloud expands Vertex AI to help customers accelerate deployment of ML models into production.

Reimagining AutoML with Google research: announcing Vertex AI Tabular Workflows - Google Cloud announces Vertex AI Tablar Workflows, expanding AutoML and building on research like TabNet.

Building AI in the cloud: An easier way with Google Cloud and NVIDIA - The One Click Deploy feature on the NVIDIA NGC catalog lets data scientists deploy directly to Google Cloud’s Vertex AI Workbench.

WebGL-powered maps features now generally available

Introducing the Google Workspace for Government Demo Series - Google Workspace for Government Demo series showcases how our tools can help agencies collaborate and work together more securely.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

How Redbox improved service resilience and availability across clusters with GKE and Anthos multicluster ingress - Redbox guides us through their journey of achieving multi regional availability for their business critical services and how they landed on Anthos.

Infrastructure Security in Google Cloud - Google Cloud’s approach to infrastructure security is unique. Google doesn’t rely on any single technology to secure its infrastructure. Rather, it has built security through progressive layers that deliver defense in depth.

How Google Cloud can help secure your software supply chain - Google Cloud just introduced its new Assured OSS service. Here’s how it can help secure your software supply chain.

Connecting Apigee to GKE using headless services and Cloud DNS - One of the biggest challenges connecting Apigee to backend services running in a GKE cluster is to avoid duplication of processing, while performing at scale. In this blog post we propose a new solution that takes advantage of GKE VPC native clusters, headless services and Cloud DNS.

Updates coming for Authorized Networks and Cloud Run/Functions on GKE

How We Deal with a Google Kubernetes Engine (GKE) Metadata Server Outage - Why the GKE metadata server failed to work, and how we fixed it.

Tutorial 1 : Setup Web Application Security Protection and Detection Lab in Google Cloud-WAF tuning - A four-part series walks you through creating a lab environment for testing against Google cloud WAF’s protections.

App Development, Serverless, Databases, DevOps

Learn how to tackle supply chain disruptions with SAP IBP and Google Cloud - SAP IBP now integrated with Google Cloud for faster, more accurate forecasting to navigate challenges with supply chain disruptions.

Even more pi in the sky: Calculating 100 trillion digits of pi on Google Cloud - Compute Engine improvements like the N2 machine family and 100 Gbps egress bandwidth allowed us to calculate 100 trillion digits of pi—a world record.

How to Connect to GCP Cloud SQL Instances in Cloud Run Servies - Connect to Cloud SQL instances with private IPs in minutes.

Commitment/Reservation expiry alerts on GCP - Implementation with a multi-cloud environment is gaining momentum with reduced complexities and seamlessly managed services as prime….

Handling multiple serverless APIs with API gateway in Google Cloud - Example of using API Gateway with serverless products.

Integrating Bitbucket Cloud Pipelines with Google Firebase using OpenID Connect - This blog post covers all the necessary steps to set up a continuous deployment pipeline from Bitbucket Cloud to Google Firebase.

Get Google Stackdriver metric to Grafana - Configure Grafana to ingest Stackdriver metrics.

What to remember if you decide to ingest logs using logging agent in Google Cloud - Explaining logging agents options.

Running serverless web, batch, and worker apps with Google Cloud Run and Cloud Spanner - Trying Cloud Run as a web app, for job triggering and worker.

Big Data, Analytics, ML&AI

BigQuery Java client library: Leverage executeSelect to get better SQL performance - Introducing the new executeSelect client library method that can provide significant query performance improvements when running a SELECT-only SQL query.

Smooth sailing: The resource hierarchy for adopting Google Cloud BigQuery across Twitter - To provide one-to-one mapping from on-prem Hadoop to BigQuery, the Google Cloud and Twitter team created this resource hierarchy architecture.

How to build a DAG based Task Scheduling tool for Multiprocessor systems using python - Scheduling Big Data Workloads and Data Pipelines in the Cloud with pyDag.

Configuring DAG and task monitoring in Google Cloud Composer - Learn how to set up your composer workflows in Google Cloud Composer and automatically close the gap in monitoring DAGs and task failures.

BigQuery 101, how to tame the beast? — Part 3 - A discussion on BigQuery properties.

Improved Storage Read API Quotas in Google BigQuery - How Google empowers it’s Data Warehouse even more.

Getting started with ML: 25+ resources recommended by role and task - Whether you are a Data Analyst, Data Scientist, ML Engineer or Software Engineer, here are specific resources to help you get started with Vertex AI.

Accelerating ML with Vertex AI: From retail and finance to manufacturing and automotive - How businesses across industries are accelerating deployment of machine learning models into production with VertexAI.

Graph Data Science on Google Cloud: Neo4j AuraDS and Vertex AI - Combine Neo4j’s AuraDS with Google’s Vertex AI for a simple way to accelerate ML development and deployment with Graph Data Science.

Cloud Composer (Airflow) for Machine Learning Data Pipeline - Data pipeline using Cloud Composer (Airflow).

How to train ML models with Vertex AI Training - A simple and scalable approach with custom container.

EvoJAX: Bringing the Power of Neuroevolution to Solve Your Problems - DL methods can solve "well-behaved" problems well, but many real-world problems are not well-behaved and non-differentiable. Neuroevolution is an alternative technology to solve these, and the recent breakthrough of EvoJAX makes it much wider applicable to those unexplored scientific and business problems with TPU/GPUs.

Various

Pick your AI/ML Path on Google Cloud - Your ultimate AI/ML decision tree.

Google Cloud CEO Thomas Kurian Makes His Move - Thomas Kurian is making his market-share takeover move as Google Cloud looks to ‘industrialize’ the cybersecurity industry, create the world’s most open cloud marketplace and completely eradicate channel conflict.

Slides, Videos, Audio

GCP Podcast - #307 FinOps with Joe Daly.

Kubernetes Podcast - #182 Cloud Native Storage, with Alex Chircop.

Security Podcast - #68 How We Attack AI? Learn More at Our RSA Panel!

Releases

Anthos clusters on bare metal - 1.9. Release 1.9.8 Anthos clusters on bare metal 1.9.8 is now available for download. Fixes: The following container image security vulnerabilities have been fixed: CVE-2022-1271 CVE-2022-22576 CVE-2022-24769 CVE-2022-27774. Known issues: For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Anthos clusters on Azure - You can now launch clusters with the following Kubernetes versions: 1.21.11-gke.1800 1.22.8-gke.2100. Windows nodes on 1.22.8-gke.2100 now use pigz to improve image layer extraction performance.

Google Cloud Armor - Google Cloud Armor Threat Intelligence (Threat Intel) is available in public preview.

Artifact Registry - Audit logs for Maven, npm, and Python repositories are now available in Cloud Logging.

BigQuery - Quotas for multi-statement queries have changed. Batch and interactive translation services are now generally available (GA), and include support for most major SQL dialects. You can now attach Resource Manager tags to datasets. The following Storage Read API quotas and limits have changed: There is now a limit of 2,000 concurrent ReadRows calls per project in the US and EU multi-regions and 400 concurrent ReadRows calls in other regions.

Chronicle - The following supported default parsers have changed (listed by product name and ingestion label): Amazon Guardduty (GUARDDUTY) Atlassian Jira (ATLASSIAN_JIRA) AWS CloudFront (AWS_CLOUDFRONT) AWS Cloudtrail (AWS_CLOUDTRAIL) AWS CloudWatch (AWS_CLOUDWATCH) AWS Config (AWS_CONFIG) AWS Elastic Load Balancer (AWS_ELB) AWS Key Management Service (AWS_KMS) AWS VPC Flow (AWS_VPC_FLOW) Check Point (CHECKPOINT_FIREWALL) Cisco ACS (CISCO_ACS) Cisco Email Security (CISCO_EMAIL_SECURITY) CrowdStrike Falcon (CS_EDR) Elastic Audit Beats (ELASTIC_AUDITBEAT) Elastic Windows Event Log Beats (ELASTIC_WINLOGBEAT) ESET Threat Intelligence (ESET_IOC) F5 BIGIP LTM (F5_BIGIP_LTM) Fastly WAF (FASTLY_WAF) GCP Cloud IOT (GCP_CLOUDIOT) HCL BigFix (HCL_BIGFIX) IBM z/OS (IBM_ZOS) Imperva (IMPERVA_WAF) Infoblox DNS (INFOBLOX_DNS) Juniper IPS (JUNIPER_IPS) Microsoft Azure Resource (AZURE_RESOURCE_LOGS) Microsoft Defender for Endpoint (MICROSOFT_DEFENDER_ENDPOINT) Microsoft Graph API Alerts (MICROSOFT_GRAPH_ALERT) Microsoft SQL Server (MICROSOFT_SQL) Okta (OKTA) Tanium Stream (TANIUM_TH) Trend Micro AV (TRENDMICRO_AV) Unix system (NIX_SYSTEM) Windows Event (WINEVTLOG) Zscaler (ZSCALER_WEBPROXY) For details about changes in each parser, see Supported default parsers.

Cloud Composer - Cloud Composer 1.18.12 and 2.0.16 release started on June 7, 2022. This version fixes a problem with autoscaling not working properly in Cloud Composer 2 environments when GKE version 1.22 is used for the environment's cluster: The issue could impact your Cloud Composer 2 environment if you created it between June 2, 2022 and June 7, 2022. (Airflow 1.10.15) Upgraded apache-beam and google provider packages to version 2022.6.1: Support impersonation_chain parameter for Dataflow runner in Apache Beam operators Added missing project_id parameter for wait_for_job method in the Dataflow operators Added key_secret_project_id parameter which specifies a project with KeyFile. Cloud Composer 1.18.12 and 2.0.16 images are available: composer-1.18.12-airflow-1.10.15 (default) composer-1.18.12-airflow-2.1.4 composer-1.18.12-airflow-2.2.5 composer-2.0.16-airflow-2.1.4 composer-2.0.16-airflow-2.2.5.

Compute Engine - Generally available: Dallas, Texas us-south1-a,b,c has launched with E2 and N2 VMs available in all three zones. Generally available: NVIDIA A100 GPUs are now available in the following additional regions and zones: Las Vegas, Nevada, North America : us-west4-b For more information about using GPUs on Compute Engine, see GPU platforms. Preview: When you create VMs in bulk, you can now use the following new values with the TARGET_SHAPE flag: ANY: Use this value to place VMs in zones to maximize unused zonal reservations.

Data Fusion - Cloud Data Fusion version 6.7.0 is in Preview. Features in 6.7.0: Connection Management is generally available (GA). Changes in 6.7.0: Increased pipeline launch and run scalability in Enterprise instances. Fixed in 6.7.0: Fixed an issue in the Group By transformation where Longest String and Shortest String aggregators returned an empty string, even when all records contained null values in the specified field.

Dataflow - Dataflow is now available in Dallas, Texas (us-south1).

Dataproc Metastore - Updated Dataproc Metastore auxiliary versions to support the Spanner database type.

Dataproc - New sub-minor versions of Dataproc images: 1.5.68-debian10, 1.5.68-rocky8, 1.5.68-ubuntu18 2.0.42-debian10, 2.0.42-rocky8, 2.0.42-ubuntu18. Dataproc Serverless for Spark now uses runtime version 1.0.13. Dataproc Serverless for Spark runtime versions 1.0.2, 1.0.3 and 1.0.4 are unavailable for new batch submissions. Dataproc on GKE Spark 3.1 images upgraded to Spark version 3.1.3. Fixed a bug where HDFS directories initialization could fail when user names in a project contain special characters. Fix a Dataproc on GKE bug that caused upload of driver logs to Cloud Storage to fail.

Cloud Deploy - The Google Cloud Terraform provider now supports creating Google Cloud Deploy delivery pipelines and targets.

Cloud Data Loss Prevention - The LOCATION_COORDINATES infoType detector is available in all regions.

Document AI - The Contract Parser is now more accurate, can extract more fields and supports higher page limits.

Cloud Functions - The Java 17 runtime (preview) now uses Ubuntu 22.

KMS - Cloud KMS is available in the following region: us-south1 For more information, see Cloud KMS locations.

Google Kubernetes Engine - You can now easily identify clusters that use deprecated Kubernetes APIs removed in version 1.22. The us-south1 region in Dallas, Texas is now available.

Load Balancing - External TCP/UDP Network Load Balancing now supports load-balancing GRE traffic.

Marketplace Partners - If you've created private offers that use a prepay payment schedule, you see new fields when you manage entitlements with the Partner Procurement API, and also additional information in your Customer Insights report.

Cloud Memorystore - Added support for customer-managed encryption keys (Preview) for Memorystore for Redis.

Cloud VPN - Cloud VPN is available in region us-south1 (Dallas, US).

Cloud PubSub - Pub/Sub is now available in us-south1 (Dallas, Texas).

Cloud Run - Session affinity is now available for Cloud Run service revisions. The following new region is now available: us-south1. You can now create and edit Cloud Run jobs using the Cloud console.

Service Mesh - 1.11.x & 1.12.x & 1.13.x. There is a known issues with the signatures of the revisions released June 9, 2022. 1.13.x. 1.13.4-asm.4 is now available. 1.12.x. 1.12.7-asm.2 is now available. 1.11.x. 1.11.8-asm.4 is now available. 1.11.x & 1.12.x & 1.13.x. The Istio and Envoy projects recently disclosed a series of CVEs that can expose Anthos Service Mesh to remotely exploitable vulnerabilities. 1.13.x. 1.13.4-asm.3 is now available. 1.12.x. 1.12.7-asm.1 is now available. 1.11.x. 1.11.8-asm.3 is now available.

SAP Solutions - BigQuery Connector for SAP version 2.1 Version 2.1 of BigQuery Connector for SAP is now available.

Cloud Spanner - Commit timestamps enable a Cloud Spanner optimization that can reduce query I/O when retrieving data written after a particular time. You can create Cloud Spanner regional instances in Dallas (us-south1).

Cloud SQL - Cloud SQL now supports faster machine type changes, with connectivity dropping to less than 60 seconds. The June maintenance changelog is now available. New maintenance versions are now available through self-service maintenance. An addendum to the May maintenance changelog shows additional security patches. Support for us-south1 (Dallas).

Cloud Storage Transfer - Storage Transfer Service now offers a merged, unified console experience for cloud and file system transfers.

Cloud Storage - Cloud Storage is now available in Dallas, Texas (us-south1 region).

VPC Service Controls - Beta stage support for the following integration: AlloyDB. General availability for the following integration: Migrate for Compute Engine.

Virtual Private Cloud - For auto mode VPC networks, added a new subnet 10.206.0.0/20 for the Dallas us-south1 region.

Cloud VPN - Cloud VPN is available in region us-south1 (Dallas, US).

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko