Welcome to issue #259 September 13th, 2021

News

Wrapping up the summer: A host of new stories and announcements from Data Analytics - August rollup of the Google Cloud Data Analytics blog, feature announcements from across the portfolio, how-to articles and customer stories.

Expanding Google Cloud’s commitment to startups - Google Cloud announces new services and support for startup customers to scale in the cloud.

Bridging the hybrid work gaps with Google Workspace - We’re bridging the gaps in hybrid work with enhancements to Google Meet and the introduction of Spaces.

Helping build the digital future. On Europe’s terms. - 'Cloud. On Europe’s Terms' is a part of Google Cloud's commitment to deliver cloud services that provide the highest levels of digital sovereignty, all while enabling the next wave of growth and transformation for Europe’s businesses and organizations.

Sqlcommenter now extending the vision of OpenTelemetry to databases - Merging Sqlcommenter, an open source object-relational mapping (ORM) auto-instrumentation library, with OpenTelemetry, an open source observability framework.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

How to check all the right boxes for your cloud migration before you get started - A new guide and checklist of essential tips for the four key parts of the migration process: Assess, Plan, Migrate, and Optimize.

To the cloud and beyond! Migration Enablement with Google Cloud’s Professional Services Organization - How Professional Service Organization (PSO) can help optimize customers' cloud journey when migrating to Google Cloud.

You’re using service accounts wrong… - A practical guide to user-service-account best practice in Google Cloud Platform.

Keyless API authentication- Launching GCP workloads from AWS - Call GCP APIs directly from AWS or Azure without secret keys.

Measuring Reliability in GCP: Step By Step SLO creation guide using Cloud Operation Sandbox - In this step-by-step guide, I will demonstrate how to configure SLOs in Cloud Operations using our learning environment, Cloud Operation Sandbox.

App Development, Serverless, Databases, DevOps

Centralizing SAP system management across clouds using SAP LaMa - SAP Landscape Management (LaMa) supports centralized management of your SAP landscape. Google Cloud recently released a free connector for our customers to deploy an existing or new SAP LaMa instance.

Enable query tagging with Sqlcommenter to understand application impact on database performance - Learn how to make Cloud SQL Insights even more powerful when you enable query tagging with Sqlcommenter in your apps.

Best practices for translating websites with Translation API - Translate your website with Google’s industry leading Machine Learning. Learn best practices for optimizing cost, performance, and security.

Stop burning money by leaving your VM’s ON — Add an Auto-Shutdown Script - Configuring an auto-shutdown on VM instances that is idle on GCP.

Google Cloud Storage Folder Wise Object Count on Google Cloud - Using log-based metric to count objects created in Cloud storage bucket.

Firestore: Copy Data from Prod to Dev or Emulator - Using Firefoo (GUI client for Firestore) to copy data from production to the local environment.

Big Data, Analytics, ML&AI

Exporting Google Cloud Monitoring Data to BigQuery Seamlessly - An open-source project to export Cloud Monitoring data to BigQuery for better control and analysis.

Running Containers on Cloud Composer with Airflow 2.0 - Running Containers on Cloud Composer (the Airflow 2.0 way).

Understanding Google Cloud PubSub - A Google-Scale Messaging Service from basics to more advanced concepts.

Add monitoring to your Google Cloud Storage streaming pipeline - In this article, you will learn how to add monitoring to the reference Google Cloud Storage streaming architecture.

Optimizing your BigQuery incremental data ingestion pipelines - Improve performance of BigQuery incremental data ingestion pipelines by pruning more data during merge step.

Ad agencies choose BigQuery to drive campaign performance - Two advertising agencies use BigQuery cloud data warehouse to streamline reporting and get real-time access to data to ensure the highest campaign performance.

Create Advanced Google Shopping Insights using Merchant Center BigQuery exports - Leverage Google provided market insights like price competitiveness and benchmarks.

Unnecessary BigQuery Optimization - Writing BigQuery query in different ways and analysing execution plans.

7 tips for trouble-free ML model training - This post covers seven common causes of ML model training failures, along with time-saving tips on how to avoid them and how to fix them.

Scalable ML Workflows using PyTorch on Kubeflow Pipelines and Vertex Pipelines - This post shows PyTorch on two pipelines frameworks, OSS Kubeflow Pipelines and Vertex Pipelines. We also share some new PyTorch components that have been added to the KFP repo, and show how to run 'KFP v2' pipelines on OSS KFP.

PyTorch on Google Cloud: How To train and tune PyTorch models on Vertex AI - With the PyTorch on Google Cloud blog series, we will share how to build, train and deploy PyTorch models at scale, how to create reproducible machine learning pipelines on Google Cloud Vertex AI and emphasize first class support for training and deploying PyTorch models on Vertex AI.

Slides, Videos, Audio

GCP Podcast - #275 Building the I/O Adventure Game with Valentin Deleplace.

Kubernetes Podcast - #162 ingress-nginx, with Alejandro de Brito Fontes and Ricardo Katz.

Security Podcast - #30 EP30 Malware Hunting with VirusTotal.

Releases

Artifact Registry - v1beta2. Maven, npm, and Python repositories are now generally available.

BigQuery - Deleting the metadata for a specific job using the bq command-line tool is now generally available (GA). Session support for BigQuery is now in Preview.

Billing - Cloud Billing Budgets & alerts now support configurable budget time periods, beyond monthly budgets In the Cloud Billing Console Budgets & alerts settings, you can now specify the time period of your budgets.

Cloud Build - Users can now configure triggers to use buildpacks to automatically build images.

Cloud Composer - Cloud Composer 2 is available in Preview. We gradually reveal Cloud Composer 2 functionality. Cloud Composer 2 uses the following Cloud Composer images: composer-2.0.0-preview.0-airflow-2.1.2 composer-2.0.0-preview.0-airflow-2.1.1 composer-2.0.0-preview.0-airflow-2.0.2.

Compute Engine - Preview: You can now review OS vulnerability report data, which is collected by VM Manager, from the Security Command Center.

Config Connector - Config Connector 1.61.0 is now available. Added the secuirtySettings field to ComputeBackendService. Added jitter to resource reconciliation reenqueue period to smooth out the traffic pattern. Fixed a bug in BigqueryJob that generates unexpected diff for 'kms_key_name'.

Cloud Dataflow - Dataflow now supports Shielded VM workers.

Dataproc Metastore - v1. The default Dataproc Metastore service creation version is changed to Hive 3.1.2.

Dataproc - The following previously released sub-minor versions of Dataproc images included a bug where the dataproc user account was broken. Added additional messages to the error messages for networking and IAM errors when creating a new cluster.

Deep Learning Containers - M79 release Updated Pytorch 1.9 containers (they were not refreshed in the last release). M79 is the last release version that has JupyterLab 1.x installed. Fixed a bug in which the home folder in custom container VMs was owned by the root instead of Jupyter.

Deep Learning VM - M79 release Updated Pytorch 1.9 images (they were not refreshed in the last release). M79 is the last release version that has JupyterLab 1.x installed. Deep Learning VM Images in Cloud Marketplace have not been updated. Fixed a bug in which the home folder in custom container VMs was owned by the root instead of Jupyter.

Cloud Firestore - Firestore triggers for Cloud Functions are now supported at the General Availability release level.

Cloud Functions - Firestore triggers for Cloud Functions are now supported at the General Availability release level.

KMS - Cloud KMS now provides a library that conforms to the PKCS #11 standard, which enables working with existing applications that use the PKCS #11 API.

Google Kubernetes Engine - The managed Filestore CSI driver for GKE is now available in GKE versions 1.21 and later to provision and manage Filestore instances for GKE workloads. Several gcloud flags used to configure which logs and metrics are collected are deprecated and replaced with new flags. (2021-R28) Version updates GKE cluster versions have been updated. Two security vulnerabilities, CVE-2021-33909 and CVE-2021-33910, have been discovered in the Linux kernel that can lead to an OS crash or an escalation to root by an unprivileged user.

GKE - (2021-R28) Version updates Version 1.20.8-gke.2101 is now the default version.

Google Kubernetes Engine Rapid - (2021-R28) Version updates Version 1.21.3-gke.2001 is now the default version in the Rapid channel.

Google Kubernetes Engine Regular - (2021-R28) Version updates Version 1.20.9-gke.701 is now the default version in the Regular channel.

Google Kubernetes Engine Stable - (2021-R28) Version updates Version 1.19.12-gke.2101 is now the default version in the Stable channel.

Load Balancing - Cloud Load Balancing now supports load-balancing traffic to endpoints that extend beyond Google Cloud, such as on-premises data centers and other public clouds that you can reach using hybrid connectivity.

Cloud Memorystore - Redis version 6.x is now Generally Available on Memorystore for Redis.

KF - 2.5.2. Kf for Anthos on-prem (Vsphere) is now available on the Public Previews page. Addressed a potential panic in the kf build-logs command. Changed flag and manifest validation for route and task fields so manifest routes are ignored when creating tasks.

Cloud Monitoring - You can now collect JVM metrics from the Ops Agent, starting with version 2.2.0.

Network Intelligence Center - Firewall Insights now provides comprehensive analysis of whether your firewall rules are overly permissive.

Notebooks - v1beta1 & v1. Due to a recent change, the iam.serviceAccounts.actAs permission on the specified service account for the notebook instance is required for users to continue to have access to their notebook instances.

Security Command Center - VM Manager vulnerability reports, which are in preview, are now available in Security Command Center Premium.

Cloud SQL MySQL - The Cloud SQL out-of-disk recommender is now generally available. Cloud SQL for MySQL now allows you to specify mysqldump options during migration from external servers.

Cloud SQL Postgres - The Cloud SQL out-of-disk recommender is now generally available. Cloud SQL for PostgreSQL now supports the min_wal_size flag.

Vertex AI - Vertex Model Monitoring is generally available (GA).

Workflows - v1. Support for callback endpoints is available in Preview.

If you have suggestion, feedback or link you want to share feel free to email me at zdenko@gcpweekly.com

Have a great week,

Zdenko