How I built a €25K machine learning rig

How to plan, buy, build, and store your 2-10 GPU machine learning servers and PCs.

Cross-validation: what does it estimate and how well does it do it?

Cross-validation is a widely-used technique to estimate prediction error, but its behavior is complex and not fully understood. Ideally, one would like to think that cross-validation estimates the prediction error for the model at hand, fit to the training data. We prove that this is not the case for the linear model fit by ordinary least squares; rather it estimates the average prediction error of models fit on other unseen training sets drawn from the same population.

5 types of machine learning algorithms

The conceptual fundamentals for Machine Learning (ML) were developed in the second half of the 20th century. But computational limitations and sparsity of data postponed the enthusiasm around artificial intelligence (AI) to recent years. Since then, computers have become exponentially faster, and cloud services have emerged with nearly limitless resources. The progress in computational power, combined with the abundance of data, makes Machine Learning algorithms applicable in many fields today.

Convert Pretrained Transformers into RNNs, Lowering Memory Cost While Retaining High Accuracy

A research team from University of Washington, Microsoft, DeepMind and Allen Institute for AI develop a method to convert pretrained transformers into efficient RNNs. The Transformer-to-RNN (T2R) approach speeds up generation and reduces memory cost.

Introducing PyTorch profiler

The analysis and refinement of the large-scale deep learning model’s performance is a constant challenge that increases in importance with the model’s size. Owing to a lack of available resources, PyTorch users had a hard time overcoming this problem. There were common GPU hardware-level debugging tools, but PyTorch-specific background of operations was not available. Users had to merge multi-tools or apply minimal correlation information manually to make sense of the data to retrieve the missing information.