Data engineering roadmap

A map for becoming a great data engineer.

Machine learning on graphs

In the first post, I present some common techniques for graph analysis that should help us better understand our data.

Parallelizing Python code

This article reviews some common options for parallelizing Python code, including process-based parallelism, specialized libraries, ipython parallel, and Ray.

How to visualize data categories in python with pandas

If you have a dataset which is divided into categories of data like: kickstarter projects, flower species or most popular car brands, then it's a good idea to visualize those data categories to see the amount of values within each category.

The sensory neuron as a transformer: Permutation-invariant neural networks for reinforcement learning

In this work, we investigate the properties of RL agents that treat their observations as an arbitrarily ordered, variable-length list of sensory inputs. Here, we partition the visual input from CarRacing (Left) and Atari Pong (right) into a 2D grid of small patches, and shuffled their ordering. Each sensory neuron in the system receives a stream of visual input at a particular permuted patch location, and through coordination, must complete the task at hand, even if the visual ordering is randomly permuted again several times during an episode.