Issue #510
August 31 2023

Hello and thank you for tuning in to Issue #510!

Once a week we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.

Seeing this for the first time? Subscribe here:

If you find this newsletter helpful to your job, consider becoming a paid subscriber here: https://datascienceweekly.substack.com/subscribe :)

If you don’t find this email useful, please unsubscribe here.

And now, let's dive into some interesting links from this week :)

Editor's Picks

• Champion-level drone racing using deep reinforcement learning
  First-person view (FPV) drone racing is a televised sport in which professional competitors pilot high-speed aircraft through a 3D circuit. Each pilot sees the environment from the perspective of their drone by means of video streamed from an onboard camera. Reaching the level of professional pilots with an autonomous drone is challenging because the robot needs to fly at its physical limits while estimating its speed and location in the circuit exclusively from onboard sensors. Here we introduce Swift, an autonomous system that can race physical vehicles at the level of the human world champions. The system combines deep reinforcement learning (RL) in simulation with data collected in the physical world…

• Google Gemini Eats The World – Gemini Smashes GPT-4 By 5X, The GPU-Poors
  Google had all the keys to the kingdom, but they fumbled the bag. A statement that is obvious to everyone. The statement that may not be obvious is that the sleeping giant, Google has woken up, and they are iterating on a pace that will smash GPT-4 total pre-training FLOPS by 5x before the end of the year. The path is clear to 20x by the end of next year given their current infrastructure buildout. Whether Google has the stomach to put these models out publicly without neutering their creativity or their existing business model is a different discussion. Today we want to discuss Google’s training systems for Gemini, the iteration velocity for Gemini models, Google’s Viperfish (TPUv5) ramp, Google’s competitiveness going forward versus the other frontier labs, and a crowd we are dubbing the GPU-Poor…
  

• Reinforced Self-Training (ReST) for Language Modeling
  Reinforcement learning from human feedback (RLHF) can improve the quality of large language model's (LLM) outputs by aligning them with human preferences. We propose a simple algorithm for aligning LLMs with human preferences inspired by growing batch reinforcement learning (RL), which we call Reinforced Self-Training (ReST)…Our results show that ReST can substantially improve translation quality, as measured by automated metrics and human evaluation on machine translation benchmarks in a compute and sample-efficient manner…

  
  

A Message from this week's Sponsor:

Hedonova outperforms S&P 500: Investors enjoy 3X higher returns

Alternative investments are gaining momentum, with 67% of institutional investors predicting that a portfolio including 20% alternatives will outperform the traditional 60/40 stock-bond investment mix with lower volatility.

Hedonova, a hyper-diversified hedge fund open to accredited investors has outperformed the S&P 500 by 17%. 

Hedonova invests in various alternative assets like equipment finance, litigation finance, startups, wine, art, etc. The SEC regulated fund is  backed by the likes of Morgan Stanley and is open to accredited investors with a low minimum investment of $5,000.

The fund was also awarded Best Multi-Strategy Hedge Fund at Hedgeweek European Awards 2023.

* Want to sponsor the newsletter? Email us for details --> team@datascienceweekly.org

Data Science Articles & Videos

• Taking a peek behind Hidden Door's AI
  CEO Hilary Mason talks about what generative AI can (and can't) do for storytelling in games..
  

• Yet Another ICML Award Fiasco
  I wrote a blog post on the ICML award fiasco: They gave an outstanding paper award to the D-Adaptation paper, that contains worse results that the ones in papers from 9 years ago. Also, this is not the first time that ICML gives awards to questionable or even plainly wrong papers. I believe this might start a serious conversation about "stochastic" awards, and the super noisy reviews in machine learning conferences…
  

• Influencers in data are doing no justice to the industry [Reddit Discussion]
  A few of them aside, most are writing stuff just to fill the gaps. Nothing meaningful, just piece after piece of barely important content…Data Twitter, on the other hand, is much more cliquey. Guarding and almost gatekeeping their world. They don't even like the LinkedIn data influencers and sometimes even hate on people in other parts of Data Twitter too…
  

• RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation
  For robots to be useful outside labs and specialized factories we need a way to teach them new useful behaviors quickly…In this work we explore dense tracking as a representational vehicle to allow faster and more general learning from demonstration. Our approach utilizes Track-Any-Point (TAP) models to isolate the relevant motion in a demonstration, and parameterize a low-level controller to reproduce this motion across changes in the scene configuration. We show this results in robust robot policies that can solve complex object-arrangement tasks such as shape-matching, stacking, and even full path-following tasks such as applying glue and sticking objects together, all from demonstrations that can be collected in minutes…
  

• GPT Pilot – a PoC for a scalable dev tool that writes production ready apps from scratch as the developer oversees the implementation
  In this blog post, I will lay out a PoC (proof of concept), with a prototype open sourced on Github, for a dev tool that uses GPT-4 to code an entire, production-ready app…The main idea is that now AI can write most of the code for an app, even 95%, but an app doesn’t work at all if all code doesn’t work completely. So, until we get to AGI, the developer will be needed to oversee the app development, acting as a team lead who gets involved when the AI developer gets stuck. I started this project thinking about a couple of things…

• Inside a Google Cloud TPU Data Center [Video]
  Get an inside look at the magic of Google Cloud TPUs, including a rare inside view of the data centers where it all happens. Customers use Cloud TPUs to run some of the world's largest AI workloads and that power comes from much more than just a chip. In this video, take a look at the components of the TPU system, including data center networking, optical circuit switches, water cooling systems, biometric security verification and more…

• GPU Puzzles - Solve puzzles. Learn CUDA
  GPU architectures are critical to machine learning, and seem to be becoming even more important every day…This notebook is an attempt to teach beginner GPU programming in a completely interactive fashion. Instead of providing text with concepts, it throws you right into coding and building GPU kernels. The exercises use NUMBA which directly maps Python code to CUDA kernels. It looks like Python but is basically identical to writing low-level CUDA code. In a few hours, I think you can go from basics to understanding the real algorithms that power 99% of deep learning today…
  

• Tensor Puzzles - Solve puzzles. Learn PyTorch
  When learning a tensor programming language like PyTorch or Numpy it is tempting to rely on the standard library (or more honestly StackOverflow) to find a magic function for everything. But in practice, the tensor language is extremely expressive, and you can do most things from first principles and clever use of broadcasting. This is a collection of 21 tensor puzzles. Like chess puzzles these are not meant to simulate the complexity of a real program, but to practice in a simplified environment. Each puzzle asks you to reimplement one function in the NumPy standard library without magic…
  

• Incredible evolution of Boston Dynamic robots [GIF] [X/Twitter]
  This is the incredible evolution of Boston Dynamic robots: [1992-2023]…
  

• Pathway from Data Analyst to Data Engineer: Tips & Takeaways [Reddit Discussion]
  I'm delivering an internal talk to other consultants at my company (primarily Data Analytics Consultants) about my personal journey to become a Data Engineer (Photographer --> DA --> DE ). I've been compiling a list of tips and takeaways to punctuate my talk and I'm hoping to get some input from the r/dataengineering brain trust. Here's what I've got (in no particular order)…
  

• Geographic data analysis in R and Python: comparing code and outputs for vector data
  In this blog post, we talk about our experience teaching R and Python for geocomputation. The focus of the blog post is on geographic vector data, meaning points, lines, polygons (and their ‘multi’ variants) and the attributes associated with them. Geographic data analysis is a broad topic and in a later post we will cover raster data, meaning gridded data such as satellite images…
  

• Teaching with AI
  We’re releasing a guide for teachers using ChatGPT in their classroom—including suggested prompts, an explanation of how ChatGPT works and its limitations, the efficacy of AI detectors, and bias…

Jobs

GUCCI Global Senior Data Scientist

For the Gucci Global Data Science team based in Milan, we are currently seeking an English speaking Senior Data Scientist.

In this role, you will report to the Global Corporate Director of Data Science and help the business in central decision making processes, have the opportunity to lead the technical development of a small team of bright and driven data scientists, collaborate with teams across different regions and areas of the business leveraging Gucci’s rich data sources, infrastructure and the power of machine learning and advanced analytics.

Influential, innovative and progressive, Gucci is reinventing a wholly modern approach to fashion.

The Gucci Data Science team is the new kid on the block, bringing fresh perspectives and a new way of working that will help the company in continuing its innovation path leveraging the power of data and ML.

Apply here
 

Want to post a job here? Email us for details --> team@datascienceweekly.org

Training & Resources

• GPT in 60 Lines of NumPy
  In this post, we'll implement a GPT from scratch in just 60 lines of numpy. We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text…
  

• A Visual Introduction to Neural Networks
  Here, we’re going to explore neural networks. Neural networks are a type of machine learning model inspired by the structure and function of the human brain. They are designed to recognize patterns and learn from data in a way that enables them to make predictions. They do this using a method called backpropagation. Before diving into the full details, let’s take a bit of a look at the history of neural nets and where they were first used…
  

• My take on “How much math do I need?” [Reddit Discussion]
  I see this question come up so many times, I wanted to write up my own “comprehensive” answer to this. Really… it depends. The answer is on a spectrum, and I wish to break down this spectrum in this post from the simplest, to the most advanced…
  

Last Week's Newsletter's 3 Most Clicked Links

• Create a roadmap for the next 5 years

• AI Forecasting: Two Years In

• Understanding Automatic Differentiation in 30 lines of Python

* Based on unique clicks.
** Find last week's issue #509 here.

Cutting Room Floor

• The Cynical Genius Illusion: Exploring and Debunking Lay Beliefs About Cynicism and Competence

• Time Travel in LLMs: Tracing Data Contamination in Large Language Models

• LLaSM: Large Language and Speech Model

• LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models

• SayTap: Language to Quadrupedal Locomotion

• WeatherBench 2: A benchmark for the next generation of data-driven global weather models

• Is it me or HuggingFace do TOO MANY things?

• I got accepted in a data officer job, and i don't know what the hell i'm doing

• Completed 18 months in Microsoft as a Data Scientist II

• Parents annoyed that I'm not hyping myself enough

• Is it ok to use an offer to negotiate a pay rise?

Thank you for joining us this week :)

All our best,
Hannah & Sebastian

P.S.
If you found this newsletter helpful to your job, please consider becoming a paid subscriber here: https://datascienceweekly.substack.com/subscribe :)

Copyright © 2013-2023 DataScienceWeekly.org, All rights reserved.