Editor's Picks

• Is Data Scientist Still the Sexiest Job of the 21st Century?
  Ten years ago, the authors posited that being a data scientist was the “sexiest job of the 21st century.” A decade later, does the claim stand up? The job has grown in popularity and is generally well-paid, and the field is projected to experience more growth than almost any other by 2029. But the job has changed, in both large and small ways...

• Overview & Applications of Large Language Models (LLMs)
  Given the rapid pace of innovation in machine learning (ML), especially in LLMs, I believe that we are beginning a wave where many significant companies supplying or utilizing this technology will be built...I’m fascinated by all of the ways LLMs will continue to become part of our daily lives, regardless of whether the LLM application is consuming the model output from the APIs of others or training their own models for specific use cases. When thinking through if an LLM application is feasible (or a good investment), I consider the following questions...

• The Great Fiction of AI: The strange world of high-speed semi-automated genre fiction
  Writer’s block is a luxury she can’t afford, which is why as soon as she heard about an artificial intelligence tool designed to break through it, she started beseeching its developers on Twitter for access to the beta test...

A Message from this week's Sponsor:

Data Science Articles & Videos

• Modeling Short Time Series with Prior Knowledge
  It is generally difficult to model time series when there is insuffient data to model a (suspected) long seasonality. Here, we show how this difficulty can be overcome by learning a seasonality on a different, long related time series and transferring the posterior as a prior distribution to the model of the short time series. The result is a forecast that is believable and can be used for decisions in a business context...

• Interview with Sir David Cox [Video]
  Last fall Trevor Hastie and Robert Tibshirani interviewed Sir David Cox, for their online course on Statistical learning. Sir David talked about a wide range of topics including history of the Cox model and his view of modern data science...

• ICML 2022 Awards
  Test of Time Award and many Outstanding Papers...

• SQL: The Universal Solvent for REST APIs
  Data scientists working in Python or R typically acquire data by way of REST APIs. Both environments provide libraries that help you make HTTP calls to REST endpoints, then transform JSON responses into dataframes. But that’s never as simple as we’d like...What if there were a way of reading from APIs that abstracted all the low-level grunt work and worked the same way everywhere? Good news! That is exactly what Steampipe does. It’s a tool that translates REST API calls directly into SQL tables. Here are three examples of questions that you can ask and answer using Steampipe...

• The Rise of Domain Experts in Deep Learning
  In this interview, Jeremy Howard, an AI researcher and the co-founder of fast.ai, iscusses what it means for different industries and even global regions now that people without PhDs from specialized research labs can build and work with deep learning models. Among other topics under this broad umbrella, he shares his thoughts on how to best keep up with state-of-the-art techniques, prompt engineering as a new skill set, and the pros and cons of code-generation systems like Codex...

• Why do tree-based models still outperform deep learning on tabular data?
  While deep learning has enabled tremendous progress on text and image datasets, its superiority on tabular data is not clear. We contribute extensive benchmarks of standard and novel deep learning methods as well as tree-based models such as XGBoost and Random Forests, across a large number of datasets and hyperparameter combinations...

• Transformers in computer vision: ViT architectures, tips, tricks and improvements
  You are probably already aware of the Vision Transformer (ViT). What came after its initial submission is the story of this blog-post. We will explore multiple orthogonal research directions on ViTs. Why? Because chances are that you are interested in a particular task like video summarization. We will address questions like how can you adapt/use ViT on your computer vision problem, what are the best ViT-based architectures, training tricks and recipes, scaling laws, supervised vs self-supervised pre-training, etc...

• The world is awful. The world is much better. The world can be much better.
  It is wrong to think that these three statements contradict each other. We need to see that they are all true to see that a better world is possible...

• Theseus, a library for encoding domain knowledge in end to end AI models
  Theseus, a library for an optimization technique called differentiable nonlinear least squares (NLS) that is particularly useful for applications like robotics and computer vision. Built on PyTorch, Theseus enables researchers to easily incorporate expert domain knowledge into modern AI architectures...

• Is Integer Arithmetic Enough for Deep Learning Training?
  Our empirical and mathematical results reveal that integer arithmetic is enough to train deep learning models. Unlike recent proposals, instead of quantization, we directly switch the number representation of computations. Our novel training method forms a fully integer training pipeline that does not change the trajectory of the loss and accuracy compared to floating-point, nor does it need any special hyper-parameter tuning, distribution adjustment, or gradient clipping. Our experimental results show that our proposed method is effective in a wide variety of tasks such as classification (including vision transformers), object detection, and semantic segmentation...

• POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging
  POET optimizes state-of-the-art ML models for training on Edge devices. Operators of the ML model are profiled on target edge device to obtain fine-grained profiles. POET adopts an integrated integrated rematerialization and paging to produce an energy-optimal training schedule...

• Transformer models: an introduction and catalog — 2022 Edition
  Just 3 months after my previous update I felt like there was already a “long overdued” further update. That is how quickly the field is evolving. In this recent update I have added the new class of text to image transformers, all the way from diffusion models to DALL-E2, CLIP, and Imagen. Another class of Transformer models I added are those that allow to use transformers to model an arbitrary agent with RL applications such as playing Atari or controling a robotic arm. Those include Trajectory Transformers, Decision Transformers, and Deepmind’s GATO. I have also included the latest open source BLOOM model. The full list of newcomers to the catalog: BLOOM, CLIP, DALL-E2, Decision and Trajectory Transformers, Flamingo, Gato, DQ-BART, GLaM, GLIDE, GC-ViT, Imagen, LAMDA, Minerva, OPT, PaLM, and Switch....

Course*

Jobs

• Senior Data Scientist, Startup Creation at Redesign Health - US
  
  As our Senior Data Scientist for our Startup Creation team, you will set up and configure the data infrastructure for our startups, and work with the startup founding team to define data driven KPIs, and implement automated statistical analyses of customer behavior. Your goal is to make all of the companies that we launch data-driven from day one.
  
  In this role, you will function as an in-house implementation team for the companies that Redesign Health launches (internally referred to as OpCos). We provide data strategy, data pipeline, data analytics and forecasting services to newly formed companies in a repeatable and scalable manner...
   

Training & Resources

• Is learning tensorflow & keras still worth it? [Reddit Discussion]
  I recently acquired Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurélien Geron. I've mainly worked with pytorch but I wanted to revise some ML/DL concepts. I probably should have thought about this before, but given the current trend of migrating from tensorflow to pytorch, is reading this book right now a step back? Thanks!...

• I start my first day as a Data Engineer next Monday, any tips? [Reddit Disscussion]
  I would like some advice on what to focus on: a) How to not be nervous with my first Engineering job, b) Did I get a good salary for my first Engineer position, c) How can I grow into a better Engineer or to even a Architect?, and d) Then any questions or advice you all can give...

• Modeling Short Time Series with Prior Knowledge in PyMC
  In this notebook I want to reproduce in PyMC the methodology described in the amazing blog post Modeling Short Time Series with Prior Knowledge by Tim Radtke to forecast short time series using bayesian transfer learning 🚀...

What you’re up to – notes from DSW readers

• Working on something cool? Let us know here :) ...

* To share your projects and updates, share the details here.

** Want to chat with one of the above people? Hit reply and let us know :)

Last Week's Newsletter's 3 Most Clicked Links

• The Data Science Trap [Reddit Discussion]

• The Data Science Trap: A Rebuttal [Reddit Discussion]

• 4 Pandas Anti-Patterns to Avoid and How to Fix Them

* Based on unique clicks.

** Find last week's newsletter here.