Editor Picks

• The Imitation of Consciousness: On the Present and Future of Natural Language Processing
  The artificial intelligence of natural language processing is arriving. In January, 2021, Microsoft filed a patent to reincarnate people digitally through distinct voice fonts appended to lingual identities garnered from their social media accounts. I don’t see any reason why it can’t work...That’s not a “new soul.” It is a mechanical tongue, an artificial person, a virtual being. The application of machine learning to natural language processing achieves the imitation of consciousness, not consciousness itself, and it is not science fiction. It is now...

• It's Not All Fun and Games: How DeepMind Unlocks Medicine's Secrets
  Hello. This is Eric Topol for Medicine and the Machine on Medscape. I have been looking forward to having this conversation with Demis Hassabis (DeepMind) for many months, if not years...I want to get into three areas: games, digitizing biology and protein structure, and drug discovery...

• This AI model tries to re-create the mind of Ruth Bader Ginsburg
  Its creators say the AI app helps ordinary people understand how artificial intelligence is progressing. Critics contend there’s much work to be done....

A Message from this week's Sponsor:

Data Science Articles & Videos

• Debezium to Snowflake: Lessons learned building data replication in production
  With the well-deserved position that Snowflake and Debezium have gained in the modern data stack, it is now fairly easy to find online resources on using these technologies. In this blog, we take it one step further by reflecting on the lessons we (at Shippeo) have learned using Debezium to replicate data at scale in near real time to Snowflake...

• VCT: A Video Compression Transformer
  We show how transformers can be used to vastly simplify neural video compression. Previous methods have been relying on an increasing number of architectural biases and priors, including motion prediction and warping operations, resulting in complex models. Instead, we independently map input frames to representations and use a transformer to model their dependencies, letting it predict the distribution of future representations given the past. The resulting video compression transformer outperforms previous methods on standard video compression data sets...

• "The Worst AI Ever" - Lessons from the GPT-4Chan Controversy
  This article contains an objective summary of a recent controversy related to an AI model named GPT-4chan, as well as a subjective commentary with my thoughts on it...the intent of this is to provide a comprehensive summary of what happened, as well as what I consider to be valuable lessons that can be taken away from it all. It is primarily for people in the AI community, but is accessible to those outside of it as well...If you are already aware of what happened, I recommend skipping the first two sections, but to still read the ‘Analysis’ and ‘Lessons’ sections...

• Discovering and Debugging a PyTorch Performance Decrease
  Over the past week, Thomas Capelle and I (Benjamin Warner) discovered, debugged, and created a workaround for a performance bug in PyTorch which reduces image training GPU throughput up to forty percent...This performance bug has been affecting fastai for an unknown amount of time...The culprit? Subclassed tensors...

• An Actually-Good Argument Against Naive AI Scaling
  My position is that continuing to scale language models under the current paradigm will give us massive but not unbounded capabilities. To be clear: I think Dr. Marcus’s argument for this position is terrible. My reasons for believing that scaling has limits have nothing to do with “true intelligence” or “symbol manipulation” or any of that nonsense. Instead, the limits come from the fundamental inadeqacy of passively-collected datasets, even at internet-scale...In other words: the internet is the bottleneck. Anyways, on to the argument...

• Google releases T5X
  T5X is a modular, composable, research-friendly framework for high-performance, configurable, self-service training, evaluation, and inference of sequence models (starting with language) at many scales...It is essentially a new and improved implementation of the T5 codebase (based on Mesh TensorFlow) in JAX and Flax...Below is a quick start guide for training models with TPUs on Google Cloud. For additional tutorials and background, see the complete documentation...

• An on-chip photonic deep neural network for image classification
  Deep neural networks are commonly implemented using clock-based processors in which computation speed is mainly limited by the clock frequency and the memory access time. In the optical domain, the lack of scalable on-chip optical non-linearity and the loss of photonic devices limit the scalability of optical deep networks. Here we report an integrated end-to-end photonic deep neural network (PDNN) that performs sub-nanosecond image classification through direct processing of the optical waves impinging on the on-chip pixel array as they propagate through layers of neurons...

• Reddit AMA: I left Google AI after 3 years [Reddit Discussion]
  During the 3 years, I developed love-hate relationship of the place. Some of my coworkers and I left eventually for more applied ML job, and all of us felt way happier so far...

• Emergent Abilities of Large Language Models
  Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence implies that additional scaling could further expand the range of capabilities of language models...

• Spancat: a new approach for span labeling
  The SpanCategorizer is a spaCy component that answers the NLP community’s need to have structured annotation for a wide variety of labeled spans, including long phrases, non-named entities, or overlapping annotations. In this blog post, we’re excited to talk more about spancat and showcase new features to help with your span labeling needs!...

• Things I Have Learned Working in an MIT AI Research Lab for a Year
  Hiall! My name is Mike and I work in the Massachusetts Institute of Technology’s Brain and Cognitive Sciences department (MIT BCS) as a research software engineer/ML Engineer. I specifically work on Brain-Score, a tool to measure how brain-like AIs are. I have been here a year, graduating from the University of Virginia in Spring of 2021...Below are 5 things that I have learned in a year of working in an MIT AI lab — some things I hope you find amusing or useful for your own journey, and some things that have profoundly impacted the way that I view life, success, knowledge, and humanity itself...

Course*

• Translate business needs into achievable data projects
• Learn a proven and repeatable approach to data analysis
• Identify the fastest path to actionable insights
• Communicate effectively with diverse stakeholders

Jobs

• Senior Data Scientist, Startup Creation at Redesign Health - US
  
  As our Senior Data Scientist for our Startup Creation team, you will set up and configure the data infrastructure for our startups, and work with the startup founding team to define data driven KPIs, and implement automated statistical analyses of customer behavior. Your goal is to make all of the companies that we launch data-driven from day one.
  
  In this role, you will function as an in-house implementation team for the companies that Redesign Health launches (internally referred to as OpCos). We provide data strategy, data pipeline, data analytics and forecasting services to newly formed companies in a repeatable and scalable manner...
   

Training & Resources

• Building voice powered To-Do list Android App with Alan AI
  How to let a voice assistant handle our To-Do list? In this article, you’ll learn how to build a To-Do list Android app that is powered by an Intelligent Voice Interface and can be controlled with voice commands...

• chalk-diagrams
  Chalk is a declarative drawing library built on top of PyCairo. The API draws heavy inspiration from Haskell's diagrams, Scala's doodle and Jeremy Gibbons's lecture notes on Functional Programming for Domain−Specific Languages...

• The next big breakthrough in AI will come from hardware, not software [Twitter Thread]
  Training giant models like PaLM already require 1000s of chips consuming several MW, and we will probably want to keep scaling these up several orders of magnitude. How can we do it? a 🧵...

What you’re up to – notes from DSW readers

• PyMC Decs are working on PyMC 4.0 - a new major release of the popular probabilistic programming library for Python that allows users to build Bayesian models with a simple Python API and fit them using Markov chain Monte Carlo (MCMC) methods. This new release comes with many new features including a new backend, JAX and GPU support, better sampling and much more. Check out the full release announcement...

* To share your projects and updates, share the details here.

** Want to chat with one of the above people? Hit reply and let us know :)

Last Week's Newsletter's 3 Most Clicked Links

• 8 surprising ways how to use Jupyter Notebook

• Some thoughts on machine learning with small data

• How to play with the GPT-3 language model

* Based on unique clicks.

** Find last week's newsletter here.