AI Digest - Thinking like transformers
#161 — June 21, 2021 | View in browser |
AI Digest
Spread the word, build the community, share the knowledge – invite your friends.
this week's favorite
What is the computational model behind a Transformer? Where recurrent neural networks have direct parallels in finite state machines, allowing clear discussion and thought around architecture variants or trained models, Transformers have no such familiar parallel. In this paper we aim to change that, proposing a computational model for the transformer-encoder in the form of a programming language. We map the basic components of a transformer-encoder -- attention and feed-forward computation -- into simple primitives, around which we form a programming language: the Restricted Access Sequence Processing Language (RASP). We show how RASP can be used to program solutions to tasks that could conceivably be learned by a Transformer, and how a Transformer can be trained to mimic a RASP solution. In particular, we provide RASP programs for histograms, sorting, and Dyck-languages. We further use our model to relate their difficulty in terms of the number of required layers and attention heads: analyzing a RASP program implies a maximum number of heads and layers necessary to encode a task in a transformer. Finally, we see how insights gained from our abstraction might be used to explain phenomena seen in recent works.
Homepage feed multi-task learning using TensorFlow
LinkedIn’s members rely on the homepage feed for a variety of content including updates from their network, industry articles, and new job opportunities. Members can engage with this content in multiple ways, including commenting on or resharing posts. It is the job of our relevance system to understand the different needs of these members and present a personalized experience that provides an ecosystem optimized to foster productive professional development.
13 essential tips for learning machine learning and data science
Once you’ve mastered the topic, it’s easy to look back and see the fastest path from noob to pro. If you only could go back in time and give yourself the roadmap. Even if I cannot do that with myself, I can do that for others. This is the objective of this article: to give you the tips I wish I knew when I started learning data science and machine learning.
Towards causal representation learning
The two fields of machine learning and graphical causality arose and developed separately. However, there is now cross-pollination and increasing interest in both fields to benefit from the advances of the other. In the present paper, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.
Real-time facial surface geometry from monocular video on mobile GPUs
We present an end-to-end neural network-based model for inferring an approximate 3D mesh representation of a human face from single camera input for AR applications. The relatively dense mesh model of 468 vertices is well-suited for face-based AR effects. The proposed model demonstrates super-realtime inference speed on mobile GPUs (100-1000+ FPS, depending on the device and model variant) and a high prediction quality that is comparable to the variance in manual annotations of the same image.
newsletters
Older messages
HyperLib: Deep learning in the Hyperbolic space
Sunday, June 6, 2021
And more news, tutorials and articles about AI, machine learning, and data science in this week's issue. #159 — June 07, 2021 View in browser AI Digest Spread the word, build the community, share
An introduction to knowledge graphs
Sunday, May 30, 2021
And more news, tutorials and articles about AI, machine learning, and data science in this week's issue. #158 — May 31, 2021 View in browser AI Digest Spread the word, build the community, share
Optimizing payments with machine learning
Sunday, May 23, 2021
And more news, tutorials and articles about AI, machine learning, and data science in this week's issue. #157 — May 24, 2021 View in browser AI Digest Spread the word, build the community, share
The modern mathematics of deep learning
Sunday, May 16, 2021
And more news, tutorials and articles about AI, machine learning, and data science in this week's issue. #156 — May 17, 2021 View in browser AI Digest Spread the word, build the community, share
Practical SQL for data analysis
Sunday, May 9, 2021
And more news, tutorials and articles about AI, machine learning, and data science in this week's issue. #155 — May 10, 2021 View in browser AI Digest Spread the word, build the community, share
You Might Also Like
Introducing SwiftUI to the team, Let loose and much more!
Friday, April 26, 2024
View in browser Hello, you're reading Infinum iOS Cocoa Treats, bringing you the latest iOS related news straight to your inbox every week. How We Got Everyone on Board with a New Technology
SWLW #596: SRE and the art of improvisation, The power of celebration, and more.
Friday, April 26, 2024
Weekly articles & videos about people, culture and leadership: everything you need to design the org that makes the product. A weekly newsletter by Oren Ellenbogen with the best content I found
Notion's New Look | In-App Notifications | Duplicated Automations
Friday, April 26, 2024
Your weekly Notion digest with the finest tip, latest news, and improvements! 🔥 ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Don’t celebrate the demise of non-competes quite yet
Friday, April 26, 2024
A slew of lawsuits are heading the FTC's way after it voted to kill non-competes View this email online in your browser By Alex Wilhelm Friday, April 26, 2024 Welcome to TechCrunch AM! I love this
The best work laptop for most people
Friday, April 26, 2024
Old Kindle? Don't do this; How to change your IP address; Net neutrality restored -- ZDNET ZDNET Tech Today - US April 26, 2024 placeholder The work laptop I recommend to most people is not made by
New 'Brokewell' Android Banking Malware Spread Through Fake Browser Updates
Friday, April 26, 2024
THN Daily Updates Newsletter cover Webinar -- Uncovering Contemporary DDoS Attack Tactics -- and How to Fight Back Stop DDoS Attacks Before They Stop Your Business... and Make You Headline News.
Apple announces an iPad event for May 7
Friday, April 26, 2024
The Morning After It's Friday, April 26, 2024. Apple has scheduled its next product showcase for May 7, a few weeks before its Worldwide Developers Conference, clearing space for even more
[AI Incubator] Enrollment ends tonight.
Friday, April 26, 2024
Text or WhatsApp me your questions
Post from Syncfusion Blogs on 04/26/2024
Friday, April 26, 2024
New blogs from Syncfusion Create Excel Table in Just 3 Steps Using C# By Mohan Chandran This blog explains how to create a table in an Excel document using Syncfusion .NET Excel Library in C# with code
ASP.NET Core News - 04/26/2024
Friday, April 26, 2024
View this email in your browser Get ready for this weeks best blog posts about ASP.NET Core! This newsletter is sponsored by elmah.io - the most advanced, yet so simple to set up, error logging and