📝 Editorial 

Machine learning (ML) is eating software, and natural language processing (NLP) is eating ML. This is one of the new catchy phrases within the ML community that captures the sentiment about the progress in NLP techniques. In the last few years, NLP has distanced itself from the rest of the deep learning field in terms of achievements both in research and practical applications. Language pretrained and transformer models such as GPT-3 have transcended the boundaries of AI labs to become part of the mainstream tech culture. With the high bar set up by NLP, we should wonder whether there is another deep learning school ready for a major breakthrough?  

Computer vision seems to be following the footsteps of NLP and making astonishing progress in the last few years. Part of that progress has been triggered by the adaptation of transformer architectures to computer vision domains, but that’s not the only catalyzer. Self-supervised learning (SSL) is one of the fastest-growing areas of deep learning and seems to be a perfect fit for tackling different computer vision challenges. Unlike NLP, computer vision problems often lack highly accurate labeled datasets, which makes it an ideal candidate for SSL techniques. Meta (Facebook) has been the big champion behind the application of SSL methods to computer vision tasks. Just last week, Meta unveiled a new version of its famous SEER (SElf-SupERvised) model for computer vision which continues to achieve remarkable milestones. Even though computer vision is considerably more complex than NLP as a general domain, the combination of SSL techniques and transformers seems to push the space's boundaries. At this rate, computer vision could soon have its GPT-3 moment.  

Share

🔺🔻 TheSequence Scope is our Sunday free digest. To receive high-quality educational content about the most relevant concepts, research papers, and developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻

🗓 Next week in TheSequence Edge:

Edge#171: we introduce DCGANs; explain the DCGAN Paper; explore Nvidia Imaginaire, a GAN Library for image and video translation. 

Edge#172: we deep dive into DeepMind AlphaCode that can generate code at the level of programming competitions.

Now, let’s review the most important developments in the AI industry this week

🔎 ML Research

Self-Supervised Computer Vision 

Meta (Facebook) AI published a paper detailing advancements in SEER, their marquee self-supervised computer vision model →read more Meta AI blog

Co-training Transformers Using Videos 

Google Research published a paper outlining a training technique that uses videos to improve action recognition in transformer models →read more on Google Research blog

Federated Learning with Differential Privacy 

Google AI Research published a detailed blog post summarizing their journey to enable differential privacy techniques in federated learning models →read more on Google Research blog

Improving QA Models in Tabular Datasets 

Amazon Research published a paper illustrating a pretraining method to improve question-answering models that work with structured tabular datasets →read more on Amazon Research blog

🤖 Cool AI Tech Releases

Next-Gen AI Chips 

AI-first chipmaker Graphcore launched a new generation of AI chips and unveiled the plans for the new AI supercomputer →read more on Graphcore blog

🛠 Real World ML

Real-Time ML at LinkedIn 

The LinkedIn engineering team published a blog post describing their ML architecture for real-time recommendations →read more their blog

Virtual Assistants at Walmart 

Walmart Labs published a blog post detailing the multi-task models used to unify models powering different virtual assistants →read more on Walmart Labs blog

Using PyTorch at Amazon Ads 

Amazon and PyTorch published an insightful blog post about their use of PyTorch and AWS Inferentia to develop and scale models for ad processing →read more on PyTorch blog

💸 Money in AI

• AI supercomputer developers Luminous Computing raised $105 million in a Series A round, with participation from investors including Gigafund, Bill Gates, 8090 Partners, Neo, Third Kind Venture Capital, Alumni Ventures Group, and others. Hiring in California, New York, Texas/US.

• Marketing solutions provider Metadata.io raised $40 million Series B funding round, led by Next47 and Resolute Ventures. Hiring remote.

• Productivity startup Time By Ping raised $36.5 million in a series B funding round led by ACME and Anthos. Hiring remote in the US.

• Automatic speech recognition startup AssemblyAI raised a $28 million Series A led by Accel. Hiring remote.

• Threat coverage optimization company CardinalOps raised $17.5 million in Series A funding led by Viola Ventures. Hiring in Israel.

• Engineering operations platform Faros AI raised $16 million in seed funding led by SignalFire, Salesforce Ventures, and Global Founders Capital. Hiring in San Francisco Bay Area/US.

• Productivity automation startup ZERØ raised $12 million Series A investment round, led by Streamlined Ventures.

• Computer vision startup for recycling industry TrueCircle raised $5.5 million in pre-seed funding round led by Lowercarbon Capital fund. Hiring in London/UK.

• Customer feedback intelligence startup Enterpret raised $4.3 Million in seed round led by Kleiner Perkins. Hiring in Banglore/India.

Acquisition

• Open-source app framework for ML and data science team Streamlit was acquired by Snowflake for $800 million. Hiring remote

*This news digest is presented by Superb AI’s team. We thank Superb AI for their support of TheSequence.

About Superb AI

Superb AI is an advanced DataOps platform looking to transform the way Computer Vision teams prepare and iterate on datasets. The Superb AI Suite provides automation products and tools across all steps of the data preparation workflow including data labeling, auditing, management and curation.