📝 Editorial 

The current era of deep learning is, without a doubt, dominated by massively large models. Since the release of models like OpenAI’s GPT-3, research labs have immersed themselves in producing larger and more scalable neural network models. This trend is undoubtfully dominant in natural language processing (NLP) but has been rapidly expanding to other domains such as computer vision and speech intelligence. Big AI labs with virtually unlimited computation resources have been certainly pushing the scalability boundaries of deep learning architectures.  

Last week, we saw two new examples of large models achieving impressive milestones. OpenAI released details about the latest version of DALL-E, one of the first deep learning models able to generate photorealistic, artistic images from natural language descriptions. Like the previous version, DALL-E 2 relies upon large pretrained models for language understanding and image generation. DALL-E 2 improves upon its predecessor by incorporating more knowledge about artists’ styles and better mechanisms to optimize images. Google has also been busy in the super model trend by unveiling PaLM, a new transformer model with an astonishing 540 billion parameters that can master many tasks, including language comprehension. One of the key innovations of PaLM is the flexibility to be trained in many hardware accelerators.  

The trend of large deep learning models continues! While many experts argue that this trend is unsustainable and that these large models are just sophisticated memorization machines, it is unlikely that the trend of super models will end any time soon.  

Share

🔺🔻 TheSequence Scope is our Sunday free digest. To receive high-quality educational content about the most relevant concepts, research papers, and developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻

🗓 Next week in TheSequence Edge:

Edge#181: we start a new series about high scale ML training; explain SeedRL; overview Horovod. 

Edge#182: we explore the topic of ethical data labeling.

Subscribe if you haven’t yet

Now, let’s review the most important developments in the AI industry this week

🔎 ML Research

DALL-E 2 

OpenAI published a research paper detailing DALL-E 2, which improves its original model to generate artistic images from textual descriptions →read more on OpenAI blog

PaLM

Google Research published a paper discussing Pathways Language Model (PaLM), a 540 billion parameter transformer model that mastered many tasks and can be efficiently trained on different hardware architectures →read more on Google Research blog

VDTTS 

Google Research published a paper outlining VDTTS, a visually-driven text-to-speech model that can generate high-quality audio from text and video frames →read more on Google Research blog

🤖 Cool AI Tech Releases

Google Cloud’s New ML Offering 

Google Cloud announced new releases such as the GA of Vertex AI Workbench and the new Vertex AI Model Registry which provide a consistent experience to manage the lifecycle of ML models →read more on Google Cloud blog

BigLake 

Google Cloud announced BigLake, a new service that unifies data lakes and data warehouses across different platforms →read more on Google Cloud blog 

DINO’s SSL demos

Meta AI Research (FAIR) launched a new series of demos that showcased the capabilities of DINO, a self-supervised learning model for computer vision →read more on DINO demo website 

🔺Open Roles at feature-oriented database platform Molecula🔻

• Site Reliability Engineer: https://grnh.se/e2b29f1e4us 

• Principal Site Reliability Engineer: https://grnh.se/c1f4f1484us 

• View all 10 open roles >

🛠 Real World ML  

Multi Text-to-Speech Model Framework 

Amazon Research published a blog post detailing a framework to enable the coexistence of multiple text-to-speech models for a given task →read more on Amazon Research blog

💸 Money in AI

AI-powered

• Contract management platform LinkSquares raised $100 million in Series C financing led by G Squared. Hiring in Boston/US.

• Cyber security platform Coro raised $60 million in a Series C round led by Balderton Capital. Hiring in Chicago and New York/US, in Tel Aviv/Israel.

• Business translation software Lilt raised $55 million in Series C funding found led by Four Rivers. Hiring globally.

• Facility design and operation platform BeamUP raised a $15 million round of seed investment led by StageOne Ventures and Ibex Investors. Hiring in Tel Aviv.

• Conversation platform Arena raised $13.6 million in a Series A round led by CRV. Hiring remote.

• Sales enablement platform Amplemarket raised $12 million in back-to-back Seed and Series A investment rounds co-led by Comcast Ventures and Armilar Venture Partners. Hiring globally.

• Data privacy automation platform LightBeam.ai raised $4.5 million in a seed funding round. Hiring.