Good morning. I’m up in Albany today for something cool you’ll be hearing about shortly. That was a tease. Keep your eye out!

In other news, a coalition of more than 100 AI researchers published an open letter last week affirming the continued existence and threat of bias and discrimination in AI.

The group of scientists is urging “policymakers to continue to develop public policy that is rooted in and builds on this scientific consensus, rather than discarding the bipartisan and global progress made thus far.”

— Ian Krietzberg, Editor-in-Chief, The Deep View

In today’s newsletter:

AI for Good: Dealing with tuberculosis

India, home to more than a quarter of the world’s tuberculosis infections, is at the center of a global battle against the disease. In an effort to meet its target of total TB elimination by 2025, the country has turned to AI-powered technological innovations. 

The details: Prominent among these is something called DeepCXR, an AI tool developed by the Institute for Plasma Research that is designed to detect early traces of TB in chest X-rays. 

“AI tools are expected to be a game-changer in detecting presumptive TB patients and quick initiation of treatment,” Minister of State for Health and Family Welfare, Anupriya Patel, said in a statement. 

While Patel highlighted India’s progress — aided and abetted by technological innovations — in reducing TB infections and deaths, doctors as recently as August said that India’s goal of eliminating TB by 2025 was “unachievable.” 

200+ hours of research on AI tools & hacks packed in 3 hours

This free 3-hour Mini Course on AI & ChatGPT (worth $399) will help you become a master of 20+ AI tools & prompting techniques and save 16 hours/week.

Get it now for absolutely free! (for first 100 users only) 🎁

This course will teach you how to:

Register & save your seat now! (100 free seats only)

Apple faces proposed class action over its lag in Apple Intelligence

Apple, already moving slowly out of the gate on generative AI, has been dealing with a number of roadblocks and mounting delays in its effort to bring a truly AI-enabled Siri to market. The problem, or, one of the problems, is that Apple used these same AI features to heavily promote its latest iPhone, which, as it says on its website, was “built for Apple Intelligence.” 

Now, the tech giant has been accused of false advertising in a proposed class action lawsuit that argues that Apple’s “pervasive” marketing campaign was “built on a lie.” 

The details: Apple has — if reluctantly — acknowledged delays on a more advanced Siri, pulling one of the ads that demonstrated the product and adding a disclaimer to its iPhone 16 product page that the feature is “in development and will be available with a future software update.”

Apple did not respond to a request for comment. 

The lawsuit was first reported by Axios, and can be read here. 

This all comes amid an executive shuffling that just took place over at Apple HQ, which put Vision Pro creator Mike Rockwell in charge of the Siri overhaul, according to Bloomberg. 

Still, shares of Apple rallied to close the day up around 2%, though the stock is still down 12% for the year. 

Transform Your Business with Advanced AI Automation

Discover how to use AI beyond basic copywriting and research. Today, it's possible to use AI to drive more sales, generate quality leads, and optimize efficiency. This exclusive webinar reveals proven strategies that enable AI to work for you — turning every facet of your business into a growth engine.

Reserve your spot now and start monetizing your business.

Meta has been quietly earning revenue from Llama, court filing shows

Meta CEO Mark Zuckerberg, a staunch defender of so-called open-source technology, wrote an impassioned defense of open AI last July, saying at the time that “a key difference between Meta and closed model providers is that selling access to AI models isn’t our business model.” 

Openly releasing Llama, according to Zuckerberg, “doesn’t undercut our revenue, sustainability, or ability to invest in research like it does for closed providers.” 

The news: But, beyond the fact that Llama isn’t really open-source by any acceptable definition, newly unredacted filings in the class action lawsuit Kadrey V. Meta revealed that Meta is actually deriving some level of revenue from its Llama models.

The finding is significant to the case, which argues that Meta didn’t just knowingly train its model on datasets of pirated books, but that the company engaged in something called “seeding,” in which a user shares stolen data with other users.

The case has revealed internal communications from Meta’s AI team that show that, in an effort to catch up to the competition, Meta began pursuing the licensing of high-quality content in the form of books. But the effort was too expensive and time-consuming, according to the documents, so Meta received approval from Zuckerberg to use datasets of pirated books instead. 

The challenges of making AI for mental health care work 

Benchmarks have become something of a holy grail in the AI field. 

When we’re dealing with systems designed to automate high-level knowledge work, the need for some method of measurement is clear. But the problem, as we’ve often discussed in this newsletter, is that benchmarks are rarely a good indicator of real-world performance, especially when it comes to generative AI. 

In some realms, and for some people, the difference might not matter. 

But it’s a piece of nuance that gets more important as these models get integrated into higher-stakes environments, as it relates heavily to levels of user trust.

This is especially true of GenAI integrations in healthcare fields, an integration that is happening in full force, today. Many GenAI systems in healthcare are back-end systems, employed by researchers to speed up drug development, for instance. But increasingly, we’re seeing the rise of AI-powered clinical assistants and notetakers designed to help out nurses and reduce the administrative burden faced by doctors. 

Ethics and challenges aside, it's an integration that requires a robust reliability calculus; an effective benchmark.  

Researchers at Stanford just proposed one for mental health, a field that has not been spared the unrelenting push of AI integration. 

The details: Current benchmarks, according to the paper, are built to mimic exams, and come complete with multiple-choice answer options. The problem with this is that, “even for humans … success in these standardized tests only weakly correlates with clinicians’ real-world performance, a disconnect that can be especially problematic in psychiatry, where diagnosis and management hinge on subjective judgments and interpersonal nuances.” 

This is even worse with the benchmarks that assess LLMs, according to the researchers, which “over-simplify the complexities of day-to-day clinical practice tasks.” 

The researchers assessed a number of off-the-shelf models from developers including OpenAI, Anthropic and Meta. The models, which aren’t designed or intended for clinical applications, performed well in the diagnosis category, collectively achieving an 80% accuracy rate. 

But they averaged accuracy rates of less than 50% for triage and documentation, clocking only a 67% for monitoring and a 76% for treatment. 

According to the researchers, the models “perform well on structured tasks,” but “struggle significantly with ambiguous real-world tasks … underscoring the limitations of current AI models in handling uncertainty.” 

Under the impression, laid out so elegantly by Psychology Today, that “like it or not, (AI) is here to stay,” we’re going to need many more benchmarks of this nature — expert-curated and as ambiguous and nuanced as humanly possible. They’re harder to train for, and they are absolute necessities when clinicians are considering the adoption of these tools. 

Experts in domains beyond computer science absolutely need to be made aware of just how reliable, and just how trustworthy, generative AI models are (or are not). Even 80% accuracy rates ought to be pretty unacceptable in a bunch of fields. 

Of course, as the paper mentions, this doesn’t address a couple of key problems that surround this integration, which include data security and privacy, consent, reliability and algorithmic bias. 

Taken together, it adds up to the risk that “AI systems could be prematurely deployed in psychiatric care, potentially leading to harmful, biased or unreliable clinical decisions.” 

But I guess you can’t have everything. 

This is the start of a start. 

🤔 Your thought process:

Selected Image 1 (Left):

Selected Image 2 (Right):

💭 A poll before you go

Thanks for reading today’s edition of The Deep View!

We’ll see you in the next one.

Here’s your view on Claude’s web search:

A third of you think it’s a bit of a weird rollout; you already use other search-enabled systems.

21% think it comes too late and 14% think it puts Anthropic back in the game.

Back in the game:

Weird rollout:

Yes.

If you want to get in front of an audience of 450,000+ developers, business leaders and tech enthusiasts, get in touch with us here.