OpenEvidence quietly makes accounts available to medical librarians

OpenEvidence is a large language model (LLM) grounded in the medical literature, with its website self-identifying as an "AI copilot for doctors that helps them make high-stakes decisions at the point of care." The free tool has become increasingly popular within the health sciences, with its website boasting that "This year, more than 100 million Americans will be treated by a clinician using OpenEvidence."

Up until now, OpenEvidence accounts have been restricted to health care professionals with a National Provider Identifier (NPI) or students that provide proof of enrollment to a health sciences college or university, which has made evaluation of the tool particularly challenging for librarians (but even that hasn't stopped us: Exhibits A and B. Librarians are a tenacious bunch!😉).

However, at long last (and likely in consequence of tenacious librarians spamming their inboxes),  OpenEvidence has quietly opened account access to medical librarians. Now, librarians have the option to register as a "medical librarian" so long as they upload proof of their current position title and employment (I was able to create my account by uploading a screenshot of my public staff page, which includes my photo and my position as a health sciences librarian). Not only does this give you unlimited queries (without an account you were only allotted 2 or 3 queries per week), but it gives you access to additional tools in OpenEvidence, such as their calculators, clinical trials locators, etc.

For those who have seen one of our Wild West presentations or our MLA poster session, or read our article on OpenEvidence, you've likely gotten the impression that my colleagues and I are more than a little skeptical of OpenEvidence; and we've actively encouraged others to closely evaluate its outputs.

However, giving credit where credit is due, OpenEvidence opening access to medical librarians is a step in the right direction, and will allow us to get an inside look into the tool. 

Now that we've got the keys to the castle, we can better search for cracks in the foundation. 🔍

Happy sleuthing, everyone! ☕

Article of Interest: Limitations of Open Evidence (by PAIJE!)

Congratulations to Paije Wilson and her four colleagues from UW–Madison on their recent publication in the Journal of the American College of Clinical Pharmacy!

Their article tackles the critical topic of AI accuracy in healthcare, providing a thorough review of current literature on OpenEvidence. By running specific pharmacotherapeutic queries, the team effectively demonstrated inaccuracies in how the tool generates responses and summarizes its sources. Crucially, the authors delve into source summarization inaccuracies—a nuance in AI performance that has largely been overlooked in current research.

Link to study: https://accpjournals.onlinelibrary.wiley.com/doi/10.1002/jac5.70237

Upcoming WHSLA Wisdom Chat July 25: Ebling Librarians will talk about Generative AI (including OpenEvidence)

Curious about generative AI? 

Attend the next WHSLA Wisdom Chat on 7/25/2025 at 1pm!

Artificial intelligence (AI) is everywhere these days. As librarians, navigating the AI landscape can seem overwhelming. This upcoming Wisdom Chat, presented by Ebling Librarians Leslie Christensen, Lia Vellardita, Susan Vandagriff, and Paije Wilson from the University of Wisconsin-Madison, will provide a critical look at popular generative AI tools within the context of literature summarization and searching. Specifically, the chat will include:

• A brief overview of the different types of AI and how they relate to each other
• The benefits and limitations of summarizers for individual articles, with an example from Elicit
• The benefits and limitations of summarizers for a body of literature, with examples from Perplexity and OpenEvidence
• An in-depth, critical discussion of OpenEvidence
• The benefits and limitations of chatbots for the creation of PubMed search strategies, with an example from ChatGPT

Attendees will walk away with knowledge of the benefits and limitations of some available generative AI tools, awareness of these tools' potential applications, and fundamental skills on how to approach these tools using a critical lens.

Join us for this presentation, followed by our usual round of sharing.

When: 1 pm on July 25, 2025 via Zoom

WHSLA Wisdom Chats are open to all WHSLA members in good standing.  If you are a current member and did not receive an invitation, please contact Michele Matucheski, WHSLA Wisdom Chat Coordinator filling in for Barb Ruggieri this year.