UX blog post: Searching for author names in PubMed

 

Image by Pat from Pixabay

 Searching for authors in databases like PubMed can feel a bit like finding a needle in a haystack (especially when you're faced with searching for entire departments of authors!). This new UX Database Tips Blog post, written by yours truly, has some tips for author searching PubMed; a discussion of some of PubMed's quirks for such searches; and an Excel workaround, which has saved me countless hours in generating searches for departments with 100+ authors.

Happy reading! May your search syntax be spotless, your internet connection swift, and your coffee (or tea) ever hot!

Can automatic indexing algorithms compare apples to Apple's?: An overview of automatic indexing in PubMed, errors, and potential band aids.

 

Image by Michal Jarmoluk from Pixabay

Inane indexing

As librarians, we've encountered some…well, let's say interesting indexing choices in PubMed records.

Such as this case, where an article about Apple iPad Pro's was indexed with the MeSH term, "Malus" (being the genus name for apple tree)…

…or this case, where an article about early identification of autism was assigned the MeSH term, "humans." Anything else? Nope. Just "humans."

So what's going on here? Well, there are a few different ways a record can be indexed in PubMed [1]:

• Manual: This is where a record was indexed manually by an indexer at the National Library of Medicine (NLM).
• Automated: This is where a record was indexed by an algorithm, with no review by human indexers.
• Curated: This is where a record was indexed by an algorithm, but was later reviewed/modified by human indexers.

If the indexing of a record seems particularly bizarre, such as in our examples, odds are the record may have been automatically indexed.

A bit of Background on Automatic Indexing in PubMed

Automatic indexing isn't new to PubMed. In 2002, automatic indexing began as simple suggestions given to indexers at the NLM as they reviewed a record [2-6]. In 2011, first line automated indexing with human review was implemented for a selection of 14 journals [2, 4, 6, 7]. This progressed to full automatic indexing for all OLDMEDLINE records in 2015, comments and backlogged citations in 2016, and, finally, full automatic indexing for all MEDLINE journals in 2022, with the NLM's newest automatic indexing algorithm, Medical Text Indexer NeXt generation (MTIX) being introduced in 2024 [4, 7, 8].

While automatic indexing has improved efficiency, it has also resulted in issues relating to precision and recall, as demonstrated in the examples I had alluded to earlier. A few studies have documented these errors more formally [3, 9-12], including an article I recently published in the Journal of the Medical Library Association (JMLA), which examined precision errors observed in a sample of MEDLINE records automatically indexed with the MeSH term, Malus, being the genus name for apple tree [13].

How do I check how a PubMed record was indexed?

For those curious, there's a nifty trick you can use to check how an item is indexed. You can do this by looking at the PubMed record in XML. To access a PubMed record in XML, simply use the URL below, replacing the highlighted number with the PMID of the record you would like to check.

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=24410017%2c11700088&retmode=xml

The indexing method is typically listed to the right of the label "indexingmethod=".

Below are screenshots of the XML of both of our examples from earlier. Sure enough, they were both automatically indexed.

Link to XML for Example 1

Link to XML for Example 2

What can we do about automatic indexing errors in PubMed?

While we can't fix automatic indexing errors in PubMed, ourselves, or wholly prevent them from occurring, there are a few ways to ameliorate their effects.

In addition to conducting additional studies on automatic indexing errors and reporting errors to the NLM HelpDesk, librarians can instruct researchers to use standardized language and the very MeSH terms they would like to see assigned to their records in the titles and abstracts of their manuscripts (at the least, this way, records may have correct MeSH terms appear alongside the erroneous ones).

Fellow librarians from the University of Iowa developed a checklist that has additional tips for helping researchers reduce the effects of automatic indexing errors in their own manuscripts [14].

Will automatic indexing algorithms ever compare, with 100% accuracy, apples to Apple's? Who knows. But, regardless of whether indexing errors cause you laughter or grief, they're likely to be sticking around for a while.

 

Works Cited / Fun Reading

1.      Incorporating values for indexing method in MEDLINE/PubMed XML [Internet]. NLM Technical Bulletin; 2018. Available from: https://www.nlm.nih.gov/pubs/techbull/ja18/ja18_indexing_method.html#note

2.      Mork J, Aronson A, Demner-Fushman D. 12 years on - Is the NLM medical text indexer still useful and relevant? J Biomed Semantics. 2017 Feb 23;8(1):8. Epub 20170223. DOI: 10.1186/s13326-017-0113-5. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5324252/pdf/13326_2017_Article_113.pdf.

3.      Chen E, Bullard J, Giustini D. Automated indexing using NLM's Medical Text Indexer (MTI) compared to human indexing in Medline: a pilot study. J Med Libr Assoc. 2023 Jul 10;111(3):684–94. DOI: 10.5195/jmla.2023.1588. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10361558/pdf/jmla-111-3-684.pdf.

4.      MEDLINE 2022 Initiative: Transition to Automated Indexing [Internet]. NLM Techincal Bulletin; 2021. Available from: https://www.nlm.nih.gov/pubs/techbull/nd21/nd21_medline_2022.html

5.      Rae AR, Pritchard DO, Mork JG, Demner-Fushman D. Automatic MeSH indexing: revisiting the subheading attachment problem. AMIA Annu Symp Proc. 2020;2020:1031–40. Epub 20210125. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8075546/pdf/139_3413087.pdf.

6.      Mork J, Jimeno Yepes A, Aronson A. The NLM Medical Text Indexer System for indexing biomedical literature. 2013. Available from: https://lhncbc.nlm.nih.gov/ii/information/Papers/MTI_System_Description_Expanded_2013_Accessible.pdf.

7.      Frequently Asked Questions about Indexing for MEDLINE: National Library of Medicine; 2023. Available from: https://www.nlm.nih.gov/bsd/indexfaq.html.

8.      Sticco A. NLM Office Hours: MEDLINE Indexing Update 2024. Available from: https://www.nlm.nih.gov/oet/ed/pubmed/02-24_oh_medline-automated-indexing.html.

9.      Aronson AR, Mork JG, Gay CW, Humphrey SM, Rogers WJ. The NLM Indexing Initiative's Medical Text Indexer. Stud Health Technol Inform. 2004;107(Pt 1):268–72. Available from: https://ebooks.iospress.nl/pdf/doi/10.3233/978-1-60750-949-3-268.

10. Fernandez-Llimos F, Negrão LG, Bond C, Stewart D. Influence of automated indexing in Medical Subject Headings (MeSH) selection for pharmacy practice journals. Res Social Adm Pharm. 2024 Jun 12. Epub 20240612. DOI: 10.1016/j.sapharm.2024.06.003.

11. Moore DAQ, Yaqub O, Sampat BN. Manual versus machine: How accurately does the Medical Text Indexer (MTI) classify different document types into disease areas? PLoS One. 2024;19(3):e0297526. Epub 20240313. DOI: 10.1371/journal.pone.0297526. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10936797/pdf/pone.0297526.pdf.

12. Brief Communication – concerning algorithmic indexing in MEDLINE. Journal of EAHIL. 2024 03/17 [cited 2025/04/09];20(1):18–21. DOI: 10.32384/jeahil20604. Available from: https://doi.org/10.32384/jeahil20604.

13. Wilson P. Sometimes the apple does fall far from the tree: a case study on automatic indexing precision errors in PubMed. J Med Libr Assoc. 2025 Oct 23;113(4):318–26. DOI: 10.5195/jmla.2025.2110.

14. Allen C, Carol H, Riley S, Deberg J. Developing an Author Checklist to Improve Discovery of Published Articles in the Era of Algorithmic Indexing. 2025. Zenodo. https://doi.org/10.5281/zenodo.17282091

Thanks for reading!

PubMed Features to Save Your Time

 

Knapp M. PubMed features to save your time. J Hosp Librarianship. 2024;24(1):1-9. doi: 10.1080/15323269.2023.2291284. Epub 2023 Dec 7. PMID: 38645880; PMCID: PMC11029388.

A brand new article was recently published in The Journal of Hospital Librarianship by the NNLM's Molly Knapp, highlighting some of the time-saving features of PubMed including the following:

• Computed Author Sort (aka authorbot)
• PMID search
• Phrase Index Searching
• Phrase searching using proximity search
• Case Studies:
• Show me everything in PubMed with the ultimate cheat code: all[sb]
• Show me ALL the systematic reviews in PubMed
• Why does PubMed ignore my search terms?

Even if all of these features are old hat for WHSLA Members, it's always good to review what's available as far as time-saving features in PubMed. Maybe you'll pick up some new tips and tricks!

Is MEDLINE different across platforms? You bet.

A co-worker recently shared this University of Michigan Taubman Talk featuring the authors of a 2021 PLoS study on how MEDLINE performs across different platforms: "MEDLINE search retrieval issues: A longitudinal query analysis of five vendor platforms". 

MEDLINE platforms included:

• PubMed
• ProQuest
• EBSCOhost
• OVID
• Web of Science

Which of these 5 performed the "best"? Access the recording and accompanying slides here. 

MEDLINE is the National Library of Medicine's (NLM) premier bibliographic database that contains references to journal articles in life sciences, with a concentration on biomedicine.

PubMed Update: Proximity Search Now Available in PubMed

  Reprinted from: NLM Tech Bull. 2022 Nov-Dec;(449):e4.  Posted 29 November 2022.

The National Library of Medicine is pleased to announce that proximity search capabilities have been added to PubMed. Users can now search for multiple terms appearing in any order within a specified distance of one another in the [Title] or [Title/Abstract] fields.

How to Build a Proximity Search in PubMed

To create a proximity search in PubMed, enter terms using the following format:

"search terms"[field:~N]

• Search terms = Two or more words enclosed in double quotes.
• Field = The search field tag for the [Title] or [Title/Abstract] fields.
• N = The maximum number of words that may appear between your search terms.

For example, to search PubMed for citations where the terms "hip" and "pain" appear with no more than two words between them in the Title/Abstract search field, try the search:

"hip pain"[Title/Abstract:~2]

Search results may include hip pain, hip-related pain, hip joint pain, hip/groin pain, hip biomechanics and pain, pain after total hip arthroplasty, pain in right hip, and more.

See the PubMed User Guide and view the proximity searching tutorial for more examples and information about proximity searching in PubMed.

Figure 1: PubMed search page for hip pain.

PubMed Proximity FAQs

• When should I use proximity search?
• Is there a limit to the number of terms I can enter in a proximity search?
• What N value should I use?
• Can I combine proximity searches with Booleans and other search terms?
• Can I specify the order my terms appear in a proximity search?
• Can I use proximity search in any field in PubMed?
• Can I use abbreviated search field tags for proximity searching?
• Can I use truncation (*) in a proximity search?
• Will PubMed translate the terms in my proximity search with Automatic Term Mapping (ATM)?
• Can I search for an exact phrase in proximity to other terms?
• My proximity search terms are highlighted in the search results with more than my specified N distance between them--what happened?
• Where can I find more information and help for proximity searching in PubMed?

When should I use proximity search?

Proximity search adds another useful tool to your search toolkit. Searching for the same terms using a variety of techniques (e.g., combining terms with AND, searching for an exact phrase) and comparing the results can help you decide which option(s) to use. Proximity searching can be particularly helpful when searching for concepts that may be represented in multiple ways, or to capture variations of a phrase. For example, you may wish to find information about "healthcare rationing." The proximity search: "rationing healthcare"[tiab:~1] can retrieve citations where this concept appears as "healthcare rationing," "rationing of healthcare," "rationing strategies in healthcare," "rationing limited healthcare," and more.

Is there a limit to the number of terms I can enter in a proximity search?

There is no limit to the number of words you can search together with one proximity operator; however, the more words you include, the narrower your search becomes.

What N value should I use?

What N value to use will depend on your search. Try changing the N value and comparing the results to find what works best for your search. 

A higher N value creates a broader, more comprehensive search; this will typically retrieve more results overall, but some of these results may be less relevant. Using the Boolean operator AND to combine terms may be more appropriate than proximity searching with a large N value.  

A lower N value creates a narrower, more precise search; this will typically retrieve fewer results that are highly relevant but may exclude other relevant results.

If N=0, the quoted terms will appear next to each other--with no other words in between.

Can I combine proximity searches with Booleans and other search terms?

Yes, proximity searches can be combined with other search terms using Boolean operators, for example:

"standard care"[tiab:~2] AND lung cancer

Multiple proximity searches can also be combined using Booleans within one query, for example:

"standard care"[tiab:~2] AND "hip arthroplasty"[tiab:~1]

Can I specify the order my terms appear in a proximity search?

No, the order your search terms appear in results cannot be specified in a proximity search in PubMed.

Can I use proximity search in any field in PubMed?

Proximity search is available only in the [Title] and [Title/Abstract] fields.

Can I use abbreviated search field tags for proximity searching?

Yes, you can use the full search field tags [Title] and [Title/Abstract] or the abbreviated search field tags [ti] and [tiab] interchangeably.

Can I use truncation (*) in a proximity search?

Proximity searching is not compatible with truncation (*) in PubMed. If the double quoted terms in a proximity search include a wildcard (*), the proximity operator will be ignored.

Will PubMed translate the terms in my proximity search with Automatic Term Mapping (ATM)?

Automatic Term Mapping is not applied to proximity searches.

Can I search for an exact phrase in proximity to other terms?

You can combine a phrase search and a proximity search in the same query using Boolean operators; however, it is not possible to specify that an exact phrase appears within a certain distance to other terms.

For example, with the query "acute migraine treatment"[ti:~1] AND "pain management", the terms "acute," "migraine," and "treatment" will appear in the title with no more than one word between them, and the phrase "pain management" can appear anywhere in the record.

My proximity search terms are highlighted in the search results with more than my specified N distance between them--what happened?

The highlighting feature in PubMed does not incorporate query syntax; it uses simple term matching to show words from your query highlighted in bold when those word(s) appear in the search results. Check the full abstract if you are searching in the Title/Abstract field and you don't see the terms from your proximity search highlighted together in the preview "snippet" shown in the summary search results display.

Where can I find more information and help for proximity searching in PubMed?

Please view the Proximity Searching tutorial, and see the PubMed User Guide for more information about proximity searching.

If you have questions or feedback about this new feature, please contact the Help Desk using the Help link available at the bottom of every page in PubMed.

NLM Technical Bulletin

ISSN 2161-2986 (Online) 

Content not copyrighted; freely reproducible.

U.S. National Library of Medicine 8600 Rockville Pike, Bethesda, MD 20894
National Institutes of Health NIH...Turning Discovery into Health® Last updated: 29 November 2022.

Down the rabbit hole

Did you ever see an article trending on PubMed and then wonder to yourself "WHY on earth is that trending now?"  (Especially when an article that probably SHOULD be trending is much further down the list)Then you click on it because the title is so horrifying and curiosity gets the better of you (which lots of other people probably did too, and now the article is suddenly trending on PubMed).  

Before you know it, you're finding other related horrifying medical articles and staring at the reference list wondering why so many pregnant women were gored by cattle in 1887 that it warranted a study.

WHAT DID I JUST READ???

PubMed Update: Improvements to Email, Cite, and Save/Email/Send To

 

The National Library of Medicine (NLM) is continuing to enhance the PubMed user experience based on usability testing, analytics, and feedback from users.

Recent PubMed updates include emailing up to 1,000 citations, saving your most recently used Cite format as a preference, and convenient access to the Save/Email/Send to buttons.

Email up to 1,000 citations

You can now email up to 1,000 citations at a time (increased from 200). Batches of more than 200 citations will be divided into multiple emails, the subject line for each email will include a label indicating [1 of 5], [2 of 5], etc., and results are numbered continuously across emails.

Cite button remembers your citation style preference

Using the Cite button for an item opens a pop-up window where you can copy the citation formatted in four popular styles, with NLM style selected by default (see Figure 1). Changing the citation format to a different style will now be remembered as a preference until your browser data and cookies are cleared.

Please ensure that your browser and provider/institution allow cookies and other browser data for your customized settings to be remembered between visits. See "Cookies" in the PubMed User Guide for more information about how cookies are used and steps to troubleshoot common issues.

Figure 1: Change the Cite button citation style format using the drop down menu.

Improved workflow for Save, Email, and Send to

The Save, Email, and Send to buttons are now available to use from anywhere on the Search Results and Clipboard pages. After selecting one or more items, the Save, Email, and Send to buttons will appear at the top of your browser window (see Figure 2). Clearing selected items or returning to the top of the page will hide this interface.

Figure 2: The Save, Email, and Send to buttons appear after selecting one or more items.

By Jessica Chan
National Center for Biotechnology Information

The New & Improved PubMed (R) is Here!

Guest post by the PubMed Team at the National Library of Medicine’s National Center for Biotechnology Information.

This article first appeared on the NLM Musings blog on May 12, 2020.  

Since our last blog post, our PubMed coders have been hard at work preparing for the full transition to the new and improved PubMed. The latest features have been added, and beginning May 18, you can experience the new PubMed too!

The new PubMed features a modern interface with enhanced search results, including highlighted text snippets to help you preview an abstract while scanning your results list, and updated web elements for easier navigation. The new Best Match sort order uses advanced machine-learning technology and a new relevance search algorithm to bring you the top-ranked results.

All of these improvements are intended to connect you with the world’s leading sources of biomedical information faster and easier than ever before.

A Great Experience for All Devices

Staying connected is more important than ever. That’s why it was one of our primary goals to deliver the same great experience to mobile as well as desktop devices.

Whether you want to create an RSS feed to keep you up to date, save items to a My NCBI collection, or have your perfectly-crafted search automatically deliver the latest results, the responsive design means you can have it all from your phone and your laptop. In fact, responses from our mobile users were so overwhelmingly positive, we decommissioned the old, separate mobile site this past March.

Same as it Ever Was

Once the new PubMed becomes the default site, your existing links will be automatically redirected — meaning you won’t need to manually update your links to PubMed citations or search results. Your My NCBI saved searches and collections will continue to work in the new PubMed.

Want to Start Learning to Use the New PubMed?

We recognize that even positive changes can be challenging to adapt to, so we added several resources to help you, and the people you support, navigate the new site. From training to technical support, we’ve got you covered.

Please take a minute to read the New PubMed Transition FAQs. This page is likely to answer your general questions about the transition.

Our Trainer’s Toolkit provides you with instructional materials that you can customize and share. Whether you want to learn about the new PubMed for your own use or to train others, this is a great place to start. The series of nine quick tours, each only 1 to 4 minutes long, can be viewed online or embedded in course management software. You’ll also find slide decks, handouts, and webinar recordings all designed for sharing and reuse.

How PubMed® Works is a series of four 90-minute online classes offered by NLM and the National Network of Libraries of Medicine. Recordings will be available for viewing after each session ends for those who can’t attend or would like to view the material again.

The comprehensive PubMed User Guide is available from the homepage and under the “Help” link on every page in PubMed. It starts with a list of frequently asked questions, allowing you to jump to short, easy-to-follow instructions for finding and using your favorite features. As with our other resources, you can copy the text into your own training materials, trifolds, and user guides.

We’re here to help

Click on the green Feedback button on any screen in the new PubMed to write to the help desk. When the Feedback button is retired, the NLM Support Center link will remain on every page in PubMed. That is the best way to let us know what is — and isn’t — working for you.

We’re committed to keeping you informed! Subscribe to the NLM Technical Bulletin and PubMed New and Noteworthy for the latest news and new releases.

We’re just getting started!

We’re always looking for ways to improve PubMed. Just as we’ve done for the past 24 years, we’ll continue to add features and data to stay current as technology, publishing standards, and our users’ needs evolve.

Please think about other ways that NLM can help you, and share your ideas with us.