Health Care Reviews Tank Following COVID Pandemic

PHILADELPHIA— After the COVID-19 pandemic struck, online reviews of health care facilities dropped significantly, and they have not yet fully recovered, according to a new analysis led by researchers at the Perelman School of Medicine at the University of Pennsylvania. More than half of reviews on the online platform, Yelp, now are negative, flipping the pre-COVID picture. The findings are published today in JAMA Network Open. 
“Online reviews can tell us information about the patient experience that traditional reporting metrics, like hospital-administered patient experience surveys, might miss,” said the study’s lead author, Neil Sehgal, ME, an associate fellow in the Penn Leonard Davis Institute of Health Economics. “These reviews can help hospitals understand what matters most to patients and their support networks in near real time.” 
By analyzing all reviews of health care facilities in the United States on the online platform Yelp dated from 2014 through 2023, Sehgal, co-author Anish Agarwal, MD, an assistant professor of Emergency Medicine, and their team saw that the percentage of positive—four- and five-star—reviews dropped from 54.3 percent before March 2020 (marked as the beginning of the COVID pandemic in the United States) to 47.9 percent after. 
In fact, from the latter half of 2021 on, the researchers found that positive reviews were never more than 50 percent. 
“We analyzed Yelp reviews of health facilities nationwide—which includes hospitals, urgent care centers, doctors’ offices, and more,” said Sehgal. “Our analysis shows how public perception of health care changed after COVID-19. And with this information, hopefully, health care professionals can work to improve upon what’s been driving these opinions.”  
Feedback ups and downs  
In analyzing the reviews, the researchers used a language processing technique to tease out the most common topics from the reviews, organized them into themes, and measured how they changed over time.  
The themes that saw the greatest change in mentions between the pre- and post-COVID periods were “insurance and billing issues” and “customer service and staff behavior.”  
One theme mentioned less in negative reviews after the arrival of COVID—and a renewed, supercharged focus on hygiene: Facility cleanliness. 
Trends emerge along ethnic and geographic lines 
While positive reviews of health care facilities on Yelp, as a whole, declined over time, health care facilities in rural areas already had lower ratings at the start of the time period examined, and the differences became more significant after COVID struck. Post-COVID, rural facilities were 23 percent less likely to have positive reviews, compared to health facilities in urban areas, which were 7 percent less likely to have positive reviews. 
Looking at data on race, researchers found that health facilities in the sample of neighborhoods that they analyzed that had higher proportions of Black or white residents were more likely to have significant declines in positive reviews.  
But when it came to areas with higher-than-average Hispanic populations, facilities still saw a decline in positive ratings, though the dips were not as pronounced.  
“At the moment, we cannot understand what is driving these changes among different populations,” Agarwal said. “But this could help us think critically on what questions to ask next and how to better deliver care around the nation in the future.”  
Researchers also identified ethnic correlations around the most mentioned complaints. Reviews for facilities in areas with higher Black and/or Hispanic populations criticized “insurance and billing issues” more often, while the facilities in areas with higher white populations found fault with “wait times” more often.  
“Many of the themes have clear overlap but are distinct across geographic areas, which is interesting and begins to highlight a growing need for health systems and facilities to be more intentional and tailored to their local communities’ needs—and opinions,” said Agarwal.  
Moving forward, the researchers hope to look deeper into the content of the reviews to better understand how attitudes and opinions break down along socio-ethnographic lines, giving health care professionals better tools for discussing issues that are important to individuals and assuaging concerns. 
This study was partially funded by National Institutes of Health grants (NIH NHLBI R01HL1-141844, NIH/DHHS R01 MH127686, NIH-NIMHD:R01MD018340, and NIH K24 HL157621). 
Penn Medicine is one of the world’s leading academic medical centers, dedicated to the related missions of medical education, biomedical research, excellence in patient care, and community service. The organization consists of the University of Pennsylvania Health System and Penn’s Raymond and Ruth Perelman School of Medicine, founded in 1765 as the nation’s first medical school.
The Perelman School of Medicine is consistently among the nation’s top recipients of funding from the National Institutes of Health, with $550 million awarded in the 2022 fiscal year. Home to a proud history of “firsts” in medicine, Penn Medicine teams have pioneered discoveries and innovations that have shaped modern medicine, including recent breakthroughs such as CAR T cell therapy for cancer and the mRNA technology used in COVID-19 vaccines.
The University of Pennsylvania Health System’s patient care facilities stretch from the Susquehanna River in Pennsylvania to the New Jersey shore. These include the Hospital of the University of Pennsylvania, Penn Presbyterian Medical Center, Chester County Hospital, Lancaster General Health, Penn Medicine Princeton Health, and Pennsylvania Hospital—the nation’s first hospital, founded in 1751. Additional facilities and enterprises include Good Shepherd Penn Partners, Penn Medicine at Home, Lancaster Behavioral Health Hospital, and Princeton House Behavioral Health, among others.
Penn Medicine is an $11.1 billion enterprise powered by more than 49,000 talented faculty and staff.