In 2012, the Nobel Prize in Economics was awarded to Alvin Roth, PhD, for his work on stable allocations and the practice of marked design. A centerpiece of this work was the redesign of the Match for medical students applying for residency positions. While the Match has worked well for the past decade, increasing competition for residency positions and a spiral of increasing application numbers have cluttered the application process at the point of interview offers, before applicants can enter the Nobel-winning Match algorithm.
In combination with specialty leaders in otolaryngology – head and neck surgery (OHNS), the UCSF OHNS is at the forefront of the implementation and evaluation of a preference signaling program that has now been adopted across the nation. This program has had a major impact in otolaryngology, and data from our specialty has convinced nearly all residency specialties to incorporate signaling within the residency application process. In the current Match cycle, almost all of the 40,000 medical students who participate in the Match will participate in preference signaling.
Within otolaryngology, over 500 medical students apply for 350 otolaryngology residency positions. In the past decade, the process has become more competitive, with prospective residents submitting more applications. As reported in "Preference Signaling for the Otolaryngology Interview Market," an article in the October 6, 2020, issue of The Laryngoscope, otolaryngology applicants submitted an average of 84 residency applications, an 80% rise over 10 years. This costs upwards of $1,700 per student in application fees alone.
"The pressures and consequences of not matching in a program are so high that the fees end up being inconsequential relative to not matching," said UCSF OHNS Residency Program Director Steven D. Pletcher, MD, who was one of the authors of The Laryngoscope article.
"In this context, applicants submit an ever-increasing number of applications, and programs are overwhelmed as they try to select students for interviews. It's so hard for students to stand out among the sea of applications we receive."
As virtual interviewing became an inevitability during the 2020-2021 application cycle and away rotations were eliminated because of COVID-19, an already extremely competitive program became even more competitive.
"Everyone was struggling with how to review applications in a reasonable way," Dr. Pletcher said.
As part of an Otolaryngology Program Director Leadership Council, he promoted a signaling process adapted from the American Economic Association. That process calls for graduate students to notify their top two prospective employers of their interest. Although there had been discussion and modeling of signaling in graduate medical education, no specialties had committed to the process.
"The pandemic tipped the balance to implement this," Dr. Pletcher said. Working with his colleagues, he engaged with medical student deans, students, the Association of American Medical Colleges (AAMC) and other stakeholders to develop and implement a signaling system that enabled prospective students to alert their top five choice residency programs of their particular interest at the time of application submission.
Preference signaling allows applicants a credible and consistent method to inform programs of student interest early in the process and before interviews are offered. Each program receives a list of students who sent a signal.
The results of this initial signaling experience demonstrated a powerful impact of signals on the ability to obtain interview offers for applicants. The results were reported by Dr. Pletcher and colleagues in "The Otolaryngology Residency Program Preference Signaling Experience" in the May 2022 issue of Academic Medicine. Importantly, the study showed that both the most competitive applicants and those who struggled most to receive interview offers benefitted significantly from signaling.
These initial results have led to a rapid expansion of signaling, and preference signaling has now been adopted by 18 specialties. In the 2024 National Resident Matching cycle, it is estimated that more than 90% of applicants will participate in preference signaling programs.
With the rapid expansion of signaling, Dr. Pletcher and his colleagues felt it was extremely important to ensure that this new initiative did not hinder groups of applicants who are historically underrepresented within otolaryngology. Teaming up with researchers from the AAMC, Dr. Pletcher and his colleagues found that signals benefit applicants from across gender and racial/ethnic groups that are underrepresented in medicine. This important finding was published in "Interview Invitations for Otolaryngology Residency Positions Across Demographic Groups Following Implementation of Preference Signaling" as an original investigation with Dr. Pletcher as the lead author in the March 7, 2023, issue of JAMA Network Open.
These results are particularly exciting as preference signaling supersedes a system of informal signals with a high potential for inequity.
"It replaces a system that was set up for bias," Dr. Pletcher said. Prior to a formal signaling process, students would ask mentors and influential people from their medical school to convey interest in a program. Applicants from medical schools without an otolaryngology residency program, who are more likely to be from underrepresented backgrounds, were left without the advocacy and mentorship required for effective informal signals.
The otolaryngology experience also shows that preference signaling results in a better distribution of interview offers across the spectrum of applicant competitiveness. Working with Dr. Pletcher and his colleagues, UCSF Otolaryngology Resident Taylor Standiford, MD, modeled the distribution of interview offers with and without signaling (see figure below) and found a 30% increase in the number of interview invitations received with signaling among applicants who struggled most to receive interview offers.
This year, the orthopedics residency, which is similar in size and number of applicants to otolaryngology, is participating in signaling for the first time. However, instead of five signals, orthopedics is using 30 signals, and Dr. Pletcher says he will be watching closely to see how it turns out.
"It changes the dynamics to the point where you likely won't get an interview unless you signal," Dr. Pletcher said.
He suspects that all orthopedics programs will fill with residents, that it will be very rare for people to match at a program they don't signal, and that the number of applications people submit in the following year will decrease. "It breaks the race to the bottom to submit more and more applications," Dr. Pletcher said. It also removes unnecessary competition for interview slots from excessive applications.
The Rise of Virtual Interviews
Another major change in the process of applying for otolaryngology residencies is a shift to virtual interviews. In 2021, the shift felt inevitable from a safety perspective, said Associate Residency Director VyVy Young, MD. "By the time we were gearing up for residency interviews, we were months into the pandemic and already quite facile with virtual platforms like Zoom," Dr. Young said.
While she doesn't think the shift to a virtual format has impacted the quality of residents who end up coming to UCSF, she prefers the ability to connect with prospective students in person. The UC system has mandated that admissions interviews for all of its programs remain virtual in 2023, and the AAMC has recommended that programs conduct virtual interviews for all applicants including local applicants.
"I think people have a love/hate relationship with virtual platforms," Dr. Young said. "They love the convenience of sitting in front of a computer in sweatpants and fuzzy slippers, but also recognize the things they miss out on."
In virtual interviews, honesty is important on both ends.
Dr. Young and her team spent a lot of time examining the department website to make sure the information was up-to-date, accessible, easy to navigate and comprehensive in addressing the questions they thought students might have. They also developed a video to showcase current residents' insights into life at UCSF as well as their experiences in the program and living and working in San Francisco.
"We've had to adapt, grow, and adjust – and we've had to change the way we approach things," Dr. Young said. In spite of the changes, the quality and caliber of residents remains the same. "Fundamentally, we have a great program with tremendous opportunities, and that attracts amazing people."
This article was originally published in the Spring 2023 edition of "HeadsUp!" You can read more articles from the publication here.
