Otomatch : Top 3 Ranklist Survey

2021 Top 3 Ranklist Survey

Below are the results of the 2021 Otomatch Top 3 Ranklist Survey. There were over 270 ranklists submitted this year. The details of the ranking methodology are spelled out below. This is not intended to be a system for ranking the quality of training at different programs. People have lots of factors that go into making their ranklist for the Match including geographic ties, cost of living, family, working conditions, research opportunity and of course training quality.

I'm sure I missed several other factors and haven't mentioned any of the biases in the study design. Let me know about them in the chit chat so next year's version can be even better. This year the comments are in a second tab sorted alphabetically by program. So without further delay, here are the results...


There were 273 valid ranklists submitted (see methods below). A total of 819 ranks were submitted (including 6 for other, none and n/a).

There were 113 unique programs submitted during the survey period, and one submitted the day after the survey ended.

A little more infor on researh programs. Twenty-eight ranklists had at least one program designated as "Research." Five ranklists had two programs labeled "Research" in their top 3. Of the programs designated "Research," 10 were on at least one ranklist.


The study design was a survey asking ENT residency applicants to share the top 3 programs from their Ranklist Order List as submitted to the NRMP. The survey was officially open from February 24 to March 12, 2021. Comments were also encouraged on why applicants ranked each program in their top 3.

To rank the programs, an Otomatch Top 3 Ranklist Score (OT3RS) was calculated for each program according to the formula below. The programs were then ranked from highest to lowest OT3RS. Ties were not broken this year, consistent with last years results. This is left as an exercise for the reader.

Formula: OT3RS = 3 * (Number of Times Ranked #1) + 2 * (Number of Times Ranked #2) + (Number of Times Ranked #3)

Data Validation and Normalization

Several edits, corrections and exclusions were made to the raw data from this years results. A total of 302 ranklists were submitted this year. Several of these were discarded. Below I explain these in excruciating detail. The reason for excessive detail is because some questions have been raised about the difference between the Otomatch Top 3 Ranklist Survey and the Top 3 tab on the spreadsheet.

Responses submitted prior to February 24 were discarded (N=20). Due to changes in the program list during the test period, comments were not able to be accurately associated with programs prior to the official start date.

Responses after March 12 (midnight in any US time zone) were discarded (N=4). One response was received after midnight in the central time zone and contained a program in the central time zone. This response was not included in the preliminary results but is included in the final results. There was one other exception to the deadline. Stony Brook Medicine is listed on the results as having * votes since its only votes came after the deadline.

One response was removed due to an email notifying otomatch@gmail.com that a specific rank list had changed.

Four responses ranked less than three programs and listed "Other" as their second and/or third rank. These were assumed to be from applicants with less than three ranked programs as they wrote in none or n/a in the comment.

There were 9 ranklist combinations (same programs, same order) that were submitted more than once. One of these combinations was submitted twice on the same day and once two days later with the same comments each time. Two of these three submissions were discarded. This triplicate combination was included in the preliminary results but discarded for the final results. The other combinations had a variety of comments and were not submitted on the same day so they were not removed.

A few apologies again this year for misspelling, misnaming and duplicating programs. Cedars-Sinai was misspelled. Icahn School of Medicine (Sinai/MEE) was listed twice. The totals from both listings were combined. I can confirm there are no ranklists with "Icahn School of Medicine at Mount Sinai'' and "Icahn School of Medicine at Mount Sinai (New York Eye and Ear Infirmary at Mount Sinai)." UCSD did not have a research track this year so any ranklists with UCSD - Research were attributed to UCSD. There were no ranklists with UCSD and UCSD - Research. PCOM was listed as "Philadelphia College of Medicine'' on the survey form. An astute commenter pointed out that this should be Philadelphia College of Osteopathic Medicine in the comments.

One obviously fraudulent ranklist was removed due to ranking the same program for all three positions.


Sam Reyes would like to thank all of the participants in this year's Otomatch Top 3 Ranklist Survey. I especially want to thank those who left valuable comments.

Home | Discussion | Interview Calendar | Headmirror | About Otomatch