About the Letters Survey

With assistance from the Association of American Medical Colleges (AAMC) and Association of American Colleges of Osteopathic Medicine (AACOM), the NAAHP Communications Committee contacted Admissions deans and directors at all U.S. medical schools, allopathic and osteopathic.  We asked them a series of 16 questions (13 about various aspects of committee letters and 3 about individual letters of evaluation)—what they like to see in letters, what frustrates them, and what their preferences are in terms of format and content.  The survey was anonymous, although they could identify themselves if they wished.  We had a good response rate for surveys of this type, over 50%.   

In this section of the Toolbox you will see the 16 questions followed by response data.  In addition to the admissions officers’ answers to our specific questions, they were given the opportunity to make comments at several points during the survey, and comment they did!  We have included a selected array of these remarks demonstrating the widest possible range of opinions. 

Please note:  While elsewhere in the Toolbox we have used the term “letters of evaluation” (LOEs), consistent with the AAMC’s terminology, in our survey we used the term “letters of recommendation” (LORs).  While we understand that “evaluation” and “recommendation” truly mean different things, we use them interchangeably here in an effort to describe the individual letters written by professors, supervisors, research PIs, etc. (non-advisors) in support of our students.  Indeed, the terms are used loosely and confusingly among both advisors and health professions schools.  So, for our purposes, LOR = LOE.  And, CL = Committee Letter.

We are a big community—U.S. medical schools and pre-health advisors—and have diverse things to say about the letter-writing and letter-reading processes.  Enjoy!


National Med School Survey - PDF Download


The National Medical School Letters Survey Results

    Questions Regarding the Importance of Letters

    Authors

    Composition

    Timing

    Letter Topics

    Questions Regarding the Applicant's Ranking

    Weaknesses in Committee Letters

    Additional Thoughts on Letters


Questions Regarding the Importance of Letters

How important is it to consider a committee letter among the other components of an applicant's file?

Critical 22.92%
Very Important 43.75%
Important 21.88%
Not Important At All 1.04%
I'm Indifferent 10.42%
Total 100%

How do individual letters of recommendation help evaluate a candidate?

  • Offer different perspectives on applicant.
  • The details about why the candidate would be successful are often documented more in an individual letter.
  • Provided authors are selected carefully, they provide a more comprehensive view of the candidate across several disciplines.
  • Specific classroom performance examples provide insight on candidates and support depth of student/faculty relationship, knowledge of personal crisis, family situation, obstacles, etc., that have impacted academic performance that are not always explained by candidate, can provide insight on smaller, lesser known colleges or institutions.
  • Aside from questions of professional "fit,” one of the biggest questions we seek to answer during our admissions process is "can we teach them?". Letters addressing the student/teacher relationship, coupled with personal traits, are therefore the most helpful in our process.
  • We accept both but prefer the committee letters; the individual letter helps us see the applicant in a classroom setting.
  • Individual letters often include specific examples/behaviors of the applicant that help us understand competencies.
  • They provide more individualized context and help to build the candidate's story.
  • We look at who the applicant has asked to write their letters. We place emphasis on those letters from individuals who have had a long-term relationship with the applicant (coach, PI, employer, etc.) over a professor from one class.
  • When accompanied by a committee letter that's concise and inclusive, not very much.
  • Sometimes, we find a flag that was not mentioned in the committee letter, or something very positive that was not included in the CL.
  • Good individual letters note how well a student interacts with classmates/teammates/colleagues and pick up on interpersonal skills. It is nice to see people who know candidates well in different contexts.
  • Buttress the committee letter but importance related in large part to knowledge of candidate and relationship (mentor-mentee for example).
  • Are they consistent with one another, do they confirm what the applicant has indicated in application, do they give any warning signs?
  • Mostly on the negative side. Those letters that are less than complimentary are helpful. Positive letters are not a reliable discriminator.
  • We use individual letters of recommendation for different purposes. Academic letters speak to ability to succeed in medical school, clinical letter speak to aptitude for patient care and professionalism. Other letters speak to character and intangible qualities.
  • Sometimes you have a specific concern or question that is addressed in individual letter and not committee, also when committee letter does not provide a clear story or clear information on what they are evaluating then the individual letters are more useful as at least you know it's one person being asked to write about one person.
  • Letters play a small role since 95% of them are all positive. The actual 5% that give honest strengths and weaknesses play a huge role.
  • In the same way that committee letters help us. Essentially we are looking for evaluative comments and overall recommendation in terms of suitability for medical school and the profession.
  • When a student has a blemish, they help explain the situation. When a student has worked extra hard to put themselves through school, and therefore, may be a little short on some service or clinical hours, it is helpful to know. Basically, LORs are extremely helpful in sorting out the diamonds in the rough, and validating the student's application.
  • We look for the inter- and intra-personal skills that the letter writers can speak to. It's important for us to get a sense of how the student interacts in a professional and academic environment on a daily basis. We also look for what's not there; for example, if a student lists that they did a significant amount of research but doesn't get a letter of rec from their research faculty, it's often considered a red flag.
  • See if candidate aligns with mission of our school, verifies ability and academic performance, verifies commitment to medicine.
  • The PI letter is especially helpful in assessing research (more than the committee letter).
  • Helpful but not as important as the Committee letter. You would expect an applicant to ask an individual to write a letter on their behalf who would write a good letter for them.
  • Not much. They are always glowing.

When a committee letter is provided, what aspects of an individual letter of recommendation are not helpful?

  • Long and in-depth description of the letter writer's own accomplishments.
  • Letters from a family member or high school teachers.
  • Class rank in a letter is entirely unhelpful in our process.
  • Repeat information in application materials and/or committee letter.
  • Not helpful if they are too long.
  • We find all aspects of letters of evaluation important. Only when we see a pattern that there is no variety in the evaluations and scores given do letters then become less useful from particular schools.
  • Reviewing an applicant's performance in individual courses. AMCAS lists this information.
  • If there are a lot of superlatives not backed up with examples of what the student has done to deserve them, that is not helpful. In general, though, committee letters are typically more carefully and thoughtfully written giving a more holistic picture of an applicant than is gleaned from individual letters. Occasionally, the committee letters include information the applicant has not shared (additional clinical work/service, background information) and this is so helpful.
  • Excessive detail already available in the CL.
  • All are helpful.
  • Review of academic record or other activities touched upon in the committee letter. Repetition is generally not helpful.
  • All aspects of a letter of recommendation included with a committee letter are helpful.
  • Anything that is used redundantly in all letters.
  • Both letters are important to us especially if they support each other's position about the applicant. However individual letters that do not support the committee letter are very informative also. They two letters typically have a different perspective on the applicant.
  • Not getting the actual written letters as part of the packet. When the committee letter is a summary of others' observations, but only snippets of the original letters are included, it makes me wonder what was not included in the overall committee letter that was written by the original letter writer.
  • Too much detail. The committee letter should summarize the applicant's overall suitability for medicine but should not reiterate what is stated in letters of recommendation that may be attached.

Authors

Preferred Authorship for Committee Letters

Single Advisor 20.0%
Individuals on Staff and/or Faculty Who Have Met with Applicant 41.11%
Group Effort Co-Written by Full Committee 38.89%
Total 100%

Credibility Relating to Author Background and Credentials

Background/
Credentials
Critical Very important Important I'm indifferent Not important at all
Science Faculty Member 38.37% 33.72% 23.26% 4.65% 0.00%
Humanities / Social Science Faculty member 5.81% 45.35% 26.74% 17.44% 4.65%
Former Employer 4.65% 31.40% 39.53% 19.77% 4.65%
Physician 22.09% 34.88% 23.26% 16.28% 3.49%
Volunteer Coordinator 4.76% 30.95% 35.71% 26.19% 2.38%
Extracurricular Supervisor (e.g. coach, student affairs dean, leader of a program) 5.81% 33.72% 34.88% 22.09% 3.49%
Other 38.10% 23.81% 14.29% 4.76% 19.05%

Other Important Author Types

  • We require D.O. & committee/advisor/science faculty letter.
  • Researcher; Research PI.
  • It's more the experience than the author's background.

Composition

Composition of a Committee Letter Packet

Committee letter with letters of recommendation (LOR) attached, but no quoting/excerpts of LORs in the committee letter. 27.37%
Committee letter with LORs attached and some quoting/excerpts from LORs when relevant. 58.95%
Committee letter with LORs summarized in composite fashion, but not LORs attached. 7.37%
No committee letter; individual LORs uploaded into AMCAS separately 6.32%
Total 100%

Timing

When the Committee Letter is Turned to for Guidance

Initial Screening 1.05%
Decision to Interview 45.26%
Post-Interview Decision 23.16%
Wait List 0.00%
Other 10.53%
Total 100%

Letter Topics

Important Topics to Include in the Letter

Question Critical Very
Important
Important I'm
Indifferent
Not Important
At All
Clinical Experience/Patient Exposure 32.22% 40.00% 20.00% 4.44% 3.33%
Research 3.30% 28.57% 45.05% 19.78% 3.30%
Community Service 32.97% 39.56% 20.88% 3.30% 3.30%
Leadership Experience 20.65% 44.57% 28.26% 4.35% 2.17%
Professionalism 68.48% 27.17% 4.35% 0.00% 0.00%
Depth of Commitment to Medicine 47.83% 34.78% 13.04% 3.26% 1.09%
Family Background 5.43% 23.91% 39.13% 21.74% 9.78%
Context for Poor Academic Performance 44.57% 35.87% 15.22% 4.35% 0.00%
Context for Gaps in Academic Progress 41.30% 36.96% 16.30% 4.35% 1.09%
Context for Gaps in Employment Post-College 17.58% 31.87% 17.58% 23.08% 9.89%
Health or Other Personal Issues 18.89% 32.22% 27.78% 14.44% 6.67%
Context for Institutional Action 63.04% 28.26% 7.61% 0.00% 1.09%
Context for Felony +/or Misdemeanor Convictions 65.22% 21.74% 8.70% 3.26% 1.09%

Interpersonal Competencies to Include

Question Critical Very
Important
Important I'm
Indifferent
Not Important
At All
Service Orientation 41.76% 45.05% 8.79% 3.30% 1.10%
Social Skills 57.61% 35.87% 5.43% 1.09% 0.00%
Cultural Competence 50.55% 41.76% 5.49% 1.10% 1.10%
Teamwork 54.95% 37.36% 7.69% 0.00% 0.00%
Oral Communication 51.09% 39.13% 7.61% 2.17% 0.00%

Intrapersonal Competencies to Include

Question Critical Very
Important
Important I'm
Indifferent
Not Important
At All
Ethical Responsibility to Self and Others 74.73% 21.98% 3.30% 0.00% 0.00%
Reliability and Responsibility 73.63% 24.18% 1.10% 1.10% 0.00%
Resilience and Adaptability 61.54% 31.87% 5.49% 1.10% 0.00%
Capacity for Improvement 50.55% 40.66% 6.59% 1.10% 1.10%

Additional Personal Qualities Preferred

  • Emotional intelligence, maturity.
  • Motivation/determination for the profession. General reputation.
  • Ability to accept feedback.
  • Humility.
  • Collaboration, capacity for growth, sensitivity to needs of others.
  • Opportunities for growth, areas that need to be strengthened.
  • Empathy, emotional intelligence, ability to respect opinions and ideas different from their own, ability to put self aside for the good of the team, ability to collaborate.
  • Altruism, help-seeking behavior when needed.
  • Experience with rural and/or underserved/disadvantaged communities.
  • Integrity, motivation, compassion.

Questions Regarding the Applicant's Ranking

Is it important to highlight the applicant's recommendation ranking?

Critical 26.37%
Very Important 47.25%
Important 17.58%
I'm Indifferent 6.59%
Not Important At All 2.20%
Total 100%

Why is the ranking level important?

  • It is important to see where the candidate falls in relation to his or her peers.
  • Actual rankings, with general percentages of how many students end up in each ranking, help us determine the relative rigor of the Committee evaluation process. Strong letters are always considered, but a collective opinion that a certain candidate is ranked in the top 10% of candidates reviewed that year adds more weight and context to the evaluation.
  • We like to know where this applicant falls among all your recommendations for the year. We also like to know which factors you are taking into consideration in making your ranking, and what aspects of the applicant's preparation you find lacking.
  • It is nice to see how the applicant stacks up among their school cohort, but I know how difficult it can be to give a global recommendation.
  • In general, the "rank" or rating of a student is really the only valuable part of the committee letter we focus on.
  • Hopefully the overall level of recommendation will reflect the summaries of the individual LORs.
  • It is NOT helpful to have, say, a 5-level scale if 35% is the top level & 50% is the second tier (leaving 15% in the average or less range).
  • As advisors, you know the coursework and activities that your applicants have pursued better than we can for most undergraduate institutions. You can better assess whether someone has challenged themselves academically, whether they have had meaningful participation in research and/or service activities, etc. I don't care so much about an "overall rank” as much as a ranking within each area (academics, research, service, etc.). That way, schools can decide which pieces are important to them.
  • It depends on how the ranking was achieved. We do not care if someone doesn't get the highest mark because of their numbers. That is not what we need the letter for in the first place.
  • Regardless of the impression(s) of a Pre-Health Committee, it is the Admissions Committee of a particular medical school that makes the ultimate decision.
  • The content of the committee letter is more important than the overall rank.
  • If an advisor has worked with an individual on a longitudinal basis, which is what I would hope would happen, I think that a level of recommendation is very helpful.
  • Using superlatives without ranking makes it very difficult for us.
  • Committee letters are variable in what they cover and how they are written, therefore an overall rank is sometimes the only information that is meaningful.
  • It's one of the best things about a committee letter vs individual letters.
  • Our committee finds it particularly important to put into context how an applicant compares to others applying to medical school from that institution. It is also interpreted as to the level of enthusiasm by the health professions committee for the applicant in terms of suitability for medical school and the profession of medicine.
  • Every university has their own set of standards for how they rank, and we often see large variances between what one college may say for each category. We would prefer the written LORs with their direct observations.
  • I rather dislike comparisons among the current cohort. Better an overall assessment.
  • Only important if ranking scale is included in the letter.
  • Ranking is very fickle and only applies within a narrow pool--it is only narrowly valid and not generalizable.
  • This helps to distinguish between the very best applicants versus average applicants--our committee is wary of a premed committee who recommends everyone "very highly.”

Are committee letters that offer multiple levels of recommendation preferred?

Yes 53.85%
No 46.15%
Total 100%

Is it important to share how many applicants from my institution fall in the overall category of recommendation within the given timeframe?

Critical 23.08%
Very Important 36.26%
Important 19.78%
I'm Indifferent 17.58%
Not Important At All 3.30%
Total 100%

Why is sharing how many applicants fall within the overall category of recommendation at my institution significant?

  • We will not base our decisions off this. It might be nice information to have, but I wouldn't create more work for you all unless it's something you all were normally tracking and wanted to provide that information.
  • Important for context, and to know that the applicant may or may not be an outlier.
  • Tier levels should be roughly a bell-shaped curve (only a few at the highest & lowest level, most in the average (+ or -) range.
  • Some schools may view the majority of their candidates in the very top category. Just helpful to know what the denominator is for each grouping.
  • Some schools are just focused on getting students into medical school, not with what kind of classmate or physician they might be.
  • Context is very important, hence needing to know number within each category. Attempt should be made to normalize the curve, and if that's not possible, to at least balance the number of applicants within each group so that no one group has the vast bulk of applicants.
  • Without this information, we have no way to interpret the level of support.
  • It gives the process at the institution credibility because it is truly engaged in an evaluative process.
  • Each college is different in the numbers of students who are moving onto the health professions. Some schools may have a large pre-health student group, others may not. The numbers of how many fall into each category is meaningless and not really helpful on our end.
  • I think this tends to hurt "average" applicants.

Weaknesses in Committee Letters

What are the top most-common weaknesses in letters?

Typos 10.10%
Lack of Concision 20.67%
Lack of Consistency Among Letters from the Same School 17.79%
Overly Formal Style 3.85%
Overly Casual Style 7.69%
References to High School 17.79%
Other 22.12%
Total 100%

Other Common Weaknesses

  • We request that if there are reasons for concern with the candidate then they are clearly stated. It is hard to read between the lines.
  • Providing a generic "guarded" review due to concern of legal ramifications.
  • Copying and pasting letters [from other documents] and forgetting to change name of applicant.
  • Preferably, letters will focus on personal qualities/characteristics of the student and not overly on academics, which we can see in their transcripts.
  • Vague terminology, lack of decisiveness, expecting the reader to "read between the lines.”
  • Letters not matching overall rating.
  • Every applicant is well above average.
  • Not telling me anything above what I can read in the application itself, redundancy that does not help distinguish the applicant.
  • Letters from authors that obviously don’t know the applicant very well.
  • Failure to at least address "red flags" that the applicant addresses in the AMCAS application.
  • Grammar.
  • Unwillingness to take a stand on major strengths and weaknesses.
  • Having no idea what a particular adjective means in the summary.
  • Using a template that has been used for another student and referring to a female as 'he' or a male as 'she' in the context of the letter. This makes the Committee wonder if the letter truly is representative of the actual applicant we are reviewing.
  • Repeating information that can be gathered from other parts of the application, not being critical and honest about students’ strengths and weaknesses.
  • The long introduction that has nothing to do with the applicant.
  • What seems to be "fluff" or over-selling the applicant.
  • Lack of specific attributes or situations.
  • Missing critical information related to competencies.
  • Letters without behavioral examples.
  • Talking too much about your school.
  • The Lake Wobegon letter syndrome, where all the women are strong, all the men are good-looking, and all the children are above average.
  • Neutrality.

Additional Thoughts on Letters

Authorship

  • Having a letter from a committee on an applicant is helpful to our admissions committee, if the letter is being written with key aspects in mind: (1) the applicant has been "interviewed" by the committee writing the letter (2) the letter is very honest in the evaluation (i.e., not all letters are good and glowing; can point out flaws) (3) letter does not have any typos or refer to someone other than the applicant
  • Letters from Premedical Committees are much very much appreciated. Members of the Admissions Committee know the time it takes to prepare committee evaluations and especially appreciate the honesty with which these letters are written. Truly outstanding students are presented, as such, but those who need more preparation are also honestly rated with the appropriate ranking. Each school knows its students best, so we very much appreciate the honest appraisals and use them in the overall ranking of each interviewed applicant.
  • We . . . require a D.O. physician letter. These letters are the ones that are most difficult because they don't always share the detailed information we are looking for, but we do not fault the student when this happens. That is why we look a lot at the committee letters.

Reviewing Letters

  • Because I read so many, for initial screening I read the bottom line which is usually the final paragraph so this is important.
  • I responded "indifferent" to the initial committee letter question because we don't hold them in any higher regard as compared to individual letters. In general, we find that committee letters don't often add to the applicant file that much but are useful when there are discrepancies between the committee letter and the individual letters in reference to the student's professionalism, communication, etc.
  • It would be very helpful if one day every pre-health office uses one format for better comparison and consistency.
  • Many of us deal with large numbers of applicants from large public undergraduate campuses where there are no committee letters. A challenge for us is how to balance the types of letters these applicants have submitted, and the overall less intense advising received by those applicants, with the highly enthusiastic committee letter from a private liberal arts college.
  • The letters are the last thing we look at in the screening process and really do determine whether or not an applicant is interviewed. A committee letter is always preferred but we recognize that many schools do not provide that service.
  • Unless there is a scale provided as to the ranking of the students in areas of service, research and clinical exposure relative to the other students at the undergraduate institution, I am sorry to say but we find the committee letter not that helpful and go directly to the individual letters.
  • We don't weigh committee letters over individual letters; we are more interested in the information in the letter.
  • We factor LORs and CLs when deciding on interviewing the candidate and then again at post interview. If we are on the fence with an applicant the stronger the letter the more likely applicant will be accepted.

The Value of Letters

  • A committee letter is a chance to help with interpretation of the application. Of course there may be bias in the letter (and in a way one would hope that in the vast majority of cases there is positive bias), but it's up to the reader to take that into account and use the interpretation to help sway them one way or another. It's a fine line between providing honest interpretation and putting the applicant in the best possible light and that is the way it has to be. I truly appreciate when concerns are forwarded, but realize that is often not applicable or unknown to the committee and therefore appreciate that the thoughts presented are based on an entire pool, experience of the letter writers, and the desire to support their own.
  • A large number of our applicants do not submit a committee letter. They send us up to 6 letters (a minimum of 3) from sources they feel are well situated to comment on their preparation and vouch for their skills, abilities, and experiences. Our committee does not penalize an applicant for not having a committee letter . . . What our committee does not appreciate are advising systems that require an applicant to be "endorsed" by the school or they cannot apply. These models raise concerns that advisers may be unfairly limiting candidates or pre-screening our pools for us (largely based on GPA and MCAT cut-offs). When a school has a premed committee and a student chooses to apply outside that process, we wonder why, but we still give the applicant fair consideration.
  • A thoughtful, concise committee letter is far superior to alternatives.
  • I know that committee letters are a lot of work but we do appreciate them!
  • I know that committee letters are very burdensome for colleges and universities, but I do feel that they really add context to the application. Individual letters can be extremely variable, particularly if coming from a referee who does not frequently write LORs.
  • In the end, we have to distinguish among outstanding young people we haven't met. Therefore, letters that give us a sense of the person and the level of enthusiasm relative to other applicants are the most helpful.
  • Our admissions committee really relies on the committee letters. Thank you for all the work you do to help us evaluate your students!
  • We believe that a well-constructed committee letter is the single most important document in the file. We look to this letter for background, life experience, context, and perspective in our review of other materials. We start by reading the committee letter and then move on to other parts of the file.
  • We can't thank advisors enough for the hard work and thoughtfulness that goes into the writhing of these letters. Without your help so many times we would just be flying blind. I don't know how you do everything you do but it is greatly appreciated.
Copyright 2013 by NAAHP Terms Of Use Privacy Statement