MCATs, Affirmative Action, and DEI, Oh My

[Don2 (Don1 Revised)]

This is a space to discuss these things without derailing some other threads about current events. In particular, US college admissions into pre-med schools has come up and different averages across demographic groups in MCATs and how that could play a role.

Thoughts?

 

[Politesse]

As a professor, I'll throw in my opinion that the MCATs, like all multiple choice standardized exams, are at best an incomplete representation of student ability. I am not an expert in its study area, but did review a bank of social and psych section example questions recently, and I don't think you would need to have taken a college level couse in the behavioral sciences to pass it. It also heavily overemphasizes psych theory and theories, which is going to be very important for sone medical fields especially anyone that is going to be prescribing psychoactive meds, but is no sort of preparation on its own for a cross-cultural clinical exchange, or even a post-connectome cog science discussion. So what is it testing for? Ability to cram your way through a test prep booklet? Serious medical programs are correct to backpedal on the over-emphasis on this particular metric. There are no 60 multiple choice questions that can, on their own, tell you if a student is ready for medical school.

 

[Derec]

As a professor, I'll throw in my opinion that the MCATs, like all multiple choice standardized exams, are at best an incomplete representation of student ability.

It would be a tall order to expect any single metric to be a complete "representation of student ability". But I think it is the single most important one because it is an objective measure of academic ability and also because everybody, regardless of race, socioeconomic status or family connections takes the same exam and is thus measured by the same yardstick. No other metric used in med school admissions has that going for it.

I am not an expert in its study area, but did review a bank of social and psych section example questions recently, and I don't think you would need to have taken a college level couse in the behavioral sciences to pass it. It also heavily overemphasizes psych theory and theories, which is going to be very important for sone medical fields especially anyone that is going to be prescribing psychoactive meds, but is no sort of preparation on its own for a cross-cultural clinical exchange, or even a post-connectome cog science discussion.

Yes, Psych/Soc is my least favorite section for these reasons. It is also the only section where the psychology/sociology knowledge base that is expected is not part of prerequisite classes such as biology, chemistry and physics. However, P/S also includes general questions about studies (including very basic statistics), which is relevant to medicine given that physicians are expected to stay informed on developments in their field.

So what is it testing for? Ability to cram your way through a test prep booklet?

You have to have a solid base of knowledge of relevant topics. But it is not a test of just regurgitating facts. It is not just a knowledge test, but also a reasoning test, especially with passage based questions. You are expected to take what you know and integrate it with the information from the passage to come up with the answer.
And then there is the infamous CARS - "Critical Analysis and Reasoning Skills" - section. It requires no prior knowledge. It just asks you some questions about passages about random and often obscure topics. Since there are four MCAT sections (Chem/Phys, Bio/Biochem, Psych/Soc and CARS) this one is 25% of your entire score.

Serious medical programs are correct to backpedal on the over-emphasis on this particular metric.

But they are not backpeddaling at all. Top medical schools expect high MCAT scores from their applicants, higher than MD average, and MD schools expect higher scores than DO schools. It's just that they all expect lower scores from applicants from favored races and ethnicities.

There are no 60 multiple choice questions that can, on their own, tell you if a student is ready for medical school.

Of course not. But it is an important metric. As I said, single most important one. Also, it's ~60 questions per section, not total.

 

[Bronzeage]

The redeeming feature of a multiple choice test is they are easily and quickly graded. Any qualifying process which gives each applicant a fair evaluation quickly breaks down once the number of applicants surpasses a fairly small number. As with almost everything in life, time and resources are the limiting factors. If you have to test a lot of people, there isn't another alternative.

Back in the 1970's, there was a push to license automotive mechanics. This terrified the auto manufacturers because it would drastically shift the balance of power in dealership service departments. If the law required a certified mechanic, the dealer would have to pay market rate for a fairly rare person. That couldn't be allowed to happen, which led to a push for "certification". A license is a document issued by the government, but anybody can issue a certificate. In reality, it was an elaborate public relations scheme(scam). Certificates were awarded to mechanics who signed up for and took and ACT style test. In the beginning, there were 8 different categories. The tests were fairly well written, with a combination of questions which could only be answered through some kind of training and others which required knowledge only learned through experience. You got an actual certificate and patches to sew on your uniform. Soon after the program began, "mechanic" became a dirty word and we were now called technicians. More public relations hype.

Although automotive engineering and design has evolved a common denominator, there are still enough differences in brands to require the tests to be very generic. A test that is too hard or too easy accomplishes nothing.

Whether this improved the quality of automotive service is another question. Maybe, maybe not.

As it is in most states today, a certification is purely voluntary, although it does make a service manager's job a little easier when it comes to hiring. The reality is, it really makes no difference. If a person wants to charge me for a haircut, he or she has to have a license issued by a state board, and usually at least a diploma from some kind of accredited school.

If a person wants to work on your antilock brakes, all he or she needs is your car keys.