Wednesday, September 21, 2011

Use Common Sense with Assessments

I worked for 19 years at the nation’s premier testing company, and one thing I learned was not to be dogmatic about assessments. Even the makers of the mighty SAT exam freely admit that their scores do not predict college achievement as well as high school grades do. The SAT scores still have value, however, because when combined with high school grades, they give a better prediction than the grades could alone.

Something similar can be said about assessments that are designed to help you with career choice: Alone, they probably are not the best way to identify a promising career. But combined with other sources of information, they can be very helpful.

Incautious assessment users sometimes forget that psychological instruments are not as precise as a yardstick. Every assessment has a certain margin of error, meaning that although its scores put you into category x, there is a chance that you really belong in the neighboring category y. In baseball games, it’s usually pretty obvious whether a ball has landed in fair or foul territory. But imagine what accuracy would be possible if the ball were ten times as big as a beach ball and even more squishy: Even the instant replay would not resolve a borderline hit.

In baseball, it’s also helpful that the foul lines are at the edges of the area that the batter is aiming for. Now imagine what the game would be like if a line ran right down the middle of the field, and the score of the game depended on which side of the line the enormous beach ball fell on. For good measure, imagine that both right-handed and left-handed batters tended to hit the ball towards the middle. Now you have an idea why I’m reluctant to use the Myers-Briggs assessment. It attempts to place you on one side or the other of a bipolar range (e.g., introvert or extravert), although the normal distribution that is so typical of psychological characteristics clusters most people near the middle. There’s a very high likelihood that your actual location on this continuum is close enough to the middle that the inevitably imprecise measurement of the instrument will assign you to the wrong side of the middle.

Although I prefer to use assessments based on the Holland types, I have to caution that they also are unable to achieve a pinpoint focus. For the Holland types, let me change the metaphor from baseball to hopscotch, where a lot depends on which square your marker lands in. On the playgrounds where I grew up, the squares tended to be at least 10 inches across, but imagine how the game would be if they were only 3 inches across and your marker were a beanbag the size of a dinner plate. Fortunately, the Holland rationale is conceptually more forgiving of ambiguity than is the Myers-Briggs rationale, because it is not bipolar and accepts the notion that bordering types (for example, Realistic and Investigative) share some characteristics. If I can’t tell for certain whether my beanbag fell in the Realistic of Investigative square, I probably would not err greatly by favoring occupations coded RI, IR, or even just R or I.

But the really big mistake would be to rely solely on the assessment, whatever its rationale. Here are some other indicators of your interests and preferences that you should consider:
  • In what school subjects did I get the most enjoyment and the best grades?
  • What activities am I reluctant to drop at dinnertime?
  • What are my favorite sections of the newspaper or of news websites?
  • At a party, what kind of people would I be able to sustain a conversation with?
  • If you could meet the world’s greatest ____________ and get that person to share his or her secrets of success, what field would that person’s achievement be in? (Romance doesn’t count.)
Still another indicator is the opinion of a family member or of a friend who knows you very well. Explain the concepts that the assessment is designed to measure and ask this person which characteristics describe you best.

Making decisions is hard. People naturally tend to seek the quickest and easiest way to decide between x and y. This is one reason why people often perceive an assessment as the beginning and ending of the decision-making process. Instead, use an assessment as the start of a gradual process of self-discovery.

1 comment:

  1. Excellent advice. In particular, the idea that assessments are not meant to provide "the answer." I had someone ask me once, why doesn't this assessment tell me the 5 best careers for me? He did not want more options than that, thinking that there should be a test that offered such a degree of accuracy.

    I also like your suggestions about discussing an assessment's concepts or results with someone who knows you very well. In the Internet Age, sometimes people forget the power and value of one-on-one, verbal communication. I know that sometimes I've learned things about myself that way that I would not have otherwise.

    ReplyDelete