The Importance of the Sample

In my work researching and writing about occupations, I encounter a lot of statistics. And this year, with an election coming ever closer, we are likely to see the results of many surveys of voters. I want to emphasize that numbers reported from surveys tell less than half of the story. They are the results of mere tabulation. What makes the numbers meaningful is the nature of the sample. Or, to put it another way, you can’t understand what a study tells you unless you understand the sample it’s based on.

To illustrate this point, I like to bring up two anecdotes. I think you’ll find them interesting even if (maybe especially if) you’ve never taken a course in statistics.

The first anecdote is based on the research that social scientists did when they essentially invented the science of jury selection. This happened in 1972, when seven radicals were about to go on trial in Harrisburg, Pennsylvania, for conspiracy to raid draft boards and destroy records, among other planned antiwar actions. This was a time of great political polarization and in a place that is characterized by political conservatism. The researchers, working on behalf of the antiwar activists’ lawyers, wanted to find a way to predict the political leanings of jurors so the lawyers could seat a jury that would be less conservative than one chosen at random from the Harrisburg population. The lawyers would not be able to ask the potential jurors flat-out about their politics; instead, they needed an indirect way to assess this.

The social scientists surveyed citizens of that community to identify their political attitudes and then correlated these attitudes with other facts about the jurors. They discovered that the surest way to predict a Harrisburger’s politics was to ask how much education the person had: The more educated the person was, the more conservative that person’s politics.

The researchers eventually realized why this was so: Young people in Harrisburg who became highly educated acquired the occupational mobility to leave the region if they were not conservative; therefore, the sample of highly educated people who remained had to be quite conservative. If the results of their survey surprised you, it’s because you didn’t stop to think about what the sample really was: not everyone who ever lived in Harrisburg, but rather those who remained—by choice or because they were less able to move out.

The second anecdote is from the Second World War. British bomber planes flying missions over Germany were often shot down by anti-aircraft fire. The Royal Air Force wanted to shield vulnerable parts of the aircraft with armor, but they wanted to use a minimal amount of armor to avoid weighing down (and slowing down) the planes. The RAF commissioned the statistician Abraham Wald to examine the planes after bombing missions to determine where on the planes’ undersides it was most critical to apply anti-flak armor.

Wald counted bullet holes in the planes and recommended that armor be applied where there were the fewest bullet holes.

This may seem like a mistake to you. Maybe you’re thinking that armor is supposed to protect against anti-aircraft fire, so shouldn’t the RAF have armored the places that got hit the most?

Again, consider the sample: Wald was not looking at every bomber that flew a mission, but rather those that returned from missions. Bombers that got shot down were removed from the sample. The bombers that returned and made up the sample were the ones that were hit only in places that were not critical for staying airborne. The places where the surviving planes were not hit, therefore, were the most likely to be critical and in need of armor.

If you’re wondering why I’m writing about this subject in a blog about careers, consider this blog entry a look at how complicated statisticians’ work can be, not so much in terms of the mathematics, but rather in terms of the concepts that must be understood.

The nonstatistical lesson to take away from these anecdotes is that you have to be careful when you make a generalization about a population—for example, the notion that educated people are more liberal politically (or, to draw on today’s politics, the notion that people of one religion are a greater threat to security). Such generalizations may be true in some global sense, but the particular population you are dealing with may really be a subset of the global population, either self-selecting or selected by some exterior factor you have not considered. The global generalization may be a poor fit for this subset, or the subset may be a misleading basis for a global generalization.

Should I Sign That Noncompete?

It is a paradox of today’s job market that employers want ever-greater flexibility in their ability to shed workers but simultaneously want to reduce workers’ flexibility in seeking employment. Specifically, employers increasingly are imposing noncompetition agreements (“noncompetes”) that can seriously limit workers’ ability to find jobs elsewhere. According to a White House report (PDF), an estimated 30 million Americans, nearly one-fifth of the workforce, are bound by these agreements, and roughly 37 percent have been so bound at some time during their careers. Perhaps the agreements themselves have not proliferated but merely their enforcement. Whichever is the case, “The law firm Beck Reed Riden LLP found a 61 percent rise from 2002 to 2013 in the number of employees getting sued by former companies for breach of non-compete agreements.”

The White House looked into this matter out of concern that noncompetition agreements can hamper the economic recovery. “Non-competes can reduce workers’ ability to use job switching or the threat of job switching to negotiate for better conditions and higher wages, reflecting their value to employers. Furthermore, non-competes could result in unemployment if workers must leave a job and are unable to find a new job that meets the requirements of their non-compete contract. In addition to reducing job mobility and worker bargaining power, non-competes can negatively impact other companies by constricting the labor pool from which to hire. Non-competes may also prevent workers from launching new companies.”

In some states, most notably California, employment laws make noncompetition agreements essentially unenforceable.  It is thought that the absence of noncompetes is one of the factors that have contributed to the towering success of the Silicon Valley. Job-hopping is a normal part of career building in the tech industry there. In fact, job-hopping is one of the reasons that employers have traditionally tended to cluster together geographically with others in the same industry, even when access to natural resources or transportation infrastructure is not a factor. Think of New York for finance, Nashville for music, or Detroit for automobiles.

Noncompetes reduce the efficiency of these industry clusters. As a result, The New York Times reports that some states are trying to limit the reach of noncompetes in hopes of duplicating one of the factors of the Silicon Valley environment: “Hawaii banned noncompete agreements for technology jobs last year, while New Mexico passed a law prohibiting noncompetes for health care workers. And Oregon and Utah have limited the duration of noncompete arrangements.”

I live in New Jersey and have personal experience with this kind of shackling. In the late 1990s, my employer required that I sign a noncompetition agreement as a condition for receiving a raise. I complied, although it bound me not to compete for one year, and after a downsizing only a few years later, the agreement seriously limited my work as a consultant. The crowning irony was that only a few years after I began consulting, my old employer came back to me in need of my consulting services and presented me with a contract that contained another noncompetition agreement—this one binding me for two years.

I refused to sign it, and with no hesitation or bargaining, they struck that paragraph from the contract. Since then, I have been asked by another employer to sign a noncompete and have again refused, with no adverse consequences.

What should you do if an employer confronts you with a noncompetition agreement? First, you should investigate whether it is enforceable in your state and for your occupation. To be totally sure, you may want to consult a lawyer, but you can get useful preliminary information from a downloadable chart at the website of Beck Reed Ridin, LLP.

It’s usually a good idea to negotiate with your employer over the terms of the noncompete. If you’re lucky enough to have some bargaining power, such as a very desirable skill set, you may be able to convince the employer to strike the agreement entirely. If not, you may be able to get the employer to relax some of the terms. For example, you may suggest altering the agreement to restrict you only in a certain geographic area or only from working for certain employers. You may be able to reduce the duration of the restriction. You may get the employer to accept wording based on the conditions of your future separation—for example, that the restriction will apply only if you quit, not if you are terminated.

Be sure to examine the fine print of any noncompetition clause. (Again, a lawyer may be helpful.) For example, some agreements include the onerous requirement that the ex-employee will have to pay any legal fees that the employer incurs as part of enforcing the agreement. Such additional burdens may also be negotiable before you sign.

Understand that one reason employers like to impose noncompetition agreements is that they fear you will carry company secrets to a competing organization. It is reasonable for the employer to ask you to sign a nondisclosure or confidentiality agreement with wording that is separate from noncompetition.