From Kahneman, Sibony, and Sunstein's book Noise

We get things wrong. But understanding the anatomy of the error can be more important than the judgment itself.

Errors can be thought of in two ways: bias and noise. Bias is when errors are in the same direction. Noise is variability in judgments that should be identical.

Noise can be good. Disagreements and contrarian thinking are essential ingredients to innovation—the market tests competing strategies. But there are many decisions where noise is a problem. Those of us who think analytically often believe that random errors cancel each other out. However, in performance ratings, judgment calls, and measurement, noise is highly detrimental to companies.

We can sometimes correct bias by examining the decision-making process.

What to look for in reducing decision-making bias?

But even we when eliminate bias, there still exists systemic noise in the system that causes wrong decisions. Kahneman describes two types of noise: level noise and pattern noise. Level noise is variation across individuals. In a performance review, some reviewers are more generous than others. That's level noise. Judgment scales are ambiguous ("on a scale of 1 to 10"), and words may mean different things to different people ("exceeds expectations"). Pattern noise is the difference in the personal responses of people to the same things. It could be due to differences in principles or values that a person holds, consciously or unconsciously.

How do you reduce noise?

What's measured gets managed. Kahneman and the authors did a study on the level of noise in a company. The executives estimated differences ranged from 5% to 10% in the organization. The results were shocking. They showed that the "noise index" ranged from 34% to 62%.

Divide and conquer. Breaking decisions up into independent tasks reduces the tendency for people to distort or ignore information that doesn't fit the emerging story. Structured interviews are a great way to put this into practice.

A decision made after careful consideration is always better than a snap judgment. Kahneman suggests that professionals shouldn't be given information that they don't need and could bias them, and calls this sequencing the information.

Scales that use comparisons are less noisy than absolute scales.

Group discussions often create noise. Averaging different independent decisions will reduce noise but may not tackle bias.

These lists were taken from Kahneman, Sibony, and Sunstein's book Noise: A Flaw in Human Judgment.