Goodhart’s Law Isn’t as Useful as You Might Think

Metadata

• Author: Cedric Chin
• Full Title: Goodhart’s Law Isn’t as Useful as You Might Think
• Category: #articles
• Summary: The text discusses how Amazon’s Weekly Business Review (WBR) process addresses Goodhart’s Law, which states that when a measure becomes a target, it ceases to be a good measure. Amazon focuses on controllable input metrics and output metrics to drive desirable outcomes. The WBR process involves continuous refinement of metrics through trial and error to ensure they are effective in achieving business goals. The text also highlights the role of Finance in certifying data and investigating metrics presented during the WBR to prevent manipulation. Overall, the WBR process at Amazon serves as a mechanism for leadership to synchronize and build a causal model of the business, aiming to avoid Goodhart’s Law-type situations.
• URL: https://commoncog.com/goodharts-law-not-useful/

Highlights

• When people are pressured to meet a target value there are three ways they can proceed:
  1. They can work to improve the system
  2. They can distort the system
  3. Or they can distort the data (View Highlight)
• if you want to improve some process, you have to ignore the goal first, in favour of examining the process itself (View Highlight)