07-source-material / web-archives / cedric-chin-process-behaviour-charts-more-than-you-need-to-know

extracted

Process Behaviour Charts: More Than You Need To Know

By Cedric Chin

Table of Contents

  1. The Practical Questions
    1. Can you use this technique on any kind of ‘process’?
    2. What are process behaviour charts used for?
    3. How do you construct process behaviour charts?
    4. How do you interpret process behaviour charts?
    5. What is the purpose of the moving range chart?
    6. How do you calculate the median XmR chart? When should you use median?
    7. How many data points do you need to be confident of your process limits?
    8. Do you need to know the probability distribution of your data?
    9. What are some common gotchas?
  2. Theoretical Questions
    1. Why use the term ‘process’?
    2. Why three sigma limits?
    3. Would this work for any single-humped distribution?
    4. Can you use standard deviation instead of three sigma limits?
    5. How are the constants derived?
  3. Wrapping Up

Welcome, Ben Wilson

Thank you for being a member! Updates from across the membership: New Cases

Recent Forum Topics

Note: this is Part 5 in aseries of blog posts about becoming data driven in business. You may want to read the prior parts before reading this essay. The last time we looked at process behaviour charts was in How to Become Data Driven, which laid out the Statistical Process Control (SPC) practitioner’s approach to, well, become data driven. A quick recap of that essay is:

  1. To become data driven, you need to understand variation.
  2. There are many ways to understand variation. The approach that SPC practitioners recommend is to use process behaviour charts. The simplest process behaviour chart is known as the XmR chart.
  3. Once you start using such charts, the SPC argument is that your organisation will come to understand variation, and will thus slowly adopt the style of thinking necessary to do continuous improvement.

I’ve spent more time digging into the underpinnings of process behaviour charts since publishing that piece. This post pulls what I’ve uncovered into one resource, which should hopefully make it easier for you to put these ideas to practice. Bear in mind that I’m only at the beginning of application in my own work; I don’t yet have practical advice for XmR chart usage. I expect to update this piece in a few months with notes from use. For now, treat this as a distillation and a map for further reading.

The Practical Questions

We’ll start with more practical ‘how to’ notes, and then move on to more theoretical ‘why the hell does this even work?’ questions.

Can you use this technique on any kind of ‘process’?

The word ‘process’ in ‘process behaviour charts’ is a little deceptive. It doesn’t mean ‘industrial process’ or even necessarily some kind of business workflow that you run in your company. In my early readings on process control, I was a little surprised to see XmR charts of shopping mall footfall and of monthly receipts of insurance premium payments. In Donald Wheeler’s Making Sense of Data there is a particularly striking chart of daily peak exhalation flow rate readings by an asthma patient. None of these measures are traditionally associated with industrial processes (or, hell, with processes that one may run inside a company). I was expecting more typical metrics like ‘in-process inventory’ or ‘on-time shipping’. So what gives? In the original 1931 book where he invented the approach, American statistician Walter Shewhart used the word ‘phenomenon’ instead of ‘process’:

A phenomenon is said to be controlled when, through the use of past experience, we can predict at least within limits, how the phenomenon may be expected to vary in the future. Here it is understood that prediction within limits means that we can state, at least approximately, the probability that the observed phenomenon will fall within the given limits. The word ‘phenomenon’ captures a little of what we’re looking for. The truth is that there are only three requirements for using a process behaviour chart:

  1. All data points come from the ‘same system of causes ’. You are assuming that this system displays some kind of predictable, steady-state, routine variation.
  2. All data points are measured the same way and by the same method.
  3. The data should be arranged in chronological order (which implies you need to know when each measurement was taken — you do want to plot a time series, after all).

Or, to quote Wheeler: “The XmR Chart is intended for use with sequences of values that are logically comparable.” What does ‘same system of causes’ mean? Well, let’s say that we’re trying to identify exceptional variation in an asthma patient’s peak exhalation flow rates. Our patient takes two measurements a day: one in the morning, before medication (let’s call this measurement X), and one in the evening, after taking medication (let’s call this measurement Y). For the sake of expediency, the patient’s doctor plots both values on the same time series, so the graph reads: X1, Y1, X2, Y2, X3, Y3 and so on. This is a perfectly good time series, but it is not suitable for turning into an XmR chart, as measures X and Y are not logically comparable. In fact, we may say that the two sets of readings come from two different systems of causes. The doctor must separate these two readings and plot an XmR chart for measure X and for measure Y separately. The overall point I’m making is that ‘process’ is more like ‘some underlying system of causes’ than it is ‘rigid business workflow’. As an example, I have subjected website visits to the process behaviour chart approach — a perfectly reasonable use for a perfectly normal business ‘phenomenon’.

What are process behaviour charts used for?

In a sentence: they are used to separate signal from noise. The main problem that you have when you’re looking at any metric is that you often don’t know if a change represents something significant. Sales is down 12% this month — is that something to worry about? Is it seasonal variation? Should you wait a bit longer before you investigate? Conversely, if your boss yells at you to investigate now, are you going to waste all the time you spend digging into it — because it turns out to be routine variation? The truth is that all natural processes display some amount of variation. Process behaviour charts give you a superpower: you are now able to tell if the variation you’re looking at is routine or exceptional. This may be used in a variety of ways:

  1. You may investigate with confidence when exceptional variation shows up (either good or bad). No more “am I going to waste my time investigating this?” and “uhh, is this bad?” Conversely, you may ignore routine variation with the same confidence.
  2. You know when your process improvement activities are beginning to give you results — because it will show up as exceptional variation. This in turn means that you’re better able to discover controllable input metrics to your process.
  3. Finally, you know what your next steps are if you want to improve any process. Some terminology: if a process displays only routine variation, it is predictable. If a process displays both routine and exceptional variation, it is unpredictable. Hence: if the process is unpredictable, you need to first investigate and then remove exceptional variation. If the process is predictable, then you need to completely rethink the process. (A predictable process is already running the best it can, and the only way to change process behaviour is to fundamentally change the underlying process.)

How do you construct process behaviour charts?

I’ve already gone through the basics in a prior essay, but here’s a recap. Let’s assume that we’re going to use the average to plot our XmR charts.

  1. Plot a time series for the metric (X) you’re interested in.
  2. Calculate and then plot the moving range for that same metric. The moving range is simply the difference between two successive data points in the X metric. Note that there should be no negative moving range numbers here — we’re interested in the magnitude differences, not whether the numbers go up or down.
  3. Calculate the average for the business metric and plot the average as a straight line through the time series.
  4. Calculate the average for the moving range and plot the average as a straight line through the moving range time series.

You’re going to end up with two charts that look like this: The next thing we’ll need to do is to draw the limits.

  1. To calculate the Upper Natural Process Limit: multiply the average moving range by a scaling factor 2.66 and add the product to the average X. 𝑈𝑁𝑃𝐿=𝐴𝑣𝑒𝑟𝑎𝑔𝑒𝑋+2.66𝐴𝑣𝑒𝑟𝑎𝑔𝑒𝑚𝑅
  2. To calculate the Lower Natural Process Limit: subtract the 2.66×𝑎𝑣𝑒𝑟𝑎𝑔𝑒𝑚𝑅 product from the average X. 𝐿𝑁𝑃𝐿=𝐴𝑣𝑒𝑟𝑎𝑔𝑒𝑋−2.66𝐴𝑣𝑒𝑟𝑎𝑔𝑒𝑚𝑅
  3. Finally, calculate the Upper Range Limit (the upper limit for the moving range) by multiplying the average moving range by the scaling factor 3.269: 𝑈𝑅𝐿=3.269×𝐴𝑣𝑒𝑟𝑎𝑔𝑒𝑚𝑅

You’ll end up with your final process behaviour chart, like so: There is also a version of the XmR chart that uses median instead of average. We’ll talk about that in a second.

How do you interpret process behaviour charts?

For simplicity’s sake, there are only three rules that you should use. These are known as the ‘Western Electric Zone’ rules in the SPC literature. (Note the implication here: over the years there have been many additional rules — often added by software packages or by consultants wanting to make a buck from extra complexity. Statistician Donald Wheeler argues that you really only need these three.) A process behaviour chart is said to display exceptional variation if: First, a data point lies outside the natural process limits. You should investigate that data point immediately (and should expect to find a clear, strong cause of exceptional variation). Second, three out of four consecutive data points lie either within the top 25% of the upper natural process limit, or within the bottom 25% of the lower natural process limit. In this case you should expect to find an exceptional cause of variation that is moderate but sustained. An easier way to visualise this rule is to imagine a line drawn exactly in the mid-point between the process limits and the average line (the blue dotted lines below). If three out of four consecutive data points lie between these lines and the process limit lines, you have exceptional variation. Third, eight data points in a row lie on one side of the average line. In this case you should expect to find an exceptional cause of variation that is weak but sustained. You’ll notice that the XmR chart I’ve chosen as an example above displays all three rules. In this case, you’ll want to investigate instances of Rule One first, since these would yield the strongest sources of exceptional variation, and provide the clearest opportunities for improvement. Why does this work? The high level intuition here is that the process limit lines are estimates of three sigma around the average. Any data point that lies outside these limits are highly likely to be variation from some exceptional cause, instead of routine variation from the process itself. This explains Rule One. The other two rules come from some theoretical basis, but mostly from empirical testing. (An earlier version of the third rule was ‘seven data points’, but experience has since caused practitioners to revise it to eight). It is worth noting here that the magic of XmR charts is not really in the reading of the charts, but in taking action based on what the charts tell you. Such action will result in better, deeper understanding of your business. The way SPC practitioners talk about this is to point out that the XmR chart are not designed for precision, but for action. So, for instance:

  1. You will get more insight into your business by investigating exceptional variation, and then deciding what to do about it once you’ve isolated the cause.
  2. Often, what you want to do when you’re looking at some process is to improve it. With an XmR chart in hand, what you’ll do is to take a stab at changing the process, and then … wait for exceptional variation. If none shows up, even after a suitable time lag, you know that you’ll have to go back to the drawing board. In this manner you may figure out all the controllable inputs into your metric of interest.

Consequently, XmR charts only estimate three sigma, they don’t precisely calculate it. And that simplicity is ok for the purpose of taking action. Once again, since this is important: the magic of process behaviour charts lies not in the creation of the charts, or even the reading of the charts, but in the action you take in response to variation. (Sources: Understanding Variation and Making Sense of Data)

What is the purpose of the moving range chart?

So far, we’ve mostly discussed the interpretation of the XmR chart using the X chart, not the mR chart. There are three uses for the mR chart:

  1. If a point exceeds the Upper Range Limit on the moving range chart, this signals a break in the time series — that is, a change in the underlying process so large that you likely have to plot a new set of process limits after the break.
  2. The moving range chart may be used to detect chunky data — a failure mode for XmR charts. (We’ll talk more about chunky data in the ‘common gotchas’ question below). If the mR chart has only three possible values below the Upper Range Limit, then you have chunky data, and the XmR chart is not likely to be useful to you.
  3. Finally, Donald Wheeler argues that the presence of an mR chart next to an X chart is a sign that the process limits were calculated correctly, using the formulas above. This is apparently a good thing, because so many software packages or process consultants will insist on calculating three standard deviations instead of three sigma when plotting limits. (We will discuss why this is not correct later).

I’m not so sure about the importance of that third point, but … ok. The main takeaway I have from the literature is that the bulk of your exceptional variation detection will occur from the X chart, not the mR chart.

How do you calculate the median XmR chart? When should you use median?

Earlier in this article we talked about plotting an XmR chart using averages. In addition to plotting process limits using averages, we may also opt for an alternative method that uses median values. Why is this useful? In most cases calculating the average would suffice. But in certain scenarios, you may encounter data with excessively large moving ranges — either because there is large exceptional variation, or because there just happens to be one or two extremely large moving range values. As a result, your average moving range and your computed limits may both be inflated when you plot process limits around an average. In such a situation you should compute using a median. Medians are less efficient than averages in their use of data, but are also less susceptible to extreme values. The formulas for computing process limits using medians are slightly different, but otherwise the process to plot an XmR chart with them are the same:

  1. Plot a time series for the metric (X) you’re interested in.
  2. Calculate and then plot the moving range for that same metric.
  3. Calculate the median for the business metric and plot the median as a straight line through the time series.
  4. Calculate the median for the moving range and plot the median as a straight line through the moving range time series.
  5. To calculate the Upper Natural Process Limit: multiply the median moving range by a scaling factor 3.14 and add the product to the median X. 𝑈𝑁𝑃𝐿=𝑀𝑒𝑑𝑖𝑎𝑛𝑋+3.14𝑀𝑒𝑑𝑖𝑎𝑛𝑚𝑅
  6. To calculate the Lower Natural Process Limit: subtract the 3.14 x mR product from the median X. 𝐿𝑁𝑃𝐿=𝑀𝑒𝑑𝑖𝑎𝑛𝑋−3.14𝑀𝑒𝑑𝑖𝑎𝑛𝑚𝑅
  7. Finally, calculate the Upper Range Limit (the upper limit for the moving range) by multiplying the median moving range by the scaling factor 3.86: 𝑈𝑅𝐿=3.86×𝑀𝑒𝑑𝑖𝑎𝑛𝑚𝑅

The resulting XmR chart may be interpreted and used the same way you would an average XmR chart.

How many data points do you need to be confident of your process limits?

In general, the more data points used in constructing a process behaviour chart, the better. But recall that the goal of the XmR chart is not to ‘get the right number’ but to ‘take the right action’. As Donald Wheeler puts it: so long as the computed limits provide an adequate description of the process, they are sufficient. What this means: computed limits with just five to six data points are good enough to start acting on. And if you are in the midst of improving a process, waiting for five to six new data points should be enough for you to say — with some confidence — that your change has affected the process. Here’s an example, taken from Understanding Variation : Note two implications from the chart above:

  1. The process manager moves on after just four data points at one point in the chart (right when they introduce a new formula). You’re allowed to do this! My read is that the process engineers were likely very confident that their formula change would result in a permanent dip in material costs.
  2. Waiting for five to six data points implies waiting for results after each change , and not moving on the next process improvement change too quickly. This is the Deming cycle in action. Imagine what this means, though — it means waiting a couple of months just to be sure that your efforts have resulted in a real benefit! What kind of manager is patient enough to do so? The answer: a wise one.

Natural process limits tend to solidify around 17 data points. The limits settle at 24 data points or more. In practice, you may start taking action after you have computed limits for as few as five to six data points.

Do you need to know the probability distribution of your data?

No, you do not need to know the distribution of your data before hand, nor do you need to normalise your data. In fact, as we’ll see later on in this guide, XmR charts were designed for the common situation where you don’t know (or can’t know) the distribution for your data. Another way of stating this is that XmR charts are designed to work with all single-humped distributions. The one caveat here is that XmR charts only work with single-humped distributions. (A single humped distribution is usually one that comes from just one system of causes). We will examine why in a bit. For now, a real world example should illustrate what this means: let’s say that you’re charting the behaviour of footfall to a shopping mall. You find that your weekend data shows consistently higher footfall than on weekdays — which is common sense; malls are busier on weekends. In this situation, we may say that your distribution is really ‘double-humped’ (one peak for weekdays, and another peak on weekends). Another way of thinking about this is that your data really comes from two different system of causes. Solution: you should separate the two sets of data, and draw one XmR chart for weekdays, and another XmR chart for weekend data. This would allow you to spot changes in either process clearly.

What are some common gotchas?

Here are some edge cases: Chunky data — XmR charts are designed to be conservative: that is, the chart errs on the side of no signal even when there is exceptional variation, instead of giving you a false alarm. But this attribute fails when an XmR chart is plotted with chunky data. Data for an XmR chart is chunky when there are fewer than three possible range values (excluding zero) on the moving range chart, below the upper control limit of that chart. As an example, imagine that you’re working in a chemical production plant, and you want to chart out the occurrence of spills. Spills occur very rarely. If you plot spill occurrences on a time series, you’ll get something quite useless: This is not going to give you a usable XmR chart. Spill occurrence is considered chunky data, since there are only two values if you plot out a moving range: 0 and 1. However, you may redo this chart by measuring the ‘days between spills’, which in turn will give you ‘Spills per Day’ and ‘Spills per Year’. This will yield the XmR chart below: This allows you to tell if there has been a change in the underlying process that leads to spills, and allows you to act appropriately. Autocorrelated data — Autocorrelation occurs when successive data points are too highly correlated with each other. An example might be the daily closing of the Dow Jones Industrial Average — which is highly correlated because each day’s activity starts from the previous day’s closing number. (If you calculate the correlation coefficient of the data below, you’ll find that it equals ~0.95). XmR charts deal fairly well with correlated numbers, but fail when the correlation coefficient between two successive numbers is higher than 0.7. This tends to result in process limits that are too tight. The easiest way to avoid this is to think about how fast your process can change, and then avoid sampling the process faster than that process may change. As an example, it does not make sense to measure temperature changes in a boiler every 15 seconds — variation in boiler temperature does not change that quickly! An XmR chart constructed from a pile of 15-second readings will have weird process limits. Measuring once an hour makes more sense. Too many data points — A related problem to the two gotchas above is that of too much data. The following chart is taken from a Quality Digest column, and represents quality data that is collected eight to 10 times a day over several months. It is unreadable and therefore useless as an XmR chart. As mentioned earlier, you’ll want to collect data at a frequency that matches the process’s ability to change.

Theoretical Questions

The questions that follow are less immediately practical, but still useful. Fair warning: the answers here get very technical very quickly. Read on if you want to know why these techniques even work, but skip ahead to the conclusion if you just want to get to work.

Why use the term ‘process’?

Earlier, we talked about how ‘process’ does not necessarily mean ‘business process’, and how Walter Shewhart originally used the term ‘phenomenon’ when introducing process behaviour charts as a process control technique. So why not use ‘phenomenon’? I don’t have an actual citation for this, but I believe the term ‘process’ is more useful due to its affordances: as an operator, you want to look at any business chart (including and especially revenue) and think “ok, this is the result of some real world process, and I don’t know what the inputs are to this process, but I’m going to find out.” Treating your business as a process that you may decompose turns out to be a very powerful way of thinking about business, as we’ve already discussed.

Why three sigma limits?

At first I thought that selection of the three sigma limits was related to the so-called ‘three sigma rule’ — that is, the observation that 99.7% of all values are covered within three standard deviations of the mean in a Gaussian distribution. But that would rely on an assumption that all the distributions we see in the real world are Gaussian, or at least Gaussian-like. This cannot be the case, can it? As it turns out, Walter Shewhart was asking a different series of questions. He observed that when we see variation in the real world, the first question we should be asking is: is this data we’re looking characterised by one probability model? (You can imagine variation as random draws from a bowl of numbers). Or is the data we’re looking at affected by multiple probability models? (Imagine that you’re drawing from multiple different bowls of numbers — which should be the case if there is exceptional variation present in your process). On top of that, let’s say that the variation we spot is characterised by one probability model. How can we tell if that probability model changes over time — say if we do things to change the underlying process? These questions boil down to “is the variation that we’re seeing only routine variation, and thus natural to the process, or is the variation we’re seeing both exceptional and routine in nature?” This is known as the ‘homogeneity question’. Many statistical techniques assume that our random numbers are drawn from just one probability model. This is not the case in the real world! As Donald Wheeler writes: “this implicit assumption of homogeneity, that is part of everything we do in traditional statistics classes, becomes a real obstacle whenever we try to analyse data.” What Shewhart has done is to say something like “ok, if we can establish some limits of variability for most of the distributions out there , then if an observation is found outside the limits, looking for an assignable cause is worthwhile.” Another way of saying this is that if an observation is found outside the limits, we know that the data is non-homogenous. Naturally, whatever limit we choose should be as conservative as possible, so that we don’t go on wild goose chases. Shewhart then writes, in Economic Control of Quality of Manufactured Product : “If more than one statistic is used, then the limits on all the statistics should be chosen so that the probability of looking for trouble when any one of the chosen statistics falls outside its own limits is economic. (…) We usually choose a symmetrical range characterised by [𝑡-sigma] limits. Experience indicates that 𝑡=3 seems to be an acceptable economic value.” The term sigma denotes a standard unit of dispersion. What Shewhart is saying is that for the vast majority of probability distributions out there, spotting a data point outside of the three sigma limits should be enough to convince us that there is exceptional variation present. The odds of a false alarm won’t be as good as 0.03% as it is with the three sigma limits on a Gaussian distribution, but Shewhart argues that the false alarm rate is really low for the vast majority of the distributions we will see in the real world. We go through the evidence for this in the next section.

Would this work for any single-humped distribution?

Yes. This is by design. I’ll paraphrase Donald Wheeler for this answer, and then summarise in plain English. Wheeler writes: On pages 275-277 of Economic Control of Quality of Manufactured Product Shewhart discussed the problem of “establishing an efficient method for detecting the presence” of assignable causes of exceptional variation. He began with a careful statement of the problem: if we know the probability model that characterises the original measurements, 𝑋, when the process satisfies the differential equation of statistical control, then we can usually find a probability model, 𝑓(𝑦,𝑛), for a statistic 𝑌 calculated from a sample of size 𝑛 such that the integral: ∫𝐵𝐴𝑓(𝑦,𝑛)𝑑𝑦=𝑃 will define the probability 𝑃 such that the statistic 𝑌 will have a value that falls in the interval defined by 𝐴 and 𝐵. This is the approach that most of us would take, having gone through undergrad statistics. Let’s call this the statistical approach:

The Statistical Approach : Choose a fixed value for 𝑃 that is close to 1.00, then for a specific probability distribution 𝑓(𝑦,𝑛), we find the critical values 𝐴 and 𝐵. Then when 𝑌 falls outside the interval 𝐴 to 𝐵, the observation may be said to be inconsistent with the conditions under which 𝐴 and 𝐵 were computed. This approach is used in all sorts of statistical inference techniques. And, as the argument goes, once you fix the value for 𝑃, the values for 𝐴 and 𝐵 will depend upon the probability model 𝑓(𝑦,𝑛). But this requires us to know what the probability distribution is. Shewhart rejected this approach. He pointed out that in the real world, we will “never know 𝑓(𝑦,𝑛) in sufficient detail to set up such limits.” So he inverted it: Shewhart’s Approach: Choose generic values for 𝐴 and 𝐵 such that, for any probability model 𝑓(𝑦,𝑛), the value of 𝑃 will be reasonably close to 1.00. Such generic limits will still allow a reasonable judgment that the process is unlikely to be predictable when 𝑌 is outside the interval 𝐴 to 𝐵. Ultimately, we do not care about the exact value of 𝑃. As long as 𝑃 is reasonably close to 1.00 we will end up making the correct decision virtually every time. After consideration of the practical issues of this decision process Shewhart summarised with the following: “For these reasons we usually choose a symmetrical range characterised by limits 𝐴𝑣𝑒𝑟𝑎𝑔𝑒±𝑡𝑠𝑖𝑔𝑚𝑎 symmetrically spaced in reference to [the Average]. Tchebycheff’s theorem tells us that the probability 𝑃 that an observed value of 𝑌 will lie within these limits so long as the quality standard is maintained satisfies the inequality 𝑃>1–1𝑡2 We are still faced with the choice of 𝑡. Experience indicates that 𝑡=3 seems to be an acceptable economic value.” And this has been shown to be true through experience. What Wheeler is pointing out, and what Shewhart is saying in his 1930s statistician way, is that it is unrealistic to expect the business manager or the production engineer to guess at the probability model of their data. It’s far better to assume that the operator does not know the probability model, and to fix a set of limits that reduces the risk of false alarms (that risk is lower the higher the value of 𝑃) — regardless of what probability distribution is represented in the data. But note here that even that question is malformed. The question that Shewhart wanted to answer is really “is this process data even homogenous?” — that is, does the data show routine variation only? Attempting to fit a probability model to non-homogenous data is a waste of time. Rather, Shewhart wanted to quickly show that a set of data displays only routine variation (to hell with what probability model the data comes from), so that if it is found to not be the case, we know to investigate. The way Shewhart put this, in his old-timey 30s era statistician language, is: “we are not concerned with the functional form of the universe, but merely with the assumption that a universe exists.” Which, um, nicely done. And indeed this is what Shewhart accomplished. In theory, the Chebychev inequality guarantees that the three sigma limits will cover at least 89% of the area under the probability model (which puts 11% or so of data points outside the limits, and therefore gives us a 1 in 10 false alarm rate). But in practice, when used with distributions that we find in the real world, Wheeler has found that the three sigma limits give values for 𝑃 that are much, much closer to 1.00. To show this, Wheeler took 1143 probability models and used the skewness and kurtosis properties of the models to place these distributions on a shape characterisation plane. He then calculated the coverage of the three-sigma intervals for each of these 1143 models. The results are displayed below (read the full article for details): Of the distributions, the three sigma limits provide better than 98% coverage for all mound shape probability models. (Wheeler points out that most real-world histograms are mound-shaped, which likely explains why Shewhart was satisfied with the three-sigma limits in practice). Wheeler then notes that J-shaped distributions will have better than 97.5% coverage. Since mound shaped and J-shaped distributions effectively cover most (if not all) distributions you will encounter in real life, you should expect a less than 2.5% false alarm rate when using the three sigma limits. Wheeler takes special pleasure in saying that this is more conservative than the 5% risk of a false alarm that is used ‘in virtually every other statistical procedure’. The one caveat (and this is where our warning about double humped distributions comes from): In the U-shaped region most of the probability models will have 100% coverage. However, some very skewed and very heavy-tailed U-shaped probability models will fall into the Chebychev minimum. Offsetting this one area of weakness is the fact that when your histogram has two humps it is almost certain that you are looking at the mixture of two different processes, making the use of three-sigma limits unnecessary. In sum: you don’t have to worry about the probability distribution of your data. Regardless of underlying distribution, XmR charts should be able to do what they say on the tin. (Source)

Can you use standard deviation instead of three sigma limits?

No, you cannot. The XmR chart examines a collection of values to see if they might have come from one underlying and unchanging process, or if they show evidence of process changes. The standard deviation statistic, 𝑠, is a global statistic, commonly computed using all the data. The assumption behind this computation is that the data can be logically considered to be one large homogenous collection of values, all obtained from the same underlying and unchanging process. Put differently, while the XmR Chart examines the data for evidence of non-homogeneity, the global standard deviation statistic, 𝑠, assumes the data to be completely homogenous. But that defeats the purpose of the XmR chart! Recall: you don’t know if the metric you are looking at contains routine or exceptional variation, and you want to find out. Computing 𝑠 assumes that your data only shows routine variation, which may or may not be the case. Therefore, drawing process limits using 𝑠 will result in the wrong conclusions.

How are the constants derived?

For the sake of brevity, I’ve uploaded a PDF scan of the relevant section of Making Sense of Data here.

Wrapping Up

One of the remaining questions that I have is “why aren’t process behaviour charts more widespread?” Of course, one of the ironies here is that while XmR charts may have started in manufacturing, most modern lean manufacturers no longer use them due to the sheer deluge of data they have from computerised production. They have moved on from the kind of vanilla SPC we’re learning here. But statistical process control is used in other fields: medicine, for instance — the asthma case study in Making Sense of Data is but one example — which perhaps gives you a sense of the kinds of ‘natural’ processes it might be useful to use process behaviour charts with. If you do a google search for ‘run charts’ — which is a precursor form of the XmR chart — and you’ll find a ton of resources from medical websites. Here is one example. But why not other fields? Why not tech? Why not business in general? Process behaviour charts are simply a tool to separate exceptional variation from routine variation, to help you tell signal from noise. That seems applicable to a lot of domains. I don’t have good answers here. I will say that I find it irritating to look at a raw time series today, without at least an average line plotted through the middle, because I do not know if I’m looking at something meaningful. I can’t tell if it’s signal or noise. I yearn for process limits, and I feel mild irritation when I hear someone say “ok that’s good right?” when they see a number go up. I have one final note. One important takeaway from this series is that it is variation that is important to recognise and understand, not necessarily XmR charts that you should use for everything. I’ve not called this out explicitly, but Amazon, for instance, doesn’t demand the use of XmR charts to recognise variation. But they sure as hell have a deep understanding of variation in all of their metrics work. On my reread of Working Backwards , the ideas that I’ve dug into during this series stand out all the more starkly. Bryar and Carr write things like:

Charlie Bell, an SVP in AWS and a great operational guru at Amazon, put it aptly when he said, “When you encounter a problem, the probability you’re actually looking at the actual root cause of the problem in the initial 24 hours is pretty close to zero, because it turns out that behind every issue there’s a very interesting story.

In the end, if you stick with identifying the true root causes of variation and eliminating them, you’ll have a predictable, in-control process that you can optimize (emphasis mine). Now, as effective as the WBR process can be, it can also go astray in several ways, including poor meeting management, focusing on normal variations rather than signals , and looking at the right data but in the wrong way (emphasis mine). Contradictory as this may sound, variation in data is normal. And unavoidable. It’s therefore critical to differentiate normal variation (noise) from some fundamental change or defect in a process (signal). Trying to attach meaning to variations within normal bounds is at best a waste of effort and at worst dangerously misleading. It’s bad enough when someone proudly explains how their herculean efforts moved their key metric up by 0.1 percent this week, taking precious time away from more important things. Worse, if that same metric went down by 0.1 percent, you could easily waste time chasing down the root cause and “fixing” an issue that’s really nothing more than normal variation.

At Amazon, understanding what’s normal is the responsibility of the metrics owner, whether that’s an individual contributor or a manager of thousands. Many statistical methods, such as XmR control charts, can highlight when a process is out of control. For us, however, experience and a deep understanding of the customer most often turned out to be the best way to filter out the signal from the background noise. For the most part, metrics are reviewed daily by their owners and weekly in the WBR, so that expected fluctuations become familiar and exceptions stand out (emphasis added). You don’t have to use process behaviour charts if you don’t want to. But you must understand variation if you want to become data driven. XmR charts is simply one way to do so. This is Part 5 of theBecoming Data Driven in Business series. The next part is here: What to Think When Looking at a Chart. Originally published 10 May 2023, last updated 26 January 2024. This article is part of the Operations topic cluster, which belongs to the Business Expertise Triad. Read more from this topic here→

Welcome, Ben Wilson

Thank you for being a member! Updates from across the membership:

New Cases

Recent Forum Topics

Previous Post

← Mental Strength in Judo, Mental Strength in Life

Next Post

Lesley Sim on Skill Acceleration in Ultimate →

Member Comments

More in Business Thinking

### Actually Effective Corporate Leadership Training ### How to Make Sense of AI ### How to Find Demand: All the Ideas from The Heart of Innovation, Simply Stated ### A Few Notes on Moats