Run Charts

Overview
Run charts are one of the most useful tools in quality improvement. They allow us to:

• Monitor the performance of one or more processes over time to detect trends, shifts or cycles.
• Allow us to compare a performance measure before and after implementation of a solution to measure its impact.
• Focuses attention on truly vital changes in the process.
• Assess whether improved performance has been sustained.

Run charts are a valuable tool at the beginning of a project, as it reveals important information about a process before you have collected enough data to create a Stewhart control chart.

Characteristics of a run chart

• On the X axis you have data in some sort of chronological order e.g. Jan, Feb, Mar
• On the Y axis you have the measure of interest e.g. %, count
• Once the data points are connected you put a centre line (CL) between the graph. For a run chart the CL is called the Median.

The median is the number in the middle of the data set when the data are reordered from the highest to the lowest value. If the number of observations is even, the median is the average of the two middle values.

A typical run chart

How to create a run chart

Step 1 – State the question that the run chart will answer and obtain data necessary to answer this question.

For example, if you were looking at how long it takes to travel to work in the morning you will make note of the time taken (in minutes) to get to work over a period of a month.

Step 2 – Gather data, generally collect 10-12 data points to detect meaningful patterns.

Step 3 – Create a graph with vertical line (y axis) and a horizontal line (x axis).

• On the vertical line (y axis), draw the scale related to the variable you are measuring.

Please note: it is good practice to ensure the y axis covers the full range of the measurements and then some (e.g. 1 ½ times the range of data). This is to ensure the chart can accommodate any future results.

• On the horizontal line (x axis), draw the time or sequence scale.

Step 4 – Plot the data, calculate the median and include into the graph.

Step 5 – Interpret the chart. Four simple rules can be used to distinguish between random and non-random variations.

Interpreting a run chart

There are four rules that can be used to interpret a run chart. Non-random variation can be recognised by looking for:

• Rule 1 – Shift
  Six or more consecutive points either all above or all below the centre line (CL). Values that fall on the CL do not add to nor break a shift. Skip values that fall on the median and continue counting.

Rule 1 – shift

Rule 2 – Trend

Five or more consecutive points all going up or all going down. If the value of two or more successive points is the same (repeats), ignore the like points when counting.

Rule 2 – trend

• Rule 3 – Too many or too few runs

A non-random pattern is signalled by too few or too many runs, or crossings of the median line. If there are too many or too few runs, this is a sign of non-random variation. To see what an appropriate number of runs for a given number of data sets, refer to following statistical table. An easy way to count the number of runs is to count the number of times the line connecting all the data points crosses the median and add one. If the number of runs you have are:

• Within the range outlined in the table, then you have a random pattern.
• Outside the range outline in the table, then you have a non-random pattern or signal of change.

Rule 3 – Too many or too few runs

• Rule 4 – An astronomical data point

This is a data point that is clearly different from all others. This is a judgement call. Different people looking at the same graph would be expected to recognise the same data point as astronomical.

Rule 4 – An astronomical data point

By applying each of the four rules, you can evaluate the run chart for a signal for change (through a non-random variation). However, it is not necessary to find evidence of change with each of the four rules to determine that a change has occurred. Any single rule occurring is sufficient evidence of a non-random signal of change.

Additional resources 

• The run chart: a simple analytical tool for learning from variation in healthcare processes – Rocco J Perla, Lloyd P Provost and Sandy K Murray

Science of Improvement on a Whiteboard, with Robert Lloyd, Vice President, Institute for Healthcare Improvement

Control Charts

Overview

The control (Shewhart) chart is a statistical tool used to distinguish between variation in a measure due to common causes and special causes. A run charts is a powerful tool for detecting non-random variation but they are not sensitive in detecting special causes. Hence they are used in the early stages of an improvement project. Once more data is available, control charts are much preferred. Control charts are commonly referred to as Shewhart charts, name after Walter Shewhart who did early work in industry to develop this method.

Variation in a measure of quality has its origins in one of two types of causes:

• Commons Causes – those causes that are inherent in the system (process or product) over time, affect everyone working in the system, and affect all outcomes of the system.
• Special Causes – those causes that are not part of the system (process or product) but arise because of specific circumstances.

Characteristics of a control (Shewhart) chart

• On the X axis you have data in some sort of chronological order e.g. Jan, Feb, Mar
• On the Y axis you have the measure of interest e.g. %, count
• Once the data points are connected you put a centre line (CL) between the graph. For a control chart the CL is called the Mean.

The mean is calculated by taking all the values and dividing by the number of values (also known as the average).

• Control charts also contain control limits. You have an upper control limit (UCL) and a lower control limit (LCL).
• The control limits define the boundaries of expected common cause (random) variation around the mean.
• The upper and lower control limits are classically known as sigma limits.
• Another variation of a control chart to a run chart is the number of data points required. Typically a control chart needs around 15 data points (preferably 20) whereas a run chart can be made using a minimum of 10 data points. The reason being is the mean is more sensitive to point to point variation.

Which chart to use

Download our a flowchart to help you pick the appropriate chart (Click the image above)

Unlike a run chart, there are different types of control charts. Depending on the type of data you have – attributes (classification or counts) or variable (continuous); – and the purpose of analysis, different types of control charts should be used.

The most commonly used charts are:

An example of a typical control chart

Interpreting a control chart

The Shewhart chart provides a basis for taking action to improve a process (or system). A process is considered to be stable when there is random distribution of the plotted points within the limits. For a stable process, action should be directed at identifying the important causes of variation common to all the points. If the distribution (or pattern) of points is not random, the process is considered to be unstable and action should be taken to learn about the special causes of variation.

• The way you interpret a control chart is by placing zones on the chart
• There are three sigma limits above the centre line and there are three sigma limits below the centre line
• Most software will name these limits as C, B and A starting from the centre line.

So for the limits above the centre line, the first limit will be called C the next will be called B and the next will be called A (working upwards). Use the same technique for limits below the centre line working downwards.

• Now that we have these zones we can apply the ‘test for special causes’.
• This is being able to look at the data and ask “do I have special causes?” or “do I have random common cause variation?”.
• Typically data that vacillate between the upper and lower control limits is common cause variation. But there are ways to detect if you have special patterns in the data.

You can use the following rules to interpret the chart:

Please note: Limits A, B and C are not shown in rule 1,2 or 3 as they are not used. These “zones” are only used in rule 4 and 5.

• Rule 1 – 3 Sigma violation

When you have a data point that exceeds the UCL/LCL, that a demonstration of a special cause. That is data that has exceeded the upper estimation of variation of the process by the UCL/LCL.

Rule 1 – 3 sigma variation

• Rule 2 – Shift
  8 or more data points hanging above or below the centre line is a demonstration of a shift.

Rule 2 – shift

• Rule 3 – Trend

Six or more consecutive points all going up or all going down. If the value of two or more successive points is the same (repeats), ignore the like points when counting.

Please note: When Shewhart charts have varying limits (due to varying number of measurements within subgroups), Rule 3 should be used with caution. Theoretically, it is not correct but it still gives useful information in practice.

Rule 3 – Trend

• Rule 4 – 2/3
  This rule relates to how the data relay themselves within zones A, B and C. When you get two out of three consecutive points in zone A (outer one-third of chart), that’s an indication of a special cause.

Note: When there is not a lower or upper limit on one side of the centre line (for example, on a standard deviation chart with fewer than six measures in a subgroup or on a P chart with 100% as a possible result for the process), Rules 1 and 4 DO NOT apply to the side missing the limit.

Rule 4 – 2/3

• Rule 5 – >15
  15 or more data points hugging the centre line (between zones C). In a normal distribution, you should have around 60% of the data near the mean of the distribution (+/- 1 standard deviation). When you get a pattern like this, you’re exceeding the 68%.

Rule 5- >15

What does each rule tell us?

• Rule 1 quickly identifies sudden changes in the measure.
• Rule 2 identifies small, sustained changes (like small improvement to a process).
• Rule 3 detects a small, consistent drift in a process (trend).
• Rule 4 adds additional sensitivity to detect changes that have not yet triggered Rule 1 or Rule 2.

The formula to calcualte the control limits differ for each type of control chart so producing control charts requires specialist software (i.e. QI Charts for excel).

Additional resources

• Statistical process control as a tool for research and healthcare improvement – 
  J C Benneyan, R C Lloyd, P E Plsek

Science of Improvement on a Whiteboard, with Robert Lloyd, Vice President, Institute for Healthcare Improvement

PDSA

1. Useful Resources for Running Your Project

QI Life is your main resource for running your QI Project, all team members can contribute to project using this online workspace. You can plan your project with driver diagrams and develop change ideas, you can test ideas using PDSA (Plan Do Study Act) cycles and record all of your data within a system that automatically plots run and control charts for you.

Click here to access your account or sign up>>

2. Model for Improvement-Overview

Throughout your QI project you should use the Model for Improvement.

It is essentially a method for structuring and carrying out an improvement project. If you

are looking to carry out a QI project, we will help guide you to develop your idea and test it out using this simple framework.

The model consists of two parts. The first three questions help us define what we want to achieve, what ideas we think might make a difference, and what we’ll measure to help us understand if change is an improvement.  You will have already worked through this by completing your charter.

The second part is the PDSA (Plan Do Study Act) cycle – outlining the steps for the actual testing of the change ideas. The cyclical nature allows the change to be refined and improved through repeated cycles of testing and learning. This provides a vehicle for continuous improvement.

Watch these two short videos with Bob Lloyd from the IHI who explains more….

3. Model for Improvement – Part 1 – The Framework 

The first three questions of the Model for Improvement will effectively form the framework for your QI project.  If you have completed a charter, you will already have worked through these three questions:

 What are we trying to accomplish?

Take the quality issue you’ve identified and turning it into an aim.  Also think why your improvement project matters to patients and whether there is a business case for it.

How will we know that a change is an improvement?

This question focuses on one thing.  Measurement.  Measuring key parameters linked to your project will allow you to track improvement over time and will help you identify quality problems but also opportunities.  For every project we normally use between 5-8 outcome, process and balance measures.

What changes can we make that will result in improvement? 

To make an improvement you’ll need to make changes, but how do you start to identify potential changes?  One technique for attacking this problem is using driver diagrams.

 Please refer to starting your QI project for further information on all of the above.

4. Model for Improvement – Part 2 – The Engine

The second part of the Model for Improvement is effectively your engine for developing, testing and implementing changes.   This is carried out by using Plan, Do, Study, Act (PDSA) cycles.

Plan-What will happen if we try something different?

• What is our objective in this cycle?
• What questions do we want to ask and what are our predictions?
• Who will carry this out? (Who? When? How? Where?)

Do-Let’s try it!

• Carry out your plan
• Document any problems
• Begin data analysis

Study-Did it work?

• Complete data analysis
• Compare results to your predictions
• Summarise your results

Act-What’s Next?

• Ready to implement?
• Try something else?
• Next cycle?

PDSA cycles allow you to take change ideas you have created, try them in practice, learn what is or isn’t working with them and then adjust your approach.  It is rare to achieve absolute success through your first PDSA cycle.  Most commonly you will need to adjust your change idea through a number of PDSA cycles before it starts to work reliably in actual practice.  The important point to note is that failure is not the end and can be a useful thing!  By meeting on a regular basis as a team and going through PDSA cycles you will be doing something called rapid cycle testing.  This will allow you to see meaningful change within months that would otherwise take years.

5. Using Data for Improvement

In the early stages of your project you will already have identified a number of measures that you will want to use to show whether your project is starting to create improvement.

But how do you use this data you are collecting to show whether you are or aren’t improving?

In improvement methodology we collect small amounts of data but regularly and use things called run charts or control charts to look at how this varies over time.  Below is an example of a run chart.

Run or control charts are particularly focused on looking at one thing.  Variation.  It is important to understand that everything varies over time.  A run chart acts a bit like a camcorder, showing you every up and down.  Snapshot audits (that you may well have done before) are a bit more like a camera, taking a picture of what things look like at just one point in time.

To be able to show that things have improved we not only need to be able to show the things have changed, but also that this is not a one off.  In other words, the change has been sustained.    Run or control charts allow us to see if this has happened.

QI Life give allows you to enter your project data quickly and easily, it then automatically generates the run and control charts for you identify trends, shifts and other points of significance.

There are also many other tools you can use in your Quality Improvement work, as such we’ve built an Improvement Tools section of the microsite to help you navigate these.

If you prefer to use Excel you are also able to access some specialist software called QI Charts which will enable you to easily create run and control charts in Excel. Please visit these sections for further details.