Improving Our Forecasting: Smooth It and Scale It!

Patient Safety Quality Monthly

• Email|
• Print

February 20, 2010

Boy did it snow! Big storms all up and down the East Coast caused organizations and even the whole government to move into reaction mode. The weather reports gave enough early warning so that airlines could cancel flights and reposition planes, but it still was a mess—especially if you were traveling that week. Although this was a big event that ultimately had a lot of impact, there was enough warning to do a lot of risk mitigation.

The next week another storm headed for New England, and the weather reporters were forecasting 10–12 inches of snow in some areas. Schools closed, activities were canceled, and work let out early. But the storm didn't materialize. In this case, there was a lot of action, but did it really mitigate risk? Could it have actually caused more risk?

Forecasting the risk of snowstorms is a tough job—typically one that we put a lot of faith in even though it is "just a forecast." We may not recognize it, but we are also in the forecasting business. Although we aren't forecasting the weather, aren't we forecasting falls, medication errors, harm events, mortality, quality of care, potentially compensable events, as well as future satisfaction? In our world of patient safety and quality, people are putting a lot of faith in our forecasts.

What can we learn from the recent snowstorms that can help use do better forecasting?

Think about the difference between a discrete event and a trend

Chicago and Buffalo were slightly amused at the impact the snow had on Washington, DC. Why? Because for these two cities, this kind of storm was a known part of the trend that they were used to. In Washington, DC it was an unusual, emergent event. When we are alerted to a potential emergent event that is outside of our trend, we move into reaction mode. Depending on how big the alert is and how well managed our organization is, this response may range from prepared to knee jerk.

Now think about your event or complication data that you present to your leadership team. Does it show an impressive spike? That communicates an "emergent problem" or "storm on the horizon" and can move your organization into reaction mode, which may not be completely appropriate and may divert resources from other more important areas.

Typically this happens for two reasons:

• First, we probably didn't smooth our data and provide a longer term trend for comparison. If, rather than seeing an apparent spike in the forecast (actually the "hindcast") we see a longer term direction of movement and the rate of change, this can help us focus on the more realistic longer term trend and make better decisions.
• Second, we may have been careless in the selection of our scale. I am sure you have seen the graphs with the "jaggies" and perhaps even a Mt. Everest. Often they are purely artifacts of the automatic scaling function of our software. If you make the scale small, more snowstorms look like "the big one." Big visual spikes can cause big actions, even when the data is not big.

Smooth It and Scale It

So what can we do to help make our forecasts more effective? Consider making these two techniques important tools in your Patient Safety & Quality toolkit.

1. Always smooth your data. Consider using a rolling average that is based on the number of periods of data that you are sharing. Set the expectation that people will start saying, "Can we drop a smoothed line on that?" This is easy to do in Excel or other packages. Remember that a smoothed average may shift your curve, so do a quick adjustment for that if necessary.
2. Base your scale on your acceptable limits, not the magnitude of your data sample. Consider always putting your acceptable limit on the graph and basing your scale on it. If your data is unacceptably outside of the limit, go ahead and expand the scale, but if your data is just a faint vibration and there is a lot of range between the data and the acceptable limit, avoid the temptation to expand the scale "just so we can see it better." That turns those little bumps into big snowstorms and can elicit the wrong reaction from the end user of your data. If you continue to have too much room between your actual data and your limit, you may need to move the limit by raising the bar.

Just like we watch the snow forecast to decide how we will plan our day, people in our organizations look at our Patient Safety and Quality data and try to make good decisions from it. Help them out: SMOOTH IT and SCALE IT!

Question of the Month | Expert Advice & Guidance

•  I've heard that medication containers have to be labeled in areas other than procedural locations or the operating room. Is this true?

• MM.05.01.09 requires medications to be labeled whenever they are prepared but not administered immediately. The term "administered ... Read More...