On nonstandardization

Years ago I spent time in the US Navy as a Seabee. I had a job as an equipment operator (EO). Driving different heavy equipment seemed easy enough. See me in the picture below with my rough-terrain forklift.

Figure 1 Your favorite blogger in younger years serving his country

Figure 1. Your favorite blogger, in younger years, serving his country

At times, I got to hang out with my Seabee buddies during construction tasks. A friend of mine became a construction electrician (CE). At a work site I remember asking him why it took so long to do the wiring (I wanted to get back to our barracks). He made some remark about electrical currents, materials, and standards. My brilliant remark? “Standards, who needs them?”

Funny how education changes us! If present-day, older Rick could meet past, younger Rick I might try and talk some sense into him about standards. But then again I might just tell him to invent Google or Facebook, or maybe Googlebook.

Standards

Over time I have come to appreciate standards like nobody’s business. Our modern society owes so much to standards that I can hardly do justice to the topic in one blog post.

Standardization describes the process of employing world-class specifications that govern the construction and delivery of services, systems, and products (International Organization for Standardization, 2014). This process results in a standard, or “…an agreed-upon way of doing something” (Spivak & Brenner, 2001, p.1).

Standards occur in industry and commerce, health care, and almost every mature scientific discipline. Standards have brought us, literally, from the dark age to the digital age. Everywhere you look you can see the power of standards. Examples:

The smart phone we can’t live without has a standard operating system that allows app makers to construct everything from Angry Birds to Lose It!

The houses in which we live and the buildings in which we shop and work came into existence through a series of standards: (1) construction workers using materials born of adhesive standards, cement and concrete standards; (2) builders following masonry standards, roofing standards, wood standards; and (3) inspections to ensure the structure met fire standards, insulation standards, and engineering standards.

When we visit medical professionals they use standards to determine if our vision, hearing, heart rate, and temperature indicate health or that we need help.

The list could go on and on. As stated by Thomas, the then President of American Society for Testing and Materials: “Standardization is indispensable to life in this century. It is virtually as indispensable as the air we breathe. And like the air we breathe, it is invisible to all except its technicians” (Spivak & Brenner, 2001, p.v).

Complete lack of standard in graphs

Take a moment and look at time series graphs from any education, psychology, sociology, economics, or other social science journal.

The line graphs will show change across time. The line graphs will also shine in the dull light of nonstandardization.

Take the example of two images I pulled from a quick Internet search. No doubt you have already noticed both graphs vary in construction. But Rick pulled those from the Internet, flagship journals surely wouldn’t fall prey to nonstandardization. They do. Every single one of them.

Figure 2 Two graphs

Figure 2. Two examples of nonstandard linear graphs (NSLGs)

What difference do you see? The size of each graph differs, the proportions of vertical to horizontal axes vary, the data points diverge, and the scaling deviates from one graph to the next (note the top graph uses sessions for a time unit on the horizontal axis, a major no-no for time series graphics).

Why should it matter? Why should any of us care that almost everyone uses nonstandard linear graphs (NSLG)? Let’s answer the question with another set of questions.

Would we care if our smart phones used nonstandard operating systems? Yes, because nonstandardization affects all the gaming, productively, and photograph apps we like.

Should we care if our buildings fell under the domain of nonstandardization? Yes! They might fall over in high winds, bombard us with noxious gases due to improper building materials, or have roofs that can’t keep the water out when it rains.

And what if our health care professionals employ nonstandardization with the devices they use to measure different health indicators ? If we don’t care about quality health care service then yes, let’s embrace the lack of standards and let every single health care device maker rig their own technical specs.

We use graphs to make decisions, in some cases high stake decisions. Professionals and nonprofessionals alike use time series graphs such as the line graph to detect subtle and dramatic changes in a measured quantity across time.

Having NSLGs serve as the basis of our main decision making/evaluative tool comes with its hazards. For this blogpost, I will not lay out all the inherent limitations with a linear graph, they exist and many have pointed out their informational shortcomings (shameless plug: The Precision Teaching Book). But I will discuss two problems with nonstandardization.

NSLG changes

Two basic problems *always* exist with graphs that live in the land of nonstandardization:

1. Slope changes based on the size and proportions of the graph
2. Variability changes based on the size and proportions of the graph

The picture below (Taken from Kubina, Kostewicz, Brennan, & King, 2014) illustrates the differences in slope changes. Each NSLG has the same data, but the axes underwent manipulation (by the way, you find textbooks that encourage graph makers to play with the data and see what looks best – Oy vey!).

Figure 3 slope changes

Figure 3. Graphs with the same data but different slopes due to changed axis sizes

Look at the difference in the slopes. Ask yourself, do you think a person would evaluate how fast the data changed based on the two slopes? Yes, yes they would. The slope in the first graph appears to rise more steeply than the second graph. Logic compels the graph reader to conclude the data changed more quickly in graph 1 than graph 2. The data haven’t changed but we have two different conclusions!

Variability suffers the fate as slopes when nonstandardization rules the roost. Take a look at the graph below.

Figure 4 variability changes

Figure 4. Graphs with the same data but different variability due to changed axis sizes

Notice the great degree of variability in graph 1 and the smaller variability envelope in graph 2. The graph readers judge high variability in one graph but interprets the other graph’s variability as moderate. As Charlie Brown would say…

Figure 5. Charlie Brown after he looked at a NSLG

Figure 5. Charlie Brown after he looked at a NSLG

Variability indicates how much control exists in a condition. For example, in a condition in which a teacher implemented an intervention, a high degree of variability points to weak control of the intervention (because the data bounce all around; the less regularity in the occurrence of behavior the less influence an intervention has exerted).

Conclusion

What do we think about nonstandardization? Not good but it does do something. A house made with nonstandard materials will stand but may have hidden, and potentially catastrophic failings. Apps made with nonstandard operating systems will have very limited appeal, market penetration does not follow without everything working on the same platform.

How well should we trust nonstandard linear graphs? Certainly the NSLG tells us something but at a price. Like the examples mentioned previously, nonstandardization can contain concealed, nasty surprises (How will we see the real slope line and variability envelope?).

Should members of scientific communities like psychology and education continue to embrace and celebrate nonstandard linear graphs? Should high stakes decisions that affect the lives of school-age children or adults with severe behavior problems continue to occur on nonstandard linear graphs? Or, if you graph your own behavior don’t you deserve better?

References

International Organization for Standardization (2014, June 4). What are standards? Retrieved from http://www.iso.org/iso/about/discover-iso_meet-iso/about.htm

Kubina, R. M., Kostewicz, D. E., Brennan, K. M., & King, S. A. (2014). A Critical Review of Line Graphs in Behavior Analytic Journals. Submitted for publication.

Spivak, S. M., & Brenner, F. C. (2001). Standardization essentials: Principles and practice. New York: Marcel Dekker.

This entry was posted in Uncategorized. Bookmark the permalink.

6 Responses to On nonstandardization

  1. Janet Dolan says:

    Posting this in the Academy as we speak. I recently attended a workshop where the 5 others at my table complained about standards and the lack of creativity allowed in their school. I find I am more free to be creative once the standards are met. Thank you for a great post, Rick

    • RickKubina says:

      Great comment Janet! Though perhaps the standards your peers were talking about are “debatable standards.” We understand the tolerance of certain materials and when we apply a standard to them we fully understand the pressure that material can take. But in education, states are coming up with standards by having groups of people convene and based upon their opinions they lay out educational standards. Those standards are far cry our the standards we have in Precision Teaching!

  2. Mandey McFarland says:

    Hello Dr. Kubina,

    I’m new to the SClistserv, and a recent graduate with my Master’s of Education in Autism and Applied Behavior Analysis. Just wanted to say that I think this was a very informative and helpful post. As someone early in my career and working in a location that is behind the times in regard to behavior analysis and education, this was a great reminder that when I approach the topic of graphing, I should teach this from the start. Thank you for sharing!

    • RickKubina says:

      Thanks Mandey! Because you are starting out and are new the whole world of science is ahead of you! Glad you found Precision Teaching, you will find it the best system we have for measuring individual behavior.

      Graphing is so critical to what we do. It always surprises me when people in our field blatantly dismiss the actual medium we use for visual analysis (nonstandard linear graphs). It’s like astronomers not caring about how far they can see with their telescope. All we can do is point out the facts and hope our good colleagues see the light (hey that’s a nice astronomy pun 😉

  3. Ian Spence says:

    If we have standards, we can compare outcomes across programs and interventions. And if we have standards in charting our results, we can compare without using uber-measures (like State tests, and statistical analyses,) which always hide the important subtleties.

    And if the standards are applied to charting our outcomes, each behavioral intervention can become a “test case” from which we can learn something about how individuals learn best – and just maybe the child, teacher, manager, can make an impact in changing how schools teach.

  4. RickKubina says:

    Ian, wonderful! Great comment.

Leave a Reply

Your email address will not be published. Required fields are marked *