Visual Estimation of Effort: RIP
Jan 12, 2012
The functional capacity evaluation (FCE) is the oft-overlooked segue to unwarranted diagnostic testing, extended and unnecessary treatment and indemnity awards that are not justified. This occurs for one simple reason: Standard testing methods do not accurately and objectively classify validity of effort. For example, there are many studies which have discredited the coefficient of variation (COV), Rapid Exchange Grip (REG) testing and the Bell Curve as indices of effort. For a list of those studies, go here: http://xrts.com/Studies that Discredit Standard Hand Strength Assessments of Validity of Effort.pdf. The aforementioned references are from peer reviewed journals, published over the last 26 years. Static (isometric) testing, according to the findings of a study of more than 130,000 subjects, does not accurately predict dynamic lifting capacity.[1] Furthermore, static testing systems are not capable of accurately classifying validity of effort. Static testing failed to identify at least 50% of the feigned weakness sessions in the first published controlled study on this subject—even though static testing has been used for this purpose for approximately 40 years.[2] In a recently-published study, feigned weakness was detected during isokinetic testing only 31% of the time, using a very sophisticated statistical analysis (abstract found here: http://www.jelectromyographykinesiology.com/article/S1050-6411%2811%2900132-5/abstract).[3] If the FCEs for your claims are using these methods, they are no longer legally defensible if challenged by an attorney who understands the research in the field of functional testing.
The sole remaining assessment typically performed during an FCE for the purpose of classifying validity of effort is a lifting evaluation. For many years, the “state of the art” has been the so-called “visual estimation of effort.” This approach is the basis for all of the most familiar “brand name” FCE protocols. These methods rely on observations of “movement patterns,” the degree of “involvement” or contraction of various muscle groups during a lifting activity and a series of other visual observations that are used for the alleged purpose of determining “how hard” a person is working or whether or not that person is cooperating.
When the published studies that have been used to promote the visual estimation of effort approach are examined critically, these things should be apparent:
- Visual estimation of effort protocols are only different versions of the same model, not distinctively different from one another. They use different terminology—sometimes—to describe what the evaluator reports observing during the lifting assessment. The biggest differences are in the brand names, the logos and the formatting of the reports.
- The observations that are said to be made during these protocols are not possible to make because they involve visually observing muscle groups that are completely, or at least substantially, concealed by clothing. In fact, one published study suggestions that visual observation of rhomboid contraction is evidence of “heavy lifting.”[4]. Unfortunately, it is not possible to see the rhomboids unless the patient is literally dissected because the rhomboids lie beneath the trapezius—and of course, the trapezius is beneath an article of clothing. In other words, the evaluators simply cannot see what they are supposedly watching during the test.
- Some of the observations used in such protocols pertain to “acceleration” or changes in “velocity.” Unless high tech equipment (camera systems, accelerometers, etc.) is used, such observations are really nothing more than “impressions” which may or may not be accurate. They are rough estimates at best—and never demonstrably “true.”
- None of the “operational definitions” (the descriptions of what the evaluators are observing, or said to be observing) have themselves been shown to be valid measures for the purpose for which they are used. For example, there is no research that addresses how much acceleration or velocity during the lifting event changes as a result of a lifting a heavier or lighter workload. How, then, is it even possible to use “acceleration” or “velocity”—which cannot be ascertained with the naked eye—as an indicator of effort or if it has not been shown to be related to “working hard” or “being cooperative?” Perhaps it is the same kind of technology which allows for observation of the rhomboids with the naked eye (http://www.xrts.com/High Tech Visualization Methods.jpg).
- Issues related to visual acuity, foveal vision and peripheral vision are also limiting factors to this methodology. Related to these issues is “inattentional blindness.” Dan Simons has studied this subject matter extensively and shown that we miss large, unexpected events about half the time when our attention is diverted while making the observations.[5] Simons’ videos, now classics in the field of cognitive psychology, are found here: http://dansimons.com/videos.html. (Simons’ findings, by the way, have implications not only for functional assessments but they are also being put to good use in improving workplace safety. His book, The Invisible Gorilla, is an insightful and very nice read that will appeal to many in the field of health and safety.)
In addition to the practical problems with making accurate observations, there is an even more fundamental reason that the visual estimation of effort should not be considered as an acceptable method of assessing clients who are involved in medical-legal cases. In medical-legal cases, evidence would ideally be objective—and in no stretch of the imagination can “observations” ever be considered as “objective.” Observations are judgments or impressions, nothing more. When challenged by an attorney who is educated on this issue, such observations become as easy to challenge as “eye witness” testimony in criminal cases. In fact, knowledgeable attorneys can deconstruct such protocols every time if they understand the methods used in standard protocols—even when the classification of effort is “right.” Why is this so? It is true because unsound, non-scientific methodology which has the “right answer” is no more supportable than the “right answers” provided by the person who controls the Ouija board.
To compound the difficulties with the visual estimation of effort approach, a recent study has reported these findings that risk managers may find of interest:
- Trained and experienced therapists, when tasked with watching videos to apply operational definitions to classify the relative difficulty of lifts performed by subjects who were lifting 25%, 50%, 75% and 100% of maximum lifting capacity, accuracy was approximately 40%. This accuracy rate only marginally exceeds chance. Notably, the untrained lay subjects in this study who watched the same videos had the same level of accuracy!
- Years of experience, education and number of tests performed by the trained and experienced therapists did not correlate with accuracy.[6]
The visual estimation of effort is simply not a scientific approach—the kind of careful approach that is required when the physical health of a worker and the financial health of an employer are on the line. Instead, it is an interpretive approach—a Ouija board approach, if you will, because it that puts the entire outcome of the FCE into the hands of the evaluator. Bad methods lead to inaccurate conclusions. Inaccurate conclusions lead to bad case management and treatment decisions. Bad decisions lead to poor outcomes. Poor outcomes result in higher losses for the employer and sometimes lasting harm to the employee. But this is what you get when a field has been allowed to use systems in which the results are “interpreted” instead of relying on measurements that can in some way be analyzed for consistency of effort. Improvements to testing systems should include:
- Repeated measures. This is common sense. Test but re-test. Trust but verify.
- Better methodology. We advocate the use of “distraction-based” testing principles. Distraction-based tests, as defined by Waddell may years ago, are “non-emotional, non-surprising and non-hurtful.” In other words, portions of the test are repeated in ways that are either not obvious, or they are done in a way that makes it difficult to “cheat.”
- Standardized statistical analysis of consistency of effort, applied the same way every time to every person that is tested.
An approach which includes these elements protects not only the employer, but also the employee. They offer the best and most direct approach to not only controlling costs, but also in potentially reforming our compensation systems to an extent that is not possible by legislative “reform” or various price control tactics.
For more information pertaining to standard testing methodologies and some recent—perhaps startling—findings, go here: http://xrts.com/FCEs_Legally Indefensible Methodologies.zip. For published studies pertaining to new methodology, follow this link: http://xrts.com/FCEs_Legally Defensible Methodologies.zip For information pertaining to ways to challenge adverse FCEs that are in your files, go here: http://xrts.com/Forensic Dissection_Second Edition_Redacted.pdf. The third link contains information in Chapter 13 regarding our record of litigation successes in helping attorneys challenge FCEs which have used poor methodology which has had an adverse relationship on the interests of their client.
It can be a new day in the field of claims management if risk managers are aware of recent advances in the field of functional testing and take the necessary steps to protect their client’s best interests. Absent an objective assessment of validity of effort, conclusions regarding “lost function” are nothing more than speculation which feeds the expensive expert witness culture and potentially leads to abuse, i.e. unfair treatment for the injured worker or the employer. In contrast, much of the contentious nature of litigated cases—and, therefore, many of the expenses—will be significantly controlled when both sides of compensable injury claims demand definitive and objective information pertaining to validity of effort.
References
1. Feeler L., St. James J.D., Schapmire D.W. (2010). Isometric strength assessment, Part I: static testing does not accurately predict dynamic lifting capacity. Work. 37(3), 301-308.
2. Townsend R., Schapmire D.W., St James J.D., Feeler L. (2010). Isometric strength assessment, Part II: static testing does not accurately classify validity of effort. Work. 37(4), 387-394.
3. Almosnino, S., Stevenson, J.M., Day, A.G., Bardan, D.D., Diaconescu E.D. & Dvir, Z. (2011). Differentiating between types and levels of isokinetic knee musculature efforts. Journal of Electromyography and Kinesiology, 21(6), 974-981.
4. Isernhagen, S,J., Hart, D.L. & Matheson, L.M. (1999). Reliability of independent observer judgments of level of lift effort in a kinesiophysical Functional Capacity Evaluation. Work, 12(2), 145-150.
5. Simons, D.J., Chabris, C.F. (1999). Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception, 28, 1059-1074.
6. Schapmire, D.W., St. James, J.D., Townsend, R. & Feeler, L. (2011). Accuracy of visual estimation in classifying effort during a lifting task. Work, 40(4), 445-457.
Other court decisions can be found here: http://xrts.com/Algonzino_Iowa Decision.pdf and here http://xrts.com/Knapp v. McKay.pdf. For more information, go to www.xrts.com. Direct communications to Darrell Schapmire, ds@xrts.com, or call 309-449-5483.
About The Author
Darrell Schapmire and James St. James are the co-developers of the X-RTS Hand Strength Assessment.
Mr. Schapmire has independently developed the X-RTS Lever Arm, a simple mechanical device used to classify validity of effort during a lifting assessment. He received his Masters Degree in exercise physiology in 1991 from Benedictine University, Lisle, IL.
Dr. James is a faculty member in the field of experimental psychology at Millikin University in Decatur, IL. These products are incorporated into the X-RTS FCE, a testing protocol which has recently been involved in a major court decision in Ilinois (http://xrts.com/Clewell.pdf).
