Polygraph and why we like research by Jarrod Rockwood
Among my favorite things about polygraph is that each test we do is like a baby scientific experiment. We have our subject. We expose them to different stimuli (questions) and record physiology which can then be subjected to numeric rules of interpretation. When we do this process well, we improve our ability to make credibility decisions by leaps and bounds. Human judgement of credibility is slightly better than chance (54%). If left to our own devices, without the polygraph equipment we might as well pull out an 8-Ball toy and ask it if someone is being honest. But with a polygraph we reach into the 80s and 90% range for assessing veracity. I believe that the scientific method can be used in our personal practice to inform decision making and to maximize our effectiveness.
I want to use an example that has been on my mind for some time. I use the example just because it is something I have thought about, measured, and analyzed (scientific method). To be clear I am not an academic and I know that there are people far superior to me that need to conduct the research in the field of polygraph. I appreciate and lean on these great minds each and every day. Yet, I would encourage us all to be curious, to ask questions, and to test our assumptions. But the trick is not to confirm what we already believe… but to challenge our beliefs. Learning comes from expanding the boundaries of what we know, not continuing in the “same old same old same old” because “that is what someone told me” in basic academy. I want to set the stage a little bit here with a classic psychological experiment that was conducted many years ago by a research named Dr. Peter Wason.
Here is the gist. People were given a short series of 3 numbers. They were then encouraged to make guesses as to the rules that guided the formulations of those three numbers by formulating additional 3 number series and getting feedback on their guess. Researches would then give dichotomous yes/no answers to the inquiries of the subjects. Let’s say that we have 3 subjects that all get to participate together and learn from one another (building from one another’s questions). For example: JD (subject) is introduced to the objective of the game (figure out the underlying rules that guide the number series) and is then given the numbers 2, 4, 6. JD is a smart guy and immediately catches on and says 8, 10, 12. Straight face researchers says “yes.” JD then takes another stab at it to test his hypothesis and says 30, 32, 34. Once again his approach is validated. JD goes on to formulate various versions of progressing even numbers +2 from one another. Each time the researcher confirms his approach is correct. At this point JD is totally confident. This test was silly.
The solution is so simple. It is just progressing series of even numbers, right? Well let’s try something new. Ted is given the same instructions and he follows JD’s path initially, but then he tweaks it just slightly. He asks about a series such as 10, 20, 30. The research indicates this pattern fits within the rules of the original progression. JD could have confirmed dozens of versions of even numbers increased by +2 and felt really confident about his conclusion; but in one fell swoop Ted has destroyed JD’s original theory. He is on to something else. He then tries something different to gain more information. He checks out a different series 20, 30, 10. The research informs him that does not fit the rules. Perfect, Ted’s hypothesis that it has to be even increasing numbers of equal distance must be correct. Then comes along Sally. Sally is given the same assignment but the first series she tried was 10, 11, 12. The research confirm this fits within the rules. What did we learn? Odd numbers can be used as well. She tries 11, 13, 15. Again, confirmed. So Sally is an out of the box thinker and she tries something totally off the cuff. What about 13, 28, 111. “Yes,” the researcher confirms, this fits the pattern. She tries 111, 28, 13. “No.” You get the idea. So what is the rule that Dr. Wason had developed? A list of numbers in ascending order. It does not matter the number, odd or even, distance between that number, etc. But it must have three numbers, and they all must ascend in order. Very few subjects were able to discover the underlying rules, why? Confirmation Bias- the tendency for people to reinforce preexisting beliefs. The mind has to manage a lot of information and one of the ways it has become efficient at its task is to filter all things into categories (schemas).
The primary rule of operation is that we confirm things we already think and believe. That is why two people can read the same article about a mass shooting and one person will walk away feeling like we need to ease up rules on concealed weapons so that we can protect ourselves and others from crazies; and another person reading the same article will walk away believing that all guns should be banned from general citizenry. The process of repeatedly making observations of the same phenomena is not sufficient to equate The Scientific Method. Karl Popper, believed that the core of The Scientific Method is Empiricism where we look to falsify our hypothesis in order to expand our knowledge using observation and measurement. That should be sufficient background to launch into a discussion about you and I as we sit in our polygraph suite. Much of the research that is done in the field is going to be done by academics that specialize in such. It is possible for us, as polygraph examiners, to learn key issues from academics that perhaps do not perform polygraphs on a day to day basis. For that matter we can potentially learn from research in other areas of credibility assessment (recent article from Eye Detect emphasizing the importance of not asking passive questions like did you lie in that questionnaire, in favor of direct questions about the area of concern like have you used any illicit substances in the last 5 yrs being a good example). So let me personalize this a bit to my own observations in my polygraph suite. I am a fairly new examiner. I completed my basic training in 2015. In that there were no polygraph schools in my state, I left my family and stayed with friends of friends in the area where the school was being held. In that I did not know anyone in the area and my natural inclination in new situations is to hyper analyze, I spent a lot of those three months reading polygraph articles, books (such as Fundamentals of Polygraph Practice, by Krapohl & Shaw which is a must read for all new examiners; and Credibility Assessment: scientific research and application by Raskin, Honts & Kircher). When I was in basic academy we were told (unverified hypothesis) we were the first class to ever receive instruction in Relative Line Length (RLL) for scoring pneumos as part of our primary training. We of course covered feature scoring as well, but most the focus was on RLL. My mentor had sent me charts to practice scoring (N-60). All of these charts had been scored by him previously. After scoring out the first 20 charts there were 4 examples of charts where I had scored out Inconclusive (IC) and he had scored out either a pass/fail (conclusive). That is 20% of that small sample of charts resulted in an IC, where my mentor had a definitive outcome. I found that to be disconcerting, and the only difference between the way we both scored the charts was the use of RLL vs Feature Scoring. So, I abandoned RLL and went on to score out the remaining 40 charts using feature scoring. In those additional 40 charts only 2 had the same issue where I had scored it out as IC and he had a conclusive decision. That is 5% of the larger sample.
A pattern was created in my mind. RLL tends to drift a chart in the direction of IC. I have repeated this opinion often when interacting with other examiners and felt strongly that this was FACT! But at the same time, I am generally a huge advocate for computer algorithms and lean on them in favor of hand scoring. This creates a huge internal conflict for me over how to make decisions, especially when there is cross talk between my hand score and the computer and whether to use RLL scoring algorithms or features. After attending a great Advance Academy training in Boise, where they gave some very convincing statistics on RLL, I decided to venture back into this arena with greater Empiricism. In place of a handful of charts, scored when I had literally just learned how to do it, I decided to start tracking a large swath of information for each chart I collected. Over the last year I have collected data on hundreds of my own charts.
The format was developed for multiple reasons, but I want to focus on RLL for the purposes of this conversation. I also need to add for this next section that I stumbled into a project that Ray Nelson is working on and I was able to coordinate with a handful of other examiners tracking the information above, so as I talk numbers for the next section this is cumulative of the various examiners that participated in that study. The overt purpose for my creating this format was to look at the effects of RLL on decisions. Remember, my initial assumption based on 20 charts back in basic academy was that RLL had a tendency to draw charts in the direction of IC. Here is what I found when using a much larger data set, multiple examiners, and more experience. I was WRONG! I have preached my perspective frequently with other examiners and as comments in trainings as I stood on my soap box talking about cautions of RLL. Here are the quick numbers on it. The group of us had collected a total of 483 charts. Of those 115 were feature scored and the vast majority were scored using RLL (368). Of those 368 charts 15% (56) were IC based on HCT. So inconclusive were relatively rare overall and slightly higher than what is reported in the MetaAnalytic Survey of polygraph research that reported roughly an 11% IC rate. I stated previously in my little sample of 20 charts there were 4 examples (20%) where the HCT created an inconclusive result when using RLL. It is problematic to make conclusions based of small samples because variations are overrepresented. It is important to gather as much data as possible. Research with a single examiner is also less valuable than multiple examiners because idiosyncrasies of the examiner may influence the numbers… but that being stated the point is not to publish formal research based on your individual experience, but to use the scientific method to analyze your process. What was found with this larger more methodical look at the dynamic of RLL pulling charts into IC showed that this happened 21% of the time. So at this point I can pat myself on the back for coming to the correct conclusion (like JD in my earlier example). But wait, when I expand my scope and look at it from another perspective (like Susan from my earlier example), there is something else that I can learn! 38% of all the inconclusive charts actually resolved using RLL. Almost 2x as many charts were resolved using RLL scoring than were created (my originally often preached bias). Should I be embarrassed about my previously held, and oft articulated, belief. I would argue “no.” Because we are all wrong about something. The danger is in not knowing what we are wrong about.
The scientific method gives us an empirical way to peer under the hood and check our perspectives and opinions. Those that are self-conscious and worried about their work product are not the examiners that make me nervous. It is the confident self-assured examiner that knows for sure every outcome is correct every time. I am not saying to be timid or not trust your charts, the odds are in your favor, but let’s be curious as we try to make decisions about how we do what we do. Just the other day I was at our local polygraph association semi-annual meeting. We were reviewing the most recent polygraph journal, scoring out some charts together, and connecting. During the discussion someone inquired as to the value of a photoplethysmograph (PPG/PLE) as a polygraph component. More specifically he wanted to know how often the PPG was able to resolve an IC chart. He was debating if it is worth investing financially to purchase one. The research on PPG is incontrovertible. It is a great asset in decision making, and is 2nd only to the EDA in providing diagnostic information regarding credibility. That is pretty exciting stuff. But the opinions in the room varied widely as to if and how valuable it is as a tool. One member in particular talked about his frustration with the discordant reactions between the EDA and the PPG. The assumption seemed to be (if I did not overgeneralize what he was trying to communicate) that they virtually always contradict one another. That is good questions. It sparks curiosity. Now what do you do with that curiosity. I would propose that a simple spread sheet that tracked positive versus negative reactions in all CQ/RQ positions comparing the PPG to the EDA (most diagnostic feature by far as beautifully described by Raymond Nelson in our last journal). If this examiner were to track that information for, let’s say, at least 100 charts. He would then have a wealth of information about his assumption. The numbers would easily tell the story. With a few quick calculations he would have a clear answer to the question.
That stated it is important to recognize to generalize the information it would be important to have a large sample from multiple examiners; otherwise it may just recreate the same confirmation bias demonstrated in the initial example. My invitation to all of us is that we be curious. We track areas that spark our interest. And we let the numbers tell the story. Don’t be embarrassed by errors, those are what make our theories strongest. The profession becomes better by individual examiners paying attention to their beliefs, testing them out, and adapting accordingly IN THEIR OWN POLYGRAPH SUITE! Professional researchers will always have a jump on us. We should pay attention to what they have to say. But we do not have to blindly follow. They learned the information through the same process that we can. So, let’s get out there and become micro scientific researchers of our own practice, looking to see where we can prove our opinions as incorrect with empirical analysis. Quite frankly, it can be exhilarating to learn something new based on your own efforts.
