What a great article! I think you have a very interesting suggestion for addressing the Accuracy problem – if I may paraphrase, simply normalize it within each lab. But I think there is a bigger problem here that I don’t see much discussion on, probably because the data is lacking to do any real analysis, but the problem is real. And that problem is Precision.

You mention a couple of times how the labs’ results can be Precise, but not necessarily accurate. For those that don’t know the difference, here is a link to a graphic that pretty clearly lays it out:

https://www.google.com/imgres?imgurl=http://cdn.antarcticglaciers.org/wp-content/uploads/2013/11/precision_accuracy.png&imgrefurl=http://www.antarcticglaciers.org/glacial-geology/dating-glacial-sediments-2/precision-and-accuracy-glacial-geology/&h=1363&w=2040&tbnid=d92uiG4-wAF2aM:&tbnh=140&tbnw=211&usg=__5_SC4MoAraOOU7FSsHT6e0OAS2k=&vet=10ahUKEwjSw9iUx-vTAhUXwWMKHXvGDmgQ9QEIKzAA..i&docid=DP3vEoGBlFj3NM&sa=X&ved=0ahUKEwjSw9iUx-vTAhUXwWMKHXvGDmgQ9QEIKzAA

Simplifying a bit, Precision is repeat-ability. It is a measure of how consistent the results are, as opposed to whether they are actually on the mark or not. Suppose we have a sample that we somehow know is exactly 20%, and we get 5 samples tested, and the results are 14%, 15%, 15%, 16%, and 15%. These results are not very Accurate (they’re off by roughly 25% of the known value), but they ARE fairly precise, repeatable, and consistent, differing by +/- 1% from each other.

The degree of Precision is usually expressed in terms of digits, or orders of magnitude. For example, most college statistics classes will teach that 15% (two digits) is less precise than 15.0% (three digits), which in turn is less precise than 15.00% (four digits). Why is that? The lower number of digits implies a greater variability, which means less measured repeat-ability. For example, contrast the numbers above with a different sample set that tested at 15.01%, 15.00%, 15.00%, 14.99%, and 15.01%. The first data set is 15% +/- 1%, while the second data set is 15% +/- 0.01%. They’re both 15% (not very Accurate), but the second data set is 100x more Precise.

I would argue this is what our labs are missing (both across labs and within labs). Unfortunately, we may not be capturing the data that is necessary to examine Precision, and the fact that we see reporting happening with 4 digits of Precision by labs that don’t seem to be able to reliably re-produce results with more than 1 digit of Precision seems to me to be the bigger problem.

Please give some thought as to how we might be able to examine repeat-ability and Precision both within a lab, and across labs. I’m guessing that it would involve collecting a lot more data than we do today, so that we could somehow know that any given group of samples came from the same “thing”, and thus *should* be repeatable, as opposed to from different “things” that rightly ought to be different. Then we could group those samples, and take a standard deviation, or something like that.

But until we have repeat-ability, and degrees of Precision that match our reporting, I’m not sure how much Accuracy really matters. If we have two samples from the exact same flower, same location, same timing, same lab, and they test at 13% and 27%, how reliable are either of those numbers? Even normalizing won’t fix this.

Thank you Nick, this is amazing feedback the details of which really improve the suggestion considerably! I’m absolutely in concurrence and would add what you say here as canonical improvements to the “silver bullet”!

In fact, this paradigm would encourage unscrupulous labs to withhold results that are abnormally high (quite the reverse of their current temptations), as high values would have the effect of skewing all other results to be lower. To cheat under this paradigm, the lab would have to test a whole bunch of low-THC trim and call it flower in order to skew the rest of their results up. Doesn’t make cheating impossible, but would make it much, much more difficult. And – of course – this correction method would only work well with flower, where the population of samples is roughly the same between labs. Extracts vary quite widely in concentration between processors and between extraction methods and even between runs, so extracts would be more difficult to pin down this way.

I’ll bring it up with the packaging and labeling committee at the LCB. I fear I already know the LCB’s response: “traceability system constraint.” Their software isn’t ready for such complexity as this. Keep up the good work. This issue is a huge burden to the industry and a solution is desperately needed. It’s a black eye for all of us.