In archaeology there is a popular phrase that says that “Absence of evidence is not evidence of absence.” In essence, it means that the fact that nothing was found does not mean that nothing exists. It merely means that it wasn’t found. For example, let’s assume that I conduct an archaeological dig in my back garden in Michigan and I find no indication that there was a Potawatomi settlement there. There are those who would then claim that these Native Americans did not live here, whereas all that can legitimately be stated is that I did not find any evidence of them. In fact, given the topography and location, it’s quite likely that there was at least a seasonal camp nearby, but based on my research I cannot make an accurate statement one way or the other.A similar issue exists in drug development. There has been a long-standing dispute between Clinical Data Management and most of the other functional areas involved in clinical trials that is centered around the use of “Not Done” and “None” data fields. A “None” box is completed on the Case Report Form (CRF) when there were no findings of a particular sort for a study subject. This is typically used to record that no adverse events were observed, or that no concomitant medications were taken. Similarly, the “Not Done” box is checked when a test or evaluation was not done. This avoids leaving a blank on the CRF, which would usually be queried by the monitor or the data manager.
There has been significant resistance to using these data fields. They are administrative fields, in that they do not contain data that are analyzed, and given that every additional field collected increases the costs of the trial, they would seem to be good candidates to drop. In addition, the FDA has typically said that they want to see only the data that represent that something happened, and not lists of “Not Done” and “None” fields. The CDISC submission standards do not define “Not Done” fields, so organizations that are modeling their entire data stream on CDISC have nowhere to put these fields. Finally, the fields create opportunities for superfluous data queries when they have been marked but there are also data values present.
All of these are good reasons for dropping “None” and “Not Done,” but they are all trumped by the fact that if “None” and “Not Done” are excluded, we cannot say if the tests were not performed, or if they were performed but the site did not complete the CRF. These are obviously two very different situations. For example, an ECG CRF that records no findings will mean one thing if the test results were not transferred from the source documents, something very different if the test was not done, and something different again if the test was done and there were no abnormal findings. In the first and third cases, we can retrieve the results and know whether or not there were significant findings. In the second case, we can’t know one way or the other.
It is important to note that there is no reason why the presence or absence of “Not Done” and “None” should affect the structure of the submission, nor the data presented in tabulations and listings. Subjects who had no findings and those whose evaluations were not done can easily be suppressed from the display. The net effect is to increase the reliability of the displays, as reviewers can be certain that blanks and absent records indicate that the data are really not available, rather than just omitted. We can then be certain that, for the purposes of analysis and submission, the absence of data really is evidence of absence.
Photo courtesy of Archaeology Magazine, Archaeological Institute of America
