Not My Job???

There is something I just don’t get about the whole contracting-out-data-management thingy. Sometimes it's done because of lack of resources, or lack of internal expertise, which is understandable and part of a flexible business model. Sometime, though, organizations look at data management and decide it is not a core competency, and that's why they outsource it or offshore it, or whatever the flavor du jour is, and that is the part I don't get. Many don't stay with only one CRO, and the studies are spread around. Often the protocol is tossed over the wall, and the CRO is expected to develop the database, cleaning rules, CRFs, etc., using their own approaches. Sure, the sponsors review the specs that the CROs produce, but either don't have the time or else the expertise in-house to assess them for relevance, completeness, and consistency with other vendors in the study, let alone poolability with other studies in the development program. If data are not consistently defined and captured, then pooling them may not be valid, and thus conclusions drawn from the data may not be valid.

Data management is about the design of the data, about its definition and assurance of its quality, with quality meaning "fit for its intended purposes." It’s about knowing how to capture data efficiently and accurately, and in accordance with the word and spirit of the protocol. Without the data, the study drug/biologic/etc is just another pile of white powder (or whatever) sitting in a jar. The data are what tell the story of the safety and the efficacy of the treatment. The data are the basis for the label, for marketing, for answering questions from the FDA and the public, and for finding new potential indications. The data are the foundation of the whole thing.

So help me understand.

What is it about this whole process that
ISN’T COMPLETELY ABOUT THE DATA??????

How on earth can data management be a non-core competency!?

Photo: the world at night, from orbit. NASA website.

Can You Create Well-Designed Consistent CRFs for the Site?

It's all about perspective...

The title question was posed at the recent SCDM Data Quality webinar. Most respondents answered “yes”. My answer is “it depends”. From a given sponsor’s perspective, the answer is “yes”, but from a site’s point of view, it is definitely “no”. Sites may never do more than one study with any given sponsor, so from their perspective, the CRFs (or EDC system) is never seen again. Each set has slightly (or very) different questions, with different answers, grouped on different pages, and with different completion instructions. Visits have different names, forms for non-completed visits may or may not have to be returned to the sponsor, and entry edit checks fire for different reasons. If sites don’t keep these rules straight, we think they produce poor quality data.

So what is the solution? The CDISC CDASH project will resolve some of these dilemmas. It identifies the minimum set of data fields for most common study designs, along with CRF completion guidelines. Many fields are linked to standard terminology, ensuring that code lists are consistent. This goes encourages similar content, and that will help the sites, but a major obstacle remains. On the whole, each company’s data management group believes that it has the best answer to each of these challenges; it has developed the best and highest quality solutions, practices and procedures, and few have any interest in changing. If one assumes that most have had submissions accepted then one of two possibilities must be true. They are either all “good enough” for the purposes of the development project, or all the QC, QA, audits and oversight are so sloppy that they fail to detect the flaws in these processes. Granted, there are differences required by some study designs, indications, and drugs/biologics vs devices, but my experience suggests that these are very minor.

That leads to the inevitable conclusion that these variations are a matter of preference, and do not impact quality. Some will be more efficient, precise, or suited to habit, but they achieve the same result. Think about what happened the last time you were shifted to another project in your organization. Chances are that you had to learn new rules, and until you learned them, you were more likely to make mistakes. There is no reason to believe it is any different for the sites.

Why, then, can’t we agree upon common practices and rules and approaches for these common activities? Are we so convinced of our own superiority that we refuse to change? Are we afraid that others will steal our good process ideas and get to market first? Are we just “used to doing it that way” or have “always done it that way” or believe that “regulations or Biometrics or Clinical require that we do it that way”? In other words, if it ain’t broke, don’t fix it? Well, I argue that it is “broke”. It is “broke” for the sites, and if it ain’t fixed we’ll continue to lose investigative sites, and pour ever more resources into trying to inspect quality into the data, and miss the opportunity to maximize the CDASH revolution.

So what do you think? Do you agree? Disagree? Want to throw turnips? Should life be more consistent for the sites at the expense of our processes? We’re a pretty inventive bunch – I bet we could find ways to be efficient while collaborating with the sites to improve consistency.

I’ve started a list of practices that I think could be harmonized. Do you agree with the list? What can you add? What shouldn’t be there? How do you approach these activities and why? Are your reasons based in concrete need or historical habit or the belief that someone else internally “won’t like it”? How can we create a sponsor/site forum that would be trusted by both groups? Who might have to collaborate internally and externally to make this happen? Let’s talk!

1. When a subject terminates early, do CRFs for all visits after termination have to be returned to Data Management? (applies to paper only, I assume)
2. Do sites complete new AE and Con Meds forms for each subject at each visit, or are existing forms updated? i.e., are AEs and Con Meds info captured in a visit-based style or a log-based style?
3. How should visits be referenced? Can they be standardized to 1, 2, 3, etc., with a possible variation for course-based studies (e.g., oncology, although that could probably be Visit 1, 2, 3 etc within each course)?
4. When should we start capturing AEs and SAEs? Informed consent? Start of treatment (technically it can’t be an AE if treatment hasn’t started)? SAEs at informed consent and all AEs at treatment start? Should it depend upon whether the study requires potentially harmful screening procedures? On something else?
5. Once the CDASH data fields have been finalized, can we agree on a consistent layout? What should it be for each of the domains?
6. For Adverse Events forms, should the layout be portrait with one AE per page or screen, or landscape with multiple lines per page or screen? What are the pros and cons of each? Note that this is not talking about how they are stored in the database – just how the site would see them.
7. Should monitoring guidelines be given to the sites? Put in the site’s study manual? Included in the EDC application? If not, why not?
8. Should the list of edit checks be given to the sites? If not, why not? Don’t we want the sites to understand what the data should look like? If so, how can they be presented so that they are accessible and understandable?

Each of these, and many more, can be discussions in their own right, and the answers to each depend upon any number of assumptions about the underlying processes, but surely, if we really want to, we can make this work. I look forward to your comments.

Photo: Romulus, Ann Arbor, Michigan. c. 2008, Kit Howard.

Who Are You?

The other day I heard the old song by The Who “Who Are You”. It has been a while, and I can’t follow most of the lyrics on the track, but it doesn’t really matter. What struck me, as I was thinking about our business, is “Who Are You?” We each have a number of roles we play.

We are Pharmaceutical Professionals. We might be a monitor, or clinical scientist, or programmer, or data manager, or any one of a plethora of other roles. In those roles we look at scientific puzzles, and try to ask the best questions to clarify the unknown to formulate the quickest reasonably concrete result. In that world we have control – we are the scientists, the providers of answers, the managers of data, the spinners of results. We tend to think of our day-to-day existence as a set of tasks. This database must be closed. That report must get written. The other article must get published so the following ad will appear “true”. This is how we earn our living, how we pay our mortgages, how we pass the time...

We have other roles as well. Roles that only surface when things go wrong. Our parents become ill, and suddenly we are trying to find out whether the medication or combination of medications prescribed for them are likely to help and/or have intolerable side effects. Where can we find the information? Some of it is on WebMD, some in the Merck manual, some perhaps in our own company data banks, some we get from colleagues in other companies. We discover that the information is fragmented, hard to come by, spun to minimize the negative and maximize the positive.

There is often no way to tell what subsets of patients are most likely to benefit, either because the information is not published, or it is published in scientific journals not available to the general public, or it is buried in reports so dense that x-rays could not penetrate them. Or maybe the information simply doesn’t exist – no one has ever done a study to see if that particular combination of medications in that particular combination of co-morbid conditions is beneficial or harmful or ineffectual…

So where can we turn? The pharma companies, our employers, don’t have any incentive to do these studies. The government bodies such as the NIH have their own research agendas. Who can tell us? In fact, given that there is an almost infinite number of possible combinations of medications and co-morbid conditions, can we ever hope to have any solid evidence? Maybe we have to make do with fragmentary data extrapolated from studies that don’t really reflect our situation in an environment where the optimal treatment may be unidentifiable and in any case would not be covered by insurance.

Then, eventually, we become the patients. Not subjects, for those are the ones participating in the trials that may or may not provide any benefit for them as individuals. The patients. Ill, disabled, frightened, tired, without the energy or mental strength to do the research to make sure that the treatments we are prescribed (by the doctor who may have graduated bottom of the class) are not going to worsen some other condition we have, or perhaps are just the usual run-of-the-mill treatments that don’t incorporate the latest findings.

As we get older, more and more of us join the ranks of the patients. Then we retire, and find that not only are we the patients, we are also the marginalized elderly, those who don’t work, who don’t contribute, who are seen as a burden, whose “entitlements” take an ever larger chunk of the federal government’s budget. As the baby boomers grow older, and the generation following is dramatically smaller, who will pay for our aging?

It is very clear to me that we have no choice. We have a moral responsibility to our elders, to ourselves and to our children to fix this system. We have to make drug discovery, development and marketing not only faster and less costly, but even more importantly, dramatically more transparent and more driven by the needs of the patients. It may not be immediately apparent how this can be compatible with corporate goals, but I submit that if we do not find a way to satisfy both imperatives, we as employees, we as caregivers and we as patients will be among those who suffer the consequences.