For those of you reasonably familiar with my fairly wordy style of writing… Hmm. That doesn’t do it. It’s usually good to learn new things, and recently I was made aware that… No, I don’t think that’s it either. OK. Here goes. I’m practicing being less wordy!

That was hard! My philosophy has always been “write it down, then edit to 50% of the size.” While editing is still good, I don’t have the bandwidth for that much rewriting, so I’m trying something new. This blog will still be a space for observations and rants, but my goal is to practice getting to the point quickly. Maybe in time it will become a habit! So here goes.

Recently I learned about OpenClinica, a free open-source clinical trials software system from Akaza Research. It apparently has been used in many trials by government, academia and in the industry. They join a growing trend of organizations no longer willing to accept industry’s rules (excuses?) of high costs and selective pipelines where treatments are developed only if they match a marketing strategy. Organizations like OneWorld Health, founded by Victoria Hale (one of my very few heros), a non-profit drug company that develops treatments for third world diseases. Or like the Parkinson’s Disease Society, that funds research to the tune of £4M per year.

Granted, these organizations have an advantage. They often receive their compounds from companies that have found them and don’t want to develop them, so much of the cost and risk of the initial research is mitigated. They generally don’t have shareholders demanding high profits. But they also don’t expect 7, 8 or 9 figure salaries, and it seems that their primary goal is treat patients, whereas much of industry seems to regard that as second to profits and personal wealth.

Most of us want to do the right thing, though, and one way is reconsider what we keep confidential and what we share. No, not chemical formulae or marketing strategies! I’m talking about clinical trials processes. How we develop our protocols, and SAPs, and CRFs. How we recruit our subject, monitor our data, resolve our queries. What we query. How we pass data between functional areas. There are efficient and inefficient ways to do these things, and we’ve been reinventing them for 20 years as new companies start and existing companies reorganize. It’s time for everyone to be open about how they do these things, and let the best approaches bubble to the top. And don’t tell me this is part of competitive advantage – no one out there is getting drugs to market dramatically faster than anyone else!!! Here’s a radical concept – let’s compete on the merits of the treatment or device itself!

Photo: St. Johns Wort, Auckland Botanical Gardens, Auckland, New Zealand. c. 2008, Kit Howard.

An Industry Evolving. Or not.

An updated version of the entry that was here can be found at http://www.clinpage.com/article/a_deep_process_problem/C12

Not My Job???

There is something I just don’t get about the whole contracting-out-data-management thingy. Sometimes it's done because of lack of resources, or lack of internal expertise, which is understandable and part of a flexible business model. Sometime, though, organizations look at data management and decide it is not a core competency, and that's why they outsource it or offshore it, or whatever the flavor du jour is, and that is the part I don't get. Many don't stay with only one CRO, and the studies are spread around. Often the protocol is tossed over the wall, and the CRO is expected to develop the database, cleaning rules, CRFs, etc., using their own approaches. Sure, the sponsors review the specs that the CROs produce, but either don't have the time or else the expertise in-house to assess them for relevance, completeness, and consistency with other vendors in the study, let alone poolability with other studies in the development program. If data are not consistently defined and captured, then pooling them may not be valid, and thus conclusions drawn from the data may not be valid.

Data management is about the design of the data, about its definition and assurance of its quality, with quality meaning "fit for its intended purposes." It’s about knowing how to capture data efficiently and accurately, and in accordance with the word and spirit of the protocol. Without the data, the study drug/biologic/etc is just another pile of white powder (or whatever) sitting in a jar. The data are what tell the story of the safety and the efficacy of the treatment. The data are the basis for the label, for marketing, for answering questions from the FDA and the public, and for finding new potential indications. The data are the foundation of the whole thing.

So help me understand.

What is it about this whole process that
ISN’T COMPLETELY ABOUT THE DATA??????

How on earth can data management be a non-core competency!?

Photo: the world at night, from orbit. NASA website.

Can You Create Well-Designed Consistent CRFs for the Site?

It's all about perspective...

The title question was posed at the recent SCDM Data Quality webinar. Most respondents answered “yes”. My answer is “it depends”. From a given sponsor’s perspective, the answer is “yes”, but from a site’s point of view, it is definitely “no”. Sites may never do more than one study with any given sponsor, so from their perspective, the CRFs (or EDC system) is never seen again. Each set has slightly (or very) different questions, with different answers, grouped on different pages, and with different completion instructions. Visits have different names, forms for non-completed visits may or may not have to be returned to the sponsor, and entry edit checks fire for different reasons. If sites don’t keep these rules straight, we think they produce poor quality data.

So what is the solution? The CDISC CDASH project will resolve some of these dilemmas. It identifies the minimum set of data fields for most common study designs, along with CRF completion guidelines. Many fields are linked to standard terminology, ensuring that code lists are consistent. This goes encourages similar content, and that will help the sites, but a major obstacle remains. On the whole, each company’s data management group believes that it has the best answer to each of these challenges; it has developed the best and highest quality solutions, practices and procedures, and few have any interest in changing. If one assumes that most have had submissions accepted then one of two possibilities must be true. They are either all “good enough” for the purposes of the development project, or all the QC, QA, audits and oversight are so sloppy that they fail to detect the flaws in these processes. Granted, there are differences required by some study designs, indications, and drugs/biologics vs devices, but my experience suggests that these are very minor.

That leads to the inevitable conclusion that these variations are a matter of preference, and do not impact quality. Some will be more efficient, precise, or suited to habit, but they achieve the same result. Think about what happened the last time you were shifted to another project in your organization. Chances are that you had to learn new rules, and until you learned them, you were more likely to make mistakes. There is no reason to believe it is any different for the sites.

Why, then, can’t we agree upon common practices and rules and approaches for these common activities? Are we so convinced of our own superiority that we refuse to change? Are we afraid that others will steal our good process ideas and get to market first? Are we just “used to doing it that way” or have “always done it that way” or believe that “regulations or Biometrics or Clinical require that we do it that way”? In other words, if it ain’t broke, don’t fix it? Well, I argue that it is “broke”. It is “broke” for the sites, and if it ain’t fixed we’ll continue to lose investigative sites, and pour ever more resources into trying to inspect quality into the data, and miss the opportunity to maximize the CDASH revolution.

So what do you think? Do you agree? Disagree? Want to throw turnips? Should life be more consistent for the sites at the expense of our processes? We’re a pretty inventive bunch – I bet we could find ways to be efficient while collaborating with the sites to improve consistency.

I’ve started a list of practices that I think could be harmonized. Do you agree with the list? What can you add? What shouldn’t be there? How do you approach these activities and why? Are your reasons based in concrete need or historical habit or the belief that someone else internally “won’t like it”? How can we create a sponsor/site forum that would be trusted by both groups? Who might have to collaborate internally and externally to make this happen? Let’s talk!

1. When a subject terminates early, do CRFs for all visits after termination have to be returned to Data Management? (applies to paper only, I assume)
2. Do sites complete new AE and Con Meds forms for each subject at each visit, or are existing forms updated? i.e., are AEs and Con Meds info captured in a visit-based style or a log-based style?
3. How should visits be referenced? Can they be standardized to 1, 2, 3, etc., with a possible variation for course-based studies (e.g., oncology, although that could probably be Visit 1, 2, 3 etc within each course)?
4. When should we start capturing AEs and SAEs? Informed consent? Start of treatment (technically it can’t be an AE if treatment hasn’t started)? SAEs at informed consent and all AEs at treatment start? Should it depend upon whether the study requires potentially harmful screening procedures? On something else?
5. Once the CDASH data fields have been finalized, can we agree on a consistent layout? What should it be for each of the domains?
6. For Adverse Events forms, should the layout be portrait with one AE per page or screen, or landscape with multiple lines per page or screen? What are the pros and cons of each? Note that this is not talking about how they are stored in the database – just how the site would see them.
7. Should monitoring guidelines be given to the sites? Put in the site’s study manual? Included in the EDC application? If not, why not?
8. Should the list of edit checks be given to the sites? If not, why not? Don’t we want the sites to understand what the data should look like? If so, how can they be presented so that they are accessible and understandable?

Each of these, and many more, can be discussions in their own right, and the answers to each depend upon any number of assumptions about the underlying processes, but surely, if we really want to, we can make this work. I look forward to your comments.

Photo: Romulus, Ann Arbor, Michigan. c. 2008, Kit Howard.

Who Are You?

The other day I heard the old song by The Who “Who Are You”. It has been a while, and I can’t follow most of the lyrics on the track, but it doesn’t really matter. What struck me, as I was thinking about our business, is “Who Are You?” We each have a number of roles we play.

We are Pharmaceutical Professionals. We might be a monitor, or clinical scientist, or programmer, or data manager, or any one of a plethora of other roles. In those roles we look at scientific puzzles, and try to ask the best questions to clarify the unknown to formulate the quickest reasonably concrete result. In that world we have control – we are the scientists, the providers of answers, the managers of data, the spinners of results. We tend to think of our day-to-day existence as a set of tasks. This database must be closed. That report must get written. The other article must get published so the following ad will appear “true”. This is how we earn our living, how we pay our mortgages, how we pass the time...

We have other roles as well. Roles that only surface when things go wrong. Our parents become ill, and suddenly we are trying to find out whether the medication or combination of medications prescribed for them are likely to help and/or have intolerable side effects. Where can we find the information? Some of it is on WebMD, some in the Merck manual, some perhaps in our own company data banks, some we get from colleagues in other companies. We discover that the information is fragmented, hard to come by, spun to minimize the negative and maximize the positive.

There is often no way to tell what subsets of patients are most likely to benefit, either because the information is not published, or it is published in scientific journals not available to the general public, or it is buried in reports so dense that x-rays could not penetrate them. Or maybe the information simply doesn’t exist – no one has ever done a study to see if that particular combination of medications in that particular combination of co-morbid conditions is beneficial or harmful or ineffectual…

So where can we turn? The pharma companies, our employers, don’t have any incentive to do these studies. The government bodies such as the NIH have their own research agendas. Who can tell us? In fact, given that there is an almost infinite number of possible combinations of medications and co-morbid conditions, can we ever hope to have any solid evidence? Maybe we have to make do with fragmentary data extrapolated from studies that don’t really reflect our situation in an environment where the optimal treatment may be unidentifiable and in any case would not be covered by insurance.

Then, eventually, we become the patients. Not subjects, for those are the ones participating in the trials that may or may not provide any benefit for them as individuals. The patients. Ill, disabled, frightened, tired, without the energy or mental strength to do the research to make sure that the treatments we are prescribed (by the doctor who may have graduated bottom of the class) are not going to worsen some other condition we have, or perhaps are just the usual run-of-the-mill treatments that don’t incorporate the latest findings.

As we get older, more and more of us join the ranks of the patients. Then we retire, and find that not only are we the patients, we are also the marginalized elderly, those who don’t work, who don’t contribute, who are seen as a burden, whose “entitlements” take an ever larger chunk of the federal government’s budget. As the baby boomers grow older, and the generation following is dramatically smaller, who will pay for our aging?

It is very clear to me that we have no choice. We have a moral responsibility to our elders, to ourselves and to our children to fix this system. We have to make drug discovery, development and marketing not only faster and less costly, but even more importantly, dramatically more transparent and more driven by the needs of the patients. It may not be immediately apparent how this can be compatible with corporate goals, but I submit that if we do not find a way to satisfy both imperatives, we as employees, we as caregivers and we as patients will be among those who suffer the consequences.

The Blossoming of Standards

It recently came to my attention that the National Institute of Neurological Disorders and Stroke has been conducting a project to define a data elements dictionary that can be used in trials it sponsors. The intent is both to facilitate data collections and storage as well as to promote data sharing. While there are many similarities in the data needed for different therapy areas, there are also many differences in both content and in structure. This makes creating a single data dictionary something of a challenge.

Tackling this challenge would be a tall order in a company, but there everyone (at least in theory) reports into the same organization, and, provided management has the intestinal fortitude to do so, common approaches can be mandated. The NINDS project has an even greater challenge, in that it has a significantly smaller degree of control over the structure of the studies it sponsors and thus must provide carrots rather than sticks (i.e., give investigators incentives to cooperate, rather than force them to do so).

This project is an example of an increasingly popular trend towards sharing data in the academic and government research environments, and to a lesser extent in industry (the latter being driven mostly by the need to create consistently structured data for regulatory submissions). CDISC is the most familiar one to those of us in industry, and it is aimed primarily at regulatory agencies. Outside of industry, I know of HL-7, which is (among other things) a set of messaging standards for defining and exchanging data between healthcare providers. There is also the National Cancer Institute with its caBIG project, which is a set of data dictionaries and processes for standardizing the definition, collection and storage of data from oncology trials sponsored by NCI. There may well be other projects.

It is wonderful to see such awareness of and enthusiasm for standardization. This can only help to increase efficiency, reduce costs, and enhance the safety and efficacy of medical treatments. On the other hand, it is a concern that, although these organizations do have some awareness of each others’ work, they are still working mostly independently. Not only is there some duplication of effort, but they will inevitably make different decisions and produce slightly different results. Anyone involved in such a project understandably has a tremendous sense of ownership and will be committed to his or her own decisions. At some point the various schemas will have to be harmonized, and this will require a great deal of work both to achieve the harmonization and then to apply the results back to the source systems. Because of this, it is critical that the organizations already developing standards be open, transparent and vocal about their work so that any other organizations contemplating initiating a standards project will join an existing one and work toward ensuring their needs are met there. By the same token, it is encumbent upon the existing projects to be open to the needs of others so that the number of projects is limited and eventual integration is not even more daunting.

Photo: Phyllachne colnsoi. Haast Pass, South Island, New Zealand. Flowers approx 3 mm diameter. c. 2006, Kit Howard.

The Washington Monument was Built by Aliens!

It has been said that figures don’t lie but liars figure. In his book Fads and Fallacies (1) Martin Gardner presented this cautionary tale as part of his examination of the numerical myths associated with the Great Pyramid in Egypt.

… If one looks up the facts about the Washington Monument in the World Almanac, he will find considerable fiveness. Its height is 555 feet and 5 inches. The base is 55 feet square, and the windows are set at 500 feet from the base. If the base is multiplied by 60 (or five times the number of months in a year) it gives 3,300, which is the exact weight of the capstone in pounds. Also, the word "Washington" has exactly ten letters (two times five). And if the weight of the capstone is multiplied by the base, the result is 181,500 —a fairly close approximation of the speed of light in miles per second. If the base is measured with a "Monument foot",(2) which is slightly smaller than the standard foot, its side comes to 56½ feet. This times 33,000 yields a figure even closer to the speed of light.

And is it not significant that the Monument is in the form of an obelisk—an ancient Egyptian structure? Or that a picture of the Great Pyramid appears on a dollar bill, on the side opposite Washington's portrait? Moreover, the decision to print the Pyramid (i.e., the reverse side of the United States seal) on dollar bills was announced by the Secretary of the Treasury on June 15, 1935—both date and year being multiples of five. And are there not exactly twenty-five letters (five times five) in the title, "The Secretary of the Treasury"?

It should take an average mathematician about fifty-five minutes to discover the above "truths," working only with the meager figures provided by the Almanac.

One definition of the quality of data is its “fitness for use”, which implies that the use has been defined. There is increasing interest at the FDA and within many companies in establishing data warehouses that will allow for meta-analyses that are orders of magnitude bigger than any possible today. We must be careful that, in our haste to mine the data for interesting relationships we do not use the data in ways that were never intended and create our own mythologies.

(1) Martin Gardner, Fads & Fallacies in the Name of Science. Dover Publications, New York.
c. 1957. pg 179.
(2) Derived by dividing the length of one building stone by 25, i.e., five times five.