Complexities in Reporting SAEs: What Do the Regs Really Say?

“If a patient is randomized to the study but never receives study drug, must site staff report serious adverse events (SAEs) for this patient?”

This excellent question was recently asked in the CDM group on LinkedIn. The responses ranged from assertions that regulations require that either serious or all AEs must be reported starting at informed consent (IC), to suggestions that, in the absence of any internal company guidance, they should be captured as they can be eliminated from the analysis based on the “definitely not related” assessment.

I think the answer is probably “it depends”! As with most aspects of clinical research, deciding what to collect and to report requires understanding the risks associated with the different choices. That requires understanding, among other things, the context of the question, where and when the research occurred, and who intends to use the data and for what. Below you will find my thoughts on some of these factors.

(Incidentally, if anyone can point me to a place in a regulation where it defines “on study” as beginning with signing IC, and/or where it clearly states that SAEs must be captured beginning at IC, I’d be grateful. I was unable to locate any such statement, but my sources are primarily US regulation and guidances, ICH guidances, ISO standards, and some European regulations and guidances.)

The Question

The original question was whether SAEs occurring after randomization but before treatment must be reported. Because of the range of responses in the Group, I have also expanded the question to “Must all AEs and/or SAEs occurring on study be reported?”

There are two elements to consider here.

• The first is the meaning of “reported.” Does it mean that AE/SAEs must be reported on a CRF? Or does it mean that AEs/SAEs must be reported to regulatory authorities under the expedited reporting rules? Or does it mean that AEs/SAEs must be reported in the study report? Each of these is a valid question, and the answers depend upon a number of factors, some of which are discussed below.
  
• The second is the meaning of “on study.” Is it when IC is signed, or at randomization, or the beginning of baseline, or something else? One could use the term “enrolled” instead, but it turns out that neither term is clearly defined. It does imply that timing plays a role, and this is explored later in this article.
  

What Do the Regulations/Guidances/etc. Say?

As mentioned above, I did not find any regulations or guidances that defined when to start capturing AEs/SAEs beyond those potentially associated with study treatment. Here is what I did find that relates to this discussion.

ICH E2A Clinical Safety Data Management: Definitions and Standards for Expedited Reporting

• Focuses on the requirements for expedited reporting, rather than on determining the timing of what should be captured
  
• Lists the minimal information necessary to send a report, which includes a treatment having a plausible causal relationship with the event
  
• States that there are circumstances where certain SAEs may be exempt from routine reporting, such as when the SAE may be the primary outcome and expected. (KH: This indicates that there is no absolute rule that all SAEs must follow expedited reporting, although this does not speak to whether they are captured.)
  

Definition: Adverse Event (from ICH E2A):

Any untoward medical occurrence in a patient or clinical investigation subject administered a pharmaceutical product and which does not necessarily have to have a causal relationship with this treatment. An adverse event (AE) can therefore be any unfavourable and unintended sign (including an abnormal laboratory finding, for example), symptom, or disease temporally associated with the use of a medicinal product, whether or not considered related to the medicinal product. (emphasis mine)

Different regulations/guidances have slightly different wording, but all require a potential association with treatment, so if the event occurred prior to treatment it cannot, by definition, be an AE. Thus, a sponsor could capture only those AEs beginning with study treatment and be technically completely compliant.

Definition: Serious Adverse Event (from ICH E2A):

During clinical investigations, adverse events may occur which, if suspected to be medicinal product-related (adverse drug reactions), might be significant enough to lead to important changes in the way the medicinal product is developed (e.g., change in dose, population, needed monitoring, consent forms). This is particularly true for reactions which, in their most severe forms, threaten life or function.

This is part of the definition of an SAE, and indicates some reasons why expedited reported should happen for SAEs. As SAEs are, by definition, a kind of AE, they too cannot occur prior to treatment initiation. In reality, there are times when treatment may not be the only factor to consider, especially with respect to SAEs.

ICH E3 Structure and Content of a Clinical Study Report

• 5.3 PATIENT INFORMATION AND CONSENT: How and when informed consent was obtained in relation to patient enrolment, (e.g., at allocation, pre-screening) should be described.
  
• 10.1 DISPOSITION OF PATIENTS: (…) It may also be relevant to provide the number of patients screened for inclusion and a breakdown of the reasons for excluding patients during screening, (…)
  

Item 5.3 suggests that IC and enrollment are not necessarily the same time point, and 10.1 reminds us that subjects can be excluded during screening, prior to treatment.

ICH E6 Consolidated Good Clinical Practices

• In section 6.9.2, it states that the clinical trial protocol should include The number of subjects planned to be enrolled. In multicentre trials, the numbers of enrolled subjects projected for each trial site should be specified. Reason for choice of sample size, including reflections on (or calculations of) the power of the trial and clinical justification.
  

This implies that enrollment refers to the subjects needed for analysis, not those screened.

21 CFR Part 312.62

(b)Case histories. An investigator is required to prepare and maintain adequate and accurate case histories that record all observations and other data pertinent to the investigation on each individual administered the investigational drug or employed as a control in the investigation. (…) The case history for each individual shall document that informed consent was obtained prior to participation in the study.

The regulation does not define what “participation” means, but since signing IC means agreeing to participate in the study, anything that happens after IC is, by definition, participating in the study. It’s a bit circular, but there it is!

ISO 14155.2 Clinical Investigation of Medical Devices for Human Subject – Good Clinical Practice

Definition 3.32: Point of Enrollment - time at which, following recruitment, a subject signs and dates the informed consent form

This brings up an interesting point:

1. Patients sign IC prior to participating in the study, meaning prior to any study procedures being performed (including those for screening)
   
2. Screening procedures are performed both to determine eligibility and, when appropriate, establish baseline values
   
3. If eligible, the subject continues. It is only at this point that the subject can be considered to be “on study.” Prior to this, eligibility has not been established, and so the subject must not be in the study, as GCP does state that ineligible subjects should not be included! (ICH E6 4.5.1 & 6.5.1)
   
4. Randomization may or may not occur at this point, depending upon the study design (there may be washout, baseline observation or other periods prior to randomization).
5. All of which means that the ISO definition of enrolment does not follow the same logic as the other regulations and guidances.
   

Even here it’s not completely clear, because later in the same standard, the Monitoring requirements state that the monitor should verify that:

f) signed and dated informed consent forms have been obtained from each subject at the point of enrollment and/or before any clinical investigation-related procedures are undertaken,
g) only eligible subjects as defined in the CIP are enrolled in the clinical investigation,

If only eligible subjects are enrolled, and enrollment happens at the time of IC, then all subjects who provided IC must be eligible, which cannot be true because screening procedures have not yet begun!

FDA’s Detailed guidance on the collection, verification and presentation of adverse reaction reports arising from clinical trials on medicinal products for human use, April 2006

5.1.1.2 Other safety issues requiring expedited reporting Other safety issues also qualify for expedited reporting where they might materially alter the current benefit-risk assessment of an investigational medicinal product or that would be sufficient to consider changes in the investigational medicinal products administration or in the overall conduct of the trial, for instance: (…)
c) new events related to the conduct of the trial or the development of the investigational medicinal products and likely to affect the safety of the subjects, such as: - a serious adverse event which could be associated with the trial procedures and which could modify the conduct of the trial, - a significant hazard to the subject population such as lack of efficacy of an investigational medicinal products used for the treatment of a life-threatening disease, (…)
5.1.2 What should not be reported? Expedited reporting is not usually required: - for reactions which are serious but expected, - for non-serious adverse reactions whether expected or not. It is generally not necessary to report events that are considered unrelated to the investigational medicinal product.

This last part speaks particularly to the distinction between capturing the SAEs and doing the expedited reporting. Section c. above brings in the point that study procedures may also cause SAEs, and although SAEs by definition must occur during treatment, study procedures happening during pre-treatment periods may also be important.

The take-home message is that one has to think about the context of the definitions, the spirit of the regulations/guidances/etc., and the specifics of the trial in order to determine the best course.

The Importance of Context

Why do we capture AEs and SAEs in a trial? We are trying to determine if the treatment causes undesirable effects that outweigh its benefits. We want to tease out the relevant events from the “background noise.” If randomization has happened appropriately, and there are no other sources of treatment assignment bias, and the investigators understand how to identify treatment emergent AEs, then the incidence of any given non-treatment-related AE in each treatment group should be the same, as should the incidence of the AE in the pre- and during-treatment periods. This suggests that, absent any additional risk factors or protocol design requirements, it should be unnecessary to capture AEs prior to treatment.

The question then becomes what to capture and/or what should follow expedited reporting rules. The following are other factors to consider.

• Screening procedures: if the screening procedures are invasive or otherwise risky, capturing AEs/SAEs prior to randomization may be desirable, as it may influence the conduct of the trial, and/or the requirements for patient monitoring after the product is approved. Whether they should be subject to expedited reporting would depend upon an assessment of the other factors below.
  
• Indication/population: how severe is the indication? If the subject population is quite ill, and SAEs are expected, then it may be appropriate to capture the SAEs, but not do expedited reporting. This should be defined a priori in the protocol after discussion with regulatory authorities, the company’s regulatory affairs and clinical colleagues.
  
• Time frame: If the decision is to capture SAEs for subjects who were randomized but never receive treatment, what time frame should be used? Should they be captured only for subjects who didn’t receive treatment because of the SAE? Should the subjects be monitored for the same follow-up duration as treated subjects? Should the SAEs be subject to expedited reporting, considering that it is known the subject was not on treatment?
  
• How much is known? If very little is known about the indication, treatment, study population and/or expected SAEs, then it would be appropriate to capture and report more information, as it is more likely that an event would be unexpected. As noted above in the FDA’s AE reporting guidance Section 5.1.2, if certain SAEs are already known to occur and this is documented in the Investigator’s Brochure, expedited reporting may be unnecessary, even if they are still “reported” on the CRF.
  
• Geographic location: where is the study being conducted? The regulatory authorities in different regions may have different requirements or preferences for how much should be captured/reported.
  
• Study design: randomization does not necessarily happen when the subject is enrolled, meaning that IC is signed and all eligibility criteria are met, and the subject is cleared to continue in the study. There can be washout periods, baseline observation periods, or other epochs that occur prior to randomization, and collecting AEs and/or SAEs may or may not be necessary. Much depends on the procedures performed and whether there is interest in comparing AE/SAE incidence before and after treatment.
  

Consequences of Reporting and Not Reporting

There are other, perhaps less obvious, consequences to these decisions.

• Capturing SAEs: capturing SAEs requires providing considerably more information than is necessary for AEs. This is an additional burden on the site. It can also be a burden and expense for the sponsor, as it requires additional attention at each stage from data entry to data management to analysis and report writing.
  
• Reporting SAEs: The burden is even greater when the expedited reporting processes are followed, as this requires completing additional forms, informing the sponsor and also the Institutional Review Board (IRB)/Ethics Committee (EC). The IRB/EC can be swamped with reports of routine events, which reduces their effectiveness. Finally, the receiving regulatory authorities have to distinguish between important events and those that are reported because the site/sponsor/etc. is being ultra conservative. This impairs their ability to respond to the critical events.
  
• Analyzing the data: analysis and study reporting happen using the data that are captured. Whether SAES from randomized-but-not-treated subjects are included in the general intent-to- treat analyses or are put in a separate table is the choice of the biostatisticians, medical writers and clinicians responsible for the study report.
  

Bottom Line

I don't think it's possible to answer the overall question with an absolute statement of yes or no.

• It is usually appropriate to capture SAEs for subjects who have been randomized but not treated, but they generally don’t need to follow expedited reporting.
  
• It’s also usually unnecessary to follow expedited reporting for SAEs for subjects who have signed IC but have not been randomized (assuming randomization is when they are enrolled), but whether to capture them is more ambiguous.
• Generally, it’s not required to capture AEs for subjects who signed IC but weren’t randomized.
  

The caveat is that there are exceptions to every one of these cases, as suggested by the earlier discussions. Like so much of what we do, the decision requires judgement. Regardless of your decision, it is good practice to define clearly in the protocol what is meant by “enrolled,” “on study” and any other similarly unclear term.

In order to make the right decision,

• Educate yourself on the variables involved, and read the regulations
  
• Gather the relevant questions and information
  
• Talk to the other functional areas – this decision cannot (MUST NOT!) be made in a silo, because other perspectives and knowledge bases are required to ensure that all angles are covered
  

Your Regulatory Affairs group may have already spoken with the regulatory agencies about this; a good time to bring it up is at the early Phase 2 or end of Phase 2 meeting with the regulators. Whatever decision is made, be sure that it is documented fully, including the rationale for each element, because you may need that documentation later to justify the decision. There are few absolutes in our business, and nowhere is that more true than when dealing with regulations.

Many thanks to those who posed and responded to the question on LinkedIn. The thread was accessed on 6 December 2009 at http://www.linkedin.com/groupAnswers?viewQuestionAndAnswers=&discussionID=10132121&gid=7 7402&commentID=8843270&goback=.anh_77402&trk=NUS_DISC_Q-subject#commentID_8843270

Disclaimer: The author does not work for a regulatory authority, and the material in this article is based on reading the regulations/guidances and applying her own experience and observations. Each company should confer with their internal experts and the appropriate authorities to determine the best approach for their situation.

Structuring Clinical Data: AE Seriousness Fields

Even in a world with data standards, clinical data can be structured in different ways and still be standards-compliant. This is especially true when the standards define what to collect, but not how, or when, or in what combination. This article takes one example, the serious adverse event (SAE) fields, and explores several designs and the circumstances in which they could be appropriate. All examples are CDISC-compliant, but are not necessarily compatible with each other, emphasizing the need for careful thought in implementation.

Premise

Most people, when designing an adverse event (AE) case report form (CRF), believe they know the right way to capture AE Seriousness data. A random selection of 10 people would produce at least 3 different opinions, each of which can be best practice in a specific situation. Implement a design based on one set of assumptions in another environment, however, and best practice can become seriously flawed. This article explores different data capture designs for AE Seriousness and discusses where they would be appropriate.

Defining “AE Seriousness”

The US regulations and the ICH guidances are uncharacteristically clear about what constitutes a serious adverse event, and they are consistent with each other. The ICH definition of “Adverse Event” is

Any untoward medical occurrence in a patient or clinical investigation subject administered a pharmaceutical product and which does not necessarily have to have a causal relationship with this treatment.

A serious AE is a one that meets the above definition and additionally has one or more of the following characteristics:

Fatal Life-threatening
Requires in-patient hospitalization or prolongs hospitalization
Results in significant or persistent disability/incapacity
Congenital anomaly/birth defect

“Overdose” was originally on the list but was removed some years ago. Medical device guidance adds “serious injury/serious illness”, which is defined as “necessitates medical or surgical intervention to preclude permanent impairment of a body function or permanent damage to a body structure.” There can also be additional AEs that are serious for a given trial based on requests from the regulatory authorities. SAEs must be reported promptly to the regulators, and the reports include many details that are not entered into the clinical database. These may include hospitalization, the circumstances of death, autopsy reports, and a full written narrative of the case. The clinical database generally treats SAEs and AEs the same, other than SAEs bring flagged to indicate that they are serious.

Storing Serious AE Data

Historically, while non-serious AE data were stored only in the clinical database and managed by the clinical data managers, SAEs were often handled by the sponsor’s regulatory or drug safety group. As CRFs were often not completed for weeks or months after the subject’s visit, this enabled the required expedited regulatory reporting and supported ongoing safety monitoring. Although electronic data capture and reporting systems are more available, and there are a few integrated solutions, CRF data are still generally not entered real-time, and most organizations still store this information in two different systems, each of which may contain partial data. While it must all be included in the AE domain for an SDTM-compliant submission, that merely means that it must be available, not that it must be stored in one place.

Some Considerations

Current industry standards (such as CDISC’s CDASH and SDTM) provide two approaches to recording in the clinical database whether the AE is serious – a single yes/no field, and a list of fields defining the serious criteria that are each marked either with a yes/no question or a “check all that apply” structure, as shown here.


Recommendations

• The standards do not specify how to use the fields or how they should interact. Here are some things to consider when making this decision.
• Flagging AEs as serious in the clinical database is necessary. It allows the AEs to be summarized separately in the study report, and identifies those that must be reconciled with the safety database. Generally, CDASH discourages capturing the same information twice, which implies that the form should either ask if the AE was serious, or capture the list of criteria.
• CRFs, whether paper or electronic, are usually not completed during or promptly after the subject’s visit, making them a poor trigger for the SAE reporting activities, and suggesting that the individual criteria list should not be used for that purpose.
• Generally, especially for shorter lists, individual yes/no questions are preferable to “check all that apply,” especially when it is critical to evaluate each item. This is consistent with the CDASH recommendations. Although there is no firm cutoff number above which “check all that apply” is acceptable, the decision should be driven by the importance of evaluating each item and how much additional CRF completion work would be created by individual questions.

Following are some different options and the circumstances in which they would be appropriate.

Situation	Field inclusion
The safety database captures the serious AE information including the serious criteria and The seriousness criteria are readily available to the summary programmers	Clinical database should contain just the one yes/no field
Paper CRFs are used	The clinical database should contain either the one yes/no field (if the criteria are available from the safety database) or the individual criteria fields, but not both
The safety database does not contain the individual criteria or The data are not readily available to the summary table programmers	Clinical database should contain just the individual criteria fields
The clinical and safety databases are integrated so that data capture is performed only once	Clinical database should contain the individual criteria fields
The EDC system uses the one yes/no question to determine whether to bring up the individual criteria questions	Clinical database can contain both sets of fields as long as entry checks ensure data consistency
The EDC system asks the individual criteria questions and if any is “yes”, the one yes/no question is derived to “yes”. This may be used to trigger access to the safety database, but this approach should only be used if there are system requirements or some other compelling reason to do it.	Clinical database can contain both sets of fields, but only the individual criteria fields should be entered

Conclusion

As suggested at the beginning of this article, there isn’t one correct answer to the question of what fields to include. That decision needs to be made in consultation with multiple functional areas to ensure all needs are met, and requires the application of judgment. Like so much of clinical data design, the more thought and collaboration there is, the higher the quality the result.

How Bad Does It Have to Get?

A recent study in PLoS One by E. Ray Dorsey, et al, found that between 1995 and 2005, the money spent on biopharm and medical device research has increased hugely across the board, and by as much as 369% in at least one therapeutic area. We have not, however, seen a comparable rise in the number of drugs approved by the FDA. Although the authors readily admit that their conclusions are based on a “sample of convenience,” they ring true to those of us in the industry. Whether you think that new drugs cost $600M or $1.2B to bring to market doesn’t really matter – we have thrown ever more money at a constantly diminishing return.

The authors suggest that now the limiting factors are non-financial, and solutions include bringing people and ideas together in more productive ways, bringing down the cost of clinical trials (though they don’t explore what this would take), openly disseminating the results of negative trials, and increasing the flow of people and ideas between organizations.

These all point to the need to revise and harmonize our processes, break down the functional area silos, increase the intelligent standardization of data and processes, and develop approaches that are nimble, flexible and adaptable. We need greater transparency in regulatory discussions and decision-making, which will help us to interpret regulations better and stop being so ultra-conservative in our work. Most of all, we need to understand that breaking the work into ever smaller pieces and parsing them out to ever more internal and external groups guarantees that no one sees the full picture, and so no one can control it. All of these changes must happen not only within organizations, but across them as well, to result in the best possible solutions.

The current approach is both failing and unsustainable. Neither government nor the public will tolerate the current mix of risk, benefit and cost. Unless our clinical trials enterprise changes dramatically, we will find ourselves forced to lower the prices we charge while maintaining the current costs. That is not a future that we, as future patients, should support.

The referenced study is at http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0007015
Photo: Dwarf ginseng, reputed to have calming medicinal properties. Taken at Sharon Hollow Nature Preserve, April 2007.
© 2009, Kit Howard

Upholding the Scientific Method

Many thanks to Mark Uehling, from ClinPage, who distilled my recent rants to their pure essence, namely:

By not sharing our clinical trial operational processes, we are violating the most basic tenet of scientific research, which is to share methods and data in order to progress. Without this, we jeopardize the future of our entire enterprise.

Amethyst crystal embedded in clear quartz, photo from www.wilenskyminerals.com

For those of you reasonably familiar with my fairly wordy style of writing… Hmm. That doesn’t do it. It’s usually good to learn new things, and recently I was made aware that… No, I don’t think that’s it either. OK. Here goes. I’m practicing being less wordy!

That was hard! My philosophy has always been “write it down, then edit to 50% of the size.” While editing is still good, I don’t have the bandwidth for that much rewriting, so I’m trying something new. This blog will still be a space for observations and rants, but my goal is to practice getting to the point quickly. Maybe in time it will become a habit! So here goes.

Recently I learned about OpenClinica, a free open-source clinical trials software system from Akaza Research. It apparently has been used in many trials by government, academia and in the industry. They join a growing trend of organizations no longer willing to accept industry’s rules (excuses?) of high costs and selective pipelines where treatments are developed only if they match a marketing strategy. Organizations like OneWorld Health, founded by Victoria Hale (one of my very few heros), a non-profit drug company that develops treatments for third world diseases. Or like the Parkinson’s Disease Society, that funds research to the tune of £4M per year.

Granted, these organizations have an advantage. They often receive their compounds from companies that have found them and don’t want to develop them, so much of the cost and risk of the initial research is mitigated. They generally don’t have shareholders demanding high profits. But they also don’t expect 7, 8 or 9 figure salaries, and it seems that their primary goal is treat patients, whereas much of industry seems to regard that as second to profits and personal wealth.

Most of us want to do the right thing, though, and one way is reconsider what we keep confidential and what we share. No, not chemical formulae or marketing strategies! I’m talking about clinical trials processes. How we develop our protocols, and SAPs, and CRFs. How we recruit our subject, monitor our data, resolve our queries. What we query. How we pass data between functional areas. There are efficient and inefficient ways to do these things, and we’ve been reinventing them for 20 years as new companies start and existing companies reorganize. It’s time for everyone to be open about how they do these things, and let the best approaches bubble to the top. And don’t tell me this is part of competitive advantage – no one out there is getting drugs to market dramatically faster than anyone else!!! Here’s a radical concept – let’s compete on the merits of the treatment or device itself!

Photo: St. Johns Wort, Auckland Botanical Gardens, Auckland, New Zealand. c. 2008, Kit Howard.

An Industry Evolving. Or not.

An updated version of the entry that was here can be found at http://www.clinpage.com/article/a_deep_process_problem/C12