FDA has a long history of using what we currently call real-world data (RWD) and real-world evidence (RWE) to monitor and evaluate the postmarket safety of approved drugs. RWE has also been used historically to support effectiveness, but on a more limited basis.
What does RWE stand for in clinical trials?
- Journal List
- Perspect Clin Res
- v.12(3); Jul-Sep 2021
- PMC8323556
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more about our disclaimer. Perspect Clin Res.2021 Jul-Sep; 12(3): 171–174.
- Electronic health records (EHRs)
- Medical claims, billing data, and insurance data
- Data from product and disease registries
- Patient-generated data, including from in-home-use settings
- Data gathered from other sources that can inform on health status, such as mobile devices.
Findings from clinical trials cannot be generalized to population at large due to the stringent eligibility criteria. RWE studies complement clinical trials by generalizing the findings from clinical trial to general population. Furthermore, RWE can provide information on other areas, such as natural history and course of disease, effectiveness studies, outcome studies, and safety surveillance.
What is the use of RWE?
What Is Real-World Evidence? – Real-world evidence (RWE) is the evidence derived from real-world data. RWE is clinical evidence regarding the use, potential benefits, and/or risks of a medical therapy. This type of evidence can be gathered by examining data from a variety of different types of research such as observational studies (both prospective and/or retrospective), randomized trials, large simple trials, and pragmatic trials.2
What is the difference between RWE and clinical trials?
What Is Real-World Data (RWD)? – In order to understand real-world evidence and its role in healthcare, we must first define real-world data, or RWD. Real-world data is any data that is collected in the context of the routine delivery of care, as opposed to data collected within a clinical trial where study design controls variability in ways that are not representative of real-world care and outcomes.
- Clinical data from electronic health records (EHRs) and case report forms (eCRFs). This data provide patient demographics, family history, comorbidities, procedure and treatment history, and outcomes.
- Patient-generated data from patient-reported outcome (PRO) surveys, These data provide insights directly from the patient, and they help researchers understand what happens outside of clinic visits, procedures, and hospital stays.
- Cost and utilization data from claims and public datasets. They provide information regarding healthcare services utilization, population coverage, and prescribing patterns.
- Public health data from various government data sources. These add critical information to enable stakeholders to best serve the needs of the populations they serve.
What are the different types of RWE?
Real-world evidence (RWE) research is an increasingly important component of biopharmaceutical product development and commercialization. The growing industry need for broader information on real-world effectiveness and safety-both of which will impact the eventual reimbursement and utilization of new products-is driven by regulators, public and private payers, and prescribers, all of whom seek to better understand the impact of a new product in a real-world setting.
- The result is that real-world evidence is now included earlier in the research and development phase.
- Real-world evidence (RWE) research is an increasingly important component of biopharmaceutical product development and commercialization.
- The growing industry need for broader information on real-world effectiveness and safety-both of which will impact the eventual reimbursement and utilization of new products-is driven by regulators, public and private payers, and prescribers, all of whom seek to better understand the impact of a new product in a real-world setting.
The result is that real-world evidence is now included earlier in the research and development phase. Today, a regulatory authority may require post-approval research or monitoring to determine real-world safety, due to concerns about a product’s safety when it becomes widely prescribed, while payers may require validation of a product’s real-world clinical value and cost-effectiveness to determine optimal formulary placement.
Data about real-world patient experience also has the potential to improve the quality and delivery of medical care, reduce overall costs and improve outcomes by accelerating the understanding of how best to incorporate new therapies and technologies into everyday clinical practice. Essentially, these data help fill the knowledge gap between clinical trials and actual clinical practice.
While clinical trials remain the gold standard for drug approval, more information is needed on how specific drugs perform within different age groups, such as the elderly, and different genders, races and ethnicities, as well as differences in disease severity and unstudied co-morbid conditions for use of products in the real world, as reflected in Figure 1.
Figure 1. Patients Exposed to Inadequately Assessed Health Care Technologies Source: HealthCore Biopharmaceutical sponsors, providers, payers and regulators are eager to investigate real-world data for insights that can increase quality, drive down costs and improve patient safety. For example, several years ago, Anthem, Inc.
asked HealthCore, its outcomes research subsidiary, to conduct a study comparing the effectiveness of oral asthma controller medications to that of inhaled corticosteroids. HealthCore found that when members took their medication as directed, those taking inhaled corticosteroids had fewer inpatient and emergency room visits and lower total health care costs.
But among members who were not taking their medication properly, those taking oral asthma controllers were less likely to have inpatient and emergency room visits. The insurer’s pharmacy and therapeutics committee used this real-world evidence, in addition to the overall evidence supporting these therapies, to keep the oral controller medication on a less-expensive, preferred tier and remove the prior authorization requirement for its use so its members could continue to use the product that worked best for them.
This research was conducted when the products were far along in their life cycles, raising the question about what a difference this information could have made on this product’s reimbursement if it had been conducted, shared and acted on earlier. HEOR in Today’s Environment Health economics and outcomes research (HEOR) generates evidence of the economic value of new and existing products and helps decision-makers and payers determine reimbursement policies and insurance coverage for new and existing therapies.
HEOR has become a more integral part of traditional R&D, impacting clinical trial design and the management of economic endpoints within these trials. With major global shifts in health care systems to control costs, a growing number of Phase III trials now include economic endpoints and analyses, alongside the clinical outcome measures, to demonstrate the cost-efficacy of new products.1 A meaningful real-world research study design for a new product begins with a deeper understanding of its real-world environment.
Access to robust real-world health data, including administrative claims data and other clinical data sources, together with the technology and analytics capability to effectively analyze that data is essential. Other prerequisites are scientific excellence, the scope and scale of operations to efficiently execute studies, expertise in clinical trial design and HEOR, and knowledge of related regulatory requirements.
Real-World Research Approaches Real-world research is an area of methodological innovation. Compared to clinical trial data, RWE data more closely describes how the product will perform in a broader, more representative population over a longer timeframe, and provides information on comparators and outcomes that are not part of the clinical trial protocol.2 There is a wide range of study types and complexities for a typical RWE research program.
RWE can be divided into two types: primary data, collected specifically for research purposes; and secondary data, collected for other purposes. Primary data are generally obtained from study-specific case report forms, electronic medical and health records, and/or clinical outcomes assessments.
- These data are collected in interventional Phase IV studies and in non-interventional prospective observational studies, patient registries and health surveys.
- Secondary data are often obtained from clinical chart reviews, registries and/or insurance claims databases, and are used in retrospective database studies or as an input to prospective study design or hybrid studies.3 Pragmatic and Explanatory Trials Trials of health care interventions often are described as either pragmatic or explanatory.
Explanatory trials generally measure efficacy, the benefit a treatment produces under ideal conditions, often using carefully defined subjects in a research clinic. These specialized studies recruit as homogeneous a population as possible and aim primarily to further scientific knowledge.4 By contrast, pragmatic randomized clinical trials measure effectiveness, the benefit the treatment produces in routine clinical practice, and are carried out after product approval.
- A pragmatic trial reflects variations between patients that occur in real clinical practice and aims to inform choices between treatments, as portrayed in Figure 2.
- These trials should represent patients to whom the treatment will be applied, and test whether an intervention works in real-world circumstances.
Pragmatic trials offer a scientific method of research for policymakers and clinicians, and serve as real-world evidence sources for decisions, such as for funding, regulations, policy and organizational changes.5 Figure 2. Schematic of the relationship between explanatory and pragmatic trials.
The wide based of the pyramid depicts the relatively higher proportion of explanatory trials. Source: Reproduced from Dialogues in Clinical Neuroscience with the permission of the publisher (Les Laboratoires Servier, Suresnes, France). ©Les Laboratoires Servier.6 An example of a pragmatic RWE study is a new Sanofi Aventis trial to determine the efficacy and health outcomes of Toujeo, a long-acting basal insulin to improve glycemic control in adults with type 2 diabetes.
Currently recruiting patients, the objective of the study is to demonstrate the clinical benefit of the drug in achieving specific therapeutic targets in patients initiating basal insulin therapy in a real-world setting. The secondary objective is to compare the drug to commercially available basal insulins after initiating insulin therapy in a real world setting, in terms of patient persistence with the assigned therapy, differences in patient- and provider-reported outcomes, and health care resource utilization, including hospitalizations and emergency department or other provider visits, and health care costs.7 Cluster RCTs In a cluster randomized trial (CRT), groups of subjects, rather than individual subjects, are randomized to receive one of the interventions being studied.
- Clusters may include provider practices, families, schools, clinics, health plans, towns and others.
- An example of the application of a CRT is in the assessment of health promotion programs, where it is problematic to design a study in which one member of a cluster (e.g., a town, a community or a patient in a group practice) can be exposed to the full impact of a program while another member would have no exposure.
A CRT can provide a way to better approximate the real-world setting for drug/program combinations. With the increased interest in pragmatic clinical trials, comparative effectiveness research and community health promotional activities, the use of CRTs has been growing.5 RWE Support Organizations Several organizations have been formed to help the life sciences industry supply practical information about its products.
- The Patient-Centered Outcomes Research Institute (PCORI) was established to help patients and clinicians improve key health outcomes.
- The group identifies critical research questions, funds patient-centered comparative clinical effectiveness research and disseminates evidence-based results.8 The Institute for Clinical and Economic Review (ICER) is an independent source for evaluating the value of new drugs and providing a transparent, objective basis for price negotiations and coverage decisions.
Over the next two years, ICER will produce 15-20 public reports on new high-impact drugs near the time of FDA approval, and calculate a benchmark price for each new drug anchored to the real benefits of the drug.9 Another organization, the American Society of Clinical Oncology (ASCO), is the world’s leading professional organization representing physicians who care for cancer patients.
Recently ASCO published a conceptual framework to assess the value of new cancer treatment options based on clinical benefit, side effects and costs. The framework is intended as a user-friendly, standardized tool doctors can use to discuss with patients the relative value of new cancer therapies compared with established treatments.10 Advancing Capabilities for RWE Studies To meet the increasing demand for RWE studies, sponsors and clinical research organizations (CRO) are ramping up their capabilities in this growing area of interest.
Many are forming partnerships and building capabilities to leverage the opportunities offered by RWE. As an example, PPD and HealthCore recently established a collaboration to further expand their services to help biopharmaceutical clients demonstrate more quickly and cost-effectively how their products will perform and benefit patients in the real world.
Through this collaboration, PPD and HealthCore have all the capabilities needed to plan, execute and deliver all types of RWE studies, in particular, pragmatic Phase IV trials. This alliance combines extensive experience and expertise in clinical trial design, HEOR, medical affairs research, epidemiology, access to large databases of integrated medical and pharmacy claims and lab results, innovative real-world research designs, analytics, and qualitative research.
Together, their RWE capabilities include: observational research, post-approval safety studies, pragmatic Phase IV trials, patient, product, and disease registries, patient-reported outcomes, and economic evaluation. Looking Forward: Outlook for RWE Studies RWE studies serve many purposes and are creating new opportunities for evidence generation in drug development.
- The EMA guidance 11 requires the collection of risk-benefit data in post-authorization safety studies.
- Pharmaceutical companies now must take a more granular approach, examining different subpopulations to determine their respective risk-benefit balance.
- There is also an increasing demand from payers to conduct observational studies on a new product’s effectiveness, and payers and clinicians are eager for more detailed health outcomes data to inform prescribing and reimbursement decisions.12 The outlook for RWE generation-stand-alone or in conjunction with clinical trials-is promising, with the potential to improve health outcomes and cost-effectiveness of new health technologies.
The demand for RWE is growing and is unlikely to subside as health care decision-makers become increasingly aware of what it offers. Biopharmaceutical companies and CROs need to stay at the forefront of developments in RWE, data sources, analytic techniques and study methodologies to ensure they are able to optimize patient access and formulary placement of new products.
- Mark Cziraky is Vice President of Research at HealthCore; and Michael Pollock is Vice President, Real World Outcomes for PPD.
- References 1. Brooks K.
- CRO Outlook & Market Trends.
- Contract Pharma,
- June 5, 2013.
- Accessed at: http://www.contractpharma.com/issues/2013-06/view_features/cro-outlook-market-trends,2.
Fiminska Z. Real World Evidence: Maximize Benefits to Healthcare. Eye for Pharma, Feb.26, 2015. Accessed at: http://social.eyeforpharma.com/market-access/real-world-evidence-maximize-benefits-healthcare,3. Mack C and Lang K. Using Real-World Data for Outcomes Research and Comparative Effectiveness Studies.
- Drug Discovery & Development,
- Nov.4, 2014.
- Accessed at: http://www.dddmag.com/articles/2014/11/using-real-world-data-outcomes-research-and-comparative-effectiveness-studies,4.
- Maclure M.
- Explaining pragmatic trials to pragmatic policy-makers. CMAJ,
- May 12, 2009;180(10): 1001-10003.5.
- Owalski CJ and Mrdjenovich AJ.
Studying group behaviour: cluster randomized clinical trials. American Journal of Clinical and Experimental Medicine, Vol.1(1), 2013: 5-15. Accessed at: http://article.sciencepublishinggroup.com/pdf/10.11648.j.ajcem.20130101.12.pdf,6. Patsopoulos NA. A Pragmatic View of Pragmatic Trials.
Dialogues Clin Neurosci,2011;13(2):217-224.7. A “Real World” Trial to Determine Efficacy and Health Outcomes of Toujeo (ACHIEVE CONTROL REAL LIFE STUDY PROGRAM). Sanofi. Accessed at: https://clinicaltrials.gov/ct2/results?term=Toujeo+Real+World+Trial&Search=Search.8. Why PCORI Was Created. Accessed at: http://www.pcori.org/about-us/why-pcori-was-created,9.
Emerging Therapy Assessment and Pricing. Institute for Clinical and Economic Review. Accessed at: http://www.icer-review.org/etap,10. ASCO Publishes Conceptual Framework to Assess the Value of New Cancer Treatment Options. American Society of Clinical Oncology.
June 22, 2015. Accessed at: http://www.asco.org/press-center/asco-publishes-conceptual-framework-assess-value-new-cancer-treatment-options,11. Post-authorisation safety studies (PASS). European Medicines Agency. Accessed at: http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/document_listing/document_listing_000377.jsp&mid=WC0b01ac058066e979,12.
Christel M. “Evidence” Trail Elusive. Applied Clinical Trials, June 30, 2014. Accessed at: http://www.appliedclinicaltrialsonline.com/evidence-trail-elusive-0,
What is the difference between RCT and RWE?
RCT = randomized clinical trial, RWE = real-world evidence. RCTs versus RWE18.
Variables | RCTs | RWE |
---|---|---|
Setting | Experimental setting | Real-world setting |
Follow up | Designed | In actual practice |
Treatment | Fixed pattern | Variable pattern |
Study group | Homogenous | Heterogeneous |
What is the source of RWE?
1. Introduction – Real-world evidence (RWE) research has received attention in the recent decade and is playing an increasing role in health care decisions, As shown by the recent COVID-19 pandemic, RWE may help to broaden the view on patient populations and multiple medications not being included or implemented in classical clinical trials,
Therefore, the interest in whether non-randomized RWE can help to supplement the evidence of randomized controlled trials (RCTs) or aid in clinical decision-making is growing. Globally, growing activities of international initiatives and the release of legal frameworks by regulatory authorities have been observed in this field,
The British Medicines and Healthcare Products Regulatory Agency (MHRA) defines RWE as information produced from data routinely collected on patients, such as electronic health records (EHR) as well as disease and patient registries, The Food and Drug Administration (FDA) defines RWE as “clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD”,
The US Congress defines real-world data (RWD) utilized in RWE as “data regarding the usage, or the potential benefits or risks, of a drug derived from sources other than traditional clinical trials”, The FDA has expanded the definition of RWD as follows: RWD “are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources”,
Thus, RWE research is based on a number of sources, including EHRs, claims and billing activities, laboratory data, hospital data, product and disease registries, patient-generated data including in home-use settings, data gathered from other sources that can inform on health status (e.g., mobile devices), and data linkage approaches ; see Figure 1, Sources of real-world evidence (RWE) research. The present review indicates, in its first chapter, current RWE gaps and highlights the chronological development of RWE, including the actions taken and guidelines released by the regulatory authorities.
- The second chapter describes the current situation of RWD in accelerating the orphan drug market and early drug assessment.
- In the third chapter, the contribution of real-world results during the COVID-19 pandemic and how it impacted the understanding of vaccine research is described; the fourth chapter reviews the importance of high-quality cancer registries as a growing RWE source in oncological research.
The following two chapters, chapters five and six, explore the ways in which RWE facilitates RCTs and digital health technology, respectively. In chapter seven, four FDA-funded projects of RWE are introduced. The current review generates a chronological overview of RWE development over the past decade and contextualizes lessons learned from research in oncology and the COVID-19 pandemic with actual perspectives from clinicians.
What is an example of real world data?
What Is Real-World Data? – Real-world data (RWD) is defined as “the data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources.” 1 Sources of RWD include, but are not limited to:
- Electronic health records (EHRs);
- Claims and billing activity;
- Product and disease registries;
- Data gathered from other sources such as mobile devices, wearables such a pedometers and smart watches, etc.1
What is the use of RWE in regulatory submissions?
The CDRH Experience with RWD and RWE – Jeff Shuren, M.D., J.D. The FDA’s Center for Devices and Radiological Health (CDRH) clarified how we evaluate the relevance and reliability of RWD to determine whether they are sufficient for generating the types of RWE that can be relied on for FDA regulatory decision-making for medical devices, in a 2017 guidance on Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices,
RWE can be leveraged to bring new products to market, evaluate the safety and effectiveness of existing products for new uses, and assess the continued performance and safety of products once on the market. The experience of CDRH, exemplified by a recent analysis, illustrates the wide range of RWE that has been used in regulatory decisions.
Today, CDRH published the results of that analysis, which include 90 publicly available examples of different types of regulatory submissions supported by RWE. These examples come from the full continuum of clinical and device areas throughout CDRH and across the medical device total product life cycle but do not comprise an exhaustive list of all submissions that have relied on RWE.
What are the disadvantages of RWE?
Benefits and limitations of RWE – Real-world studies reflect clinical experiences across a broader and more diverse distribution of patients than RCTs, They often include larger sample sizes and longer follow-up periods. This can allow assessment of long-term outcomes and provide additional safety information after drug approval.
In addition, as these studies use existing data sources, they can be more economical and time efficient than RCTs. Limitations include inconsistencies in collecting electronic data and also missing data. To maximise validity and applicability, RWD should be accurate, consistently collected and verifiable to a similar level as prospective clinical trials.
In addition, recognition of the limitations of each study is also important.
What is the difference between real world data and real world evidence?
Real-World Data (RWD) are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources. Real-World Evidence (RWE) is the clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD.
What are the key parameters which decides the most valuable source of RWE for a biopharmaceutical manufacturer?
What type of RWE is most valuable to a biopharmaceutical manufacturer? – Alwardt : Most biopharma companies know the questions they want to ask. The questions determine the source of the RWE. The questions they most commonly ask fall into two areas: treatment patterns and drug efficacy.
What are the advantages and disadvantages to using real-world clinical data as opposed to clinical trials?
The advantages & limitations of real-world studies –
RWE has broader generalizability than clinical trial data and can provide insights to the effectiveness and safety of a drug during routine care; however, limitations include the potential for bias and confounding factors that are controlled for in RCTs.
How many types of RCT are there?
A user’s guide Alejandro R Jadad 2 Types of randomised controlled trials
There are many terms used to describe and classify RCTs. Some RCTs explore different aspects of the interventions. RCTs can be designed according to the participants’ exposure to the interventions. RCTs can be classified according to the number of participants. RCTs can also be classified by the presence, absence, or degree of strategies to control bias. RCTs can take account of the preferences of eligible individuals.
Randomised clinical trials (RCTs) can be used to evaluate different types of interventions in different populations of participants, in different settings, and for different purposes. Once investigators ensure that allocation of participants to the study groups is random (to call the study an RCT), they can design the study using strategies to match the characteristics of the interventions they want to study, the resources they have available, and their academic, political, marketing, or clinical motivations.
Over the years, multiple terms have been used to describe different types of RCTs. This terminology has evolved to the point of becoming real jargon. This jargon is not easy to understand for those who are starting their careers as clinicians or researchers, because there is no single source with clear and simple definitions for all these terms.
In this chapter, I will describe the most frequent terms used to describe different types of RCTs. I will do my best to classify them in a way that will be easy for you to follow, understand, and remember. Some of the terms apply specifically to RCTs, whereas others may also be applied to other study designs.
Some terms are mutually exclusive, some overlap considerably, and some complement each other. On occasion, I will include terms that are used to describe other types of studies that are not necessarily RCTs, to ensure that you are aware of the differences between them. RCTs can be classified according to: (1) the aspect of the interventions investigators want to explore; (2) the way in which the participants are exposed to the interventions; (3) the number of participants included in the study; (4) whether the investigators and participants know which intervention is being assessed; and (5) whether the preferences of non-randomised individuals and participants are taken into account in the design of the study ( Box 2.1 ).
RCTs that explore different aspects of the interventions Depending on the aspects of the interventions that investigators want to evaluate, RCTs can be classified as: explanatory or pragmatic; as efficacy, effectiveness, or equivalence trials; and as phase I, II or III.
- What is the difference between explanatory and pragmatic trials? Explanatory trials address whether or not an intervention works.
- If the intervention works, then these trials attempt to establish how such intervention works.
- Typically, these trials are designed in such a way that the results are likely to yield a ‘clean’ evaluation of the interventions.
To achieve this, the investigators set strict inclusion criteria that will produce highly homogeneous study groups. For instance, investigators designing an explanatory study of the effects of a new antihypertensive drug could decide to include only patients aged between 40 and 50 years, with no coexisting diseases (that is, diabetes) and exclude those receiving other particular interventions (b-blockers).
Explanatory trials also tend to use placebos as controls, fixed regimens (that is, 20 mg by mouth every 6 hours), long washout periods (if patients have been taking diuretics, for instance, those drugs will be stopped for a sufficient period of time to ensure that they are ‘washed out’ of their bodies), intention to treat analysis (see Chapter 3 ), and focus on ‘hard’ outcomes (that is, blood pressure recorded at specific times after a detailed and standardised process).
Pragmatic trials (also called management trials) are designed not only to determine whether the intervention works, but also to describe all the consequences of its use, good and bad, under circumstances mimicking clinical practice.1 To achieve this, pragmatic studies tend to use more lax criteria to include participants with heterogeneous characteristics, similar to those seen by clinicians in their daily practice.
In addition to the more lax inclusion criteria, pragmatic trials tend to use active controls (that is, the new antihypertensive drug vs a b-blocker), flexible regimens (that is, 20 mg orally every 6 hours, reducing or increasing the dose by 5 mg according to the degree of blood pressure control and adverse effects), and analysis of the patients who received the interventions (see Chapter 3 ).
Pragmatic trials do not preclude the use of ‘soft’ outcome measures, such as measures of sexual function or quality of life. Although both explanatory and pragmatic approaches are reasonable and even complementary, it is important that you understand that they represent the extremes of a spectrum and that most RCTs include a combination of elements from each.
- The key issue is whether the investigators achieved the best combination of elements to answer their (and your) questions.
- What is the difference between efficacy and effectiveness? RCTs are often described in terms of whether they evaluate the efficacy or the effectiveness of an intervention.
- These two concepts are frequently misunderstood.
Efficacy refers to whether an intervention works in people who receive it.2 Trials designed to establish efficacy (also called efficacy trials) tend to be explanatory trials, because they are designed to yield a ‘clean’ evaluation of the effects of the intervention.
In this particular case, however, the investigators are not so interested in finding out how the intervention works. Instead, their main goal is to include participants who will follow their instructions and who will receive the intervention. The extent to which study participants follow the instructions given by the investigators is called compliance or adherence.
High compliance is easy to achieve when the administration of the interventions is completely controlled by the investigators or by other health professionals, who are not acting as investigators but are supportive of the study (that is, an RCT evaluating the effects of coronary artery bypass surgery and those of angioplasty in patients with unstable angina).
This is more difficult when the interventions are not administered by the investigators but by the participants themselves, when the study has a long duration, and when the interventions have to be administered several times a day. Returning to the example of the antihypertensive drug discussed in the previous section, compliance will depend on the extent to which the participants take the antihypertensive tablets as prescribed for the whole duration of the study.
The investigators in charge of this study may choose to include patients who have already shown high compliance in other studies. Effectiveness refers to whether an intervention works in people to whom it has been offered.2 These RCTs, also called effectiveness trials, tend to be pragmatic, because they try to evaluate the effects of the intervention in circumstances similar to those found by clinicians in their daily practice.
- The design of effectiveness trials is usually simpler than the design of efficacy trials, because effectiveness trials tend to follow lax inclusion criteria, include flexible regimens, and allow participants to accept or reject the interventions offered to them.
- Typically, effectiveness trials evaluate interventions with proven efficacy when they are offered to a heterogeneous group of people under ordinary clinical circumstances.
On occasions, trials are designed not to detect possible differences in efficacy or effectiveness between two or more interventions, but to show that the interventions are, within certain narrow limits, ‘equally effective’ 3 or equally efficacious. These trials are called equivalence trials.
- Often, these trials seek to demonstrate that a new intervention (or a more conservative one) is at least as good as the conventional standard treatment.
- Investigators who engage in equivalence trials make efforts to minimise the risk of suggesting that the interventions have equivalent effects when in fact they do not.
Strategies to minimise this type of risk are described in Chapters 3 and 4, What are phase I, II and III trials ? These terms are used to describe the different types of trials that are conducted during the evaluation of a new drug. Phase I and II trials are not usually randomised.
As the name suggests, phase I trials are the first studies conducted in humans to evaluate a new drug. Phase I trials are conducted once the safety and potential efficacy of the new drug have been documented in animals. As the investigators know nothing about the effects of the new drug in humans, phase I trials tend to focus primarily on safety.
They are used to establish how much of a new drug can be given to humans without causing serious adverse effects, and to study how the drug is metabolised by the human body.4 Phase I trials are mostly conducted on healthy volunteers. The typical participant in a phase I study is one of the investigators who developed the new drug, either an employee of a pharmaceutical company or a member of a research team at a university.
People with diseases for which there is no known cure (that is, AIDS and certain types of cancer) often participate in phase I trials. As mentioned above, these trials are often not randomised, and not even controlled. Usually, they are just series of cases in which the participants are given incremental doses of the drug, without a control group, while they are monitored carefully by the investigators.
In addition to the inherent limitations of case series, the main problem of this type of trial is that, if the participants are patients, those who are studied at the beginning are likely to receive very low doses which are unlikely to be effective, whereas those studied later are at greater risk of receiving toxic doses but are also more likely to benefit if the drug is effective.
- After the safety of a new drug has been documented in phase I trials, investigators can proceed to conduct phase II trials.
- These are trials in which the new drug is given to small groups of patients with a given condition (usually about 20 per trial).
- The aim of phase II trials is to establish the efficacy of different doses and frequencies of administration.
Even though phase II trials focus on efficacy, they can also provide additional information on the safety of the new drug. Often, phase II trials are not randomised, particularly when the therapeutic effects of the new drug can be measured objectively.
- For instance, if a new drug has been designed to treat a type of cancer that is associated with a high mortality rate, the investigators will conduct a phase II trial in which about 20 patients will receive the drug while tumour response, mortality and adverse effects are monitored carefully.
- If the drug is judged to be ineffective or excessively toxic, no more trials will be conducted.
If the drug produces a good response (that is, ‘fewer patients than expected’ die), however, and patients tolerate its adverse effects, the investigators can proceed to a phase III trial. When the effects of the new drug are assessed using subjective measures (that is, pain relief with a new analgesic drug), the investigators can use a randomised design in which they will compare the effects of the new drug with a placebo (see below) to ensure that the effects observed in the small groups of patients can be attributed to the new drug and not to other factors (that is, a placebo effect).
- Phase III trials are designed and conducted once a new drug has been shown to be reasonably effective and safe in phase II trials.4 Phase III trials are typically effectiveness trials, because they seek to compare the new drug with an existing drug or intervention known to be effective.
- This existing drug is usually regarded as the current standard treatment.4 Most phase III trials are RCTs.
There is an additional group of studies called phase IV trials. The term ‘phase IV trial’ is used to represent large studies 3 that seek to monitor adverse effects of a new drug after it has been approved for marketing.4 These studies are also called postmarketing surveillance studies.
- They are mostly surveys and seldom include comparisons among interventions.3 The term ‘phase IV trial’ can also be used to describe promotional strategies to bring a new drug to the attention of a large number of clinicians.4 In either case, phase IV trials are not RCTs.
- RCTs according to the participants’ exposure to the interventions Depending on the extent to which the participants are exposed to the study interventions, RCTs can have parallel, crossover, or factorial designs.
What is a parallel design? Most RCTs have a parallel design. In these studies (also called parallel trials or RCTs with parallel group design), each group of participants is exposed to only one of the study interventions. For instance, if a group of investigators uses a parallel design to evaluate the effects of a new analgesic compared with those of a placebo in patients with migraine, they would give the new analgesic to one group of patients and placebo to a different group of patients.
- What is a cross-over design? An RCT has a cross-over design when each of the participants is given all the study interventions in successive periods.
- The order in which the participants receive each of the study interventions is determined at random.
- Crossover trials produce within participant comparisons, whereas parallel designs produce between participant comparisons.
As each participant acts as his or her own control in crossover trials, they can produce statistically and clinically valid results with fewer participants than would be required with a parallel design.5 The time during which each of the interventions is administered and evaluated is called a period.
The simplest crossover design includes only two periods. Returning to the example of the new analgesic, if the same group of investigators uses a crossover design, they would randomise each patient to receive the new analgesic first and then the placebo, or vice versa – the placebo first and then the new analgesic.
Crossover trials are not always appropriate. Every time you read the report of a crossover trial, you should explore the extent to which some basic rules are followed. The interventions should be used in chronic, incurable diseases Patients who are cured by one or more of the interventions will not be eligible to enter subsequent periods of a crossover trial.
- This means that the ability of the crossover trial to produce within patient comparisons is lost and the baseline characteristics of the participants in each period are no longer the same.
- Ignoring this rule can bias the results of a crossover trial substantially, particularly when a crossover trial is used to compare the effects of a drug that can cure a disease with those of a placebo 6 (see Chapter 3 ).
The effects of interventions should have rapid onset and short duration This minimises the risk of drop out within each period and helps to keep the number of participants stable across periods. In addition, if the effects of the intervention are of short duration, they are less likely to persist during the administration and evaluation of another intervention.
When the effects of an intervention are still present during the evaluation of another, such effects are called carry-over effects. If any observed difference between the interventions can be explained by the order in which the interventions were given to the participants, this is called a treatment-period interaction and it can invalidate the trial.
Carry over effects can be predicted when the duration of the effects of the interventions are well known. In these cases, carry over effects can be prevented by separating the study periods by a period of time of sufficient length to enable the participants to be free of the influence of the intervention previously used by the time they receive the next intervention.3 This amount of time is also known as a washout period.
- The condition (or disease) must be stable If the disease is stable, the circumstances at the beginning of each period are more likely to be the same than if the disease is not stable.
- For instance, a crossover design to evaluate the new analgesic will produce more valid results in patients with migraine than in patients with postoperative pain because the intensity of postoperative pain tends to decrease with time.
Even if the new analgesic was studied only in patients with migraine, the results of a crossover trial would be more valid if it included patients who have suffered similar episodes of migraine for many years than in patients who present episodes of migraine of different and unpredictable duration or intensity.
- All the differences between the study periods that are caused by disease progression, regression, or fluctuation are called period effects.
- Carry over and period effects are known as order effects.
- Both can be assessed and removed from the comparisons by using statistical manoeuvres that are beyond the scope of this book, but that are described elsewhere.7 What is a factorial design? An RCT has a factorial design when two or more experimental interventions are not only evaluated separately, but also in combination and against a control.
For instance, a factorial design to study the effect of an opioid (that is, morphine) and a non-steroidal anti-inflammatory drug (that is, ibuprofen) for the treatment of cancer pain would mean that patients will be allocated randomly to receive ibuprofen only, morphine only, a combination of morphine and ibuprofen, or placebo.
This design allows the investigators to compare the experimental interventions with the control (that is, morphine vs placebo), compare the experimental interventions with each other (that is, morphine vs ibuprofen), and investigate possible interactions between them (that is, comparison of the sum of the effects of morphine and ibuprofen given separately with the effects of the combination).
RCTs according to the number of participants RCTs can include from one to tens of thousands of participants, they can have fixed or variable (sequential) numbers of participants, and they can involve one or many centres. Is it possible for an RCT to have only one participant? The answer is yes.
These RCTs are called ‘ n -of-1 trials’ or ‘individual patient trials’. Basically, they are crossover trials in which one participant receives the experimental and the control interventions, in pairs, on multiple occasions and in random order. These trials provide individual, rather than generalisable, results.
They can be very useful when it is not clear whether a treatment will help a particular patient. You may find yourself in this situation, for instance, when you have a patient with a rare disease and there are no trials supporting the use of the treatment in that particular disease, or when the patient does not have a rare disease, but the treatment has been evaluated in studies that include very different patients.8 Typically, the number of pairs of interventions varies from two to seven.
- Usually, the number of pairs is not specified in advance, so that the clinician and the patient can decide to stop when they are convinced that there are (or that there are not) important differences between the interventions.
- The success of n -of-1 trials to guide clinical decisions depends largely on whether the patient is willing to collaborate and on whether the rules described in relation to crossover trials (see above) are followed.
A detailed description of how to design, conduct, and analyse n -of-1 trials is beyond the scope of this book, but can be found elsewhere.9 What is a mega-trial? ‘Mega-trial’ is a term that is being used increasingly to describe RCTs with a simple design (usually very pragmatic) which include thousands of patients and limited data collection.10, 11 Usually, these trials require the participation of many investigators (sometimes hundreds of them) from multiple centres and from different countries.
- The main purpose of these large simple trials is to obtain ‘increased statistical power’ and to achieve wider generalisability.
- This means that their aim is to increase the chances of finding a difference between two or more interventions, if such a difference exists.
- This issue will be discussed in more detail in Chapter 5,
What is a sequential trial? A sequential trial is a study with parallel design in which the number of participants is not specified by the investigators beforehand. Instead, the investigators continue recruiting participants until a clear benefit of one of the interventions is observed, or until they are convinced that there are no important differences between the interventions.12 These trials allow a more efficient use of resources than trials with fixed numbers of participants, but they depend on the principal outcome being measured relatively soon after trial entry.
- What is a fixed size trial? In a fixed size trial the investigators establish deductively the number of participants (also called sample size) that they will include.
- This number can be decided arbitrarily or can be calculated using statistical methods.
- The main goal of using statistical methods to calculate the sample size is to maximise the chance of detecting a statistically and clinically significant difference between the interventions when a difference really exists (see Chapter 5 ).
RCTs according to whether the investigators and participants know which intervention is being assessed In addition to randomisation (which helps control selection bias), the investigators can incorporate other methodological strategies to reduce the risk of other biases.
- These biases and the strategies to control them will be discussed in detail in Chapter 3,
- I have brought this issue to your attention in this chapter because the presence, absence, or degree of one of these strategies has been used to classify RCTs.
- This strategy is known as ‘blinding’ or, perhaps more appropriately (but rarely used), ‘masking’.
In clinical trial jargon, blinding or masking represents any attempt made by the investigators to keep one or more of the people involved in the trial (that is, the participant or the investigator) unaware of the intervention that is being given or evaluated.
The purpose of blinding is to reduce the risk of ascertainment or observation bias. This bias is present when the assessment of the outcomes of an intervention is influenced systematically by knowledge of which intervention a participant is receiving. Blinding can be implemented at at least six different levels in an RCT.
These levels include the participants, the investigators, or clinicians who administer the interventions, the investigators or clinicians who take care of the participants during the trial, the investigators who assess the outcomes of the interventions, the data analysts, and the investigators who write the results of the trial.
As you might expect, in many studies those individuals who administer the interventions, take care of the participants, assess the outcomes, or write the reports are the same. Depending on the extent of blinding, RCTs can be classified as open, single-blind, double-blind, triple-blind, and quadruple-blind.
What is an open RCT? An open RCT is a randomised trial in which everybody involved in the trial knows which intervention is given to each participant. Most trials comparing different surgical interventions or comparing surgery with medication are open RCTs.
- What is a single-blind RCT? A single-blind RCT is a randomised trial in which one group of individuals involved in the trial does not know the identity of the intervention that is given to each participant.
- Usually it is the participants or the investigators assessing the outcomes who do not know the identity of the interventions.
Single blind designs are used frequently to evaluate educational or surgical interventions. For instance, investigators could evaluate different educational strategies in patients who would be unaware of the different types of strategies that are being compared.
- In addition, two surgical procedures could be compared under single-blind conditions with the use of identical wound dressings to keep investigators blind to the type of procedure during the assessment of the outcomes.
- What is a double-blind RCT? A double-blind RCT is a randomised trial in which two groups of individuals involved in the trial do not know the identity of the intervention that is given to each participant.
Usually, these two groups include the participants and the investigators in charge of assessing the outcomes of the interventions. To be successful, double-blinding requires that the interventions must be indistinguishable to both the participant and the investigator assessing the outcomes.
- Usually, the interventions are known to them just as A or B.
- When the experimental intervention is new and there are no standard effective interventions that could be used as controls, the investigators use an inert substance, or placebo, which has the same appearance and taste as the experimental intervention.
These double-blind RCTs, in which the control group receives a placebo, are also called double-blind, randomised, placebo controlled trials. When the RCT is designed to compare a new intervention with a standard treatment, the RCTs are called active-controlled.
Achieving double-blinding in active-controlled trials is often difficult and frequently requires the use of what is called a double dummy. In a double-blind, double dummy RCT, each group of participants receives one of the active interventions and a placebo (in this case called a dummy) that looks and tastes the same as the other intervention.
The double dummy technique is particularly useful when the investigators want to compare interventions that are administered by different routes or that require different techniques of administration. For instance, a double-blind, double dummy RCT would be the ideal study design to compare one intervention that is given as a tablet with another that is given by injection.
In such a trial, the participants in one of the study groups would receive a tablet with the active drug and a placebo injection, whereas the participants in the other group would receive a placebo tablet and an injection with the active drug. Problems with the way in which double-blinding is implemented, evaluated, and described in RCTs will be discussed in Chapters 3 and 4,
What is a triple-blind or quadruple-blind RCT? In a triple-blind RCT, three groups of individuals involved in the trial do not know the identity of the intervention that is given to each participant. These groups could include the participants, the investigators giving the intervention, and those evaluating the outcomes (if the latter two are different); or the participants, the investigators evaluating the outcomes, and the data analysts.
- If one more group is unaware of the identity of the intervention that is given to each participant, then the trial becomes a quadruple-blind RCT.
- This could easily be achieved by making the investigators who write the results of the trial unaware of the identity of the interventions until the time at which they complete the manuscript.
Even though it is easy to make a double-blind trial into a triple or quadruple-blind trial, it happens very rarely. RCTs that take into account the preferences of non-randomised individuals and participants Eligible individuals may refuse to participate in trials, either because they have a strong preference for one particular intervention (if there are several active interventions available) or because they do not want to receive a placebo.
- Other eligible individuals may decide to participate in a trial despite having a clear preference for one of the study interventions.
- The outcomes of these individuals, whether they enter the trial or not, may be different from those participants who do not have strong preferences.
- The outcomes of the individuals who do not participate in the trials or of those who participate and have strong preferences are rarely recorded.
There are at least three types of RCTs that take into account the preferences of eligible individuals, whether they take part in the trial or not. These trials are called preference trials, because they include at least one group in which the participants are allowed to choose their own preferred treatment from among several options offered.13, 14 These trials can have Zelen’s design, a comprehensive cohort design, or Wennberg’s design ( Fig 2.1 ).
What is a trial with Zelen’s design? In a trial with Zelen’s design, eligible individuals are randomised before they give consent to participate in the trial, to receive either a standard treatment or an experimental intervention. Those who are allocated to standard treatment are given the standard treatment and are not told that they are part of a trial, whereas those who are allocated to the experimental intervention are offered the experimental intervention and told that they are part of a trial.
If they refuse to participate in the trial, they are given the standard intervention but are analysed as if they had received the experimental intervention.15 The main advantages of Zelen’s design are that almost all eligible individuals are included in the trial and that the design allows the evaluation of the true effect of offering experimental interventions to patients.
The main disadvantages are that they have to be open trials and that the statistical power of the study may be affected if a high proportion of participants choose to have the standard treatment. To overcome the ethical concerns of not telling patients that they have been randomised to receive the standard treatment, the original approach proposed by Zelen can be modified by informing participants of the group to which they have been allocated and by offering them the opportunity to switch to the other group ( Fig 2.1a ).
This design is also known as double randomised consent design.16 Even though this modified design overcomes the ethical concerns of the original Zelen’s design, it does not solve the problems associated with lack of blinding and potential loss of statistical power.14 What is a trial with comprehensive cohort design? A comprehensive cohort trial is a study in which all participants are followed up, regardless of their randomisation status ( Fig 2.1b ).
In these trials, if a person agrees to take part in an RCT, he or she is randomised to one of the study interventions. If the person does not agree to be randomised because he or she has a strong preference for one of the interventions, that person will be given the preferred intervention and followed up as if he or she were part of a cohort study (see Chapter 7 ).16, 17 At the end, the outcomes of people who participated in the RCT can be compared with those who participated in the cohort studies to assess their similarities and differences.
This type of design is ideal for trials in which a large proportion of eligible individuals are likely to refuse to be randomised because they (or their clinicians) have a strong preference for one of the study interventions.16 In these cases, it could be said that the study is really a prospective cohort study with a small proportion of participants taking part in an RCT.16 One of the main limitations of this type of design is that any differences in outcomes may be explained by differences in the baseline characteristics of the participants in the randomised and non-randomised groups.18, 19 What is a trial with Wennberg’s design? In a trial with Wennberg’s design eligible individuals are randomised to a ‘preference group’ or an ‘RCT group’ ( Fig 2.1c ).
- Those individuals in the preference group are given the opportunity to receive the intervention that they choose, whereas those in the RCT group are allocated randomly to receive any of the study interventions, regardless of their preference.
- At the end of the study, the outcomes associated with each of the interventions in each of the groups are compared and used to estimate the impact of the participants’ preferences on the outcomes.
Preference trials are rarely used in health care research. They are, however, likely to become more frequently used as consumer participation in health care decisions and research increases. References 1. Sackett DL, Gent M. Controversy in counting and attributing events in clinical trials.
N Engl J Med 1979; 301 :1410-12.2. Fletcher RH, Fletcher SW, Wagner EH. Clinical epidemiology: the essentials, 3rd edn. Baltimore, MD: Williams & Wilkins, 1996.3, Armitage P, Berry G. Statistical methods in medical research, 3rd edn. Oxford: Blackwell Scientific, 1994.4. Pocock SJ. Clinical trials: a practical approach,
Chichester: Wiley, 1983.5. Louis TA, Lavori PW, Bailar JC III, Polansky M. Crossover and self-controlled designs in clinical research. In: Bailar JC III, Mosteller F, eds. Medical uses of statistics, 2nd edn. Boston, MA: New England Medical Journal Publications, 1992:83-104.6.
Han KS, Daya S, Collins JA, Walter SD. Empirical evidence of bias in infertility research: overestimation of treatment effect in crossover trials using pregnancy as the outcome measure. Fertil Steril 1996; 65 :939-45.7. Senn S. Cross-over trials in clinical research, Chichester: John Wiley & Sons, 1993.8.
Guyatt G, Sackett D, Taylor DW, Chong J, Roberts RS, Pugsley S. Determining optimal therapyrandomized trials in individual patients. N Engl J Med 1986; 314 :889-92.9.Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine, 2nd edn.
New York: Little, Brown & Company, 1991.10. Woods KL. Mega-trials and management of acute myocardial infarction. Lancet 1995; 346 :611-14.11. Charlton BG. Mega-trials: methodological issues and clinical implications. J R Coll Physicians Lond 1995; 29 :96-100.12. Altman DG. Practical statistics for medical research,
London: Chapman & Hall, 1991.13. Till JE, Sutherland HJ, Meslin EM. Is there a role for preference assessments in research on quality of life in oncology? Quality of Life Res 1992; 1 :31-40.14. Silverman WA, Altman DG. Patients’ preferences and randomised trials.
- Lancet 1996; 347 :171-4.15. Zelen M.
- A new design for randomized clinical trials.
- N Engl J Med 1979; 300 :1242-5.16.
- Olschewski M, Scheurlen H.
- Comprehensive cohort study: An alternative to randomized consent design in a breast preservation trial.
- Methods Inform Med 1985; 24 :131-4.17.
- Brewin CR, Bradley C.
Patient preferences and randomised clinical trials. BMJ 1989; 299 :684-5.18. Paradise JL, Bluestone CD, Rogers KD, Taylor FH, Colborn DK, Bachman RZ, Bernard BS, Schwarzbach RH. Efficacy of adenoidectomy for recurrent otitis media in children previously treated with tympanostomy-tube placement: Results of parallel randomized and nonrandomized trials.
Explanatory and pragmatic trials Efficacy and effectiveness trials Phase I, II, and III trials
RCTs according to how the participants are exposed to the interventions
Parallel trials Crossover trials Trials with factorial design
RCTs according to the number of participants
From n -of-1 to mega-trials Fixed size Sequential trials
RCTs according to whether the investigators and participants know which intervention is being assessed
Open trials Single blind trials Double blind trials Triple and quadruple-blind trials
RCTs according to whether the preferences of non-randomised individuals and participants are taken into account
Zelen’s design Comprehensive cohort design Wennberg’s design
What is real-world data science?
Introduction – Per the definition by the US FDA, real-world data (RWD) in the medical and healthcare field “are the data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources”. The wide usage of the internet, social media, wearable devices and mobile devices, claims and billing activities, (disease) registries, electronic health records (EHRs), product and disease registries, e-health services, and other technology-driven services, together with increased capacity in data storage, have led to the rapid generation and availability of digital RWD,
The increasing accessibility of RWD and the fast development of artificial intelligence (AI) and machine learning (ML) techniques, together with rising costs and recognized limitations of traditional trials, has spurred great interest in the use of RWD to enhance the efficiency of clinical research and discoveries and bridge the evidence gap between clinical research and practice.
For example, during the COVID-19 pandemic, RWD are used to generate or aid the generation of real-world evidence (RWE) on the effectiveness of COVID-19 vaccination, to model localized COVID-19 control strategies, to characterize COVID-19 and flu using data from smartphones and wearables, to study behavioral and mental health changes in relation to the lockdown of public life, and to assist in decision and policy making, among others.
What is RWE in Germany?
Operations – RWE operates in Asia-Pacific, Europe and the United States. The group is organized around four core areas:
- Offshore Wind
- Onshore Wind/Solar
- Hydro/Biomass/Gas
- Supply & Trading
In addition to these core areas, there is a Coal/Nuclear segment. In the UK, RWE fully owns RWE Generation UK plc., which operates a number of natural gas and renewable energy power stations across the UK. RWE’s last coal-fired power station in the UK, Aberthaw Power Station in South Wales, was closed in March 2020.
The company owns the gas-fired power stations Staythorpe in Nottinghamshire, Pembroke in West Wales, Little Barford in Bedfordshire, Great Yarmouth in Norfolk, and Didcot ‘B’ Station in Oxfordshire. It has previously closed its Didcot ‘A’ Power Station in Oxfordshire, Littlebrook Power Station in Kent, Fawley Power Station in Hampshire, and Tilbury Power Station in Essex.
RWE jointly owns one third of the Urenco Group with E.ON, The remaining stakes are held by the British and Dutch governments, with one-third each.
What is real world insights?
How is QIAGEN Real-World Insights driving impact? – QIAGEN Real-World Insights (RWI) is a professional and personalized pharmaceutical development service dedicated to finding the fastest, most cost-effective route to move therapies from research to market. RWI provides access to real-world genomic data, helping pharmaceutical companies confidently answer critical questions, such as:
Which biomarkers to target? What variants to include? Which patients to select?