A pilot experiment, pilot study, pilot test or pilot project is a small-scale preliminary study conducted to evaluate feasibility, duration, cost, adverse events, and improve upon the study design prior to performance of a full-scale research project. [1] [2]
Pilot experiments are frequently carried out before large-scale quantitative research, in an attempt to avoid time and money being used on an inadequately designed project. A pilot study is usually carried out on members of the relevant population. [1] A pilot study is used to formulate the design of the full-scale experiment which then can be adjusted. [1] [2] The pilot study is potentially a critical insight to clinical trial design, recruitment and sample size of participants, treatment testing, and statistical analysis to improve the power of testing the hypothesis of the study. [2] Analysis from the pilot experiment can be added to the full-scale (and more expensive) experiment to improve the chances of a clear outcome. [1] [2]
In sociology, pilot studies can be referred to as small-scale studies that will help identify design issues before the main research is done. Although pilot experiments have a well-established tradition, their usefulness as a strategy for change has been questioned, at least in the domain of environmental management. [3] Extrapolation from a pilot study to large scale strategy may not be assumed as possible, partly due to the exceptional resources and favorable conditions that accompany a pilot study. [1]
In clinical research, studies conducted in preparation for a future randomized controlled trial are known as "pilot" and "feasibility" studies, where pilot studies are a subset of feasibility studies. [1] [4] A feasibility study asks whether the study should proceed, and if so, how. A pilot study asks the same questions, but also has a specific design feature: in a pilot study, a future study is conducted on a smaller scale, [1] [5] which, if having produced positive results, may lead to a Phase I clinical trial. [6] The use of pilot and feasibility studies to estimate treatment effect is controversial, with ongoing methodologic discussion about appropriateness. [7]
A checklist was published in 2016 to provide guidance on how to report pilot trials. [8]
In engineering, a pilot trial may be conducted to understand the design problems, learn the correct technique’s or to capture unknown requirements prior to building a prototype. It may use prototype parts or simply samples to see which are successful and which are not, prior to more significant development effort. A pilot can typically be differentiated from a prototype by being significantly different in build, if not in function i.e. it is not intended to be developed into the end product, but to learn how to design and build the end product successfully.
Evidence-based medicine (EBM) is "the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. ... [It] means integrating individual clinical expertise with the best available external clinical evidence from systematic research." The aim of EBM is to integrate the experience of the clinician, the values of the patient, and the best available scientific information to guide decision-making about clinical management. The term was originally used to describe an approach to teaching the practice of medicine and improving decisions by individual physicians about individual patients.
Meta-analysis is a method of synthesis of quantitative data from multiple independent studies addressing a common research question. An important part of this method involves computing a combined effect size across all of the studies. As such, this statistical approach involves extracting effect sizes and variance measures from various studies. By combining these effect sizes the statistical power is improved and can resolve uncertainties or discrepancies found in individual studies. Meta-analyses are integral in supporting research grant proposals, shaping treatment guidelines, and influencing health policies. They are also pivotal in summarizing existing research to guide future studies, thereby cementing their role as a fundamental methodology in metascience. Meta-analyses are often, but not always, important components of a systematic review.
A randomized controlled trial is a form of scientific experiment used to control factors not under direct experimental control. Examples of RCTs are clinical trials that compare the effects of drugs, surgical techniques, medical devices, diagnostic procedures, diets or other medical treatments.
Clinical trials are prospective biomedical or behavioral research studies on human participants designed to answer specific questions about biomedical or behavioral interventions, including new treatments and known interventions that warrant further study and comparison. Clinical trials generate data on dosage, safety and efficacy. They are conducted only after they have received health authority/ethics committee approval in the country where approval of the therapy is sought. These authorities are responsible for vetting the risk/benefit ratio of the trial—their approval does not mean the therapy is 'safe' or effective, only that the trial may be conducted.
In a blind or blinded experiment, information which may influence the participants of the experiment is withheld until after the experiment is complete. Good blinding can reduce or eliminate experimental biases that arise from a participants' expectations, observer's effect on the participants, observer bias, confirmation bias, and other sources. A blind can be imposed on any participant of an experiment, including subjects, researchers, technicians, data analysts, and evaluators. In some cases, while blinding would be useful, it is impossible or unethical. For example, it is not possible to blind a patient to their treatment in a physical therapy intervention. A good clinical protocol ensures that blinding is as effective as possible within ethical and practical constraints.
In published academic research, publication bias occurs when the outcome of an experiment or research study biases the decision to publish or otherwise distribute it. Publishing only results that show a significant finding disturbs the balance of findings in favor of positive results. The study of publication bias is an important topic in metascience.
Proof of concept, also known as proof of principle, is a realization of a certain idea, method or principle in order to demonstrate its feasibility, or viability, or a demonstration in principle with the aim of verifying that some concept or theory has practical potential. A proof of concept is usually small and may or may not be complete.
A systematic review is a scholarly synthesis of the evidence on a clearly presented topic using critical methods to identify, define and assess research on the topic. A systematic review extracts and interprets data from published studies on the topic, then analyzes, describes, critically appraises and summarizes interpretations into a refined evidence-based conclusion. For example, a systematic review of randomized controlled trials is a way of summarizing and implementing evidence-based medicine.
Drug development is the process of bringing a new pharmaceutical drug to the market once a lead compound has been identified through the process of drug discovery. It includes preclinical research on microorganisms and animals, filing for regulatory status, such as via the United States Food and Drug Administration for an investigational new drug to initiate clinical trials on humans, and may include the step of obtaining regulatory approval with a new drug application to market the drug. The entire process—from concept through preclinical testing in the laboratory to clinical trial development, including Phase I–III trials—to approved vaccine or drug typically takes more than a decade.
The Bristol stool scale is a diagnostic medical tool designed to classify the form of human faeces into seven categories. It is used in both clinical and experimental fields.
In science, randomized experiments are the experiments that allow the greatest reliability and validity of statistical estimates of treatment effects. Randomization-based inference is especially important in experimental design and in survey sampling.
In fields such as epidemiology, social sciences, psychology and statistics, an observational study draws inferences from a sample to a population where the independent variable is not under the control of the researcher because of ethical concerns or logistical constraints. One common observational study is about the possible effect of a treatment on subjects, where the assignment of subjects into a treated group versus a control group is outside the control of the investigator. This is in contrast with experiments, such as randomized controlled trials, where each subject is randomly assigned to a treated group or a control group. Observational studies, for lacking an assignment mechanism, naturally present difficulties for inferential analysis.
In epidemiology, reporting bias is defined as "selective revealing or suppression of information" by subjects. In artificial intelligence research, the term reporting bias is used to refer to people's tendency to under-report all the information available.
In natural and social science research, a protocol is most commonly a predefined procedural method in the design and implementation of an experiment. Protocols are written whenever it is desirable to standardize a laboratory method to ensure successful replication of results by others in the same laboratory or by other laboratories. Additionally, and by extension, protocols have the advantage of facilitating the assessment of experimental results through peer review. In addition to detailed procedures, equipment, and instruments, protocols will also contain study objectives, reasoning for experimental design, reasoning for chosen sample sizes, safety precautions, and how results were calculated and reported, including statistical analysis and any rules for predefining and documenting excluded data to avoid bias.
The experience sampling method (ESM), also referred to as a daily diary method, or ecological momentary assessment (EMA), is an intensive longitudinal research methodology that involves asking participants to report on their thoughts, feelings, behaviors, and/or environment on multiple occasions over time. Participants report on their thoughts, feelings, behaviors, and/or environment in the moment or shortly thereafter. Participants can be given a journal with many identical pages. Each page can have a psychometric scale, open-ended questions, or anything else used to assess their condition in that place and time. ESM studies can also operate fully automatized on portable electronic devices or via the internet. The experience sampling method was developed by Suzanne Prescott during doctoral work at University of Chicago's Committee on Human Development with assistance from her dissertation advisor Mihaly Csikszentmihalyi. Early studies that used ESM were coauthored by fellow students Reed W. Larson and Ronald Graef, whose dissertations both used the method.
The Jadad scale, sometimes known as Jadad scoring or the Oxford quality scoring system, is a procedure to assess the methodological quality of a clinical trial by objective criteria. It is named after Canadian-Colombian physician Alex Jadad who in 1996 described a system for allocating such trials a score of between zero and five (rigorous). It is the most widely used such assessment in the world, and as of May 2024, its seminal paper has been cited in over 24,500 scientific works.
A cluster-randomised controlled trial is a type of randomised controlled trial in which groups of subjects are randomised. Cluster randomised controlled trials are also known as cluster-randomised trials, group-randomised trials, and place-randomized trials. Cluster-randomised controlled trials are used when there is a strong reason for randomising treatment and control groups over randomising participants.
The phases of clinical research are the stages in which scientists conduct experiments with a health intervention to obtain sufficient evidence for a process considered effective as a medical treatment. For drug development, the clinical phases start with testing for drug safety in a few human subjects, then expand to many study participants to determine if the treatment is effective. Clinical research is conducted on drug candidates, vaccine candidates, new medical devices, and new diagnostic assays.
In medicine, a stepped-wedge trial is a type of randomised controlled trial (RCT). An RCT is a scientific experiment that is designed to reduce bias when testing a new medical treatment, a social intervention, or another testable hypothesis.
In an adaptive design of a clinical trial, the parameters and conduct of the trial for a candidate drug or vaccine may be changed based on an interim analysis. Adaptive design typically involves advanced statistics to interpret a clinical trial endpoint. This is in contrast to traditional single-arm clinical trials or randomized clinical trials (RCTs) that are static in their protocol and do not modify any parameters until the trial is completed. The adaptation process takes place at certain points in the trial, prescribed in the trial protocol. Importantly, this trial protocol is set before the trial begins with the adaptation schedule and processes specified. Adaptions may include modifications to: dosage, sample size, drug undergoing trial, patient selection criteria and/or "cocktail" mix. The PANDA provides not only a summary of different adaptive designs, but also comprehensive information on adaptive design planning, conduct, analysis and reporting.