![]() ![]() |
|
PresidentOur Centers |
BEST AVAILABLE SCIENCE: METRICS FOR EVALUATION OF SCIENTIFIC CLAIMSFor several decades, the staff of the Institute for Regulatory Science has been involved in the development of a process to evaluate the validity of scientific claims. We chose the term Best Available Science (BAS) to generically sort out the validity of claims and counter claims. These efforts led to the development of Metrics for Evaluation of Scientific Claims (MESC). Our initial review of literature could not find the term “best available science”, much less a definition for it. Ultimately, our choice was based on the following:
During the evolution of BAS/MESC we recognized the need to identify basic principles that must be used in assessing scientific claims. We identified four principles that historically have led the scientific community in virtually all new findings: Open-mindedness Principle implies the willingness to accept new knowledge. Skepticism Principle requires that those who make a scientific claim must provide sufficient evidence supporting their claim. Universal Scientific Principles imply that there are certain principles applicable to virtually all scientific disciplines. Reproducibility Principle implies that any qualified investigator should be able to reproduce the claim.
PILLAR: STANDARDIZATION OF SCIENTIFIC INFORMATION Scientific information can be generically classified into the following classes: Class I – Proven Science This class consists of scientific laws sometimes called scientific principles and all other information that have been unequivocally confirmed and are generally accepted. The cornerstone of this class is that any investigator who has the proper equipment and the necessary skills can reproduce it. Similarly, this class of scientific information does not require assumptions or any other prerequisite for its proof. This class also includes the application of scientific laws to various branches of commerce and industry— provided the laws are correctly interpreted. Class II – Evolving Science The overwhelming scientific advances in virtually all disciplines are evolving science. Partially Reproducible Science: Formerly referred to as Rationalized Science or Scientific Extrapolation, this class includes certain segments of regulatory science information including predictive models. Although it builds upon Proven or Reproducible Evolving Science, it uses assumptions, extrapolations, and default data to derive its results. An important characteristic of this class is its level of reproducibility. Whereas the scientific foundation of this class meets the Reproducibility Principle, the choice of assumptions, mathematical processes, default data, and numerous other prerequisites are inherently arbitrary and thus are not necessarily reproducible. Evidence-Based Science: This class attempts to correlate systematic observations performed in accordance with Universal Scientific Principles to an effect. There is extensive literature covering this class including a large segment of epidemiology. Experience shows that correlation does not necessarily imply causation and as expected, some correlations have correctly identified their cause but others have proven to be unrelated. Much of evidence-based medicine belongs to this class. Hypothesized Science: This class consists of an organized response to an observation, an idea, or any other initiating thought process. Imagine that an event is observed by a scientist who cannot explain it by any known scientific principles, approaches, or other processes. Subsequently, the scientist hypothesizes one or more potential causes. Whereas some Hypothesized Science has proven to be worth pursuing, often the conclusions are proven to be wrong. Scientific Judgment: On occasion, decisions must be made without having the needed scientific information including basic principles, the necessary data, and other scientific requirements. The methodology for expert judgment is reasonably well-developed, and consists of asking a number of presumably knowledgeable individuals to give answers to specific questions and statistically assess the results. However, this class is often an educated guess. Speculation: This class consists of information that cannot meet standards described in any of the above classes. It is often based on the intuition of an individual who wants to stimulate a discussion or initiate a research project. Fallacious Information Information that clearly falls into the purview of science but is inconsistent with the three classes identified above is fallacious information. This information is often called “pseudo science”, “junk science”, or “politically-processed science”. PILLAR: CATEGORIZATION OF RELIABILITY OF SCIENTIFIC INFORMATION There are rational and reasonable uncontested methods to resolve scientific controversies. Briefly, scientific information is divided into the following four distinct categories: Category 1 - Personal Opinions: Expression of views by any individual is included in this group. At best, this category can be used to initiate the study of a scientific issue. Category 2 - Gray Literature: Written information that has not been subjected to an independent peer review is included in this category. This is the favorite category of government agencies, advocacy groups, and individuals who want to promote an idea. Experience shows that in the overwhelming majority of cases, this category does not meet the requirements of scientific acceptability. At best, this category should be used to initiate a study. Category 3 - Peer‑Reviewed Science: Information subjected to an independent peer review constitutes this category. Peer review is the foundation of scientific acceptability. There are numerous requirements for acceptability of peer review. Briefly, the individual who is chosen as a reviewer must be a “peer” to the author of the study, and must have no conflict of interest. In addition, the author of the study must respond to criticism by the peer to the satisfaction of an uninvolved person or organization. Category 4 - Consensus-Processed Science: This category consists of information resulting from a process used to resolve scientific disputes. The prerequisite for this process is the formation of a group of peers under the auspices of an organization that is uniquely qualified to do so. Category 4 is a formal process requiring consideration of qualifications of the members of the consensus panel and their potential conflict of interest, which is contrary to the view that is often expressed. PILLAR: AREAS OUTSIDE THE PURVIEW OF SCIENCE
|
|||
About RSI |
Science at RSI |
Centers
|
Publications Regulatory Science and Technology |
|||
© Copyright 2011 - RSI: Institute for Regulatory Science Webmaster: Johnson Ventures of Virginia, LLC |
||||||