BAS/MESC: Best Available Science and Metrics for Evaluation of Scientific Claims

For several decades, the staff of the Institute for Regulatory Science has been involved in the development of a process to evaluate the validity of scientific claims. We chose the term Best Available Science (BAS) to generically sort out the validity of claims and counter claims. These efforts led to the development of Metrics for Evaluation of Scientific Claims (MESC). Our initial review of literature could not find the term “best available science”, much less a definition for it. Ultimately, our choice was based on the following:

1. Best connotes scientific information with the greatest degree of excellence and authenticity based on sound logic, which includes good judgment, reasoning, or evidence.  It indicates the highest level of reliability.  However, it suggests that there is also scientific information that does not meet the highest level of reliability.  It implies that on occasion a specific area of science has not reached a reasonable level of maturity.  There is an implication that one has to generically identify the level of reliability of that specific area of scientific information. Consequently, it implies the need for some sort of standard.
2. Available connotes scientific information that is accessible and attainable, i.e., capable of being obtained or used, based on No. 1 above.
3. Science excludes any subject that is not science.

During the evolution of BAS/MESC we recognized the need to identify basic principles that must be used in assessing scientific claims. We identified four principles that historically have led the scientific community in virtually all new findings.

Open-mindedness Principle implies the willingness to accept new knowledge.

Skepticism Principle requires that those who make a scientific claim must provide sufficient evidence supporting their claim.

Universal Scientific Principles imply that there are certain principles applicable to virtually all scientific disciplines.

Reproducibility Principle implies that any qualified investigator should be able to reproduce the claim.
 
The BAS concept and MESC derived from it are based on three important elements as follows:

1.   Standardization of scientific information
2.   Categorization of reliability of scientific information
3.   Areas outside the purview of science

1. STANDARDIZATION OF SCIENTIFIC INFORMATION

Scientific information can be generically classified into the following classes:

Class I – Proven Science

This class is the equivalent to scientific laws in the classical process.  In the BAS system, it consists of scientific laws—sometimes called scientific principles—application, and virtually proven science.
 
Class IA ‑ Confirmed Science:  Scientific laws and all other information that have been unequivocally confirmed and are generally accepted constitute Class IA.  It is recognized that each scientific law or scientific fact has its limitations and conditions for its validity.  The cornerstone of this class is that any investigator who has the proper equipment and the necessary skills can reproduce it. Similarly, this class of scientific information does not require assumptions or any other prerequisite for its proof. 

Class IB ‑ Applied Science:  This class consists of application of scientific laws to various branches of commerce and industry. The only prerequisite for information to be placed into this class is the correct interpretation of scientific laws. Much of the Engineering disciplines and a large segment of other applied sciences fall into this class.

Class IC – Virtually Proven:  This class consists of information whose reliability has been uncontested yet there is insufficient proof to be placed in the proven science class.  The process for placing scientific information into this class consists of: 1) ensuring that no reliable observation contradicts information in this class and 2) there is no alternative system that describes the observed phenomena. Possible examples of this class include both general and special relativity theories.  These theories attempt to describe the universe, forces operating in the universe, and their interrelationship.

Class II – Evolving Science

The overwhelming scientific advances in virtually all disciplines are evolving science. 
 
Class IIA – Reproducible Evolving Science:  Reliable information dealing with a subject that is not completely understood constitutes the core of this class. The key factor in placing information into this category is reproducibility.  Although the observations that are the foundation of this class may not necessarily have general applicability, they are clearly and unambiguously reproducible by those with appropriate skills and equipment.  Advancements in virtually all branches of science including physics, chemistry, biology, and many other scientific disciplines are based on the desire of investigators to improve knowledge in this class.

Class IIB – Extrapolated Science:  The scientific foundation of information placed in this class is derived from proven or evolving science.  However, it uses assumptions, extrapolations, and numerous similar processes in deriving its results and conclusions.   A characteristic of this class is that the investigator who is trying to reproduce this class must not only have proper skills and the necessary equipment, but must also accept the asserted scientific foundation, assumptions, choice of mathematical processes, default data, and numerous other prerequisites.  Most predictive models and a large segment of contested scientific information fall into this class.

Class IIB – Hypothesized Science:

This class consists of an organized response to an observation, an idea, or any other initiating thought process.  Imagine that an event is observed by a scientist who cannot explain it by any known scientific principles, approaches, or other processes. Subsequently, the scientist hypothesizes one or more potential causes and communicates the observation and the hypothesized cause to the scientific community. As this and numerous other examples indicate, the information in this class is unique and requires attention by the scientific community. However, it is imperative to recognize that any conclusion or societal decision based on this class is likely to cause undue burden for not only the individuals involved but also for the society as a whole.

Class III – Borderline Science

As the title implies, this class is not necessarily science.  The closest part in the classical process to this class is hypothesis.

Class IIIA ‑ Scientific Judgment:  On occasion, information is provided to the society that lacks scientific foundation. Similarly, often decisions must be made without having the needed scientific information including basic principles, the necessary data, and other scientific requirements.  The methodology for expert judgment is reasonably well-developed and consists of asking a number of presumably knowledgeable individuals to give answers to specific questions and statistically assess the results.  However, this class is often an educated guess.

Class IIIB ‑ Speculation:  This class consists of information that cannot meet standards described in any of the above classes. It is often based on the intuition of an individual who wants to stimulate a discussion or initiate a research project.

Fallacious Information

Information that clearly falls into the purview of science but is inconsistent with the three classes identified above is fallacious information. This information is often called “pseudo science”, “junk science”, or “politically-processed science”. 

2.  CATEGORIZATION OF RELIABILITY OF SCIENTIFIC INFORMATION

There are rational and reasonable uncontested methods to resolve scientific controversies.  Briefly, scientific information is divided into the following four distinct categories:

Category 1 - Personal Opinions:  Expression of views by individuals regardless of their training, experience, and social agenda are included in this group.  Personal opinions are seldom, if ever, BAS.  At best, this category can be used to initiate the study of a scientific issue.  Note that the standard process of the news media is reliance upon this category in its reporting of scientific issues.

Category 2 - Gray Literature:  Written information prepared by government agencies, advocacy groups, and others that has not been subjected to an independent peer review is included in this category.  This is the favorite category of government agencies, advocacy groups, and individuals who want to promote an idea.  In fact, this category is the more organized and written form of personal opinions.  Again here, at best, this category should be used to initiate a study.  Experience shows that in the overwhelming majority of cases this category does not meet the requirements of scientific acceptability.

Category 3 - Peer‑Reviewed Science:  Information subjected to an independent peer review constitutes this category.  Peer review is the foundation of scientific acceptability.  There are numerous requirements for acceptability of peer review.  Briefly, the individual who is chosen as a reviewer must be a “peer” to the author of the study, and must have no conflict of interest.  In addition, the author of the study must respond to criticism by the peer to the satisfaction of an uninvolved person or organization.

Category 4 - Consensus-Processed Science:  This category consists of information resulting from a process used to resolve scientific disputes.  The prerequisite for this process is the formation of a group of peers under the auspices of an organization that is uniquely qualified to do so. Category 4 is a formal process requiring consideration of qualifications of the members of the consensus panel and their potential conflict of interest, which is contrary to the view that is often expressed.

3. AREAS OUTSIDE THE PURVIEW OF SCIENCE

 The concept of BAS deals with classification of scientific information and evaluation of scientific assertions.  It specifically excludes areas that are outside of science. The inclusion of faith, ideology, beliefs, or any other non-scientific objectives in assessing the validity of scientific information is inconsistent with the foundation of BAS or MESC.

APPLICATION OF BAS/MESC

One of the key applications of BAS/MESC is a topic that was recently brought to the attention of the staff of the RSI.  The media, certain policy makers, many advocacy organizations, and others claim that an issue is settled; the science has spoken; the toxicity of agent is scientifically proven; science has established the benefit of a medical procedure; and numerous others. According to the BASS/MESC

A scientific issue is only then settled if any investigator who has the necessary skills and, depending upon the nature of the issue, relevant equipment and facilities is able to reproduce and confirm it.

About RSI

HOME
OUR MISSION
CONTACT US
CENTERS

Science at RSI

BAS/MESC
What is Regulatory Science
Voice of Science

Centers

TIMC
ICESH
RAC
SESC
STARC