Inlay

The idea that internal validity comes first and external validity comes after is a misconception—particularly in economics. In reality, research design involves a trade-off between the two, and the goal is to choose an optimal point on the validity frontier. www.degruyterbrill.com/document/doi...

The Credibility Revolution advances internally valid research designs intended to identify causal effects from quantitative data. The ensuing emphasis on internal validity, however, has enabled a neglect of construct and external validity. We show that ignoring construct and external validity within identification strategies undermines the Credibility Revolution’s own goal of understanding causality deductively. Without assumptions regarding construct validity, one cannot accurately label the cause or outcome. Without assumptions regarding external validity, one cannot label the conditions enabling the cause to have an effect. If any of the assumptions regarding internal, construct, and external validity are missing, the claim is not deductively supported. The critical role of theoretical and substantive knowledge in deductive causal inference is illuminated by making such assumptions explicit. This article critically reviews approaches to identification in causal inference while developing a framework called causal specification . Causal specification augments existing identification strategies to enable and justify deductive, generalized claims about causes and effects. In the process, we review a variety of developments in the philosophy of science and causality and interdisciplinary social science methodology.

Late to the conversation, but I'm kind of unbothered by this lack of replication. If the study is not internally valid to begin with, then being able to replicate it only means replicating systematic biases. And something tells me this is likely to be a bigger problem in the literature.