Multiplicative Interaction Models - Matt Golder


Articles

  1. William Berry, Matt Golder, & Daniel Milton. 2012. "Improving Tests of Theories Positing Interaction." Journal of Politics 74: 653-671. [replication materials][online appendix]
  2. Abstract: It is well established that all interactions are symmetric when the effect of X on Y is conditional on the value of Z, the effect of Z must be conditional on the value of X. Yet the typical practice when testing an interactive theory is to (1) view one variable, Z, as the conditioning variable, (2) offer a hypothesis about how the marginal effect of the other variable, X, is conditional on the value of Z, and (3) construct a marginal effect plot for X to test the theory. We show that the failure to make additional predictions about how the effect of Z varies with the value of X, and to evaluate them with a second marginal effect plot, means that scholars often ignore evidence that can be extremely valuable for testing their theory. As a result, they either underestimate or, more worryingly, overestimate the support for their theories.

  3. Thomas Brambor, William Roberts Clark, & Matt Golder. 2006. "Understanding Interaction Models: Improving Empirical Analyses." Political Analysis 14: 63-82. [literature survey][replications]
  4. Abstract: Multiplicative interaction models are common in the quantitative political science literature. This is so for good reason. Institutional arguments frequently imply that the relationship between political inputs and outcomes varies depending on the institutional context. Models of strategic interaction typically produce conditional hypotheses as well. Although conditional hypotheses are ubiquitous in political science and multiplicative interaction models have been found to capture their intuition quite well, a survey of the top three political science journals from 1998 to 2002 suggests that the execution of these models is often flawed and inferential errors common. We believe that considerable progress in our understanding of the political world can occur if scholars follow the simple checklist of dos and don'ts for using multiplicative interaction models presented in this article. Only 10% of the articles in our survey followed the checklist.

  5. William Roberts Clark, Michael Gilligan, & Matt Golder. 2006. "A Simple Multivariate Test for Asymmetric Hypotheses." Political Analysis 14: 311-331. [replication materials]

Abstract: In this paper, we argue that claims of necessity and sufficiency involve a type of asymmetric causal claim that is useful in many social scientific contexts. Contrary to some qualitative researchers, we maintain that there is nothing about such asymmetries that should lead scholars to depart from standard social science practice. We take as given that deterministic and monocausal tests are inappropriate in the social world and demonstrate that standard multiplicative interaction models are up to the task of handing asymmetric causal claims in a multivariate, probabilistic manner. We illustrate our argument with examples from the empirical literature linking electoral institutions and party system size.

Below you will find various information related to these articles. If you find any of this information useful, my co-authors and I would be grateful if you could cite the relevant paper(s).

 

Recommendations

Five Key Predictions

In "Improving Tests of Theories Positing Interaction", my co-authors and I recommend that scholars try to make as many of the following five predictions as possible when evaluating a simple interaction model such as:

Y = β0 + βxX + βzZ + βxzXZ + ε

  1. The marginal effect of X is [positive, negative, zero] when Z is at its lowest level.
  2. The marginal effect of X is [positive, negative, zero] when Z is at its highest level.
  3. The marginal effect of Z is [positive, negative, zero] when X is at its lowest level.
  4. The marginal effect of Z is [positive, negative, zero] when X is at its highest level.
  5. The marginal effect of each of X and Z is [positively, negatively] related to the other variable.

All five predictions can be subsumed in a single hypothesis about how the marginal effect of X varies with Z and a single hypothesis about how the marginal effect of Z varies with X. For example, you might have:

Specification and Interpretation

In "Understanding Interaction Models: Improving Empirical Analyses", my co-authors and I make several recommendations about the general specification and interpretation of interaction models:

  1. Use multiplicative interaction models whenever the hypothesis they want to test is conditional in nature.
  2. Include all constitutive terms in the model specification.
  3. Do not interpret the coefficients on constitutive terms as if they are unconditional marginal effects.
  4. Do not forget to calculate substantively meaningful marginal effects and standard errors.

In "Improving Tests of Theories Positing Interaction", my co-authors and I discuss how to interpret several different prototypical sets of results -- see pages 7-10.

 

 

Marginal Effect Plots

In "Improving Tests of Theories Positing Interaction" my co-authors and I make several recommendations about the construction of marginal effect plots:

  1. Conditional theories are typically strong enough to generate predictions about how the marginal effect of X on Y varies with Z and how the marginal effect of Z on Y varies with X. In these cases, scholars should evaluate their predictions by constructing marginal effect plots for both X and Z.
  2. The horizontal axis of a marginal effect plot should extend from the minimum observed value in the sample for the variable being plotted to the maximum observed value.
  3. A frequency distribution for the variable on the horizontal axis should be superimposed over each marginal effect plot. Although it depends to some extent on the context, we believe that a combination of a histogram and a rug plot has many virtues.
  4. Report the estimated product term coefficient along with its t-ratio or standard error somewhere in each marginal effect plot.

An example that follows these recommendations is shown below [replication code][detailed explanation of code]

 

 

 

Computer Code

Here you can find Stata code for producing a marginal effect plot for one of the interacting variables, in this case X, based on three different types of multiplicative interaction models. It should be easy to adapt this code to deal with other types of interaction models. Before looking at these specific examples, I suggest examining the recommendations for marginal effect plots in general by clicking here.

 

Marginal Effect Plot for X: An Interaction Between X and Z

[Code][Detailed Explanation of Code]

The following example is a marginal effect plot for X based on the results from a linear-interactive model taking the following basic form:

Y = β0 + β1X + β2Z + β3XZ + ε

 

"Marginal Effect" Plot for X: An Interaction Between X and Z in a Probit Model

[Code][Detailed Explanation of Code]

The following example is a "marginal effect" plot for X based on the results from a probit interaction model taking the following basic form:

Pr(Y=1) = Φ (β0 + β1X + β2Z + β3XZ + ε)

The code for this example can easily be modified to produce "marginal effect" plots for other non-linear models such as duration models, count models, ordered models etc. See the detailed explanation of the code.

 

Marginal Effect Plot for X: An Interaction Between X, Z, and W

[Code][Detailed Explanation of Code]

The following example is a marginal effect plot for X based on the results from a linear-interactive model taking the following basic form:

Y = β0 + β1X + β2W + β3Z + β4XW + β5XZ + β6WZ + β7XWZ + ε

 

 

Standard Errors

The document below lists two tables illustrating a variety of multiplicative interaction models, marginal effects, and standard errors. The tables are based on those in Aiken and West (1991).


 

 

Survey of the Literature

In "Understanding Interaction Models: Improving Empirical Analyses" my co-authors and I conducted a systematic examination of three leading, non-specialized political science journals (American Journal of Political Science, American Political Science Review, Journal of Politics) from 1998 to 2002. During the five year period from 1998 to 2002 we found 149 articles that employed interaction models of one variety or another. We coded each article for whether they implemented the four recommendations that we made in our article. A summary of our results are shown in the table below.

Recommendation
Yes
No
Total

Include all constitutive terms
107 (69%)
49 (31%)
156
Interpret constitutive terms correctly*
38 (38%)
63 (62%)
101
Provide range for marginal effect
86 (55%)
70 (45%)
156
Provide measure of uncertainty
34 (22%)
122 (78%)
156
* Only 101 articles interpreted constitutive terms

'Include all constitutive terms' is self-explanatory. 'Interpret constitutive terms correctly' means not interpreting the coefficients on constitutive terms as unconditional marginal effects. 'Provide range for marginal effect' and 'Provide measure of uncertainty' require the analyst to calculate the marginal effect for some independent variable for at least one value of the relevant modifying variable other than zero and provide some measure of uncertainty such as a standard error or confidence interval. We were very liberal on these last two criteria and coded articles that reported predicted probabilities under two or more different scenarios as having met our recommendations even though these are not marginal effects or first differences (or the quantities of interest). If we had not done this, considerably fewer articles would have been coded as having implemented our recommendations.

A complete list of these articles, along with additional details, can be found by clicking here (Sample). We do not mean to suggest that the conclusions reached in any of these articles are necessarily wrong. After all, we have not conducted detailed reanalyses of all of these studies. However, we do believe that there is a potential for some conclusions in these articles to be incorrect. This is why we encourage people to conduct replications of these studies.

Although my co-authors and I went to great pains to avoid any errors in our survey, we are only human and errors may remain. In some cases, it was hard to code particular articles because it was not always clear how certain variables were constructed or what model specification was actually used. If your article is included in our sample and you believe that it has been erroneously classified, we would be happy to hear from you. Our goal in conducting this survey is not to cause offence but to improve future empirical research. As a result, we will immediately correct any errors. Please email Thomas Brambor, William Roberts Clark, or Matt Golder.

 

 

Replications

In the survey of the literature conducted in "Understanding Interaction Models: Improving Empirical Analyses", my co-authors and I listed several articles that estimated multiplicative interaction models that omit at least one constitutive term, interpret the coeffcient on constitutive terms as unconditional marginal effects, or fail to calculate marginal effects and standard errors across a substantively meaningful range of the modifying variable(s). We believe that these types of mistakes are so common in the literature that analysts should critically re-evaluate, and where necessary re-specify, models employing interaction terms before using their results as the basis for future research. Substantively different conclusions from those in the original analyses often arise when this is done.

Below we present several replications of analyses using multiplicative interaction models that we have conducted in the course of our own research. We hope that others will also conduct replications of other analyses using such models and will add them to the list below. This might be a useful assignment for a first or second semester class in quantitative methods. If anyone would like to do this, they should send the replication to Matt Golder at mgolder@psu.edu.


Home Page