Response to: Adam B. Cox & Thomas J. Miles, Judging the Voting Rights Act, 108 Colum. L. Rev. 1 (2008).
One notable difference between early empirical legal scholarship and the more recent sophisticated contributions to the literature is scholars' goal of identifying cause and effect relationships. Professors Cox and Miles's recent study of judicial decisionmaking provides a terrific example of this new-generation work.1 The authors investigate whether personal attributes such as ideology, race, or gender cause judges to favor (or disfavor) plaintiffs' claims under section 2 of the Voting Rights Act. The study is a valuable contribution to the emerging body of empirical scholarship exploring causal relationships, and to the work on judicial decisionmaking and voting rights litigation in particular.2
Causal inference, as opposed to making claims about mere correlations, is, of course, an ambitious undertaking. Investigators must spend time and energy exploring the underlying relationship between and among the variables of interest in order to identify possible bias and confounding in their data and, importantly, to address these perceived problems with appropriate conceptual and statistical methods.3 If bias and confounding exist but are not—or cannot be—remedied, scholars must exercise humility in reporting empirical results: They may point to interesting correlations in the data, but causal claims would be completely unjustified.
In this Response, we use Professors Cox and Miles's study of judicial decisionmaking to explore what is at stake when legal scholars present empirical findings without fully investigating the structural relationships of their data, or without explicitly stating the assumptions they make in order to draw causal inferences. We do not intend merely to identify the limitations of Cox and Miles's work (and by implication, those of many other empirical studies published in the extant legal literature); rather, we plan to introduce a new methodology that is intuitive, easy to use, and, most importantly, allows scholars to systematically assess problems of bias and confounding. This methodology—known as causal directed acyclic graphs—will help empirical researchers identify true cause and effect relationships when they exist, and at the same time posit statistical models with appropriate controls, in order to better justify causal claims. While this methodology has become popular in a number of disciplines—including statistics, biostatistics, epidemiology, and computer science—and is widely believed to be a valuable tool for empirical research, it has yet to appear in the empirical law literature. Accordingly, our goal is to offer a brief introduction of the method and to initiate discussion as to its worth in empirical legal studies.
I. Confounding in Cox and Miles's Study
We begin in Part I.A by briefly outlining Cox and Miles's study of judicial decisionmaking in the Voting Rights Act context. Then, in Part I.B, we note that empiricists seeking to make causal claims must address the potential problems of bias and confounding by exploring the underlying relationships that exist between and among the variables in their study. Cox and Miles, like most empiricists, did not explicitly clarify their assumptions with respect to these relationships, but we note that their modeling approach suggests they believe the variables of interest are independent—a very strong assumption that is not likely to be warranted given our knowledge and understanding of the real world. For this reason, we believe the authors' results suffer from confounding, and thus causal inferences are inappropriate. Finally, in Part I.C, we develop an alternative approach for exploring the effects of judicial attributes on voting. When we compare the results of the two strategies, we find that Cox and Miles appear to have both over- and underestimated the effects of judicial attributes on decisionmaking due to the problems of bias left unaddressed in their model.
A. Cox and Miles's Study of Decisionmaking in the Voting Rights Context
Professors Cox and Miles investigate the effects of a judge's personal attributes on judicial decisionmaking in the voting rights context and provide the first systematic evidence that both ideology and race are closely related to pro-plaintiff outcomes. Specifically, the authors' study indicates that Democratic judges and African American judges are more likely than Republican or white judges to find liability under section 2 of the Voting Rights Act, but that age and gender exert no such effects. In specifying their statistical models, and in making these empirical claims, Cox and Miles seem to have assumed that the effects of ideology and race on judicial outcomes are free of confounding—a very strong assumption, and one we believe their data does not support. If confounding indeed exists, then Cox and Miles's empirical results are likely to be biased. Our purpose in writing this Response, as noted above, is not to quibble about the precise size and direction of the effects presented by the authors, but rather to introduce a new methodology for systematically identifying and addressing confounding variables, the primary source of the empiricists' problems in the estimation process, and illustrate how it could have improved Cox and Miles's analysis.
Before we begin our re-analysis of Cox and Miles's data and introduce the new methodology, however, we would like first to note that we were generally very impressed with Cox and Miles's empirical study; indeed, we believe the methodology introduced here supports many of the authors' qualitative findings. While we ultimately find that the authors overstated some of their results, others are actually strengthened by our approach. Further, applying causal directed acyclic graphs allows numerous other interesting causal relationships to emerge that were hidden by Cox and Miles's methodological strategy. In short, causal directed acyclic graphs not only allow for more precise estimates, they can also bolster an investigator's empirical claims.
B. Hidden Assumptions About Data Can Lead to Unjustified Causal Claims
Causal inference requires empirical researchers not only to identify the nature of the relationship between and among the variables under investigation, but also to determine how possible confounders fit within the framework. Confounders are variables that affect both the outcome and the explanatory variable and, of course, can lead to biased (even spurious) empirical findings if not appropriately addressed. In Cox and Miles's study, the authors hope to explain judicial votes (the outcome variable) with ideology (the explanatory variable), and thus they must search for additional variables that could lead them to report spurious correlation between these two variables. For example, if race, gender, or age directly affects judges' political preferences as well as the propensity to cast a pro-plaintiff vote, then failure to account for these variables will induce bias in Cox and Miles's estimate of the outcome of interest (ideology).
In order to explore the underlying structure of the data for purposes of identifying possible confounders, it is useful to construct a diagram. While we defer discussing the formal rules and principles for devising such a diagram until later in Part II, we find it illustrative at this point to visually chart some of the possible sources of bias in Cox and Miles's analysis. First, consider Figure 1, depicting one possible set of relationships for Cox and Miles's data. Figure 1 indicates that race, gender, and age do not confound the effects of ideology on judicial voting because none of these variables affects both ideology and judicial decisions; the variables all affect judicial decisions, but not each other. Ideology, race, gender, and age are, therefore, independent of each other but causally related to judicial decisions. Thus, Cox and Miles would not need to adjust for interdependence between these characteristics in order to generate unbiased estimates of the effect of the ideology on the likelihood of a judge voting for liability under section 2 of the Voting Rights Act.
Figure 1: A Simple Depiction of the Underlying Structure of Some Variables in Cox and Miles's Data

Figure 2, however, is also a possible depiction of some of the underlying relationships of interest. Numerous studies have suggested that personal characteristics—including race, gender, and age—correlate with one's political preferences, and that these factors are also likely to affect judicial decisions.4 If this is indeed true, Cox and Miles must include background characteristics in their statistical model for purposes of identifying the true effects of ideology. Absent these controls, any uncovered effects are likely to be overestimated, underestimated, or, possibly, entirely spurious.
Figure 2: A More Complex Depiction of the Underlying Structure of Some Variables in Cox and Miles's Data

Cox and Miles, like all empirical researchers, must make assumptions about their data before estimating causal relationships and reporting empirical results. The authors, in short, must presume that Figure 1, Figure 2, or some other set of relationships exists when specifying a model and interpreting the estimates from a regression as to the effect of ideology on judicial votes. Our point, however, is more fundamental: While empirical researchers must, and always do, make assumptions about their data, these assumptions are almost always left unstated. As a result, consumers of the legal literature are left with two options: (1) interpret results in a manner that presupposes authors have made good assumptions about their data and have addressed any and all problems of confounding and bias; or (2) parse the models presented in an effort to identify whether the authors have inadvertently excluded confounding variables, and then interpret results accordingly. If legal empiricists fully recognized that all modeling exercises presuppose certain relationships between and among the variables of interest and, in turn, sought to make their assumptions about the data transparent, it would be reasonable for readers to choose option (1); otherwise readers must choose option (2).
C. Reestimation of Cox and Miles's Data to Account for Confounding
While Cox and Miles do not explicitly posit a theory about the underlying structure of their data, their interpretation of their empirical analyses seems to assume that race, gender, and age do not affect ideology. To see this, consider Table 1 below, which reproduces the authors' empirical results.5
Table 1: Cox and Miles's Table 5, Columns (1)–(3), Indicating the Likelihood of Voting For Section 2 Liability: Probit Regression Analysis Focusing on Political Affiliation
|
Variable |
(1)
Cox & Miles Table 5, col. 1 |
(2)
Cox & Miles Table 5, col. 2
|
(3)
Cox & Miles Table 5, col. 3 |
|
Judge was Democratic Appointee |
.145** (.035) |
.151** (.037) |
.158** (.044) |
|
Judge was Democratic Appointee * Year Was After 1994 |
- |
- |
-.021 (.072) |
|
Year Was After 1994 |
-.123** (.050) |
- |
- |
|
Case Occurred in South |
.016 (.057) |
- |
- |
|
Appellate Case |
-.084 (.051) |
-.102* (.053) |
-.103* (.053) |
|
Challenge to At-large Election |
.104 (.070) |
.078 (.069) |
.077 (.070) |
|
Challenge to Reapportionment Plan |
.054 (.073) |
.034 (.073) |
.034 (.073) |
|
Challenge to Local Election Practice |
.005 (.059) |
-.018 (.062) |
-.018 (.062) |
|
Plaintiffs Were African-American |
.027 (.068) |
.114* (.064) |
.114** (.064) |
|
Case Occurred in Jurisdiction Covered by §5
|
.046 (.063) |
.045 (.069) |
.045 (.068) |
Notes: * indicates significant at 10% level; ** indicates significant at 5% level. With the exception of Model (1), all regressions include fixed-effects controls for judicial circuits and years. Standard errors in parentheses.
As Table 1 indicates, Cox and Miles used three different models to estimate the effects of ideology on judicial voting; each model includes a variety of case characteristics—such as whether the plaintiff challenged an at-large election or reapportionment. Additionally, Model (1) includes binary variables indicating whether the case occurred in the South and whether the case occurred after 1994. Models (2) and (3) exclude the binary variables for geography and era, but include fixed effects for circuit and year; Model (3) also includes an interaction term between a judge's ideology and era.6 Presumably, the authors rely on these three different models in an effort to account for a variety of possible confounders, but they all fail to account for possible confounding problems associated with the race, gender, and age of the judge rendering a decision in the case. Variables measuring these judicial attributes are present in their dataset, but the authors did not take advantage of this data when estimating the effect of ideology.7
Given our claim that these background characteristics affect both ideology and judicial decisions, as depicted in Figure 2, we believe every one of the coefficients for ideology reported by Cox and Miles and reproduced in Table 1 is confounded, and thus causal inference is not warranted. Accordingly, we refit the data to the same three models used by Cox and Miles but included the confounding variables race, gender, and age, and present these new estimates in Table 2 below.
Table 2: Reestimation of Data in Cox and Miles's Table 5, Columns (1)–(3), to Account for Confounding
|
Variable |
(1a)
VanderWeele & Staudt Model |
(2a)
VanderWeele & Staudt Model |
(3a)
VanderWeele & Staudt Model |
|
Judge was Democratic Appointee |
.121** (.03) |
.104** (.04) |
.099* (.05) |
|
Judge was Democratic Appointee * Year Was After 1994 |
|
|
.01 (.08) |
|
Year Was After 1994 |
-.14** (.03) |
|
|
|
Case Occurred in South |
.01 (.04) |
|
|
|
Appellate Case |
-.09** (.04) |
-.11** (.04) |
-.11** (.04) |
|
Challenge to At-large Election |
.08* (.04) |
.06 (.05) |
.06 (.05) |
|
Challenge to Reapportionment Plan |
.03 (.05) |
.008 (.05) |
.008 (.05) |
|
Challenge to Local Election Practice |
.005 (.04) |
-.02 (.04) |
-.02 (.04) |
|
Plaintiffs Were African-American |
.01 (.04) |
.11** (.04) |
.11** (.04) |
|
Case Occurred in Jurisdiction Covered by §5
|
.05 (.04) |
.03 (.05) |
.03 (.05) |
|
Judge's Age
|
.003** (.001) |
.005** (.002) |
.005** (.002) |
|
Judge's Race |
.27** (.08) |
.36** (.09) |
.36** (.09) |
|
Judge's Gender |
.003 (.05) |
.02 (.06) |
.02 (.06) |
|
Judge Attended Ivy League College |
-.003 (.05) |
.02 (.05) |
.02 (.05) |
|
Judge Attended Elite Law School |
-.07* (.04) |
-.07* (.04) |
-.07 (.04) |
|
Judge Previously Served as Law Clerk |
.09* (.05) |
.08 (.05) |
.08 (.05) |
Notes: * indicates significant at 10% level; ** indicates significant at 5% level. With the exception of Model (1a), all regressions include fixed-effects controls for judicial circuits and years. Standard errors in parentheses.
Columns 1(a), 2(a), and 3(a) of Table 2 correspond to Cox and Miles's findings presented in Table 1, columns 1, 2, and 3 respectively, but differ in that we include race, gender, age, and education in our reestimation process. Although we have been focusing on the possible bias induced by excluding race, gender, and age for purposes of this discussion, we believe various other confounders of the relationship between ideology and judicial decision may also exist, such as education, and therefore we include them in the regression models used to derive Table 2.8
The first thing to note about Table 2 is that, at least in the voting rights context, the qualitative conclusions about the effects of ideology are robust to various sets of controls, which suggests that Cox and Miles's findings are not spurious, but may nonetheless be over- or underestimated.9
Indeed, comparing Table 1 with Table 2 suggests that Cox and Miles's approach inflated the size of the effects of ideology. After including the proper controls, we obtained estimates that were 2.4 to 5.9 percentage points lower than those obtained by Cox and Miles.10 Given the relatively small size of the coefficients in all the models, this means that due to confounding Cox and Miles overestimate the effects of ideology by 20%, 45%, and 60% in columns 1, 2, and 3 of Table 1, respectively.11 Accounting for this inadvertent exaggeration does not change the authors' underlying qualitative claim with respect to the effects of ideology in this particular study. However, in other studies it is entirely possible that failure to account for confounding variables could actually result in an incorrect qualitative conclusion because the appropriate model could have the effect of reversing the sign of the coefficient and thus a variable could have the opposite effect claimed by the author—a problem that we note emerges in Cox and Miles's data in our more extended investigation of their study.12
We also note that the overstated effects of ideology appear to be associated with the positive and statistically significant effects of race and age. Put differently, because the authors did not include proper adjustments, they unintentionally incorporate some of the effects of race and age into their coefficient on ideology. Our findings with respect to these attributes are particularly interesting given that Cox and Miles separately estimate the direct effects of race and age, controlling for ideology. Their results show that race correlates with a thirty percent increase in the likelihood of a judge's voting for liability at statistically significant levels, but that age does not have a statistically significant relation to outcomes.13 In fact, the models we presented in Table 2 suggest that the estimate of the direct effect of race, with proper control for confounding, may be as much as seventeen percent greater than the estimate of Cox and Miles. Moreover, our results indicate that the authors missed the role that age plays in the decisionmaking process. Assuming that Table 2 incorporates better assumptions about the data than those incorporated into Cox and Miles's models, then the direct effect of age on judicial voting is such that every year that a judge grows older, he or she has a 0.3% to 0.5% increased probability of finding liability. Accordingly, the oldest judge in the dataset is eighteen or thirty percent more likely to issue a pro-plaintiff ruling than the youngest judge.
As noted above, the point of our Response is not simply to critique Cox and Miles's empirical findings, but rather to introduce a new empirical method: directed acyclic graphs (DAGs). The DAG method is easy and intuitive to use and, as we show below, will allow empiricists systematically to explore the underlying structure of their data, thereby enabling easy identification of confounding, estimation of appropriate models, and legitimate causal claims. We now turn to the rules and principles of DAG construction.
II. The Principles and Rules Underlying Causal Directed Acyclic Graphs
To begin our exposition, we would like to remind our readers that we are only able to offer a very brief introduction to causal DAGs in this Response. We hope we convey enough information to demonstrate the underlying intuition behind and usefulness of the method, but encourage interested readers to consult other sources of information on DAGs for a more detailed analysis.14
The first step in creating a DAG is to construct a network or diagram representing the investigator's understanding of the relationships and dependencies between and among variables of interest. The graph should consist of a set of nodes (the variables) and a set of directed edges (arrows) that link the nodes. The directed edges correspond to cause-effect relationships. A path is an unbroken, nonintersecting sequence of edges that may go along with or against the arrows. A directed path is a path that follows the edges in the direction of the arrows. Relationships such as A ← B → C and A → B → C are both paths, but only the latter is a directed path as it follows the edges in the direction indicated by the graph's arrow. A node Xi that has a directed edge into node Xj indicates that Xi is the "parent" (or "direct cause") of Xj; and Xj is said to be a "child" of Xi. A node Xi is an "ancestor" of Xj if there is a directed path from Xi to Xj; and in this case Xj is said to be a "descendent" of Xi. If no node on the graph has a directed path back to itself, then the graph is said to be acyclic. Graphs that are directed and acyclic preserve the notion that causes must precede their effects and that no event can be its own cause.15 For a directed acyclic graph to be considered a causal DAG, one must ensure that all common causes of any two variables on the graph are also on the graph; this ensures that the graph captures the various possible confounding relationships.
Consider Figure 3, which depicts a DAG representing the relationships among and between five separate variables: seasons of the year (X1), sprinkler systems (X2), rainfall (X3), wet pavement (X4), and accidents (Y).16 The DAG shows that X1 is a parent of both X2 and X3; that X2 and X3 are parents of X4; and that X4 is a parent of Y.
Figure 3: A Directed Acyclic Graph Representing the Relationship Among Five Variables

Figure 3 reflects our intuitions, understandings, and beliefs about the world and is meant to convey underlying assumptions of analysis. The absence of a direct link between X1 and Y, for example, captures our understanding that the influences of seasonal variation on sidewalk accidents is mediated through various other conditions, and that the variations themselves are not direct causes of accidents. Springtime, for example, does not directly cause one to slip on the sidewalk; rather, springtime leads to more rain and higher levels of sprinkler use, which in turn lead to wet pavement—the direct and proximate cause of observed accidents in this model.17
Now suppose we have collected all the data for the variables depicted by the nodes in Figure 3 and would like to estimate the causal effect of a specific variable, say X2 (sprinkler systems) on Y (sidewalk accidents). Before doing so we must specify a model that includes the proper control or adjustments to avoid confounding. Adjustments are essentially equivalent to dividing the population into groups that are homogenous relative to some factor, say Z, and assessing the effect of the variable of interest on the outcome in each homogenous group and then averaging the results.18 Such a procedure is often carried out in conjunction with modeling by means of regression techniques similar to that used by Cox and Miles in their study and presented in Table 1 above.19
Recall from above that confounding variables have an effect on both the target explanatory variable and the outcome of interest. In the graphing context, this means that confounding occurs when there is a path from the explanatory variable of interest to the outcome that begins with a directed edge going into the explanatory variable. Such confounding paths are referred to as "backdoor paths." When such "backdoor paths" exist, control must be made for other variables in order to prevent this confounding. In general, if control can be made for all variables with directed edges going into the explanatory variable, then this will suffice to address confounding and one can obtain unbiased estimates of the causal effects of interest.
We see from Figure 3 that there is confounding of the relationships between sprinkler systems and accidents given the backdoor path from X2 to Y (X2 ← X1 → X3 → X4 → Y). We could address this confounding by controlling for the variable X1, indicating the season of the year. Essentially, if we fail to account for the season of the year, we might overestimate the effects of sprinkler systems on accidents given that sprinkler systems are used more in the spring, it rains more frequently in the spring, and the rain itself can cause the pavement to be wet and thus lead to an increased incidence of accidents. Adjustment must therefore be made for X1 (season of the year) in order to obtain unbiased estimates of the effect of sprinkler systems on accidents.
Now reconsider Figures 1 and 2, above, which are two possible depictions of the underlying relationships between and among the variables in Cox and Miles's dataset. The best representation—that is, the appropriate DAG upon which to rely in the modeling process—is the one that best reflects our intuitions, understandings, and beliefs about the world. Note that Figure 1 depicts a link from ideology, race, gender, and age going directly into judicial decisions but no other links between any other two variables in the data; this captures Cox and Miles's assumption that personal attributes are the parents of judicial outcomes but are otherwise independent. Figure 2, by contrast, relaxes the strong independence assumption and allows race, gender, and age to affect both ideology and judicial decisions as demonstrated by the directed edges from race, gender, and age into ideology and judicial decisions. Importantly, we believe Figure 2 better captures some of the structural relationships among variables and captures the assumption that political preferences can (and do) shift with a person's background characteristics. Put differently, on average we expect African Americans, women, and younger individuals to be more liberal than whites, men, and older persons:20 a relationship allowed by Figure 2 but not by Figure 1.
We believe the DAG methodology would have improved Cox and Miles's modeling process. Indeed, we believe that if the authors fully explored the structure of their data they would have noted the possible cause-effect relationship between race, gender, and age with ideology and would not have made the strong assumption of independence. Put differently, because Figure 2 depicts three backdoor paths from ideology to judicial decisions (ideology ← race → judicial decisions; ideology ← gender → judicial decisions; and, ideology ← age → judicial decisions), the authors would have seen the problem of confounding and included the proper adjustments to their models before making causal claims. Although we will not offer further analysis of the authors' study here, we have undertaken a comprehensive analysis of the possible underlying causal structures of the Cox and Miles data elsewhere, using causal DAGs.21 The basic intuitions of our analyses are fairly straightforward. It seems likely that in addition to the characteristics of the case, the variables race, gender, age, and education of the judge all probably affect both ideology and judicial decisions.22 In Part I, we demonstrated how these additional controls can and do change the authors' qualitative conclusions with respect to age and quantitative findings with respect to ideology and race.
Conclusion
In this Response, we demonstrate that when scholars rush to present empirical results without first considering the structure of their data, they are apt to make unrealistic assumptions about the relationships between and among the variables. These faulty assumptions, in turn, often lead investigators to ignore bias and confounding when they exist, and thus present findings that may over- or underestimate the effects of interest—and in some cases even reach incorrect qualitative and quantitative conclusions. With the help of causal directed acyclic graphs (DAGs), scholars can better avoid these problems, thereby strengthening inferences about cause and effect relationships. We have introduced the intuition behind the DAG methodology along with the basic rules and principles for constructing diagrams, and illustrated the use of these rules in the context of Professors Cox and Miles's study on judicial voting. If the authors had constructed and then relied on a DAG before estimating the effects of ideology and race on decisionmaking in the voting rights context, we believe they would have avoided overestimating the effects of ideology and underestimating the effects of race, and could have observed the actual effect of age. Although we have only demonstrated here the usefulness of DAGs in one particular context, the systematic procedure of this approach is applicable to diagrams of any shape, size, or complexity, and we believe that this methodology can be very helpful in future empirical legal research.
* Class of 1940 Research Professor of Law at Northwestern University Law School; Ph.D. student at the University of Chicago Public Policy School. We would like to thank Professors Cox and Miles for generously sharing their data, thereby making all the analyses presented here possible. Please send all thoughts and comments to Nancy Staudt at n-staudt@northwestern.edu.
** Assistant Professor of Biostatistics in the Department of Health Studies at the University of Chicago.
1 Adam B. Cox & Thomas J. Miles, Judging the Voting Rights Act, 108 Colum. L. Rev. 1 (2008).
2 Voting Rights Act of 1965, 42 U.S.C. § 1973 (2000).
3 As a trivial illustration of confounding, suppose that in analyzing children of different ages, an empirical researcher found correlation between weight and math ability. A researcher finding this correlation might (falsely) claim that heavier children are smarter, or that high math scores cause weight gain. In this context, age confounds the relationship between weight and math ability, and thus needs to be controlled for in the analysis. Specifically, the correlation between weight and math ability should be conducted within the strata of age. To avoid spurious claims, scholars must adjust for confounders in their models—such as the confounding variable of age in this example.
4 In fact, Cox and Miles themselves suggest a correlation between ideology and race. Cox & Miles, supra note 1, at 3 ("[B]ecause race and partisanship correlate closely in the United States, the partisan and racial implications of voting rights cases are often plain on their face.").
5 Id. at 38 tbl.5. For simplicity we focus our discussion on the individual judicial decisionmaking analysis of Cox and Miles rather than their analysis of panel effects.
6 Id. at 37–40.
7 Id. at 44 (presenting empirical results for additional models that separately include race, gender, and age). Tables 5 and 6 present Cox and Miles's empirical findings from twelve statistical models; ideology, race, gender, and age are all present, but not in the same model simultaneously, as indicated by the list of variables included in the first column of each table. Id. at 38, 44.
8 See Tyler J. VanderWeele & Nancy Staudt, Causal Diagrams for Empirical Legal Research: Methodology for Identifying Causation, Avoiding Bias, and Interpreting Results 12–17 (unpublished manuscript, on file with the Columbia Law Review) (March 11, 2009) (exploring in more detail causal directed acyclic graphs and issues of their application to legal research).
9 In particular, note that the coefficients on ideology is both positive and is statistically significant at the p ≤ .05 level in five of the six specifications presented in Tables 1 and 2 and at the p ≤ .10 level in the sixth.
10 We obtained these numbers by comparing the coefficients on ideology in Tables 1 and 2. In Table 2, column 1(a), for example, we estimate that liberal judges are .121 percentage points more likely to vote in favor of the plaintiff, but in Table 1, column 1, Cox and Miles estimate the effect is .145—a difference of .024 or 2.4%. Columns 2(a) and 3(a) of Table 2 are similarly compared with columns 2 and 3 in Table 1, respectively.
11 Because we believe our estimates are the better estimates given the graph in Figure 2, we calculated the level of inflation by dividing Cox and Miles's estimate of ideology's effects presented in each column of Table 1 by our corresponding estimate in each column of Table 2. For example, comparing column 1 and 1(a): .145/.121= 1.20, indicating that Cox and Miles's estimate is 20% larger than our estimate.
12 See VanderWeele & Staudt, supra note 8, at 12–17.
13 See Cox & Miles, supra note 1, at 44 tbl.6, col. 1 (reporting coefficient on race equal to .300 with standard error of .081); id. tbl.6, col. 4 (reporting coefficient on age equal to .003 and standard error of .002).
14 See, e.g., VanderWeele & Staudt, supra note 8.
15 Judea Pearl, Causality 12–40 (2000); Sander Greenland, Judea Pearl & James M. Robins, Causal Diagrams for Epidemiologic Research, 10 Epidemiology 37, 46 (1999).
16 Figure 3 presents a modified example of a causal DAG from Pearl, supra note 15, at 15.
17 Id. at 14–15.
18 Id. at 78.
19 See supra note 6 and accompanying text (describing models used by Cox and Miles).
20 A vast literature in law, sociology, psychology, and political science explores the relationship between personal attributes and political preferences. For just a few examples of these studies, see generally, Edward Carmines & James Stimpson, Issue Evolution: Transformation of American Politic (1990) (exploring role of race in voting patterns); Martin Gilens, Racial Attitudes and Opposition to Welfare, 57 J. of Pol. 994 (1995) (study of white conservative viewpoints); Tammy L. Henderson, Pamela Monroe, James Garand & Diane Burts, Explaining Public Opinion Toward Government Spending on Child Care, 44 Fam. Rel. 37 (1995) (exploring role of race, gender, and age in public policy context); John J. Ray, What Old People Believe: Age, Sex, and Conservatism, 6 Pol. Psychol. 525 (1985) (exploring effects of age and sex on ideology); Susan Welch, Are Women More Liberal Than Men in the U.S. Congress?, 10 Legis. Stud. Q. 125 (1985) (finding women in House of Representatives vote more liberally than men).
21 VanderWeele & Staudt, supra note 8, at 12–17.
22 Id.
Preferred Citation: Nancy C. Staudt & Tyler J. VanderWeele, Methodological Advances and Empirical Legal Scholarship: A Note on Cox and Miles's Voting Rights Act Study, 109 Colum. L. Rev. Sidebar 42 (2009), http://www.columbialawreview.org/Sidebar/volume/109/42_StaudtVanderWeele.pdf.



