Doing Economics

Figure 2.1
Example table of variable descriptions from Bellemare (2012).
At this point, it is time to present and discuss descriptive statistics. Here,
whereas it used to be sufficient to simply present a table of means and
standard deviations, it has become practically necessary in cases where the
variable of interest (i.e., the treatment variable) is composed of a small
number of categories to show the results of balance tests, namely tables
where each line is a variable retained for analysis, where means and
standard errors are shown conditional on treatment status, and wherein one
assesses whether the mean of each variable systematically differs across

---

treatment statuses by reporting p-values for a test of difference in means.
Though the textbook example involves only treatment and control, it is
increasingly common for studies to include more than two treatment arms,
and so any meaningful balance test must be reported for each pairwise
comparison of means. With two treatment arms, this means (i) treatment 1
versus control, (ii) treatment 2 versus control, and (iii) treatment 1 versus
treatment 2.
With experimental data, the idea behind such balance tests is to show the
reader that randomization was done properly. With observational data,
where we would not expect the data to be balanced, the idea behind such
balance tests is to assess how unbalanced the data are—an idea which
comes from the matching literature (Morgan and Winship 2015). With
perfect random assignment across treatment and control groups, there
should be fewer than 1 in 10 pairwise comparisons differing at less than the
10 percent level of statistical significance, fewer than 1 in 20 pairwise
comparisons differing at less than the 5 percent level of statistical
significance, and fewer than 1 in 100 pairwise comparisons different at less
than the 1 percent level of statistical significance. In cases where pairwise
comparisons return too many systematic differences, one should ideally
control for the relevant covariates in a regression or matching context when
estimating treatment effects.
4
Beyond the usual table of means and standard deviations and one or more
tables showing the results of balance tests, a good Data and Descriptive
Statistics section can also be used to explore the data nonparametrically by
showing kernel density estimates of the relevant variables (i.e., outcome
and treatment variables at a minimum, but also controls suspected to be the
source of treatment heterogeneity) when they are continuous, histograms of
the relevant variables when they are categorical, or cross-tabulations (i.e.,
two-by-two tables) in cases where both the treatment and the outcome are
binary.
When writing a Data and Descriptive Statistics section, there are a few
mistakes you should avoid making. The first is for the writeup to present a
boring enumeration of means. If a gender variable is merely used as a
control in the analysis, there is little use to stating in the text that “37.4
percent of respondents are female” when the reader can look that up for
herself; the only variables that typically deserve discussion here are the

---

outcome and treatment variables, any variable that is used for identification
(e.g., an instrumental or forcing variable), or anything that really stands out.
Generally, a good rule of thumb is to keep the discussion of the descriptive
statistics to a few sentences.
The second such mistake is the use of the past tense in discussing the
data and descriptive statistics. The example above stated how “37.4 percent
of respondents are female,” and not how “37.4 percent of respondents were
female.” Scientific communication in English is more effective when using
the present tense to discuss your data or results, and just as you should
avoid the passive voice, you should also avoid the past tense in research
papers, except when summarizing and concluding. Indeed, the past tense
should be largely kept for when you discuss what other researchers have
done before you, and the future tense for what you are planning on doing or
what others should be doing in the future. The present tense is ideal because
it refers to that which occupies the reader right now, which is your paper.
5
Finally, another mistake is to present numbers that either have too many
decimal places because they are too small (usually, three decimal places is
more than enough, and at any rate it is always possible to rescale a variable
to make its magnitude fit with that of the other variables) or to present
numbers that are difficult to interpret in tables, such as 1.37e + 8, or
anything other than units readers are used to dealing with (for instance, it is
always possible to express a dollar amount in thousands or hundreds of
thousands if need be). In other words, even if the empirical work regresses
the logarithm of income on the treatment variable, the table of descriptive
statistics should report the mean of the income level, not the mean of the
logarithm of income. Ultimately, although a lot of what goes into a Data
and Descriptive Statistics section might seem like useless posturing, as
stated before, a good Data and Descriptive Statistics section should allow
the reader to form reasonable expectations about the sign and the magnitude
of the estimates of interest, and to get an idea of how those estimates are
likely to vary across a given conditioning domain.