In Search of Evidence

Chapter 7
Based on these exclusion criteria, 10 Dutch and 181 U.S. respondents were omitted
from the survey. No restrictions were applied regarding sub-discipline (e.g. HR or
finance), organizational level (e.g. senior or mid-level management), or industry.
We sent an email with an invitation to participate and a secured link to the
online questionnaire to a random sample of 30,000 U.S. managers from the
leadership directory and a convenience sample of 2,972 Belgian and Dutch
managers. We emailed a reminder twice to all non-responders. The response rate
for the American sample was 3% (n = 924), while the overall response rate for the
Belgian—Dutch sample was 30% (n = 875), giving an overall final sample size of
1,566. One explanation for the difference in response rate is that a large number of
the American invitations were undeliverable, either because of incorrect email
addresses or the manager having left the organization they were listed under.
Another explanation is that, in contrast with the U.S. sample, those in Belgian and
Dutch sample were contacted through established relationships, such as alumni
organizations. Characteristics of the 1,566 respondents are described in Table 1.
The typical respondent was a 50- to 59-year-old male manager with a master’s
degree and more than 10 years of experience in the field of general management.
Statistical Analysis
The first step we took in preparing the results of the questionnaire for use was
to examine the nature of our missing data. In this case, the rate of missing values
ranged from 0% to 19.3% (mean 9.2%), thus emphasizing the need to choose an
appropriate method for approximating these missing values. We considered the
data to include patterns of missing at random and missing completely at random,
which necessitates using a robust imputation technique. Specifically, we opted for
the multiple imputation (MI) approach using the Markov chain Monte Carlo
method (Rubin, 1987). This method uses an iterative version of stochastic
regression imputation to create multiple copies of the data set (in our case 20),
each of which contains different estimates of the missing values. The advantage of
MI over other missing data approaches is that the sample size is preserved, and the
parameter estimates for all of the imputed datasets are pooled, providing more
accurate estimates. To represent the broad trends uncovered in our analyses, we
reported categorical data as percentages, while continuous data were reported as
means and standard deviations. We tested a possible association between level of
education, level of experience (tenure), and attitude towards evidence-based