Meta-Analysis Programs & Datasets
| Field, A. P. & Gillett, R. (2010). How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology, 63, 665-694. |
Getting Started
This website contains the files associated with Field, A. P. & Gillett, R. (2010). How to do meta-analysis. British Journal of Mathematical and Statistical Psychology, 63, 665-694. Please see this article for a full description of the files and how to use them.
The programs are SPSS command syntax files (written for IBM SPSS 16.0 or later) and R programs (written for R 2.7.2 or later). The datasets are SPSS data files based on published meta-analyses in the field of psychology.
- Create a new folder Meta-Analysis in the Documents folder of your PC/Mac. (In Windows 7 this folder appears as 'My Documents' in explorer but is called 'Documents', in versions of Windows older than Vista it is actually called 'My Documents' and you will need to edit the syntax files accordingly - see below).
- Download the programs and datasets on this webpage to the Meta-Analysis folder
- The setup program for R may be downloaded from http://www.stats.bris.ac.uk/R/ (for Windows and Mac)
- R should be installed on the same PC/Mac as SPSS
- Familiarity with R is not required to run the programs
IMPORTANT: Many of these syntax files have a command in them that changes the directory to be your 'Meta-Analysis' directory. This line of code will need to be changed depending on your operating system:
- Text in file: cd "%HOMEDRIVE%%HOMEPATH%\Documents\Meta-Analysis". This works for Windows Vista and Windows 7.
- Windows XP and earlier: change the syntaxso that 'Documents' reads 'My Documents' (i.e. cd "%HOMEDRIVE%%HOMEPATH%\My Documents\Meta-Analysis").
- MacOS: change the syntax to delete the stuff before 'Documents' (i.e. cd "Documents\Meta-Analysis").
Basic Meta-Analysis
Each program for basic meta-analysis provides:
- An estimate of the average effect size of all studies, or any subset of studies
- A test of homogeneity of effect size that contributes to the assessment of the goodness of fit of the statistical model
- Elementary indicators of, and tests for, the presence of publication bias
- Output for both fixed-effects and random-effects models
Example 1: Correlation
- Start up SPSS and open the dataset Cartwright-Hatton_et_al_2004.sav
- On the top menu bar, click File | Open | Syntax and go to the Meta-Analysis folder
- Select the syntax file Meta_Basic_r.sps and, when it has opened, click Run | All on its top menu bar. (Windows XP and Mac users note that you need to edit the file as described in getting started)
Example 2: Correlation (Hunter Schmidt Only)
- This file differs from the one above in that it conducts only the Hunter-Schmidt Method. However, unlike the Meta_Basic_r.sps file it includes a more complete version of the H-S method. It adds the corrected correlations (corrected for measurement error), observed variance of the corrected correlation, the standard deviation of the corrected correlations, 80% credibility interval around the corrected correlations, and variance attributable to study artifacts). This syntax file does not form part of the Field & Gillett (2010) paper and was adapted from our syntax by Maja Osolnik (Saint Cloud State University). We inciude it here as an additional file that might be useful.
Example 3: Standardised Difference Between Two Means
- Start up SPSS and open the dataset Else-Quest_et_al_2006.sav
- On the top menu bar, click File | Open | Syntax and go to the Meta-Analysis folder
- Select the syntax file Meta_Basic_d.sps and, when it has opened, click Run | All on its top menu bar. (Windows XP and Mac users note that you need to edit the file as described in getting started)
Example 4: Difference Between Two Proportions
- Start up SPSS and open the dataset Pozzulo_&_Lindsay_1998.sav
- On the top menu bar, click File | Open | Syntax and go to the Meta-Analysis folder
- Select the syntax file Meta_Basic_D_h.sps and, when it has opened, click Run | All on its top menu bar. (Windows XP and Mac users note that you need to edit the file as described in getting started)
Moderator Variable Analysis
A moderator variable is a predictor variable, with a known value for
each study, that has been proposed (preferably on theoretical grounds)
as a candidate to explain heterogeneity among effect sizes.
The programs for analysing the influence of moderator variables use weighted multiple regression to supply:
- An evaluation of the impact of continuous moderator variables on effect size
- An evaluation of the impact of categorical moderator variables on effect size
- Tests of homogeneity of effect size that contribute to the
assessment of the goodness of fit of a statistical model incorporating
a given set of moderator variables
- Output for both fixed-effects and random-effects models
Each program is run by means of an individual launcher program which
minimises the need for user input and provides a template to facilitate
customisation.
Coding Categorical Moderators
A categorical variable with c categories should be coded using the first c integers, 1, 2, ..., c, to denote the categories.
Example 4: Correlation
- Start up SPSS and open the dataset Tenenbaum_&_Leaper_2002.sav
- On the top menu bar, click File | Open | Syntax and go to the Meta-Analysis folder
- Select the syntax file Launch_Meta_Mod_r.sps and, when it has opened, click Run | All on its top menu bar (Windows XP and Mac users note that you need to edit the file as described in getting started)
- Note that the last line of the syntax file launches a macro Moderator_r that has four parameters
| | Moderator_r r=r n=n conmods=( ) catmods=(genexp). | |
- The four parameters take as values the names of variables in a dataset
| | r | = name of the correlation variable, e.g., r=cor | |
| | n | = name of the sample size variable, e.g., n=nsub | |
| | conmods | = names of continuous moderator variables, e.g., conmods=(arousal accuracy) | |
| | catmods | = names of categorical moderator variables, e.g., catmods=(gender religion) | |
Example 5: Standardised Difference Between Two Means
- Start up SPSS and open the dataset Else-Quest_et_al_2006.sav
- On the top menu bar, click File | Open | Syntax and go to the Meta-Analysis folder
- Select the syntax file Launch_Meta_Mod_d.sps and, when it has opened, click Run | All on its top menu bar (Windows XP and Mac users note that you need to edit the file as described in getting started)
- Note that the last line of the syntax file launches a macro Moderator_d that has five parameters
| | Moderator_d d=d n1=n1 n2=n2 conmods=( ) catmods=(age source). | |
- The five parameters take as values the names of variables in a dataset
| | d | = name of variable representing the standardised difference between two means, e.g., d=delta | |
| | n1 & n2 | = names of variables representing sample sizes of group 1 and group 2, e.g., n1=ng1 n2=ng2 | |
| | conmods | = names of continuous moderator variables, e.g., conmods=(arousal accuracy) | |
| | catmods | = names of categorical moderator variables, e.g., catmods=(gender religion) | |
Example 6: Difference Between Two Proportions
- Start up SPSS and open the dataset Pozzulo_&_Lindsay_1998.sav
- On the top menu bar, click File | Open | Syntax and go to the Meta-Analysis folder
- Select the syntax file Launch_Meta_Mod_D_h.sps and, when it has opened, click Run | All on its top menu bar (Windows XP and Mac users note that you need to edit the file as described in getting started)
- Note that the last line of the syntax file launches a macro Moderator_D_h that has six parameters
| | Moderator_D_h
p1=p1 p2=p2 n1=n1 n2=n2
conmods=( ) catmods=(age source). | |
- The six parameters take as values the names of variables in a dataset
| | p1 & p2 | = names of variables representing the proportions in group 1 and group 2, e.g., p1=pg1 p2=pg2 | |
| | n1 & n2 | = names of variables representing sample sizes of group 1 and group 2, e.g., n1=ng1 n2=ng2 | |
| | conmods | = names of continuous moderator variables, e.g., conmods=(arousal accuracy) | |
| | catmods | = names of categorical moderator variables, e.g., catmods=(gender religion) | |
- The output contains the results of two separate analyses of the same data: first, using effect-size measure D and, second, using effect-size measure h
Sensitivity of Effect-Size Estimates to Publication Bias
To supplement the elementary tests for publication bias described
above, programs for producing funnel plots and assessing the vulnerability of effect-size
estimates to publication bias are provided. They are written in R and have been adapted from Vevea & Woods (2005). The programs employ sensitivity analysis to investigate the impact of moderate and severe
levels of bias. Effect-size estimates may be considered robust to the
extent that they would be relatively unaffected were a substantial
level of bias to be present.
Each program supplies:
- An ordinary unadjusted estimate of effect size
- Four adjusted estimates of effect size that indicate the potential impact of
- moderate one-tailed bias
- severe one-tailed bias
- moderate two-tailed bias
- severe two-tailed bias
- Output for both fixed-effects and random-effects models
R should be installed on the same PC/Mac as SPSS, as described in Getting Started. Notive that ythere are Windows and Mac versions of the files, the only difference is one line of code that 'finds' the publication bias data from your earlier analysis. On Windows the codeline is
fp <- paste(Sys.getenv("R_USER"),"Meta-Analysis","Pub_Bias_Data.sav",sep="\\")
but on a Mac (MacOS 10) it needs to be
fp <- paste(Sys.getenv("HOME"),"/Documents/Meta-Analysis","Pub_Bias_Data.sav",sep="/")
The different files differ only in this line of code. You can edit this line of code to look for the file "Pub_Bias_Data.sav"if you saved it somewhere other than ~/Documents/Meta-Analysis/
Example 7: Correlation
- Start up SPSS, open the dataset Cartwright-Hatton_et_al_2004.sav, and perform the basic meta-analysis described in Example 1 to create an intermediate file required by R.
- Start up R and on the top menu bar click File | Open Document, go to the Meta-Analysis folder, and select the file Pub_Bias_r.R
- In the window that opens up to display the file contents, right-click anywhere to get the menu and choose Select All, then right-click a second time and choose Run Line or Selection (on a Mac cmd+A selects all, and cmd+enter runs the selected code)
Example 8: Standardised Difference Between Two Means
- Start up SPSS, open the dataset Else-Quest_et_al_2006.sav, and perform the basic meta-analysis described in Example 1 to create an intermediate file required by R
- Start up R and on the top menu bar click File | Open Document, go to the Meta-Analysis folder, and select the file Pub_Bias_d.R
- In the window that opens up to display the file contents, right-click anywhere to get the menu and choose Select All, then right-click a second time and choose Run Line or Selection (on a Mac cmd+A selects all, and cmd+enter runs the selected code)
Example 9: Difference Between Two Proportions
- Start up SPSS, open the dataset Pozzulo_&_Lindsay_1998.sav, and perform the basic meta-analysis described in Example 1 to create an intermediate file required by R
- Start up R and on the top menu bar click File | Open Document, go to the Meta-Analysis folder, and select the file Pub_Bias_D_h.R
- In the window that opens up to display the file contents, right-click anywhere to get the menu and choose Select All, then right-click a second time and choose Run Line or Selection
(on a Mac cmd+A selects all, and cmd+enter runs the selected code)
Links related to Meta-Analysis or our Article
Web page by Andy Field and Raphael Gillett, 2009, Last Updated September 2010