Cogmed In-Brief Response: Is working memory training effective? A meta-analytic review
Working memory (WM) training deserves thoughtful review and it is natural that the methods be challenged and scrutinized by skeptical researchers. Unfortunately, a discerning review of the current literature is a difficult undertaking and can lead to questionable analysis and flawed conclusions as found in Melby-Lervåg and Hulme (2012). This meta-analysis is marked by a failure to recognize the key differences between WM training programs and the serious limitations inherent in comparing these programs. Below, please find take home messages from the Melby-Lervåg and Hulme paper. For a more detailed response to this meta-analysis, click here. For a meta-analysis of solely Cogmed Working Memory Training studies, click here.
WM is important and domain general
The authors’ recognition of WM as domain general and related to attention is consistent with the extant literature in the field, as well as, the claims made by Cogmed.
WM is impaired in diverse clinical populations
Cogmed agrees with the authors in their recognition that WM deficits are pervasive in certain clinical groups.
Randomized, placebo controlled studies are needed
WM training studies should indeed strive to employ rigorous research design and methodology. In order to foster this type of research, Cogmed actively supports over 60 ongoing studies using Cogmed Working Memory Training where almost all studies are randomized and where 65% of the 34 actively controlled projects utilize a placebo control. Combination of diverse group of sample populations, ignoring distinctions between individuals with and without WM deficits. The authors draw an inappropriate comparison of the effect of training on people with widely different baseline WM abilities.
Combination of diverse age range of age groups
The WM literature supports the notion that one’s WM capacity is related to their developmental stage. Unfortunately, the authors combine participants from ages 10 to over 50 years and create arbitrary definitions for coding the age groups. For example, older children range from ages 11 to 18 years.
Comparison of heterogeneous research methods
Misleadingly, the authors amalgamate wide range of training types, programs, & protocols and create arbitrary definitions for coding the differences between these variables. For example, training time was coded as either 8 hours or less vs. 9 hours or more with no rationale for a split.
Cognitive measures used to assess change combined into arbitrary categories
The type of assessment used to assess and interpret change after WM training matters. Unfortunately in this analysis, assessments were combined from different studies and linked together by arbitrary definitions. For example, the Stroop test was used as sole measure of attention.
Failure to include relevant measure of reading
Misleadingly, the authors included studies with measures of accuracy and fluency of word and non-word reading but ignored WM training research with measures of reading comprehension, an ability more tightly linked with WM.
Failure to include analysis of behavioral rating data and to distinguish between developmental disorders
In order to make justified conclusions about the behavioral impact of WM training, behavioral rating scales must be considered. Notably, the authors fail to include measures of behavior contained within reviewed papers. The conclusions about WM interventions stating that they cannot be recommended to ameliorate symptoms of some developmental disorders are based on an analysis that was never done (and possibly purposely excluded).
Study inclusion and exclusion criteria applied inconsistently
Inclusion and exclusion criteria for studies to be used in a meta-analysis must be applied uniformly. Unfortunately, some important and widely available studies meeting inclusion criteria were excluded and vice versa. Also, unpublished works were included while notable published works were not. The study selection approach is questionable.
Taking these points into consideration, one should plumb the findings of this meta-analysis of WM training literature and be aware of the limitations on drawing usable conclusions. For an Article Summary of the Melby-Lervåg and Hulme paper, please go here.