Elsevier

NeuroImage

Volume 84, 1 January 2014, Pages 299-306
NeuroImage

Can structural MRI aid in clinical classification? A machine learning study in two independent samples of patients with schizophrenia, bipolar disorder and healthy subjects

https://doi.org/10.1016/j.neuroimage.2013.08.053Get rights and content

Highlights

  • We separated patients with schizophrenia and bipolar disorder based on their sMRI scans.

  • We trained a support vector machine (SVM) model to do this in a 1.5 T ‘discovery set’.

  • Using cross-validation the model obtained a classification accuracy of > 80% in this set.

  • Applying this model to an independent 3 T ‘validation set’ yielded 66% accuracy.

  • Patterns of brain abnormalities differ between schizophrenia and bipolar disorder.

Abstract

Although structural magnetic resonance imaging (MRI) has revealed partly non-overlapping brain abnormalities in schizophrenia and bipolar disorder, it is unknown whether structural MRI scans can be used to separate individuals with schizophrenia from those with bipolar disorder. An algorithm capable of discriminating between these two disorders could become a diagnostic aid for psychiatrists. Here, we scanned 66 schizophrenia patients, 66 patients with bipolar disorder and 66 healthy subjects on a 1.5 T MRI scanner. Three support vector machines were trained to separate patients with schizophrenia from healthy subjects, patients with schizophrenia from those with bipolar disorder, and patients with bipolar disorder from healthy subjects, respectively, based on their gray matter density images. The predictive power of the models was tested using cross-validation and in an independent validation set of 46 schizophrenia patients, 47 patients with bipolar disorder and 43 healthy subjects scanned on a 3 T MRI scanner. Schizophrenia patients could be separated from healthy subjects with an average accuracy of 90%. Additionally, schizophrenia patients and patients with bipolar disorder could be distinguished with an average accuracy of 88%.The model delineating bipolar patients from healthy subjects was less accurate, correctly classifying 67% of the healthy subjects and only 53% of the patients with bipolar disorder. In the latter group, lithium and antipsychotics use had no influence on the classification results. Application of the 1.5 T models on the 3 T validation set yielded average classification accuracies of 76% (healthy vs schizophrenia), 66% (bipolar vs schizophrenia) and 61% (healthy vs bipolar). In conclusion, the accurate separation of schizophrenia from bipolar patients on the basis of structural MRI scans, as demonstrated here, could be of added value in the differential diagnosis of these two disorders. The results also suggest that gray matter pathology in schizophrenia and bipolar disorder differs to such an extent that they can be reliably differentiated using machine learning paradigms.

Introduction

Currently, the diagnosis of psychiatric disorders such as schizophrenia and bipolar disorder is based predominantly on their clinical manifestations. While psychiatrists can establish the presence of illness (as distinct from its absence) with relative ease, discrimination between several possible diagnoses is far more complicated, especially in the early phase of schizophrenia and bipolar disorder. The availability of additional (objective) measures would assist psychiatrists in the process of diagnosis, with obvious benefits to efficiency of treatment and improved outcome. Magnetic resonance imaging (MRI) has proven to be an effective technique to detect structural brain abnormalities at group-level in schizophrenia patients (meta-analyses: Haijma et al., 2012, Olabi et al., 2011) and those with bipolar disorder (meta-analyses: Kempton et al., 2008, McDonald et al., 2004). Unfortunately, statistical group differences do not translate to discovering deviations from normal on an individual basis and therefore are not sufficient as a diagnostic aid.

Using machine learning techniques, promising results have been obtained for the classification of schizophrenia patients and healthy subjects based on MRI scans. Pioneering work was done by Davatzikos et al. (2005), followed by numerous other investigations. The support vector machine (SVM; Fan et al., 2008, Ingalhalikar et al., 2010, Koutsouleris et al., 2009, Pohl and Sabuncu, 2009, Vapnik, 1999) and the Discriminant Function Analysis (Karageorgiou et al., 2011, Kasparek et al., 2011, Leonard et al., 1999, Liu et al., 2004, Nakamura et al., 2004, Takayanagi et al., 2011) are the most frequently used methods (for an overview of schizophrenia classification studies using structural MRI, see Nieuwenhuis et al. (2012)). We recently demonstrated in two large independent samples that a classification model built from one data set can be used to classify new subjects as schizophrenia patients or healthy subjects with 71% accuracy. To the best of our knowledge, no studies have been published investigating the use of MRI to separate bipolar patients from healthy subjects or schizophrenia patients [although one study combined structural MRI brain measures and neuropsychological test scores for this purpose (Pardo et al., 2006)]. Given the brain abnormalities found in bipolar disorder and schizophrenia and the differences between these abnormalities (Arnone et al., 2009, Ellison-Wright and Bullmore, 2010, Hulshoff Pol et al., 2012, Koo et al., 2008, McDonald et al., 2005, Qiu et al., 2008, Rimol et al., 2010, Rimol et al., 2012), it may be fruitful to apply these classification models to help separate these two disorders. We train three SVM models to separate patients with schizophrenia from those with bipolar disorder, healthy subjects from patients with schizophrenia, and healthy subjects from patients with bipolar disorder. Although it could be of theoretical interest to build a three-group classifier that separates the three groups in a single step, our approach addresses the clinical relevant issue of separating the two disorders using MRI. Furthermore, it provides brain patterns that discriminate between the respective groups, which can be analyzed to indicate which features are unique to the discrimination between schizophrenia and bipolar disorder. We test the predictive power of the models both in the dataset they were built on and in an independent dataset.

Section snippets

General

In this study we used two datasets. The first set, called discovery sample, was used to build classification models for the separation of healthy subjects and patients with schizophrenia and bipolar disorder. The models were tested on this set too. On the second set, called validation sample, no models were built; this independent sample was used to test the generalizability of the models built on the first set.

Subjects — discovery sample

Schizophrenia patients (SZ), patients with bipolar disorder (BP) and healthy

Results

The classification results of the models are given in Table 2. In the discovery sample, the models involving SZ patients performed best, with classification accuracies of 86% or higher for the models including all subjects (M). SZ patients can thus be separated well from HC subjects and from BP patients (p < 0.001). The separation of BP patients and HC subjects, on the other hand, turned out to be less accurate: 67% of the HC subjects were correctly classified (p = 0.02) and only 53% of the BP

Discussion

The purpose of this study was to classify patients with schizophrenia, bipolar disorder, and healthy controls on the basis of their structural MRI scans. We used a support vector machine (SVM) to create three models from gray matter density images, each separating two of the three groups. We confirmed that it is possible to separate schizophrenia patients from healthy subjects with an average accuracy of 90%. Additionally, we demonstrated that schizophrenia patients can be separated from those

Acknowledgments

We thank Dorret Boomsma for providing us with healthy twin subjects from the Netherlands Twin Registry. We thank and Astrid van der Schot for assistance with data acquisition and processing.

References (49)

  • B. Olabi et al.

    Are there progressive brain changes in schizophrenia? A meta-analysis of structural magnetic resonance imaging studies

    Biol. Psychiatry

    (2011)
  • P.J. Pardo et al.

    Classification of adolescent psychotic disorders using linear discriminant analysis

    Schizophr. Res.

    (2006)
  • L.M. Rimol et al.

    Cortical thickness and subcortical volumes in schizophrenia and bipolar disorder

    Biol. Psychiatry

    (2010)
  • L.M. Rimol et al.

    Cortical volume, surface area, and thickness in schizophrenia and bipolar disorder

    Biol. Psychiatry

    (2012)
  • A.M. Shepherd et al.

    Systematic meta-review and quality assessment of the structural brain alterations in schizophrenia

    Neurosci. Biobehav. Rev.

    (2012)
  • N.C. Andreasen et al.

    The Comprehensive Assessment of Symptoms and History (CASH). An instrument for assessing diagnosis and psychopathology

    Arch. Gen. Psychiatry

    (1992)
  • D. Arnone et al.

    Magnetic resonance imaging studies in bipolar disorder and schizophrenia: meta-analysis

    Br. J. Psychiatry

    (2009)
  • C.-C. Chang

    A library for support vector machines

  • D.L. Collins et al.

    Automatic 3-d model-based neuroanatomical segmentation

    Hum. Brain Mapp.

    (1995)
  • T. Dashjamts et al.

    Alzheimer's disease: diagnosis by different methods of voxel-based morphometry

    Fukuoka Igaku Zasshi

    (2012)
  • C. Davatzikos et al.

    Whole-brain morphometric study of schizophrenia revealing a spatially complex set of focal abnormalities

    Arch. Gen. Psychiatry

    (2005)
  • J. Endicott et al.

    A diagnostic interview: the schedule for affective disorders and schizophrenia

    Arch. Gen. Psychiatry

    (1978)
  • S.V. Haijma et al.

    Brain volumes in schizophrenia: a meta-analysis in over 18 000 subjects

    Schizophr. Bull.

    (2012)
  • B.C. Ho et al.

    Long-term antipsychotic treatment and brain volumes: a longitudinal study of first-episode schizophrenia

    Arch. Gen. Psychiatry

    (2011)
  • Cited by (0)

    1

    Authors contributed equally.

    View full text