Graphical Abstract

Phrase 2

This project aims to create an inexpensive and effective multi-class classification machine learning algorithm that will predict what type of psychiatric disorder a patient has based on their Resting-State brain f-MRI scan.

Background

Over the past decades, over 50% of the United States population was diagnosed with a psychiatric disorder (CDC, 2021). The most common types of psychiatric disorders include major depressive disorder (MDD), anxiety disorder, bipolar disorder (BD), dementia, attention-deficit/hyperactivity disorder (ADHD), obsessive compulsive disorder (OCD), post traumatic stress disorder (PTSD), and Schizophrenia (WHO, 2022). Patients suffering from psychiatric disorders deal with symptoms ranging from excessive sadness to reduced ability to focus. A patient may become detached from both reality and society leading them down a dark path. In the short term, these symptoms may not seem horrible, but in the long run, they can lead to the complete breakdown of one’s mental and physical health. A major component of understanding these disorders is coming to the realization that the patient’s experience varies case-by-case, meaning that every person's symptoms are unique to them and not everyone should receive the same treatment (Salters-Pedneault, 2021). Psychiatric disorders can take over a patient’s life, not only damaging their relationship with the things and people they love, but also their relationship with themselves.

The largest concern regarding mental illnesses has to do with the treatment process. Since treatment is case-dependent, it is crucial that doctors are able to properly classify patients. Current methods of classification include inputting the patients Magnetic Resonance Imaging (MRI) scans into picture archiving and communication systems (PACS). According to Doctor Kovvuru from the University of Arkansas for Medical Sciences, psychiatrists then use this system to transfer the images to Sectra which is a software that biomarks the scans, but does not analyze them. Biomarkers, measurable factors that act as significant points that appear on the images, help doctors differentiate between the MRI scans of different disorders (Rasetti et al., 2011). The main biomarkers that these scans check for are gray matter density, subcortical volumes, cortical thickness, fractional anisotropy, mean diffusivity, axial diffusivity, radial diffusivity, and functional connectivity (de Vos et al., 2020).

In order to identify these biomarkers, specific types of MRI scans are used: anatomical MRI scans, structural MRI scans, and diffusion MRI scans. Going into more depth about each specific scan, an anatomical scan is used to study the shape, volume, and developmental changes of the brain; therefore, this model was used by de Vos et al. (2020) when considering gray matter density, subcortical volumes, and cortical thickness. The next type of scan was a diffusion MRI scan which is mainly used for detecting tumors and monitoring tumor response to treatments over a period of time; hence why de Vos et al. (2020) used it for the calculation of anisotropy, mean, axial, and radial diffusivity. Lastly, they used a resting state functional MRI scan which is used for displaying normal and abnormal functional brain connectivity in different conditions. From this, they derived the functional connectivity and the amplitude of low-frequency fluctuations (de Vos et al., 2020). It is crucial that the right types of MRI scans are put to use when trying to diagnose a patient because without the right details, it is extremely difficult to correctly classify.

However, even with the assistance of the detailed biomarkers, doctors are still not able to properly inspect the scans because the biomarkers are outweighed by the amount of overlapping phenotypes displayed in the MRI scans. The mental health disorders with the most similar phenotypes are MDD, BD, ADHD, and Schizophrenia. Major Depressive Disorder is the greatest cause of disabilities and affects about 10% of the world. In 2013, it was identified that 60% of people that were classified as having Major Depressive Disorder were actually misdiagnosed (Bloomberg School of Public Health, 2013). Patients with MDD are most commonly misdiagnosed with BPD, a disorder that leads to extreme mood swings. The effects of Bipolar Disorder go to two different extents: hypomania, emotional heights, and depression, emotional lows. (Gao et al., 2018). Another neurological disorder is Schizophrenia, a neurological disorder where patients envision reality in an abnormal way because of hallucinations and delusions. ADHD is a neurological disorder that leads to low attention spans and hyperactivity. ADHD is most commonly diagnosed in a person's childhood. All of these disorders can be both genetic or acquired (Galderisi et al., 2017).

Machine learning algorithms have been commonly used to help recognize patterns and analyze biomarkers. There are many different types of machine learning algorithms, but in order to look for specific traits, classification algorithms are the most common. When creating a machine learning algorithm, Neural Networks are used. Although not every classification algorithm, such as random forests, requires Neural Networks, it is the most efficient approach (Xin et al., 2019). Neural Networks are a series of algorithms that recognize patterns in a data set. In order to increase an algorithm’s accuracy, models are optimized by repeatedly tampering with/testing kernels, layers, inputs, outputs, and weights. Layers receive the weighted input from the previous layer and pass these values as output to the next layer. In this process, they transform the input with a set of mostly non-linear functions before passing it on. This cycle repeats multiples with the inputs and outputs. Layers are the highest-level building block in deep learning, and there are many different types of layers, including activation and output layers. Activation layers control how well a model learns the training data. The most common example of this would be softmax which helps produce a multinomial probability. A multinomial probability is a type of probability distribution that allows one to find specific numbers for many different outcomes, given that the outcome has a fixed probability that it will happen. Output layers are the final layer where predictions are received. The two most common output layers are Rectified Linear Unit (ReLU), which prevents exponential growth by linearizing them, and Sigmoid, which applies a sigmoid function to the input so that the output is between 0 and 1 (Kim et al., 2022).

When looking at MRI scans to identify biomarkers, in specific, the three most common machine learning algorithms used when considering image classification are Convolutional Neural Networks (CNN), Support Vector Machines (SVM), and Transfer Learning. Convolutional Neural Networks are mainly focused on processing data like images. They are useful because they have a massive data capacity, but on the downside, they have a slower operation time and need more RAM and memory storage, which is hard to get access to as it is expensive. Support Vector Machines are used for binary classification, but since this project will focus on four different mental disorders, a binary classification wouldn't be useful. The positive side to SVMs is that they have a distinct margin that separates between classes, but on the negative side, they are not good for large data sets. Following along with this, if the number of features is greater than the sample number, then the SVM will underperform on it, meaning it will not train properly and will be inapplicable to other datasets. Transfer learning is the process of taking previous algorithms and adjusting them to fit the criteria of your project. The three most common transfer learning methods for classification are Inception V3, ResNet50, and VGG16 (Wahlang et al., 2022). All three of these methods are types of convolutional Neural Networks that have different layers and layer densities.

By improving upon these models and using machine learning algorithms to help not only display the biomarkers but also help classify between the disorders, the amount of misdiagnoses can be reduced. If patients are being misclassified, then their treatment will also be incorrect, leading to horrible side effects. For example, with the wrong medication, patient’s can deal with problems including memory impairment and addiction to that substance (Alyssa, 2021). These effects will in turn, lead to a worsened condition since the patient was expecting a positive outcome but will instead be in distress and sadness as they are actually going in the wrong direction.

Although the idea of machine learning in the medical field is not new, the model constructed in this study compared four different neurological disorders (MDD, BD, ADHD, and Schizophrenia), unlike any project done before. Introducing machine learning and image classification to the realm of neurological disorder diagnosis is an extremely recent discovery. Beyond this, most classifications are binary, meaning they classify between the null and alternative. This method identifies biomarkers for all four disorders and identifies specific biomarkers. Patients with neurological complications deserve to get better, not worse, because of a mistake in their treatment process. They have ensured their trust in this treatment, and it will only make their neurological decline even more if we are not careful with their medication process.

Background

I started off my project by looking for datasets that contained at least 200 MRI scans of each disorder. Although I couldn’t get access to a dataset with all four disorders, I was able to obtain MRI scans for each disorder from separate databases. In order to combat the issue of different practices taking MRI scans differently in terms of sizing and markings, I had to preprocess my data. I started off by resizing and adding filters to my images. The reason behind this is that if the layout of the images weren’t the same, then that meant the algorithm wouldn’t be classified based on the actual MRI scan. After this, I had to create more images using data augmentation. This means that I created more modified images based on what I already had by adding things like rotations. After creating my first two algorithms, I realized that in order to produce a higher accuracy, I needed to narrow the scope of what the model was checking for. I started off this process by identifying and labeling biomarkers in each specific MRI scan. This may seem counterproductive since the machine learning algorithm should theoretically be able to identify the biomarkers through the training process. Still, it was actually most beneficial for the biomarkers to be pre-highlighted because then it guaranteed that those were the specific parts of the MRI scan that the algorithm would look for.

After this, I split my data into a training, testing, and validation group. The testing set is the data you use to evaluate your model, the training set is the data you use to create and train the algorithm, and the validation set is the data you use to ensure that your algorithm goes through an unbiased evaluation. The training set should be the section with the most points because that is where your algorithm starts to recognize the patterns, and then the testing set is where the algorithm applies what is learned. When deciding what models to use, I started off by creating a decision matrix (Appendix 1). Although I ended up using all three models in the end, it was beneficial to have an idea of what the predicted outcomes would look like and which models to prioritize. There are many different image classification algorithms, such as decision trees, random forests, support vector machines (SVM), and convolutional neural networks (CNN). Outside of building our own model from scratch, machine learning also allows us to use transfer learning which essentially is the process of taking previous algorithms and adjusting them to fit the criteria of the project. Some common transfer learning models include InceptionV3, ResNet50, and VGG16 (Wahlang 2022). For my model in specific, the three models I am using are InceptionV3, Convolutional Neural Networks (CNN), and Support Vector Machines (SVM). Although SVMs are only used for binary classification, they are still useful when comparing each disorder MRI scan to MRI scans with no disorders because they produce a higher accuracy.

InceptionV3 is the main model being used for the image classification. At the top, we can see that the number of categories being classified is four. The second line of code displays which transfer learning model is being used: InceptionV3 in this case. Rectified Linear Unit (ReLU) is the activation layer that was used meaning that this is how the data was trained. RelU also helps prevent exponential growth. There were 64 neurons that replicate the human brain used in this model. There is a positive correlation between neurons and high accuracy, however, the number generally does not exceed 64 neurons since it also causes a major increase in run time. At the end of the code, I included SoftMax as my output layer as it helps produce a multinomial probability. Output layers are the final layer where predictions are received. At the end of my code, I included my training methods. I included 20 epochs meaning that every data point will be run through 20 times. I had 200 data points which was my batch size and I created labels to analyze the results when they were printed.

The confusion matrix was created to compare the actual true values to the predicted true values: true values meaning correctly classified. The reason I chose a confusion matrix as my statistical test is that it shows the probability of getting correct classifications. The rows represent the four classes of classification: no disorder, Schizophrenia, MDD, and BD. This confusion matrix compares the recall, precision, and f-1 score of all of them. As seen on the scale to the side, the lighter boxes produced the best results. The difference between precision and recall is how good the model was at predicting categories, and recall is how good the model was at detecting categories. The f1-score measures the algorithm's real accuracies by comparing the precision and recall values. The reason why overfitting is bad is that it masters an algorithm based on training data but then doesn’t end up applying this mastery to other datasets, which creates biases. In order to combat this problem, I used data augmentation, meaning that I created artificial data to increase the points in the training data set by using pre-existing data.

Figure 2

ROC/AUC graphs for Schizophrenia vs. BD modified InceptionV3 model

Figure 3

Multi-Class Support Vector Machine Confusion Matrix