STEM I

STEM with Scientific and Technical Writing is taught by Dr. Crowthers. In this class, we are not only exposed to scientific literature, but we also write our own. The first half of the year involves our own independent research projects, while the second half is a lot more group-oriented, where we develop technologies to assist with challenges in our community.

Using a Machine Learning Model to Prevent Misdiagnosis of Narcolepsy

Overview

This project aims to create a machine learning model that can effectively classify narcolepsy and epilepsy EEG signals in order to prevent the likelihood of misdiagnosis. Although datasets were limited, this provides a foundation for the development of future models that use EEG scans of these two disorders.

Abstract

Sleep constitutes about a third of an individual’s lifetime, and can be tremendously affected by various sleep disorders. Narcolepsy is a sleep disorder that affects an individual's sleep-wake cycles, leading to excessive daytime sleepiness (EDS) and cataplexy. However, EDS and cataplexy can be mistaken for symptoms of epileptic seizures, leading to the misdiagnosis of narcolepsy. Electroencephalograms (EEGs) record the electrical activity of the brain, making them an invaluable tool in the diagnosis of both sleep and neurological disorders. To help prevent misdiagnosis, machine learning models were trained with EEG data obtained from sleep clinics. Data included patients diagnosed with narcolepsy and epilepsy, in addition to healthy controls. Two models, InceptionV3 and Random Forest, were trained with the EEG datasets for classification. Data was preprocessed before training and testing to ensure standardized formats across datasets; this assisted in making the data more suitable for analysis and classification by the algorithm. The model's performance was evaluated using accuracy, precision, recall, and F1-score for classifying narcolepsy and epilepsy. This study provides the foundation for a classification system for narcolepsy and epileptic seizures through the use of EEG recordings. It also presents a foundation for future algorithms that may use narcolepsy EEG data to improve understanding of the disorder. Further work can be done to refine the model so that it can better aid clinicians, potentially leading to faster and more accurate diagnoses and improved patient outcomes.

Graphical Abstract

Research Proposal

Phrase 1

Currently, narcolepsy can be easily misdiagnosed as epileptic seizures.

Phrase 2

This project aims to create an effective and computationally inexpensive classification model that can predict the type of disorder based on data from an EEG.

Background Infographic

Background

Sleep is an important part of life, taking up about a third of a person’s lifetime. Yet, there are many sleep disorders that can affect an individual’s ability to sleep or their experiences while sleeping. Narcolepsy, a rare and chronic neurological disorder that negatively impacts the control of sleep-wake cycles, manifests in different and sometimes even dangerous ways throughout the day in patients. Narcoleptic patients experience excessive daytime sleepiness (EDS), even if they felt rested after waking up. This leads to the possibility of falling asleep in the middle of an activity, such as talking. In more dangerous cases, this can happen when a person is driving, leading to fatal car accidents (de Mello et al., 2013). When this sudden onset of sleep occurs, the person experiences rapid eye movement sleep (REM); in this phase of sleep, they are unable to move. Another potentially life-threatening symptom of narcolepsy is cataplexy. The two types of narcolepsy, Type 1 (NT1) and Type 2 (NT2), are differentiated by the presence of cataplexy as a symptom in those with NT1. Cataplexy involves sudden weakness in the muscles, rendering them limp or unable to move. Often triggered by strong emotions such as laughter, cataplexy can range from a mild period of weakness in a few muscles to a complete collapse of the body (NINDS, 2023).

Due to the rarity of the disease, affecting about one in 2,000 people, the pathophysiology of narcolepsy has yet to be fully explored. Narcolepsy with cataplexy is characterized by the deficiency of hypocretin, also known as orexin–a neuropeptide that plays a role in stimulating wakefulness. The loss of hypocretin-producing neurons in the hypothalamus of the brain contribute to their deficiency in narcoleptic patients (Liblau et al., 2015). Furthermore, symptoms of narcolepsy, particularly cataplexy, can look similar to epileptic seizures (Diukova et al., 2021). As narcolepsy appears to have common symptoms with epilepsy, misdiagnosis is not rare, especially in the case of children and adolescents due to the onset of narcolepsy (Baiardi et al., 2015). Misdiagnosis presents a significant diagnosis challenge in clinical practice, and electrophysiological studies are necessary for differential diagnosis (Baiardi et al., 2015). An electroencephalogram (EEG), frequently used in the diagnosis of sleep and neurological disorders, records the electrical signals from the brain. The Multiple Sleep Latency Test (MSLT) checks EDS by measuring the time it takes for someone to go from the start of a daytime nap to the first signs of sleep, known as sleep latency. In addition, the Epworth Sleepiness Scale (ESS) is a questionnaire that is reliably used in studies to measure the frequency of daytime sleepiness (Zhang & Zhao, 2008). In the case of narcoleptic patients who are incorrectly diagnosed, it will take substantially more time to acquire proper diagnosis and treatment, if at all.

Recently, many machine learning algorithms have been trained to help identify different disorders from EEG recordings. As EEGs are an important diagnosis tool for both sleep disorders and neurological conditions, there is an abundance of EEG data for both narcolepsy and epilepsy patients. Machine learning classification models are trained with datasets of both healthy and affected individuals; as these models become increasingly accurate, their current clinical implications play a growing role in diagnosis of diseases. When using EEG recordings for image classification, one of the most commonly used algorithms is a convolutional neural network (CNN) (Pawan & Dhiman, 2023). CNNs are useful for working with two-dimensional data, and are composed of many layers–the highest-level building blocks in deep learning. Each layer will receive input from a previous layer, and after transforming the values it obtains, it then passes these values to the next layer. The convolutional layers of a CNN perform calculations at each part of the image, which helps it identify specific details. Afterwards, the pooling layers take this data and simplify it. Activation functions in the output layer such as the Rectified Linear Unit (ReLU) and sigmoid are then used in learning relationships between features. Ultimately, the model is able to extract key features and improve its accuracy on predictions through more training data. One of the most common models used is Inception V3, a type of CNN; it is 48 layers deep and offers both good accuracy and computational cost, making it a robust model for image classification. In addition, ensemble models such as random forests utilize the predictions of many different models and merge them together to evaluate their own classification system. This aims to even out the biases of individual models, and leverages the collective decisions and intelligence of all the models that it draws from (Mahajan et al., 2023).

Methods Infographic

Methods

Differentiating between narcolepsy and seizure disorder can be challenging due to overlapping symptoms. This study aims to develop a machine learning model that utilizes electroencephalography (EEG) data to improve differential diagnosis accuracy.

This project was done using the Python programming language, incorporating libraries and modules such as TensorFlow and NumPy. Convolutional neural networks (CNNs) and ensemble learning models, including InceptionV3 and random forest respectively, were used for dataset training. CNNs are used in EEG recognition due to their ability to pick out key features for classification (Pawan & Dhiman, 2023). The computation efficiency that results from these layers is a significant advantage of the model, as the amount of storage on the computer that ran the algorithm was limited. In addition, the random forest model was trained and tested. Random forests are resilient to EEG noise, reducing the risk of misclassification due to external factors that interfere with the desired signal. Different individuals present with varying EEG patterns within each disorder. Random forests can effectively adapt to this heterogeneity, potentially improving generalizability and clinical applicability (Rigatti, 2017).

Data for each disorder was collected from online datasets. In addition to data from patients with narcolepsy and seizure disorders, a healthy control group was obtained to set a baseline reference that helped the model in pinpointing key features of each disorder. Standardized acquisition protocols using EEG equipment ensure data reliability. Informed consent was obtained from all participants diagnosed with narcolepsy or seizure disorder, adhering to ethical guidelines and anonymizing personal information. The datasets consist of recordings labeled as either characteristic of narcolepsy, seizure disorder, or a healthy control. Data quality control procedures include artifact detection and removal, bandpass filtering, and channel selection. Removing artifacts, signals that interfere with the desired data, will help with enhanced reliability and accuracy of results. Bandpass filtering allows the algorithm to focus on a specific frequency range of signals, since various brain waves with different frequencies are presented on each EEG recording. Additionally, by focusing on specific EEG channels, the computational efficiency and validity of the model is further improved. To compensate for using datasets from different sleep clinics and hospitals, pre-processing was done on each dataset before using them to train or test the model.

Data was split into training and testing sets, which comprised 70% and 30% of the data respectively, which is a common practice in machine learning to avoid overfitting. The training set was used to train the model, while the testing set was used after the training set to assess the model’s accuracy, precision, recall, and F1-score for both narcolepsy and seizure disorder classification, in addition to healthy patients.

Results

Analysis

The random forest model achieved an 86.5% accuracy in differentiating epilepsy patients from healthy controls. This suggests that machine learning models can be trained with EEG data to effectively distinguish between certain neurological disorders. Random forests are known for their robustness to noise and ability to handle diverse data. This suggests the potential for developing a model that can be applied to various clinical settings and patient populations. In addition, the InceptionV3 CNN achieved 79.1% accuracy. Both of these models highlight the need for further optimization and exploration of different algorithms and model architectures specific to narcolepsy EEG data.

A confusion matrix was developed to help summarize the performance of each algorithm. The boxes in the confusion matrix represent true positives, false positives, false negatives, and true negatives. They are used to calculate precision and recall, which in turn are used for calculating the F1 score. The precision and recall of the InceptionV3 model were 69.9% and 81.5% respectively, and the F1 score was calculated to be 75.2%. The F1 score is the harmonic mean of precision and recall, and evaluates the overall performances. The precision and recall of the Random Forest model were 86.3% and 84.8% respectively, and the F1 score was calculated to be 83.4%.

Discussion/Conclusion

A major limitation of this study is the lack of compatibility between the obtained datasets. As the narcolepsy data was not able to effectively be quantified, there were issues with what could be done for a multiclass model for narcolepsy and epilepsy. Another potential limitation of this study is the sample size. As data for training and testing was limited, a higher accuracy of the model could be achieved with a greater abundance of data and greater compatibility between datasets. The study utilized data from online, potentially limiting the generalizability of the model to broader populations. Future research could incorporate larger datasets to ensure the model's effectiveness in different clinical settings. While the random forest model performed well for epilepsy classification, the InceptionV3 CNN only achieved 61.3% accuracy. This highlights the need for further optimization and exploration of different algorithms specific to narcolepsy EEG data. Further research is needed to evaluate the model's performance in real-world clinical settings. This includes assessing its effectiveness in tandem with other diagnostic tools and how clinicians can use these tools to more accurately make a diagnosis. By analyzing variable importance measures in the random forest model, researchers could potentially identify specific EEG features that differentiate between narcolepsy and other disorders. This knowledge could contribute to further understanding the pathophysiology of narcolepsy and guide future diagnoses.

Overall, this study explores the application of machine learning in diagnosing different neurological disorders. As narcolepsy and epilepsy are frequently misdiagnosed due to physical symptoms that can be mistaken, this model provides the foundation for a potentially effective diagnostic tool in clinical practice. The results from the random forest model suggest that this approach has the potential to improve the accuracy and efficiency of narcolepsy diagnosis. However, it is shown in this study that data must first be made compatible, as there were difficulties in integrating the narcolepsy data and turning the binary classification model into a multiclass classification model. Addressing the limitations and conducting further research is crucial to ensure the development of a robust tool that can be used in a clinical setting.

References

February Fair Poster