Automating tasks in Healthcare with Machine Learning

The following blogpost is aiming towards the healthcare industry, with the objective to enlighten how machine learning can automate processes, and thereby reduce workload for nurses and doctors. A case will be shown on predicting whether a given tumor in the breast is malignant or benign, but could as well be a picture from a X-ray detecting whether a bone is broken or not.

We have seen an increase in doctors and nurses who experience stress in their working days. By using machine learning, we can automate some of these processes, helping doctors and nurses overcome the continually stress they are on.

Stress in the public sector

The number of people suffering from stress within the public sector has increased over the past years in Denmark. According to Arbejdsbevægelsens Erhvervsråd, in 2016 almost 25 % of workers in the public sector experienced symptoms of stress. This is an increase of 17 % over a four year period [1]. Additionally, according to the publication Arbejdsmiljø i Danmark, one of the most stressed industries is hospitals. Specific doctors experience more stress than people in other industries [2].

We know that the Government reduces the budget for hospitals, as well as every second nurse have expressed a busy workday that is affecting the safety of patients [3].

Let us look at the circle of cost, quality and delivery (speed). The idea of this model is, that you cannot have them all, and therefore must choose to focus on primarily two, and only two. The government has decided to reduce costs.

As expressed in Sygeplejersken, every fifth nurse experience that he/she cannot get professional assistance when needed [3]. Additionally, patients become upset if they have to wait for a long time, as well as emergencies can happen requiring immediate attention, most nurses and doctors may see themselves forced to put their focus on delivery (or speed). In a self-reported survey amongst nurses, it can be seen that those who have to work fast, experience that it has a direct effect on the quality of their work. Additionally, all but 3% in that same group experienced that it directly affected the quality of their core competencies.

However, for those who are passionate about their job, it would be hard to see the quality be reduced, which then might stress these employees more, as they are forced on maintain a high quality. With help from machine learning it is possible to automate some of the processes the staff at hospitals have which in turn can reduce the number of tasks expected to be performed by the nurses and doctors. One of these tasks could be to detect whether a given tumor in the breast is malignant or benign, or it could be to use computer vision to detect broken body parts in X-ray scanning.

Predicting whether a breast tumor is malignant or benign

Following dataset is from Kaggle and aims to predict whether a given tumor is malignant or benign by help of 30 features. The dataset is rather small consisting of:

The first thing we do is to change our output variable from B and M to a binary: 0 and 1. After that, we remove the NaN-values before making a correlation HeatMap using Spearman’s correlation.

Next step is to split our dataset into train and test, providing a training size of 80% and a test size of 20% of the observations.
We will use a Support Vector Machine algorithm to classify the given tumor, as our dataset is a binary problem, not extremely complex (that would have called for neural networks), and is a rather small dataset with only 569 observations. Support vector machine algorithm is extremely powerful for problems like these!
After splitting our dataset in to train and test, we use GridSearchCV to find the best C-value for our support vector machine:

1
2
3
4
5
6
7
8
9

model = GridSearchCV(model,
                     parameters,
                     cv=5,
                     verbose=1,
                     n_jobs=3)
model.fit(X_train, y_train)
params = model.best_params_
X = params.get(“C)
print(C) 

The C-value is then used for our support vector machine:

1
2

model = svm.SVC(C, gamma=“scale”)
model.fit(X_train, y_train)

With standardized dataset, and all features except the null-values and ID we get an accuracy of:
Train: 0.9869
Test: 0.9649
AUC: 0.9623
However, we can tune the model, by removing features with none to weak correlation that punishes our models accuracy. After adjusting our model, we end up removing 10 features totally. That means, our model now “only” takes 20 features for predicting the outcome with a higher accuracy of:
Train: 0.9956
Test: 0.9737
AUC: 0.9643

We save the model, and then test the model on the entire dataset, to test how well our model performs on a larger scale, providing an accuracy score of:
Confidence: 0.9912
AUC: 0.9882

Conclusion on using machine learning for predicting the tumor

We have created a model that can predict correctly, in almost all cases, whether a given tumor is benign or malignant.
It is crucial to make sure our model is good at predicting false negatives: The model predicts the patient do not have cancer but they do.

When testing the model on the entire dataset and making a confusion matrix, our model shows no false positives but 5 false-negatives which is great considering how small the dataset is.

Yet, sending 5 patients away with the news that the tumor is benign when it in reality is malignant is not satisfying. Thankfully, a lot of machine learning model’s output value actually comes out as probabilities or confidence levels. What you can do for reducing the risk of false negatives is to create a warning system, so if the probability is around 0.3 or 0.4 (the model would give an output of 0, but is in reality not sure at all), a doctor should take a look at the results to double-check. On that way, doctors only have to look at those cases, where the model is insecure. Additionally, as the model is re-trained on new data and the dataset becomes larger, the probability for having 5 false-negatives will be reduced as more data means that the model have more information to use, when finding patterns.

Taking the model to production

The model is strong in predicting whether a given tumor is malignant or benign based on only 569 observations. It does however demand information about 20 features to be able to make the prediction. This would take time for a nurse or doctor to type in, and might take longer than investigating it manually as they are doing today. However, it might be possible to automate the input based on the information retrieved from the scanner of the tumor, making it a full-automated process. This process should of course be tailored to the information a given hospital is using to make predictions, but could provide more time for doctors and nurses to do other important tasks. By taking the data from a mammography, it would additionally be possible to automate the process of detecting whether the given patient has a tumor, with computer vision algorithms. In this case the doctor only has to investigate where the model is unsure (based on a confidence level), or detects cancer.

In other processes involved with the healthcare, machine learning can be used for documentation. By using algorithms for speech-to-text, we can create a model that can reduce the time it takes doctors and nurses to do the required documentation in the patient chart. Google’s Live Transcribe has for instance proven how the technology today is ready! Doctors and nurses can with a speech-to-text algorithm do the documentation verbally, and the algorithm will detect what is said, and write that down! Additionally, machine learning can be used to reduce the amount of machinery breakdown by using predictive maintenance.

GDPR and healthcare

We know machine learning in health care is going to be big as it has the potential to automate processes and make it easier for doctors and nurses to do their job. However, with GDPR it has become harder to research in health sector as sensitive data is needed to predict the probability of future events or train a model to classify given diagnosis and probabilities. It is not necessary to know a person’s personal identification number, however information such as the gender, age and sometimes weight and height is needed. When creating a model for predicting the probability of breast cancer in a mammography picture, it is necessary to get pictures of both healthy and cancer-infected breasts to be able to learn to distinguish between the two as well as finding patterns. However, these pictures can be anonymised so that they can not be traced back to the person involved which is important doing in cases like this one.

// Maria Hvid, Machine Learning Engineer @ neurospace

References

[1] Emilie Lichtenberg (2018) Flere oplever stress - især blandt offentligt ansatte. Arbejderbevægelsens Erhvervsråd, august, 9., 2018, Copenhagen.

[2] Arbejdsmiljø (n.d.) Hvor mange er stressede? (https://amid.dk/viden-og-forebyggelse/psykisk-arbejdsmiljoe/stress/viden-om-stress/hvor-mange-er-stressede/)

[3] Sygeplejersken Danish Journal of Nursing (2015) Hver anden sygeplejerske: Travlhed truer patientsikkerheden(6) (https://dsr.dk/file/13057/download?token=pJg0Mpz-)