Breast Cancer Prediction using Random Forest Algorithm in Python

In this Python tutorial, we will analyze the Wisconsin breast cancer dataset for prediction using random forest machine learning algorithm. The Wisconsin breast cancer dataset can be downloaded from our datasets page.

Random Forest Machine Learning Algorithm

Random forests are a decision tool that is used to classify pieces of data and help guide machines to make decisions. A random forest has the same basic structure as a decision tree. It is a machine learning algorithm that combines a large number of possibilities with the probability of event occurring. The random forests algorithm is particularly helpful with a machine learning training set. Some machine learning operators want their artificial intelligence systems to work within a small set of parameters. Others want a more expansive data set that goes beyond the abilities of individuals.

Most data professionals will often opt for a decision tree if it is at all possible. The decision tree is an approach to probability and percentages that is taught to many individuals in high school. A decision tree can be understood by people who know nothing about any other data algorithm. They may not be familiar with random forests and may be worried about the applicability of such a complex system. But random forests have their own applicability in many instances. A decision tree may overfit for a training set and will reduce the number of possible outcomes that the operation can provide percentages for.

Possibilities and Probabilities

Random forests involve a machine learning program analyzing a wide variety of potential decision tree and then viewing how those decision trees are developing. The random forest algorithm can detect patterns on what decisions and probabilities are available and then understand those patterns through a series of algorithms. The random forest algorithm then presents a set of possibilities with a set of probabilities just like a decision tree. The main benefit of random forests is that they can overcome potential limitations caused by a basic decision tree.

The random forest algorithm is also easily visualized. A wider variety of individuals can view and understand its decisions than those made by other sophisticated algorithms. But even though the random forest algorithm is simpler than some other algorithms, it is still vastly more complicated than a decision tree and cannot be as easily followed. Sometimes, a decision tree may be all that is needed and all that an individual should use for their particular machine learning problem.

Random Forest Classifier Model

The RandomForestClassifier() from sklearn is a very simple model to build by adding a few parameters to reduce over-fitting. A random forest is a meta estimator that fits a number of decision tree classifiers on various sub-samples of the dataset. In addition, uses averaging to improve the predictive accuracy and control over-fitting. The sub-sample size is always the same as the original input sample size but the samples are drawn with replacement if bootstrap=True (default).

Import Packages and Diabetes Dataset

Train and Test Data

Before we can create the training and test data, we must remove the ‘id’ column. This column provides no value on the prediction on breast cancer and can be removed with the del function from pandas.

Build the Random Forest Classifier Model

The below model will use the n_estimators parameter with an integer of 100. The n_estimators parameter is the number of trees in the forest and is optional parameter with a default of 10.

Input:

Output:

The random forest with 100 trees gives us an accuracy of 94.41% on the testing data but 100% on training set which is an indicator of over-fitting. We can implement the max_depth parameter that will set at 5.

Input:

Output:

By adding the max depth of 5, we improved the training set accuracy to 99.30% and the testing set accuracy score from stayed the same at 94.41%.

Random Forest Classifier Feature Importance

The RandomForestClassifier() has an attribute that returns the feature importance of each feature.

Input:

Output:

Random Forest Classifier Feature Importance Plot

Input:

Output: