Breast Cancer Dataset Analysis, Visualization and Machine Learning in Python

In this Python tutorial, learn to analyze and visualize the Wisconsin breast cancer dataset. This tutorial will analyze how data can be used to predict which type of breast cancer one may have.  By analyzing the breast cancer data, we will also implement machine learning in separate posts and how it can be used to predict breast cancer. The purpose of this tutorial is to help provide a basic understanding of data cleansing, data exploration, feature selection, model evaluation and model selection.

Importance of Data Mining

The field of data mining is one of the most robust technological fields in the modern digital economy. More and more companies are hiring data professionals and spending time and money perfecting their data procedures. Data mining operations are becoming essential to modern competition in the 21st century economy. They are helping factories and health care offices save thousands of dollars every year in their operations. They are allowing companies to microtarget advertisements and get more out of each dollar spent. A continuing trend has been to start hiring individuals in this field who will help organize these data approaches.

Data Manipulation and Predictions

These individuals are tasked with learning a massive amount of information in a short period of time. They have to be familiar with the dozens and hundreds of different ways that a computer can manipulate a large amount of data and use that manipulated information to make sense of data. A data professional may have to streamline a thousand different procedures or make sense of a million data points.

In order to be successful, a data mining professional must be familiar with all of these procedures and how to use them. They must know how to analyze and apply common forms of machine learning. These forms are applicable for a wide variety of data situations and can help a data professional classify, organize, and use data in order to make predictions.

Breast Cancer Tumors

A mass of abnormal tissue is known as a tumor. There are two types of tumors: benign, which are non-cancerous and malignant, which are cancerous.

Benign Tumors

In most cases, a doctor diagnosing a tumor as benign will most likely be left alone.  Benign tumors are not generally aggressive around the surrounding tissue and in some cases, may continue to grow. If the tumor continues to grow and cause discomfort by pressing against surrounding organs and causing pain, the tumor would be removed.

Malignant Tumors

Malignant tumors are aggressive and cancerous because damage the surrounding tissue and may be removed depending on the cancerous and aggressive on the severity or aggressiveness of the tumor.

Breast Cancer Dataset Analysis

The below few section will completed data analysis of the breast cancer dataset before we work into the visualizing the breast cancer dataset.

Load the Breast Cancer Dataset

The first step is loading the breast cancer dataset and then importing the data with pandas using the pd.read_csv method. This will save the object containing digits data and the attributes associated with it.

pd.read_csv()

breast_cancer.columns

We can use the breast_cancer.columns method to print out the column names.

Input:

Output:

breast_cancer.head()

The breast_cancer.head() method will print out the data for the first 5 rows.

Input:

Output:

breast_cancer.shape

The breast_cancer.shape() method show the total of columns and total of rows.

Input:

Output:

breast_cancer.info()

The breast_cancer.info() method will provide all necessary information about the breast cancer dataset in one output.

Input:

Output:

Breast Cancer Diagnosis Size

The .size() will return an int representing the number of elements if an object. If it’s a series, it will return the number of rows and if a DataFrame it will return the number of rows times number of columns.

Input:

Output:

Breast Cancer Data Visualization

The seanborn package offers a count plot that is thought of a histogram across a categorical, instead of quantitative, variable. The count plot will plot the counted of observations in each categorical bin using bars.

When inputting data, the data can be passed in a variety of formats including below:

• A DataFrame in long-form will determine how the data are plotted in which the case the  x, y, and hue variables will determine how the data is plotted.
• A DataFrame in wide-form will plot each numeric column.
• An array or list of vectors.

Import Packages

Seaborn Diagnosis Countplot

The seaborn package provides a function called color_palette() that allows discrete color palettes. This will provide an interface to generate colors in seaborn, but also provide Hex color codes to your liking. In this example, we will provide the Hex color codes of #FF1493 (deep pink) and #FF69B4 (hot pink).

Input:

Output:

Pandas Pairwise Correlation

The Pandas breast_cancer.corr() is used to find the pairwise correlation of all columns in the breast cancer dataframe. Correlation is used when referencing the strength of a relationship between two variables have a high/strong correlation means.

We must first drop the ‘id’ column as it provides no value when analyzing the dataset.

Input:

Output:

Seaborn Correlation Heatmap Matrix

A heatmap is a two-dimensional graphical representation of data values that are contained in a visualized matrix.

The seaborn Python package allows the creation of heatmaps which can be tweaked using matplotlib tools. For this example, we will be using the matplotlibs ListedColormap to customize the colors of the heatmap.

Input:

Output:

Highest Correlation Breast Cancer Features

In order to plot the next few plots, we must be able to analyze which features have the highest correlation.

The breast_cancer_corr data frame has 30 rows and 30 columns. This means when we sort the features with the highest correlation, the dataframe output will have have 900 rows.

breast_cancer_corr.abs()

The abs() function is used to return the absolute value of a number.

high_correlation.unstack()

The unstack() function in a dataframe unstacks the row to columns.

high_correlation_unstack.sort_values(ascending=False)

The sort.values() function will sort the values in ascending or descending order. Also, we will print the high correlation between 30 and 35 because the first the 29 values will have a correlation of 1.000000.

Input:

Output:

From the above we can see the highest correlation between features that do not equal 1.0000. In the few section we can use any of the above, but for simplicity we will use the radius_worst and perimeter_worst.

Seaborn High Correlation Scatterplot

The seaborn scatterplot will be used to plot two high-correlation variables from the Wisconsin breast cancer dataset. The two variables that have a high correlation are radius_worst and perimeter_worst will provide analysis in the strength of that relationship with the available statistical data.

Input:

Output:

Seaborn High Correlation Pairplot

The seaborn pairplot is made to visualize the relationship between two variables, where the variables can be continuous.

Input:

Output:

Machine Learning Algorithms for Breast Cancer

Machine learning and data mining go hand-in-hand when working with data. Machine learning algorithms are referred from data mining and other big data tools that make use of big data. When working with large sets of data, it can be processed and understood by human beings because of the large quantities of quantitative data.

The Wisconsin breast cancer dataset can have multiple algorithms implemented to detect the diagnosis of benign or malignant. The below machine algorithms will be implemented with the breast cancer dataset in separate tutorials to fully focus on each algorithm.

Python Machine Learning Algorithm in Scope:

• • Decision Trees Algorithm
  • Gradient Boosting Algorithm 
  • k-Nearest Neighbors Algorithm
  • Logistic Regression Algorithm
  • Random Forests Algorithm
  • Support Vector Machine Algorithm

R Programming Machine Learning Algorithm in Scope:

• • k-Nearest Neighbors Algorithm