Grocery Shopping Impulse Purchases with Apriori Algorithm and Association Rules in R

In this R tutorial, we will be focusing on purchases for groceries and how one has impulse purchases. Many times we go into a store for something and end up buying many other items and sometimes even forget what we came into the store for; well at least I do. These impulse buys are no coincidence, as retailers use sophisticated data analysis techniques to identify patterns that will drive retail behavior.

In the past, many recommendation systems were based on the subjective intuition of marketing professionals and inventory managers or buyers. Machine learning has been applied with the increase of bar-code scanners, computerized inventory systems, and online shopping trends increasing.

What are Association Rules?

Association rules are the building blocks of a market analysis are the items that may appear in any given transaction. There are groups of items that are surrounded by brackets to indicate that they form a set or an item set that appears in the data with some regularity. This result of a market basket analysis is a collection of association rules that will specify patterns that are found in the relationships among those items.

Most association rules are used for market basket analysis, but there are other potential applications:

1. Searching for interesting and frequently occurring patterns of DNA and protein sequences in cancer data.
2. Patterns among purchases or medical claims among credit card fraud or insurance use.
3. Identify combinations of behavior that precede customers dropping their cellular phone service or upgrading their cable television package.

This report will be using market basket analysis.

Install and Load Packages

Below are the packages and libraries that we will need to load to complete this tutorial.

Input:

Import the Grocery Dataset

Since we will be using the groceries dataset, you will need to download this dataset. This dataset is already packaged and available for an easy download from the dataset page or directly from the below

1. Groceries – groceries.csv
2. Grocery Rules – groceryrules.csv

Input:

View the Grocery Dataset

str() function

Input:

Output:

summary() function

Input:

Output:

inspect() function

We can use the inspect() function to view the first five transactions.

Input:

Output:

itemFrequency() function

Also, we can use the itemFrequency() function to view the most frequently bought items.

Input:

Output:

Top 20 Grocery Items Plot

This plot will show decreasing support and diagram of the top 20 grocery items.

Input:

Output:

Grocery Items Transaction Visualization

The below may be hard to read, but it will show the 5 transactions, with four items each. I like this visualization because it’s very useful for data exploring.

Columns that could be filled all the way down could indicate items that are purchased in every transaction; a problem that could arise if a retailer’s name or ID was included int he data.

In addition, depending on the season or holiday, toys, candy or a turkey could be more common. This type of visualization will be very powerful if the items were also sorted into categories.

Matrix with 5 rows and 169 columns:

Matrix with 100 rows and 100 columns:

Train the Model with Apriori Algorithm

We will be using the Apriori algorithm to explore and prepare the groceries data.

Input:

Association rules learning with Apriori Algorithm

The Apriori algorithm is the most-widely used approach for efficiently searching large databases for rules. The algorithm name is derived from that fact that the algorithm utilizes a simple prior believe about the properties of frequent itemsets. Below are a few strengths and weakness of Apriori:

Strengths

1. Capable of working with large data of transactions(retail)
2. Results in rules that are easy to understand
3. Great for data mining and discovering unexpected kofwledge in data

Weaknesses

1. Not very helpful for small data
2. Requires effort to separate the true insight from common sense
3. Easy to draw spurious conclusions from random patterns

The support of an itemset or rule measures how frequently it occurs in the data. The rule’s confidence is a measurement of its predictive power or accuracy. It is defined as the support of the itemset containing both X and Y divided by the support of the itemset containing only X. Rules like {peanut butter} –> {jelly} are known as strong rules, because they both have high support and confidence.

Input:

Output:

Input:

Output:

Input::

Output:

Input:

Output:

Improve the model

Improvement is best by sorting and evaluation rules based on different criteria. Depending on the objectives of the market basket analysis, the most useful rules might be done with the highest support, confidence, or lift. Below we will be using the sort() function with vector operators, we can obtain a specific number of interesting rules.

We will be using the best 5 rules according to the lift statistic and can be examined using the below.

Input:

Output:

Subset of Berries

If one wanted to put together a campaign to sell berries, one would need to learn if berries are often purchased with other items. We can do this by creating a subset() function which will provide a method to search for subsets of transactions, items, and rules.

One can see below that berries are often purchased with whipped.sour cream, yogurt, other vegetables and whole milk.

This function is very powerful for marketing and how to target an audience with other items for purchase.

The function subset() is very powerful and below are a few topics to remember:

1. The keyword items explained should match an item appearing anywhere in the rule. One can limit the subset on the left or ride side by using lhs or rhs.
2. The operator %in% means that at least one of the items must be found in the list you defined. For instance, one could write a subset to search for berries or yogurt as such; %in%c(“berries”, “yogurt”)
3. Additional operators are available to partial search (%pin) and complete search (%ain).
4. Subsets can be limited by support, confidence, or lift. For example, one could search for ruled with confidence greater than 50 percent; confidence > 0.50

Input:

Output:

Additional Subsets for Groceries

Let’s search multiple items(berries and yogurt) with a confidence level greater than 60 percent.

Input:

Output:

Saving Association Rules

By using the write() function we can export the grocery rules or you can find them on the R-ALGO Engineering Big Data dataset page.

Input:

The association rules are frequently used to find useful insights in the transnational databases. Users are able to extract data from databases without any prior knowledge of what patterns a person may wish to seek. When we used the Apriori algorithm was used for the grocery data, this was used by setting minimum thresholds of interestingness and reported any associations meeting that criteria.