I am trying to solve a rather difficult issue at my job right now. We are interested in installing a set of automatic trays in our warehouse, each of which can hold $N \in \{5, 6, \dots, 20 \}$ unique items. I would like to choose items for each tray according to how likely they are to get ordered with each other. So, if item $A$ is always ordered with item $B$, and item $C$ sometimes goes with item $D$ but always goes with item $E$, $A$ & $B$ go on one tray, while $B$ & $D$ go on another (and perhaps $D$, but only if there's room).
The information I have is as follows.
A dataset of every order from our history (every item included in that order)
A list of the most common items ($a_0, a_1, \dots , a_n$), sorted by how often it is ordered
A matrix containing the probability that item $a_x$ is featured with any other item $a_0, a_1, \dots , a_n$.
Currently, the probability is calculated as the number of orders featuring both $a_x$ and $a_y$ divided by the number of orders featuring $a_x$. However, I believe this may be non-ideal, and could instead be calculated as the number of orders featuring $a_x$ and $a_y$ divided by the total number of orders.
This square matrix is approximately $500 \times 500$, but for all intents and purposes is an arbitrary $n \times n$.
A similar question was asked in this question (Clustering algorithm to cluster objects based on their relation weight), whose matrix is the same structure as mine, however the two answers seem to disagree, so it is unclear which option is actually the best.
I should be able to get more, but this is what I currently have.
Is there a specific algorithm that is best suited to this problem? Previously, I was planning on using a K-Means clustering algorithm, based on the aforementioned probability matrix. This seemed like a good idea, but, from that question and Meltem Tutar's article, it seems that an alternative clustering algorithm such as SVD- or kernel-based K-means algorithms might be better. I believe that using standard k-means clustering may result in items with similar (but otherwise unrelated) kinds of probability will get clustered. (For example, 5 items that all have an 80% chance of being included with item A but a near-zero percent chance of being included with each other, will get grouped together, while item A will get clustered with other items. This may be incorrect, though.)
Ultimately, the solution does not need to be perfect, just mostly optimized and, most importantly, relatively easy to implement. The big caveat to this question is that I am not a mathematician or programmer, I am a manufacturing engineer. That being said, I do have recent experience in both (I am familiar with the concepts of kernels, SVD, and eigenvalues/vectors, but I have limited knowledge beyond that; I also have some coding experience).
To summarize, the design goals of the algorithm are:
- relatively simple coding implementation.
- fairly accurate, but not necessarily 100% perfect optimization.
- cluster items such that items with a high likelihood of sharing orders share their cluster.
- each item may only go in one cluster.
- ability to control the number of items in each cluster.
- ability to control the number of clusters (~100).
