3

This might not be a very good question, but I would still ask if it's beneficial to do EDA before running a clustering algorithm?

I understand that EDA helps us generate good and helpful insights into the data, which is crucial in data understanding. If we leave aside standard checks and manipulations like - removing outliers, scaling, removing constant value columns, removing null/'zero' value columns, etc. and if we have 20-30 features. How will EDA help me in producing good and sensible clusters? Is it even necessary to do the EDA before clustering?

Note: I am using k-means

Akash Dubey
  • 696
  • 2
  • 5
  • 19

3 Answers3

0

From a data management, data engineering, and data analytics perspective, basic EDA will force you to slice and group your data with liked data types. Creating a situation where you will be forced to do basic data management and data engineering to treat data quality and integrity issues before doing advanced work on it. For example, low quality data is data that is incomplete, poorly encoded, or simply hard to use as-is.

From an statistics and artificial intelligence and machine learning (AIML) perspective, data is represented in different form factors, which means you cannot do clustering if you have, for example, a mix of continuous and discrete values or numerical and categorical data.

From data retrieval and natural language processing (NLP) perspective, EDA helps you see patterns too. It helps you create a baseline for whatever problem you're trying to solve or investigate.

Overall. For EDA to work (for you to drive the experience), you have to ideate and visualize your objective first. Running data into python (sklearn, PyTorch etc) or R (Caret, dplyr etc) without a goal, scope, and/or purpose will give you some values (result), but you will not be driving the experience. The python and R libraries other people wrote will. This will mean you will not be able to explain your EDA to your professor (if school project) or to your boss (if part of job project)

Full Array
  • 224
  • 1
  • 7
0

EDA helps in cleaning data (like handling missing values or outliers, if any), but also in discover some patterns across features; for this scope is very useful do a multivariate analysis, with conditional charts, where in a single graph you combine 2/3 features (think about conditional scatterplot for example).

However like in previous answer it's crucial have and objective, and EDA helps in making a good clustering that well meets your goal.

Alessio Calvagno
  • 1
  • 2
0

How would you know you have to do cluster analysis before looking at your data ?

Setting aside data quality questions (which you should never do), a bare minimum of EDA will help you :

  • Know if it's relevant to do a clustering analysis (rarely imo)
  • Know if K-means is the best clustering tool (rarely imo)
  • Get an idea of the number of the clusters

Then you should do some EDA after, to understand what are the clusters you have selected.

Edit: Basically, it will help you answers this kind of question : How do I interpret my result of clustering?

Lucas Morin
  • 2,775
  • 5
  • 25
  • 47