alternatives to pyspark to query a csv file locally (no distributed computation needed)

Question

I am reading a csv file with pyspark to extract some information from it. I am running pyspark locally and I do not need distributed computation. On the contrary I need some plotting library such as matplotlib to plot data. I know there are some workarounds to do it with pyspark but I found it very complicated.

My question is: is there any alternative to pyspark, possibly python-native, that allows to query a csv file allowing to use python libraries?

score 2 · Answer 1 · edited Jan 14 '21 at 15:01

2

pandas can do it. It can read in a csv file with a single line and put the data in a dataframe that you can manipulate.

edited Jan 14 '21 at 15:01

Stephen Rauch

1,831
11
23
34

answered Jan 14 '21 at 09:09

hayderOICO

21
2

alternatives to pyspark to query a csv file locally (no distributed computation needed)

1 Answers1