0

I have two really large arrays that can't fit in memory at once. I am interested in plotting a 2D histogram of these two arrays with a colorbar. In my case I loop over subsequent chunks of arrays and keep appending first data to x and y, such that:

n_bins=99
histogram = np.zeros(shape=(1,n_bins), dtype=np.float64)  
histogramy = np.zeros(shape=(1,n_bins), dtype=np.float64)

and below happens in each (for) loop iteration when desired chunk length of x is reached:

add_coip, _ = np.histogram(np.asarray(x)[:], bins=(xedges))
add_coipy, _ = np.histogram(np.asarray(y)[:], bins=(yedges))

histogramx += add_coip.astype(np.float64)
histogramy += add_coipy.astype(np.float64)
x,y=[],[] # reset lists to append next chunk

Though I get histogramx and histogramy binned properly, I am unable to plot a 2D histogram using these arrays. Note that histogramx and histogramy doesn not contain data, rather a binned-version of the data. Any suggestions please?

PS: I understand using plt.bar as in this post but that's for a 1D plot!

nuki
  • 101
  • 5
  • Are you using NumPy with Matplotlib to plot? – Shannon Starr Jan 25 '22 at 22:43
  • @ShannonStarr: I actually tried both `plt.hist2d` and `numpy.histogram2d` and had no luck since the arrays have already-binned data instead of the original data. Any suggestions please? – nuki Jan 26 '22 at 00:30
  • Maybe use matplotlib.pyplots.step with documentation here: https://matplotlib.org/3.1.1/api/_as_gen/matplotlib.pyplot.step.html (Sorry I am not totally sure) – Shannon Starr Jan 26 '22 at 00:37
  • @ShannonStarr: Yeah, that actually generates a 1D histogram. My objective here is to generate a 2D histogram. Let me know if you have any other thoughts? – nuki Jan 26 '22 at 17:03
  • I think it might be bar3d in matplotlib https://matplotlib.org/stable/gallery/mplot3d/3d_bars.html – Shannon Starr Jan 26 '22 at 21:35

1 Answers1

0

In case someone else comes across this, you can use plt.matshow as explained here, it has many of the options available that work for plt.hist2d.

valentinwust
  • 1
  • 1
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Sep 24 '22 at 19:09