0

I have two pandas dataframe (let's call them df1 and df2), with 132 columns each and one used as index (index) and i need to merge df2 to df1 using these rules:

  • Update If an index of df2 is in df1 update the specific row of df1 with the values from df2
  • Insert If a new df2 index is found append the row to df1

The first dataframe has 500000 rows and this operation will be executed every day with a new df2.

I'have tried with pandas.update() but it seems to just solve the first point of my problem (Update) while pd.concat([df2[~df2["index"].isin(df1["index"])], df1]) seems to just solve the Insert problem.

Is there a smart way using Pandas or other packages to do this?

Duccio Borchi
  • 209
  • 4
  • 13
  • If you look in the first duplicate link at the accepted answer, you see it is about updating and inserting. – Michel Oct 13 '22 at 10:38

0 Answers0