Python "With open(file) as f" is not saving the content if intrupt in a loop

Question

I mostly use with open('file.txt', 'w') as f: for writing (reading as well). today I notice something weird.

I was crawling a site and there was normal pagination.

While True:
   # visit url, get/scrape data
   # save data in text file
   # find next link(pagination) 
   # loop till next url is available.

for saving data first I did use with

with open('data.txt','w') as f:
    While True:
       # visit url, get/scrape data
       f.write(some_scraped_data)
       # find next link(pagination)
       # loop till next url is available.

but when I run this script and if some exception occurred, this loop gets terminated and no data save in data.txt file

but when I do f = open('data.txt','w') then whatever data is crawled is saved(till exception occurred) even I didn't put f.close()

f = open('data.txt','w')
While True:
    # visit url, get/scrape data
    f.write(some_scraped_data)
    # find next link(pagination) til next url is available.

my question is how can we achieve same thing with with. and I'm just curious where everybody recommends with for file handling and its not supporting this feature.

PS: I'm not so experienced in python. so if you find this question silly, I'm sorry

Answer 1

According to the documentation, the use of 'with' will close the file correctly if an exception occurs, so that is a good approach.

However, you could try f.flush() to get the buffer written to disc. More on flush here: what exactly the python's file.flush() is doing?