In my S3 bucket, I have a folder structure of rootFolder1/year/month/day/myfile.csv. Depending on the time, the rootFolder folder could have multiple subfolder, organised by year, month and date, with each date folder containing multiple csv files. There could also be a rootFolder2 in my bucket, with the same folder structure.
I know for downloading object from S3, I can use the getObject method where I can pass in the params object with the bucket Key like so:
var params = {
Key: "book/2020/01/01/all_books_000.csv",
Bucket: 'my-bucket-test'
}
s3.getObject(params, (err, data) => {
if (err) throw err;
// do something with data
})
But in my case I don't want to hardcode the Key in. Is there any elegant way I can call getObject where I can dynamically set the Key, so it downloads the files under each date folder?
edit: for some further clarity, these are the keys I get back when I call s3.listObjects
'author/',
'author/2020/',
'author/2020/01/',
'author/2020/01/01/',
'author/2020/01/01/all_authors_000.csv',
'book/',
'book/2020/',
'book/2020/01/',
'book/2020/01/01/',
'book/2020/01/01/all_books_000.csv',
'book/2020/01/01/all_books_001.csv'