How to download a big file from Dropbox with Node.js?

Question

I want to implement a big file downloading (approx. 10-1024 Mb). I've already succeeded to get a file from Dropbox:

operationResult = await dbx.filesDownload({
    path: `/${CONFIG_STORAGE.uploader.assetsPath}/${fileUUID}`
});

Then I bundle the received file with a meta-data and I return it to my Node.js server:

fileMIME = mime.lookup(operationResult.name);

const downloadResult = Object.freeze({
    fileBinary: operationResult.fileBinary,
    fileLength: operationResult.fileBinary.length,
    fileMIME,
    fileName: operationResult.name,
    isSucceeded,
    message
});

return downloadResult;

Now I convert a Buffer, I got from Dropbox, into a Readable stream and pipe it back to a client:

res.setHeader("Content-Disposition", "attachment; filename=" + downloadResult.fileName);
res.setHeader("Content-Type", downloadResult.fileMIME);

const fileReadableStream = new Readable();

fileReadableStream.push(downloadResult.fileBinary);
fileReadableStream.push(null);

fileReadableStream.pipe(res);

Up until now everything is clear and works. Here I face a first pitfall: I need somehow to trigger a download process in browser.

In many examples, some small image or JSON are used, which we can completely load into RAM, make operations, e.g. transforming to Base64, assign it to a.href, and trigger a.click(). But since my file is 10-50 Mb I'm not sure if such approach is a correct one.

I've already tried Fetch API:

const response = await fetch(`${host}/download?fileName=${fileName}`, {
    credentials: "same-origin",
    method: "POST",
    mode: "cors"
});

const a = document.createElement("a");
a.href = response.text();
a.download = "MyFile.pdf";
a.click();

But I always get Failed - No file error. I also tried to use jQuery AJAX, and XMLHttpRequest (XHR), but still no file is downloaded.

Perhaps, there is something I'm missing. How to get a 10-1024 Mb file from a server?

P.S. I never thought that such a trivial task, as a file downloading, can be so complicated.

Mike · Accepted Answer · 2020-07-10T15:28:32.417

I've solved the issue by switching from filesDownload to filesGetTemporaryLink, which returns a link to a file instead of the file itself. Then I trigger a downloading of this link.

The final result:

operationResult = await dbx.filesGetTemporaryLink({
    path: `/${CONFIG_STORAGE.uploader.assetsPath}/${fileUUID}`
});

const downloadResult = Object.freeze({
    fileLength: operationResult?.metadata.size,
    fileLink: operationResult?.link,
    fileMIME: mime.lookup(operationResult?.metadata.name),
    fileName: operationResult?.metadata.name,
    isSucceeded,
    message
});

return downloadResult;

Then I send the output to a client:

res.json(downloadResult);

On a client-side I get it via await/async Fetch API call:

const fileResponse = await fetch(``${host}/downloadDocument`, {
    body: JSON.stringify({fileUUID: fileName}),
    cache: "no-cache",
    credentials: "same-origin",
    headers: {
        "Content-Type": "application/json"
    },
    method: "POST",
    mode: "cors"
});

const fileData = await fileResponse.json();

const aTag = document.createElement("a");

aTag.href = fileData.fileLink;
aTag.download = fileData.fileName;
aTag.click();

As a result, a server doesn't have to deal with files at all, no extra CPU, RAM, or traffic impact, no matter what size of a file I'm trying to download.

How to download a big file from Dropbox with Node.js?

1 Answers1

Linked