I keep on finding myself wanting to download and check the integrity of the download immediately in a script, but I haven't been able to find the right incantation of sha256sum.
MY_SHA256=e147f0392686c40cfd7d5e6f332c6ee74c4eab4d24e2694b3b0a0c037bf51dc5
sha256sum some_binary | sha256sum --check ${MY_SHA256}
How can I take the sha256sum of a new file and compare it with a known hash immediately?
I have downloaded an archive file and an accompanying checksum file. Here is how I verify that the hash of the downloaded archive matches the hash from the downloaded checksum file:
echo "$(cat archive.tar.gz.sha256) archive.tar.gz" | sha256sum --check --status
The --status flag prevents all stdout output (more effective than --quiet). I then need to rely on the return code to determine if they matched, which is what I want anyway since I'm going to be using this in a script.
Example:
echo "67574ee0039eaf4043a237e7c4b0eb432ca07ebf9c7b2dd0667e83bc3900b2cf kali-linux-2019.2-amd64.iso" | sha256sum -c
In case you have the sha256sum file, you can directly use it:
sha256sum -c "kali-linux-2019.2-amd64.iso.txt.sha256sum"
Explanation:
In the above example, you have
echo "<known SHA 256 sum of the file> <name of the file>" | sha256sum -c
sha256sum -c option can either read the SHA256 sum from a sha256sum file or from STDIN. In case you don't have the sha256sum file, then using the echo command you can provide the same details contained in a sha256sum file.
In case you have the sha256sum file, you can directly use it:
sha256sum -c "<sha256sum file name>"
Note:
Alternatively, you can use shasum -a 256 instead of sha256sum where -a specifies the algorithm to be used.
You can see that sha256sum --check takes the output of a previous (regular) sha256sum run: it takes hashes and filenames via stdin, and compares them against actual files.
So the obvious thing to do is to manually give it the output in the format it wants:
$ echo "da39a3ee5e6b4b0d3255bfef95601890afd80709 motd" | sha1sum --check
motd: OK
# 1. Check to see if file "filename" has this expected hash:
# `expected_checksum_hash`
echo "expected_checksum_hash filename" | sha256sum --check
2. Check to see if these two files ("path/to/file1" and "path/to/file2")
have the same checksum hash
echo "$(sha256sum "path/to/file1" | gawk '{ print $1 }') path/to/file2"
| sha256sum --check
OR (same as #2 just above)
file1_hash="$(sha256sum "path/to/file1" | gawk '{ print $1 }')"
&& echo "$file1_hash path/to/file2" | sha256sum --check
Note: you can use sha256sum or sha512sum in any of the examples below. These are the recommended and most-robust cryptographic checksums, with sha512sum, of course, being stronger.
There is also md5sum, but it isn't as robust, but is still commonly used for data integrity checks. Whenever possible, I recommend you use sha256sum or sha512sum instead. Wikipedia states that md5sum is still good for data integrity checks, but is "no longer deemed secure" and shouldn't be used for cryptographic purposes. So, just use sha256sum or sha512sum above, instead.
There are even more, however. Here is a list of the various checksum program you can technically use in any of the examples below:
sha1sum
sha224sum
sha256sum
sha384sum
sha512sum
shasum # general-purpose tool, requires specifying the algorithm
md5sum
sha256sum path/to/any/file
Example:
$ sha256sum FoxitReader.enu.setup.2.4.4.0911.x64.run.tar.gz
6b579bd4ecdf86f7e70a009886c511da0b5085b831b0d6afc42442cabc249b90 FoxitReader.enu.setup.2.4.4.0911.x64.run.tar.gz
Notice that the output of the sha256sum command is the numerical checksum hash followed by the file name this checksum corresponds to. You can store this checksum into a file named sha256sum.txt like this:
sha256sum path/to/file > sha256sum.txt
Now, assuming you want to check the integrity of the file against this known checksum in that file, you can test the file again like this:
# This causes the program to re-do the checksum of the file specified inside
# sha256sum.txt, and then compare it to the checksum in that same file. If they
# (the re-calculated checksum and the previously-stored checksum) match, it will
# output the name of the file followed by "OK".
sha256sum --check sha256sum.txt
Example:
$ sha256sum --check sha256sum.txt
FoxitReader.enu.setup.2.4.4.0911.x64.run.tar.gz: OK
You can also manually pipe these things (the expected checksum hash and filename) to the checksum program, like this. This is really useful for when you need to check a downloaded file against a known checksum published online where you downloaded it. This way you can check for data integrity to ensure the downloaded file was downloaded successfully.
# 1. pipe to the checksum program directly
echo "expected_checksum_hash filename" | sha256sum --check
2. OR, manually create the checksum file, and then run it on that file
as done above
echo "expected_checksum_hash filename" > sha256sum.txt
sha256sum --check sha256sum.txt # same as previously done above
Example of option 1 just above:
$ echo "6b579bd4ecdf86f7e70a009886c511da0b5085b831b0d6afc42442cabc249b90 \
> FoxitReader.enu.setup.2.4.4.0911.x64.run.tar.gz" | sha256sum --check
FoxitReader.enu.setup.2.4.4.0911.x64.run.tar.gz: OK
file1 to file2:Sometimes you have two downloaded files, or two copies of what you think are the same file, and you just want to ensure they are in fact the same (or different). Building on the information above, there are a few ways to do this.
Manually check the checksum of each file, manually looking at the hashes to ensure they match:
sha256sum 'path/to/file1'
sha256sum 'path/to/file2'
# now visually inspect both hashes
OR [RECOMMENDED] automatically test file1 against file2:
# Do some trickery to compare the hash of file1 agains the hash of file2.
# Effectively, what we have done is this:
# `echo "checksum_hash_from_file1 path/to/file2" | sha256sum --check`
# This therefore is checking to see if the hash from file1 matches the hash
# from file2.
echo "$(sha256sum "path/to/file1" | gawk '{ print $1 }') path/to/file2" \
| sha256sum --check
OR (same as just above)
file1_hash="$(sha256sum "path/to/file1" | gawk '{ print $1 }')"
&& echo "$file1_hash path/to/file2" | sha256sum --check
The way this works is that first it checks the checksum of file1, piping the output (hash and filename) to gawk, which is the GNU version of awk, which is a pattern-matching and text processing language. The gawk '{ print $1 }' command simply says to strip the first space-separated text field (indicated by $1), and retain it only. That's the checksum hash from file1. Then, we append the path/to/file2 and pipe this whole thing to be checked, as done previously above.
In effect, we are tricking the checksum program into thinking we have a previously-obtained hash from file2, and we'd like to check it against a newly-calculated hash from file2. Since we used the hash from file1, however, but the filename of file2, we know that if it passes it is really saying file1 and file2 have the same hash, and are therefore identical files.
Example:
# technique 1
$ echo "$(sha256sum "FoxitReader.enu.setup.2.4.4.0911.x64.run.tar.gz" \
| gawk '{ print $1 }') FoxitReader.enu.setup.2.4.4.0911_NEW.x64.run.tar.gz" \
| sha256sum --check
FoxitReader.enu.setup.2.4.4.0911_NEW.x64.run.tar.gz: OK
technique 2
$ file1_hash="$(sha256sum "FoxitReader.enu.setup.2.4.4.0911.x64.run.tar.gz"
| gawk '{ print $1 }')"
&& echo "$file1_hash FoxitReader.enu.setup.2.4.4.0911_NEW.x64.run.tar.gz"
| sha256sum --check
FoxitReader.enu.setup.2.4.4.0911_NEW.x64.run.tar.gz: OK
If you have the checksum file, you can simply do a sha256sum --check. For example:
% ls
elasticsearch-7.9.3-darwin-x86_64.tar.gz # File to check
elasticsearch-7.9.3-darwin-x86_64.tar.gz.sha512 # Checksum file
% sha512sum --check elasticsearch-7.9.3-darwin-x86_64.tar.gz.sha512
elasticsearch-7.9.3-darwin-x86_64.tar.gz: OK
The easiest way, the file with the sha512sums (in my case, "clear-36010-live-desktop.iso-sha512sums") and the file to verify in the same directory, and run
cat clear-36010-live-desktop.iso-sha512sums | sha512sum --check
Note: the content of the sha512sums file is both the value and the filename.
cat clear-36010-live-desktop.iso-sha512sums
8a47e79fc14721b5d0084f03c50ae66d6d7f7eeebb8d80a1dd168b2a2d5e31cb11efd69e202230b85914d159e0a0fbe20d57df08b825fa5aec0eea62fc1d2e64 clear-36010-live-desktop.iso
not exactly answering the question but I made a function shacheck which compares 2 files using their sha256 hash and reports if files are identical or not (used to check file copy went well for example). I put this function in .bashrc and checked, it works. Any comments appreciated.
Listing:
#GOAL: check 2 files are identical using sha256
#$1=1st filename ; $2=2nd filename
shacheck () {
if [ "$#" -ne 2 ]; then
echo "number of arguments passed is $#";
echo 'This function requires exactly 2 parameters, file 1 and file 2 and check they are the same using sha256 checksum';
return -1;
else
number of arguments passed is 2
if [ -f $1 ]; then
ARG1ISFILE=1
else
ARG1ISFILE=0
fi
if [ -f $2 ]; then
ARG2ISFILE=1
else
ARG2ISFILE=0
fi
#echo "$ARG1ISFILE" $ARG1ISFILE
#echo "$ARG2ISFILE" $ARG2ISFILE
TWOARGSAREFILE=$(($ARG1ISFILE + $ARG2ISFILE))
#echo "$TWOARGSAREFILE" $TWOARGSAREFILE
if [ $TWOARGSAREFILE != 2 ]; then
echo "both arguments have to be plain files"
return -1;
else
#echo "both arguments are plain files"
CHECKSUMFILE1=$(sha256sum $1 | awk '{print $1}')
CHECKSUMFILE2=$(sha256sum $2 | awk '{print $1}')
if [ "$CHECKSUMFILE1" = "$CHECKSUMFILE2" ]; then
# echo "files are identical. \$CHECKSUMFILE1 = $CHECKSUMFILE1 \$CHECKSUMFILE2 = $CHECKSUMFILE2"
#what I want as printed format:
# files are identical. new line
# CHECKSUM FILE 1 = $CHECKSUMFILE1 new line
# equals new line
# CHECKSUM FILE 2 = $CHECKSUMFILE2 new line
echo -e "files are identical.\n"
echo -e "CHECKSUM FILE 1 = $CHECKSUMFILE1 \n"
echo -e "equals \n"
echo -e "CHECKSUM FILE 2 = $CHECKSUMFILE2 \n"
else
echo -e "files are different.\n"
echo -e "CHECKSUM FILE 1 = $CHECKSUMFILE1 \n"
echo -e "is different than \n"
echo -e "CHECKSUM FILE 2 = $CHECKSUMFILE2 \n"
fi
fi
fi
}
I had an scenario where the sha256 files did not have the file name in it, no properly formatted SHA256 checksum lines found, as an example:
cat file.sha256
5b2266edc48f958687ab0f597f62dde1c6f01693d988e272411d081292991476
so I had to get each file with .sha256 read it to get the sha and then format the input to sha256sum command, like this:
for file in *.sha256; do echo $(cat $file) $(basename $file .sha256) | sha256sum --check; done
Many software projects provide a checksum file some-release.zip.sha256 in addition to the downloadable file some-release.zip.
On one hand, the filename-checksum entries in .sha256 files use relative paths. On the other hand, the sha256sum command takes the current working directory from which it is called into account.
Assuming some-release.zip.sha256 is located within directory /tmp/, the following example invocation will work regardless of the current working directory, and will keep the current working directory after execution as it was before execution:
(cd /tmp/ && sha256sum --check some-release.zip.sha256)
On successful checksum verification, it will print
some-release.zip: OK
to stdout and exit with exit status 0.
For OSXUsers (MacUsers) it´s possible to use:
echo "CHECKSUM" | shasum -a 256 PATHTOFILE | shasum -c
It´s no script, but it works for OSX ;-). Maybe someone can use it.
