Does someone have good recommendation for (Windows) checksum utilities similar to dvdsig? Ie. something to recursively create md5 (or sha1, or sha512 or whatever) checksums for all the files in a folder and all its subfolders, and easily verify their integrity at any time, e.g. if you have copied them all to another media?

dvdsig is otherwise just what I need, but it seems to report quite many "invalid data" errors already when scanning files, not really telling why it considers the data invalid. To me it would appear it has problems at least with extra characters in filenames, very long paths/filenames, and possibly tiny files? Also, I have no idea why dvdsig uses so TINY font, I sometimes have problems seeing what exactly it is writing on the screen, especially when it is writing grey text on blue background. Also, I haven't found a way to e.g. copy&paste the report anywhere, to check in more detail which files have failed the verification.

Features I'm looking for:

- hopefully portable (like dvdsig, e.g. just put the exe in the root directory and run it there)
- simple to use to create checksums for all files in the current folder and subfolders, and to verify them later
- hopefully the report can be saved
- doesn't have similar problems with obscure filenames etc. as dvdsig
- preferably freeware, but I am willing to pay something if it is good

Some I've tried so far:

1. <span class="bold">md5summer</span> (freeware): it felt promising, but I have to select all the folders in the root directory, ie. I can't tell it to just create checksums of all folders/files in the current directory. Also it seemed to me it was still creating md5 checksums, even though I told it to create sha1 checksums? Also its "About" page mentions Windows versions up to XP, so is this an abandoned utility? I like it though how it gives you detailed feedback while processing the files.

2. <span class="bold">Corz Checksum</span>: on paper this sounds promising. While it includes a portable version, it seems you really should install it to get the most out of it. I wish it was more visible when creating the checksums, now it just shows some tiny text of the currently processed file in the upper left corner of the screen. This is nagware, asking for a donation. Fine by me, if it ends up being what I am looking for.

3. <span class="bold">MD5 and SHA Checksum Utility</span>, it appears at least the freeware version doesn't handle subfolders. I understood there is a (commercial?) Pro version which does, but I don't dare to buy such without trying first.

Two extra questions: is md5sum reliable enough for integrity checking, or is there a reason to prefer sha1, sha256, blake2 etc. instead?

Also, are there similar utilities for Linux, or is there the idea to create a bash script of my own which does it using md5sum or sha1sum?

Personally, I use fsum. It's a command line utility (graphical frontends are available, but I find them unnecessary). It can use a number of different algorithms, but defaults to MD5. Basic use is

fsum -r *.*>checksum.txt - to generate sums for all files (*.*), recursively through all sub-folders (-r), dump to file checksums.txt
fsum -c -jf checksum.txt - to check files against the file checksum.txt (-c), displaying only the failures (-jf)

The text file can be named however you choose. If you don't specify a file, results will be printed on screen.

Alternatively, when I want to directly compare two files or folders, I use WinDiff. It's a Microsoft program, not sure if it comes with Windows by default or with the Windows SDK specifically.

Maybe the Microsoft File Checksum Integrity Verifier would meet your need? It can recurse through all files/folders.

hth

Never used it that way, but perhaps RapidCRC?

This is what I used to use when I was still on Windows. So, yeah, another recommendation from me.

If you use advanced file systems which are resilent to bit rot, they handle the checksums automatically for you. Also, if you have your data in a redundant array of disks, when the file systems detects corruption it can automatically repair your files from the good redundant copies. So you don't even need to re-download them again.

The 3 big (or only?) resilent filesystems are ZFS, Btrfs and ReFS.

ZFS is old and proven. You can easily implement ZFS on a NAS using FreeNAS. I tried this, but found FreeNAS to be a bit buggy.

Btrfs is considered stable by Oracle who develop it, but it is still in development. Some features are experimental and these are not stable (like raid5 and raid6 arrays). But if you stick with a raid1 (mirror) array, then it should be ok. (Should always have another backup elsewhere anyway). Btrfs features and fixes are dependent on Linux kernel version. So best to pick a distro that is not using an ancient kernel, probably best to be on latest actually.

ReFS is available in Windows 8 and 8.1 I believe. Through the use of the "Storage Spaces" feature. I think this is supposed to be the easiest method, altough I have never tried it myself.

I'm personally running Btrfs in Arch Linux (because it keeps up-to-date on kernel version) on a HP ProLiant G7 N54L MicroServer. It's like a custom built NAS without the bugginess of FreeNAS. Also I have installed a remote access card into it so I can do everything remotely, like turning it on, rebooting, going into BIOS, installing OS from ISO remotely, etc. Absolute dream machine. But if you are new to Linux it may be a daunting task to setup.

If you happen to have Cygwin or Mingw installed, you probably already have a utility called md5sum, however I somehow don't see much of a recursive option in the win32 binary while I thought the GNU version offered a recursive option. I know there's a way to list the files and pass them off to other command lines using the back quote(s), but the exact method escapes me at this moment.

Yeah, I'm really looking forward to using these. I was going to use Btrfs when I recently installed the latest Linux Mint, but opted for the default ext4, as Btrfs was not offered without jumping through some hoops.

Anyway, for now I mostly work with Windows PCs, hence I am looking for something there, mainly for things like copying lots of files from one big USB hard drive to another, and wanting to make sure afterwards all files are still intact. Sometimes they haven't been.

Well the new "Storage Spaces" feature in Windows 8/8.1 can apparently use ReFS. The theory is it is supposed to allow anyone to easily create with a few clicks, reliable, bit rot resilent storage with redundancy. It is also supposed to be easy to grow it as needed by just plugging in extra devices, USB stick here, external HDD there. It all gets added to a dynamic pool (or pools?). I did briefly read that there was some catches with it though. I have never tried it myself, so I cannot really comment for sure.

Both fsum and rhash seemed interesting, but I marked rhash as the solution as it appears to be newer, and it appears to have a Linux version too.

I'm quite sure I've heard about fciv before...

Now using this, coupled with "Tail for Win32" (so that I can also see while the checksum file is filled, like tail -f). Trying it now with sha256, not sure if it matters in the end.

Even better that the same utility is available also for Linux, already installed it there with "sudo apt-get install rhash".

Total Commander also has plugin for SHA1/MD5 checksum generation/verification.