It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
Lexor: As I can see it is the same utility in the background and Frontend just adds GUI.
By looking at http://www.slavasoft.com/fsum/ I, again, do not really understand hot to construct the command I need.
Can you show me a working command to verify files located in FOLDER1 and FOLDER2 from my example?

BTW: When checking these utilities additional question comes to my mind: is md5 checksum "enough" to verify locally stored files or I should move to something like sha to reach better "reliability"? And then: which one? There are multiple versions of sha.
Without the GUI and with Fsum you can use a batch script. If fsum.exe is in the same folder as FOLDER1-3 in your example from above you can create the following batch file

check_md5.bat:
fsum -md5 -c -jf -dFOLDER1 file1.md5
fsum -md5 -c -jf -dFOLDER2 file2.md5
fsum -md5 -c -jf -dFOLDER3 file3.md5

(with the option -jf it prints only failed lines)

And for the scenario with checking different folders after some weeks you have to create multiple batch files to do so.

The md5 and sha1 checksums are outdated because they are insecure. The current standard is sha256. If you want to use sha256 then the format of sha256 files created by Total Commander is incompatible with Fsum.

Total Commander creates:
sha256sum *file.abc

Fsum wants this format:
sha256sum ?SHA256*file.abc

With sha256sum the batch file looks like this:

check_sha256.bat:
fsum -sha256 -c -jf -dFOLDER1 file1.sha256
fsum -sha256 -c -jf -dFOLDER2 file2.sha256
fsum -sha256 -c -jf -dFOLDER3 file3.sha256
Post edited November 18, 2021 by toma85
What I usually do with TC when verifying my archive is do a search (ALT+F7) for "*.md5", feed the results to a listbox, select all (CTRL+A), then run the Files > Verify Checksums command. TC then will verify all selected MD5 hash files one after the other and give a compiled result at the end.
Attachments:
crc.png (20 Kb)
avatar
nightcraw1er.488: This utility works well: http://md5deep.sourceforge.net/
avatar
Lexor: I do not quite understand the options from http://md5deep.sourceforge.net/md5deep.html info page.
Can you show me a working command to verify files located in FOLDER1 and FOLDER2 from my example?

avatar
Timboli: FSUM which is open source but has silly licensing is one of the few programs that will hash a folder.
avatar
Lexor: As I can see it is the same utility in the background and Frontend just adds GUI.
By looking at http://www.slavasoft.com/fsum/ I, again, do not really understand hot to construct the command I need.
Can you show me a working command to verify files located in FOLDER1 and FOLDER2 from my example?

BTW: When checking these utilities additional question comes to my mind: is md5 checksum "enough" to verify locally stored files or I should move to something like sha to reach better "reliability"? And then: which one? There are multiple versions of sha.
It’s a command line program, so you call it like an exe with some parameters. The synopsis section shows all possibly parameters and description explains them. You can run the exe from command prompt, or put it in a batch program. I would suggest generating a batch file from your metadata or from a directory and some code (I.e directory listing, if file is a certain type, then add call to batch file. Will see if I can post an example tomorrow.
avatar
toma85: Without the GUI and with Fsum you can use a batch script. If fsum.exe is in the same folder as FOLDER1-3 in your example from above you can create the following batch file

check_md5.bat:
fsum -md5 -c -jf -dFOLDER1 file1.md5
fsum -md5 -c -jf -dFOLDER2 file2.md5
fsum -md5 -c -jf -dFOLDER3 file3.md5

(with the option -jf it prints only failed lines)

And for the scenario with checking different folders after some weeks you have to create multiple batch files to do so.
OK, It seems that if I'm going to use command line version then I need to know the exact names of files with checksums? Am I right? I was hoping for something like this:

fsum <some options> FOLDER1

... and FSUM will "automatically" enter FOLDER1 (and FOLDER1's subfolders) and look for any md5 files.

avatar
toma85: The md5 and sha1 checksums are outdated because they are insecure. The current standard is sha256.
Yes, I know about better security level, but: does this "md5 is outdated" also mean "less reliable than sha256" for detecting HDD data errors?

avatar
toma85: If you want to use sha256 then the format of sha256 files created by Total Commander is incompatible with Fsum.

Total Commander creates:
sha256sum *file.abc

Fsum wants this format:
sha256sum ?SHA256*file.abc
Has TC implemented its own standard? Which one is the official one?

avatar
nightcraw1er.488: It’s a command line program, so you call it like an exe with some parameters. The synopsis section shows all possibly parameters and description explains them. You can run the exe from command prompt, or put it in a batch program. I would suggest generating a batch file from your metadata or from a directory and some code (I.e directory listing, if file is a certain type, then add call to batch file. Will see if I can post an example tomorrow.
Yes, I know how to use command line programs. ;) I was just hoping you can show me some direct command like:

md5deep <some options> FOLDER1

... and md5deep will "automatically" enter FOLDER1 (and FOLDER1's subfolders) and look for any md5 files.

avatar
soosgjr: What I usually do with TC when verifying my archive is do a search (ALT+F7) for "*.md5", feed the results to a listbox, select all (CTRL+A), then run the Files > Verify Checksums command. TC then will verify all selected MD5 hash files one after the other and give a compiled result at the end.
OMG, this is so clever idea! And it's probably the best answer for me so far as well. Thank you for this post. :D
avatar
Lexor: OK, It seems that if I'm going to use command line version then I need to know the exact names of files with checksums? Am I right? I was hoping for something like this:

fsum <some options> FOLDER1

... and FSUM will "automatically" enter FOLDER1 (and FOLDER1's subfolders) and look for any md5 files.
You have to write a program/script here that traverses the directory and subdirectories to search the md5-files and list them somewhere. Then it traverses the list. For each list entry it calls

fsum -md5 -c -jf <entry in list>

The Fsum Frontend which I posted earlier has such a function under "Verify checksum files". You can use also the TC checksum files here. You can add a folder and it searches the folder for checksum files.

avatar
Lexor: Yes, I know about better security level, but: does this "md5 is outdated" also mean "less reliable than sha256" for detecting HDD data errors?
Filesystems like ZFS are using sha256-sums for detecting duplicates on the HDD and file integrity. There are cases where you can generate the same md5-sums for two different pieces of data. The same can happen for sha1-sums. This is important for data transmission. The ZFS filesystem is used for servers with lots of data transmissions. If you are just looking to check for file corruption or differences md5-sums should be enough when the source is trusted.

avatar
Lexor: Has TC implemented its own standard? Which one is the official one?
I usually see checksums in the format that is also used by TC. The extra "?SHA256" in front of the "*" is something that Fsum needs to understand the file. The Fsum Frontend is different here. It actually doesn't use Fsum. If you want to use the frontend for Fsum, you have to use the fsum.exe and go to the Options menu (checkbox and Fsum location).
Post edited November 18, 2021 by toma85
avatar
toma85: You have to write a program/script here that traverses the directory and subdirectories to search the md5-files and list them somewhere. Then it traverses the list. For each list entry it calls

fsum -md5 -c -jf <entry in list>
Since I move directories around (and that's the main reason I can't create one checkum file for all subfolders and each subfolder has to have its own checksum file) then this method is not suitable for my needs - I would need to modify this script every time I move any subfolder(s) to some other location.

avatar
toma85: The Fsum Frontend which I posted earlier has such a function under "Verify checksum files". You can use also the TC checksum files here. You can add a folder and it searches the folder for checksum files.
Frontend looks like much nicer option but I think I prefer the idea connected to TC posted here by @soosgjr as I already have / use TC and this way doesn't need anything more to be installed.

avatar
toma85: Filesystems like ZFS are using sha256-sums for detecting duplicates on the HDD and file integrity. There are cases where you can generate the same md5-sums for two different pieces of data. The same can happen for sha1-sums. This is important for data transmission. The ZFS filesystem is used for servers with lots of data transmissions. If you are just looking to check for file corruption or differences md5-sums should be enough when the source is trusted.
I know that two different files can have the same checksum and your last sentence is the most important to me.

I was just wondering if md5 checksum is "less sensitive" than sha256 and how often it's possible to have a problem of "a few bytes" changed (caused by HDD reading errors) which is not reflected in a change of md5 file checksum.
avatar
toma85: Without the GUI and with Fsum you can use a batch script. If fsum.exe is in the same folder as FOLDER1-3 in your example from above you can create the following batch file

check_md5.bat:
fsum -md5 -c -jf -dFOLDER1 file1.md5
fsum -md5 -c -jf -dFOLDER2 file2.md5
fsum -md5 -c -jf -dFOLDER3 file3.md5

(with the option -jf it prints only failed lines)

And for the scenario with checking different folders after some weeks you have to create multiple batch files to do so.
avatar
Lexor: OK, It seems that if I'm going to use command line version then I need to know the exact names of files with checksums? Am I right? I was hoping for something like this:

fsum <some options> FOLDER1

... and FSUM will "automatically" enter FOLDER1 (and FOLDER1's subfolders) and look for any md5 files.

avatar
toma85: The md5 and sha1 checksums are outdated because they are insecure. The current standard is sha256.
avatar
Lexor: Yes, I know about better security level, but: does this "md5 is outdated" also mean "less reliable than sha256" for detecting HDD data errors?

avatar
toma85: If you want to use sha256 then the format of sha256 files created by Total Commander is incompatible with Fsum.

Total Commander creates:
sha256sum *file.abc

Fsum wants this format:
sha256sum ?SHA256*file.abc
avatar
Lexor: Has TC implemented its own standard? Which one is the official one?

avatar
nightcraw1er.488: It’s a command line program, so you call it like an exe with some parameters. The synopsis section shows all possibly parameters and description explains them. You can run the exe from command prompt, or put it in a batch program. I would suggest generating a batch file from your metadata or from a directory and some code (I.e directory listing, if file is a certain type, then add call to batch file. Will see if I can post an example tomorrow.
avatar
Lexor: Yes, I know how to use command line programs. ;) I was just hoping you can show me some direct command like:

md5deep <some options> FOLDER1

... and md5deep will "automatically" enter FOLDER1 (and FOLDER1's subfolders) and look for any md5 files.

avatar
soosgjr: What I usually do with TC when verifying my archive is do a search (ALT+F7) for "*.md5", feed the results to a listbox, select all (CTRL+A), then run the Files > Verify Checksums command. TC then will verify all selected MD5 hash files one after the other and give a compiled result at the end.
avatar
Lexor: OMG, this is so clever idea! And it's probably the best answer for me so far as well. Thank you for this post. :D
I may need to reread this tomorrow as it’s late now, it one of the options you can pass To the command prompt is to check all subdirectories, this will check every file and every subdirectory within a given root. TBH with these types of things, I like to link to my metadata, so for instance the data about games I have. I wouldn’t want to be copying and pasting, or doing manual processing.
Oh, there is other inbuilt windows functionality as well:
https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/certutil

And powershell as well:
https://infosecscout.com/md5-checksum-on-windows/

I use certutil a fair bit on windows, it’s useful but does not have the recursion of md5deep, but if you have the metadata to hand that doesn’t matter.
avatar
Lexor: I was just wondering if md5 checksum is "less sensitive" than sha256 and how often it's possible to have a problem of "a few bytes" changed (caused by HDD reading errors) which is not reflected in a change of md5 file checksum.
MD5's output is 128 bits (half of SHA256's), so in theory it can have 2^128 possible distinct results, or somewhere close to it. Due to something called the avalanche effect, even modifying a single bit in your data can produce a radically different result. Considering these two things the chances of a file producing the same checksum through a sheer random change, like bit rot, are essentially astronomically low.

Algorithms that produce more secure hashes, like the SHA2 family, are generally aim to prevent intentional attacks through hash collisions. There's nothing necessarily wrong with using them for data integrity verification purposes, you're just end up paying more processing power for something that's extremely unlikely to make a difference in the end. Though you have to consider the amount of data you have when deciding if that matters to you or not.
avatar
Lexor: I have subfolders and each subfolder has its own *.md5 file inside.

Is there a way to verify all these md5 checksums in one go (from the common parent directory)?
Do I really need to verify all these checksum files one by one?

If it's not so simple then maybe it can be done with a help of some batch file?

I also have Total Commander installed but I can't see such option - maybe there is some plugin for it?
I have Total Commander too,but if i understood what you want to do (years ago i used Total Commander)

search for Hashcheck Shell Extension 2.4.0.55 (should be on SourceForge)

It adds a tab to the system (no clue if there's a version other than one for Window) -> it's more faster to use

if you download a file from internet you copy the hash provided (it's often the case) and paste it in hashcheck to verify
it works with sub-directories when you want generate one hash.

it works with multiples types of hashes. (basic ones and most advanced ones)

You should generate a hash from the root folder for all subfolders (that's what i do). I guess it's possible to do with another tool or creating a script to do the job (but it's beyond by knowledge).Also It's beyond my knowledge if it's possible to do what you discribe 3 posts above mine : but if you move the root directory and not the subdirectories inside : i don't see the problem... that why i fail to understand what you really need , if you need to move a subfolder the hash in the root folder must be updated in one way or another (even with for your needs)
Last option i know : try portableapps.com : there are multiples tools

https://portableapps.com/apps/utilities/checksum_control_portable
https://portableapps.com/apps/utilities/rapid-crc-unicode-portable
https://portableapps.com/apps/utilities/winmd5sum_portable

at least you could try them whithout messing with the registry since those are portable versions

Hope that helps. and sorry if i don't help at all.

avatar
Lexor: I know that two different files can have the same checksum and your last sentence is the most important to me.

I was just wondering if md5 checksum is "less sensitive" than sha256 and how often it's possible to have a problem of "a few bytes" changed (caused by HDD reading errors) which is not reflected in a change of md5 file checksum.
At least for this point i could tell you what i do :

MD5 for local use -> never got a problem with only using MD5 for checking file integrity on local HDDs.
and more advanced ones to verify files downloaded : security reason.

I could be wrong with that use, but at least it always worked fine.
Post edited November 19, 2021 by DyNaer
avatar
Lexor: Example: let's take I have these six files.

C:\FOLDER1\file1.data
C:\FOLDER1\file1.md5
C:\FOLDER2\file2.data
C:\FOLDER2\file2.md5
C:\FOLDER3\file3.data
C:\FOLDER3\file3.md5

I want to be in main C:\ and, for example, I would like to be able to verify contents of FOLDER1 and FOLDER2. And week after I would like to verify contents of FOLDER2 and FOLDER3. All by running one command (without going in/out every folder) and without creating one checkum for all subfolders in main folder.
How about rhash, which is also a command-line application, found here: http://rhash.sourceforge.net

you can just use this command:
rhash.exe -cr --crc-accept=md5 -r C:\FOLDER1\

which will check every .md5 file it finds in FOLDER1 and all its subfolders. You only need to know the path to the folder, there's no need for the exact path/filename for each .md5 file contained within, or writing additional script files.
Post edited November 19, 2021 by MadalinStroe
avatar
Lexor: OK, It seems that if I'm going to use command line version then I need to know the exact names of files with checksums? Am I right? I was hoping for something like this:

fsum <some options> FOLDER1

... and FSUM will "automatically" enter FOLDER1 (and FOLDER1's subfolders) and look for any md5 files.
I've not tried it with .md5 files, but FSUM has a recurse option as well, so you don't need to specify filenames. In fact, it will give you a single MD5 result for the folder and its content.
avatar
Lexor: OK, It seems that if I'm going to use command line version then I need to know the exact names of files with checksums? Am I right? I was hoping for something like this:

fsum <some options> FOLDER1

... and FSUM will "automatically" enter FOLDER1 (and FOLDER1's subfolders) and look for any md5 files.
avatar
Timboli: I've not tried it with .md5 files, but FSUM has a recurse option as well, so you don't need to specify filenames. In fact, it will give you a single MD5 result for the folder and its content.
He does NOT want to calculate MD5! He wants to verify the checksum files he already has.
avatar
Timboli: I've not tried it with .md5 files, but FSUM has a recurse option as well, so you don't need to specify filenames. In fact, it will give you a single MD5 result for the folder and its content.
avatar
MadalinStroe: He does NOT want to calculate MD5! He wants to verify the checksum files he already has.
No he is wanting to compare the content of two or more folders using the MD5s.
I have suggested another way to do that.
I also don't discount the possibility of FSUM being able to use the existing .md5 files ... just never tried that myself.

P.S. See my first reply, which my second reply is an extension of.
Post edited November 19, 2021 by Timboli
avatar
DyNaer: if you download a file from internet you copy the hash provided (it's often the case) and paste it in hashcheck to verify it works with sub-directories when you want generate one hash.
I asked my question because I wanted to check integrity of locally stored files by using their locally stored md5 checksums, not during other system operations (like download of files).

avatar
DyNaer: You should generate a hash from the root folder for all subfolders (that's what i do).
As I posted earlier (and it seems you did not read my post) that way will not work in my case.
I often move my directories around (for example: C:\FOLDER1\DATA1\ can be moved to C:\FOLDER3\DATA1\) so generation of only one file with all checksums in root folder will be useless because it will become obsolete quite fast.

avatar
DyNaer: MD5 for local use -> never got a problem with only using MD5 for checking file integrity on local HDDs.
and more advanced ones to verify files downloaded : security reason.

I could be wrong with that use, but at least it always worked fine.
Thank you for this opinion. :)
Post edited November 19, 2021 by Lexor
avatar
MadalinStroe: How about rhash, which is also a command-line application, found here: http://rhash.sourceforge.net

you can just use this command:
rhash.exe -cr --crc-accept=md5 -r C:\FOLDER1\

which will check every .md5 file it finds in FOLDER1 and all its subfolders. You only need to know the path to the folder, there's no need for the exact path/filename for each .md5 file contained within, or writing additional script files.
If this tool works like you said (thank you for the example) then for me this is the first acceptable command line tool in this topic. :)

Question: What does "-cr" option in your example do? I can't find it in the manual - there is only "-c".

avatar
MadalinStroe: He does NOT want to calculate MD5! He wants to verify the checksum files he already has.
avatar
Timboli: No he is wanting to compare the content of two or more folders using the MD5s.
@MadalinStroe is right. I have locally stored files with their locally stored md5 checkums - checking integrity of these files after some backup is just one of additional options, not the main reason why I asked my question.
Post edited November 19, 2021 by Lexor