MD5

Dec. 16th, 2009 08:01 am
slowfox: Slowfox' default icon (Default)
[personal profile] slowfox
MD5 is a hashing algorithm that attempts to verify that a given file's integrity hasn't been compromised (during a download process, or in the process of being copied from A to B etc).

Typically, you'll have a file reference given on a web-site and, somewhere else (this is important, more later), there'll be a long string of seemingly random characters called the MD5 Sum, or MD5 Hash or whatever:

c59b048c992804d165aed10170f003dc

What happens is that the algorithm works through the source file, and maps the first element to something new, and then feeds that result into part of the computation for the second element, and then the result of that into the third element etc. When it reaches the end of the file, the whole result gets fed back into the algorithm again. And again, and again for a specified number of iterations (loops). The end result of these calculations boils down to the specific MD5 Sum for the file.

I use WinMD5Sum on the work PC:
md5screenshot

The idea behind MD5 hashing is that even tiny changes in the original file result in significant differences in the end MD5 hash.

For example, I created a text file called hello.txt, which contained the phrase Hello, world..

The MD5 sum of this worked out to be 45d2c2d506211d17f99a3eb8de863f36

By changing the last character to a comma - Hello, world, - the MD5 sum changed to c59b048c992804d165aed10170f003dc, immediately telling me that the file's changed from the original.

An MD5 sum is always 32 characters long, yet can be generated for files of any size. Pretty obviously, then, there will be various different kinds of files that result in the same hash - these are known (with the industry's predictable fondness for dramatic vernacular) as 'collisions'. Nonetheless, the risk of collision is pretty small (if, for example, by repeatedly going through the MD5 hashing algorithm, all files eventually boiled down to a single value, it'd obviously be useless) - that said, boffins have managed, now, to successfully construct amended files that generate the same MD5 hash, but this requires a fair bit of work and an accommodating starting point.

Anyway, the idea is that you see a file - exciting_prog.exe - on a website and download it. By using an MD5 checksum, you can verify (beyond reasonable doubt) that the file you've downloaded is the file that the website intended you to download.

(which, by the way, is a loooooooooooooooooong way from saying it's safe).

The major caveat should be obvious: the situation where the file's MD5 sum is listed in the same directory as the file itself. Consider: if nefarious malfeasants have managed to hack the server and place a malicious file in the benign one's place, then since they've clearly got access to the server, it'd be trivial for them to also replace the posted MD5 sum with the sum to match their own malicious file.

Date: 2009-12-16 08:33 am (UTC)
linaelyn: (exercise-pushup by dropsofsunshine on LJ)
From: [personal profile] linaelyn
Excellent, informative post. Thank you!

Date: 2009-12-16 09:24 am (UTC)
alicit: Cheshire cat pointing to your right (Default)
From: [personal profile] alicit
OMG, I haven't understood a thing after [personal profile] slowfox. *Feels dumb*

I think I'll go make myself another coffee.

Date: 2009-12-16 11:53 am (UTC)
schnurble: (Default)
From: [personal profile] schnurble
I've learned about MD5 hashes a while ago when I wrote a little prog that posted my tweets to LJ - the LJ API also accepts your LJ password as MD5 hash, so you don't have to send the password in clear text through the net. And in that case, it's good that several different strings boil down to the same MD5 hash, or you could easily extract the original password from the hash (of course, this is still not 100% safe, but safer than clear text)

Date: 2009-12-16 11:00 pm (UTC)
From: (Anonymous)
Interesting, I'm now curious to see where this is heading.

oops

Date: 2009-12-16 11:00 pm (UTC)
carolanne5: (Default)
From: [personal profile] carolanne5
Oops ... could of sworn I was still logged in.

Profile

slowfox: Slowfox' default icon (Default)
slowfox

Style Credit

Expand Cut Tags

No cut tags
Page generated Jul. 25th, 2025 01:15 pm
Powered by Dreamwidth Studios