How hard would it be to create a 1k file from the SHA1?

[hermanm]

I was thinking instead of actually transferring file pieces, if we transmitted the SHA1 in small enough pieces, couldn't we just create the actual 0s and 1s on our end? Even with brute force, how long would 1k take? Create a rainbow lookup table and couldn't we potentially generate all the data we need for a file from our computer?

[Ultima]

1 KiB = 1024*8 bits = 8192 bits

2^8192 = GINORMOUS NUMBER

Somehow, I find it highly doubtful you or anybody else will be generating a SHA1 lookup table for every possible 1 KiB piece.

Additionally, hashes aren't collision-free, meaning that a single SHA1 hash may potentially belong to multiple pieces. Even if you had a lookup table, you might still end up picking out a piece that passes the hash check, yet, is still the incorrect piece.

[hermanm]

> GINORMOUS NUMBER

How much data exactly is 2^1892?

> a single SHA1 hash may potentially belong to multiple pieces

I was under the impression that it was mathematically possible, but statistically insignificant? Would adding CRC check on top of SHA1 take care of this issue?

[Ultima]

For reference: 2^128 = 340,282,366,920,938,463,463,374,607,431,768,211,456

So you'd have to keep a table of that many 128-bit (16-byte) chunks of data along with each of their 20-byte SHA1 hashes, along with (if you choose) their 4-byte CRC32 checksum, basically 40*(2^128) bytes just for all the combinations of 128-bit chunks of data. That's not even counting storage overhead.

Imagine 8192-bit (1 KiB) chunks. It'd only grow exponentially, and exponential growth is VERY quick. That's (1024+20+4)*(2^8192) bytes without storage overhead. In short, it's completely impractical.

Trying to generate collisions is even more impractical, and you'd still run into the fact that SHA1 hashes aren't guaranteed to be unique for each piece of data.

[hermanm]

Thanks for indulging me and taking the time to explain. I'm just getting a grasp of the concept of petabyte. I can't really conceptualize a number that big.