Jump to content

Partially deleted downloads?


ajones81

Recommended Posts

I don't know if this is normal uTorrent behaviour so just thought I'd ask. I had a multi-file torrent and selected and downloaded only a few files to a single directory. Then I deleted the torrent but left the data behind (obviously).

Later, I realised I had left one file unselected and so I re-downloaded the *same* torrent from the *same* site, selected only the missing file I needed and pointed it to the same destination directory with the earlier files in it.

Immediately, I saw that uTorrent had renamed *some* of the previously downloaded files with an .!ut extension and was saying that those files hadn't been downloaded fully. All were showing up as being skipped and around 90%+ complete, which is strange considering they had been at 100% before I loaded the torrent again. I didn't know what to do, so I simply started them up (priority = normal) and completed them once again.

So what I want to know is this - a) since those files had already been downloaded and the 2nd time around I didn't select them from the list, how come instead of lying untouched they got partially deleted? B) why didn't this happen to *all* the files I downloaded the 1st time and only to *some* of them?

It's especially funny, considering that the files had not been touched after being downloaded and so it's not as if they could have been modified and thus re-downloading of the affected parts was required or something. I had even copied one of those files to a different directory (after the 1st download session) and once it got re-downloaded the 2nd time around, I did a file compare (using crc32, md2, md4, md5, sha1, sha256, sha384 and sha512 digests computed for each) and the two copies came out the same. So, what gives?

Link to comment
Share on other sites

http://forum.utorrent.com/viewtopic.php?pid=261604#p261604

If you delete the torrent job from the list, µTorrent will delete the partfile it uses to store the unneeded parts of pieces. Once those get deleted, µTorrent can't tell if a piece you have part of is corrupt or not, so it has to chuck the entire piece. That's just how things are.

I might have a better explanation lying around the forums if you search for posts containing bittorrent piece* made by me.

Link to comment
Share on other sites

Ok, I think I got your explanation partially, though I still don't understand why some files were spared when even their partfile was gone same as for the rest?

Also, what you say should hold good *only* if I had explicitly selected to re-download those files, right? Since I never selected those files the 2nd time round (meaning I'm happy with the versions I have), why should uTorrent take it upon itself to go check them, suspect they are corrupt and download the (chucked and now missing) pieces again? uTorrent should optimally concern itself only with files I *want* to download in the current session. Thus, even if I deliberately corrupt a file, it should be *my* decision if I want to overwrite it with a good copy or leave it as it is, right?

Finally, it just occurred to me, what if I had downloaded another torrent containing a file with the same name as one I had already downloaded but with different contents? Would uTorrent have discarded my original file fully and downloaded the new version? If so, this would be a really bad 'feature' in my opinion. Even if this doesn't hold true, I'd still like the original issue of unselected files being tampered with resolved please.

Link to comment
Share on other sites

Some files are spared because they aren't edge cases where the full piece isn't there because of the deleted partfile.

1 2 3 4 5 6 7

[AAABBBBB][BBCCCDDD][DDDDDDDD][DEEEEEEE][FFFFGGGG][GGGGGGGG][HHH]

Assuming this represented the torrent contents in piece form, each letter being the data from a single file...

  • Even if the user chooses to skip files A, D, and H, µTorrent will still download pieces 1, 2, 4, 5, and 6 and store the unwanted data from files A and D in the partfile.
  • If the user wants to carry on the download after removing the job from the list (which would remove the partfile), then then pieces 1, 2, and 4 would be chucked after the hash check on the existing data (take note: this is where the pieces are chucked) and redownloaded becase because they share pieces with the skipped files, and are (therefore) incomplete after the partfile is removed.
  • Pieces 5 and 6 would be left alone, and files F and G wouldn't be touched because they're not the edge cases, and don't share data with the pieces that contain skipped files A, D and H.
  • If you choose to download only file D after re-adding, then pieces 2, 3, and 4 will be redownloaded (as they were incomplete), and they will all have to be redownloaded.
  • Choosing to download file H after re-adding won't affect any other pieces, as it doesn't share pieces with any other file.

That's just the price that needs to be paid for selective file downloading to work. Do note that it's generally rare for files to not share pieces with other files, so the above example is just a bit contrived (but it does get the main points across).

µTorrent can't keep incomplete pieces because it uses SHA1 on the entire piece; that's the whole point behind splitting the conglomerate data into pieces in the first place. If the piece is missing data, it will most definitely fail the SHA1 check because it's not identical to the reference piece hash included in the .torrent file's metadata.

Link to comment
Share on other sites

Ultima, thanks for the detailed explanation. I got it, though somehow I still find it funny that the one missing file I chose to download shared its pieces with so many other perfectly downloaded files. I mean, just how many files can one file overlap with? Shouldn't it be at the most two (one on each side)? Sorry if I sound noob-ish, but this concept is rather new (but quite interesting) for me.

Also, since you mentioned SHA1 being used on a piece-wise basis rather than a file-wise basis, then I guess the problem I theorised will be true?

Finally, it just occurred to me, what if I had downloaded another torrent containing a file with the same name as one I had already downloaded but with different contents? Would uTorrent have discarded my original file fully and downloaded the new version? If so, this would be a really bad 'feature' in my opinion.

If this is indeed true, then that makes it quite dangerous to point a torrent at the same download directory used earlier, 'cos uTorrent will overwrite all files with the same name, whether you like it or not. I do hope I'm wrong regarding this...

Link to comment
Share on other sites

Files with identical names will be hash checked. If they are different files they will be overwritten.

This is perfectly normal since you should not be saving identically named files to the same directory, just like you wouldn't put two files with the same name in a folder when moving something in explorer (though explorer asks you if you want to replace the file first). Imagine what should happen then if an incomplete file (from another torrent with the same content if you want) is resumed. A partially completed file would obviously have a different hash, so should it then start from scratch again?

Link to comment
Share on other sites

I mean, just how many files can one file overlap with? Shouldn't it be at the most two (one on each side)?

There isn't any limit, really. For all BitTorrent cares, you can have a million 0 byte files being described by the .torrent file (all being contained within a single piece), and it wouldn't make a difference. The way the .torrent file is generated is by clumping all the files' contents together, and then split the data into pieces and hash them from there. If you have... five 200KiB files (totalling 1000KiB) clumped next to each other in a .torrent file with a 1MiB piece size (1024KB), the theoretically, all five files can actually fit within a single piece without any problems.

Link to comment
Share on other sites

Ok, got the general drift of piece-wise allocation. Thanks Ultima and 5618.

Moral of the story? Always set the destination directory for a torrent (esp. one that has been downloaded earlier) to an empty one rather than reusing the old destination directory, unless you're absolutely sure that none of the file names overlap or you actually want to overwrite the existing files with different versions.

Even if we set the destination directory to an existing one with files in it, a warning from uTorrent after pressing Ok in the Add New Torrent dialog would be nice to have in case of identical file names, thus giving a chance for the user to either change the destination directory or go ahead and overwrite if he so pleases. Can this be implemented?

Link to comment
Share on other sites

Even if we set the destination directory to an existing one with files in it, a warning from uTorrent after pressing Ok in the Add New Torrent dialog would be nice to have in case of identical file names, thus giving a chance for the user to either change the destination directory or go ahead and overwrite if he so pleases. Can this be implemented?

No, I do not think so, since it'd have to ask each time you wanted to resume a torrent. People would get confused, maybe think they would need to start from 0% and generally clicking "No". That's what I think anyway.

Link to comment
Share on other sites

No, I do not think so, since it'd have to ask each time you wanted to resume a torrent.

Since when does uTorrent ask for the destination directory to be set every time you want to resume a paused/stopped torrent?

The warning would be useful in the specific case that you loaded a new torrent, or an old torrent that you had previously downloaded from and deleted. In such cases you'd see the Add New Torrent dialog and it could check the destination directory you set and warn you if files with the same names already existed. So I don't see where the problem is with this, 5618.

@Ultima/Firon: Should I start a separate request thread for this, 'cos I think it might be quite useful indeed.

Link to comment
Share on other sites

  • 1 month later...

One thing I don't understand is why utorrent doesn't always check the existing utorrentpartfile if there is one, and make use of the data if it's correct. I wanted to reset the ratio count on one multi-file torrent I had (to make the ratio rules count for the whole 1.4GB instead for one 2MB piece I had downloaded), so I did this - stopped the torrent; moved the downloaded folder complete with utorrentpartfile to a new location; removed the torrent from utorrent; opened the torrent again and pointed it to the new location and started it. This had the result that utorrent started to download the missing fractions of the pieces all over again, and obviously didn't give a damn about that all that data already existed in the existing utorrentpartfile. I was back at square one.

Now, I find this really strange, because if you do this with two different installations of utorrent, like if you download something on your usb-stick (at school maybe...) and then want to seed it from home, _then_ utorrent checks the content of the utorrentpartfile and all is good, 100% downloaded. I don't see any reason for this different behaviour? Is there one or is it a bug?

Link to comment
Share on other sites

removed the torrent from utorrent

Once you remove the torrent job from µTorrent, it deletes the partfile, and logically so. Once you remove the torrent job, there should be no reason to keep the partfile any longer. If you need to keep it, then back it up, and restore it after you've removed the torrent job from µTorrent and before you've readded it.

Link to comment
Share on other sites

But that's what I did? I tried it both with 1.7.2 and 1.6.1: resuming a complete download including partfile from another installation of utorrent - no problems. Trying the same with a backup - utorrent doesn't make use of the existing partfile, and starts to download the data that's actually already there, all over again.

edit: Ah, now I understand what's been happening - you always have to add the torrent in stopped mode and do a Force Re-Check to resume a download this way. utorrent doesn't check for a partfile when it's doing the hash check "by its own". I could have sworn it used to do that, but obviously not.

I guess I should make a post about this in the bug forum, if no one can point out any disadvantages with utorrent checking for a good partfile in both cases.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...