completed files still !UT, force recheck restarts them

[personalmountains]

Good day.

I have two particular torrents that are causing problems right now.

Unfortunately, their content and origin prevent me from listing them

here.

They both have rather high availability (25 and 20) and I get decent

speed. Once they finish downloading, I find that although they have

been moved to my "completed" directory, some of the files still have

the !UT extension. Looking at these files, I find that they are _not_

complete.

If I force a recheck, they end up at about 60% and 50% each and

continue to download to completion. At that point, rinse and repeat:

some files are still !UT and a force recheck brings them back to

about the same percentage. As far as I can tell, they never go higher

than 60% and 50%. What data is downloaded (and then considered invalid

by the recheck) is unclear to me.

There is nothing of interest in the logger tab.

This problem never happened to me before, nor does it affect any other

torrents I am downloading concurrently. It would seem that these two

torrents in particular are causing trouble. Based on their name and

content, they don't seem to have been created by the same user, but

I might be wrong since the files are related.

Windows 7, uTorrent 2.2.1 build 25130. The complete and incomplete

directories are on a NAS (although this was never a problem and I

think is irrelevant.)

Thanks.

[moogly]

And if you try with a torrent downloaded on your local HDD?

[personalmountains]

> And if you try with a torrent downloaded on your local HDD?

I just did, same behaviour.

However, I was wrong on two points:

1) I realize now that the percentage *is* going up each time the

torrent is rechecked and restarted. I'll keep doing it to see

if it will eventually finish. I've done it about 10 times for each

of the two problematic torrents so far. Both are around 1.35gb.

2) The log does give interesting messages. I got 10 failed hash

checks and 13 bans. 12 of those are have no specific message

(although they all have between 64 and 112k mentioned as "bad") and

one says "too many pex messages."

I also noticed that some of the files in the directory are duplicate:

one with !ut and the other without. Both are different and the !ut

mostly has zeros. Neither are complete.

Finally, I want to confirm that the data that is kept by the recheck

is good while the data that is discarded is invalid (whether corrupted

or zeros, I'm not sure.)

But still, I do not understand why uTorrent can mark a torrent as

being complete (by moving it to another directory and changing its

status in the list) while at the same time keeping files with the

!ut extension and failing rechecks.

Thanks.

[moogly]

Are the files audio, like MP3s?

[personalmountains]

> Are the files audio, like MP3s?

AVIs (all xvids) and JPGs.

This is the current file listing:

torrent 1 (now rechecked to 85%)
  1.avi      713mb    looks complete
  1.avi.!ut  713mb    incomplete
  2.avi.!ut  714mb    incomplete

  1.jpg      380k     complete
  2.jpg      380k     complete

torrent 2 (now rechecked to 49.8%)
  1.avi      715mb    incomplete
  1.avi.!ut  715mb    looks complete
  2.avi.!ut  715mb    looks complete

You'll notice that in t1, 1.avi looks complete, while 1.avi.!ut is

incomplete, but that the reverse happens in t2.

The only way I can verify whether a file works or not is by opening

it. vlc seeks correctly with no artifacts on the "looks complete"

files, although I can't be sure that the data is all correct. The

"incomplete" files either do not open or seek incorrectly.

When I first started this thread, T1 had been rechecked to ~60%

several times. The next recheck brought it to 85% and it has stayed

there for the last two. T2 is still at 49.8% and has been this way for

the last ~10 rechecks.

I'll try it a few more times and then will probably delete the files

and start from scratch. This way I'll see whether it's a problem with

files being corrupted locally or if I'm being fed bad data (which

passes the crc, however unlikely that is.) The local corruption is

also unlikely since it happens on two drives (although I only did a

"set download location" from one drive to another, which may have

carried the bad data with it.)

What I'm also not sure is on which files uTorrent is now working,

either the avi or avi.!ut. On T1, process monitor shows write access

only to 2.avi.!ut. This gives me the impression that 1.avi is

complete, but that 1.avi.!ut is a leftover. I did not notice the

content of the files tab.

On T2, only 1.avi.!ut is being written to. The files tab for T2 shows

0.2% for 1.avi and 99.9% for 2.avi. [update: 2.avi.!ut has been

written to once and is now complete, see below]

Update:

While I was writing this, both T1 and T2 completed.

I did two rechecks on T1 immediately, but uTorrent reports it as 100%

complete, so I just lost a test subject. The final file structure is

as follows:

torrent 1 (now rechecked to 85%)
  1.avi      713mb    looks complete
  1.avi.!ut  713mb    incomplete
  2.avi      714mb    looks complete

  1.jpg      380k     complete
  2.jpg      380k     complete

So 2.avi.!ut was correctly renamed 2.avi; the !ut file is gone.

I tried a diff on 1.avi and 1.avi.!ut, but I'm getting memory

exhaustion before I'm able to see how many bytes are different. The

files are definitely not the same.

I'm restarting T1 right now from scratch to see what happens.

As for T2, a recheck brought it back to 49.8% (where it started on the

last recheck), discarding everything that was just downloaded. The

file structure is still the same.

1.avi is back to 0.2% and 2.avi to 99.9%. Looking at process monitor,

1.avi.ut! has been written to continuously in 256k chunks, all showing

SUCCESS.

So although T1 went up from 60% to 85% to 100% in the last rechecks,

T2 is still stuck at 49.8%. I'm restarting it again. Nothing was

logged for this file since I restarted it.

I'm baffled.

Update 2:

> I'm restarting T1 right now from scratch to see what happens.

T1 finished without any issues after an hour. The files are complete

and the temporary !ut files are gone.

T2 failed again and is stil stuck at 49.8%.

[moogly]

Yes, try to redownload the files on your local HDD to test if there is a local file corruption.

In addition this issue can occur if some softwares like media indexers/libraries try to retag the files (especially with audio files like MP3), so the hash changes and uT doesn't reach 100% after a forced recheck.

VLC is nota good media player to test broken files, because it is able to play broken/partial files without displayiong an error. WMP is less clever and if the file is broken, it tell you it is not able to play the file.

[personalmountains]

> Yes, try to redownload the files on your local HDD to test if there

> is a local file corruption.

Sure. I was trying to avoid this in case you wanted me to test

something particular with these files, but I guess not.

> In addition this issue can occur if some softwares like media

> indexers/libraries try to retag the files (especially with audio

> files like MP3), so the hash changes and uT doesn't reach 100% after

> a forced recheck.

Yeah, but these video files are being downloaded among other video

files on the same drive and I'm quite sure I have nothing automated

that fiddles with these files.

Even if some program was modifying these files on the fly, they

wouldn't go back down to 0.2%. uTorrent would only re-download the

modified parts.

> VLC is nota good media player to test broken files, because it is

> able to play broken/partial files without displayiong an error.

But with visible artifacts and seeking issues for most incomplete

video files.

The reason I thought this was a problem with uTorrent was mostly

because of what's happening with T2: 1.avi is marked as 0.2% in the

files tab but 1.avi.!ut plays flawlessly in vlc, mpc and wmp. This

tells me that the data that was downloaded is fine, but that uTorrent

does not consider it to be.

Unfortunately, I can't find any kind of checksum for those files so

I can't be sure that 1.avi.!ut is in fact complete. But it's certainly

not at 0.2%. Something is happening so that uTorrent is rejecting

perfectly good data.

I'll keep the T2 files somewhere, re-download the whole thing and

(if it finishes correctly, like I think it will) I'll compare the

checksum of 1.avi.!ut and the new 1.avi.