Random speed drops

[Greg Hazel]

>Is it always the same disk, or even the same file?

All my torrents are on the same hard disk right now. But, it also happens if I'm running torrents across multiple drives. It also happens if I'm only maxing out with relatively small torrents. The only common thing I can think of is that most if not all torrents I'm usually running have a piece size of 4mb or higher.

Every other application can read/write the disk, it is only utorrent that hangs up reads. Infact, utorrent can keep writing as well (if the downloads aren't too fast), only reads die off.

I've had a torrent capped at 1mbyte/s download, and it kept going even as the disk reads stalled. Though, after I set it back to unlimited speed (5-6mbyte usually, depending on ISP load, could go up to 11mbyte), disk writes stalled as well, and so did this downloading torrent. This torrent was only around 3gb, not even that big.

Are you sure that writes don't keep going because it's filling up the cache? Higher speeds would fill up the cache faster... If Windows is blocking on a disk read, there's no much uTorrent can do except wait. It's surprising to me that other application could read and write from the disk as well.. are you sure they're not reading from the filesystem cache? Testing this is quite tricky. You might try:

- Make a largish file called 'foo'

- Reboot

- Do -not- read 'foo'

- Get uTorrent to stall on a disk read

- Try to open 'foo'

Let me know if that works or not. Also, try to catch the disk read which uTorrent is waiting on in "Process Monitor" http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

That will show the size of the read and such.

Just to be clear, 1.6.1 and versions without the "Disable Windows cache" options should perform the same as having those boxes unchecked. This should cause the Windows cache to grow indefinitely, which is why I added the option to bypass it. Your experience matches that, but also Windows stalls from time-to-time if you check those boxes?

[zyrobs]

Right, true. Disk writing keeps going, but it will end up stalling too as no ACTUAL disk writes are done (as viewed by Process Monitor). It probably writes to cache, which eventually fills, and the download will stall due to no disk activity.

I was checking utorrent in process monitor, and it seems utorrent does not touch any of the torrented files at all while these stalls are happening. It only synchronises with some Directx files in system32 (maybe because of aero?), and it periodically updates resume.dat.

Now, here's something curious. All the torrents I'm seeding are on F:\, and all the torrents I'm downloading are on E:\. Both stall sooner or later, as I've described above. I can still access the drives fine. More importantly, and this is the interesting part, if I access data on F:\, the stalls stop and utorrent will resume reading files. I've tried opening a 300mb rar, it fixed the stalling. I've tried opening a few byte txt, it fixed the stalling. In fact, if I just navigate into a a folder in F:\, that too will fix the stalling.

On the other hand, doing anything on E:\ will not fix the stalling, so the stall probably happens somewhere on the disk reading part.

Entering a not-filesystem-cached folder/file has a short lag, barely enough to notice it. I didn't notice it before I was looking, since I'm so used to windows lag. However, if I access *anything* on F:\ when utorrent is operating normally, anything comes up instantly. When I try accessing anything when utorrent is stalled, there is a minor lag. Like, half a second. Even when I'm doing only as much as opening a folder I've yet to navigate to. If I navigate to a folder I was in before (back/forward buttons), it still loads fast, obviously as it is cached.

If it matters, all my drives are SATAII with AHCI enabled (ga-p35-ds3 mobo), and I have both "improve drive performance" boxes ticked in at the device manager, for all drives.

Edit: if you make a dev build with debug logging, I'd be glad to test it, as long as it functions with private trackers.

[Switeck]

So long as private trackers allow uTorrent v2.0, they probably won't be the wiser.

[Greg Hazel]

Right, true. Disk writing keeps going, but it will end up stalling too as no ACTUAL disk writes are done (as viewed by Process Monitor). It probably writes to cache, which eventually fills, and the download will stall due to no disk activity.

I was checking utorrent in process monitor, and it seems utorrent does not touch any of the torrented files at all while these stalls are happening. It only synchronises with some Directx files in system32 (maybe because of aero?), and it periodically updates resume.dat.

When the stall ends, do you see an event in Process Monitor which has a high Duration? (you can turn this column on, or look in an event's Properties.)

[zyrobs]

There is an ongoing read event when the stalls are happening, when this ongoing event finishes (with a duration of 60+ seconds), the stall ends as well.

tcp/udp connections are still going on while the stalls happen, so do Registry reads. But file reads will not continue until that last file read finishes. Inbetween the two file reads, no other Operation has a noteworthy duration, even the longest ones (syncing with the directx files) take 0,0000176 seconds.

edit: note that I have process monitor filtered to display utorrent.exe only, I'll check without filters too.

edit2: no high duration operations by other apps either.

[DiamondTear]

I had a similar problem with large (over 4GB) torrents, so I thought I'd share my solution for anyone else stumbling into this thread. This probably won't help zyrobs.

utorrent: 1.8.5

connection: 100/100 ethernet

problem: weird speed drops when uploading at 2MiB/s+ (like this: )

solution: Override disk cache in utorrent options. I set mine to 512MB, but utorrent doesn't seem to be using it all, so leave "reduce memory usage" on. It appears that utorrent wasn't increasing its cache usage as it should with override disabled and was stuck at 32MB, which wasn't enough.

[Greg Hazel]

There is an ongoing read event when the stalls are happening, when this ongoing event finishes (with a duration of 60+ seconds), the stall ends as well.

Can you paste the details of some of the stalls here? Is there any common element between them?

[zyrobs]

No common elements that I could figure out.

Date & Time: 2010.01.05. 0:13:39

Event Class: File System

Operation: ReadFile

Result: SUCCESS

Path: F:\xxx.zip

TID: 348

Duration: 61.4457257

Offset: 583 289 856

Length: 131 584

I/O Flags: Non-cached

Priority: Normal

Date & Time: 2010.01.05. 0:18:36

Event Class: File System

Operation: ReadFile

Result: SUCCESS

Path: F:\yyy.zip

TID: 348

Duration: 61.3944236

Offset: 10 451 456

Length: 131 584

I/O Flags: Non-cached

Priority: Normal

Date & Time: 2010.01.05. 0:20:32

Event Class: File System

Operation: ReadFile

Result: SUCCESS

Path: F:\zzz.zip

TID: 348

Duration: 62.3238769

Offset: 120 681 984

Length: 131 584

I/O Flags: Non-cached

Priority: Normal

[Greg Hazel]

Ok. It really looks like Windows is the cause, but maybe it's the a sector on the harddisk? I believe drives are configured to retry on CRC failure for quite awhile. Could you try reading those bytes by hand somehow? Try copying the file, that should do it. See if you get a similar delay. Although once it's in the cache it won't have to retry a bunch anymore.

Also, see if there are any S.M.A.R.T. warnings for that drive - your harddrive manufacturer probably provides a tool to check, or you might try SpeedFan ( http://www.almico.com/speedfan440.exe ) - be sure to run (not just install) it with Administrator privileges. Their online analysis seems to be the best way to read the results.

[zyrobs]

There are no SMART warnings, the drive is 2 weeks old, and was stress tested with HD Tune for a full day when I got it. I take HD errors very seriously.

Also, this happens across all drives, and if I manually browse into any non-cached folder on a drive, the stalls are cut short. So I doubt that it's a disk error.

[Greg Hazel]

Well, there's some probably serious reason that Windows is taking >60 seconds to read 128kB. uTorrent can't be delaying the completion of the read - failing to read the result for a long period of time would not show up in Process Monitor that way. That measures time in and out of the kernel.

Possible reasons include:

- A serious Windows bug, which Microsoft should know about right away

- A drive controller error

- A harddrive failure (which doesn't seem to match, as you described)

Since it doesn't affect networking or anything else it seems safe to rule out CPU and memory as problems.

Perhaps you could set up a large RAM drive, and see if seeding from that works well? That would isolate Windows and the drivers and uTorrent from actual hardware: http://www.mydigitallife.info/2007/05/27/free-ramdisk-for-windows-vista-xp-2000-and-2003-server/

[ayaha]

Edited by Admin: Sorry, this is not the same issue.

[zyrobs]

Using a RAM drive will probably end up difficult, since I don't have enough RAM - and finding a torrent to max out my line could prove difficult, especially with a small torrent. But I'll look around.

However, first I will try running utorrent from a different account, with the utorrent datfiles copied over, and see if the errors still happen on a new account with the default settings. It's possible that there is something misconfigured on my user account, since I've been importing it for a long time now..

[Greg Hazel]

I really hope there's nothing user-related which could cause such delays... But if you search turns up anything please post about it

[DreadWingKnight]

I just deleted a post recommending old versions.

DO NOT RECOMMEND OLD VERSIONS!

[Greg Hazel]

Old versions do not work better anyway. We already confirmed that unchecking the Windows disk cache options makes the newest versions act like versions before those options existed.

[Sawyer22]

Well I have the same problem here. 4x 20/20Mbit. Routing table edited for the 4 connections.

route ADD 73.0.0.0 MASK 255.0.0.0 $ime1

route ADD 73.255.255.255 MASK 255.255.255.255 $ime1

route ADD 77.0.0.0 MASK 255.0.0.0 $ime1

route ADD 77.255.255.255 MASK 255.255.255.255 $ime1

route ADD 81.0.0.0 MASK 255.0.0.0 $ime1

route ADD 81.255.255.255 MASK 255.255.255.255 $ime1

route ADD 85.0.0.0 MASK 255.0.0.0 $ime1

route ADD 85.255.255.255 MASK 255.255.255.255 $ime1

Works fine with Vuze, I'm getting constant ul speeds with absolute no speed drops at all, but with utorrent I have huge speed drops all the time.

Will try some tweaks suggested here but I have mostly tried all there is.

http://www.shrani.si/f/2i/VJ/40cv6LGV/capture.jpg

EDIT :

Well. Done everything suggested here. Even upped the memory to 1000Mb. It sitll happens. Maybe less ofthen but still.

http://www.shrani.si/f/3H/12p/35O2jcJP/capture.jpg

http://www.shrani.si/f/3M/u5/JKJMJTn/capture2.jpg

[Greg Hazel]

Sawyer22: Do you have the same symptoms? If you run Process Monitor, can you find some IO operations taking ~60 seconds?

[Sawyer22]

I was testing Vuze right now and I was able to stretch 3 torrent up to 8.4MB/s and I didn't notice any speed drops at all. On the other hand with utorrent which finds peers much faster then Vuze. I have hard time reaching that speeds. It hit 9.1MByte/s at one point. Then I checked task managed and one core of my quad CPU which is a q9300 clocked to 3GHZ with 8GB memory was at 25%. Fully loaded. Yet azerus hardly reached 10% at 8.4MBytes per second upload speed.

I will have to check again for that IO. Running Windows 7 64Bit and not Vista like the others here also.

I could setup a TeamViwer remote control later today when I get home if you want. However I'm in Europe so It will have to be somewhere between 12 - 24 CET time.

[Greg Hazel]

Did you try unchecking the "Disable Windows caching of .." boxes?

[Sawyer22]

Yes I did. However utorrent is now also restared what I didnt do before. Running now dunno since 30+ mins and the speed didnt drop once under 5.5MByte/s. Mostly its around 7.0+MB/s. Looks like it doesnt stop anymore and cpu is normal also. I did enable the net low cpu to true if this has anything to do with it however.

EDIT:

http://www.shrani.si/f/a/1q/1pWDeZ4w/capture.jpg

Thats now more then 1 hour without a single speed drop. Will test again with net low cpu on off to see if this made the difference as I read this can help if you miss configured something badly.

[Greg Hazel]

Great! It sounds like you're experiencing exactly the same issue. If you run it with those boxes checked and run Process Monitor, can you capture a log during when the speed drop occurs?

[Sawyer22]

That is now about 30 mins give or take a minute with net low cpu off.

http://www.shrani.si/f/2g/PC/1DXc5aIL/capture.jpg

Again no total stops like I was getting earlier. Are this zigzags still considered to be problematic slowdowns or what should I check? My settings are like this since the last 2 test. Checked already.

http://www.shrani.si/f/1K/pR/n31IqZ9/capture2.jpg

I'm gonna enable netlimiter for another test. This is the only other variable other then not restarting utorrent which I have changed since I had problems and sicne its working way better.

[Greg Hazel]

The zig-zags are probably harmless. Can you increase the resolution to 1 second or 5 seconds and let it run for a few minutes and post a graph of that?

[Sawyer22]

Yeah I believe to, that the the zigzag are harmless.

Process monitor is that? http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

or do you mean resource monitor in Windows 7?

Resolution 1 sec with resource monitor.

http://www.shrani.si/f/3k/Lb/15GSpZ18/capture.jpg