Disk Overload

[zYx]

I've changed it to xx/1Mbit I'll see if it gets any better, however someone told me to run Norton Disk Doctor which i did and the program stack on one of mine external usb hdd. I don't know, but it may be related to this message 'disk overload' as once I got that message everything else freezes..

[spacemansg1]

i have same prob only mine says disk overloaded 100% im useing 1.7.7

im running xp pro sp2

intel® Core™ Due CPU

E8200 @ 66GHz

2.67GHz, 2.00 GB RAM

so can anyone tell me why its doing this and what i can do to stop it... oh i have no other programs running and all background programs running are only the ones that the system needs to run.

i have a 500g external hdd on usb 2. no other usb devices are connected

[aa]

EDIT №2: I just realized that this problem has not been solved completely, and I am confident that fragmentation is among the causes of the problem.

Every time you need to write to a file, the computer reads where on the disk the file fragments are, and then appends another fragment to the end of it. So reducing the number of writes reduces the number of times the computer has to read your file to see where to put the next fragment, and that in turn reduces the disk overload. For example, my cluster size is 512 bytes (I know, it's ridiculously low, but it was Vista's suggestion). So if there is only one fragment, it requires 1 read of 512 bytes to write 512 bytes. If there are 100000 fragments (a minimum for my files b/c bad settings initially), it has to perform 100000 reads of however many bytes each to write 512 bytes. So that is 200,000,000 reads to write 1 mB. So if you increase the number of bytes the system remembers to, say, 8000, that already reduces the number of reads for me to 12,500,000 reads/ mB. Also, defragmenting my hard drive took a few weeks because of its horrible fragmentation (and required special software; VISTA stopped after reducing it to from around 400,000 to around 250,000 per file, and redefragmentation did not help), but stopped the disk overload error. So that is quite a plausible possibility from my viewpoint. That is also why System Restore did not work.

EDIT: I meant to start a thread for this; please answer my question there.

This problem is similar to the one in http://forum.utorrent.com/viewtopic.php?id=5187, but not really.

Someone told me to turn on compact allocation, and I did, and I started downloading this huge group of files, but I cannot play them as they download because of the compact allocation issue. The worst thing is, it pre-allocated all those files, and now it is merely fragmenting my hard drive really bad and causing it to give a disk overload error. Is there any way I can return those pieces to where they belong now that I have the compact allocation back off? Moreover, if there is no way, is it going to restore them when it finishes downloading? Because I have been downloading since about the beginning of May, and I would not want to spend another four weeks to figure out that the files are irrepairably mangled.

[jewelisheaven]

... mangled? Bittorrent is byte-exact. When you are 100% complete you have a copy of the source. If you want to keep issues separate in different threads; do so. If you don't, keep each problem separate...

You have the files... they are there, if you consistently get disk overload problems, get faster disks or run fewer torrents simultaneously. The INITIAL disk overload is aggravated when people use 1) compact and 2) preallocate. Is there a reason you need to preallocate?

You need to learn math... With a 512 bytes cluster, 2048 operations are done for 1 MiB. Fragmentation is usually an issue with near-full disks... perhaps you should rethink your organization as well as settings... i.e. download to some temp drive, and then move-when-complete to your external/storage/whatever drive so you don't have to deal with constant seeking while downloading.

[aa]

Well, actually, no, in the other thread I was asking a question. In this thread I was trying to offer a possible solution to other people's problems. But the reason it fragments is because when it downloads a piece it writes it. Then a piece from another torrent finishes downloading, so it writes it too. By the time the pieces written get back to the first torrent, chances are, there is a piece written from every other torrent. So there would be 700 clusters between each of the consecutive clusters of the file.

What was even more idiotic is that even though I turned sparsing to false to keep it from fragmenting, I kept compact allocation on, so the fragmentation still happened because of the compact allocation, but I had the full space reserved, so I could not use the space for anything else, and all I could do was to sit there and wait for it to continue fragmenting the disk.

[DJRacer]

I am d/l at about 300-400k until recently. Disk Overload 100% stops the d/l and I cant find the diskio.write_queue_size option in the advanced properties. Anyone got any suggestions?

[Firon]

Please read the help file available on the download page for more details on UI options.

[coljung]

I am d/l at about 300-400k until recently. Disk Overload 100% stops the d/l and I cant find the diskio.write_queue_size option in the advanced properties. Anyone got any suggestions?

Same problem here.

Please read the help file available on the download page for more details on UI options.

I just read it and did not find any info regarding diskio.write_queue_size.

[Firon]

Because that option doesn't exist. Look up the disk cache in the manual.

[quickcycles]

I know this is an old thread now but people have used it recently so I hope it's ok.

I'm running 1.7.7 and constantly get disk overload when running torrents, It used to happen when I hit speeds of around 40MB/s but now I'm lucky if I can get to 1MB/s before it maxes out.

It also seems to eat into my 'mem usage' too, it's the highest I've ever seen anything.

I've looked into the guides on disk cache, I tried adjusting the settings but still no joy?

Please can someone help? guide me on what the best settings when speeds are upto 100MB/s

I've done a disc defrag and check the HDD for errors but nothing?

Thanks,

QC

[DreadWingKnight]

Have you checked to make sure your drives haven't lapsed into PIO mode on the properties of their controllers?

[quickcycles]

sorry,

it's a scsi hard drive the main drive and its not in PIO mode

[Firon]

Uninstall and reinstall the controllers anyway.

[quickcycles]

I don't think we can do that - we're on a dedicated server?

QC

[Firon]

If it's a dedicated server, you should have no issue removing it from the device manager.

[meon]

i'm having an issue of disk overload also.

recently, seeders make torrents with 16mb pieces in order to get small torrent file.

i always get disk overload with such torrents.

i'm on very high bandwidth (1Gbps port in data center), using raid0 hdd for downloading.

usually, with 4mb-piece torrents, i can download at 50-60mb/s with write cache going between 20-50mb and upload speed goes to around 60-70mb/s.

however, with 16mb-piece torrents, write cache started to increase, then disk overload occurred almost every time.

i use utorrent 1.6.1 with 250mb overrided cache

diskio.coalesce.write set to false

unlimit download is set

i am in some kind of competition on local lan connection. i need to upload as much as possible (as fast as possible). we start downloading a torrent at the same time from one seeder and whoever gets the most uploaded data wins.

i need to get rid of this disk over load so that utorrent can upload as much as possible.

any recommendation?

[DreadWingKnight]

You use an unsupported version.

Upgrade before we will even start troubleshooting.

[bit4bit]

hey guys ive read most of your posts and judging by the date of the post i hope you all figured this out before now :-)

ok i dont know why, how, or well any reasoning that i could prove but...

DISK OVERLOADED 100% right? yeh this sucks

i played around with the diskio value and no results, so please stop posting on that, really...

******************************************************************

******************************************************************

change the disk cache size in the advanced settings > disk cache > basic settings

make sure it already is and if not check the box "Override automatic cache size....."

set the value to anywhere between 64 and 256 (my value = 64MB) works great for me :-D

******************************************************************

previously my value was 1024MB and it was checked, if i set it back to that value disk overload(DO....im lazy) immediately comes back,as it will for anything above that value

however 8MB was too low, immediately after setting to this value DO came back and dl's stopped

16MB worked great but i never stayed with it for more than a minute, but really anything between 32MB - and 128MB is ideal in my opinion, some of you may know more in this area but this definitely stopped the DISK OVERLOAD problem

i played with this for a while and i have mine left on 64MB and i havent had a disk overload since,

speeds are back up above 1Mb/s if there are available sources :-)

the only reasoning i can give it is that if the file is too large something along the line whether it being the cpu or hard disk struggles to access the cache efficiently

im not saying this will work for everybody but i do think you will have great success with this

play around with the value for it and see which performs best for your connections and cpu

******************************************************************

i might suggest further testing of different values to simply see if it can offer better speeds of some sort

and for clarity i must give credit to someone previously in the post who gave me the idea,

username: aa

his post talking about the clusters that someone else seemingly was correcting him on something of sorts, but his logic makes sense, and i think disk overloaded might be a poor error description because it may simply be the cpu's inability to read a large file continuously for long amounts of time while constantly processing Mb's of data streaming in, just a thought, and yeh i could be totally in left field on this so correct me if you know any better

FYI im running e6300 currently at stock speed 1.86 ghz

4GB pc-6400

two main hard disks are P ATA

all torrents are saved, all such data of torrents are on:

WD My Book 1TB using eSATA (FYI i love the drive and i havent had any problems with it, would reccomend for you guys as well!)

so as you all can see nothing killer about my machine at all so hopefully this post can help everyone :-)

side note the 4GB ive never come close to using it all, in winXP or Ubuntu, even when playing two full length bluray movies, kills my cpu but never touches more than half of that RAM

also this is a stock winXP im running and its so slow with 1Mb/s or more coming in downloading, im looking into a revised or tweaked copy to see if there is any advantage for connection speeds and downloads

just some food for thought guys,

happy downloading,

-BIG Earl

[aa]

Have you tried defragmenting your disk with JKDefrag? How long did it take? Is your sparsing off? Do you have the «Pre-allocate all files» turned on? If you answer, I might suggest other things to do.

[Rinkue]

Not sure if this will help any one else but here it goes. I stared getting this same error yesturday and came here to see if I could find out what happened. I tried everything here and nothing worked. So I started looking around Windows and I noticed that after a explorer crash that the disk write-caching had been disabled during the crash and after re-applying it No more issuse with disk overload.

Well that worked for a short time then it started again. After thinking about it and trying to nail down the problem. I started having this problem whenI installed windows 7 beta and thought that it could be an issue but I used it with no issuse for a few days right up to the point that I installed Norton's AV because McAfee that I would normally use will not work with Windows 7. So after stoping Norton's before I start uTorrent I can download with out the disk overload I will try it this way for a few days to see if it keeps up.

[digitalage]

diskio.write_queue_size is not available in the latest release (1.8.2).

I got slowly dead Windows and didn't know why (see http://forum.utorrent.com/viewtopic.php?id=55083).

I guess is Disk Overload fault. I payed attention to this and increased diskio.coalesce_write_size to a values up to 16777216 (16 MB), but I was still getting Disk Overload for speeds above 4-6 MB/s (depending on how many torrents I was downloading). I noticed that as more active torrents I run, less speed I got without Disk Overload. I should say that my link allow me to download to speeds up to 8 MB/s or even more. Based on experiments, I let diskio.coalesce_write_size with 16 GB and forced maximum download speed to 2048 (2 MB/s), which allow me to run 8-10 torrents simultaneous without crashing. If I limit the number of simultaneous active torrents, it allows me to increase the download speed without Disk Overload. Keep in mind that the speed and other characteristics of your hard-drive may have influence on the experiments I did. Anyways, just to let you know, I run torrents on a separate 80 GB HDD.

[Switeck]

Disk overload is probably triggered at least in part by how many peers you're downloading/uploading to at once. Fewer upload slots even at slightly faster total speeds probably are less likely to trigger disk overloads.

Antivirus and file indexers make disk overloads immensely more likely, as they try to read the files over and over again when changes are made.

[digitalage]

Switeck, what you say makes perfectly sense. However, speeds increase as time pass and we need a way to avoid such problems.

To developers:

Guys, you need to find a way to keep the good work you've done so far with uTorrent. Somehow, such things as "Disk overload" should be avoided by making uTorrent act intelligently. It has no sense to have an engine that work extremely fast, if the hardware won't fit with it. It's useless. And users have not always the knowledge to report their experience. It wouldn't heart if betas have more advanced features, so that techy users would play with it and report. I'm one of them. Let's keep quality of uTorrent, in respect to its developers and users worldwide.

[Ultima]

That diskio.write_queue_size has been removed, and the disk cache settings have been moved to Preferences > Advanced > Disk Cache (see manual) since at least µTorrent 1.6. If you don't understand what the advanced settings are for (you don't, if you're messing with diskio.coalesce_write_size as you are), then you shouldn't be touching them.

Finally, if there is a hardware limitation, how would you expect µTorrent to get by the bottleneck? Work some kind of impossible magic? Or are you suggesting that µTorrent simply not show "Disk Overload," in which case it'll just be at fault for not being more forward about the cause of a slowdown?

[optimus prime]

Just to add my 2 cents, IMO, hardware limitation has got nothing to do with it, now I could be wrong but, there are plenty of users here posting this problem who have powerful rigs. My POS I use to download on is an old Toshiba TE2000 laptop with only 256mb of ram. The torrents are stored on a 3.5" hard drive in a hard drive dock connected to the lappy via USB1!!! Now thats limitations!

I once had this problem on this lappy and I solved it and now I do not see the problem anymore.

On XP. What I did to fix it was:

Under preferences, goto advanced then disk cache, I have unticked "Overide automatic cache size...", ticked "Reduce memory usage when cache is not needed". Under Advanced cache settings I have unticked "Enable caching of disk writes", ticked "Enable caching of disk reads", unticked "turn off read caching if the upload speed is slow", ticked "remove old blocks from cache", unticked "increase automatic cache size when cache thrashing", ticked "disable windows caching of disk writes" and unticked "disable windows caching of disk reads".

Now that was for XP but when I got the disk overload problem my up and down load would virtually stop, now it doesn't.