Fragmentation. What can be done?

[DDRs]

Hi, I have been wondering for a while why my downloads disk has been performing badly for some time, and I believe I have discovered the answer.

The attached pictures show my disk straight after defragmentation, straight after downloading a 111MB file through uTorrent, and then again straight after deleting that 111MB file. (not sure what the gap in the green is about, but it is irrelevant to this)

It is a 640GB disk (596GB formatted), it can handle contiguous files at around 70MB/s.  It handles these highly fragmented files at anything between 10MB/s and 30MB/s, usually at about 15MB/s.

If you the imagine how much extra work the heads have to do to deal with even more data, the mechanical wear involved, and how much this is going to affect system performance for someone who only has 1 partition to work with, I'm sure you will agree there must be a better way.

I seem to remember an option at one time to set out the disk space required for each file when you start a download, but I can't seem to find it now.  As an IT consultant/repairman I would recommend people use this option (if it is available) by default, or they risk a higher chance of losing their drive and data.

[DreadWingKnight]

Not a lot can be done since disk allocation is typically handled by the OS and not by uTorrent.

uTorrent already requests a preference to sequential blocks but if the OS doesn't give them, uTorrent doesn't get them.

Also, Torrent downloads are NOT sequential, so any drive benchmarks that only consider sequential data transfer are useless.

[DDRs]

If that is the case, then why does windows handle all my other files differently?

[DreadWingKnight]

Because all your other files download in a different order.

[DDRs]

I mean in general.  Windows has improved the way it places files so as to avoid defragmentation over the years, Even before that it would start files at the first chunk of contiguous space so it would at least all be in one place, if possible.

Wasn't there an option in uTorrent at some point to allocate the disk space for the file before it starts downloading?  Wouldn't this be contiguous?

Edit: where would I find the settings you describe in your next post?

[DreadWingKnight]

Again, no guarantees, but you can go turn on preallocate with sparse disabled and try.

[DDRs]

Found the Pre-Allocate all files option in the options, has made a world of difference.  files now copy from one disk to another at 3 times the speed, disk mechanisms won't wear out as quickly, and it won't slow people who only have 1 partition's computers to a crawl.  No additional side effects from what I can see.  I highly recommend people use it and Utorrent developers enable it by default.

[rafi]

What is your advanced->diskio.parse_files set to?

For me, enabling pre-allocate  delayed the startup of downloads considerably, since they have a bug there, where they unintentionally (due to a bug) zero the whole file on pre-allocation.  This creates disk-overhead when you start downloads of large files.

Also, if what you say is true, uTorrent pre-allocates files (even w/o this option set) once the first block is written to them. There shouldn't be any difference IMHO as to the way it asks to allocates them (continuous). If, as you've shown , there is a difference - it's a bug.

[DDRs]

diskio.parse_files is set to true.  I've not changed this setting.

I used to see the pre allocation delay that you described which is why I initially welcomed the changes that eliminated it.

Now, with the single "Pre-allocate all files" ticked in the general tab, I see "Pre-allocating" appear in the status of each new torrent, but only for a spilt second.

Checking the drive image with perfect disk confirms to me that files are now being allocated continuously.



Ignoring the top 2 thirds which consist of defragmented files, gaps where defragmented files have been removed, and Green/Orange blocks that Perfectdisk has deemed to need moving based on a 60 day threshold, The bottom third is where newly downloaded files have been placed continuously.

[rafi]

With which version/build of uTorrent is it (that you do not see any delay)?

[DDRs]

42094

[rafi]

Well, with the latest stable 3.4.7 42330 - the delay (overload writing zeros bug) is still there , and just adjusting sparse_files to false seems good enough ...

[DDRs]

So it's been fixed? without having to tick the box?  My advanced settings doesn't have a defrag files option.

Edit, Read you post below, are you saying that sparse files and pre-allocate essentially have the same effect, just a different way of going about it?

Will try sparse disabled for a while.

[rafi]

Not "fixed", this is the behavior... You do have the option:  diskio.sparse_files ..... The problem is that the default is true, and you need it to be false...