Jump to content

question on settings for disk cache & diskio.sparse_files for my pc


Thunderbolt

Recommended Posts

comp:

c2d E7200 (2x 2.53 L2: 3mb fsb:1066)

3 GB ram

2x500 GB raid-0

100mbit up & down

NTFS file system

goal: best up- and downspeed = 'avoiding disk overload 100% as much as possible'

now I have 2 questions regarding cache and diskio.sparse files:

1. how much cache should I set with this comp I have? it could very well be possible that I have 3-4 active files with full speed running + upload. should I increase the cache size the better my comp is? or decrease it then? or even let it be utorrent handle it itself (big filesizes)?

2. whats the advantage of having diskio.sparse files activated? or whats the disadvantage better said, else it could be default. I heard the disk could become quite fragmented after a while, and disk overload problems could occur more often then, especially with multiple active files. is this true? is there a difference if the disk gets fuller and fuller over time? what would you recommend? and in how far would it be better or worse to activate this option, depending on the cache size?

thanks!

Link to comment
Share on other sites

For cache options that reduce disk activity use the manual, make sure your current transfer mode is NOT PIO. You can check this in the device manager, IDE controller properties.

Regarding sparse files,

Enabling this option causes µTorrent to allocate only the data that it writes, but will inform the filesystem of the file's size (so that it can attempt to reserve enough contiguous space on the hard drive without having to physically zero all of the space out for the file). Even though space is reserved for the file, no space will be taken for the unwritten parts of the file. Enabling this option may potentially lead to increased disk fragmentation in rare cases where the drive does not have enough free space available to honor the space reservation for sparse files. Here are some things to take note of when using this option:

If you enable sparse files, AND full disk compression, then the undownloaded data wont take up data on the drive, it'll be "compressed" to its downloaded size. If the file is uncompressed, even though it's not technically taking up space, the full size will be counted on your used space. I'm not sure how disk compression will affect fragmentation, considering a glitch in one of the beta µT's it doesn't seem to have been taken into account.

So.. it's ok to have on, and even more useful if you're short on space.

Link to comment
Share on other sites

I meant the utorrent disk cache settings, which you can specify yourself if you like. I guess you meant the HDD settings in windows

they are SATA2 btw

and im still using 1.7.7 or do you recommend switching to the latest beta version?

edit: nvm guess you meant that as well :P

so is it possible to say when the diskio.sparse_files option begins to slow down the overall speed? for example when the disk is half full or something like that?

Link to comment
Share on other sites

If the bus is fast enough you won't get disk overloaded message. I'd start with 200 MiB of cache and go from there. Some people use over 1024 MiB... but don't go over that without using 1.8

Fragmentation increases with disk used %. Sparse files have little to do with it. Windows does all this in the background.

If you're trying to minimize disk usage, in the 1.8 Disk Cache pane, set your cache, disable reduce memory usage, disable write out untouched pieces, disable write out pieces immediately, disable turn off read cacheing if too slow, enable increase cache when thrashing, and depending upon which Windows enable disable windows cacheing of writes AND reads.

Link to comment
Share on other sites

but wouldnt uncheckmarking the option "turn off reading cache if upload speed is too slow" slow down my traffic as well? I only want to take usage from HDD as long as it makes my overall traffic faster (or at least not slower as well)

regading 1.8, I've had some crashes with larger file sizes, files >5-6 GB. it just would load up its cache and then do nothing with those 256 block pieces. left it running for 2 hours, nothing happened.

it this a know problem right now, or did I maybe just screw with cache settings here?

Link to comment
Share on other sites

Uhm, a block is 16 KiB. Pieces can be anything from 32 KiB to 4 MiB (some make larger but uT limits to 4 MiB, which is 256 blocks). Based upon need both read and write cache used will not exceed the set disk cache number. I'm not sure the inflationary % when it's thrashing though.

I don't know what you mean... if uT crashed, did it create a dumpfile? The cache is supplementary to direct disk access. If uT crashed and you didn't restart, of course nothing happened. What's the question/behaviour you're asking about?

Link to comment
Share on other sites

didnt know about a dumpfile yet, so I dont know.

what I meant is it basically crashed while trying to save all those block of a ~12 GB file or something like that. I left it running for 12 hours, but no further activity happened. so in the end I terminated utorrent.exe via task manager and restarted windows, because I couldnt delete everything from the directory that file created then.

this happened as well with another file with ~7GB of size. then I switched back to 1.7.7 and it was fine for me.

Link to comment
Share on other sites

> If you enable sparse files, AND full disk compression, then the undownloaded data wont

> take up data on the drive, it'll be "compressed" to its downloaded size.

If you're using NTFS, sparse file will take only take up the actually downloaded amount while reporting the full size in Explorer. Go to Properties to see file size and then right below it is the actual disk space used. NTFS compression has no bearing on this disk space used, other than actually reducing the space used further. My point - NTFS Compression is not required to use sparse file nor to achieve disk space savings. There really isn't a good reason to use NTFS Compression any since disk space to cost ratio has come down significantly since Windows NT 4.0. Also, if you download video or other files that are further compressible, you're wasting CPU and disk time by using NTFS compression.

> Uhm, a block is 16 KiB.

Probably a good idea to set your NTFS cluster size to 16 KiB on your initial disk format. This will match the Bittorrent block size and reduce fragmentation. A standard NTFS cluster size of 4 KiB could potentially require 4 reads or 4 writes if each cluster is not located sequentially on the disk. You can minimize fragmentation even with sparse file turned on by using a temp disk for incomplete torrents, then then have µTorrent move your files to a different drive upon completion. The above recommendation only really becomes useful though if your disk activity is the performance bottleneck. I'm sure other users will know better, but I don't see disk being the bottleneck unless you're running 100+ Mbit or have a LAN peer.

Link to comment
Share on other sites

I mistyped, if compression is NOT used, the files unsparse size will be counted in the HDD's properties. Enabling compression gives a more accurate reading of allocated data in the HDD properties. And NTFS compression DOES prove useful in some scenarios.

Side note, I read that once you have enough data that the unsparse size exceeds the partitions capacity, it causes some "weird things" to happen.

Link to comment
Share on other sites

"Bad things happen" when the number of sparse blocks exceeds the hard-coded limit in the NTFS driver. Theoretically there is a ... large limit, but in the current implementation (up through Vista SP1) it's limited to some "small" number, probably 2^16-1, or maybe even 2^8-1. It's a known problem to MS, so people keep posting to MS about it. It's the reason sparse files is off in Vista. There's een reference that ~ 1 year ago another filesharing app (can't remember which) also disables sparse flag in their program due to the limit.

Link to comment
Share on other sites

that cluster size changing thing is a good idead. 2 questions though:

-is it possible or better put still effective to change it when windows is already installed?

-is it then connected with data loss? (apart from the possibility that I dont have that much free space afterwards which is needed for this cluster rezising, because I will gain more 'loss' by that)

Link to comment
Share on other sites

It's best to NOT resize clusters on exising working SYSTEM volumes, that is the partition you boot windows from. With any operation there's a % chance of data loss.. like if you lost power during conversion. What cluster size would you be converting from? You'll only lose noticeable freespace wher eyou have an abundance of files < cluster_size. I remember the first time I converted from FAT to FAT32, I got back 25% of my 2 GiB hard drive ><

Link to comment
Share on other sites

What he is saying is files less then 16 whatever will still take 16 whatever, he is also saying that if you lose power, it would be bad. You should have battery backup when doing potentially data corrupting ops :P.

Link to comment
Share on other sites

Well, it IS supposed to increase performance. Anyway, it's always a good idea to invest. I got to use my computer to watch tv for about 10 minutes! Of course I was just showing off to a friend, it would have lasted longer but I connected my stereo to it :D. Really it's meant to buy time for power critical functions.

Link to comment
Share on other sites

You might consider changing the cluster size only if your current setup is inadequate. Your system sounds like it is fast enough now to keep up with torrent activity especially since you're running a striped volume...

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...