uTorrent generates huge load when downloading partial torrents

[bugboy]

uTorrent generates a ~uTorrentPartfile when you download a torrent and choose to skip some of the files inside the torrent. This file contains the leftovers, but the file is changed very frequently and is as large as the total size of the data inside the torrent. Because the file is changed very frequently it is backed up every hour when you use time machine. This generates a lot of load and fills up the back-up space pretty fast.

It would have been better to skip the file, but I guess this isn't possible. I would rather see that there is a dedicated directory that contains these left-overs that is removed when the torrent has completed. Because these files won't change as much as the partfile it would generate less load.

[Ultima]

Partfiles will never be removed on completion -- only on actual torrent job removal. Why? Because partfiles are needed for proper re-seeding should the torrent be started again after completion.

In what way would it "generate less load" by moving partfiles into their own dedicated directory? They'd still get written to as often as they always do.

[bugboy]

After completing the download the file isn't changed anymore, so it won't be backed-up. Suppose I download a 10GB torrent file that takes 50 hours to complete. Then Time Machine will backup a 10GB large file 50 times, which takes up 500GB space (as large as most time capsules).

It would be a lot time-machine friendlier to create the skipped files in another folder for which parts are received. These files won't be changed that much and such a directory could be excluded for time machine. I know it is a little bit specialized case, but because time machine is an integral part of OS X I think that a Mac version of uTorrent should behave nicely :-)

For now I think I won't skip any files anymore, but use priorities. Files that need to be downloaded will be marked with 'high' and the others with low. When the 'high' files are completed, then I stop uTorrent and delete the 'low' files that are (partially) downloaded.

[Ultima]

I'll be frank and tell you that I'm not familiar with how Time Machine decides how/what to back up I understand what you're trying to get across here, but I'm just not certain that isolating the partfiles in their own directories would make a bit of difference.

- When Time Machine backs up, it backs up entire directories instead of changed files...? That itself sounds like silly behavior as is. Can't you tell Time Machine to ignore certain directories (namely, your incomplete downloads directories)?

- When a partfile changes, that means the other files being downloaded will also change anyway. How does moving the file out prevent those other files from being backed up...? Wouldn't the directory in which the partfiles are being stored also get backed up by Time Machine anyway?

- Partfiles are used to prevent entire files from being allocated, so it really shouldn't be taking as much space as normal unless the files you're skipping are all rather small... Is this the case?

[bugboy]

The exact algorithm of time machine is also unknown to me. I don't think it is documented, but I have heard it uses the Spotlight indexing service. You can simply say it backs up files that have been changed since the last backup.

You can tell Time Machine to ignore certain directories, so an incomplete directory would be great, so you can exclude it completely. I know aMule uses this approach. Incomplete torrents won't be backed up, but I don't think this is a problem. Unfortunately, the current uTorrent version for the Mac doesn't support this feature.

I guess if you use smaller files (like an MP3 discography of an artist), then only small files will be backed up which is faster. But forget about these small files. An 'incomplete' directory would be a lot better :-)

I am not very familiar with Mac OS development (I am a MS developer), but I guess you can use sparse files on Mac OS too to prevent that the entire file will be allocated on disk. If it does, then it seems that Time Machine doesn't support this and backs up a whole lot of empty space.

[ArthurDent006.5]

I'm sticking a related-but-not-exactly-the-same comment into the same topic. If that's bad, feel free to tell me to start a new topic.

Based on an Ars Technica article, I think that: every hour, Time Machine collects:

- the set of directories that have changed, from some sort of file-system-watcher service

- for each directory, the set of files that have metadata different from the last backup

and copies out the whole contents of each file.

I have a torrent that I'm in the middle of, that consists of 11 large (3 GB) files. uTorrent downloads bits of all of them concurrently, causing all of them to be backed up every hour. I can manually tell it to skip almost all of them, in order to get only one large file backed up, for a few hours, while it downloads, but I don't know that I've chosen the optimal file.

It would be nice to have a limiting option to say something like:

- concurrently work on a set of files with an aggregate size only up to 1GB, or a single file