Jump to content

Copy torrent to USB-flashdrive in real time using binary diff?


Recommended Posts

I have a device which plays media from usb, and I use it quite often together with a usb-flashdrive. It's a drag though having to first wait for the download to the HDD, then wait for the transfer to the flashdrive.

I know it's bad to be directly downloading to the flashdrive since utorrent makes a lot of write/read operations. The question I have is how much does it wear the flashdrive down? Let's say one downloaded 5 torrents a week to it, how long would it take before the flashdrive failed? If it's not "that bad" I can just keep using cheap ones.

Having said that I'm only getting 100-300kb/s (from what normally is a 10-11mb/s torrent) when downloading to the flashdrive, it should be 4-5mb/s as that's what I get when copying files from windows, and 5mb/s is the write specifications of the drive. Is this slow torrent downloading a limitation with flashdrives or is there some setting I can change?

Above 2 problems got me thinking. What if one could use a file synchronization tool with binary diff (compares two files and only updates the changes instead of creating a full new file) which monitors the torrent folder on the HDD in realtime and copies/updates the torrentfile(s) to the flashdrive as utorrent downloads the data?

So I tried using freefilesync with the file content option (which from the description is binary diff) to try to synchronize the torrent to another folder. First it works (when the second folder is empty). The second time, when it tries to compare changes, I get error message about the file being locked by utorrent. Is there a way around this? Is it possible to make this work at all or doomed to fail? There's an option in freefilesync to copy locked files using Volume Shadow Copy Service and it's ticked, so I don't understand what the problem is?

Edit: I found out freefilesync does not use voluem shadow copy during binary file comparison and therefore can't access the locked file.

Edit2: I found rsync can copy only changed parts of a file - this in conjunction with shadowspawn.exe (https://github.com/candera/shadowspawn) to copy the locked torrent via volume shadow copy achieves what I wanted. However rsync takes too long to compare before starting the copy, so it's not a viable solution.

Link to comment
Share on other sites

After a certain point large flash drives are likely to fail for other reasons then worn down flash memory, due to wear leveling techniques. Many UFD's are terrible at random write performance, so torrenting being slow makes sense. You may be able to compensate for this with more aggressive caching in uTorrent and/or enabling caching for it in Windows.

Wikipedia says 100,000 write cycles is typical, but mentions 10,000 cycles. You may want to research that further but lets use 10,000 cycles and a 16 GB flash drive as an example.

UFD size * 1024 (speed is in MB, converting GB to MB) * 10,000 (how many times you can write a cell) / write speed in MB/s / 60 (seconds to minutes) /60 (minutes to hours) / 24 (hours to days) /365 (days to years) = years of life

16 * 1024 * 10,000 / 5 / 60 / 60 / 24 / 365 = 1.039066463723998 years. This would be the WORST case scenario, constantly writing to the drive without end, based on the drive size and write cycle limit given. Depending on how often you torrent, you probably don't have to worry at that size flash or greater. YMMV.

Link to comment
Share on other sites


This topic is now archived and is closed to further replies.

  • Create New...