Torrent Merge

[Brandy Down Under]

On several occasions, I have been downloading a very large torrent that no longer has any 100% seeds.  Looking about, I've found other related torrents that contain the same files (or almost all the same anyway).. Trying to download all of these torrents can be time consuming (4 weeks or more estimate each) and disk intensive (300GB each). While I can't do anything about the disk utilization, I could accelerate the process by manually copying files from one torrent to the other. I'd stop the torrents and then select the merge direction based upon percentage of completion for each file.  When this process was complete, I'd force a recheck. I've been able to change the newer torrents from low amounts like 1% or 2% to an overall completion around 95%.  Even though none of the torrents had a 100% seed, I was able to complete at least one, with close to 99% for the others.

 

And yes, I seeded these until I got to a ratio over 10.  A dirty job, but somebody's got to do it.

 

So, my question is:  Is there some way that two torrents could be merged?  Or at least compare the file hashes to identify files that could be merged.  That would eliminate the mistakes that can occur by manually merging two files that don't belong - loosing what percentage that the file did have.

 

An extension to this request is cross merging of the pieces in each of the files.  The hash of corresponding pieces could be computed and compared to the values in the torrent file.  When there is a match on one side or the other, that piece could be move to the file that doesn't have it.  Another extension would be to change the download locations so that the two torrents share the same actual file.  That would be ideal, but probably too complicated to keep the two torrents in synchronization.

 

 

[DreadWingKnight]

Because the hashes in torrents are per-piece and not per-file, it's extremely difficult at the best of times to bridge torrents.

Your existing file tetris technique is actually the most reliable for the least effort.

[hermanm]

This one of the disadvantages of hashing multiple files to achieve the piece size. I've suggested Bittorrent change their protocol to allow for different piece sizes while creating the torrent. This way, the end of a file, is the end of a hash. At least with the Maelstrom project, this type of change would be better suited for serving files. It does add some overhead to the torrent size due to more hashes, but I think it is worth the overhead so that force rechecks can actually complete to 100% rather than 99%.