How to combine similar torrents ??

[TheDude]

Hi !

I am trying to download a torrent and it happens to be available on two different trackers.

All the files are the same, except that on one tracker, the uploader added an additional small TEXT file.

On both trackers, the torrent has few seeders (1 to 2).

So, I'd like to combine them together and get pieces from both - is this possible with utorrent ?

Or will the TEXT file cause the pieces to be different in each torrent ?

Thanks !

[Firon]

You can't download both at the same time. You can try skipping half the files on one, then half the files on other, but it may not work.

[luk32]

ummm... I'd try copying tracker list from one torrent into the second. I think, that when torrent client want's the data from the tracker, it does not borders to the same torrent only, because. I guess it's per file based, per piece checking would be too dangerous but that's my intuition. I think it's at least file 'cause when you are downloading some single episodes, they encourage you, to drop single-file torrents and use the whole-pack-one. And i belive that whole-pack torrents can cooperate with the single-file. What i'm trying to proove is, that it's not important which torrent do you use, as long as part of the content is the same, the data will be exchaged between all the users. In my opinion it would even enhanced torrents performance, 'cause, at least for the part of the data the swarm would be greater, which, i belive is, always considered better [though it's not always ture]. It's just harder to determine the seeds and peers numbers, but it's not very important, for the data sharing itself.

Anyway, it shouldn't do any bad, and i'd give it a shot.

In the worst case, you won't see any improvement.

Sorry for my english, and thx for reading .

[splintax]

I don't quite understand what you're trying to say, but...

BitTorrent hashes each individual piece of the file. That's why it bothers splitting files up into pieces, and how you can grab a large file from mutliple sources.

The reason people recommend that you upload files in .rars is because sometimes even after BT's hashing problems occur and a rar needs to be re-grabbed. Though I've never experienced this problem, apparently some people do (crappy BT client I guess ).

[luk32]

but i think that TheDude want's to combine the same files from two torrents, or better said "combine two torrents" which content varies just a little.

I was wondering why and how this would be possible.

I suppose that you misunderstood examle i given, i meant, that sometimes people are relasing files, week by week, as the ...for example it is aired on TV, and then, when the whole series ends, they are buliding one "large" torrent with all the single releases still as separate files though, i think you understand. If not i'll try to do the "łopatology" you have 13 torrents with one episode per torrent, and then one with all the eps included (like 13ep in torrent). And i think, that folk with the single-file torrent can exchange the data with the guy who is interested in dowloading the whole series and have the "large" torrent.

Basing on this I am implying that you can share the same data "between two torrents". It's not properly named, but it looks kind like this. Is more, like you can exchange the files (or more universal the data) independently of the torrent file you use for it as long as it the same data [preety logical].

And when you could share the data regardless the torrent, but the data itself, theoretically you can combine two torrents [since they "contain" "shared" files or any the same data].

I think that depends on the torrent protocol, or the tracker.

I know my english is spined and twisted. I apologize for that and for it's inunderstandability

If somebody still doesn't understand something, just point it out and i'll try to explain.

[DreadWingKnight]

You can't be actively downloading two torrents with different infohashes into the same set of files.

Download one torrent, hash check the second before starting and resume it.

And when you could share the data regardless the torrent, but the data itself, theoretically you can combine two torrents [since they "contain" "shared" files or any the same data].

I think that depends on the torrent protocol, or the tracker.

Combining two differnet torrents (torrents with different infohashes) to download isn't possible. The protocol doesn't permit it. The tracker has no say in this.

Seeding two different torrents from the same batch of data is possible, since the locks on the file don't prevent multiple torrents from reading at the same time.

In the original post's example, it's NOT possible to combine the torrents.

[osm0sis]

yeah, unfortunately it's not possible. usually because of differing peice sizes and whatnot, what's neat is sometimes if you're downloading one and you stop and switch, if the second tracker's torrent has a smaller peice size it'll take pretty much all of your previously download data and be able to hash it so you'll pick up where you left off and not lose much of a %.

[luk32]

I'd like to clarificate, that by "combining" i never meant downloading or having active two torrents operating physically on the same files at one time. This would be... um. collquially saying ... very very bad; hard to handle, synchronise and many other worse things.

I was just thinking, if it's possible, why and how could work migrating between two torrents with the same data (both already downloaded or possesed and "contained" torrent ).

It'd be not far from "combining" two torrents.

How i see this working and why protocol and tracker dependent?

'casue if the protocl itself doesn't provide the solution, the smart tracker could do it.

Basically it's like: Client wants the data asks the tracker who have it and gets an answer.

Now that could work like that, Client asks about particular piece of a particular file. Then tracker don't need to point on the guy who have the same torrent file, but just this particular piece of file.

So if you were able to identify the file, it's not important which torrents clients use. Torrent might exist only to provide about what to ask.

Also i can see two version of the same thinking on different level of data identifying. First, when tracker assumes, that two clients "operate" on the same data, when they have the same torrents, i belive, that's why the torrent's hash exists.

Thay could exchange data as long as the have the same torrents.

And second case where the data identifying is even "closer to the ground", I mean you could share the data with someone [mutually of course] as long as it would be the same piece. It don't even need to be the same file, since two files could contain the same binary data. Sure it's very unlike to happen for two files, to contain by any chance the same [let's say] 128kb of code. But, i'm talking just about the possibilyties.

I really don't know how torrents and trackers are implemened, i'm just thinking loud, about how could this or that be possible.

So if asking for data, would mean to ask, about the part of the file, then it's not important whether he has the same torrent or he uses other one. The important thing is, that he has, the file that you want.

Now combining two torrents would no more nor less than combine the tracker list. It's obvious that the file lists can be different but as the main point i see ability to exchange the data with users of both torrents which would be provided. Which one file list we would like would be up to us.

I hope my statements are clear, tell me what you think

Any feedback is good.

Oh and if it's like DreadWingKnight stated, then one is restrained to exchange the data with peers working on the same torrent.

Then I am asking, how it is possible, to share data between single-file[episode] torrent and those batch ones, with all the episodes included. It works somehow i belive. Or users of each torrent can't and don't communicate in fact ?

Oh... and i hope that TheDude tried to "combine" the torrents and is not waiting for conclusion whether it's theoretically possible or not.

[DreadWingKnight]

Now that could work like that, Client asks about particular piece of a particular file. Then tracker don't need to point on the guy who have the same torrent file, but just this particular piece of file.

The BT protocol doesn't permit this at all. Trackers only give out peers matching the infohash. It can't bridge torrents with different infohashes. It doesn't matter how smart the tracker is, the protocol just plain doesn't permit it.

Oh and if it's like DreadWingKnight stated, then one is restrained to exchange the data with peers working on the same torrent.

Then I am asking, how it is possible, to share data between single-file[episode] torrent and those batch ones, with all the episodes included. It works somehow i belive. Or users of each torrent can't and don't communicate in fact ?

Clients can't and don't communicate the fact that the single file torrents match files in the batch. The user needs to know the files match and attempt to resume the batch from the single files.

[luk32]

Thank you for solving the case DreadWingKnight.

Now I know that data is identiofied on the "torrent level", so basically to establish a connection and exchange the data between two users, the have to use the same torrent.

So it's preety impossible to "combine" two torrents.

The only one thing we can do is to combine the tracker lists, if our torrents are "identical", i think, that hash info is based just on files provided by the torrent and maybe some other really vunerable torrent-specific values, so tracker list shouldn't change the hash.

Still I don't think my ideas are bad at first glance they might work .

Anyway, they are forbidden in current version of the protocol.

Again, thanks DreadWingKnight for your clarification

See ya.

[Added after DreadWingKnight's post below, but not worth of another one]

Sure you're right.

It's my bad I forgot 'bout this feature.

So, we can't help in the case. I belive that what can be done is done, at least as for now.

[DreadWingKnight]

The only one thing we can do is to combine the tracker lists, if our torrents are "identical", i think, that hash info is based just on files provided by the torrent and maybe some other really vunerable torrent-specific values, so tracker list shouldn't change the hash.

µTorrent already supports adding tracker lists from added torrents to existing torrents in its download queue if the infohash matches. These are considered to be the "same torrent" for purposes of the protocol.

Some tracker packages already support exchanging peerlists between them on that side (transparent to the user).

[eyal0]

I had this problem so I wrote software to solve it.

http://code.google.com/p/mergetorrent/

To use it, first select your .torrent file. Then select the two directories where you have partially downloaded data. Then select a target directory (which can be the same as one of the first two directories). Now press the merge button.

I hope this helps you. It helped me. Good luck,

Eyal

[tymes]

Seriously, this is an excellent little app.

This is exactly what I have wanted for a few years.

Any thought on specifying both .torrent files? I have enough common sense to either use the live torrent or otherwise use the torrent with the smallest pieces, but right now I'm downloading several season of stuff that is available entirely and also per season so I was trying to optimize the download by downloading both and seeing where that took me (fortunately later, I'll be able to seed both). Unfortunately the .txt file in the torrents disjoints the pieces and you have to recognize the differences in block size and what that also means.

In the past I would use 2 clients and the 2 different torrents on each... I would be able to share information between the two different torrents by periodically re-targeting and exchanging the second set of torrents so all the info from torrents1 would become available for torrent2. This also mean that while they were merging, they were still downloading and new information was always being introduced even while the second client was offline checking pieces.

Meanwhile Vuze/Azureus has features: "Advanced/Files/Periodically Recheck Incomplete Pieces" and "Enforce exclusive file write access locking across torrents" which helped downloading different torrents to the same place

Yet, again, the mergetorrent app is stupendous.

[eyal0]

Any thought on specifying both .torrent files?

I'll think about it. Maybe I'll give mergetorrent an advanced mode that lets you specify as many .torrents as you like and as many sources as you like.

In the meantime you can use one of the .torrent files, then use the other .torrent file, then go back to the first.

I'm glad it helped you.

[drano]

eyal0

EEEEYYYOOOWWW!

mergetorrent.exe is GREAT!!!! This really saves a lot of redundant downloading that might otherwise be necessary to complete a "dead" torrent. I've wanted some code to do this forever! I don't understand why this is not a standard feature in torrent clients. This is the kind of programming that is too often left undone: where the user already has the necessary data which would be easy for the computer to manipulate as desired by the user but the user cannot find application code to do the job! This adds value to data the user already has! That is entire purpose of computing!

[bruticus0]

OMG..thank you eya! i registered just to thank you for this app. I was really in need of this to get my fav torrent completed faster! i went from two almost dead torrents, to havin a chance to get it finished. Thank you so much again.

matt

[jamak]

You CAN do it, just not with utorrent:

http://wiki.vuze.com/w/Multiple_torrents_accessing_shared_file%28s%29

[moogly]

You CAN do it, just not with utorrent:

http://wiki.vuze.com/w/Multiple_torrents_accessing_shared_file%28s%29

That's the crucial point:

This will not synchronize their piece completion states. To do the syncing you have to go to the My Torrents View and enable Rightclick > Advanced > Files > Periodically recheck incomplete pieces for each torrent that should check for pieces that have been added by other torrents.

But you can do it in µT but it's not very user-friendly: stop the torrent jobs and force a recheck.