Jump to content

Selecting downloads from a torrent containing a huge amount of files


inca

Recommended Posts

Hi

here is the catch...

I have many files to share in the same torrent and I have a community of users that will want only to download a selected list of the files present in the torrent. Because the user selection is randomly I need to put everything in one torrent. While each user will download an group of files the overall community will in average download the entire package.

So as an example:

torrent file with files A, B, C, D, E, F

User #1 will want to download only A, D

User #2 will want to download only C, D

User #3 will want to download only B, E

User #4 will want to download only D, F

User #5 will want to download all A, B, C, D, E, F

User #6 will want to download only A, E

Now this is currently feasible with some clients that allow selective download. The problem is that I will have many many files in the torrent (> 1000). To facilitate the user selection I'll have even a database driven web site where the user can query and discover which are the files he/she desires. Because of their numbers it is impossible to do torrents with every possible combination and if I would do this the files wouldn't be shared within the same swarm.

On the other hand, doing a torrent per file will force the user to download several torrent files something that I want to avoid.

So I've tried to do dynamic fake torrents that would only include the information of the files desired by each user but still pointing to the same base torrent. This fails miserably because of the initial hash code checking to see the consistency of the torrent file.

Is this possible ? or a feature that someone is thinking about ?

thanks and good luck

Link to comment
Share on other sites

BitTorrent does not have the concept of files that you think it does.

In a multi-file torrent, bittorrent only has the concept of pieces, not individual files.

The kind of individual file demand you have is not what multi-file torrents are designed for.

You're better off offering single-file torrents for all of the files individually.

Link to comment
Share on other sites

thanks for your quick response

you're right .. I know that I'm trying to adapt the design to the requirements :)

but do you think I could achieve this if I store in the db the corresponding pieces to files relation.

so the previous relation would refer to pieces and not files

torrent file with files A, B, C, D, E, F

User #1 will want to download only pieces A1, A2, A3, B1, C9,D1, D2, D3, D4

like in the selective donwload of some clients ?...

another way was to create a torrent that implicitly tells the client to not download some of the files (on/off tag somewhere?)

what do you think ?

thanks

Link to comment
Share on other sites

No I don't think you could do what you want with bittorrent.

BitTorrent is NOT a traditional p2p network.

It has NO concept of files at the protocol level.

Pieces in multi-file torrents cross files frequently (and in some cases one piece can span multiple files).

Any piece relation database will automatically fail because of this.

The kind of functionality you're after is from a traditional p2p network, not from BitTorrent.

Link to comment
Share on other sites

thanks for you direct answer

but .. wihout being pesky about it ... the file-piece relation is explicitly defined in the torrent

at the same time some bittorrent clients allow the user to select the desired files from a torrent file containing several files (e.g. azureus)

In the case of a piece spanning multiple files this is surely not optimal but at least we are not forcing the user to download the entire collection.

I really need to have a big torrent shared by some users while other users will only seed some pieces/files. For the last type and as the number of files is quite huge the trick might be fool the client into believing that the undesirable files are not being seed by anyone or that should not being downloaded. And least this type of users will be participating on the overall effort even if they only want a piece of the cake.

Did you know if someone has done some work on the prioritization of the order of piece download?

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...