DHT Support

[paritybit]

I have searched the forums looking for any idea about DHT support planned for uTorrent. I must have missed something but I was just wondering if I could get a definative (ie. Yes it will or No it isn't planned) answer if DHT support is planned for a future version of uTorrent? All I found was azureus bashing, which is cool, at least there wasn't any Java bashing.

[BlackLion]

check here for the info about DHT support. I'm waiting for it as well. I still use uT, the only time I come away from it now is to use the Comet for the DHT. UDP and DHT and this client is ready to roll.

[Burstin']

Can comeone explain the advantages of UDP and DHT to me? I have an idea about DHT but not sure.

[paritybit]

I will be switching if they get DHT support. Just out of curiousity, is it just conincidence that uTorrent looks like G3?

[paritybit]

Can comeone explain the advantages of UDP and DHT to me? I have an idea about DHT but not sure.

Not so sure about UDP just guessing it is a lightweight way to poll the tracker.

DHT is for distributed tracking of torrents so you don't need a tracker. Makes the community decentralised.

[Ultima]

paritybit: Well ludde took ideas he liked from here and there and implemented them in µTorrent, so that's why it might look like another client.

Burstin': DHT allows you to connect to peers even if there is no tracker. You can read up about it here or here.

[Firon]

DHT is for distributed tracking of torrents so you don't need a tracker. Makes the community decentralised.

It's also an alternate way to get a peerlist even when the torrent IS on a tracker, and it can virtually connect peers on the same torrent from different trackers, too

[chaosblade]

UDP trackers have no benefit other then that UDP does not send ack's back like TCP does (ACK = Acknoledgement that a request reached its destination).

Also, UDP isnt very much supported in the BT community, Is not widely used on trackers, And isnt possible to support through all trackers (read: phpweb based trackers).

[paritybit]

Dunno why they wouldn't use UDP. Would make a lot of scense. Using UDP would reduce bandwidth use a lot.

[Ultima]

Heh read the very last part of chaos' post =]

And isnt possible to support through all trackers (read: phpweb based trackers).

[silverfire]

Another problem with UDP-based communications is that data reliability isn't the main focus of the protocol. UDP is geared more towards applications such as streaming video where data speed matters much more than connection reliability. As a result, using UDP will most likely give you a lot more bad data than TCP-based transfers.

[eighto2]

hopefully its a feature with ON/OFF because decentralized tracking freaks me out

[Apothos]

hopefully its a feature with ON/OFF because decentralized tracking freaks me out

Yes, I also would like the 'on/off' switch for it. Mine will always be Off.

[Veneficus]

UDP is used for solving scalability problems with trackers since, as paritybit said, bandwidth use is reduced a lot. No, data reliability is not UDP's main focus, but I'm sure any UDP-based protocol that needs that reliability, such as tracker access, would have reliability checks built into it.

An actual BitTorrent peer-to-peer UDP protocol would be a lot more important and useful IMO - it would allow NAT traversal through such techniques as UDP hole-punching.

EDIT: Why are people afraid of decentralized tracking?! I don't understand that. The only difference is, you're asking peers about the locations of other peers, instead of asking trackers...

[paritybit]

UDP for tracker communications makes sense. You send a request, tracker sends reply. You have a couple of timeouts and it would work just fine. As for peer to peer communications UDP is great. Seeing P2P normally works by piecing together files UDP would work great because reliable delivery isn't a big issue as the client can re-request stuff and not really lose that much data. Anyway, chances are if a UDP packet is lost you will have the same amount of loss for TCP but not have the overhead.

[Veneficus]

Anyway, chances are if a UDP packet is lost you will have the same amount of loss for TCP but not have the overhead.

I'm not sure that's entirely true. For lossy data such as VoIP, yes. But BitTorrent cannot have losses. If it does, it must re-request the data. TCP does that internally and only lets the "good" data flow to the application. UDP doesn't care either way, so it's up to the application to implement data integrity. I would think that TCP handles that better than UDP with Layer 5-based control.

But, to reiterate my previous post, UDP P2P communication is necessary for NAT traversal. TCP NAT traversal is almost impossible.

[Apothos]

EDIT: Why are people afraid of decentralized tracking?! I don't understand that. The only difference is, you're asking peers about the locations of other peers, instead of asking trackers...

Well, if you are on a private tracker, I 'think' that using DHT will make it less secure, from what I have read, although I am not at all sure if this is correct....? :?:

[Veneficus]

That's very possible. I'm sure it depends on the DHT implementation. From what I understand, there are two right now - the "normal" one and the Azureus one. I don't know the details of how those two DHT systems work, but it should be fairly simple to have DHT automatically disabled for private torrents.

[sethg]

FTP is built on top of UDP, so you can still get reliable data transfer, you just have to do it one level up in the protocol stack.

[Veneficus]

No, FTP is TCP-based. Maybe you're thinking TFTP?

[sethg]

No, FTP is TCP-based. Maybe you're thinking TFTP?

You're right, my bad. TFTP is built on UDP, while full FTP uses TCP. I've always used TFTP or purpose-built protocols based on TFPT for large data transfers due to the lower overhead of UDP. Sorry if I confused anybody besides myself.

[Firon]

Well, if you are on a private tracker, I 'think' that using DHT will make it less secure, from what I have read, although I am not at all sure if this is correct....? :?:

Torrents made for private trackers can be set as "private", which disables DHT support for that torrent in the client, if the client respects the private flag. DHT doesn't make it less 'secure' on private trackers--what it does is that it allows people who aren't registered or leeches or something to bypass the tracker and get a peerlist.

[paritybit]

DHT doesn't make it less 'secure' on private trackers--what it does is that it allows people who aren't registered or leeches or something to bypass the tracker and get a peerlist.

Just wondering:

1) You could make the torrent so that it is private and clients that respect this will follow it.

2) Have an option in the client only to connect to peers provided by the tracker not peers that connect to you willie nillie.

3) In my experience DHT has been useful for finding peers on seeingly dead torrents on public trackers and has helped me a couple of times when the tracker has reported there are no seeds connected.

[sethg]

The down side of trackerless torrents is depriving private trackers of information regarding the distribution of the content through the network they coordinate. Let's say the private tracker is set up by a copyright owner who wants to run the distribution on a pay-per-download basis. One example might be a film company deciding to distribute their content via a paid torrent service as an alternative to selling them at retail. Since a torrent is the ultimate in low-cost distribution, the prices could be extremely reasonable.

However, keeping control over who can participate in the swarm is lost if any single client decides to distribute a peer list through DHT. This possibility would effectively prevent the use of torrents as a low-cost paid distribution medium. This is really a pity, as this technology has the potential to change the way the content market operates into something approaching fairness. When the prices get low enough, more people would be willing to buy the content. The advantage is in buying only the content you want, rather than paying for 100 channels of cable a month, 95 of which you have no interest in. Ultimately, the consumer pays a lot less money, and the content distributors have much lower distribution costs. They may make less money than they currently do, but with prices as they are, a growing number of people have little compunction about making illegal copies, so their current financial model is not sustainable in any case.

If they can't use torrents, they would have to resort to traditional servers, which which are controllable, but far more expensive. This is especially true if large peak loads have to be accomodated, as when a new movie is released.

If anyting is certain in all this, it's that you can't stop technology. Every attempt to do so in the past has failed. The pity is that the torrent has the potential to drop the distribution cost for legal copyright owners to next to nothing and this could have been the vehicle for a major shift in market dynamics. The distribution costs are low enough that it could permit individual artists to publish their work on a mass scale, where that was previously infeasible economically. I'm afraid DHT and other trackerless P2P systems make that unlikely, for better or worse.

The other downside of trackerless torrents is that the tracker can perform the important centralized function of banning "bad actors". Without the tracker providing peer lists, each client would have to do that function on its own. As a centralized repository, trackers are in an ideal position to collect and store information on anti-social clients. Individual clients are at a disadvantage in this regard, though other centralized servers could take over that function, much as the email blacklists (DNSBL's) are used to refuse mail from servers that send a lot of spam.

On the plus side, it has been mentioned before that trackerless systems work when the tracker is down, overloaded or sluggish. Locally obtained peer lists are more likely to be current and sharing is even possible across torrents. If that concept is extended far enough, a trackerless P2P system could act as a giant mesh that could connect clients who want to exchange content, even if they are on disparate networks. This is a pie-in-the-sky view, but trackerless torrents are clearly a very different networking paradigm. It's about as different from a traditional torrent as a torrent is from a traditional FTP server. The real magic is the distributed search functionality, like a highly distributed Gopher service.

The Skype system already approaches this ideal, though there are some supernodes in its distributed directory service. However, most of the directory information is decentralized and millions of clients are still able to find each other within seconds. Substitute a file record ID for a VoIP client name and you have the makings of an almost completely decentralized P2P system where clients can share content without explicitly searching for it. Given a set of hashes from the original content provider, you can always tell if what you receive is the same as the original, regardless of where you get it. This is not limited to movies and TV shows, it is just as applicable to book, magazine and newspaper distribution. This distribution technology could turn today's newspaper into an almost real-time medium, instead of a snapshot of yesterday. Instead of sinking tons of capital into printing presses and delivery trucks, they could spend more money creating content that people wanted to pay for.

[Ultima]

Well thought out post you've got there, Seth, but there's a simple solution that's been mentioned almost *too* many times already. This problem can be solved using the 'private' flag. If a client doesn't respect the flag, ban it.