Debate over Protocol Encryption

[ev0|]

Source : http://bramcohen.livejournal.com/29886.html

Obfuscating BitTorrent

People ask about making an 'obfuscated' version of BitTorrent often enough that it ought to be a FAQ. The reason given is that some ISPs are doing traffic shaping on BitTorrent protocol, and people wish to get around that.

There are several whole categories of reasons for not doing it. First, there's hardly any benefit. Most ISPs don't do such shaping, and attempts at obfuscation won't work for long - the ISP traffic shaping tools are already quite sophisticated, and a wire protocol which transfers a lot of data bidirectionally and consistently looks like line noise with no header is only marginally more difficult to identify then one which uses fixed ports. Obfuscating the protocol doesn't even claim to make it difficult to find out who's downloading a particular file.

Second, if done poorly it has the potential to create an outright incompatibility between obfuscating clients and non-obfuscating clients. This would cause a huge amount of performance problems and general pain, made even more ludicrous by the lack of benefit it brings in the first place. There is in fact a proposal being worked on by some people for an obfuscating protocol which would have exactly this problem. Fortunately it's quite simple to avoid this problem - simply add an extension to the tracker protocol so that a client tells the tracker that it supports obfuscation, and when a tracker gets such a request it returns, in addition to the usual list of peers, a list of peers which support obfuscation. It's very easy for trackers to support such an extension, and it has the benefit of allowing trackers to keep peers from using obfuscation, in case they're interested in making sure the ISP can cache data or just don't want it to be used for some other reason. In general, the tracker should be in control.

That was the main reason I'm writing about this - I rather suspect that some developer has gotten rate limited by his ISP, and is more interested in trying to hack around his ISP's limitations than in the performance of the internet as a whole. Hopefully no such idiocy will take place. Backwards-compatible obfuscation (as opposed to incompatible obfuscation) is still a dubious idea, but at least it isn't a harmful idea.

Back to reasons for not doing obfuscation.

Third, any cacheing which the ISP may do (and yes some ISPs do cache BitTorrent protocol reasonably transparently, much to the benefit of their users) is completely obliterated by obfuscation.

Fourth, when it comes to dealing with ISPs, obfuscation is some combination of hostile, unprofessional, and harmful. Software projects which value quality over featuritis generally steer clear of such things, especially when their potential effectiveness level is the equivalent of spitting in one's face than actual utility.

Oh, and by the way, the amount of CPU necessary to do a diffie-hellman key exchange is enough to be annoying, and if you're making a connection via a trusted intermediary, like, say, a tracker, or already have a reasonably secret piece of shared information like, say, an infohash, there's no need to use a diffie-hellman key exchange to establish a shared secret. Imagining that crypto will stop being done by dilettantes is clearly a pipe dream though.

Basically Bram says it is a bad idea, and gives some very good reasons why. What do you guys think ?

Ludde, I know you were involved in the discussions Bram is referencing, so I'd be interested in your viewpoint as well.

I have a lot of respect for Bram, and I've generally agreed with him in the past, so I'm interested in a developers point of view on this topic, especially if it will lead to an exciting thread.

[Ultima]

Hm... Bram says most ISPs don't shape, and while that's true right now, more and more ISPs are starting to shape (or at least it seems so) as the bandwidth costs rise for them... so if the apparent (to me) trend continues, the first reason will become invalid. But as of now, it's semi-true (though you'll notice that in these forums, a lot of users complain about slow speeds, and it's due to ISPs like frickin Rogers).

The second reason... well it's not done poorly, so it's backwards compatible. Kinda moot.

Third and fourth reasons... well I know nothing about, so assuming he's correct about what he's saying, then yeah, he's got a point =P

[Firon]

Well, we already knew about this, we discussed it on IRC, and various people shot down most of bram's arguments XD

Especially the caching thing, if he wanted to do that, he should've done BT using HTTP!

And the tracker argument is ridiculous, because then it'd be trivial for an ISP to modify or block the tracker requests to remove all the encrypted peers from the list.

[ASDF Commander]

Right now Bram is interested in 'legitimatizing' BitTorrent as a means of distributing legal content and has been given a lot of money to do so. I don't think he wants anything messing that up. He's part of the whole corporate thing now, if he had any links to the community he'd understand that it is a real problem for a lot of people, and it's only going to grow.

His logic might be though, that once big companies start using this protocol, ISPs won't block it for the same reason they don't block FTP/HTTP now - it would anger the majority of their customers.

/snooze, Bram = Old News (thanks for the nifty protocol though)

[Kazuaki Shimazaki]

A Layman's Perception

Obfuscating BitTorrent

People ask about making an 'obfuscated' version of BitTorrent often enough that it ought to be a FAQ. The reason given is that some ISPs are doing traffic shaping on BitTorrent protocol, and people wish to get around that.

First, note Mr. Cohen's tactic of "poisoning the well" by using the word 'obfuscated'.

Oh, and by the way, the amount of CPU necessary to do a diffie-hellman key exchange is enough to be annoying,

Bringing the last paragraph up: This shows he is quite far from reality! What kind of BT clients does he think people run. The amount of system resources required to do even basic bitorrenting on an average computer is enough to be crippling with most clients (not counting uTorrent, most of whose users survived said Crippling Clients). What makes him think that an addition of an "annoying" drag would be enough to put anybody off.

He is quite far from reality in another way too. What makes him think we are unaware that encryption = overhead, and the only question is "how much"? Does he think we want encryption for the fun of it? People went on downloading illegal torrents without encryption, knowing the laws of statistics and probability will protect almost all of them. We want it because it is between that and no BTing.

and if you're making a connection via a trusted intermediary, like, say, a tracker, or already have a reasonably secret piece of shared information like, say, an infohash, there's no need to use a diffie-hellman key exchange to establish a shared secret. Imagining that crypto will stop being done by dilettantes is clearly a pipe dream though.

1) What happens to the DHT and PEX connections, who have no trusted intermediary? OK, sir, I know you didn't like PEX, but you did put DHT into your latest Official Client (albeit as a last ditch alternate rather than the supplement it is with the others).

2) Somehow, I'm not convinced that the infohash would be a "reasonably secret piece" of information - rather, using it as the encryption key will create a nice vulnerability.

First, there's hardly any benefit. Most ISPs don't do such shaping,

Except for the present victims which fortunately I'm not one of. Mr. Cohen, do you want your protocol to be nearly inaccessible to a now measurable percentage of the Internet population?

and attempts at obfuscation won't work for long - the ISP traffic shaping tools are already quite sophisticated, and a wire protocol which transfers a lot of data bidirectionally and consistently looks like line noise with no header is only marginally more difficult to identify then one which uses fixed ports. Obfuscating the protocol doesn't even claim to make it difficult to find out who's downloading a particular file.

Even a poor defense, like BitComet's semifunctional PHE, was good enough to get it through at least some of the ISPs. Sometimes you can get past it by turning on Lazy Bitfield, which AFAIK is not even an encryption. The point is to make it more difficult - we know that if they are desperate they can cut off everything but ports 80, 25 and a couple of others.

Second, if done poorly it has the potential to create an outright incompatibility between obfuscating clients and non-obfuscating clients.

1) The present situation will certainly create an outright incompatibility between certain Internet surfers and BitTorrent.

2) There is a reason why the present PE is being pushed as a common standard.

such a request it returns, in addition to the usual list of peers, a list of peers which support obfuscation.

There is no reason for a second list. It creates a vulnerability, as Firon had already mentioned. He already thinks we can be counterdetected - does he want to make our life worse by creating a clear signature? The clear signature was precisely the kind of thing we were trying to avoid by encrypting! The guy who needs encryption can just attach to each IP and see who happens to support encryption as well.

That was the main reason I'm writing about this - I rather suspect that some developer has gotten rate limited by his ISP, and is more interested in trying to hack around his ISP's limitations than in the performance of the internet as a whole.

Note that despite the altruism involved in merely making your client available to the public (BT clients AFAIK are mostly for-free), he assumes a selfish reason. This argument is an appeal-to-motive, a cheap shot and perhaps even slander.

Oh sheesh, could it just be in response to screams from the victims of Throttling ISPs?

Hopefully no such idiocy will take place. Backwards-compatible obfuscation (as opposed to incompatible obfuscation) is still a dubious idea, but at least it isn't a harmful idea.

Concession accepted...

Third, any cacheing which the ISP may do (and yes some ISPs do cache BitTorrent protocol reasonably transparently, much to the benefit of their users) is completely obliterated by obfuscation.

Most of the users of Encryption will be on ISPs that are throttling them - the chances that such an ISP will cache are roughly zero. The rest of the users will mostly not use Encryption - they can cache off that.

Fourth, when it comes to dealing with ISPs, obfuscation is some combination of hostile, unprofessional, and harmful.

The same could be said of the ISPs that are throttling them, making this a desirable feature in the first place...

[Saribro]

Third, any cacheing which the ISP may do (and yes some ISPs do cache BitTorrent protocol reasonably transparently, much to the benefit of their users) is completely obliterated by obfuscation.

Most of the users of Encryption will be on ISPs that are throttling them - the chances that such an ISP will cache are roughly zero. The rest of the users will mostly not use Encryption - they can cache off that.

What is also funny about that argument, is that he first calls the point of encryption into question because of the "small" number of ISP's that shape BT-traffic, and then uses an even smaller amount of BT-caching ISP's as an argument against encryption. Yeeey hipocrisy .

[ev0|]

Hmm,

I still haven't made my mind up on this yet, but I do see a huge disconnect between Bram and the rest of the (majority of the anyway) community.

I guess it depends on the way you spin it, but you have all made very good points so far, especially Kazuaki Shimazaki.

[boo]

ev0, cohen work with RIAA and MPAA, so ofcourse he would be against encryption.

[bleh]

as far as I'm concerned, after cohen sold out to the organisations, he lost whatever credibility he gained by making this application.

And as far as his arguments go I believe that Kazuaki shot them down fairly well

[hofshi]

Kazuaki Shimazaki: thank you for you post. It was an enjoyable read.

[Falcon4]

Bleh: BitTorrent was never designed for illegal P2P, I'm sure (or at least reasonably certain). I don't really think he lost credibility, but I do think he lost a lot of respect.

I just don't understand why it's not possible for a standard client, or an encryption-enabled client, to auto-detect each other's type? Isn't it possible to send a test message, or place some kind of identification in the client (like an ignored comment or something in some header), that tells each other if they support encryption?

Does there really have to be a complete change in the BT protocol to support this encryption?

And then... now, I don't know exactly how BT works as far as data being sent and received, but am I to understand that all data going through the torrent is being encrypted? Why does the data itself need to be encrypted? Far as an ISP is concerned, I'm sure they think the whole thing is just random junk... if the identifier as a BT packet is encrypted.

*sigh* I'm so confused.

[osm0sis]

Does there really have to be a complete change in the BT protocol to support this encryption?

And then... now, I don't know exactly how BT works as far as data being sent and received, but am I to understand that all data going through the torrent is being encrypted? Why does the data itself need to be encrypted? Far as an ISP is concerned, I'm sure they think the whole thing is just random junk... if the identifier as a BT packet is encrypted.

*sigh* I'm so confused.

For encryption to work, a spec needs to be agreed on. For example if two clients both support encryption but support different kinds/implementations of encryption (as was the problem with closed-source BitComet), they obviously can't work together in an encrypted mode. Everyone agreeing on a spec and implementing it would technically make it part of the protocol i assume. As for header Vs. complete encryption, i think encrypting everything is just safer, especially if what your sending is sensitive material or semi-legal. i'm sure there's another good reason for it as well, i'm just too lazy to read all these threads over again.

hope this helped a bit.

[Falcon4]

A lot - I can't believe I didn't think of that. Guess I just "assumed" everyone would be using the same encryption... (don't even ask me what a "salt" is - I'd say you're talking about dinner.)

Far as data goes... well, if you're transmitting your mail backup or a whole batch of credit card numbers, I guess those would count as legit reasons to use encryption (as far as to push forward the implementation of an encryption protocol). But semi-legal material? Well, looking at it from all angles, it seems that would be a good reason to encrypt the data as well. Despite the pieces coming in a completely random order, I imagine it would be possible to monitor and assemble pieces to know someone's downloading something... they're "not supposed to be". So that's pretty valid too.

Yep, that helped a lot! Thanks

[Firon]

µT only encrypts everything, it does not let you encrypt the handshake only.

The reason is that handshake only is worthless, all an ISP has to do is look for any kind of BT message and then close/throttle the connection. Takes a little longer sure, but still effective.

[osm0sis]

furthermore, it's great that µTorrent and Azureus have agreed on a encryption spec in their newest betas, might nicely steal the thunder away from BitComet for users on throttled ISPs.

edit: Firon, right, that was the one i couldn't remember. thanks

[Falcon4]

Anything to steal thunder from that crap BitComet is an awesome development.

Keep it up!

[Firon]

However, old versions and other clients (read: Az) can still do handshake only, so if you see an "e" in the Flags column, that's the case.

[r00ted]

Right now Bram is interested in 'legitimatizing' BitTorrent as a means of distributing legal content and has been given a lot of money to do so. I don't think he wants anything messing that up. He's part of the whole corporate thing now, if he had any links to the community he'd understand that it is a real problem for a lot of people, and it's only going to grow.

His logic might be though, that once big companies start using this protocol, ISPs won't block it for the same reason they don't block FTP/HTTP now - it would anger the majority of their customers.

/snooze, Bram = Old News (thanks for the nifty protocol though)

Yes, he is part of the "corporate world" now.....but I've heard of these issues from users, who were attempting to grab (download/upload/share/seed/whatever) linux distros via bittorrent.

BitTorrent has it's legitimacies...he/we should really attack/fight the ISPs, and show THEM the light, that BitTorrent is for more than just piracy and such.

[bleh]

Falcon4: what I meant to say, is that I won't trust him anymore, since he's more likely to make his application "kosher" now with the media giants to please them.

Which might make him turn it into a parody of the original idea.

.. come to think of it.. that would be more like a tragedy.

[Falcon4]

That... is for sure. It scares me to think he was "paid" to find ways to make BT "legit". What's he gonna do, put spyware in the official client? People still use that ugly-ass thing, so he could have an edge over the blissfully ignorant by putting that kind of thing in...

He's definately switched sides in most peoples' minds since he got "bought out". Now he's more of the guy to worry about, instead of the guy we used to be rooting for and hanging on every word of. I still remember going to bittorrent.com and seeing that neat little homebaked site about how BT works.

But that's definately no more...

[splintax]

Well, we already knew about this, we discussed it on IRC, and various people shot down most of bram's arguments XD

Heh, damn right we did

Aside from the MPAA alliance issues, I also think that Bram likes to crap on about stuff to make himself feel smart. I seriously think a lot of the stuff he blogs about is written to perpetuate the "oh I have Asperger's, I'm a misunderstood genius" side of things. BitTorrent is really not that revolutionary. It's just that the addition of trackers makes it more efficient than eD2k for most purposes.

[SledgeDG]

BitTorrent has it's legitimacies...he/we should really attack/fight the ISPs, and show THEM the light, that BitTorrent is for more than just piracy and such.

You really think an ISP throttles users because he suspects illegal content??

It's the shere amount of data transmitted they are fighting against.

In other words plain cut down on profits...and in situations like that you can argue with a profit-oriented business guy until you're blue in the face

-DG

[Ares]

This is now being discussed on Slashdot, for anyone who might be interested:

http://yro.slashdot.org/article.pl?sid=06/02/06/2039241

-Ares

[MabeShark]

I'm all about some good encryption, especially increased upload/download speeds, but does anyone know what the difference is with the encryption in BitComet and the upcoming encryption in µTorrent and Azureus? From my understanding, BitComet is closed source, so the encryption in it would only work from BitComet to BitComet users. BitComet just encrypts the BT headers though (I guess that's all you need to encrypt), is that what is µTorrent and Azureus are doing too, encrypting the header?

[hotrice]

Please make encryption an option that is on by default. All of the major Canadian cable operators have throttled Bittorrent traffic.