BitComet cheats, so µTorrent should IGNORE it!

[Firon]

Creating a collision for a single SHA1 would take hundreds of computers on a distributed network.

Sounds like an awful lot of resources for making a collision for a SINGLE hash (nevermind the fact that a torrent has one for every piece). I don't think it's really feasible.

[foxyshadis]

Still, the chances that movie studios are going to rent supercomputers for a few months or years to poison a single torrent is pretty ludicrous. That's the level of difficulty it would take to find a matching hash with the constraint of being the exact same large size. They'd be much better off seeding bad torrents and flooding torrents with leechers.

TTH, even with a hash as simple as CRC32, would be somewhat better than a longer SHA hash, since every piece would have to pass multiple checks. But it's too remote a possibility to matter in the near future.

[edit] Natch, firon's too fast for me

[Shernand]

From Emule Plus Docs :

Enable countermeasures against unfair clients

Many clients have some unfair options that harm legit eMule users (i.e. less slots available, no upload, hash stealing and more). By enabling this option eMule Plus will try to overcome this wrong behaviour. Leecher MODs are banned after detection and a message will be kept in the debug log if 'Send message to banned users' option is disabled.

Since there's a similar story with bitcomet i'd be great to see some countermeasures implemented in utorrent, optional, of course.

Personal opinion : if it was me i'd have an option to connect to bitcomet clients ONLY if they're the only ones avaiable to connect, for example, let's say that you have Max number of connected peers per torrent set to 50, that ONLY if i can't reach those 50 with FAIR clients, then connect to bitcomet clients. Personally i think that won't hurt GOOD clients and will give bitcomet a run for its money. Just my 2 cents

[Switeck]

Still, the chances that movie studios are going to rent supercomputers for a few months or years to poison a single torrent is pretty ludicrous. That's the level of difficulty it would take to find a matching hash with the constraint of being the exact same large size. They'd be much better off seeding bad torrents and flooding torrents with leechers.

Actually, they have a 3rd option.

They have either specially designed programs that act like BitTorrent clients or a special packet filter program that allows them to connect to regular torrents and spam bad file pieces to everyone. They wouldn't even need to do it at high speeds -- just 1 bad byte to every chunk out there can force a redownload of the whole chunk...which can take awhile if the chunk's 1 MB or larger.

Personal opinion : if it was me i'd have an option to connect to bitcomet clients ONLY if they're the only ones avaiable to connect, for example, let's say that you have Max number of connected peers per torrent set to 50, that ONLY if i can't reach those 50 with FAIR clients, then connect to bitcomet clients. Personally i think that won't hurt GOOD clients and will give bitcomet a run for its money.

That's certainly another possibility asides from the throttle, ignore, or ban ideas posted so far. I'd say it falls between ignore and ban ideas in severity.

Another idea mentioned was when BitTorrent leech clients disconnect/reconnect to force uploads from other peers/seeds is to simply treat them like they never disconnected at all. That doesn't punish them for hostile behavior, but it would still require the same kind of programming as my ignore idea.

[AshenSugar]

ok didnt read whole thredas it is quite long, first off blocking bitcomment alone to me is a waist of time.

allowing users to set clients and even peers into a ban list is GOOD, I use this feture in another torrent client to remove people who are cheating. Its all manual.

Sure it may not seem fair but its a feture that i personaly prefer.

Also i got NO problem with the dht thing, i got another client(not commet) that can ignore privat tages if you want it to, and when trackers go down I resort to using that client, sure it dosnt report the ratio but when im doing a 4gb download and im stuck at 75-99% i sure as hell dont want to be sitting around for days waiting for a tracker to come back up.

i dissagree with sixshots quote on the first page(last post) i feel that blocking truely bad clients is GOOD for the health of bittorrent as a whole, sure it may suck for somebody who is to stoborn to try another client but it keeps things FLOWING insted of letting them stagnate, also keeps people from cheating and downloading without uploading (those ppl really erk me)

honestly i dont want any default client/peer banning but I would LOVE the option to block slectivly via right click in peer list, this is a VERY handy feture.

well thats just my oppenion. dont force ppl to use utorrent but allow people to block cheaters and flacky clients, its alwase possable to remove a manual ban if a client that somebody banned gets fixed

[Herra]

Whether or not all of this holds ground is anyones guess, but it's always amusing to read arguments that use CAPPED words to make a point of any kind.

It always reminds me of how people have the habit of raising their voice when talking to foreigners who don't understand the language at all. Because talking louder really helps in that case. REALLY.

[splintax]

Maybe I should rephrase what I said. Fixing these problems is, as others have said, the responsibility of maintainers of the BT protocol. This includes tracker admins, as they try to stop it from being abused by doing things such as banning BC. I stand by what I said with regard to µTorrent treating all torrent clients equally. If it's just µTorrent doing it, first, it will do almost nothing to stop BitComet's effect, and second, it makes µTorrent an "unfair" client.

As for hash cracking.. look, there are checksums in the TCP segment (based on one's complement, IIRC), usually checksums on layer 2 and there's checksums for every piece with SHA-1. SHA-1 is still more than secure enough for BitTorrent. Maybe if you're making hashes for highly sensitive passwords or something, the move to SHA-256 is justified, but otherwise, no. Poisoning with bad data only happens when you have a shitty hashing algo, like Kazaa's, which only hashes 256 KB of data every few megabytes, IIRC..

[Switeck]

honestly i dont want any default client/peer banning but I would LOVE the option to block slectivly via right click in peer list, this is a VERY handy fe[a]ture.

You can already ban by ip in µTorrent -- just add the ip to the ipfilter.dat file. That can even be done without restarting µTorrent or any torrents running.

Maybe I should rephrase what I said. Fixing these problems is, as others have said, the responsibility of maintainers of the BT protocol.

Concerning the BitComet cheating method I've mentioned, there is no problem with the BitTorrent protocol, it was actually Azureus and µTorrent (at least used to) that was disobeying the letter and spirit of the BitTorrent protocol. BitComet was just exploiting that fact.

[AllWeasel]

Hi there. This will be my first post, so I'll get the preliminary stuff out of the way. I've spent many hours reading messages on this forum, but I'm sure there are plenty more that I haven't read yet. Assume that I would have read something if I was aware of it, and respond in an appropriate, warm manner based on that assumption.

I write software for a living, very complex software. I've been doing it for well over 20 years. Please don't assume that *I* don't understand something simply if/when I do a poor job of expressing my point. Rather, assume that I understand the problem - I'm just not making *my* point very clearly to you.

I'd first like to say thank you to the developers of utorrent (yeah I know the FAQ tells me how to make the micro symbol, but it's just too silly a thing to bother looking it up). I've tried many BT clients and utorrent is my new favorite - but not because it's small. It's because the developers have a clear design model and an elegant implementation of that model. I admire elegance in code and I tip my hat to those that wrote utorrent. The size (or lack thereof) is simply a reflection of that design and discipline. Nice job guys.

Ok, so much for the intro. Hi everyone - "top of the day" to you all.

I gathered from a few comments on the board here that the developers are opposed to allowing users to hack-up the clean behavior of the code and/or apply arbitrary human prejudice to banning other clients. I absolutely agree in principle. However, a good BT client should not require any hacking to behave properly and provide a dis-incentive for cheating clients. The problem is that many (most) of the people advocating police action are looking at the problem completely wrong.

Switeck got it right (IMO) when he said that the problem was not BitComet. The problem was a broken seeder that placed three times the normal emphasis on a fresh connection. That behavior is broken and contrary to the white papers that describe the *desired* BT choking behavior. The correct response to the "client abuses" is to make sure that our favorite client (utorrent) is indifferent to those "loop holes", "exploits" (or whatever you prefer to call them).

I honestly can't imagine how someone could implement an algorithm that would allow a just-disconnected peer to get back into the small pool of connected peers anytime soon. They must actually attempt to reconnect dropped peers intentionally, or it just makes no sense.

I've see a dozen different implementations of BT and I've never seen one that didn't hit your peer with a serious penalty if you drop and try to reconnect again. Any BT client that opens a connection simply because your peer asks, is just stupid. Typically, they either queue the request for later or they just deny / immediately close the connection. They do this until a bunch of the current peer connections have expired and the pool of peers reaches the lower threshold whereupon they initiate a bunch of connections themselves or answer the pile of pending connection requests.

The only case where peers should be able to reconnect anytime soon, would be an insanely small swarm that isn't big enough to exceed the max threshold of desired peer connections. In that situation, you have no choice but to let them participate in the swarm anyway. Again, any form of "banning" BitComet peers isn't going to help the problem with a small swarm. In fact, it would cause some real problems with tiny swarms.

The BT client Rufus has a really nice way of dealing with the BitComent class of problem and it's exactly the spirit expressed in the BT white papers. Rufus allows for tunable choker parameters. The most effective (and least unfair) choker is the "Leoxv" (named after the person that conceived the idea) choker.

What the Leoxv choker does (in spirit) is to accumulate statistics on the peer's "tat" rate. IOW, how much bandwidth is that peer willing to give me back? If it falls below a user-defined threshold, the choker algorithm will *tend* to drop that peer from the pool of desirable peers, but it doesn't just dump them willy nilly. Peers that consistently send more data will in turn be allowed to continue unchoked *and* they get data sent back to them relative to their contribution compared to the other peers in "the short list". This is a layperson's explanation; the algorithm is more complex than that, but I haven't found enough documentation on it to fully understand it's finer points of operation. I'll try to keep my comments focused on how I know it to behave from my own experience.

The end effect is that any peers or clients that "play tricks" can not do anything artificial that will actually improve the KB/s that our choker is looking for. Peers that maintain good KB/s ratios, will become "preferred" and peers that have the lowest KB/s ratios will eventually fall out of the pool of viable peers and get disconeected in favor of a new (optimistically) better producing peer. In practice, the algorithm is quite subtle and very interesting to watch. Over time, (even with fewer connected peers) the download rate climbs steadily and doesn't jump all over the place. It produces noticibly higher DL rate with great consistency and requires no babysitting as long as the threshold values are in the right ballpark. Better than that, our upstream data is being sent to the cream of the anti-leechers which makes me feel all warm and fuzzy.

The Leoxv choker uncovers the most stable peers and the best producers for us. This promotion of good producers indirectly includes whether or not the peer has lots of pieces we want and of course whether the Internet connection between us is any good etc.. It's not about who has more raw bandwidth, it's about who can deliver us the most bytes of what we want in the least amount of time - over a long period of time. We reward that set of circumstances by responding in kind with lots of data and few interruptions.

We don't care what client they're using. If they deliver the goods, then they stay connected and we can bounce data back and forth until one of us is displeased with the arrangement, or runs out of things to send the other peer.

I must warn you that the Leoxv choker is very unfriendly to your own computer when you're not putting out per-peer as much as you claim to expect from each peer in return. If you set the minimum KB/s value to less than you're sending out, then you'll indeed locate the "big givers" out there, but those same peers will ignore you because they have a dozen other peers in their pool that *are* willing to give back the volume that they are taking. The spirit of tit-for-tat (and/or smart choker algorithms) will insure that you get starved from the "big boys" in short order. Since your own Leoxv choker is effectively disconnecting all the "small fries" from your peer pool, you'll get starved-out and have no peer activity in a very short period of time.

What the Leoxv choker does (in practice) is to quickly weed out the clients that send 0.8 KB/s and waste both your peer-pool space and your precious time negotiating for that wimpy datarate. In essence, Leoxv probably (I'm a little foggy on the exact details ...) looks at their u/d ratio, their upload byte count, and also the actual rate of data that they are sending to us, and it rewards the best producers with constant data packets from us. What it does not attempt to do is to punish anyone or punish any specific clients. All that matters is whether the peer delivers the bacon or not. This philosphy is exactly what the BT white papers advocate as a good choker policy.

Whether utorrent implements Leoxv exactly or simply rolls a leecher-unfriendly choker of their own is not important. What is important is that the choker reacts positively to get us ever-closer to our goal instead of trying to punish someone or some thing for a perceived evil deed. The leechers or leech-minded clients will quite naturally become insignificant because we've got our attention on the peers that are the most symbiotic and compatible with our goals. Everything else is just noise and gets cast aside. When BitComet becomes the world's worst BT client, in terms of datarate, then it will take it's natural Darwinian path of getting improved or simply dying a slow death as smarter clients pass more data to other smarter clients.

The second very important part of the Leoxv choker and something the utorrent developers are not keen on - is that the "minimum flow threshold" must (by nature of the beast) be user-configurable to work well in various situations.

Currently, I have 1Mb/s upstream bandwidth and I tend to run no more than 2 torrents at a time. Assuming that one of them is a download, I'll tend to configure the download as "high" priority and assume that the majority of the upstream bandwidth is available for it (generally speaking). I'll let the BT client work out the details of the bandwidth distribution.

In the Rufus client, the configuration of upload slots is simpler because you can tell it what global per-peer average KB/s rate you'd like to have and it will work out the number of peers needed in the pool to achieve that average rate at any given moment. This makes it quite simple to use a slightly lower KB/s value for the Leoxv threshold. If we can't find any big producers, then Rufus will just add more peers to the unchoked pool until the average rate works out or we're at the max upload KB/s rate. Whatever they can give back is what we live with. Of course, if we find one or more big producers, then we begin a data swapping orgy at whatever speed they can muster. In this case, we have a very few high-flow peers which is a lot more efficient as well as being fast. It's a win-win deal because the other peers are getting lots of data from us.

However, if I've got four seeds going, and I've set the Leoxv to 20 KB/s per-peer, for example, then I'm not going to be able to keep up with that rate myself. Therefore, I must drop the Leoxv threshold down to a range that I can sustain *while* I'm also seeding the other torrents. It obviously gets more complicated to choose the optimum threshold and the tweakers out there will have lots of fun with it, but the bottom line is that any good choker that produces anti-leecher behavior will have to allow for user tuning - for a number of very good reasons.

I think if the utorrent developers can get past the stigma of allowing the user to tweak the choker parameters, the solution to BitComet (or any other leecher-friendly behavior) is really quite simple and elegant. The choker (IMO) is the correct place to put code that chooses which clients get a higher priority than others.

P.S.

I'll try to submit the "Leoxv choker as an enhancement" request later, but I want to bundle it with the notion of named packages (or profiles) of pre-configured parameters for operating in different conditions. This would help prevent a lot of the creeping-featurism that is starting to occur here. I want to give it a bit more thought before proposing it.

[Switeck]

The problem was a broken seeder that placed three times the normal emphasis on a fresh connection. That behavior is broken and contrary to the white papers that describe the *desired* BT choking behavior. The correct response to the "client abuses" is to make sure that our favorite client (utorrent) is indifferent to those "loop holes", "exploits" (or whatever you prefer to call them).

A distinction must be made here, as your description is slightly vague. It is explicitly and implicitly in the BitTorrent protocol for optimistic unchoke to give 3 times the normal emphasis to NEW connections that presumeably have nothing to share. This might also include connections that have something to share but everyone already has those parts. So it is not breaking BT protocol to give the extra emphasis to ANY new ip regardless of whether or not it has anything to share.

But that does not include giving 3 times the normal emphasis to quickly-reconnecting peers that are known to have something to share. It is only these that might be cheating. Or their connections might be 'flakey'. A simple snub could sort out the cheaters from the noncheaters, as the noncheaters would upload to you if their optimistic unchoke is working ok. Trying to upload to a constantly disconnecting peer isn't very productive...even a short snub of 1-5 minutes would be enough to tell if the connection is stable or not, while an automatic reconnecter would be having fits under that snub. The key of my idea is to make the snub SHORT and conditional. ONLY reconnections should be affected by it.

µTorrent already maintains a pool of ips for torrents and doesn't connect to every one of them by default if there's more than a handful. µTorrent would need to store info on each ip as though it were connected -- then and only then is it possible to tell if an ip is a new peer or a reconnecter.

I honestly can't imagine how someone could implement an algorithm that would allow a just-disconnected peer to get back into the small pool of connected peers anytime soon. They must actually attempt to reconnect dropped peers intentionally, or it just makes no sense.

Reconnecting should be a good thing -- there's various reasons why a connection may 'break' for a second but still be good for hours straight after that. µTorrent already has a terrible problem with maintaining reasonable numbers of connections at least in the last 'stable' build (v1.3).

Even reconnecters should be rewarded with some of your upload bandwidth *IF* they are uploading to you at a reasonable speed. Banning them is not the solution, especially with BitComet having such a huge percentage base of all BitTorrent users.

[Firon]

µTorrent already remembers download and uploaded data from peers, even if they reconnect. In fact, it remembers it throughout the entire session (even if you stop and start the torrent), I believe... So implementing something like this would probably be not that hard.

[AllWeasel]

It is explicitly and implicitly in the BitTorrent protocol for optimistic unchoke to give 3 times the normal emphasis to NEW connections that presumeably have nothing to share.

I've not found anything on the bittorrent.com web site that mentions this. Can you point me at whatever protocol description you're using? I don't mean this antagonistically, I'm seriously interested in reading it. I can't see any point in commenting much further until I know what you (already?) know on the subject.

What I meant by peer pool was not the leacher list of the swarm, I meant the peers that are currently connected. For example, a torrent swarm has 3305 leachers and 381 seeds. A BT client has 100 peers currently connected and 10 upload slots. Do you really think that allowing the next leecher (that attempts to connect) to not only connect, but actually unchoke them in the process - is a good idea? Logic would tell us to add them into the pool of peers that we *could* connect to and process them in turn - NOT unusually out-of-turn.

For the record, if we have 3000+ peers to choose from, then an already-disconected-once peer has NOT got my interest. Maybe after I look at the other 2999 peers that don't have re-connection issues, I might reconsider your connection request. Until I look at the other peers, I really don't want to settle for sub-optimum tit-for-tat candidates.

In my universe, I have 1 MiB upload to offer, so I bloody well expect some serious flow from *every* peer thats in the pool. In some respects, the Leoxv approach would be like changing utorrent's "disconnect when inactive for 5 min" algorithm to "disconnect the peer if average incoming KB/s is less than 2.5 over the last 5 min" instead. Of course it's more intelligent than that, but that "drop-the-droolers" part is the part that dumps poor-flow peers and keeps the pool filled with relatively higher flowing peers. All other aspects of tit-for-tat and optimistic unchoke are retained.

I'm not suggesting that utorrent adopt this as the sole choker, I'm hoping the developers will offer this as an optional choker that can be used to skip past leechers and automatically weed out leecher clients (or leecher-tuned clients) naturally, without "banning" anything or "hurting" anyone. There is nothing unfair about me not wanting to talk to peers that can not produce the kind of flow that I'm willing to give them in return. In fact, discarding them and moving the data onward to several other peers gives them a better chance to get the data faster and expands the seed pool in less calendar time.

If I had a 128 Kib upstream, then I'd set the Leoxv threshold much lower because I *could* be productive to the swarm at those lower flows. Trying to keep 130* upload slots open because the "dribblers" can't accept more than 2 Kb/s of my upload BW is terribly inefficient. I want to *choose* to move on to other peers instead of choking them and waiting 5 minutes for them to drop.

*the 130 was an editorial exageration, I usually run 30-40 upload slots when I want to extort the leechers

[Switeck]

It is explicitly and implicitly in the BitTorrent protocol for optimistic unchoke to give 3 times the normal emphasis to NEW connections that presumeably have nothing to share.

I've not found anything on the bittorrent.com web site that mentions this. Can you point me at whatever protocol description you're using?

It's here:

http://www.bittorrent.com/protocol.html

In the very last paragraph, the very last sentance:

"To give them a decent chance of getting a complete piece to upload, new connections are three times as likely to start as the current optimistic unchoke as anywhere else in the rotation."

(From Gulliver's Travels, the argument now is BIG Endian versis little endian...what the definition of "new connections" is.)

One could argue that "new connections" means new to the torrent tracker, new to the individual peers, or already seen by the peers but reconnecting.

I think even though an ip may have been added to your peer list hours ago, the first time your client connects to it...that ip qualifies as a "new connection" regardless of whether it already has something to share with others. A peer may have alot of the torrent, but if everyone else has those parts already then essentially they have nothing to share. This will hopefully allow your connection to quickly test whether new ips (to you) will return tit-for-tat or be a greedy leech. It is in a sense a selfish act because the upload to the new ips will only continue past 30 seconds (end of the optimistic unchoke) if the new ip uploads back to you.

Also debateable is the meaning of the 2 "as" in the phrase: "new connections are three times as likely to start as the current optimistic unchoke". Does that phrase mean "new connections are 3x MORE likely to BECOME the current optimistic unchoke" or "uploads start to new connections 3x MORE likely THAN the current optimistic unchoke"? (Man eating shark I say.) The first means new connections are optimistically unchoked very often. The second means new connections are automatically uploaded to separately from the standard optimistic unchoke.

The first interpretation seems the most likely and makes the most sense. Part of the BitTorrent's principles is to keep the protocol simple. (I don't really know where that's stated, that's just a feeling I get from reading about it.)

Having 2 types of random uploading that don't follow tit-for-tat guidelines is not simple. The more uploads are occuring at once, the longer it takes for a complete torrent chunk to be sent to any given peer so they can share it. Thus, multiple random uploads (which don't count against the standard max upload slot limit) instead of the single optimistic unchoke (which counts against the standard max upload slot limit) is contrary to that important goal.

[AllWeasel]

Ah, ok I see where you're getting the "protocol specification" from. It is unfortunate that BramCohen chose to name both the protocol and the implemented client code the same name. Notice the disclaimer before the section that you are using as a "protocol spec".

The choking algorithm described below is the currently deployed one. It is very important that all new algorithms work well both in a network consisting entirely of themselves and in a network consisting mostly of this one.

With all due respect to Bram, the code he is refering to is more than 3 years old and it was terribly naive to begin with. He did virtually no research as to the consequences of multi-peer focussed attacks on the protocol - for example. He provided no means for peers to ack as trackers aka; DHT, which should have been integrated into the protocol so that data rate stats could be shared and verified to root out lying cheating client implementations. Yada yada yada.

My intial point is that you're using his comments on the then-current implementation of some python code as the protocol template for what constitutes legal behavior in other clients. Read the paragraph above and tell me where it says I can't do whatever floats my boat as long as it plays well with itself and his client?

I still assert that stacking the deck for new peers is not necessary and it *is* provably unhealthy for the swarm when BitComet is around. By the definition above, his implementation fails to play fairly due to over-stacking the deck for broken peer behavior. I must say however, that I am probably not fully qualified to judge the statistical merit of the stacked deck. I do understand what he's attempting to solve, I just don't think his implementation is the correct solution.

On the other hand, lets investigate the reality of a modern BT client. I regularly see swarms of several thousand peers. No modern client would attempt to connect to every client in the swarm, so they provide a setting for the maximum *connected* peers allowed in the choker pool at any given moment. From that choker pool will be chosen the X number of upload slots.

Imagine a current pool of 50 connected peers from a swarm of 5000 peers. If any of the 50 peers disconnects and tries to reconnect, they should (if the programmer has any brains) be put at the 5001th queue position to get into the pool of 50. Again, if the programmer has any brains, the peer will then be disconnected so that the local TCP stack and the NAT router won't have to keep a TCP socket pair open for a gazillion hours (for absolutely no reason). Otherwise, the client opens themselves up to a DDOS attack and NAT owners curse you and windows computers crash the TCP stack and bluescreen and life just suxxors.

So, BitComet doing the disconnect/reconnect thing will have no positive (or cheating) effect with large swarms. Getting a small optimistic squirt after waiting for 4999 other peers to get in and finished is not exactly a boon to data movement. Therefore, the only case when the disconnect/reconnect behavior might stack the deck (in a sane client) is when the peer pool *is* the swarm. Thus all "new" peers are connected immediately. In this very limited condition lies the exploitaable hole in Bram's broken code.

Because I've been a software engineer for a couple of decades, I'm going to make an educated guess that his code doesn't just answer/connect peers asynchronously. I'd bet that he would check the pool size during the choker pass and either initiate or answer sockets at the same place in the cycle that the choker code is going to decide who gets the next round of unchokes.

So, he probably gives the new entry into the pool (aka; the new connection) a 50/50 chance at each of the "4" upload slots (minus the one being lotto'd as the optimistic one) which is 3 slots @ 50% chance, giving the 3 times more likely verbiage [i'm guessing here]. That would be a reasonable explanation and the code would be easy to implement.

Ok, now fast-forward to modern clients that have a hell of a lot more upload slots. The algorithm I described would not scale at all, so you'd have to just fudge the statistical probability for the "new" peer. But why? Just to give a stranger a better shot at leeching from us? Personally, this is where I think Bram got off to a bad paradigm. He is hell-bent on pre-loading the other guy with some data, but he goes through some real mental gynastics to avoid using a block-based TFT. Instead, he kust tosses some unspecified KB/s at a freshly connecting peer on the off chance that it will return something.

To be actually-fair instead of dialup-charitable, we ought to toss them a specific number of blocks and then see how many blocks we get back. But all this is just goofy because I'm reasonably sure that we already know whether that peer has anything we want or not before we even send a byte to him.

From a byte-based TFT point of view, this makes zero sense. However, from a be-charitable-to-dialups point of view, this quasi-rate-based TFT behavior (and a couple of others) all tend to squirt a bit of un-deserved data towards the low-bandwidth users. I'm sure that in Bram's mind, his ideal of "fair" was to let the speedy cable guys subsidize a little love for the dialups to help them get more reasonable D/L times and the overall D/L time would be more "fair" in the grand scheme of everyone gets the file. Unfortunately, just capping his upload or using an obscene number of upload slots will allow a broadband client to appear as a client in need of this charity.

Anyway, getting back to the goofy part... why would any sane client that already knows an incoming peer has nothing it wants, want to even connect to that peer? If you *must* connect to find out what it has to offer (this may be necessary if the tracker data is stale), then why continue at all if it can't TFT for you? Again Bram's subsidizer mindset colors his python implementation - I think. In a swarm that is greater than my allowed peer pool, why connect to a peer that I know will get snubbed and disconnected in 5 minutes? It makes no sense at all, let alone connect to him and then give him a triple shot at being the optimistic unchoke. Charity for low bandwidth users is the only answer that I can think of.

Regardless, the modern client allows many more than the naive 4 upload slots of the initial BT client (for good reason). I assert that a modern client can fully respect the spirit of BT (the protocol) and optionally, also fully support Bram's concept of subsidizing the needy while also not providing positive incentive for clients that screw the swarm's health.

For example, we could provide one dedicated upload slot (of marginal BW) to be used as the charity pipe. Any clients that don't qualify for TFT can queue for the charity pipe (just before we disconnect the little leeching buggers <evil grin>). Or perhaps the anti-leecher choker puts dribblers into the charity pool on their way out the door? That way, the peer pool stays loaded with peers that I actually have a good chance of TFT'ing with. A side-effect of not needing to jack the odds of optimistic unchoke for new arrivals is that it's easier to answer/open a block of peers in one quick shot to keep the peer pool full. I'm amazed at how poorly utorrent manages to keep the peer pool near the max level, BTW. It seems to me that with a properly-modest peer pool, it would take only a few choker cycles to figure out that a peer is inactive or we've been snubbed. I think 300 seconds is at least 150 seconds too long to wait - for my broadband connection. A dialup user might well be more forgiving, but if I have 3000+ peers to choose from and you sit on your ass for 3 minutes, I'm inclined to cut you and move on to the other 2999 peers that might just be able to TFT with me.

Oh and before anyone points out that NOT feeding peers with no possibility of TFT would lead to an empty client starving to death, I'd like to suggest that again thinking outside the box and providing the "charity pipe" for that purpose would actually make the code and the data flow much simpler. You see, I believe that seeding and TFT'ing are two wholy different things and they require completely different algorithms to be efficient at doing either of them. Having a dedicated pool of pure-leechers would allow a code thread to apply pure-seeder logic to determine who to unchoke. This would (I believe) result in a better allocation of non-TFT data flow and it would also allow the TFT algorithm to concentrate only on peers that TFT - without tripping over goofy special cases for newly connecting peers and other things like that.

In any case, those ideas are off the top of my head as I write this, so they will clearly have flaws. But the point is, that a BT client can implement a (radically?) different peerpicker and/or choker algorithm and *still* fully support the spirit of the BT protocol. However, we have to be careful not to confuse Bram's implementation, Bram's notion of "fair" and the actual protocol definition.

Granted, I understand that there are subtle reasons for Bram's code that aren't bodly mentioned in the protocol explanation, but that's the part I call the BT "spirit" because the BT protocol operates on those assumptions even if it doesn't rely on the actual quirky implementation of the python code. For example, pre-giving or optimistic uploading is crucial to the way BT operates. OTOH, I think Bram's charity to dialup users is socially necessary, and provides a benificial marketing kind of attraction to keep BT highly popular, but the BT protocol doesn't *need* the charity factor to keep the swarms healthy.

I'm wondering how cool a feature it would be to have a charity KB/s setting in utorrent? I know that I'd allocate 10 or 20 KB/s to boost the poor dialup guys, if I *knew* that the damn broadband BitComet-esque clients couldn't cheat and suck it up.

Also debateable is the meaning of the 2 "as" in the phrase: "new connections are three times as likely to start as the current optimistic unchoke".

I read that as ... "new connections are three times as likely to be selected as the *next* optimistic unchoke - at the moment they connect." and I described my guess as to why it was implemented that way above.

This will hopefully allow your connection to quickly test whether new ips (to you) will return tit-for-tat or be a greedy leech.

Unfortunately, the slow leechy clients will take the data slowly and then leave you hanging waiting to some data back until you finally snub them or disconnect them. The whole idea of new peers getting bonus shots at the optimitic slot is broken IMO as I also mentioned above.

Finally, trying to pull this thing back towards the topic it's posted in... like the LeoXV choker concept, I believe that a developer can think outside the box of Bram's sample client (which has always been a toy) and come up with a method of negative results for leechers and positive results for "good neighbors" while not allowing a broken client to cheat or extort the swarm for self-interest. I don't think any focussed banning behavior is needed to accomplish this. Not for BitComet and not for any other client.

Oh BTW, something I forgot to mention about the LeoXV choker in Rufus was that it has a setting to NOT request blocks from peers that have a share ratio higher than X to 1. This is the part of LeoXV that everyone forgets to mention. G3Torrent already had an adjustable choker, but LeoXV extended that idea to prohibit your own client from sucking data off of seeders that have already given a lot back to the swarm. I like that idea a lot. I like to know my client is always being a good swarm citizen.

Ack I have to sleep a bit before I get up again. Later.

P.S. Please excuse the typos and such. A few years ago, I broke my back and had some spinal damage that occasionally causes my fingers to mis-hit the keyboard when I'm not watching them. Any errors in logic or reason ... well I have no excuse for those, I'm just getting old.

[DrS]

- Fix: Better detection if BitComet clients threw away my requests.

That's something, but not nearly enough to have a substantial effect. I haven't notice a difference, nothing at all.

Anyone who still isn't convinced BitComet and other cheaters are a significant problem should just look at the peer details in their client. 9 out of 10 you've given far more to a BitComet peer than received in return, no matter how far your or their downloads are progressed. At least, that's my experience. And that's not because they can't match my upload: I've got an average cable connection. I'm going to re-install Azureus (and stuffer) to do the seeding with. With µTorrent I have to seed well over 300% to get a single copy out, ridiculous.

Many seem to be of the opion it's completely up to the maintainers of the protocol -I guess they're waiting for Bram to write BitTorrent 2 or something- and tracker admins to address. I think that's rather naive. A client should do what's best for its user. In that sense, BitComet is a really good client; it's able to give its user high download speeds at a relative low input of bandwidth. That's why it's so popular. But a good client should also do what's best for the swarm: efficient use of the user's bandwidth. At the moment µTorrent does neither.

I think many would appreciate a comment by Ludde on this, I would. At least we would know his opinion about all this. I'd like to know if he acknowledges there is a serious problem to begin with, I doubt he does. If he does, what he'll do about it.

I really like Switeck's and AllWeasel's ideas on the matter. Their solution(s) is(/are) much more elegant than the stuffer like feature I requested. (I still like that to be implemented though as the current µTorrent is fair game to cheaters.)

Switeck, AllWeasel, keep it up.

[Martin Levac]

AllWeasel, what mechanism does the LeoXV choker have to determine share ratio of any given peer to not request data from this peer?

Does the client have a method to send share ratio info about itself to other clients?

Do other clients have a method to accept and use this info?

Do other clients have a method to send this info about themselves to the client?

Other than those who use the LeoXV choker.

Does the LeoXV choker have a method to bypass this share ratio/do not request mechanism if the client that it queries about it wants to keep seeding indefinately or until a higher ratio is reached? Some of us want to seed until a local share ratio is reached and it may be higher than the one the other client has set to not do requests on us.

ML

[Nefarious]

huh, i've been seeing a lot of 0.60 bitcomet users but none 0.61 they love the way 0.60 cheats and therefore not gonna upgrade? tahts just low

i still say a re-estructuration of the .torrent files making the torrent block clients instead of the tracker is better, places like isohunt.com and torrentspy.com that adds another trackers to the .torrent files is making the block in trackers useless...

[Kazuaki Shimazaki]

huh, i've been seeing a lot of 0.60 bitcomet users but none 0.61 they love the way 0.60 cheats and therefore not gonna upgrade? tahts just low

Let's be fair. Give them some time. For one thing, many users are laymen that may not even know the issues. Another factor is that BitComet is getting a bad name, so other sites may not be so hot on telling you there is an update (used to be BC is listed on AnimeSuki as one of the clients, now it is not). A third factor is that BitComet does not update every day like uTorrent. We are all sitting here for the next gift from Ludde. We probably won't be this dedicated if uTorrent updates every six months.

[moose999]

The private UKNova tracker blocks BitComet, and more power to them!!

[AllWeasel]

AllWeasel, what mechanism does the LeoXV choker have to determine share ratio of any given peer to not request data from this peer? [...]

I really don't know any particulars as I haven't downloaded the source code. Frankly, I'm not a fan of the python language and reverse engineering a BT client doesn't sound like an exciting way to spend my weekends. However, I did try to find everything I could that describes the design or the empirical operation of it.

Reverse engineering the Rufus client is not necessary nor desirable for many reasons, not the least of which is stealing someone's intellectual property without so much as dinner or a kiss.

Actually, the G3torrent client originated the idea of the tunable choker. Rufus split the source code tree and continued the development while G3torrent languished. Later Rufus' author collaborated with LeoXV and added the "throttle-thyself" aspects. LeoXV added the non-leecher and self-choking concepts to make sure that *my* client is a healthy swarm member.

I don't have any details on where the numbers come from, but unlike trackers where you live and die on ratios (prompting the desire for clients to lie), LeoXV doesn't live or die on ratios. Whatever source of stats is currently being used in the GUI is probably close enough to provide a fair yardstick for a threshold-based choker.

LeoXV just "prefers" higher bandwidth clients by not uploading to the low bandwidth dribblers. Optimistic unchoke still sends them as much as they can handle (or I can give), but if they fail to send a high volume response next time, then my client will download, but not upload to them. I'm sure that after a few non-responsive optimistic unchokings, the LeoXV choker just stops uploading to them entirely (a snub).

Again, the LeoXV design also makes my client "prefer" not to abuse seeds that have already contributed many times their share when other seeds are available. Instead, my client would prefer to pull more from other seeds or connect to other seeds instead (whenever possible). It does this by "desiring" a seed in an inverse proportion to how much that seed has already given. IOW, it will hit the new kid on the block before they go offline and leave the old guys alone.

From what I've seen, the LeoXv implementation isn't a binary thing, it's a subtle sliding-preference thing that over ten or twenty minutes will taper the peer pool and generally make *my* client a far better swarm member in all respects. The idea, as far as I can determine, is to use new seeders disproportionately; to avoid the established old seeds. This adds to the health of the swarm by both catching quasi-leechers before they go offline (after just-completing their DL) -and- by politely using seeders (like me) that wish to upload the initial 150% as quickly as possible so they can rotate to the next torrent and seed it to 150% (or 200% or 300% etc.).

The global effect of downloading from (but not uploading to), the leechy dribbler clients does several positive things for the health of the swarm.

1) It maintains the first-unchoke-is-free slant that Bram Cohen likes. This allows the well-behaved but bandwidth-limited clients to prove themselves to be a good client. If they can produce at a rate anywhere above the threshold I set, then my client usually sends them far more KB/s than they sent me. Again, this preserves Bram Cohen's modem-users-need-more-love concept of fairness. More importantly, it allows a low BW client to adopt a "fair" configuration that actually gets a decent DL rate in return for playing fair.

2) Low upload BW (leech-configured) clients find themselves uploading to peers that don't return the favor. They begin to get snubbed more and more as time goes by and their DL rate falls into the toilet. Since any *negative* response like disconnecting or banning my IP only helps me avoid their leechy client, they have virtually no way to "hurt" me or not play fair with me. In all cases, my client's response is to download from them without uploading to them. In fact, the only way they can escape being snubbed is to (deity forbid) become a well-behaved swarm member. My client will immediately provide positive reinforcement if they choose to play fair at any time. This provides both a strong positive reinforcement and a strong negative reinforcement to play fair. Greed will dictate that they adopt (generally) fair settings or "suffer the snub".

3) The long term effect of snubbing is that the *other* client will drop *you* and not *want* to reconnect to your client. This is exactly want we want. In fact, no "smart" banning algorithm could ever accomplish making *them* not want to connect in the same way that snubbing can. It's a slow gentle way to make *my* client an undesirable target for their leeching, and it's done simply by not looking like an easy target anymore. Nature takes care of the rest. If you walk down the street wearing a Rolex and a fine silk suit, then you look a lot more inviting than the barefoot guy wearing a burlap sack.

4) At first blush, it would seem that when we apply Bram's mandate of behaving in a swarm of *all* LeoXV chokers, there is an opportunity for low BW clients to find themselves peer-less because all the big fish have set their choker at 10KB/s and the low BW modem users can't get above the threshold. Well, yes, this is true. However, by weeding out my client as a viable source of tit-for-tat, the "offended" client will seek other modem users that will be much more willing to TFT *and* the TFT rates will be much more symbiotic.

A modem user could never attract my client's attention in a TFT tug-of-war with another cable-modem peer, so they are better off finding a different peer with similar bandwidth. Also, even if they *did* get my client's attention, they couldn't handle my 1 Mib upload rate anyway, so they'd be wasting my unchoke slot and blocking packets coming to them from other peers (that are a better match for their BW). By blocking them out and wasting the other upload slot, the modem user will become less desirable to all of them. This leads to starve-feast-starve and decays into a very inefficient data exchange. You want to keep both ends of a TCP connection *pushed* full of data so that the network driver doesn't have to stop and wait for data.

If I understand correctly, my client will send a "choked" message when I snub them, so they aren't getting blind-sided. Since my optimistic unchoke response will likely send 2-3 times more data at them than they can send to me, they won't get screwed initially, but they *will* get snubbed over time if they continue trying to TFT with me. Note however, that an interesting thing happens to well-behaved (but low BW) clients. If they continue to send my client data non-stop and they *are* above the threshold, then once in a while my client sends them a burst of data. That one burst is likely to be more than they would get from another modem-user during the same time period. So, over time, it *may* work out as a viable strategy for them, although to watch it happen it *appears* like my client is abusing them. But if I'm sending choke messages, then nobody is getting abused - they are choosing to "woo" me into a bit of TFT once in a while. Again, nature takes its course and our two clients work it out. Or they move on.

Now, when a client can't get over my threshold, they get snubbed and this sounds like an opportunity to screw the little fish and leave them out in the cold. However, I've tried a "mean" setting on the LeoXV threshold to see what would happen and I was greatly relieved to note that after a short while, my peer list was barren and my DL rate was 1/4 of my UL rate. By snubbing the-little-fish, I shot myself in the foot because many little-looking fish are actually just big fish that don't know you yet. Like your client, they have to shuffle through a number of leechers and smaller fish to settle-in and discover other big fish that will offer TFT in the right (matching) BW. My snubbing them too early tells them that I'm not really a big fish (or I'm not playing fair) and they should move on.

In a poor quality swarm, setting a moderate threshold can dramtically improve the quality of the whole swarm and produce viable seeders much quicker. In a high quality swarm, it is advantageous to set the threshold as low as possible and allow all peers to compete for your best TFT. In playing with the adjustable choker, you quickly realize that settings below 1.5 KB/s have little effect and settings over 3.5 KB/s will usually tend to get you snubbed - unless you're in a jumbo swarm. Note that my experience is all on public trackers as nobody has invited me to join a private one.

You have to consider that the LeoXV algorithm isn't a silver bullet, nor is it the only game in town. For instance, it occured to me that a more viable choker could be constructed by looking at the number of blocks being passed instead of looking at the raw data rates. It is more difficult to determine the best TFT candidate with this sort of algorithm, but it would virtually put an end to perenial leeching - especially if the client used persistent storage to keep track of total blocks sent/received from each client over a long period of time.

Yes, I realize this is traditionally the job of the trackers, but IMO, it really belongs in the client in today's post-naive leech-infested reality. I also believe the future is trackerless peer networks. While a built-in database engine is probably not in-line with the minimalist nature of utorrent, it would make it possible to construct an exceptionally intelligent and ultimately very fair BT client. It also allows my client to download a Debian distro CD from you today and then give you back a copy of that new Fedora release next week. When you pop online, a background process notices that you're "an old friend" and gives you a higher multiplier when calculating how much juice you get from any of my active torrents.

Unlike private trackers where you might have a great share ratio for your porn files, you have no reputation with my client because I don't want any of your porn. If you gain a good share ratio with me, it's because we share an interest in the same kind of material and you've treated me fairly in the past - perhaps more than fairly. I'd definitely like to return the favor some day.

I believe that the world is so screwed up today because we lack personal accountability. The tracking of specific TFT behavior as it affects me directly would promote personal accountability and that would be a good thing. You could *also* implement a tracker that published the personal experience of many peers and allowed them to form their own conclusioons based on this information.

A crude approximation of this behavior could be accomplished with a traditional BT client. The client could simply log the necessary block counts into a text file and an external program could crunch the data. There would have to be some kind of text file or other method implemented to tell the client that a peer was more or less desirable. A simplified approach would be a text file with each IP on a line with a "preference" value next to it. The external program would periodically parse the BT client logs and update the "preference" values. When the BT client opens a connection to an IP, it would grab the current preference value from the file.

Since IPs would be a relatively poor way to track peers over the long term, this could spur a trend toward a unique (yet somewhat untraceable) peer ID. This probably sounds scary to people that traffic in illegal material. I don't think promoting the fair and legal use of BT and discouraging the leeching / illegal use of BT to be a bad thing really. Besides, it's not like the RIAA can't get a court order to make your ISP rat-out your IP == real-life identity, anyway.

All this was drifting a bit off-topic, but the point is that alternate chokers and alternate methods to apply positive and negative feedback *without* trying to ban specific clients is, IMO, a better approach than a wholesale ban on a piece of code. Don't shoot the car, coerce the driver to use a different road instead.

[Switeck]

There is only ONE "charity pipe" -- optimistic unchoke only goes to ONE peer at a time. There is not a separate unchoke for new peers, they are simply more likely to become the next optimistic unchoke peer. ALL other upload slots are devoted to Tit-For-Tat (TFT). What's happening is BitComet's foot-in-the-door trick is causing REAL new peers to be starved and have very slow ramp-up times. This has been expressed repeatedly in many other threads about µTorrent's slow start-ups -- which also point out that µTorrent isn't trying to get alot of connections quickly to compensate. It's my opinion that lots of connections shouldn't be needed to ramp-up to a minimum download speed, but that's not going to be true until a few clients change implimentations or get depreferanced by other clients.

It does little good for you to toss random blocks to a new peer if those random blocks don't complete a chunk for it. But even then the new peer would have nothing yet to share with you...because its first chunk you already have! A similar problem occurs in torrents with 1 seed and nearly every peer having x% -- no matter who you trade your "rare" chunk with, they probably will have nothing to return to you at this time.

...actually, I see 1 problem with the LeoXV choker. In some torrents, you might connect only to BitComet clients with average upload speed per upload slot below 1 KB/sec. They may have fast upload speeds in total but spread their upload speed across so many peers (probably on different torrents as well). BitComet itself doesn't have much of a problem with this, as it seeks LOTS of connections for itself -- even spamming the tracker to get them. Your client would attempt to snub all of them since none would satisfy your choking criteria, and would be a true leech asides from the optimisitic unchoke.

What if BitComet was simply never considered or considered with lower odds than other clients for optimistic unchoke by µTorrent?

...especially if seeding or super-seeding.

[AllWeasel]

I think we have multiple topics being discussed, which is perfectly fine, but it can lead to misunderstanding sometimes. I had brought up several ideas, some of which spoke about how Bram Cohen's "naive" client behaved. Some spoke about how the BT "protocol" works in theory and then I went off to mention BT protocol in practice and finally, how a slightly altered BT protocol might be implemented.

1) I don't believe that *any* attempt to identify a specific client and treat that client in a special way is viable. It would simply take too much effort and the only way you could keep up with the "abusive client of the week" is to do exactly what the virus scanner industry has done. Even if you could find the paying customer base for this ongoing war on abusive clients, the BT protocol would simply die in a very short period of time.

It would be very difficult to identify which client (let alone which version of that client) you're talking to. Now add the complication of pro-leechers that willfully attempt to engage in leechy behavior. They will actively try to make it impossible to identify themselves. Today BitComet, tomorrow the next generation leecher client disguised as utorrent itself, or mainline or whatever.

2) Identifying clients is not viable, and I believe it is trying to solve the synptom, not the problem. The problem is the behavior, not the "abusive client of the week". Actually attacking the behavior is probably not viable or healthy for the swarm as a whole either. Offensive approaches, in general, just tend to get a client banned or snubbed in the real world. Being the only client on the block to refuse connections to BC would make you the only client not DL'ing anything. Many trackers would ban you simply for being unfair/biased in your peer-connection selection.

Instead, I believe that a good defensive or passive [aka; passive-aggressive] behavior will be more successful and easier to implement. If all clients stop uploading to peers that send less than X KB/s, then eventually, all peers (and intentional leechers alike) will be forced to change their settings to allow a healthier swarm. So, I don't care if they are leechers, poorly configured clients, or poorly designed clients, they will all have to change their behavior to adopt swarm-friendly settings or they will be leeched-upon and receive nothing in return. Thus, the behavior will change - without regard for why the behavior was broken before.

3) I suspect that Ludde is philosophically against any "special" treatment of clients, and I *know* that I am against this approach; if for no other reason than it's like performing brain surgery with a chainsaw. Also, because of item #1 above and the technical simplicity/elegance of implementing #2 above.

Based on which suggestions Ludde has implemented, and which things he proclaims will "never" be implemented, it would seem that utorrent is perhaps the least likely BT client that would [attempt to] implement a client-specific ban mechanism. It was this premise that caused me to suggest a more uniform way to approach the problem. Note that by client-specific, I mean something more involved than just identifying a "well-meaning but buggy" client by it's name+version.

4) The LeoXV-inspired choker, found in Rufus, is *not* the first choker to attempt the snubbing of leech-like peers; specifically, peers that don't upload a reasonable BW or flow rate. In fact, the G3Torrent choker does exactly that and only that. It was the shortcomings of that approach that caused LeoXV to go further (I think). What the LeoXV "improvements" do, are limit the damage caused by your own client when it is in a situation that would normally induce leech-like behavior when following the [naive] BT protocol. Also, it allows for tuning this behavior to maximize the health of the swarm on a per-torrent basis. I only mention this because I fear that many people think that the concept of threshold-based chokers == LeoXV, which is really not *only* untrue, it goes against the very nature of LeoXV's modifications to the threshold-based choker algorithms.

LeoXV clearly saw the potential problems associated with snubbing all low BW peers. He also recognized the unique conditions when this behavior was detrimental to the health of the swarm. If the last (or an extremely rare) piece, has to come from one of those low BW peers, then exceptions must occur. If the swarm consists only of low BW peers, then exceptions must occur.

He allowed for this paradox in an interesting way. He included a setting that determines the maximum data size allowed before the choker rules kick-in. As time goes by, I think I'm understanding more and more about how his modifications affect the behavior of a threahold-based choker. I wish that I could find a record of his thoughts on this - instead of just trying to learn by empirical observation.

5) As you mentioned previously, I think we both agree that allowing BC to reconnect *and get preference* while doing so, is wrong and clearly unhealthy for the swarm. I believe that when Bram Cohen refers to a "new" peer, he does not conceptually include a reconnecting peer in the definition of "new". Although his papers don't actually consider the situation of reconnects, and comment on this behavior, clearly they *should* address this situation and other similar "abusive" issues. Actually, I believe that the entire optimization that he is talking about does *not* apply to swarms that are larger than the current peer-pool limit; but that is another topic.

Clearly, anyone that understands the intended behavior of the swarm, and the actual behavior of naive clients during repeated peer reconnections, will also understand the damage that this causes to the swarm. I think all clients that behave this way (in cases where the swarm is larger than the peer-pool), are simply bugged.

However, to behave this way when you have less than the desired number of peers in your current peer pool, well, that's starting to split hairs at the implementation level. Yes, it's a significantly better behavior to not prefer reconnecting peers, but it may be a significantly big PITA to check for this specific behavior - depending on how the client code was designed.

However, in light of the 300 pound gorilla BitComet dominating the BT client universe, I think it may actually be better for the swarm to simply ignore the optimization of giving "new" peers a better than average shot at the optimistic unchoke slot. Again, if one could implement the "new-peer optimization" *and* not reward a BT client for constantly reconnecting, then that would be the best of both worlds. Even when ignoring the optimization, if the peer pool size is reasonable, then the optimistic unchoke will get to everyone fairly quickly.

There is only ONE "charity pipe"... [snip]

Yes, I concur. What I was talking about in regards to a "charity" pipe applies to alternate BT implementations, such as a TFT algorithm based on byte counts instead of momentary spurts of BW. The "weak spot" in alternate algorithms would be the lack of "random" rewarding of non-producers. This is an important function to maintain healthy swarms. The "classic" BT implementation of using the optimistic unchoke currently provides this function (and other functions).

I think the optimistic unchoke also plays heavily in the determination of BW when a client is not actually seeding, but it has no peers that are currently "interesting". Note: I could be wrong about that. It seems to me, that the optimistic unchoke is the only mechanism that directs BW to other peers when no peers have a desirable piece.

I think most clients have provisions to cope with this and they increase the number of unchokes dynamically to meet the BW allocated for uploading. I assert that splitting the normal TFT - and the optimistic unchoke operations, into two different physical queues, would allow for a smarter and ultimately more fair way to distribute "charity" bytes back to the swarm.

By charity, I mean non-TFT bandwidth that will presumably never return any desired pieces to the client. This "charity" traffic is crucial to the correct operation of the swarm and is a major contributer to high BW swarms that allow a client to download data at a rate of nearly 100 times their own upload rate. Allowing BT clients to have 50 upload slots per swarm effectively kills this charity BW and also severly cripples the purpose and effectiveness of the optimistic unchoke.

Anyway, the discussion of alternate (better ?) BT algorithms is off topic here and really needs it's own thread to discuss the possibilities properly. Even then, it may not be on-topic in this forum at all. I doubt Ludde is going to jump at the chance to completely re-design utorrent to try some of our alternate implementation models. Personally, I like a good theoretical conversation, so it's all good for me, it's just that it's probably OT here.

actually, I see 1 problem with the LeoXV choker. In some torrents, you might connect only to BitComet clients with average upload speed per upload slot below 1 KB/sec. They may have fast upload speeds in total but spread their upload speed across so many peers (probably on different torrents as well). BitComet itself doesn't have much of a problem with this, as it seeks LOTS of connections for itself -- even spamming the tracker to get them. Your client would attempt to snub all of them since none would satisfy your choking criteria, and would be a true leech asides from the optimisitic unchoke.

I'll assume that you mean threshold-based chokers in general and not the LeoXV implementation (which I now believe actually has provisions to avoid this problem) in particular. I might say facetiously, that if you're connected to nothing but BC peers and you are actively banning them - then you're truly screwed - much worse than snubbing them with a threshold-based choker. But seriously, yes, the downfall of a ridgid threshold-based choker (TBC) is that in a pool of peers that can't overcome the threshold that you've set, you are effectively doing nothing but leeching - which eventually leads to you being snubbed by everyone. Ain't it a bitch to do the right thing sometimes?

The big question to ask - one that Bram asks, is what happens if the swarm consists of nothing BUT clients that behave this way? Well, I think it's safe to assume that anyone taking the time to use and tune his TBC, is probably concerned about the health of the swarm and is also smart enough to configure his client to deliver more per-peer upload BW than his choker threshold. To do otherwise will starve you out of peers, so it's kind of a no-brainer. If not, then certainly the client could impose these limitations automatically "on behalf" of the [ignorant] user. Regardless, the answer to Bram's question is yes, it's absolutely a good situation - provided that the limitation on your upload rate is your settings and not the network or a slow modem, bad ISP, too many hops etc..

So, this is where your (valid) concerns come into play. What if a peer is using a well-behaved and well-intentioned client, but the connection simply won't allow enough data to overcome your threshold? What if *I* am the only peer in another continent and all the others "seem" to be poorly behaved peers (although they see each other as very healthy)? These situations and others like them require a relaxing of the ridgid threshold settings - an exception if you will. The problem is that it gets exponentially difficult to decide when to make those exceptions and not render the TBC completely useless.

Partly due to your comments, I spent some more time looking the LeoXV implementation and I now believe that he addresses this problem in an interesting way - one that I hadn't considered before, such that the "settings" I was looking at completely escaped my attention. I just glossed over them thinking they were just parameters like you'd enter for a disk cache. They mean *something*, but the effect is obvious and linear so you don't really give them much thought. Only a diehard tweaker would futz with them.

I now think that I was being obtuse and I didn't comprehend what I was really looking at. There is a line labeled "DL Threshold to start choker" with two settings labeled "Piece size" and "Multiplier". I think the idea is to NOT use the TBC behavior for data packets below the selected "piece size" setting.

I'm NOT comfortable saying that I understand the behavior of the multiplier setting because my empirical observation has not been conclusive. However, when a typical BT client starts getting into "dire straights" trying to find rare pieces and also in endgame mode, it starts requesting sub-pieces, often to multiple peers at the same time. It is precisely at these "desperate" times when we wish the TBC wasn't being used. It's sort of a left-handed way to figure it out, but it makes perfect sense really.

At the very time when we're trying to collect little sub-pieces and we're willing (out of greed) or forced (due to need), to drop the TBC in order to get them, the TBC naturally disengages on it's own. Dropping the TBC for other peers so they can complete their own endgame is also important for the swarm. This case is also handled correctly. Dropping the TBC for tiny torrents (naturally tiny piece sizes) is probably also a smart thing to do. Again, this desirable behavior is a natural side effect of the "piece size" settings. I don't think it elliminates the problem, but it does mitigate it during the times that the swarm could best be served by the TBC not being there.

In response to the attempt to perfect the art of "the TBC that wasn't there", I'd (strongly) assert that the swarm, BT, and the universe in general, would all be far better off if those clients where choked to death and leeched unrelentingly by everyone. The quicker people realize that they need to supply a *limited* number of *quality* upload slots, the better. If that means that I don't participate in perpetuating a sickly swarm, then I'm ok with that sacrifice. I'll go find the information or files somewhere else, or not at all. IMO, it's better than degrading myself to the least common denominator just so I can be "one of the gang" and get my file. IMO, the worst thing one can do in that situation is to encourage those low-yielding peers by collecting the dribbles and then sending them juicy high BW bursts in return.

'Twould be better to let them die the death of a thousand cuts by leeching them into oblivion. IMO, you have to apply pressure to make people tune their clients for the health of the swarm - and they will eventually discover that it actually helps them in the end.

Something else I have been thinking about is the way that everyone assigns malice to BC's behavior. Clearly, some things are design decisions or bugs that have conveniently "not been fixed yet", however there are some things like the reconnect behavior that I believe are simply side effects of it's internal design. I think the peer connection logic is implemented as a separate thread of execution. That thread just tries to keep the connected peer pool between the min and max settings that the user has entered. This thread is responsible for limiting the rate of connections and the number of in-process connection requests. Probably, it checks the number of times that the same IP has been connected etc..

Another part of the code is probably responsible for controlling which peers are worth keeping and which ones have snubbed it. If a snub is interpretted as a technical failure, as opposed to an asserted effort to disengage the client, then BC's response to being snubbed might be to disconnect and "call back" for a better line. So, the IP is passed into the queue of the "connection" thread and forgotten. If the connection thread's queue is fairly short, and the snub-detection code is agressive, then this snub-disconnect-redial cycle might happen very quickly. BC may well be too simplistic to detect this endless loop across the two threads.

If BC's targetted client was correctly not considering it to be a "new" peer simply because it tried to re-connect, then BC would find itself at the back of the queue, never get any data upon repeated reconnects, and the BC users would start getting pissed-off at this behavior. Putting the motivation (pain) back on them is the solution, because trying to ban them will simply make you the odd-man-out in a BT universe dominated by BC clients.

In summary, *I* think leeching blindly (and without remorse) from all clients that are poorly configured (or intentionally leechy) and uploading generously to the well-behaved clients, is a better (and more viable) solution than a holy war on BitComet.

[Switeck]

µTorrent already remembers download and uploaded data from peers, even if they reconnect. In fact, it remembers it throughout the entire session (even if you stop and start the torrent), I believe... So implementing something like this would probably be not that hard.

It's still really needed:

http://forum.utorrent.com/viewtopic.php?id=5181

The only way the results that poster is seeing makes sense...is if µTorrent is being hit repeatedly by a reconnecter.

[HyToFry]

Why not just choke new clients for thirty (or so) seconds? That wouldn't hinder the download *too* much, but it would make constant disconnecting/reconnecting a fruitless effort. Legit "new" clients would still get their 3x boost so they had something to share, but clients which tried to abuse the feature would be crippled for a large portion of their download life.

Sorry, didn't read the entire giant thread, so if I stepped on some toes there, then change my post to "I vote for the 'choke new clients for thirty seconds' idea."

*edit

And while you're at it... why not just choke the clients which are known to be malicious? ;->

[AllWeasel]

Sorry, didn't read the entire giant thread, so if I stepped on some toes there, then change my post to "I vote for the 'choke new clients for thirty seconds' idea."

*edit

And while you're at it... why not just choke the clients which are known to be malicious? ;->

Q: When did a developer ever let users vote on how to implement an algoritm?

A: Never.

I vote to shoot people that don't read the whole thread before they open their mouths and offer advice.

A) It WOULD adversly affect the swarm to delay 30 seconds.

You can't figure out which ones are malicious clients, because you can't trust ANY information they give you.

C) You'd know all that if you had read the entire thread.