Dynamic upload slots / bandwidth management

[U.N.Owen]

Please correct me if I misunderstood protocol implementation in uT, but to me it seems to follow original bittorrent spec with regard to upload slots: for each torrent uT tries to open no more than N connections out of which M are active slots (with N and M specified in settings), and upload as fast as possible to unchoked peers, constrained only by this user's up capacity and other peers' down capacity. I think this algorithm can be improved in the following way: IF the client is consistently downloading at less than maximum line speed THEN dynamically limit the speed of each upload slot to, say, 110% of the respective rate we're getting in return AND open additional slots to utilize freed upload.

In other words, I think uTorrent is too rigid and inefficient in bandwidth allocation and randomly wastes too much of its scarce resources. For each peer it has either "on" state (user uploads some above-threshold amount to us so we throw all we got at him/her) or "off" state (user is slow, so we snub them). It would make sense to replace this binary relationship with true linear TFT: the more you give, the more you get back. Looks like any peer dealing with current behaviour has incentive to find "just right" amount of upload and exploit this by withholding extra. Proposed bandwidth allocation scheme would proportionately reward high-speed reciprocators.

[Ultima]

Upload slots aren't completely rigid. By default, if µTorrent finds itself uploading too slowly (less than 90% of the set upload limit), it'll open additional upload slots.

[Switeck]

And if you set your upload slots too high for your current upload speed max, uTorrent won't use max upload slots either...it may only use 1 per torrent if upload speed is too low.

[U.N.Owen]

Ultima, that is true, but completely unrelated to the point. I'm trying to explain that unless I horribly misunderstand uT's opaque inner workings from surface experiments, it is inefficiently downloading things by trying to divide allotted upload bandwidth equally between peers in our active peer set. The less speed we get from a particular peer, the less that peer's threshold for kicking us out of its active peer set is likely to be. Why upload hundreds of KBs per second to someone giving us only tens of KBs in return where we could throttle that particular upload to same tens of Ks, probably retain download speed from that peer and use extra bandwidth for opening other connections that are going to bring us more downstream, while our upload contribution to the swarm would remain the same.

[Switeck]

uTorrent already seeks out the most generous peers and gives them more of its own upload.

There's just a minimum amount each upload slot gets, if possible.

What uTorrent DOESN'T do is work like BitTyrant and create more upload slots simply because the "X" best peers aren't uploading back to you "fast" enough. (Where "X" = max upload slots per torrent.)

[U.N.Owen]

>uTorrent already seeks out the most generous peers and gives them more of its own upload.

True, but it does so only by choking the worst uploader, which is a terribly inefficient way to do things. Dynamic rate adjustment can react to all peers at once at the timescale of bandwidth averaging window; on the other hand current implementation chokes only one peer each round and doesn't make distinction between best and worst uploaders (whose contribution could differ by more than an order of magnitude) as long as they are included in active peer set.

>What uTorrent DOESN'T do is work like BitTyrant

Thanks for the BitTyrant pointer. I've read their paper and looks like they already implemented even more aggressive algorithm, which (naturally) gives superior results. Do you, or anyone else, by any chance know BitTorrent,Inc. position on adding the option to use bittyrant-like choker/shaper in uT in the future? If it is left for developers to decide, what is their opinion?

[Switeck]

Tit-for-Tat exists in uTorrent, or uTorrent wouldn't qualify as a BitTorrent client.

[U.N.Owen]

Switeck, are you involved in marketing? TFT exists in uTorrent, what a surprise! And burger flippers receive money for their job. So, we'll just sweep the issue about six-digit salaries of professionals under the rug and be done with that.

I have to repeat myself for the fourth time, my posts don't imply uTorrent doesn't work. My claim is that it could be improved. Bittyrant-like scheme is already proven to be ~70% more efficient, see their paper. Possible real acceptable answers to that:

1. Your data is wrong, uTorrent is the most efficient client, here's our data.

2. uTorrent isn't the most efficient, but devs are too lazy to develop it further.

3. uTorrent isn't the most efficient, but devs are prohibited from developing it further.

4. uTorrent isn't the most efficient, and devs don't care.

5. uTorrent isn't the most efficient, and it should stay so because we believe in socialism and want to force altruism by subsidizing asymmetric low-speed users at the expense of symmetric high-speed users.

Answer that you gave:

"uTorrent already works, so yeah".

[Switeck]

"True, but it does so only by choking the worst uploader, which is a terribly inefficient way to do things."

You seem to have no understanding either of the BitTorrent protocol or why BitTyrant is even by the designer/s own admissions probably not good for general-purpose use. (Meaning if everyone uses it, everyone will likely be WORSE off.)

"Efficiency" can be measured in many ways. Maximizing download speed at the expense of upload speed...is not a good one. And that comes at a price on a marginal torrent with few or no seeds.

[thelittlefire]

If you're going to be flippant, you're no better than a troll.

You ask for clarification on points you obviously have no knowledge of and then lack foresight to listen... What about the logic and presentation is lacking?? Switeck obviously cares, and your ad hominem is insulting. Go away.

[U.N.Owen]

thelittlefire:

>you're no better than a troll. Go away.

Your post, on the other hand, was full of deep insights on the protocol design. Thank you for your meaningful contribution to the dialogue.

Switeck:

>"Efficiency" can be measured in many ways. Maximizing download speed at the expense of upload speed...is not a good one.

Neither BitTyrant nor "110% back" scheme give less upload to the swarm. They are only redistributing their upload bandwidth to increase their download. Upload = same. Download = more.

Oh, and one more thing. I've searched this forum for "BitTyrant" and found your post, dated 2008-09-15, where you state that

"uTorrent's/BitTorrent's tit-for-tat system needs to be rethought and strengthened. This may mean uTorrent needs to use a download/upload strategy more like the selfish BitTyrant client...that will basically only upload (quickly at least) to those that upload back to it."

But this is exactly what I'm trying to convey here. What arguments made you change your opinion on this matter?

[Switeck]

Downloading a large torrent in 1/2 the time discourages reaching equal levels of uploading. While this is not a "fault" of the protocol, it represents potential disaster on public torrents where hit-and-running is already prevalent.

To download at max speed...upload speed MUST be sacrificed. At least 1 KB/sec upload speed is used per every 40 KB/sec of download speed attained. In worst-case conditions, the ratio may become more than 1 upload for every 10 download. So downloading at 100 KB/sec comes at the price of upload speed being reduced anywhere between 2 and 15 KB/sec from max. While this may not sound like much, it can represent a significant fraction of people's total upload speeds.

BitTyrant almost completely does away with the concept of regular optimistic unchoke slot. The optimistic unchoke slot causes uTorrent to give heavily to peers that have returned nothing despite opportunity to do so. This does not mean I want the optimistic unchoke slot to go away completely, only that either through malice, ignorance, and laziness it is sometimes getting "gamed" by other BitTorrent clients...such as the ones that set global max upload speed to 1 KB/sec.

[U.N.Owen]

Downloading a large torrent in 1/2 the time discourages reaching equal levels of uploading only if the user will stop the torrent right after download completes. Slow DSL users who need to throw their tiny upload at the task of getting next file are, on average, more likely to do so than fat symmetric pipe users who aren't even reaching their top line speed either way most of the time.

BitTyrant almost completely does away with the concept of regular optimistic unchoke slot.

..and yet it achieves 170% download performance. Looks like BitComet does it too. That means the protocol is broken, not those clients.

Reluctance to add the option of using alternative choker/shaper is a subtle way of enforcing user behavior through the means of client compliance (as opposed to protocol design). Doing so is a Bad Thing, because it can be locally broken.

Consider a toy model bittorrent-like protocol, where peers exchange encrypted chunks and then trade decryption keys for equal amounts of data with central tracker acting as an escrow agency. No matter how clever and inventive the client is, it just cannot cheat the protocol, therefore the rational decision for each peer is to provide X uploaded bytes in exchange for X downloaded. Consider another case, where peer can't stop the torrent until it reaches 1:1 ratio only because client says so. Here it is trivial to circumvent restriction by implementing another client without such (mis)feature, by patching existing client, by adding some firewall rules, etc.

I have to say that in current situation uTorrent looks more like the latter, not the former. By writing a bunch of iptables classifiers or, even easier, by switching to another client (since the protocol is kinda open, with the exception of uTP) each peer independently acting in one's own interest can increase his/her download speed at the cost of learning another client (and longterm global speed decrease, but "to switch" is a strictly dominant strategy in any case). Can you guess the equilibrium the whole system will end itself in?

This does not mean I want the optimistic unchoke slot to go away completely, only that either through malice, ignorance, and laziness it is sometimes getting "gamed" by other BitTorrent clients...such as the ones that set global max upload speed to 1 KB/sec.

Setting global max upload to 1 KB/s is an irrational decision (unless you're on 14k dialup or (pay per volume uploaded and don't care about DL times)), ignorance and laziness will be punished by decreased download speeds. Such peers could be virtually eliminated from the pool if other clients, including uTorrent, learned selfish behavior. Only because of stupidity on part of optimistic clients such peers can enjoy more-than-1KB/s downloads and have the luxury of not having to learn from experience.

On the other hand, "malice" can not be prevented. Good design practice is that you expect end users to try to exploit and sabotage every part of the system under their direct control in attempts to secure more resources for themselves. Good protocol is distinguished by such incentives built in, so that equilibrium strategy is stable and deviating from it will immediately decrease payoff to the defector.

[DreadWingKnight]

Setting global max upload to 1 KB/s is an irrational decision

And yet hundreds of users do it every day. Look at what happened when we enforced download limits on low upload speeds.

[thelittlefire]

You capitulated to the whiners, that's what happened.

[Switeck]

Don't bother mentioning BitComet as an example of how to do things...

Go watch this for how many people set up uTorrent:

http://www.youtube.com/watch?v=NVq2GkzNrY8&feature=related

MANY people stop torrents shortly after they start seeding regardless of the speed of their connection.

[ben130g]

I always have my upload / download speed set to max - It seems not to have any effect - except for a minute or so at the beginning an the d/l speed MAY be low but soon picks up to max d/l speed. but seems to automatically limit my upload speed to around two thirds of max.Put it this way I d/l say 1 gbs file six or seven times faster than upload or even more which then seems to take forever to return. O.K. so i could limit my d/l speed Yeah,sure.

[Ultima]

Um... huh?

[Switeck]

ben130g, are you using settings from the 2nd link in my signature that roughly matches your connection's upload speed max?