uTorrent 2.0 fail?

[Greg Hazel]

Hmmm... Was it good when I actually disabled TCP connections after switching to uT2.0?.. I set bt.transp_disposition = 10... Everything goes nice, I'm having almost maximum theoretical speed all the time and able to use browser without stopping torrents...

I think that's great! If you're getting maximum throughput, clearly your connection could not be more used. You are contributing to the swarm as much as possible, and not paying a congestion penalty.

However, there are TCP-only peers out there you can't upload to. If the swarm ever shrinks to be small enough, maybe one of them won't be able to get data because he can't connect to you. If you're downloading a file with 50k peers, I wouldn't worry about it. If there are only 10 peers, I would say it's probably an issue.

The thing same happened when encryption became an option in the BitTorrent protocol. As with that situation, you should set your settings to suit your needs, and just be aware of the effect that has.

[Lord Alderaan]

I just want to add that with encryption it was only useful to set it to forced if it actually helped you circumvent ISP throttling.

The same goes for µTP, if you aren't being throttled then setting µTP to forced (which is what a setting of 10 in bt.transp_disposition entails) is counter-productive (both for you, specific peers and the whole swarm). At least until µTP becomes widespread.

[DaVrOS]

Just to add to this - here are some examples from members on our site (I am an admin on a mediunm to large private site):

Its not just seeding, its downloading too. I am using a server to download and was using a utorrent 2.0 full swarm just yesterday to download a torrent. Getting 1kb/sec download speeds because no-one is connecting to me due to 2.0 preferring itself is not good.

I had to wait ages for 1 person in the entire swarm who wasn't using 2.0 to connect to me before I could download anything.

Utorrent in its current form should be banned everywhere.

The test that was done before with 40 connections is actually relevant when you take RSS feeds into account and lets say 200 people with the same hosting company - which is a common thing nowadays on private trackers.

The big problem I find is that Private sites works different to public in that they are generally Ratio driven - therfore the individual members ratio is important and if it falls too low then they get banned by that site automatically.

Now if you have a UTP heavy swarm and they are not using a UTP cient then they DO suffer in that instance.

I did notice in the upcomming 2.1 changelog that there was a race condition change to spread UTP and TCP more evenly. Was that done to try and fix the condition of "accidental preference of UTP" caused by the faster UTP connection speeds?

Even the staff here seem to agree that there is preferencing:

eg:

2010-02-18 05:58:23

DreadWingKnight

Re: "Unethical" Features?

The "preferencing" you're seeing is an artifact of the uTP support in 2.0 and will go away as more clients support uTP.

Support for uTP actually pre-dates the 2.0 branch entirely.

There has always been a way to turn it off.

2010-02-18 06:07:31

Firon

Administrator

It connects faster than standard TCP and is far more effective in preventing network congestion, but basically until other clients implement it, 2.0 clients have a small advantage as a side effect. It isn't anywhere near as big of an advantage as it's made out to be, though.

Additionally. With UT 2.0 or 2.01 on my home connection I only download at about 650KB/s but with 1.8.2 (14458) I max my conenction out at 1.2 MB/s on downloading. With BOTH speeds I never had or have any problem browsing the internet. I am a little concerned about how much it actually throttles me. 50% is quite a drop.

[Greg Hazel]

I did notice in the upcomming 2.1 changelog that there was a race condition change to spread UTP and TCP more evenly. Was that done to try and fix the condition of "accidental preference of UTP" caused by the faster UTP connection speeds?

No. This race condition only occurred when the peer supported both TCP and uTP. If the two connections completed at the same time, the bug was that each side might pick a different connection to drop, resulting in no connection at all.

[DreadWingKnight]

I've said it before in this thread and I'll say it again.

Seedboxes on these sites are ALREADY causing more damage than uTP is capable of causing.

[Firon]

- Fix: simultaneous uTP and TCP connection race condition

is already in 2.0.1 and not quite. It was a race condition that left both ends disconnected when it happened. It was actually pretty common.

And that test isn't relevant. Even with RSS, they don't all get it at the same time. It does take time to check the feed (and it only checks every so often too) and actually grab the .torrent.

[DaVrOS]

I agree seedboxes are a problem but they are not the discussion here are they. UTP and uTORRENT are I though?

Plus my post had multiple questions/comments.

I find the one from that "uploader class" that I quoted interesting in re: to his downloading.

[DreadWingKnight]

They're banning one thing causing a relatively minor problem in regards to ratios and yet allowing another thing causing MAJOR problems in regards to ratios.

Stinks of hypocrisy to me.

[Firon]

Sure, but you can't sweep the problem seedboxes present under the rug when the argument given here is about protecting your users.

edit: argh you edited your post. >:|

Additionally. With UT 2.0 or 2.01 on my home connection I only download at about 650KB/s but with 1.8.2 (14458) I max my conenction out at 1.2 MB/s on downloading. With BOTH speeds I never had or have any problem browsing the internet. I am a little concerned about how much it actually throttles me. 50% is quite a drop.

That is not normal behavior. It's just 2.0 doing weird things (largely due to the TCP rate control and other bugs with that). We found a bug serious enough that warranted making another 2.0 (speed drop every 60 seconds).

2.0.1 has a great deal of fixes and tweaks which should solve overthrottling and poor performance in almost every case.

[DaVrOS]

So now it wont allow tcp and utp connections at t the same time. (which is a good thing of course) - that will now mean that UTP will connect first and then TCP or will it fully randomize?

I should add that I actually take care of the client bans on our site and we currently do not allow UT 2.0 or 2.0.1.

The problem of course is not really UTP - it is that other clients do not support UTP.

In the meantime there has been some talk of a private UTP flag where sites could turn off the UTp feature with the client supporting that "switch".

I did put this to firon privately but was met with some hostility. I do love uTorrent as a client but for sites - there isa bigger picture that just 1 client - no matter how popular.

oh and my above psot said 2.0 OR 2.0.1 - as I tried both.

[DreadWingKnight]

Did you keep up with the ever-changing 2.0.1 builds?

[Firon]

It does uTP and TCP at the same time, just like before. The problem was that each side would drop a different one, so neither side actually had a connection established. End result was no connectivity.

I said it before, and I'll say it again: a flag to disable uTP is not a solution. It's a stupid idea.

[Greg Hazel]

So now it wont allow tcp and utp connections at t the same time. (which is a good thing of course) - that will now mean that UTP will connect first and then TCP or will it fully randomize?

It will try both, and prefer to keep the uTP connection.

The problem of course is not really UTP - it is that other clients do not support UTP.

The problem is not really that other clients don't support uTP either. If there is a problem, it is in the way uTorrent balances between TCP and uTP (but I have yet to see a case which demonstrates the issue).

In the meantime there has been some talk of a private UTP flag where sites could turn off the UTp feature with the client supporting that "switch".

I don't mean to be rude, but this is not a solution and will not be implemented. Fixing the bug is better for everyone.

[DaVrOS]

The staff here still continue to say things like:

2010-02-21 02:02:22

DreadWingKnight

I never claimed to be nice.

Re: uTorrent 2.0 fail?

BitComet actually did their preferencing on purpose.

uTorrent is by accident.

So they admit there is preferencing. Weather it is by "accident" or is deliberate. That is not an ideal situation for a swarm with multiple clients in it.

It is also what we have been seeing. Can you see what I mean by the example if my first post where I quoted that downloader?

[DreadWingKnight]

And yet the less ideal situation of seedboxes pummeling residential broadband users in ratio still isn't being addressed.

[Greg Hazel]

The important note with seedboxes in my mind is that they do negatively affect the ratio system, as noted by everyone here, but are not considered to be a problem because generally they are good for the network as a whole. uTP is also good for the network as a whole, regardless of the impact on ratio distribution. It seems odd to point fingers at one of these but not the other. Both are good but the ratio metric fails to reflect this.

[DaVrOS]

Seedboxes are unfotunately something we cannot control now with 120,000 to 140,000 peers.

A single client - we can.

I have been keeping an eye on the peer count each day since some sites banned azureus and since we have not allowed the UT 2.0 < series and I will say there has been no noticable drop in peers - yet.

I think with the current fixes and problems that UTp has experienced so far (you guys have had to make new builds to fix issues already) is that it is still a relatively new and unknown technology that will need much streamlining before it can be adopted in a mainstream enviroment on ratio driven sites.

[Greg Hazel]

Seedboxes are unfotunately something we cannot control now with 120,000 to 140,000 peers.

A single client - we can.

So you would control seedboxes if you could? What about the positive aspects?

I think with the current fixes and problems that UTp has experienced so far (you guys have had to make new builds to fix issues already) is that it is still a relatively new and unknown technology that will need much streamlining before it can be adopted in a mainstream enviroment on ratio driven sites.

I think you're forgetting that ratio sites are not mainstream. The majority of BitTorrent users are using public sites. uTP is clearly ready to be adopted there, since it has been, and public swarms are working well.

uTP is relatively new to you, but it is not new or unknown to us. We've been using uTP for several years now, and it was researched heavily before we ever implemented it. Yes, we release improvements and fixes. uTorrent always receives improvements and fixes - it's part of actively developing software.

So the question is, when will ratio driven sites be ready for uTP? Is there some test or benchmark which you are waiting for uTP to pass? If you can provide one, we can get uTP working for you in extremely short order.

[DaVrOS]

No personally I would not change the seedbox aspect but the other staff here seemed to want to turn the topic towards that as a "comeback" for the UTp discussion so I felt it merited a response.

Ratio sites have possibly become more mainstream in the last year or so with the larger public sites such as Mininova changing its direction and having a traffic drop of approx 80%. ISOHUNT appealing a current courtcase that they lost. TPB being blocked by many countries, etc.

I guess the main criteria for any site is if there is a significant drop in peers due to a decision of allowing or not allowing something.

On the other side of things - it would be a decision based on how it is interacting with the non UTP clients and if is affecting them adversely - which currently it seems to be.

I know Alus suggested a possibly partial fix but I am not sure I understand it as you say that UTP and TCP connections ARE shared fairly.

his suggestion was:

<alus> what would be an acceptable solution here? leaving some % of the connections for TCP?

[Greg Hazel]

I know Alus suggested a possibly partial fix but I am not sure I understand it as you say that UTP and TCP connections ARE shared fairly.

his suggestion was:

<alus> what would be an acceptable solution here? leaving some % of the connections for TCP?

I am "alus", btw. My suggestion was to leave some connection slots open for TCP. However, given the (vague) descriptions I've read of the problem here, I'm not sure that would be sufficient. Specifically, the report is that TCP connections time out or "go away" after awhile. So reserving slots for TCP would not help.

[DaVrOS]

Sorry I should have remember you are Greg. I haven't used your email in while.

Would leaving some of the connection slots open for TCP be an advantage in this situation like I first mentioned.

A site member wrote:

Its not just seeding, its downloading too. I am using a server to download and was using a utorrent 2.0 full swarm just yesterday to download a torrent. Getting 1kb/sec download speeds because no-one is connecting to me due to 2.0 preferring itself is not good.

I had to wait ages for 1 person in the entire swarm who wasn't using 2.0 to connect to me before I could download anything.

He was obviously not using uTorrent 2.0 or larger as his client.

from what I can see his last clients used were

uTorrent/1770

and

rtorrent

[Greg Hazel]

Would leaving some of the connection slots open for TCP be an advantage in this situation like I first mentioned.

A site member wrote:

Its not just seeding, its downloading too. I am using a server to download and was using a utorrent 2.0 full swarm just yesterday to download a torrent. Getting 1kb/sec download speeds because no-one is connecting to me due to 2.0 preferring itself is not good.

I had to wait ages for 1 person in the entire swarm who wasn't using 2.0 to connect to me before I could download anything.

He was obviously not using uTorrent 2.0 or larger as his client.

from what I can see his last clients used were

uTorrent/1770

and

rtorrent

It's difficult to know. It seems highly unlikely that all uTorrent 2.0 peers were completely full on connections to each other without accepting/making connections to any other clients. Even if that were true, than a new uTorrent 2.0 peer who joined the swarm would have exactly the same difficulty getting any connections, uTP or not. So that wouldn't have anything to do with uTorrent at all. If the swarm was made up entirely of rtorrent TCP peers they could do the same thing, right? That seems like a highly unlikely scenario which is not new at all.

So, I don't think enough data was gathered to draw any conclusions here.

[Lord Alderaan]

The following rant is not on the technical issue but an attempt to address some of the emotional/communicative issues that might undermine the actual technical discussion. It is not intended to start a new discussion, at least not in this thread.

Even with hundreds of private trackers the majority of the hundreds of millions of BitTorrent users operate on public trackers. If I'd have to wager a guess the private trackers only service a few percent (maybe up to 10%) of the total BitTorrent userbase.

However the private trackers often contain the more experienced and innovative BitTorrent users and are able to set precedents that will affect the BitTorrent userbase as a whole. We've seen this with the grapevine effect.

imho this potential of private trackers to set a precedence actually reinforces the point that a disable-uTP flag is a bad idea. The issue it would 'fix' is only temporary because the µTP devs don't want the reported behavior either. They might be having difficulty determining what exactly is going wrong in the described situations but they are willing to fix whatever it is and have already addressed some possible causes.

And in any case such a flag is trying to solve the symptoms of a bug without trying to solve the actual bug. And in the meanwhile such a flag could actually be the downfall of µTP altogether. Obviously the devs don't want this, they believe in µTP and put a lot of work in it.

A lot of other people might think 'good riddance' but they aren't looking at the long term big picture. µTP is an technology that does have good reasons. And includes many advantages that also improve private swarms (such as: faster connecting, less overhead, less strain on (consumer) network equipment and address the only valid argument for ISP throttling)!

Maybe some of these people (no not you) are ignoring all that purely because they feel µTP is being 'forced' upon them. The fact is, it isn't purposefully being forced, it's the usual cycle of innovation. Just think back, almost all BitTorrent innovations of the last few years have been 'forced' in a similar way upon at least one group or another. Think of the Private flag, DHT, PEX, PE, RSS, HTTPS, UDP trackers, etc and how exactly each of these eventually became widely supported

Maybe µTP wasn't ready to be used in the wild yet, or maybe putting it out there will help greatly by quickly surfacing many issues that would have taken years of testing to find otherwise.

Who knows? And what does it matter now? Because the issue described above is obviously real. And the Devs seem to be unable to reproduce it or to explain it theoretically. This makes it really hard to track down and solve this issue. If the issue isn't solved by 2.0.1 then they are gonna need more detailed information.

So have patience, try to stick to the actual issue instead of other pretty much unrelated side issues. I hope this post is enough to put the most of these issues to rest. If not maybe it's a good idea to start a separate topic so that the technical discussion here is undisturbed.

colors added for readability, not emphasis

[Firon]

So, I was trying to replicate the problems TraderJones saw, but I couldn't get it to happen.

I tried to replicate a 1.8.x (TCP) client getting kicked off in low-peer situations when there's two or more 2.0+ clients. My 1.8.5 held onto the connection for an hour, before I just stopped the torrent.

I also was able to successfully connect with TCP to the uTP-enabled client, even with more than one 2.0+ client in the swarm. Maybe the problem just doesn't happen with 2.0.1 anymore.

This was tested with different machines and different internet connections with an isolated swarm (just the test clients).

[TraderJones]

Thank you for your followup Firon, I was wondering if any tests had been done... I've been buried at work and then got a nasty virus (physical not computer) that put me even further behind. I'll talk around with the sites I'm active with and see if we can open things up for 2.01 for a bit.