uTorrent 2.0 fail?

[Firon]

I'm gonna try to find the time to perform the test again with a few more clients to make sure it doesn't happen with a larger number of clients involved. I'll try it with all of them being 2.0.1 but one.

We're going to be pushing out a new 2.0 ~today with a fix for the new uTP header (we discovered a bug in it). 2.0.1 18408 already has the fixed header, but we're pushing it to 2.0 as well just so we can have an established base of clients with support for it.

[TraderJones]

Will the updated 2.0 have the fixes that 2.0.1 and 2.1 have been updated with for the TCP vs uTP connections?

[Firon]

Not most of them. Some are fairly substantial changes we don't wanna introduce into 2.0.

[davechr]

Scenario;

-XP user has the default tcpip.sys (10 half open connections)

-The uTorrent default of 5 connections per second is used

-uTorrent randomly tries 10 unconnectable peers that only support TCP in the first 2 seconds

-tcp connections time out after 30 seconds

Will uTorrent 2.0+ only connect to uTP peers for the next 28 seconds, or will it wait for the TCP half opens to close before trying more uTP based connections? Could the be causing the imbalance private trackers are complaining about?

[rafi]

@davechr:

1. As you've implied yourself, this scenario is not relevant to non-XP systems that are now growing in numbers, and do not have this limitation (and neither does uT when installed on most of them)

2. Many (if not most) XP/P2P users have patched their systems to increase the limit, and can setup uT accordingly to 60 half open connections (though, it's not recommended)

3. It is also statistically not very common to have many completely unresponsive peers for a long time. If only some of them respond, the connection attempts will resume

4. The same capability is available also in 1.8x/pre-2.x uTorrent clients if only the user set the bt.transp_disposition setting like in 2.x (=15)

5. Would you set up uT to also limit TCP connections in Vista or Win7 as well ? ban Win7 & Vista users for being "unfair" to XP users ? Ban also 1.8x clients because of this possibly setting ? would you add an option to also hold up ALL uTP connections after 5 unresponsive uTP connection attempts for a similar timeout (and at the same time keep TCP attempts going in Vista/Win7) ? Should technical improvements be avoided as long as there is a single Win98/XP user around ?

And to answer your question - yes. Here is a log when all PC traffic is intentionally blocked with a firewall:

http://pastebin.com/fAR8Nzf1

Regardless of any reservations raised here - uTP issues should and probably are being attended to in the upcoming 2.0.x betas & release, and I hope will make it better than TCP or at least as good as TCP in speed, overhead and congestion control.

[davechr]

I'm aware the probability of this scenario is quite low, but given enough end users with enough torrents over enough time it wasn't out of reach. It should also be noted, that with larger files this would eventually iron itself out as the client has at least tried each peer once, or has reached maximum peers.

I would not do any of those things, and in addition I would not ban unconnectible peers either because they also hurt the swarm in terms of connection rates. It would seem banning a client that supports uTP also hurts the swarm because that client is increasing connection rates. Unfortunately it increases connection rates only to other uTP clients, which creates bias towards uT2.0 (if maximum peers are reached) since its the only popular implementation of uTP.

"Would you add an option to hold up ALL connections also after 5 unresponsive uTP connection attempts for a similar timeout"

There is value in that question. This could resolve the slight bias until uTP is more widely adopted. Since its statistically improbable to attempt connections to so many unconnectible peers, and the problem already exists with the previous TCP-only implementation, in a real world scenario no one would notice either way. However, this negatively affects the swarm where an end user just wants as many peers as quickly possible, so I can see the argument against this implementation. This would also be really painful to the innovation of uTP overcoming the half-open limit, and personally I don't think uTorrent should be accommodating XP users with an unpatched tcpip.sys at the cost of the rest of the swarm.

Thank you for the log. I figured as much but couldn't easily test.

[Lord Alderaan]

@davechr:

About the specific scenario you described, we agree that TCP is limited by both bt.connect_speed and net.max_halfopen while µTP is only limited by bt.connect_speed right?

In 1.8.5 (with default settings) µTorrent makes no outgoing µTP settings. And the default bt.connect_speed is 20. Technically meaning max 20 TCP connections per second. But the net.max_halfopen limit kicked in frequently.

In 2.0 µTP (with default settings) µTorrent makes both outgoing TCP and µTP connections. And every time the net.max_halfopen limit would kick in for TCP connections µTP would still be allowed to make new connections at the bt.connect_speed of 20 per second. And this is where the (unfair) advantage for µTP would lie.

However in 2.0 the default bt.connect_speed has been reduced to 5.* This limit will be reached a lot faster (also because both TCP and µTP connections are made simultaneously). And as a result TCP will run into the net.max_halfopen limit a lot less and when it does µTP will still be limited by the bt.connect_speed limit of 5.

So the advantage of µTP (in this respect) is already kept in check. I agree it is still not a completely level playing field. But the difference should be negligible.

And this all only applies to µTorrent run on Windows XP machines. On Vista/7 the default net.max_halfopen in 2.0 is 100 in and this basically completely levels the playing field.

On top of that it is only in private swarms where this issue could actually be noticed at all (because of the ratio system).

So I hope you agree the µTP restriction you want should only be imposed on torrents with the private flag set listed in µTorrent running on a XP box, right?

Even without such a restriction, private trackers (with many XP users) do have other options then outright banning µTorrent 2.0. Because luckily many peers on private trackers that still use XP (which are often XP fans or people who run µTorrent on second older machine) are tech savvy and either already patched their tcpip.sys and increased the net.max_halfopen limit** or can be instructed to do so using the many means a private tracker has to communicate with its peers.

In any case, the actual major issue described higher in this thread (and which is the actual cause for most of the bans) is likely the direct result of actual bugs in µTorrent/µTP. And not the direct result of how µTP is implemented. Hopefully when 2.0.1 beta goes stable it will already address this issue, without having to artificially restrict µTP or changing the implementation of µTP

P.S. This whole post is imho

* I know you already know it is 5 now but I don't know if you know that it was 20 before. And that the scenario you describe has already been reduced in likelihood by this reduction.

** Even though it has been recommended by µTorrent devs and veterans not to do this for many years

[davechr]

I agree the situation I described in my scenario is negligible, and like I said, I personally don't think utorrent should impose restrictions on utp connection rates if tcp is timing out or vice versa.

"the actual major issue described higher in this thread (and which is the actual cause for most of the bans)"

Maybe I haven't read this thread enough, or I don't know the change logs well enough... The race condition fix, uTP header size and packet size fixes, and the negligible advantage I've described, don't seem like they would affect a private tracker. Is there another (more major) issue that was fixed?

[mtakala]

Some trackers (or at least one) are also reporting strange announces, as in a not-so-fast broadband user's announces to the tracker are telling that the user has uploaded a ridicilous amount of upload (as in >200GB) in a day, when that is not physically possible. That has resulted in uTorrent being banned on that tracker from 1.8.4 onwards.

The developers have been told about this and shown the data. I wonder what the bug is and whether it is already fixed in some version?

[moogly]

Many tracker templates are broken or not well configured and report bad stats. And whatever the client used by the member.

[Lord Alderaan]

@mtakala:

Are you sure it isn't a cheater application that is spoofing µtorrent client IDs?

And as moogly said, did you detect the discrepancies in the actual announce (monitored with for example wireshark or found in detailed http-daemon logs) or did you just look at the info that was in the tracker database? Did you narrow down specific conditions client-side for when this happens?

@davechr: Since the devs are unable to actually reproduce the major issue (extreme favoritism on private trackers) atm I'm hoping that the fixes that are already done (the ones you mentioned) and the fixes that are to follow before 2.0.1 goes stable will solve the issue.

Hence my use of the word 'likely' and 'Hopefully' in the last paragraph of my previous post

[mtakala]

@Lord Alderaan: It was reportedly seen in internal testing by the tracker staff also. And the behavior clearly changed with some version of uTorrent.

[Greg Hazel]

@Lord Alderaan: It was reportedly seen in internal testing by the tracker staff also. And the behavior clearly changed with some version of uTorrent.

If there is a tracker stats reporting issue, can you please make it clear how to reproduce it, or post Wireshark logs of it occurring? We are not aware of any problems in this regard at the moment.

[mtakala]

Greg: This is a direct quote from the forum of the site I'm talking about:

"We showed one of the developers from uTorrent what their client is sending to our tracker and also what their older versions send and he was very surprised by the differences he saw."

Firon is member on that site.

[Greg Hazel]

Greg: This is a direct quote from the forum of the site I'm talking about:

"We showed one of the developers from uTorrent what their client is sending to our tracker and also what their older versions send and he was very surprised by the differences he saw."

Firon is member on that site.

When was this quote from?

[mtakala]

Greg: March 7th, 2010 so about 4 days ago. I don't know about the timeframe the site moderator is referring to in his post.

[Firon]

Greg: This is a direct quote from the forum of the site I'm talking about:

"We showed one of the developers from uTorrent what their client is sending to our tracker and also what their older versions send and he was very surprised by the differences he saw."

Firon is member on that site.

If you mean TB, then no such problem was ever found there. The only issue we found there was just that their logic for downloaded and left behavior just needed updating. No legitimate uTorrent reports bogus stats like what you are saying.

[mtakala]

OK, my apologies for derailing this thread for this issue.

[badschandau]

Almost 3 out of 5 web surfers still use XP.

http://www.w3schools.com/browsers/browsers_os.asp

Unlike Lord Alderaan, I believe that the vast majority is unpatched.

Edit: Even among the utorrent users at private sites (of which I don't know the XP-share).

[Lord Alderaan]

The composition OSes on public swarms doesn't matter. And the composition of OSes used on private tracker communities is a lot different.

And I never said most windows XP users at private sites are patched. I said they are either patched or can be instructed to patch now.

Windows XP is two OS version ago and almost a decade old. I think it is fairer to expect people (on private trackers) using this outdated operating system to patch then it is to expect µTorrent to hold back on innovation because people are still using it.

The only other option I see is to just disable µTP on XP by default (exactly like how µTorrent sets the net.max_halfopen limit to 8 instead of 100 on XP by default).

[rafi]

there is no reason why XP users will not be "upgraded"/patched to Vista/Win7 level

[Firon]

Microsoft is only providing security updates for XP. It's no longer getting actual changes.

[kylling]

Is there a way to get utorrent 2.0 to report itself as a lower version to the tracker? Like the way you can tune most web browser to report itself as IE to a web server.

[DreadWingKnight]

No, and there won't be.

[bbbl67@yahoo.com]

The only other option I see is to just disable µTP on XP by default (exactly like how µTorrent sets the net.max_halfopen limit to 8 instead of 100 on XP by default).

Maybe there should be a disable uTP on the private flag? That way, when you see a private torrent, it'll not only disable DHT, Peer Exchange, and Local Peer Discovery, but also uTP?

I know uTP hasn't got anything to do with the other three things, but maybe it'll make those private torrent site admins feel better? I mean if you're going to be going around patching the operating system components and other things just to make private trackers happy, this solution is as good as anything else.