uTorrent 2.0 fail?

[Greg Hazel]

I do my best to stay out of this side of the conversation, but let's assume that you really don't understand/know what the main objection to uT2.0s behavior is.

Thank you for taking the time to detail things here. This really is new information for us.

uTorrent 2.0+ gives (extreme) preference to other clients that are running UTP to the point of disconnecting from non-UTP clients when a UTP enabled client (uT 2.0+ obviously) is available.

What Firon said is correct - uTorrent does not actively disconnect from non-uTP clients when other uTP clients are available. When given the choice between two connections to the same client, one uTP and one TCP, it will chose uTP. If you are seeing behavior that looks like uTorrent is disconnecting non-uTP clients, please enable Connect and Disconnect logging, and send us the log(s) involved.

If an uploader is using 2.0+ and 2 leechers download the torrent, the non-UTP client will rarely connect or be able to download until the UTP enabled clients are finished. I've deliberately downloaded torrents where there are only 2.0+ clients seeding/leeching with uT 1.8.2 and it can take the better part of an hour or even several hours before one of the 2.0 clients deigns to connect to me via TCP.

In an active swarm where there are multiple non-UTP clients and 1 2.0+ client things go fine until the 2nd 2.0+ client arrives. The 2.0 clients connect and fairly soon after that the rest of the swarm can't see them anymore. Really annoying when the lone 2.0 client was the original seeder and no one else has finished. All the non-UTP clients are basically SYOL until the UTP clients are done.

I've seen this happen and been part of discussions on multiple private trackers with the exact same issue raised. uT 2.0 gives preference to other 2.0+ clients to the point of exclusion for non-UTP enabled clients. Simply not acceptable in the eyes of many sites, no different than what BitComet did.

This description makes me wonder what happens when you try to connect to those uTorrent peers over TCP. Does it complete a connection and then disconnect from you, or does it time out? I believe the way that this is different from what BitComet did is that we did not specifically write in any sort of uTorrent preferencing, and will work with you to fix this problem.

TCP connections should be made *first* and given the higher preference

When the list of ips and ports are received from the tracker, it's not clear which will support TCP and which will support uTP. If you're suggesting that we go through each and try TCP before trying uTP, that could result in serious swarm startup delays. And if, as you described, the situation changes any time a uTP peer arrives, delaying uTP connections does not fix the situation, it only delays it.

In your defense you've done a lot of work with the UTP/TCP balancing that helps to some degree, but when we test it and followup on the behavior, we're still seeing the same preferential behavior.

We would very much like to work with you and see the behavior you're describing in action, to try to come up with a fix. If you are able to come up with a reproducible case or involve me in the observation of this happening, it would be very helpful.

[TraderJones]

It does NOT disconnect from non-uTP clients to give preference to a uTP client. That is just not true. It also doesn't exclude non-uTP clients from connecting.

But there was a connectivity bug where connections will sometimes fail when uTP is enabled (due to a race condition) in the initially released 2.0. Yesterday we made a new 2.0 to fix that particular bug.

I understand that the *intent* may not be to prioritize UTP, but my experience and the experience of many others testing the new versions is that the current behavior does appear to prioritize UTP connections. I've watched it happen, others have seen the same thing. When there are two or more UTP clients in a swarm, the non-UTP clients tend to lose their connection to the UTP enabled clients. This may be related to the above listed bug, only time will tell.

I just disagree with your statement that 2.0 does not disconnect from non-UTP clients. We're just seeing too much of that behavior to accept such a statement. I will however freely admin that these disconnects are likely not intentionally prioritizing UTP. But the net effect is to "de-prioritize" non-UTP clients in such a swarm.

[Greg Hazel]

It does NOT disconnect from non-uTP clients to give preference to a uTP client. That is just not true. It also doesn't exclude non-uTP clients from connecting.

I understand that the *intent* may not be to prioritize UTP, but my experience and the experience of many others testing the new versions is that the current behavior does appear to prioritize UTP connections. I've watched it happen, others have seen the same thing. When there are two or more UTP clients in a swarm, the non-UTP clients tend to lose their connection to the UTP enabled clients. This may be related to the above listed bug, only time will tell.

I just disagree with your statement that 2.0 does not disconnect from non-UTP clients. We're just seeing too much of that behavior to accept such a statement. I will however freely admin that these disconnects are likely not intentionally prioritizing UTP. But the net effect is to "de-prioritize" non-UTP clients in such a swarm.

We're not saying that you are not observing some sort of net effect. Clearly you are. However, uT does not disconnect from non-uTP clients due to protocol. It must be some other side-effect, for example the idle timeout.

[TraderJones]

I understand that what we may be experiencing is a "side-effect" rather than an intentional behavior, but it's still one of the most important driving factors for those sites that have either banned or are looking at banning 2.0+, right or wrong.

I'll see what I can do about log data. I personally rolled back to 1.8.3 for the time being. Most of the private sites I use have instituted a 2.0 ban, so I'd have to beg / code some exceptions to get good log info.

This description makes me wonder what happens when you try to connect to those uTorrent peers over TCP. Does it complete a connection and then disconnect from you, or does it time out?

Not sure I completely follow you here... which uTorrent peers? The 2.0? If that's what you were referring to, TCP connections to 2.0 clients from non-UTP clients/version sometimes take a while to get established (I've seen it take over an hour), but once the TCP connection starts the transfer is relatively stable unless another UTP client jumps in.

Since UTP has a small(er) packet size and (therefore) more control packets flying as well, could that somehow have the net effect of slowing the TCP to the point of a timeout? I don't know but I'm just throwing that out there.

[Greg Hazel]

I'll see what I can do about log data. I personally rolled back to 1.8.3 for the time being. Most of the private sites I use have instituted a 2.0 ban, so I'd have to beg / code some exceptions to get good log info.

Why 1.8.3 and not 1.8.5? Also, why not just uncheck "Enable Bandwidth Management" in 2.x? That will disable uTP.

This description makes me wonder what happens when you try to connect to those uTorrent peers over TCP. Does it complete a connection and then disconnect from you, or does it time out?

Not sure I completely follow you here... which uTorrent peers? The 2.0? If that's what you were referring to, TCP connections to 2.0 clients from non-UTP clients/version sometimes take a while to get established (I've seen it take over an hour), but once the TCP connection starts the transfer is relatively stable unless another UTP client jumps in.

Yes, I meant uT 2.0 peers. A TCP connection cannot take hours to complete. It either completes or times out in 30 seconds.

Since UTP has a small(er) packet size and (therefore) more control packets flying as well, could that somehow have the net effect of slowing the TCP to the point of a timeout? I don't know but I'm just throwing that out there.

No. If TCP was experiencing a timeout due to a full buffer, so would uTP. uTP specifically keeps the buffer mostly empty. I imagine if TCP is experiencing timeouts, it's because bt.tcp_rate_control has limited TCP down too far, but let's repro it before we make that assumption.

[rnr]

I guess I'm not helping the situation, I have utp only forced. You should do something similar to what you do for dht when a private flag is detected. So banning of utp supported clients isn't needed.

[Greg Hazel]

I guess I'm not helping the situation, I have utp only forced. You should do something similar to what you do for dht when a private flag is detected. So banning of utp supported clients isn't needed.

Fixing the issue they are seeing is probably just as easy, and benefits public swarms as well. Disabling uTP for private trackers is not the answer - uTP is a *good* thing, and should be used as much as possible. The bug is likely in our handling of TCP in mixed protocol environments.

[Firon]

Disabling uTP is just not a viable solution, neither as a default option nor in response to some flag set in the torrent.

The only thing to do is fix the remaining issues with uTP and TCP rate control. Several bugs have already been fixed in 2.0.1, and more are being worked on still.

[rnr]

Yes, 2.0.1 fixed slow uploading problem on my side.

[Lord Alderaan]

Interesting discussion :-)

Lets hope more tracker admins read this and instead of handing out judgmental statements like

The µTorrent crew has made their choice on how they want their client to act, so we have made our choice that it's not welcome on PWT.

they will move on to statements that are more enlightening to their users and to the admins of other tracker (reducing the grapevine effect). Statements that actually explain things; that vie for understanding; that explain both positions; that list actual underlying issues and that provide a glance at the future.

Something like:

µTorrent 2.0 has implemented a new improved peer-to-peer protocol (uTP) that unintentionally but effectively causes it to favor other µTorrent peers when run under Windows XP*.

This can cause seeding issues for all our members. As a result we have decided to disallow µTorrent 2.0

This decision is final but not permanent. We will re-evaluate when either:

A) other BitTorrent clients also start implementing the new protocol,

Windows XP is no longer widespread on this tracker

C) a workable solution is implemented

For more information see this topic on the µTorrent forums.

.

Then again you could also instruct those users that run Windows XP* to disable uTP. If I understand it correctly that instruction (if you trust your users to actually do so) combined with the fix in 2.0.1 that should solve all the issues.

* For Vista users that haven't installed SP2 the solution is simply "INSTALL SP2". Therefore I do not blame µTorrent for any issue that only happens in very outdated Vista systems.

[TraderJones]

Why 1.8.3 and not 1.8.5? Also, why not just uncheck "Enable Bandwidth Management" in 2.x? That will disable uTP.

Disabling uTP for 2.0+ won't help when the tracker has banned 2.0 in general... I dropped back to 1.8.3 because that was the last stable version that did not include uTP (at least as far as I know) and was also the version I was using prior to looking into 2.0. I'm not adverse to running 1.8.5, but I already had my 1.8.3 setup archived and it was "easier" just to rollback to that.

Yes, I meant uT 2.0 peers. A TCP connection cannot take hours to complete. It either completes or times out in 30 seconds.

Let me phrase this differently and see if it helps. If two (or more) 2.0 clients are already seeding/leeching on a torrent and I download the torrent I will not see the 2.0 peers in my peer list for up to 30 minutes. Even once they appear in the list I won't get an active download for quite a while longer (as I said over an hour in some cases). I have noticed that if there are 2 peers and they finish the upload/download I then start downloading in a very short time. Does that make sense? I realize it's a very "subjective" description of the issue, but I don't know how else to describe it. I'm not saying the TCP connection is taking hours, I'm saying the 2.0 clients appear to take hours to "become available" for a TCP connection? Not sure if that helps either.....

Then again you could also instruct those users that run Windows XP* to disable uTP. If I understand it correctly that instruction (if you trust your users to actually do so) combined with the fix in 2.0.1 that should solve all the issues.

hmmm, so you're saying that you think this is an XP issue only? I can't really speak to that as I don't have Vista on my puter running uT.

for Greg, here's the results of testing done by one person on a private tracker, done around the end of Jan, so it doesn't reflect recent changes:

Here's a txt that xxxxx posted on a test he had ran...

I created a max of 20 connections

and did 40 connections

20 instances of rtorrent

20 instances of utorrent 2.0

and on another box, a utorrent 2.0 seeder

NONE of the rtorrents connected to the 2.0 Seeder

and only ONE rtorrent slot could connect to the other 20 utorrentsi n the swarm

when I turned uTP off .. it was a pretty equal between the rtorrent and the utorrent

but with uTP on, the TCP connections didn't have a chance.

the other utorrent instances favoured each other greatly, though and 18 out of the 20 had 1 rtorrent peer with the other two having 2 rtorrent peers

which rtorrent was random

tcp .. equal spread across all of them

[DreadWingKnight]

I dropped back to 1.8.3 because that was the last stable version that did not include uTP (at least as far as I know

It had uTP.

[Switeck]

uTorrent v1.8.3 had...very buggy support for uTP, and would accept ANY incoming uTP connections by default. It could even be configured to make predominately or ONLY outgoing uTP connections.

[Firon]

for Greg, here's the results of testing done by one person on a private tracker, done around the end of Jan, so it doesn't reflect recent changes:

Here's a txt that xxxxx posted on a test he had ran...

I created a max of 20 connections

and did 40 connections

20 instances of rtorrent

20 instances of utorrent 2.0

and on another box, a utorrent 2.0 seeder

NONE of the rtorrents connected to the 2.0 Seeder

and only ONE rtorrent slot could connect to the other 20 utorrentsi n the swarm

when I turned uTP off .. it was a pretty equal between the rtorrent and the utorrent

but with uTP on, the TCP connections didn't have a chance.

the other utorrent instances favoured each other greatly, though and 18 out of the 20 had 1 rtorrent peer with the other two having 2 rtorrent peers

which rtorrent was random

tcp .. equal spread across all of them

This test was done with a VM, if I recall correctly. All VMed instances started connecting at exactly the same time (and I mean down to the microsecond). This is not realistic. This never happens in the real world. And naturally, if you start at exactly the same time and have equal pingtimes of 0ms, uTP will get the slot faster than pure TCP, since it has less roundtrips to do before a connection is established. This is basically all that's going on here.

But that test is completely worthless because it never happens in the real world.

[Archatos]

Lets hope more tracker admins read this and instead of handing out judgmental statements like ....

they will move on to statements that are more enlightening to their users and to the admins of other tracker (reducing the grapevine effect). Statements that actually explain things; that vie for understanding; that explain both positions; that list actual underlying issues and that provide a glance at the future.

Something like:

µTorrent 2.0 has implemented a new improved peer-to-peer protocol (uTP) that unintentionally but effectively causes it to favor other µTorrent peers when run under Windows XP*.

So instead of stating what we're actually seeing we should give some sort of advertising for µTorrent containing false information? It does not only happen in XP, so why should that be put there? Why should we ignore that even after discussing the issues at length with other tracker admins, the µTorrent crew is here saying they've never heard about any of these issues? Why should we ignore that µTorrent representatives even still blame how private trackers work instead of admitting that µTP is causing issues?

What is being seen, is that there is either a complete lack of internal communication, or flat out bullshitting going on here. Whichever it is, it is something that can't be sorted by anyone other than the µTorrent crew themselves.

I have no problems understanding why BitTorrent Inc is trying to sell µTP as the saviour. What I will never accept is the miserably failed attempt at treating tracker admins like idiots. But of course, this will just be replied to with another politically correct remark, because God forbid that anyone here will admit that they've been trying to pull a fast one so they won't look weak infront of the fan crew.

For µTorrent to become a viable option again for private trackers, either other clients need to be convinced to implement µTP, or a way for private trackers to remove it through the tracker code will need to be supported. Both of the options are completely up to the µTorrent devs.

Asking us to do their job for them is not going to work when there are plenty of other good alternatives out there.

Again, the issues have been explained in detail multiple times. Hence "discussing" it with someone who have shown now multiple times in this thread alone that they are not willing to even attempt to understand is pointless. The information posted by TraderJones is far from new to the µTorrent crew. The fact that they act like it is pretty much speaks for itself on how willing they are to work on it.

[Greg Hazel]

Why should we ignore that even after discussing the issues at length with other tracker admins, the µTorrent crew is here saying they've never heard about any of these issues? Why should we ignore that µTorrent representatives even still blame how private trackers work instead of admitting that µTP is causing issues?

We're covering new ground in this thread, and I'm hearing about aspects I did not know of previously. I'm not sure where these other discussions occurred, but I was not part of them. I'm not blaming private trackers for the problems seen with TCP balancing.

For µTorrent to become a viable option again for private trackers, either other clients need to be convinced to implement µTP, or a way for private trackers to remove it through the tracker code will need to be supported. Both of the options are completely up to the µTorrent devs.

Disabling uTP for private trackers is not the answer. Fixing the bug is the answer. That will benefit public swarms as well, and uTP will benefit private trackers when it works with TCP to their satisfaction.

Again, the issues have been explained in detail multiple times. Hence "discussing" it with someone who have shown now multiple times in this thread alone that they are not willing to even attempt to understand is pointless. The information posted by TraderJones is far from new to the µTorrent crew. The fact that they act like it is pretty much speaks for itself on how willing they are to work on it.

TraderJones has presented new interesting information. I have yet to see or reproduce any of it first-hand.

Archatos, I can see that you're distrustful of us. I don't know what to say except we are genuinely trying to reproduce and solve this problem.

[TraderJones]

One correction, I just double checked and I'm running 1.8.2 if that makes any difference. I coulda sworn that I had updated to 1.8.3, but apparently not.

Firon, that particular test may be "unrealistic" but it does give a methodology that could be altered to test exactly the situations I've described. Just don't start the clients all at the same time and from more than one physical location and see what happens. I personally don't have the resources to do such a test, but I would think that the uTorrent dev team would..... I had more than one reason for adding it to that post.

Archatos, I agree wholeheartedly with your feelings, but even so I felt someone should make the attempt (again) to clearly state the (objectionable) behavior. If nothing comes of it, so be it. LOL, I live in the world of "politically correct responses" as opposed to "addressing the real issue" so I have a lot of experience ignoring that part and still stating my piece....

[Greg Hazel]

Firon, that particular test may be "unrealistic" but it does give a methodology that could be altered to test exactly the situations I've described. Just don't start the clients all at the same time and from more than one physical location and see what happens. I personally don't have the resources to do such a test, but I would think that the uTorrent dev team would..... I had more than one reason for adding it to that post.

You mentioned seeing an issue with a smaller swarm - 3 peers, right? I'd like to limit the resources required as much as possible to isolate it. How about this test plan:

- Start uT 2.0.1 seeder

- Start uT 1.8.5 peer

- Wait for them to transfer a bit

- Start uT 2.0.1 peer

If I understand your previous post correctly, we should see the 1.8.5 peer eventually stall and get disconnected, until the 2.0.1 peer is complete. Is that right?

[Firon]

Let's go back for a moment. It seems like everyone I've talked to is basing their evidence of uTP being bad on that VM test that was done last month in completely unrealistic conditions. Almost everyone else is just spewing out the same damn info. Aside from a few trackers here and there (who have given REAL concerns and shown real problems), most of the tracker staff have never tested a god damn thing themselves and just keep repeating the same old crap. This does make you look like a complete idiot.

The main issue being presented here is one that just shows the underlying flaws of most private trackers who are purely ratio based. Being able to connect faster and get slots at the beginning of the torrent gives you a ratio advantage because your system is broken, where only the fastest can ever get any ratio. The argument is summed up as "don't use uTorrent, it's fast. Use something slow because it's fair to everyone else". It's utterly fucking retarded and it's almost always the scenario given as the reason for banning 2.0.

Now, for the few trackers who are talking to me directly and giving me real problems and data (you are not one of these, Archatos), they are being fixed. TCP rate control has problems and completely screws speeds sometimes. There was a 60 second problem where uTP would screw itself over every 60 seconds (and you'd see it in the speed graphs). There was a connection bug where 2.0 clients would often leave themselves in a state where both TCP and uTP connectons failed. These have been fixed, and the other issues are still being fixed.

The crew here is perfectly willing to work with trackers, and we often do. Just ask Davros. But don't give me any bullshit about us ignoring 'the issues' and pretending we haven't heard it.

Now, TraderJones is bringing up a different issue which I haven't seen. But there may very well be a real problem, since 1.8 clients (or any client connected with TCP) absolutely should not be getting disconnected for no reason, especially in a small swarm.

Would you be willing to join IRC? It's probably faster than trying to do this on the forums. irc.p2p-network.net #utorrent

[TraderJones]

Sorry, somehow I hit report instead of reply....

I'm not basing my comments on the previous test Firon, I simply quoted it as an *additional* situation where someone else found that uTP appears to get prioritzed. I spend half my working day analyzing the "behavior" of another companie's poorly implemented software trying to figure out why we get bad data, so I never base my opinion strictly on my own, or one other person's experience. I have the good sense to know my own limits and blind spots and see what issues other people/organizations are reporting and how they may or may not fit with my own experience. Tossing off an expletive laced denouncement of concerns being raised by ratio based trackers doesn't help make anyone feel you are the least bit interested in helping.

Greg, The test scenario you described is one of the two situations that I personally have seen occur. The other is when a pair of 2.0 clients are connected and a 1.8 (non uTP) downloads and starts the torrent there is an extended delay before either of the 2.0 clients will connect and send data to the non uTP client. I'll be interested to see if the recent changes mentioned have made a difference here. Thanks for taking the time to take a look.

[prodigydancer]

Oh my... The amount of private tracker elitism in this thread made me sick enough to register. Very, very rarely - even in an Internet discussion - one can see such an amount of arrogance coming from people who don't even know what they're talking about.

And, as usual, private trackers admins speak like the BitTorrent community has given them the rights to represent the whole of it. This is further from truth than anything could be. Actually the larger and the healthier part of the community just despises them for being egotists they are.

Usually I think that user feedback is very good as it helps software developers but in this case "feedback" is a heap of misleading spam and "users" who are providing it are just a couple of narrow-minded trolls who only care about themselves.

---

On topic:

I'd like to thank the developers wholeheartedly. uTorrent 2.0 is as excellent as uTorrent has ever been, and uTP is a welcome new technology. I'm trying to recall another piece of software that has never failed me in years... and nothing comes to mind. (OK, maybe Nano text editor is close enough.)

[Firon]

Sorry, somehow I hit report instead of reply....

I'm not basing my comments on the previous test Firon, I simply quoted it as an *additional* situation where someone else found that uTP appears to get prioritzed. I spend half my working day analyzing the "behavior" of another companie's poorly implemented software trying to figure out why we get bad data, so I never base my opinion strictly on my own, or one other person's experience. I have the good sense to know my own limits and blind spots and see what issues other people/organizations are reporting and how they may or may not fit with my own experience. Tossing off an expletive laced denouncement of concerns being raised by ratio based trackers doesn't help make anyone feel you are the least bit interested in helping.

Greg, The test scenario you described is one of the two situations that I personally have seen occur. The other is when a pair of 2.0 clients are connected and a 1.8 (non uTP) downloads and starts the torrent there is an extended delay before either of the 2.0 clients will connect and send data to the non uTP client. I'll be interested to see if the recent changes mentioned have made a difference here. Thanks for taking the time to take a look.

That previous post wasn't really directed at you, since it was clear early on that you had issues you were experiencing through your own use. Sorry, I should've been clearer that it was directed to certain other people.

[Lord Alderaan]

@Archatos:

I'm sorry, now that I read my own post back I come of patronizing. I didn't intend that.

What I meant with the XP thing is that from the rest of the topic I understood that uTP has the advantage if TCP has to observe a max half-open connection limit. And µtorrent only implements such a limit when it is run on Windows XP because of XP's OS level limit.

I also understood another potential cause is (arguably) fixed in 2.0.1

What I meant with my whole post is that private trackers that do apply the Ban Hammer to µTorrent often do not make their reasoning clear. This leads to confused users and other tracker admins that blindly follow the decisions of their brethren because they trust them and not because they evaluated the issue themselves.

Even simply linking to topics like this in statements like that would make everything easier for everyone

I do want to say that you shouldn't speak as if for the whole private tracker community. Your claim that µtorrent is no longer a valid private tracker client is obviously false. µTorrent has been through much serious issues (LPD disrespecting the Private flag comes to mind) without losing momentum. This is just a minor issue that the devs are obviously willing to look at.

I'll stop derailing the thread now

[eliot_cougar]

Hmmm... Was it good when I actually disabled TCP connections after switching to uT2.0?.. I set bt.transp_disposition = 10... Everything goes nice, I'm having almost maximum theoretical speed all the time and able to use browser without stopping torrents...

[Lord Alderaan]

No! That is a really bad idea! Because you are now only connecting to other clients that support µTP which at this time is only µTorrent (from 1.8 onwards)!

Even if YOU still get your max speeds etc, you are denying other people your bandwidth. And BitTorrent is about working together indiscriminately.

Besides in private swarms you are denying specific people the opportunity to maintain their ratio which will make those people and probably the admins angry.