Torrent-friendly ISPs frown at uTorrent and uTP

I've been reading ISP engineers technical forum, and it has a big thread on how to block uTP connections, which is full of hatred towards uTorrent developers and uTP.

And no, these are not those crappy providers which hate torrents at all and want to block all P2P activity of their users. They are normal ISPs which want to provide the best service possible to their users, including torrents.

The problem is just the uTP protocol, which GREATLY increases PPS (packets per second) rate at the same BPS (bytes per second, "speed") rate due to small packet sizes.

The matter is that it's the PPS rate which determines the load on the ISP equipment. The engineers say when the uTorrent 2.0+ was out and installed by lots of users the ISPs equipment got greatly overloaded which caused failures, glitches and delays of service. That was not because of improved file exchange and increased user traffic, but just due to the decreased size and increased number of packets.

That means uTP is a harmful thing that produces lots of unnessessary problems to ISP. And they sure take measures to counter it.

I just wanna ask the developers: what is this stupid war for? It just harms us users. Is it possible to work out the protocols together with ISP engineers so the protocols would both carry the P2P traffic the most effectively and don't impose the unnessessary load on the network equipment?

And what do they say in this big thread?

Do you want me to copy-paste all the flame here?

I gave the short summary in the first message, the rest is the flame and technical discussions/data like firewall signatures to block uTP traffic not affecting all other UDP traffic. Also, suggestions like measures to wipe out uTorrent from BitTorrent clients list - like to block every packet with "uTorrent" in it, explaining the situation to the users suggesting the alternate BitTorrent clients, and so on.

we want a link to it ... also, uTP packets' sizes decreased in size, and not increased...

Flaming words are not really important but the technical details they propose to block/limit µTP can be interesting for the µT devs (for future discussion with ISP e.g.) and improving µTP.

It's like these private trackers arguing uTP is cheating and gives some 'advantages' to µT users, when you ask a proof, it's so rare to obtain it.

Well, the link is not a secret, here it is:

http://forum.nag.ru/forum/index.php?showtopic=55025

The problem is this forum I read is in Russian so I guess this will be little help.

uTP packets' sizes decreased in size, and not increased...

This is exactly the problem. The LESS a packet' size, the MORE packets you need to carry the same volume.

As an example, let's say you need to carry water from one pool to another. You have a tea cup and a bucket for that. What would you prefer to use if the weight difference between a filled cup and a bucket doesn't matter?

The ISP routers are machines for which the weight of containers doesn't matter much, the QUANTITY of containers (packets) is what really matters.

What uTP does is make ISPs carry water in tea cups, and you can guess they're not very happy about that.

2.0.1 does increase the packet size compared to 2.0.

by ? 2% on average ? ... completely negligent effect on total overhead . Plus setting "4" as default minimum INITIAL size is being completely overridden by the speed limiter misbehavior

Is it a problem with any bulk data packets being 600-1000 bytes in size, or only if the packets are smaller than that?

the problem is the number of packets per time unit, not their size.

as the original poster explained twice.

Packet size must matter -- if it's "ok" to have max rate max size (~1500 byte) packets, then 300 byte packets would only be 5 times as many per second all other things held equal.

600-1000 byte packets would be 1.5-2.5 times as many per second, but still significantly reducing time between packets. In a mixed-mode (TCP and uTP) torrent environment, smaller uTP packets are pointless.

You can't get very high packet-per-second upload rates with max packet sizes on typical ADSL2, because they don't have more than 1 megabit/sec upload. And with many people unable even to GET those speeds due to poor quality lines and distance from the DSLAM, this is even less of an issue.

If these ISPs cannot handle the max packet size at roughly max rate during OFF-peak (read: not evening hours from ~5 PM to ~11 PM) then the ISP is plainly torrent-hostile, though this may be by their network design issues rather than pure meanness.

Over the last weeks I've read here that if anyone is having problems with uT 2.x then:

- their tracker is at fault;

- their router is at fault;

- their OS is at fault;

- the user is at fault.

Now we hear that it is the ISP's fault, that they are 'torrent-hostile' and do that out of meanness, even if the performance problems don't happen with other clients or when uTP is disabled.

The way that any negative report on uTP is dismissed here reminds me of certain totalitarian regimes.

You're running out of excuses, maybe next we'll hear that uTP doesn't work because this world is bad.

the problem is the number of packets per time unit, not their size.

the two are correlated. If they were not small, and the overhead was not so hight - there wouldn't be so many of them per second ...

any negative report on uTP is dismissed

you can view many of my own reports on size and overheads they are not dismissed. just treated with low priority (that is - ignored for now... ) . I guess you can say that the devs wanted to experiment in the wild, and see if the world's Internet congestion was reduced. This test results are still pending...

The way that any negative report on uTP is dismissed here reminds me of certain totalitarian regimes.

I said, "In a mixed-mode (TCP and uTP) torrent environment, smaller uTP packets are pointless."

...Strongly implies I am not defending the use of tiny uTP packets.

I was asking if 600-1000 byte packet sizes were acceptable/tolerable.

But an ISP does deserve some blame if they cannot handle their OFF-peak loads.

As tim716 said, there is a problem with packets-per-second.

This problem affects ISP's devices and also subscriber's devices (soho routers with nat and/or firewall, for example).

Also there is problem with UDP itself. When routing/NATting TCP traffic, device puts record into connection tracking database only once, then uses established state. When UDP - it tracks every packet (not all devices, but many of them, depends on model and network stack). This cause, for example, hanging/freezing of soho routers, adsl or cable modems etc.

For ISP hardware situation is the same in many cases. Not all hardware designed for UDP traffic overload. This cause degradation of quality of service for all clients, which even doesn't use bittorrent. Some ISPs have to setup filtering for uTP packets to save network functionality (depends on laws, in Russia and Ukraine filtering is allowed).

Devs just can't win.

Users ask for years to have UDP support added to uTorrent, and are denied due to lack of standards and the problems a lot of network equipment has with large amounts of UDP traffic. It finally gets added in the form of uTP, and users scream bloody murder about having to change their settings, and about the problems their network equipment is having with large amounts of UDP traffic.

Just can't win.

tim716 never clarified at what packets-per-second level there was a problem -- and I am left wondering if he meant PPS of an ISP aggregate through a critical gateway *OR* PPS at the individual user level.

Much of those posts in the Russian forum he linked to are reacting to uTorrent v2.0's initial "stable" release, which had immense problems with using way-too-small packet sizes when it shouldn't even by uTorrent developers' own criteria!

If users' routers or software firewalls are crashing from the load, at least that would spare ISP equipment from a similar fate. But doesn't seem to be what's happening.

User's routers and firewalls in typical networks - have some % of users; other are connected directly. And, of course, most of users with SOHO routers thinks that troubles with their hardware are caused by provider. This is 1st.

2nd, after enabling uTP, there are saw-like traffic pulsations on inbound/outbound channels, with period 15-20 minutes (even if channel and hardware have enough reserve). This behavior also spotted by different users - they have saw-like speed graphic.

3rd, there is trouble with fragmentation - so there are fragmented UDP packets.

4th, most NICs and SOHO routers have hardware support of TCP fragmentation offloading, so 50K datagram will pass through them transparently as 1 big packet, and uTP will cause 30-100 UDP packets (so load will increase by 30-100 times, which is critical for 100...300MHz SOHO router CPU); also SOHO routers have hardware NAT offloading for TCP - but I doubt that they have UDP NAT offloading (not much time ago it was really unneeded - UDP flow hasn't high bitrates, and UDP NAT algorithm is slightly different.)

Also, it looks like some ISP-level hardware (L3 switches, etc) have troubles with UDP transfer at high rates (for ex., look at comment http://forum.nag.ru/forum/index.php?showtopic=55025&view=findpost&p=490674)

It's strange that uTP uses TCP-like behavior - ACK for each packet rather than sending big 'multipacket' (for ex., 64 packets) and waiting ACK for this 'multipacket' with per-packet receive confirmation, and then - send lost packets in next 'multipacket' with new packets.

IMHO at least it'll be good if uTorrent will try to check both UDP and TCP protocols, and select best one (or TCP if results become approx equal) for connection. And, of course, it'll be preferred if providers will have ability to configure uTorrent behavior in their networks - for ex., by config server in zone .local (in any way, there is enough technical possibilities to disable some features that cause headache in both users and admins; but I expect that radical actions - like enforced user migration to other client - will remain only in minds and forums' flame).

You can't get very high packet-per-second upload rates with max packet sizes on typical ADSL2, because they don't have more than 1 megabit/sec upload. And with many people unable even to GET those speeds due to poor quality lines and distance from the DSLAM, this is even less of an issue.

may be You think that Russia and other exUSSR countries are poor. Yes, that's why in our country poor quality lines and their quality so bad, that many people can have 64 or 128Kbit ADSL Internet. But this poor quality lines gave future for the ETTH networks, and now Statistics of UA-IX http://www.ua-ix.net.ua/57.0.0.1.0.0.phtml in top80 Internet traffic exchange network UA-IX is 20 and MSK-IX is 16.

Now many (10-15 million who have Internet people that onli in Ukraine) have high quality ETTH Internet with speeds from 2/2Mbit(in vary small city) to 100/100mbit(majority)...http://www.speedtest.net/result/750273613.png

Now average speed of Internet is 20/20Mbit...

Yes in Ukraine there are ADSL(1M/512K,2M/512K,4M/512K,8M/512K), but Number of people who use it is less then <10% it's the official statistics.

Every year number of Internet users are increase at lest on 25%, and speed increase on 50%.

My ISP increase speed from 60/60Mbit to 150/150Mbit for 500 user in last 6 month. And price of hight speed connection to Internet are less then 10$ for Not guaranteed 100/100mbit/s.

If these ISPs cannot handle the max packet size at roughly max rate during OFF-peak (read: not evening hours from ~5 PM to ~11 PM) then the ISP is plainly torrent-hostile, though this may be by their network design issues rather than pure meanness.

And a problem is that when there are no new connection and technical problem number of pps increase on 20-50%, and about this reporting more than 40 ISP admins with network size more than 1000 users (there are network >100000 users). All this start after release of ut2.0 stable.

Users who have a router after release of ut2.0 stable have problem whith speed. Before release usual router DIR-300 has 15-50/15-50 mbit in torrent, After - in average 5-10/5-10 and freeze after 10-15min. When user change utorrent from 2.0 to 1.8.X - speed increase, pps from user decrease, and equipment work five for a weeks.

let me add my five cents:

currently i've working for ISP in Moscow and we do implementing traffic filtration for utorrent 2.xx due to abnormal load on our equipment. Also i have to mention, that most of our admins (including myself) been loyal utorrent users for years.

Many of ISP are loyal for users traffic, not only torrent. But this is compulsory measure to protect our equipment. We don't want deny utorrent users and we read for what design mTp protocol.

What is the bigger problem: higher PPS ? smaller packet size ? or traffic being UDP and not TCP ?

What is the bigger problem: higher PPS ? smaller packet size ? or traffic being UDP and not TCP ?

the main problem is alot of small udp packets storming equipment in a second (i.e. PPS is critically high).

Two questions that haven't been addressed yet:

1> How small are the offending packets?

2> How many are coming through per second?

Yes in Ukraine there are ADSL(1M/512K,2M/512K,4M/512K,8M/512K), but Number of people who use it is less then <10% it's the official statistics.

...

Users who have a router after release of ut2.0 stable have problem whith speed. Before release usual router DIR-300 has 15-50/15-50 mbit in torrent, After - in average 5-10/5-10 and freeze after 10-15min. When user change utorrent from 2.0 to 1.8.X - speed increase, pps from user decrease, and equipment work five for a weeks.

Firstly, ADSL can go up to 24 mbit/sec down but is typically limited to 1-1.4 mbit/sec up (Annex M can go as high as 3.4 mbit/sec up). In Ukraine and no doubt Sweden as well ADSL may be the exception rather than the norm, but elsewhere in the world ADSL is common. Cable ISPs also have limited max upload speeds, because the shared upload channel is smaller than the download channel/s. Even if you're on a fast line, many of your connected peers/seeds may not be on "global" torrents.

A D-Link DIR-300 router is one of the worst common routers on the market. It is below what should be acceptable performance and is only fit to be mentioned when something surprisingly works on it.

As I've stated multiple times, the first release of uTorrent v2.0 "stable" was very bad about using way-too-small uTP packet sizes. uTorrent v2.0.2 should be greatly improved in that regard. If you have >1 mbit/sec usable upload bandwidth *AND* giving uTorrent >100 KB/sec upload speed *AND* giving >5 KB/sec per upload slot, then uTorrent should be using the largest uTP packet sizes basically all the time. If it is not, please give us more details so the uTorrent developers can possibly tweak/improve it!