Jump to content

Improving Traffic Locality via Biased Neighbor Selection


dapkor

Recommended Posts

  • Replies 95
  • Created
  • Last Reply

Updated my post: http://www.azureuswiki.com/index.php/User:The8472/P2PvsISPs (added point #4)

Regarding P4P, most bittorrent developers (including me) do not consider it effective to simply use local peers as long as there is no direct payoff for that. Although it would probably lower the costs for the ISP it would also hurt user experience, as to date there is no user-side advantage for doing so. As long as ISPs do not provide such incentives (see my wiki article for several of these) or at least make it less detrimental to the user to prefer local peers we simply won't implement it.

And everyone i have spoken to agrees that the "speed gains" stated in their press release is pure hyperbole and PR talk.

Link to comment
Share on other sites

This is not about exclusive use of local peer connections at all! But: fast peers (traders) shall be connected first before torrent download connection limit reached.

Amazing situation: I am telling you that I already doing this (fast peer selection) manually and it helps me decrease download times, but no1 is interested. Is there any BT simulator around? Please suggest me.

Link to comment
Share on other sites

errr.... bittorrent already prefers the fastest peers, that's how the normal unchoking works. the very point of this discussion is that local != fastest. If local peers were the fastest ones then this discussion would be moot as they'd get picked.

Link to comment
Share on other sites

The8472, cool down please. Would you be so kind and approximately explain - how unchoking will work in situation described below?

Friend of mine who is located in my LAN network but not in the same subnet, told me that nice 1GByte file released and he already downloaded half of it. There's 1024 seeders. My maximum summary download speed is limited to 64 KByte/s from outside except my LAN network where I got upto 2MByte/s. If my connection limit is 32 per tracker - how and when my BT client software will finally connect my friend to quickly download everything he got? Usually BT client does not hit him, I get bored waiting and have to connect him manually.

local != fastest

Only connection type that bittorrent creators aware is ADSL? Please explain me from where those fast nonlocal seeders come, what type of connection they have and how it can be that between them assumption "local != fastest" is also true?

Link to comment
Share on other sites

Where the list of peers on a torrent are > than your connected peers per torrent it's only random chance to connect to peers you know exist. This functionality will not change that as the multi-cast for local peers will likely get swallowed up on the ether between you and your friend.

Local peers according to uTorrent at least right now refer to peers on your LAN or current subnet according to subnet mask. That the 2(3) clans of developers who've responded feel the same way will not change. If you want this feature... convince your large ISP to talk to p2p client developers about incentives they can implement for increasing their users' experience while lowering their costs. Until such time as that happens, the client developers have no reason to code for a niche which does not benefit the swarm as a whole. I believe some ISPs use CacheLogic servers for this purpose allowing transparent mirroring of content at their boundary to allow a portion of their bandwidth to stay in-network. Unfortunately the number of ISPs which are fighting the flood of data which has precipitated since the age of streaming FAR OUTNUMBER those who work with or at least don't impede this flow of data.

DWKnight is quite logical and this point will stand until ... it doesn't :

Then we have a good old catch-22 situation on our hands. Until ISPs make location-awareness viable, client developers don't have any real reason to implement it.
Link to comment
Share on other sites

although jewelisheaven has already said most of the relevant things...

@ogden

unchoking will just work fine in your situation, once your bittorrent client connects to this peer it will be preferred as it is the fastest. and connecting can be done by manual injection in your very specific scenario. For lan connections in general you can either rely on multicast based lan peer location functionality (different subnets won't matter as long as both nodes are in the same broadcast domain, if they're not you need multicast-aware routers) or by running an additional, lan-local tracker. mixed LAN+WAN usage is a niche-application, and the general WAN-related behavior of bittorrent will not be changed to accomodate this specific situation (except for said injection and lan location options).

Please explain me from where those fast nonlocal seeders come

Seedboxes, often co-located, or users in bandwidth havens like sweden (our prime example). For swedish users swedish ones will be fast, but they don't need special locality-preference, as bittorrent already selects the fastest reciprocating peers, and unless someone is even faster than a swedish peer a swedish one will eventually prefer another swedish one.

Most ISPs that complainy about P2P run asymmetric services (not just ADSL, also cable, satellite or whatever)

Only connection type that bittorrent creators aware is ADSL?

Ok, let me qualify my statement, i thought the implications were obvious, but apparently they are not. We are talking about WAN connections here, so a LAN scenario does not count under this topic (as explained above). If we are talking about "local != fastest" that means "local does not imply fastest" and not "local cannot ever be the fastest". A local peer can be the fastest peer, but it can also be the slowest one, there is no good correlation we could build upon.

Link to comment
Share on other sites

or by running an additional, lan-local tracker

Sounds nice. Commynity or LAN/whatever DHT cache server. Oh... I know, i know - it will stay as it is.

Unchoking: If my BT connects to 32 of 1024 seeds - does it really will "walk around" until it finds the one? Usually it (uTorrent) sticks to most of them for quite long, dropping only dead-ones. How can I tune unchoking logic of uTorrent? I would like to disconnect slow seeder after 30..60sec if his speed is less than some defined by me, percentage of my total download speed for particular torrent. Why would I need to save 0.2KB/s connection in case of 500KB/s total download?

I believe some ISPs use CacheLogic servers for this purpose allowing transparent mirroring of content at their boundary to allow a portion of their bandwidth to stay in-network. Unfortunately the number of ISPs which are fighting the flood of data which has precipitated since the age of streaming FAR OUTNUMBER those who work with or at least don't impede this flow of data.

Let's see - if some ISP comes here and asks about how to set up cache server, he will be sent far far away? - That's because other ISPs do not care?

We are stuck completely :)

p.s. What about CacheLogic & CDP? - It was promised to be open..

Link to comment
Share on other sites

Ctrl-P -> Advanced -> peer.disconnect_inactive and peer.disconnect_inactive_interval . They default to TRUE and 300 respectively. The timeframe cannot be decreased below 5 minutes. :) Covered in the Manual actually.

I mentioned CacheLogic because that's what I have heard about. After the innovation with all these IHVs including bittorrent clients in NASes and such... I'm sure something else is out there. The point is the ISPs need to stop FIGHTING the data overload and start working towards managing it on their (existing) infrastructure. I reference this AWESOME article with a p-o-c site whose rollout deployment time for updating was cut to 4% of previous time due to bittorrent technologies http://torrentfreak.com/university-uses-utorrent-080306/

Link to comment
Share on other sites

It means that peer with upload speed 200bytes/sec will not be disconnected because he is not inactive. Right? grrrr... It is not good to use one of 32 connections to get 1/2500 of transfer speed (500KB/s vs 0.2KB/s).

The point is the ISPs need to stop FIGHTING the data overload and start working towards managing it on their (existing) infrastructure

If I go to ISP office - what shall I tell them? That they need to start working towards managing infrastructure or what? What exactly shall they do? - I am really curious here at the moment.

Link to comment
Share on other sites

Unless you have extraordinary influence on your ISP you need to get everyone you know and people you don't know ... who are customers of the ISP ...to begin some sort of action committee for change. ISPs are a business, and businesses are about the bottom line. If enough people start saying "I'm mad as hell and I'm not going to take it anymore" regarding degraded service, and let the ISP know they are willing to deal with up-front limits to their service, and caveats for full-usage including "your traffic must stay in the swarm" and provide mechanisms for testing this, I'm sure the "smart people" read as: engineers for both the ISP and software which can take advantage of this... will get together and have code parties

</dream> At least that's my dream.

Link to comment
Share on other sites

engineers for both the ISP and software ... will get together and have code parties

.. and make next business. In result swarm gets nothing except better commercial video/data streaming. Example: CacheLogic, now Velocix cooperation with bittorrent.com. Cache Discovery Protocol is still not open and I doubt it will be anymore. CDP easily could be solution to what we are talking about.

Link to comment
Share on other sites

Local peers are not always slow, I'd say they may be reasonably fast.

But uTorrent ALREADY rewards fast uploaders to you by uploading proportionally more back to them.

A scheme to flood the local net at faster than the regular max speed limits in uTorrent (IE: no speed limit on local peers) is probably not the best way to do it. As I've already expressed, the bottlenecks in many ISPs are also their close-to-customer internal networks (especially for cable networks!) as well as their internet gateway.

Any means to find this "spare" local bandwidth may likely be obtrusive and unwanted by ISPs.

Link to comment
Share on other sites

There can be improvment, trust me.

Example: I have DSL and I frequently saturate my download speed. Actually looking while it downloads I see local (Swedish) peers connect and upload pretty fast in the first 10 seconds or so. Then it all equalizes and every peer uploads at the same speed Say 50 people each at 5kb/s.

A typical torrents looks something like this right now:

20 US peers each at 5kb/s

15 EU peers each at 5kb/s

5 Swedish peers each at 5kb/s

2 my_isp.se peers each at 5kb/s

8 International peers each at 5kb/s

TOTAL speed 250kb/s

I would very much like a setting changing the weighted bandwidth. Say we add a setting to allow .se peers to be weighted more and my_isp.se peers to be even more weighted.

20 US peers each at 0kb/s

15 EU peers each at 3kb/s

5 Swedish peers each at 20kb/s

2 my_isp.se peers each at 50kb/s

8 International peers each at 0kb/s

TOTAL speed 250kb/s

Making no difference for me, but my isp is probably much happier. If there were no Swedish peers, it would revert to the old behavior.

Since isps have weird naming schemes this would probably call for a manual list of local isp hostmasks, but that wouldn't be too hard.

Link to comment
Share on other sites

As a seeder, there is little feedback to tell uTorrent to upload faster or slower to various peers. So as long as uTorrent is uploading to a 'broadband' connection, they'll likely get a roughly even cut of your upload speed as any other peer.

While downloading, upload speeds to other peers often vary wildly. I see a few over 5 KB/sec...a few between 1 KB/sec and 5 KB/sec...and many below 1 KB/sec...and lots more at 0.

So long as even the "good" ISPs often have contention rates of 10:1 or worse, you're not going to find a suitable means to maximize local peer download/upload speeds that will make ISPs happy. In fact, the better the method you find/use...the more likely you'll be needing to find a new ISP. :P

Link to comment
Share on other sites

We-r drifting of the point. What is needed is a Change in the protocol & clients that enables a active torrent to look for peers busy with the same torrent to mask together. This increasing the local bandwith use but decreasing the inter ISP bandwith use. This will enable ISP to increase profits deu to lower traffic cost. Hence they have more funds to re-invest in the network and so enable use in the long run to increase the average transfer speed even more.

Many ISP have no limit on the speed between users on the same network so hence the torrent spreads as wildfire. enabeling more seeds to be created in less time and so these seeds can feed users downloading in network where none or very few users are using that paticular torrent.

Lets give life to local bittorent. Lets make everyone benifit. Lets make things better ( :D cheers philipps)

Link to comment
Share on other sites

Many ISP have no limit on the speed between users on the same network

Not as many as you'd think. They tend to be restricted to specific countries.

Both Canada and the USA have VERY few ISPs with such variances between ISP-local and ISP-non-local traffic.

In Canada there are 4 major players for ISPs. NONE of them have the split-tier bandwidth limitations you claim.

Link to comment
Share on other sites

Both Canada and the USA have VERY few ISPs with such variances between ISP-local and ISP-non-local traffic.

If you care about immigration problems, then you should think about adding features which is useful in other countries where variances still exist ;)

Link to comment
Share on other sites

How is uTorrent supposed to be "network-aware" to take advantage of local networks greater speeds?

The topology of the network uTorrent connects to the internet through is virtually invisible to simple network probes.

Transparent proxies are attempts by ISPs to obfuscate network topology further.

A user's own router may restrict or cripple uTorrent's local network scanning ability, not just by being not forwarded...but also by subnet limitations.

Even a software firewall on the uTorrent-using computer would likely compound the problems further.

Without knowing a local network's limits, uTorrent must be told explicitly what ip ranges are "local" and what speeds to run at for "local" connections. (Just to know what connections are local to you would require an extensive library of information, far above and beyond simple reverse DNS lookups...and may not be accurate or complete just 6 months later.)

...Or just to occasionally attempt "infinite" speeds to all peers/seeds, (at least once to each) and anything that exceeds regular "max" speed (also defined by the user) must be a local peer.

Even assuming local peers and seeds can be found "with ease", it would be in bad taste for uTorrent with default settings to attempt to saturate the local network. In a contention-based network, where not everyone can be running at 100% bandwidth utilization at once, attempts to saturate the local network are almost always viewed as "hostile" by ISPs...and treated as such!

Although the idea may sound good, if it cannot be done without cooperation between both ISPs and BitTorrent protocol developers...it cannot be done at this time except on whatever ISPs are willing to support it.

Link to comment
Share on other sites

Just look at whats happening in Canada right now. Bell Sympatico ISP & wholesaler of the network is closing down all P2P traffic even of applications as Iwatch. How F*** up is this ???

These kind of actions are being taken bij ISP all the time. And what can be done>? well Bittorrent develpors need to set the first step. Talking to Major ISP and to Internet backbone infrastructure operators. This to enable the world to clean up the internet. Because if we really want to take advantage of the internet we need to clean up of F*** up.

Link to comment
Share on other sites

I JUST read on Tweakers.net that Comcast is going to work with Bittorrent.inc This to develop the p4p Infrastructure/protocol. This is a set in the right dirrection.

P4P : http://life.tweakers.net/nieuws/52425/p4p-routeringstechniek-moet-p2p-verkeer-efficienter-maken.html

(dutch link, Use google translator to get in english)

This just showes how good Local peep awareness is. I need a medal.

Peace

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.


×
×
  • Create New...