Improving Traffic Locality via Biased Neighbor Selection

I'm a heavy Bittorrent user. I got into it 5 years ago when i was living in Belgium. No problem Fast internet.

Now living in Nepal where i have the best consumer internet connection i have a 5 kb/s download speed to the outside world. Still downloading using bittorrent!!

one night i got connected to a peer within my ISP & my speed jumped to 120 kb/s like nothing , my upload also jumped to 120 kb/s. I got curiuse into what caused it.

I FOUND OUT: and this needs to be implemented to reduce the problems cross bandwith use between ISP. so reducing the economical cost the protocol is creating.

you can all read how to DECREASE DOWNLOAD TIME @

http://crypto.stanford.edu/~cao/biased-bt.pdf

or see the .ptt @

http://crypto.stanford.edu/~cao/talks/biased-bt-ICDCS.ppt

i'm not a develpor of the bittorent protocol but Utorrent is the offical Bittorrent client so i guess one of you will be the develpor. I have seen that it has been implemented to a certain exstent in Utorrent. But we can always do better.

Please keep me updated on any changes your making related to this on the protocol.

Lets Just make it even better, Bittorrent.

Yeah, it'll help you maybe. For most people, nearest peers will suck.

Before you make any such statments just read the Paper. or back up what you have to say.

Good thing thouse peeps from Thepiratebay are setting up a new Better protocol then Bittorent.

They will understand the problems bittorent creats in its current form.to High cross bandwidth use.

I like the idea. Its when you go into a bar, the local people are served first and they get free beer sometimes. It would save bandwidth for ISPs and people that have to pay for monthly bandwidth can use this if they are running out of their monthly allowance. Sometimes I think if there are lot of seeds, near-local peers reduce network load. I don't think near-local peer should have first priority but it should be a factor when you have plenty of seeds.

The problem is simple

a) in most cases ISP-local peers are actually slower, not faster since latency does not matter and you are more likely to find a fast peer in countries like sweden or japan

as you can see in the paper they don't really propose methods for the client itself to do anything locality-related... it requires support from the trackers instead (and PEX/DHT would thwart that anyway)

c) this whole thing relies on local peers being faster because your ISP sabotages trans-ISP bittorrent connections, which leads to the next point:

d) many swarms (the long tail actually) are sparse, which means you won't find any clients on your local ISP, the smaller your ISP, the smaller your swarm and the more foreign the content (if you are in the US, try to find someone downloading an undubbed korean movie for example...) the less likely you are to find a local peers, which means you are in conflict with point c)

If something like this should actually work a significant amount of cooperation on the ISP side would be necessary, as i described here: http://www.azureuswiki.com/index.php/User:The8472/P2PvsISPs

The only times where this would help IMO is when the local trunk line has excess bandwidth capacity but the internet gateway does not...AND you can find a peer or seed that is on the same local trunk line as you.

The problem is, on cable connections, the local trunk line may be already running near-max bandwidth regardless of how much overhead the internet gateway still has.

Shared wireless connections are even worse -- where EACH additional wireless participant cuts into the max capacity.

ADSL lines, with their dedicated bandwidth, would seem a perfect place to utilize something like this...but they're already running at max download and max upload capacity for many customers due to their great distances from their ISP's DSLAM link.

This leaves places like university campuses and giant office LANs as potential sources where this might work...but in both of those places, there's likely to be a policy to kill BitTorrent traffic rather than assist it!

I'm not saying that we have to reduce our traffic to only local peers. but create a plug in or a change to the protocol s tht a client programme is more likely to find a local peer. For some 10% of the users of bittorent this will lead to an increase in download speed and so make the protocol even more populair.

I wonder what the effect would be if this where to be implemented on mass. i excpect a HUGE reduction in the cross-ISP Bandwidth use of the bittorrent protocol.??

BTW: Bittorent is alos used for legal purpaces in Universitys and gaint companys to share files between departments ect. I worked at a major Chemical company in the Nethalands for a summer and had to download my memo's via bittorent. This is because the file is on a central server somewhere in amsterdam and to reduce internet bandwidth use we had to use Bittorent. were all our peers where on the intranet of that facility. with only the IT deprtment could download the files from the central server in amsterdam.

dapkor,

In your case, it sounds like you'd be better served with most of the BitTorrent clients you use blocking all internet ips and only allowing LAN ips, with only the IT deprtment's BitTorrent client given internet access.

I disagree with "For most people, nearest peers will suck".

I kindly suggest to read this and look at the numbers:

http://www.pandonetworks.com/p4p

Idea could be: torrent client software learns every peer IP (IP:port?) and average speed (total RX&TX bytes, total RX&TX connection time). This info is collected into indexed database and used at connection decision making time - fast peers shall be connected first.

There shall be possibility to manually add/delete/change database entries. There shall be possibility to add not only IP address but network subnets also.

I will appreciate comments from developers.

This does not work in general. Nonetheless, every time this question comes up... the cycle repeats. Remember alot of the connections out there are asymmetric, which means there is a (sometimes large) discrepancy between up and down speed and pooling together all available download bandwidth would have NO chance with say a 5:1 connection and I'm being generous here, with say a 20Mbit down 4Mbit up... Therefore if specific clients choose to do this fine, but it does not necessitate changing the protocol on a basic fundamental level...

uTorrent had this functionality as of 1.7 and still does but ONLY for LAN / Private reserved IP ranges.

--- 2008-01-15: Version 1.7.6 (build 7859)

- Change: do not use adapter subnets to detect local peers. seems to result in many false-positives

--- 2007-07-21: Version 1.7.2 (build 3458)

- Fix: Disable Local Peer Discovery for private torrents

--- 2007-07-13: Version 1.7 (build 3351)

- Fix: Determination of which peers are local on Windows 98 (and possibly other 95 varieties)

--- 2007-07-04: Version 1.7 (build 3148)

- Change: Add a limit local peers option to the bittorrent options in the preferences window

--- 2007-05-31: Version 1.7 (build 2248)

- Change: include all adapters in subnet search for local peers

- Change: consider peers in reserved local ranges as local peers

- Change: don't limit local peers by default

--- 2007-04-05: Version 1.7 (build 1065)

- Feature: Local Peer Discovery

This is not about asymmetric connections. If I got nice download speed from someone yesterday then I wish to utilise it today and in future - if he have file I need.

Realworld example: friend of mine is interested in the same TV show as me. As soon as new series released, we _both_ start downloading it at evening. We use different ISPs and we are not on the same subnet. We both have international connection around 100Kbit/s and connection to local regional backbone 100Mbit. When we both get lucky and our torrent clients make interconnection, usually we get file twice as fast. The problem is that not always uTorrent makes this interconnection! Each time I have to add my friends IP to individual torrent connections manually and pray for connection. This is annoying.

Forgot to say that we both have connection count limits (as most users do) - that's why our torrent clients not always get interconnection.

I think you missed the part above where they removed local subnet validation for your External IP.

Enforcing peer selection in the protocol is a bad idea. If your friend's IP is not in the tracker's returned peerlist of course uT won't connect to him.

I still do not see strong argumentation against biased peer selection.

The fact that ISPs don't implement incentives on their side to actually warrant such peer selection.

I get faster speeds from international (swedish and japanese) peers than I do from domestic peers.

I also can potentially get faster speeds from peers outside of the normal preference ranges.

There's too much that ISPs need to implement before bittorrent clients can actually benefit from biased peer selection.

PERIOD.

If you do not get it, and reading the arguments and thoughts against it by experts in this area don't change your mind, you will feel as though you are hitting your head against a brick wall by continuing to push the issue. If you wish to become a purveyor of improving this technology feel free to reference the documentation BT@Theory.org and Official repository and come up with your own code and submit it.

Where was expert argumentation in this thread? That most torrent users have asymmetric DSL lines? That DreadWingKnight do not get faster downloads from near peers, so nobody needs biased neighbor selection? Come on

I don't think that biased [neighbor] peer selection is about breaking BT protocol spec. My suggestion could be nice option to have in leading BT client software. Please do not get me wrong, I do not want to push anything Bittorrent.com will do it sooner or later http://www.dcia.info/documents/P4P_Overview.pdf

If you don't know who the people commenting in the thread are... more reason to stifle.

<--- I am not an expert, but I don't go proving it everywhere I go

It seems that conversation went into "i am superior, so you should be quiet now" arguments phase.

Also it seems that you [jewelish] even did not read what I suggested. - I suggested to learn average peer-peer speed disregarding location. Odds to hit someone matching both torrent seeders and known hi-speed peers is so small that in general it will not change anything except that in some cases it can help alot.

I am tired to ask for reasonable arguments, so I don't.

Have a nice day/whatever.

You all must understand one Thing. ISP Have no benifit of inter ISP traffic. It only creates cost. The same cost that you can see in europe when in the netherlands you pay internatinal tarrif to call to germany.. even if your friend is standing 3 meters to your side but you happen to be on the border.

Cross ISP = costly , Intranet ISP traffic = CHEAP like HELLL.

Peer Biase creates a lower cost. additonnally some ISP create Newsservers with Heavily downloaded files in there network. This to reduce Traffic ( example MS Updates).

Bittorrent = distribution of files As efficiantly as possible. So cheaply as possible leads to as Fast as possible. Less cost ISP , more profit on connection fee, More money to spend on Internet backbone --> FASTER TRANSFER OF FILES.

Just read up on P4P Protocol the p2p re-incarnation.

I'm Right.

Going back to DWKnight's post: Until such time as ISPs actually advertise this or get into contact with relevant developer projects about this type of behaviour being preferrable, nothing will change.

Right now as you can see from the uT 1.7 changelog there is no benefit currently to change code in uT for something which doesn't reliably work on any given ISP.

That page is a "give us your money" landing page. I admit I have actually seen pando peer(s). Advertising as a bittorrent client... for which uT has no decoding... I've never seen it done before.

Locality traffic advantages (From the PopkinPasko Presentation, link on wikipedia p4p):

- Solve the problem before we have to cope

with the problem (Related to the greedy increasing bandwidth usage, especially with the forthcoming technologies: eg. VDSL/2 )

- Decrease incentives for ISPs

to "manage" P2P traffic

- Faster downloads for users

Of course you could argue that adsl with poor upstream don't allow faster downloads immediately.

To this point, I can reply:

- I rarely achieve speeds over 40kB/s which is actually less than my upstream bw. (I have a 20M / 384k !)

- The overhead bandwidth needed to mantain the peers is extremely high (50% in my case). Lower upstream overhead means more useful upload bandwidth.

- The sooner p2p traffic gets cheaper (location-aware), the sooner the ISPs may consider upgrading upstream.

- I rarely achieve speeds over 40kB/s which is actually less than my upstream bw. (I have a 20M / 384k !)

Then you're doing something wrong either with torrent selection or with your own configuration settings unless your ISP has already started shaping you.

- The overhead bandwidth needed to mantain the peers is extremely high (50% in my case). Lower upstream overhead means more useful upload bandwidth.

Overhead with BT is only that high when a user has obscenely excessive settings for connection limits and number of active torrents.

The messages that get exchanged between peers such as "Have" messages account for at worst an additional 8 bytes per peer per 32768 transmitted (which is why excessive connection limits are frowned upon). Piece requests are the largest messages at about 14 bytes per block requested (16384 bytes per block).

- The sooner p2p traffic gets cheaper (location-aware), the sooner the ISPs may consider upgrading upstream.

Then we have a good old catch-22 situation on our hands. Until ISPs make location-awareness viable, client developers don't have any real reason to implement it.

Then we have a good old catch-22 situation on our hands. Until ISPs make location-awareness viable, client developers don't have any real reason to implement it.

Chicken and egg problem?

It is happening already - in areas like mine where international channels are expensive. Monopoly problems or whatever. In result ISPs overbook international capacities. My ISP sold me internet where max international speed is 20x times slower than national area access speed. Imagine huge campus with thousands of 100Mbps ethernet cliens connected to multigigabit backbone with outside/international connection just 100Mbps.

In result I am very interested in using nationally located trackers and making connections to peers which is in my area, minimising downloads from outside strangers.

Area/speed-aware connection decisions can also help for downloaders teaming with others: sometimes me together with my friends team-up to download some file from international tracker&peers, thus reducing download time multiple times!

I know and can prove that biased peer selection works, but at this moment there is no support from software. Only possibility to do it manually. This of course, is pain in the *ss.

people forget that verizon's p4p test was done with fios peers, who have waaaaaaaaay more bandwidth than you'd normally ever get with local peers.

people forget that verizon's p4p test was done with fios peers, who have waaaaaaaaay more bandwidth than you'd normally ever get with local peers.

Agree that there can be many network topologies around. However I refuse to accept assumption that local peers always is slow and BT must concentrate on assymetric DL:UL speed peers only. This is totally wrong in Eastern Europe and perhaps, Japan too. In eastern europe ISPs made nice ethernet&cable infrastructure (had no other choice because of Telco monopoly). Only problem here: expensive international capacities which leads to fast (upto full-speed ethernet) local, national area speeds and way slower international speeds.