Endgame strategy per priority level, per torrent

[vithos]

The feature I propose is to implement an endgame strategy for each priority level, for each downloading torrent. I'm not sure whether uTorrent has an endgame strategy to begin with; if it does, it might be simple to reimplement it in this way. For those not familiar with the endgame strategy many bittorrent clients make use of, the idea is as follows:

Your torrent is almost completely downloaded. Every piece is either downloaded or requested. However, sometimes requests stall due to the other peer's upload scheduling, or perhaps even a disruption of service. So, once the conditions of our hypothesis have been satisfied, it makes sense to start renegotiating stalled piece requests in order to complete the torrent sooner.

This idea can (and I alledge, should) be implemented for the files of each priority level. My request goes like this:

For downloading torrents, find the highest download priority which has pieces that have not been completely downloaded. If there are no pieces of this download priority that have both not been fully recieved AND are not currently requested from a peer, consider renegotiating pieces that have "stalled", where "stalled" is defined by no data transferred within some reasonable timeframe, or perhaps instead by the combination of a short timeframe and a better candidate.

I've not really read any BT client sourcecode, nor any protocol documentation, so I'm just anecdotally assuming that this kind of thing is already implemented with no regard for priority levels based on casual observation of torrents as they complete. If my assumptions were wrong, or there's otherwise something wrong with this idea, please let me know. Perhaps it would make more sense for endgame to act per file, per priority level, per torrent. I look forward to comments.

Merged double post:

On further consideration, the piece transfer mode (fast, medium, slow) might also need to be considered when deciding to renegotiate. In fact, maybe that's how the standard endgame works. Anyone know?

[Nefarious]

u dont need to double post, just use the edit button at the end of ur post to add more info

[vithos]

I was aware of the edit feature and didn't feel that "edit" accurately described the purpose of my second post.

[Honeyfrog]

I'd like something similar for newly opened torrents -- I'm tired of staring at a full screen's worth of jerk-hole peers who won't trade you a damn thing faster than 0.1k/s second unless you're also uploading. Unless a "good Samaritan" peer shows up, this can mean painfully slow acquisition of the first completed piece (which then starts trading, and DL speed improves markedly). In a torrent with 4mb pieces, it might take over half an hour.

[ICleolion]

Already requested by me in IRC. Already on list of things to do for v1.8.

[alanmc]

I'm tired of staring at a full screen's worth of jerk-hole peers who won't trade you a damn thing faster than 0.1k/s second unless you're also uploading. Unless a "good Samaritan" peer shows up, this can mean painfully slow acquisition of the first completed piece (which then starts trading, and DL speed improves markedly).

What you want is for uTorrent to implement the FastPeers extensions. That has a method which allows you to request a psuedorandom selection of pieces, typically 5 or 6 off any peer even when choked. Note, every peer calculates the same 5 or 6 pieces, so there are *only* 5 or 6 pieces you can download in this manner, it is *not* 5 or 6 pieces per peer.

As for the original post, it's a waste of time to be completely honest. Endgame mode is generally entered when there are no free pieces to download. This happens in the very very late stages of downloading which means when you enter endgame mode, you typically will complete the torrent well within the next minute or two. There's no real point in trying to "optimise" it to download pieces from high-priority files as you'll be 100% complete before you can go to the toilet and back.

[cussuol]

I'd like this resource too.

It's very useful for multi-files torrents when you want to play/test some file before the others.

[faselfasel]

Please do not confuse things. This feature request is not about newly opened torrents.

This request is about improving the priority system on multi-file torrents.

In a multi-file torrent you can use the priority system (high, normal, low, skip) to have some files to be downloaded before others. You can even avoid files by setting them to "skip".

Example: I have a torrent containing 10 files. Availability is very good. I set the first to "high", the second to "normal" and all others to "low". The priority system works quite well. It will try to download pieces for the high priority file first. Thusly this file will finish first and I can enjoy the first file while the others are still downloading.

Sometimes this works just until the last pieces of the high priority file. Then often this file will just not finish it's pieces (for reasons unknown to me). Other pieces from lower priority files will still download at high speed.

Restarting this torrent will finish the file in seconds.

I think uTorrent should somehow manage to automatically honour priorities until the very end of a file.

[schnurlos]

@faselfasel: you answered on a 1.8 year old post ...

[faselfasel]

@schnurlos: I know but this thread was referenced from another thread yesterday. I wanted to point out that this feature request is still active as it covers problems existing in current version of uTorrent.

Actually, I'd consider uTorrent's behaviour erratic and would think one could also file a bug report - if something like this is possible.

[hermanm]

Azureus has a feature where you can block upload to peers. When I am seeding, I manually block upload to new peers or peers below a certain percentage. For example, if the greatest percentage is 95%, I'll block any peer below 90% just to get a higher seed count faster and stop seeding the torrent asap. This is the opposite approach, but I do some benefit in completing torrents faster. In reality, you'd have more hit and run (at least on public trackers) which may end up hurting the swarm.

[Firon]

That's a bad feature. You shouldn't be micromanaging the upload like that.

[faselfasel]

hermanm, I don't get it. What has "blocking upload to peers" when seeding to do with this feature request? I don't see any common context.

[hermanm]

> You shouldn't be micromanaging the upload like that.

I'll do whatever I feel like. Seriously, I want to limit my upload bandwidth since I get limited GBs per month.

> What has "blocking upload to peers" when seeding to do with this feature request?

Think about it. When I block upload to peers, the peers near completion finish faster. That's what the point of the feature request is.

I'm just saying that peers finishing faster only encourages them to leave the swarm that much sooner.

[Switeck]

I have on rare occasion done the opposite -- banning peers with >80% completed on torrents of 2 GB size and larger. I figure the longer they stick around, the more they will seed...and they don't need to be connected to MY seed to complete the torrent.

Also, I have banned the only seed on a large torrent from connecting to me while I was still downloading because it seemed to be misconfigured and uploading to everyone. I was only getting 0-0.4 KB/sec download speed from it. I figured the seed uploading to fewer peers at once would mean peers would complete pieces faster.

Connecting to lots of ips (peers and seeds) while downloading can boost download speeds, (this comes at a price of causing tragedy of the commons and law of diminishing returns if you allow TOO many peers+seeds) but connecting to lots of peers while seeding reduces the seeder's effectiveness immensely. Why would you want to connect to a seed if it's not going to upload much or very often to you?

[faselfasel]

> Think about it. When I block upload to peers, the peers near completion finish faster.

> That's what the point of the feature request is.

I don't understand. Can you please explain why blocking my uploads to peers helps finishing the lasts pieces of a file I am downloading? I have to add I am not an expert to bittorrent.

[Switeck]

Supply-sided economics.

hermanm is pushing more of his upload TO the peers that are "stuck" on the last few percent.

[hermanm]

> Supply-sided economics.

The end result is the same, but if the micromanaged uploading is initiated from the client side, there is no wasted bandwidth with requesting stalled pieces.