Match BitComet's first_last default

[Honeyfrog]

This extremely popular torrent has one 800mb file of 3600 pieces; the peers are about half uT and half BitComet newer versions (the torrent is from PirateBay, where non-English peers gravitate toward BC due to its multi-lingual support).

Note the fat "dark" chunk at the beginning (10mb? more? Percentage?); that's the BitComet preference toward the beginning of the file.

-- The annoyance here is that the dark area is the part of the file that uTorrent users will receive LAST, because it is least rare. (This can be easily proven in any torrent with a massive number of peers; uT will hardly DL any pieces from the dark zone while is steadily accumulates the rest of the torrent.)

[Firon]

And? Why should we care what BC does?

[amc1]

Unless you've configured uTorrent to get the first and last pieces of the file - why would it matter that uT gets them at the end?

[The8472]

it's actually a good thing that µT does that, it evens out the availability... the whole point of rarest-first

[Switeck]

µTorrent is trying to undo BitComet's sabotage of that torrent swarm. Looks like it's failing and you want it to give up trying.

...ah well, it's a rejected request before in many forms, (sequential downloading, start+end grabbing first) it is again.

[amc1]

Err... should two Az devs really be sticking up for µT? Isn't that like crossing the proton pack streams?

[Saribro]

Consider it sticking up for good torrenting-behaviour in general .

[Honeyfrog]

Unless you've configured uTorrent to get the first and last pieces of the file - why would it matter that uT gets them at the end?

Because I DO do that. -- First_last is off by default in uT; I have no problem with that. However, when I have it enabled, I'd like it to not suck. Right now, it sucks compared to BC's implementation; and, when in a BC-heavy torrent, the part of the file I want most is the part I get last.

µTorrent is trying to undo BitComet's sabotage of that torrent swarm.

That's just an illusion: If everybody DL'd the same percentage of the file first, it wouldn't hurt the torrent at all because everybody still needs to the rest of the file anyway. (Behavior like this doesn't croak a torrent; what really kills 'em is selective downloading of only single files in thinly-peered multi-file torrents, or good old-fashioned choking of UL to 1k/s and abusing disconnect/reconnect.)

The only torrents that can be hurt by every single peer glomming onto the first 10% are torrents whose quality is crap. In such a case, many peers exit upon obtaining enough to view. -- And this is a GOOD thing, because crap is just a waste of everybody's time. (A torrent doesn't have an inherent right to survive!)

[amc1]

However, when I have it enabled, I'd like it to not suck.

So you're saying that when it's enabled - it doesn't get the first and last pieces quick enough for you? Or that it doesn't get as much of the starting block as BC does?

A torrent doesn't have an inherent right to survive!

Which seems like a really strange thing to say. Surely it's better for your client to get the rarest pieces so that, in the event that a lot of the other peers disappear? Isn't it the case that if you pick up the rarest pieces that other peers will be more likely to trade with you?

I'd be interested to hear when uT decides to get the first / last pieces with the option switched on and off in that swarm...

[Switeck]

"If everybody DL'd the same percentage of the file first, it wouldn't hurt the torrent at all because everybody still needs to the rest of the file anyway."

I would be pissed as a seeder to have 20 out of the first 30 peers insisted on ALL grabbing the first 10% from me. For a 1 GB file, that means potentially 2 GB of uploading duplicate stuff as the seeder -- or DOUBLE the size of the torrent! There's a REASON some of us hate BitComet when we're trying to seed.

Even if the first+last parts requested are only 5% of the torrent in my scenario, that's still up to 1 GB of duplicated uploading...which if you're the only unfirewalled seed or peer is a given.

This is a definite problem for people with limited monthly bandwidth quotas.

If I see a torrent that seriously stalls while downloading, I often lower its upload speed if I determine I've uploaded (often far) more than I've downloaded AND few/no peers have parts I'm missing. In short, I half give up on any torrent that doesn't download faster than 5 KiloBYTES/sec...especially if it goes 20+ minutes straight at 0 speed. That would happen quite often with a single slow seed and everyone trying to grab the exact same parts...such as the first file in a multi-file torrent. Upload bandwidth may be amply available among the peers, but have NOTHING to share because of the first-last downloading priority. The seed/s have to make up the difference in that case.

Later on, for "old" torrents, this can cause a torrent to go unseeded simply because a seed gives up after seeding x-times as much as they downloaded...and no seeds are created when it quits.

[Honeyfrog]

Switech: I would be pissed as a seeder to have 20 out of the first 30 peers insisted on ALL grabbing the first 10% from me.

The reason those early sections are "dark" (most common) is because some of the BitComet users in question are Japanese peers with insane optical-pipe bandwidth, and they very quickly feed every other BitComet peer (in a torrent with almost 10,000 peers) until they all have it. Thinking that seeds are getting selectively robbed is an illusion. Thinking that the torrent will be hurt is also an illusion because every peer still needs the whole file regardless of what part he acquires first. -- This is a phenomena entirely distinct from that of peers prioritizing files in a multi-file torrent.

AMC: Surely it's better for your client to get the rarest pieces so that, in the event that a lot of the other peers disappear? Isn't it the case that if you pick up the rarest pieces that other peers will be more likely to trade with you?

See above. I maintain that prioritizing pieces within single files is an order of magnitude LESS detrimental to prospective torrent health than prioritizing files within multi-file torrents (a very popular feature few of us would do without), not to mention 1k/s UL settings, automatic-quit upon file-completion or automatic "seed while" ratio governing.

Meanwhile, it is almost impossible for a uTorrent peer in any heavily trafficked torrent to get pieces #1-#20ish before everything else. (I should have taken that screen-clip nearer to completion, when my "Downloaded" bar was an inversed mirror-image of "Availability", with solid completion everywhere except the first part save piece #0.) Even if #0 provides codec, audio and track info, a few seconds of usually black screen prior to title credits give me very little visuals with which to acertain the true quality of the picture.

[Switeck]

For a lot of what I see, your scenario (insane optical-pipe bandwidth Japanese peers) is more an exception than a rule. I can even envision a scenario with the occasional superfast peer joining and staying a couple hours past completion that could have the odd result of "draining" the torrent of seeds. The other seeds might find while the superfast seed was there that they couldn't consistently upload to their max for that torrent, conclude they're not needed (and rightfully at that time), and leave. As a general rule, the average torrent user sees very little incentive to keep a torrent running if they've already completed the download.

The issues I pointed out will be very real if there's no fast peers. Generally, BitComet is not a good sharer in large swarms because it insists on automatically using too many upload slots at once.

Seeds may not get selectively robbed IF there's enough connectible peers, however that may not happen until the torrent swarm is surprisingly big due to the typical 60+% firewalled peers. In private torrents where Peer Exchange and DHT is disabled, typically it takes 1 or 2 tracker recontact cycle times before new peers can find one another. So for potentially the first 30 minutes, an old unfirewalled seed will receive an unfair amount of attention from firewalled peers.

When the amount seeds needs to upload to create other seeds is increased...the net effect is still the same, precisely BECAUSE every peer still needs the whole file regardless of what part he acquires first. Anytime a BitTorrent client refuses to seek out rare pieces first, they vastly increase their odds (as they reach the end of the torrent download) that they *MUST* download from a seed. Someone might seed for 200% on a half-dead torrent...and after they leave the only parts peers would consistently have is the first and last parts. Which would likely make the torrent FULLY dead.

[Honeyfrog]

For a lot of what I see, your scenario (insane optical-pipe bandwidth Japanese peers) is more an exception than a rule.

Well, a T3 or OC3 connection may be atypical, but it only takes one of those in a torrent to equal the capacity of dozens or even hundreds of other peers. Regardless of the peer make-up, virtually every PB single-file torrent with at least 100 peers displays the same dark "common" bar at the beginning, and it is therefore impossible for uTorrent peers to get pieces #1-#20 other than last.

Anytime a BitTorrent client refuses to seek out rare pieces first, they vastly increase their odds (as they reach the end of the torrent download)....

...But the clients in question are only eschewing rarest pieces when they lack the first umpitty pieces or percent of the file. After that (while still less than 5%), they behave normally. Actually, I am not even certain that they're not refusing to seek rare pieces -- in a torrent with 10,000 peers, there's really no such thing as a rare piece, and there may well be a cut-of availability number below which the behavior is not manifest. (But I'm only theorizing there.)

[Switeck]

The FIRST few pieces that a peer gets needs to be "rare" for the "BitTorrent effect" to work well. If they download a piece that everyone has...who are they going to upload it to? So they remain a complete 100% leecher for potentially the first 5% of their download and often leave shortly after they finish downloading the whole torrent. This shrinks the window-of-opportunity where they at least could be uploading while downloading...and that's good for the torrent swarm how?

With each new peer joining that wants the first-last pieces immediately, even "old" peers spend more time uploading those pieces than something to insure that the torrent doesn't become UNSEEDED when seeds leave.

[Honeyfrog]

The FIRST few pieces that a peer gets needs to be "rare" for the "BitTorrent effect" to work well. If they download a piece that everyone has...who are they going to upload it to?

Everybody that requests it, thereby making the dark bar. That this happens is already a given. Once everybody has the first part of the file, they all still need the rest, and settle down to the usual business.

So they remain a complete 100% leecher for potentially the first 5% of their download and often leave shortly after they finish downloading the whole torrent.

Actually it's more like about 2% by my eyeballing. All that's really going on here is that the first part of the file is being treated like a separate "sample" file in a multi-file torrent, but without the extra megabyte bloat and less worry that the sample file will become unpopular over time (risking loss of a distributed copy over time, causing the torrent's scrape to list it as unseeded).

In any event, the dark-bar zone is created by peers who are intrinsically fast uploaders, and who will (or at least should, theoretically) seed the rest of the file just as fast. Also, it could be argued that it's a good idea to "feed" leeches with common pieces rather than rare ones, so they'll have less "bait" to tempt connections with good peers.

[Switeck]

"Everybody that requests it, thereby making the dark bar."

But that's just it, if everyone else has it they're just leeching since they have nothing to upload to anyone. ...Unless a new peer arrives. But a new peer is more likely at first to connect to older peers and seeds, as these are the first to be registered on the tracker. So the fresh new peer that just got the first+last part of the torrent probably won't connect to the next new peer (which starts with nothing) till that newer peer gains the first+last part from some older peer or seed. By demanding these parts first, you're putting an undue burden on the seeds and older peers on the torrent.

Were it a manually-selected process, that might be more acceptable. But automatically doing it for every torrent sounds little different to me than sequentially downloading every file in a multi-file torrent. (Which has been REPEATEDLY rejected.)

With more BitTorrent clients set to grab first+last pieces initially, the rarest pieces stay rare and become more so when seeds leave. If every new peer demanded the same 2% of the torrent, a seed could easily end up with a 200% share ratio and leave the torrent with availability under 1.0 and nobody else a seed!

"causing the torrent's scrape to list it as unseeded"

Do you mean the "sample portion/file" in the torrent file or the torrent file as a whole?

[Honeyfrog]

Mind you, a BIG part of this suggestion -- which I would happily accept as a cryptic super-secret hidden mystery Advanced setting -- is to quickly detect torrents that are garbage in the first place. Such torrents, by definition, shouldn't be helped. But as things stand now, uTorrent users are the "suckers" who always find out LAST.

I read elsewhere that first_last was going to 1mb; that's nice, but it's still barely more than a few seconds of studio logo in a 700mb rip, and virtually nothing at all of anything bigger.

Right now, with anything iffy, I am compelled to use *BitComet* to discover if the file is any good.

[Switeck]

If you're seeking to determine if the torrent is poisoned or fake, a quick check of availability, connectible seeds, or tight grouping of "strange" ips is usually good enough.

Barring that, if the website (or person) you are getting the torrent from cannot report the torrent's quality...I do not perceive that to be µTorrent's fault.

1 MB of a torrent, be it 700 MB or 7 GB in size, still represents an appreciable amount of time acting as a leech of a torrent piece which by your own estimates is common. Increasing that amount further shrinks the "window" of downloading while also potentially uploading. This will increase the amount seeds have to upload to create virtual copies and more seeds. Which gets me to another point:

"Seeding an extra 25% of 700mb is annoying; seeding 25% more of 4gb is maddening; seeding 25% more of 20gb is totally intolerable."

Percentages won't work for that reason.

Really, if you feel compelled to use BitComet...you may find its automatic phone-home features (which can't be disabled) of the torrent size, torrent hash, and possibly filename of whatever it is you're trying to download quite helpful -- after all, BitComet also includes a means to rate torrents and "chat" with other peers and seeds. Your ip is recorded too, so it is easier for BitComet to track you.

[Honeyfrog]

I very seldom need to know whether or not a torrent is fake (since I'm very seldom interested in new releases); I am always interested in determining quality.

(My seeding comments concern initial-seeding mode, and the current bug which resets piece-offered position back to #0 if uTorrent is restarted.)

[Firon]

Might wanna check your threads, that piece position reset bug was fixed

edit: which you in fact already did.

[Honeyfrog]

Yep; I'm looking forward to it.