"Queued Seed" forever

[Timwi]

The limitations in the client are a direct result of the protocol, get over it.

Funny that you think this comment would encourage me to use BitTorrent more

[Saribro]

And once again you fail to understand what's going on. We're trying to make you use BitTorrent -correctly-. We'd all much rather have you use something that actually suits your needs, than have you abuse BitTorrent. Just because you want to use BitTorrent, doesn't mean you should. The right tool for the right job, as it were.

[Timwi]

I don't think I'm "abusing" BitTorrent. I guess I just came here with the assumption that it's approximately as good/useful as eMule (and possibly other P2P systems which I don't know). I was already aware of its technical shortcomings, so I expected that there are other benefits to make up for it. Now I find out that it really is as limited as I thought, but also that its users defend it religiously, claiming that it's not comparable because it's a different kind of system, and that the misbehaviours of the clients are intended and by design. I am still wondering if that is enough to explain its popularity.

[Saribro]

BitTorrent is a -different- protocol, designed for -different- uses, with -different- behaviour than eMule (and gnutella, winmw, napster, kazaa,dc++ and that sort of tools). As long as you refuse to accept this, you will never understand it, or BitTorrent's popularity. If BitTorrent doesn't suit your needs, don't try to force it to do things it wasn't designed for and then blame the clients and protocol when you fail to do so, just use something else that does suit your requirements.

[Timwi]

Why do you keep alleging that I am "forcing BitTorrent to do things it wasn't designed for"? I keep telling you that I haven't changed any settings. As long as you refuse to accept this, you will never understand that (at least part of) the fault lies with BitTorrent and/or your software. All that I'm trying to do is keep torrents running after they finish downloading (in fact people keep asking for this explicitly, to keep the files available); if your software can't do that then it doesn't fulfill the intended purpose of BitTorrent.

That BitTorrent is a different protocol, I am well aware. That it is designed for different uses, however, I disagree. Ultimately, its purpose is to download files in pieces from other people who have the same file. BitTorrent can do that, but eMule can do that and more. Can you name one thing that BitTorrent can do that eMule can't?

[Switeck]

The queued settings are trying (loosely) to decide what to seed and what to not seed at any given time. But it is NOT possible to constantly upload on all of those torrents at once...without causing far more problems than it solves.

[The8472]

> to keep the files available

you are NOT achieving this by running all seeds at once, on the opposite... all you achieve by that is spreading your upload so thing that not a single torrent will profit from your upload.

> thing that BitTorrent can do that eMule can't

letting old content die to make room for new content

eMule is basically a content-retention-system. bittorrent is a file-distribution system, once the landrush on a torrent is over and most people have it it can die and you move to the next new thing.

[Timwi]

Your attempt to make a deficiency sound like an advantage is interesting, but I don't fall for that; I don't see how it is in any way an advantage that files become extinct. If "most" people have it and then it dies, the remaining people who want it are out of luck.

But it is NOT possible to constantly upload on all of those torrents at once

I mentioned about 4 or 5 times already that that is not what I'm trying to do. I really don't know who spread this rumour and why .

[amc1]

Timwi, I've skim-read through this thread again, and I'm just going to have to requote Firon's rather excellent statement.

You still don't get it. On eMule and networks like it, you share an entire library of files, and control upload slots for that. On BT, every torrent is like a new network, and the upload slot control is per-torrent, not global. So running too many will spread your upload too thin.

When you aren't seeding, and you are queued, it means that the tracker won't report your IP and port when a new peer comes along and wants connections (and even if the tracker did report it, the client won't accept connections for that torrent anyway).

The comment about scraping (which perhaps you may have read about before) is where the client doesn't actually communicate with the tracker to say "I'm available to share this file", but it does communicate with the tracker to find out how many seeds and peers there are.

It's not unusual for BitTorrent clients to revert to a queued state for a torrent when there aren't any peers around (and there still *are* seeds around), since there's no benefit being on that torrent. When a peer comes along, the client will find out about it by "scraping" the torrent information from the tracker, and decide to go back and join the swarm.

BitTorrent excels at actually sharing bandwidth "fairly" between peers - the faster you are uploading, the faster you will normally download. I don't know the specifics of how Emule decides to distribute bandwidth, but given that BitTorrent was primarily designed to distribute content well (and to be biased against people who don't share), it's likely that the mechanism BitTorrent uses is better doing this than EMule.

There are limitations to BitTorrent, but these things are usually related to the fact that it isn't what BitTorrent was primarily designed for.

[reswobslc]

As a newcomer to the thread I see a lot of hot-headed arguing, NEITHER side understanding what the other is trying to say.

It sounds like Timwi wants to use BitTorrent to share 1000+ files, just to keep them available, most of which will be rarely downloaded except a couple at a time. It sounds like most everyone else is saying "You idiot, you don't have enough bandwidth for 1000 torrents". I think Timwi gets that.

I think the missing link is that Timwi understands his 1000 files are relatively unpopular but he wants to preserve their availability, and that he expects only a couple of them to be downloaded at a time, consistent with the limited bandwidth he has.

This SHOULD be possible with BitTorrent regardless of the protocol details, because if he is the only person seeding each of these files, there should be no peers to saturate his bandwidth with keepalives. His only regular communication will be announces. Obviously once his connection is saturated by people downloading even 30 of these files, the other 970 are essentially unavailable. I think Timwi is aware of that, but doesn't consider that his files are popular enough in the aggregate that he will be faced with that situation. Obviously if he turns out to be wrong, then the limitation is doing its job, and queueing is the next best alternative.

I also get the impression that μTorrent is behaving correctly, in that if there exists peers for more than 20 or so files, then it should not be actively seeding more files than that, regardless. (I have never attempted to seed or share that many files at a time, so I cannot say first-hand what the behavior is).

μTorrent would do exactly what Timwi is expecting IF it could simply stop actively seeding files if there are other seeders and/or no downloaders for a set period of time (5 min for example) and move on to seeding files for which there are NO other seeders. I am assuming from this thread that those idle torrents shouldn't consume any "slots" until other peers appear, as since there's no peers, there's no peer-to-peer traffic. If downloaders appear while he's the only seeder, those should consume his slots, and if seeders appear, then Timwi wants his client to consider "going away" from that torrent to concentrate on other files for which there's no seeders.

I think that Timwi is saying that regardless of the protocol, he feels μTorrent should be able to OFFER to seed an unlimited number of files so long as none of those files, or very few of them, actually has peers at any given time. Sure he may only have bandwidth to participate in 23 torrents, but he doesn't want torrents for which there are no other peers to count against that 23, nor to be made inactive.

This is not a request unique to Timwi. A lot of people would love to make their entire collection of SOMETHING available to the BitTorrent network, with the assumption that only bits and pieces of it will be of interest to the world at any given time.

[Timwi]

Yay, reswobslc understands me :-)

I notice the wording "OFFER to seed". I think there is an ambiguity in the BitTorrent terminology which contributed to the misunderstandings in this thread. I have tried to use the words "share" and "upload" instead to avoid this. It appears that "seed" means both of those. If I share 1000 files in eMule, obviously I will never have all 1000 files uploading at the same time. In the same way, I expect to be able to "seed" (share, offer, make available) large numbers of files and having µTorrent decide which ones to "seed" (upload) at any given time, while still announcing their availability on the tracker. I don't want 970 files to become unavailable just because 30 of them are already downloading. eMule solves this correctly.

[amc1]

Timwi - you might want to re-read what I said about torrent "scraping". I know Azureus can in various circumstances drop out of a swarm and keep an eye if any peers arrive by scraping.

You can choose to join the swarm if there aren't any available seeds (so that people who may discover the torrent can see that it isn't dead). I'm sure µTorrent does the same thing.

[reswobslc]

I would think that if all your slots became full with torrents you were actively seeding (that is, somebody's downloading 30 of your files), you would want to at least temporarily stop announcing on those other 970 torrents since you couldn't effectively serve them anyway. But you said it better - that is, as long as you have bandwidth to share, not being consumed by your peerless torrents, your client should be able to report to the tracker that you're seeding those torrents so they will not die out.

[Torin]

The problem is on private trackers. The BitTorrent protocol is NOT designed for a virtualy infinite number of seeders and a few leechers. Also, it is a common situation for those trackers to have some lightning speed connection users (2-4 MB UL bandwidth) and plenty of low connection speed users, ADSL and others, lets say limited to 60 kB/s upload bandwidth. The first category can keep a good ratio without thinking too much about it, the ones in the second group need to compensate the low speed and have to put more effort in order to maintain a decent ratio and stay in business. This effort translates in more storage space to keep more torrents available to others and more hours/day seeding time. It is pretty useless if they don't find a way to force the client to use the entire available upload bandwidth when seeding.

Now lets talk about the utorrent client behavior. For example, let's consider an ADSL connection xx/512 and set a limit of 50 kB/s max up speed. The build in speed guide suggests Max active torrents: 3 (xx/512). Presuming that you have completed 30 torrents, all ready for seeding, the Utorrent client will pick 3 of them fast and start seeding; the rest of them will be queued.

Assuming you are lucky and the upload speed is 20/15/10 (45 kB/s, almost the entire UL bandwidth is used). The first torrent is finished, so utorrent replaces it with another one. It is possible to find one who needs only 5 kB/s, so the new stats are 5/15/10 (30 kB/s). From Utorrent's point of view everything is fine, 3 max active torrents, decent UL speed for each, even though in this moment you only use 30 kB.

The question is, if most of yours torrents are queued, how will the client know to pick up the 3 'most wanted' torrents (in terms of UL speed), in order to keep your entire UL bandwidth used? In practice, for max 3 active torrents you can easily find a suboptimal 5/5/10 kB/s scenario. A 50 kB/s upload bandwidth is already a piece of shit, so seeing that only 20kB is used for hours is annoying.

Keep in mind that on some private trackers the leech requests number are actually pretty low, excepting a few hot new torrents most of them got 0 or max 1 leechers. If you set Max active torrents: 3 according to the guide, you will get lots of torrents in queued state and 2-3 torrents active and like 25-30 kB/s used bandwidth. If you set Max active torrents: 8 or 10, almost all torrents get the seeding status (even they are not active) and seems to be more easy for utorrent client to pick up the 'right' 3 torrents, made them active and maximize the usage of UL bandwidth. In other word, the uttorent client chooses smarter when all torrents are in seeding status then queued status.

You see, it's not about cheating with 1 kB/s upload/torrent. I do NOT want 10 active torrents at 5 kB up speed each. All I want is 3 active torrents, not 10/10/10 kB/sec but 20/15/15 kB/sec. In other words, I dont want the uttorent client to stop seeking when he reached the maximum active torrents number even that the bandwidth is not full used. If the UL bandwidth is not full used, I would like a behaviour to keep puting queued torrents in seeding status till he finally can match up 3 torrents who makes up the maximum bandwidth usage.

[reswobslc]

First, trackers can get gigabytes of space as cheaply as anyone else can, and I doubt they're running out of "space". Second, they can exclude anybody they feel is misusing them. This is rarely an issue.

I also think there is nothing wrong with uploading files with very slow bandwidth, particularly if there's few seeders. The slower the upload, the more incentive others have to stick around and contribute to the swarm than if one allows a "wham, bam, thank you ma'am" fast drive-by download.

As for how it will find the 3 most "wanted" torrents... it's not a matter of "more wanted" versus "less wanted". To seed lots of files would require that most of them are actually "unwanted". In that case, when someone gets around to "wanting" a file, the one "wanting" shall use the tracker to find the one "offering".

[amc1]

The problem is on private trackers. The BitTorrent protocol is NOT designed for a virtualy infinite number of seeders and a few leechers

Well - no. It was never designed for things like "private" trackers or other things where ul / dl ratio are important.

I know that Azureus has individual queueing options to indicate how many torrents you want active when you are just seeding - I assume uTorrent does too...

[Switeck]

Although µTorrent doesn't specifically have special max torrent limits for while seeding only, it does separate max downloading torrents from total active torrents. Downloading torrent max must be equal or less than active (seeding+downloading) torrent max.

A real problem with these limits is although they may work fine for "midsize" relatively active torrents, they can be poor settings or just plain unworkable for extremes. Really slow+large torrents download so slowly that it may take hours for you to have anything to share...and then only to discover the other peers on the torrent already have what you do, so your upload side goes idle most of the time. This type of torrent often goes through feast-and-famine behavior: upload idle for most of the time, then maxed out for a handful of minutes to be idle again possibly for an hour or more. The download side of the same torrent mirrors the upload side, only a little leading it...as you cannot upload what you haven't gotten yet.

Really small torrents often complete within minutes, so peers are usually short-lived on those torrents and once again your upload side goes idle most of the time! If the really small torrents have lots of seeds, peers disappear even quicker. If they have only 1 very slow seed which shows up only rarely, then all the peers quickly end up at the same percentage.

In other words, I dont want the uttorent client to stop seeking when he reached the maximum active torrents number even that the bandwidth is not full used. If the UL bandwidth is not full used, I would like a behaviour to keep puting queued torrents in seeding status till he finally can match up 3 torrents who makes up the maximum bandwidth usage.

Already exists:

Advanced settings, queue.dont_count_slow_dl and queue.dont_count_slow_ul -- if set to TRUE, then if a torrent is downloading AND uploading slower than 1 KiloBYTE/sec, it doesn't count as an active torrent and more torrents will be started. The 1 KiloBYTE/sec limit is hard-coded...possibly to prevent abuse.

There are SERIOUS problems with really slow upload speeds due to piece size. When already low upload speeds get subdivided further between lots of torrents and then again between multiple upload slots on EACH torrent...speeds measured in bits-per-second can result. Only whole pieces get shared. A MILLION peers each with 99% of 1 million different pieces are useless to each other.

This is why a reasonable minimum upload speed PER active upload slot must be enforced by the BitTorrent client. BitComet does not do this. It sucks. µTorrent doesn't do a good job -- it will decrease the number of used upload slots PER torrent all the way down to 1, but it won't move an active torrent to queued status unless active torrent max is exceeded.

[Torin]

Advanced settings, queue.dont_count_slow_dl and queue.dont_count_slow_ul -- if set to TRUE, then if a torrent is downloading AND uploading slower than 1 KiloBYTE/sec, it doesn't count as an active torrent and more torrents will be started. The 1 KiloBYTE/sec limit is hard-coded...possibly to prevent abuse.

queue.dont_count_slow_dl and queue.dont_count_slow_ul are both TRUE but the 1 KiloBYTE/sec limit is still very generous.

This is why a reasonable minimum upload speed PER active upload slot must be enforced by the BitTorrent client. BitComet does not do this. It sucks. µTorrent doesn't do a good job -- it will decrease the number of used upload slots PER torrent all the way down to 1, but it won't move an active torrent to queued status unless active torrent max is exceeded.

by any chance, this feature - a configurable minumum upload speed per active upload - is planned for next releases of utorrent?

[hermanm]

Is there a defined interval that a queued seed communicates with the tracker and/or peers to prioritize seeding?

[Switeck]

Yes, it is somewhat generous...and can cause LOTS of started torrents if upload speed is set really low relative total number of max active torrents.

[wyrmchild]

I've experienced the same problem as Timwi, uTorrent just doesn't rotate the active torrents as much as it should (I have to manually stop/pause some of the active ones so it actually realizes to do something with the other torrents, instead of the current active one with 0 peers) And yes, I have scraping, don't count slow ul/dl and every other fancy option enabled that should, according to you and the docs, help with this.

Edit: Except scrape stopped option is false, since having it enabled won't make uTorrent start seeding the queued ones any faster (unless uTorrent is really bugged).

Edit2: I only impose upload speed limits on torrents that have reached the desired sharing ratio, no auto-stopping since that's undesired effect. I still want to seed, but the torrents are of lesser priority for me once the seeding goal is reached.

[hermanm]

I can understand rotation not being as quick as you'd like. I guess there's a balance you have to strike a balance between availability and bandwidth efficiency in checking peer & seed status. If you remove queued seed files on your hard drive to a different location, you'll get a sense of how quickly the rotation occurs. Personally, I think the rotation interval is adequate.

[wyrmchild]

I personally would like it to be more configurable. Since I have scraping enabled, it should switch away from torrents with 0 peers within a short time period (the time taken for two announces should be enough) and return when scraping says there are new peers. Currently it seems to hang on them for days. Especially on the ones that the tracker reports as having peers but never can connect to them or never get incoming connections from them.

[cherami]

I am new to this forum so first of all hi to one and all. I experianced a problem today with all torrent except one being Queued Seed. After reading all the help could not find a reason and then found this posting. I then realised that due to an earlier change in config I had changed the box in the speed guide to a lower bit rate because I had a error message and the help file told me to lower it., I have now put it back and all my Queued Seed are back to normal. I have know idea of all that is being said on this posting, but just wanted to say that what I did worked as far as Queued Seeding was concerned.

[jewelisheaven]

Queued Seed means due to your Ctrl-P -> Queueing settings your FINISHED torrents were no longer being seeded because other HIGHER-priority torrents were in the SEEDING status. Thanks for your input.

Edit: (regarding queue.dont_count_slow_dl queue.dont_count_slow_ul )

The addition of altering the queue.dont_count_slow_dl/ul options has been added to the current in-development build of 1.8. It defaults to 1 KiB but you can change it as you see fit