Does initial-seeding have known "bytes uploaded" inefficiency problem?

[lblisters]

I have the problem that initial-seeding, even in good conditions (30-100 peers), requires 2-3 times the data to be uploaded before it is completely available. This is more than in normal seeding mode, and seems to violate the "frugal" goal. Note that I am not complaining about maxing my bandwidth, minimizing time to completion, or getting one peer to 100%.

I have read many many forum posts on super-seeding. As of June 06 utorrent was known by the admins to be buggy in this exact way. I read an offhand comment that "super-seeding was supposed to be fixed" in version 1.6. In september DreadWingKnight said utorrent took 2-3 times data size in super-seeding mode "until we fixed it". The FAQ says super-seeding mode "is supported".

Before I ask for help troubleshooting my circumstance, I just want to clarify that this is not a known issue with the utorrent 1.6 client implementation (barring all the discussion about possible strategies to deal with cheaters and bad clients). I will make my troubleshooting request in a new topic unless advised otherwise.

Thanks.

[Ultima]

Initial seeding doesn't appear to be problematic for most users, so I guess you can classify it as being "not known" ;\

What clients were the peers using in the swarm?

(Anyhow, no need to switch to another thread to troubleshoot your problem, it's fine keeping it in this thread)

[lblisters]

Thanks Ultima,

State of my current torrent: uploaded data equivalent to 120% of torrent data size, availability is 60%. This is 50% efficient. My upload is maxed out at 40kB (limited from 46kB physical bandwidth), most peers are roughly tied at the head of the pack within 1% of total availability. I'm connected to 110 out of 150 peers, all claim to be using utorrent or azureus (strict tracker.)

50% efficient seems to be relatively poor, going by DreadWingKnight's comparison.

I have been unable to find a good guide on maximizing the efficiency of initial seeding. I've spent hours googling, reading forums, and have been unable to identify the source of the inefficiency. Pointers to existing guides and answers to my questions below greatly appreciated. Apologies for the length.

I posted about a previous experience '>here.

I understand that each circumstance is unique, and that initial seeding mode can be slower due to peers blocking etc... I'm aware that utorrent is a fair client, will not add features for banning clients or peers. I've read of the dangers of assuming that some peers are bad and being ban-happy. I'm aware that cheating / abuse / spoofing does happen. I don't want to rehash those fair/hostile issues, I just want to better understand my circumstance.

In what case will utorrent upload the same piece twice? Is it only after it has uploaded all pieces once, or will it upload the same piece again if it does not become available? I haven't found a detailed explanation of utorrent's initial seeding algorithm, nor is the source open.

Is the 50% efficiency strong evidence of cheaters?

Whether cheaters or just circumstances, how can I understand and protect against inefficiency? What happened to all those bytes I uploaded, which don't seem to contribute to the total availability?

How can I detect cheats? I don't see peers reconnecting with 0%. I don't see any peers far ahead of others. All peers claim to be azureus or utorrent.

What peer statistics can I trust, and which can be spoofed? How can I detect the source of inefficiency by monitoring the peers table and log window? I've enabled all columns in peers table and am logging torrent traffic to Logger tab.

Is there a reference for what every column in the Peers table means? Didn't find it in user guide or FAQ.

Is it possible peers are getting a really high rate of failures from me? Any way to detect this?

Thanks!

[µtorrent-Guest]

are they "real" azureus? e.g. do they identify as "Azureus/number" or as "AZ2500xxx" or something like that?

if they identify as real azureus, any chance its THEM™ in the 38.100.x.y and 208.10.x.y range?

[lblisters]

Thanks utorrent-guest, those are just the type of tricks which are hard to learn from searching the archives.

Everyone shows up as utorrent/1600 azureus/2306 etc. I see none of the form AZ500xx. Some users were very unhappy when the tracker banned bitcomet, so it would not surprise me if some tried to spoof as another client.

No peers are in the 38.100.nn.nn or 208.10.nn.nn ranges.

I have temporarily put some screenshots online, of the peers table sorted different ways - most complete, most uploaded to, etc. here

I don't see anyhing obviously suspicious: rapid reconnects, stay ing at 0% complete, having way more complete than anyone else. Sometimes I get two connections from the same IP 3 seconds apart, but never more than two.

[Ultima]

It's easily possible that you're seeding corrupted pieces, but then the question comes as to why most of the peers haven't banned you yet... What router are you using?

Unfortunately, I'm not really informed on the technical details of how µTorrent does its initial seeding, so I'm short on answers and ideas =\

[lblisters]

D-Link DI-514 router, ports are forwarded. Tracker confirms my ports are open. I see now that this router is known to have problems consistently forwarding UDP ports... maybe I'm losing connections a lot due to this? Is there a utorrent setting I can change to test/mitigate this? I don't notice any messages in the log window about broken connections, would they be logged? Would dropped partial pieces contribute to the Upload statistic?

The router is a little crappy in general, it sometimes needs to be power cycled, but I've never noticed a specific P2P problem.

Argggh, I see now that "DI-5xx" is on the Due to UPnP list, my search for "DI-514" missed this. I'm forwarding both HTTP and UDP on a single port. DMZ is not on. My global connections was set to 500, although I had a limit of 160 per torrent and only my upload is active. I've never noticed large numbers of hash fails (I do watch the General tab sometimes) when downloading.

Crap, and after I thought I'd long gone through every router setting, I found a tab within a tab with a "gaming" option which was enabled! Even though DMZ was explicitly disabled... I also disabled UPnP there. I'll see whether this resolves it.

UPDATE: no, Ratio still increases 0.02 for every 0.01 increase in Availability.

[µtorrent-Guest]

1) Ok, from the pictured peers IP I guess we can now rule out generally malicious intend on your peers part :-)

2) you might want to test initial seeding that torrent with the (afaic latest) beta just to see if that changes something

http://download.utorrent.com/beta/utorrent-1.6.1-beta-build-483.exe

(just stop end close your currently running µt version and start the latest beta 1 minute after that. it will keep all the settings and continue)

[lblisters]

1) Thanks, this is good to know. I don't have much experience with malicious peers so I was not confident to judge.

2) My current upload has finally finished with a ratio a little above 2.0. Before my next upload, I will replace my router with WRT54GL to eliminate it as a possible source of problems. I will first test with my current version of utorrent (build 464) and the new router. If this exhibits the same problem, I will try with latest beta version of utorrent. I will post my results here in a week or so.

Thanks again for the help.

UPDATE:

I've upgraded router to WRT54GL with dd-wrt 23sp2 firmware, did not upgrade uTorrent, and recently started a new upload on same tracker with similar conditions. I watched in anticipation as super-seeding got up to speed, availability tracked ratio pretty closely - EG ratio .021 avail 0.017.

But over time the efficiency slipped before my eyes.. from 80% to 70% to 60%. When Ratio hit .801 and Avail was only .431 (54% efficiency) I knew my problem was not solved.

At this point I upgraded uTorrent client to latest 1.61 build and restarted torrent. Ratio is now 1.28 [upload should be complete] and Avail .606 - 47% efficiency

Now that router and client version have been eliminated as possible sources of the problem, we ruled out malicious peers, what is left? Next upload I will try using BitTornado for super-seeding. If it has the same problems, maybe my ISP is blocking or mangling packets - I would then try encryption as a workaround. If BitTornado has no efficiency problem, then i will compare all client settings carefully.

UPDATE: I switched to BitTornado, it finished the last 40% of the torrent data with 85% efficiency. Comparing preferences, I see only two differences:

1) uTorrent has been on port 32459 for months, BT is randomized between 32460 and 32800. The range 32459-32800 is forwarded, both clients confirm incoming connections.

2) BT has encryption enabled by default and also an option "break-up seed bitfield to foil ISP manipulation" enabled by default. uTorrent defaults to no encryption.

I would prefer to use uTorrent, so I guess on my next upload I'll try with encryption enabled and see what efficiency is.