descra Posted July 5, 2006 Report Share Posted July 5, 2006 hi all, First of all, uTorrent is the best bittorrent client - that's no daubt.But ther's a one serious problem ...A number of wasted data could really make you angry!I read all FAQ (propably 4 times) and I found out that is NOT the uTorrent problem and number of wasted data always depends on every single torrent - for example when torrent is broken, poisoned etc.ok, I can understand it but ......Today for some test I downloaded some movie.Resaults?uTorrent: movie 700 MB, wasted data 40 MB !Azureus: the same movie! at the same time tracking on the same tracker, almost the same number of peers'n'seeds, wasted data? - only 5 MB !In my opinion is a real bull schit that uTorrent has no influence of number of wasted data!So ....I need an expert's advice.Could you tell me whitch settings - I mean "Advance options" and others may affect a number of wasted data?I'll be really thankfull for your help and some wise advice.thanks in advance Link to comment Share on other sites More sharing options...
Ghost21 Posted July 5, 2006 Report Share Posted July 5, 2006 Why would there be any settings to increase or decrease wasted data? Link to comment Share on other sites More sharing options...
Klaus_1250 Posted July 5, 2006 Report Share Posted July 5, 2006 Could be usefull for people using low bandwidth connections (though they're less likely to have wasted bandwidth) or who have to pay per megabyte.As far as I know, wasted data occurs when the same data is downloaded from different peers (among others) and I think there are a few switches you can make to prevent this, at the cost of overall speed. But on my installation, I don't see a lot of wasted bandwidth. Link to comment Share on other sites More sharing options...
descra Posted July 5, 2006 Author Report Share Posted July 5, 2006 Why would there be any settings to increase or decrease wasted data?sorry for being not enough preciseI mean whitch settings may theoreticly influence to downloading less number of wasted data.I'm realistic and I know that generally it's NOT posibble to download some stuff without even the small number of wasted data but you should admit that 40 MB per 700 MB movie is a "bit" too much, especially when you downloaded the same movie at the same time, using different bittorrent client and the number of wasted data was a lot smaller - like I said before:the same movie on the same tracker durring very similar number peers'n'seedsuTorrent: 40 MB w.dAzureus: 5 MB w.d.So ...Could you tell me whitch setting may theoreticly influence on number of wasted data - I mean for example: maybe disk cashe or number of open upload slots or some others Advance Settings ?please help Link to comment Share on other sites More sharing options...
descra Posted July 5, 2006 Author Report Share Posted July 5, 2006 Here are the list of options witch I think they may have some influence on number of wasted data and I'll be really thankful if someone verify that and tell me where I'm wrong and why (I don't say that they may increase or decrease but I think they may something to do)1. Disk cache - "Write out untouched blocks every 2 minutes", "Write out finished pieces immediately" and "Remove old blocks from cache "2. Protocol Encryption - "Allow legacy incoming connections"3. DHT and peer exchange enabled4. Too much numbers of trackers, for example 3, 4, 7 and more5. Report a different IP to the tracker6. net.max_halfopen7. diskio.flush_files 8. diskio.smart_hash9. diskio.coalesce_writesso ... could you tell me if some of these settings above may have something to do with wasted data?I mean witch of these options in 100% don't have any influence and why ?... and witch one theoreticly may have some influence and why ?... and at last witch one for sure have something to do and of course why ?thanks in advance for your help Link to comment Share on other sites More sharing options...
DreadWingKnight Posted July 5, 2006 Report Share Posted July 5, 2006 1> No effect - Disk cache doesn't affect wether a peer will send you part of a 16k block or all of it.2> No effect - Encryption has nothing to do with the amount a peer will send you3> No effect - DHT and Peer exchange have nothing to do with piece data transfer4> No effect - Trackers have no bearing whatsoever on piece transfer5> No effect - Reporting a different IP to a tracker will either fuck you over speed-wise or get ignored by the tracker6> No effect - A half-open connection isn't sending piece data7-9> No effect - Disk IO controls have nothing to do with piece data transfer Link to comment Share on other sites More sharing options...
Switeck Posted July 5, 2006 Report Share Posted July 5, 2006 Setting bt.prio_first_last_piece to true might increase wasted data a tiny amount, as µTorrent would be particularly aggressive at trying to get the first and last piece of a torrent. (or is it 1st and last piece of each file in a torrent too?)I believe net.low_cpu set to false (default setting) means run aggressively and queue up lots of pieces at once (even duplicates from different peers/seeds?) -- while setting net.low_cpu to true would queue up fewer pieces at once. That in theory might have an effect on wasted data amounts.How many connections per torrent you allow would certainly have an effect on wasted data. If you only connected to 10 seeds/peers PER torrent, who are you going to get duplicate data from? Also, how many upload slots you allow (and how fast each uploads on average) might have an effect on how many peers are uploading back to you at once -- if you have too few, the only peers that will consistantly upload to you are the ones you are uploading to. If you have too many, then peers will upload to you very seldom and in random bursts. (optimistic unchoke for the win!) Seeds will likely upload to you regardless, but they often aren't any faster than peers and likewise are pretty random in who they upload to. Having only a few peers uploading to you quickly (because you're only uploading to a few peers quickly, because your upload slots are only 2-5) would seem to reduce the chances of duplicate data. Having LOTS of peers uploading slowly to you at once would seem to cause immense amounts of wasted data.Even running multiple torrents at once (versis 1 at a time) would increase wasted data considerably, simply because you can't be uploading as quickly per torrent and such peers on each torrent would be uploading back to you more sporadically and slower.I do know that µTorrent goes into "endgame mode" when there are fewer than 50 pieces of the torrent left to get. When that occurs, it tries more peers at once for the same pieces so it increases the likelyhood of completing the torrent at all. It really doesn't need to on torrents with lots of peers/seeds -- as that's probably where it incurs the highest wasted data as a result!Assuming it's not something more insiduous like a bug, µTorrent probably is just more aggressive at getting pieces than Azureus -- which results in more wasted data overall.A NASTY side-effect of high wasted data rates...is it is likely caused by often having multiple peers/seeds uploading each piece. If there is a seed/peer that is poisoning the torrent, they can "hide" behind the other peers/seeds -- and you'd end up kicking/banning lots of harmless peers/seeds that just had the misfortune of contributing a part of the same chunk that failed hash! Link to comment Share on other sites More sharing options...
kurahashi Posted July 5, 2006 Report Share Posted July 5, 2006 My advice:- try decreasing number of upload slots- try decreasing number of connectionsAnd last but not least make sure that your upload graph is flat. No zigzags, no electrocardiogram, just pure flatness.BTW: my usual level of wasted data is ca 0.5% (0.5MB wasted per every 100MB downloaded) Link to comment Share on other sites More sharing options...
descra Posted July 5, 2006 Author Report Share Posted July 5, 2006 First of all thank you all for your answers and took this problem seriously without any silly replys from your side So ... propably the best thing will be when I simply show you my settings I use.1. Connection: upload 12 Kb/s (my MAX upload bandwitch is 16 Kb/s and upload is independent from download speed), 3 open upload slots, maximum number peers per torrent: 902. Advance (most important settings):net.low_cpu - falsenet.max_halfopen - 20 (I've got patched TCP/IP.sys so this number above 8 is fine)queue.prio_no_seeds - truebt.scrape_stopped - falsebt.compact_allocation - falsebt.send_have_to_seed - truebt.connect_speed - 20bt.prio_first_last_piece - falsebt.allow_same_ip - falsepeer.lazy_bitfield - truepeer.disconnect_inactive - truediskio.flush_files - falsediskio.sparse_files - truediskio.use_partfile - truediskio.smat_hash - truediskio.coalesce_writes - true3. Disk cache:Write cache enabled with all optionsRead cache disabled (I read that the read cache is not need it when the upload is under 12 Kb/s) 4. DHT disabled (of course I switch it on when the tracker got down but normaly is off)so... as you see these are my settingsImportant thing is I do NOT download broken or poisoned torrents and in 99% tracker works fine.I'm not behind NATFirewall is well configuredso .... should I change some (or even only single one) of this options witch I talk about above? Like I sad before I downloaded today some 700 MB movie. Torrent wasn't broken, tracker worked fine, settings was like I said above but wasted data was 40 MB! - a bit too much, isn't it? the most funny thing is that the amount of wasted data still increase with every newer ver. of uTorrent (now I use the newest one)Azuerus wasted data: ~ a few MBuTorrent w.d.: ~ 20-40 MB!so I got laugh when I read in FAQ that wasted data has nothing to do with uTorrent settings please helpthanks, Link to comment Share on other sites More sharing options...
Nefarious Posted July 5, 2006 Report Share Posted July 5, 2006 that u tried the same torrent doesnt mean u are gonna get the same results, u could have ended connection to a peer that was causing the wasted data when in utorrent and then he was not in azureus connections as u cannot be sure u are gonna connect to the same people or if they just left the swarm, i've seen this behavior a few times when first i got 20MB wasted for 1.0GB downloaded, but then after disconnecting and reconnecting to it i got no more than another 2MB wasted for 2more GB downloaded, it's certainly unfair to compare like that unless u are always getting the same results Link to comment Share on other sites More sharing options...
descra Posted July 5, 2006 Author Report Share Posted July 5, 2006 ... it's certainly unfair to compare like that unless u are always getting the same results... but I got the same resaults all the time and that's the problem - a lot MB of wasted data I wouldn't be talk about it if it will happend sometimes but it happends always so that's why I need some wise advices Link to comment Share on other sites More sharing options...
Switeck Posted July 5, 2006 Report Share Posted July 5, 2006 Try changing these values to this:maximum number peers per torrent: 30net.low_cpu=Truenet.max_halfopen=6bt.connect_speed=10Are you by any chance running multiple torrents at once? Link to comment Share on other sites More sharing options...
Klaus_1250 Posted July 5, 2006 Report Share Posted July 5, 2006 I'm wondering to what extend waisted data depends on the quality of your internet connection / swarm size / swarm speed. I use µTorrent with quite aggressive settings, but download mostly from smaller swarms and/orslower ones than my connection (250KB/s down max were my connection is 515KB/s) and I see little waisted data. Usually it is about 0.1 to 1%.I think Kuruhashi post says the most important things. Keep your connections down and make sure you set proper up- and download speeds (pratical ones, not theoretical).I think connecting to a lot of peers and download at 1KB/s or less from them is a sure way to get waisted bandwidth. (I think that is where my waisted bandwidth comes from). Link to comment Share on other sites More sharing options...
jivan Posted July 6, 2006 Report Share Posted July 6, 2006 hi all, noob here but get something like 6 mb on 800 MB downloads. Although do keep tweaking stuff here & there. One of the things which has not been put out is the ipfilter. So use a good ipfilter. Other thn that my settings are similar :-maximum number peers per torrent: 10net.low_cpu=Truenet.max_halfopen=16bt.connect_speed=25 & my siggy gives my dl/ul ratios. Any suggestions for me anyone :- Link to comment Share on other sites More sharing options...
descra Posted July 6, 2006 Author Report Share Posted July 6, 2006 I'm wondering to what extend waisted data depends on the quality of your internet connection / swarm size / swarm speed. I use µTorrent with quite aggressive settings, but download mostly from smaller swarms and/orslower ones than my connection (250KB/s down max were my connection is 515KB/s) and I see little waisted data. Usually it is about 0.1 to 1%.I think Kuruhashi post says the most important things. Keep your connections down and make sure you set proper up- and download speeds (pratical ones, not theoretical).I think connecting to a lot of peers and download at 1KB/s or less from them is a sure way to get waisted bandwidth. (I think that is where my waisted bandwidth comes from).quality of my connection is more than good - is stable without any gaps or interraptions during downloading or uploadingI think the most important thing is what you said at the end of your post - I mean "connection to lot of peers and download at 1 Kb/s or less from them"I tried to find out any option in uTorrent settings similar to Azureus one to ignor download speed from peer when is slower than for example 1 Kb/sI didn't find that and maybe that's the key problem?I don't know, maybe this is an inner option witch we normally don't notice and haven't access to change it and it's automatic in use but I'll be thankful if someone could admit or deny if uTorrent is able to ignor specific download speed from peer for example if it's slower than 1 Kb/s ?so ..... is it posible to set in uTorrent settings the minimun speed of download from single peer under witch level that kind of connection will be ignor?p.s.When I asked if it's posible that disk cache has something to do with wasted data I got answer that NObut .... I'm still wondering about one option "Write out untouched blocks every 2 minutes"Decription of this option says: " ...will control whether µTorrent flushes the chunks from incomplete pieces to disk if the piece has been inactive for 2 minutes"so ..... theoreticly, is it posible that after 2 minutes uTorrent remove some incomplete pieces only why because they were inactive ??? ..... but in my opinion it doesn't mean that after 2 minutes they couldn't be usefull - maybe after this 2 minutes they could be still ready to use.for example, some torrent is cuts in 1 MB pieces (NOT parts like *.rar) I stared to downloading one of this pieces, I downloaded 800 KB (224 KB still to come) but sudenly BREAK - 2 minutes breakand what - that 800 KB segment will be remove ??? even that after 2 minutes could be ready to use and only wait to be finish ?Situation: I started downloading one of that 1 MB, I downloaded 800 KB but peer from whom I download it sudenly stop sending me that pieceis it posible to finishing downloading that incomplete piece from other peer? ... or it's ONLY posible to finishing downloading it from that first peer from whom we stared ?thanks a lot, Link to comment Share on other sites More sharing options...
Switeck Posted July 6, 2006 Report Share Posted July 6, 2006 Actually, if your maximum number peers per torrent is set to only 10, you don't really need a high halfopen value. The halfopen value and connect speed combined is how fast µTorrent tries new ips to get new peer/seed connections. But if you've set the limit to 10, it doesn't need to be trying 16 half-open at once with 25 new connections (in the handshake process) at a time. Klaus_1250 was saying if you download from lots of peers at only 1KB/sec or less, you get more wasted data. In translation: if you allow lots of peers (high max connections per torrent) then you increase your wasted data -- as you're downloading from MORE peers at once, at a slower speed each (if you're already downloading at >100 KB/sec), and thus peers are more likely to send you the same thing."Write out untouched blocks every 2 minutes" copies the blocks to HDD from ram, it does not then 'forget' where it was at in the block. That's not the problem-causer."some torrent is cuts in 1 MB pieces (NOT parts like *.rar)"Actually, each 1 MB piece of a torrent so set is 1 "chunk" of the torrent -- it is the smallest amount of that torrent that can be individually checked for a bad hash which means that chunk has an error in it.Once you get 800 KB of a 1 MB chunk from 1 peer, ANY other peer/seed can come along and try to finish that chunk -- and may even be requested at the same time as the original peer is still requested to send the rest! So both can end up sending the remaining 224 KB...giving you 224 KB of wasted data on 1 MB. (21.9% wastage rate!)You only have indirect control over how many chunks µTorrent tries to download at once. Then again if you put scheduler into seed-only mode, it will finish unfinished chunks (or may only finish current requests) and then only upload what it has. But that would be an absolute pain to test more than once.The settings we have suggested are meant only to reduce wasted data amounts. Please test them.Upload speed max is not totally independant of download speed. While downloading, some upload bandwidth is needed to keep the download going. From my findings, it's a 50-to-1 ratio (or worse) -- so if I'm downloading at 50 KB/sec, I need at least 1 KB/sec of upload bandwidth to keep that download going. If I was downloading at 500 KB/sec, I'd need 10 KB/sec or more upload bandwidth...a noticeable amount indeed of my total! Link to comment Share on other sites More sharing options...
kurahashi Posted July 6, 2006 Report Share Posted July 6, 2006 Upload speed max is not totally independant of download speed. While downloading, some upload bandwidth is needed to keep the download going. From my findings, it's a 50-to-1 ratio (or worse) -- so if I'm downloading at 50 KB/sec, I need at least 1 KB/sec of upload bandwidth to keep that download going. If I was downloading at 500 KB/sec, I'd need 10 KB/sec or more upload bandwidth...a noticeable amount indeed of my total!50:1 ratio? Thats really low!In my case, average ratio is 20:1. That means, every 20kB/s of download generates additional 1kB/s of upload overhead. Additionally, upload generates overhead by itself, in my case it is 8:1 (every 8kB/s of real upload generates additional 1kB/s of overhead)How does it work in practice:- my upload limit in uT is set to 24kB/s (192kb/s)- while seeding only it is using 27-28kB/s- while downloading at 60kB/s, upload used is 30kB/s- while downloading at 120kB/s - upload is 33kB/s- while downloading at 240kB/s - upload is ca 39kB/s - just to be barely safe in the parameters of my day connection (which is real 2048/320)So, to be able of using my whole 256kB/s download, I have to limit upload in uT from 40kB/s (which I have) almost by a half to 24kB/s. Sounds incredible but its true. Link to comment Share on other sites More sharing options...
Klaus_1250 Posted July 6, 2006 Report Share Posted July 6, 2006 @ kuruhasi: Are you using SACK and have you set a good MTU size? 20:1 and 8:1 sound too high. Link to comment Share on other sites More sharing options...
Switeck Posted July 6, 2006 Report Share Posted July 6, 2006 The 50-to-1 ratio (at best) was for simple file transfers between computers on my network. This lacked BitTorrent protocol overheads and was only a single connection at a time. I have the TCP receive window set to 256960, which also probably lowers overhead...at the price of higher latency due to chained packets. MTU is set to 1500 bytes. Link to comment Share on other sites More sharing options...
kurahashi Posted July 6, 2006 Report Share Posted July 6, 2006 Nah, my current ISP provides the net through normal ethernet connection and my settings are ok, sack on mtu 1500, window size = 255*(mtu-40) so...I even try to play a little with the TCPIP settings lowering mtu a little to 1492 (the only reason to do so is possible communication improvement with the remote peers which are using xDSL PPPoE connections) but I can't see any differences. Link to comment Share on other sites More sharing options...
Switeck Posted July 6, 2006 Report Share Posted July 6, 2006 Actually, it probably does take 9 bandwidth bits (or more) to transfer 1 file BYTE. This is due to all the overheads. I even figure it at 10 bandwidth bits per file byte, just to keep the math simpler.So if someone says they have 300 kilobits/sec of upload bandwidth...I say limit upload speed to no more than 30 KB/sec. It's about the same as 300 / 8 * 0.80 Link to comment Share on other sites More sharing options...
infused Posted July 6, 2006 Report Share Posted July 6, 2006 I get hardley any wasted data... and I send around 1tb a month. Most clients are uTorrent though, but not all... Link to comment Share on other sites More sharing options...
kurahashi Posted July 6, 2006 Report Share Posted July 6, 2006 Switeck, but don't confuse modem overhead with TCPIP and application overheads please. We can distinguish: modem/physical layer overhead - when your cable or xDSL modem synchronizes, for example 512/128, the real speed available for TCPIP transfers are even 15-20% lower than your synchronization speed (due to PPP encapsulation for example)TCP/IP overhead - this one is much harder to estimate, usually consist of the all the handshake and ack packets, sometimes IP packets are lost and there is need for retransmission application (uTorrent here) overhead - all the torrent communication excluding the real data exchange - with tracker, other peers, pex, dht etc.What we have to remember, is the upload cap setting in uTorrent concerns real data upload speed, without any of this overheads. Which can be really big, as seen this in practice. Link to comment Share on other sites More sharing options...
Switeck Posted July 7, 2006 Report Share Posted July 7, 2006 Yes, I know...µTorrent's upload speed cap does not consider overheads...which can add a considerable extra bandwidth burden to the line.I set 40 KB/sec as my upload max...and it ends up using close to 360 kilobits/sec upload bandwidth -- though that depends on BT traffic.I did say ALL the overheads, which in µTorrent's case are modem/physical layer overhead, TCP/IP overhead, and application/protocol (uTorrent here) overhead.And that's assuming failed hashes and lost packets are quite rare.I'm saying 9-10 bits bandwidth = 1 byte of a file transferred as a rule-of-thumb/rough estimate. But it is good enough for upload speed calculations.There is certainly a possibility that the Speed Guide's speed test is measuring bandwidth based on file transfer rates, making some of the overheads (at least modem/physical layer overhead) "transparent" in the calculation.If I recall correctly, PPPoE looks very wasteful. Link to comment Share on other sites More sharing options...
Firon Posted July 11, 2006 Report Share Posted July 11, 2006 Chalk up the wasted differences to coincidence. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.