Private flag and local peers

[sysrev]

1.7.1 ignores the private flag in a torrent when connecting to local (LAN) peers.

Steps to reproduce:

1) Load uT on machine 1 and start a private torrent running.

2) Load uT with "local peer discovery" enabled on machine 2. Load a copy of the torrent that has its announce address changed/passkey altered or invalidated in another way which doesn't alter the hash.

3) Watch machine 2 connect and download from machine 1.

This could be pretty serious for private torrents on large LANs (universities for instance).

When bitcomet ignored the private flag with PEX/DHT, it pretty much killed that client for use on private trackers.

[DaVrOS]

Yup, you are correct. Azureus Does seem to support that private flag over local peer discovery though. Wheras UT 1.7.1 does not.

[Ultima]

The behavior is being modified as we speak.

[tequilavip]

last client that did something like this got banned at 99% of private trackers (bitcomet). I liked the basic functionality of that client, but it's (earned) reputation as a "cheater's client" and it's death @ private sites is what drove me to utorrent in the first place. The scary possibility is that it is unknown to us if utorrent would also do this over an entire ISP... does it? And is/was it intentional or a bug?

This would potentially allow members to alter the announce and download a file without ever even announcing to the tracker, and even allow banned ratio cheaters to download even though the tracker may have banned their IP address from connecting, and while these downloader would not be reporting any stats to the tracker, the seeder would still get credit for all data uploaded. (if I properly understand what may be occurring -please educate me if I am wrong about this!)

Another potential consequence is that it would make cheat detection much more difficult/erroneous when comparing total data UL'd on a torrent vs total data DL'd. A member of a private torrent site uploading data to a peer who is not reporting stats to the tracker would make it look as though that seeder is falsifying upload stats, ie on a torrent with 1 seeder and zero leechers shown on the tracker, but the seeder is showing data being uploaded.

edit: this is of more concern than it should have been because it is the second stats-related "bug" in the last week... Also, I was writing as the above comment was being posted... Glad to see staff are aware of this

[Smoovious]

Dude... chill out... if the bug was reported and no action was taken to resolve it, THEN you'd have cause to make a big deal about it.

It wasn't intentional, and BC isn't the only client that has had private-flag leak bugs before. Az has had them... older versions of µT has had them...

Reputation isn't made over a bug.

If you want to ban 1.7.1 b3360 as a result of this bug, then at least it would be a valid reason for this build.

As for the broadcast scope. That all depends on the hardware. In practice, it shouldn't leave the LAN, but some networks may allow a wider scope or none at all.

-- Smoovious

[tjobo]

...I think it's a safe bet that within a few days, most private trackers will follow the bitmetv lead and ban uT 1.7.1. I guess all the "paranoia" was justified...

Sent By: tequilavip - 2007-07-21 03:25:01

You were just wating to post that, didn't you?

This bug has still nothing to do with the privacy acuses had. This has nothing to with claims that utorrent is sending information to the RIAA, MPAA or whatever.

I bet there will be a very few *if any* that actually will exploit this bug before it's fixed.

And by the way.. what about stop banning users from having an opinion in the shoutbox? I hate to use my "reserve" accounts.

[Firon]

It only does it in a LAN, not across an entire ISP. Because of that, it's limited to people that are quite local to you, and are people who you could easily cheat with anyway (if that was your intent).

[system]

"are people who you could easily cheat with anyway"

Providing you know everybody on your lan.

In a university or a big company, that's unlikely.

I'd be interested if anyone could test with windows seed boxes to see what happens in a datacenter.

Edit: don't know if you caught it on our forums, but the new download stat calculations are against the most basic specs for BT.

uploaded

The total amount uploaded so far, encoded in base ten ascii.

downloaded

The total amount downloaded so far, encoded in base ten ascii.

left

The number of bytes this peer still has to download, encoded in base ten ascii. Note that this can't be computed from downloaded and the file length since it might be a resume, and there's a chance that some of the downloaded data failed an integrity check and had to be re-downloaded.

left is specifically stated as being incalculable from the file size and downloaded because downloaded should include failed pieces.

[Firon]

They ALSO need the .torrent file to do it, so it's not like you're suddenly gonna have a huge influx of people magically stealth leeching torrents.

And as for breaking the spec... The download value is pretty meaningless anyway, except to private trackers, for whom this change is for (you know, benefiting your own users). Now you're just trying to find petty stuff to bitch about. Ratio tracking is a misuse of the BT protocol, so maybe you should just go become an open tracker right now.

[Ultima]

I still fail to see how all the "paranoia" was justified, just because of this change. Oh, so because LPD didn't restrict to non-private torrents (and even though a patch was pushed in quickly), it means the MAFIAA controls µTorrent and it's now unsafe? That was the original "reason" for banning right?

I can't tell from the PM tjobo posted up above whether it means they plan on unbanning µTorrent after the fix gets released, but oh well, here's to hoping.

[system]

The download value is pretty meaningless anyway, except to private trackers, for whom this change is for (you know, benefiting your own users). You're just being obnoxious and trying to find petty stuff to bitch about.

How is this change for private trackers?

Person a, using uT, reports 500MB of upload.

Person b, also using uT, reports 20MB of download because of hash failures.

With only 2 people on the swarm when that happens, what is the expected reaction of staff?

Either one of the users is banned, or uT is banned.

None of those outcomes helps the tracker or the users.

Having a correct total of download amounts can actually help all trackers (including public ones) in these days of certain people flooding swarms with bad data. When we can see that every peer is downloading a lot more than is expected, we can see that something is poisoning the swarm.

This change is against spec, and is definately not helpful to private trackers.

Calling me obnoxious doesn't change the fact that the only people the change helps is uT users who want a better ratio on private trackers at the risk of losing their accounts.

Might as well tell everyone to use ratiomaker, same risk, better results.

BTW, I defended uT against the claims of it having a "phone home", hardly the actions of someone looking for something to "bitch about".

Edit: in response to your edit

Ratio tracking is a misuse of the BT protocol, so maybe you should just go become an open tracker right now.

If that's the case, what about the parts of the spec where upload and download amounts are sent to the tracker. If BT was designed to always be open trackers, we would need nothing more than the event variable. There's no need to send uploaded/downloaded and left if trackers were never meant to keep track of these things.

Glad to see brams views on private trackers are really rubbing off here.

[tequilavip]

it also doesn't help when people posting in support of uttorent are cheats...

And by the way.. what about stop banning users from having an opinion in the shoutbox? I hate to use my "reserve" accounts.

he's braggin in this bug forum about violating rules of sites to which he is a member (most sites only allow one account per member)

This has nothing to with claims that utorrent is sending information to the RIAA, MPAA or whatever.

no one at our tracker ever signed on to that hysterical rumor, we simply stated that we didn't know one way or the other if new versions of utorrent were safe to use or not. Once someone gets banned at private trackers for cheating because of these "features" that now appear to be intentionally included in 1.7.1....

anway, back on topic, is it the intent of uTorrent staff to "fix" these issues? What is your latest position on the exclusion of pieces failing hashcheck from total data DL'd in announces?

[sss]

offtopic !

tequilavip, he is teasing you because u are banning people because of few words in shoutbox apparently

[Ultima]

LPD "fixed" in 1.7.2, and as for the hashfailed pieces... http://forum.utorrent.com/viewtopic.php?pid=264441#p264441

Glad to see brams views on private trackers are really rubbing off here.

Sorry, we've always been of this view since LONG before BitTorrent Inc came into the picture. I'd look for proof for you, but between Firon, DWK, and I, we just have too many posts to actually sift through to get to posts where we voiced the exact same opinion (that private trackers were never what BitTorrent was designed for, and itself is an abuse of the protocol). That said, we still try to accomodate private trackers whenever we can -- do notice that the last 2 "bugfix" releases only matter to private trackers in the first place. With or without those changes, it wouldn't make one bit of difference in how µTorrent performed if you look at it down to the basic level of just transferring data (yes, 1.7 - 1.7.1 was a mistake, but it still wouldn't have made a difference for anyone besides private tracker users/administrators).

[system]

Leaving aside the difference of opinions on private trackers and the implementation of downloaded, may I resuggest something?

I tested azureus, and yes, it does drop downloaded after a piece fails hash checking. Fired off an email to one of the devs, and a new flag will be included in the next snapshot (today or tommorow). This flag will be called either corrupt or failed, and will be used to inform the tracker of how much bad data has been downloaded.

The mechanics of how it will work will have to wait for testing, but I think it'll be that downloaded stays the same as it is, and corrupt/failed is an amount to be added to downloaded for calculating totals and checking how much bad data there is.

This would allow tracker staff to still find cheats, spot problems with poisoning and make choices as to what is fair for all users. On a tracker with 90%+ az and ut peers, they may decide that downloaded without fails is fair. On a tracker with only 10% az and ut peers, they may decide to add fails to the download until all clients have this option.

[Ultima]

Up to the devs, I guess.

[system]

Confirmation from az:

downloaded will stay as now - total good data received. Corrupted

will have the total bytes received but thrown away as the piece hash check

failed.

Someone will be contacting rtorrent/libtorrent to ask for inclusion of the flag there as well.

[TooMuchTime]

Why is this so important to you? Other trackers (i.e. mine) don't see the need to know the amount of hashfails. Azureus has had this behavior for something like 20 months without it destroying the way everyone operates. And I'll bet this change is only made in Azureus version 3, not in the 2x series, seems to be superceded.

TooMuchTime

Admin, OiNK's Pink Palace

[system]

You don't consider it important that clients should have some uniformity in reporting?

If you don't know what downloaded means from one client to the next, then the stats being kept are completely pointless.

A users ratio may or may not be better than anothers ratio, regardless of what their profile says their ratio is.

If dropping failed pieces is fair for the users, then it should be fair for all users, not just those who choose a specific client.

What does it matter if there's an extra flag anyway? You'll still receive downloaded as you want it, and other trackers will be able to decide how they want it.

It'll require no change on your tracker and gives others the freedom to do their own thing, not oinks thing.

[TooMuchTime]

Agreed that I don't need to change anything to ignore this new information, but the rest of your post makes very little sense to me. We haven't had any uniformity in reporting since Azureus implemented not reporting download where there were hashfails. As µTorrent is the client with the largest percentage of peers on my tracker (well over a million peers), I asked the µTorrent devs to adopt this behavior to level the playing field. It mystifies me that you seem to be objecting to it.

Maybe you're relying on trying to detect if total upload equals total download on any given torrent. I found that to be an exercise in frustration but it may be more doable on sites with fewer torrents and/or peers than mine. I gave up a long time ago on trying to catch every last cheater or stealth downloader.

[system]

On huge torrents, the totals probably would go very far out of synch simply with users getting disconnected (and thus missing announces). In those cases, it's understandable you don't bother

I'm not doing anything with ratios at all on my own site (very small, all friends), but I think all the data should be there for use if it's needed. On some sites the staff manually check torrents, some they don't. Thing is though, someone with incredibly bad RAM/HDD sitting on a 10mb/s download could theoretically run up 50+ GB of mismatch overnight when not reporting the failed pieces. For any site where the totals are checked, that's usually going to set off alarm bells.

The other reason as mentioned, is that a seperate key allows admins to decide what's fair.

On your tracker, with many uT peers, you decide it's fair to allow all those peers a reduced download.

On a tracker with mostly ABC peers say, they may decide it's fair to include the failed hashes for ut/az users. The alternative if they feel strongly enough, is to ban those clients. I think we both know sites who would

Now that I know az also does this, I've been in touch with them and they are rectifying the situation.

If I'd have known about it sooner, I'd have been in touch with them sooner. I've been out of the loop on client updates for a while though doing other things.

If I've been a bit shitty over this, I appologize. Had a lot of stress the past few days.

[Switeck]

There's also duplicate data that often gets downloaded as the last few pieces come in. So total amount downloaded is often (if not almost always!) greater than the size of the torrent even in the total absence of hash fails. This problem is aggravated further on big torrent swarms when lots of people foolishly connect to 100+ seeds+peers at once. BitTorrent clients that are configured to only trickle out less than 1 KiloBYTE/sec upload speed per upload slot add to that as well.

...Just another thing that makes finding cheaters and stealth downloaders harder.