BitComet cheats, so µTorrent should IGNORE it!

[jboyx4]

I have to tell you, I really don't understand half of what is being said here. I don't understand protocalls & the such. So in my noncomputer understanding brain, I'm hearing 2 things in these debates: 1.)Are they confirmed cheats? & 2.)Is it considered cheating to make aggressive clients such as Bitcomet play by the same rules? My thoughts on the first 1 are "Who Cares". If protocols(rules) are in place that discourage them & that particular client isn't doing it, then who's being harmed. & 2.)To me this is obvious. Everyone should be equal. That would be like allowing my kids to have a piece of pie, and being told by one of them "your Cheating" just for not allowing him to have 2. Childish, don't you think?

[Switeck]

I have to tell you, I really don't understand half of what is being said here. I don't understand protocalls & the such. So in my noncomputer understanding brain, I'm hearing 2 things in these debates: 1.)Are they confirmed cheats? & 2.)Is it considered cheating to make aggressive clients such as Bitcomet play by the same rules? My thoughts on the first 1 are "Who Cares". If protocols(rules) are in place that discourage them & that particular client isn't doing it, then who's being harmed. & 2.)To me this is obvious. Everyone should be equal. That would be like allowing my kids to have a piece of pie, and being told by one of them "your Cheating" just for not allowing him to have 2. Childish, don't you think?

1)Some of the cheats have been confirmed, others seem to have no evidence to the contrary, and 1 or 2 of them is only true if BitComet is hacked.

2)The ways BitComet is cheating is often bad for everyone, even other BitComet users. If a torrent tracker goes down due to overloads caused by excessive requerying by BitComet clients...it doesn't help anyone. (Sure, DHT allows them to function without the tracker, but new peers are pretty much S.O.L. till it comes back online.)

If a super-seed can't keep a torrent going because BitComet is sucking up all its upload bandwidth then not sharing what it gets...it doesn't help anyone except the cheater.

If BitComet spews lots of duplicate data and even garbage data, the time and bandwidth it takes to get the complete torrent goes up rapidly...that helps no one.

BitComet's author has been virtually silent on these issues, even though some/many of them have been known for over 6 months.

[Klaus_1250]

An unconfirmed problem:

BitComet creates incompatible torrent files, or just plain unreadable (even by BitComet!) torrent files, sometimes. Incompatibility is known to exist at least because of BitComet's strange way of handling unicode characters in filenames, but I've heard there's other incompatibilites as well -- possibly due to a pure program bug in BitComet.

I recently encountered such a file. µTorrent complained it couldn't read it, but Shareaza had no problems with it. It was created by BitComet. But I'm not sure wheter or not it was a bug in µTorrent or BitComet.

As to the cheating. The only way to know for sure is if programs like µTorrent logged and analyzed behaviour of other clients. There does need to be proof before you can say "guilty". If there is such proof, ignoring BitComet is not the way. If there is a hole in the BT-protcol that allows BitComet's behaviour, it should be solved there. If BitComet is really cheating, make sure it (and other clients) cannot do it, without ignorring them wholly. Banning should really really be only a last resort and should be done by the community as a whole.

[nightshifted]

An unconfirmed problem:

BitComet creates incompatible torrent files, or just plain unreadable (even by BitComet!) torrent files, sometimes. Incompatibility is known to exist at least because of BitComet's strange way of handling unicode characters in filenames, but I've heard there's other incompatibilites as well -- possibly due to a pure program bug in BitComet.

In creating a torrent of a directory, BitComet (and it's been doing this long enough that BitLord does it too) will not skip empty subdirectories, nor subdirectories that lead only to empty subfolders. The resulting .torrent is unusable by Mainline or, I believe, by BitTornado either. All other torrent creation utilities that I've tried, including the one built into µTorrent, skip empty subdirectories and fallow directory paths (to pick a word out of the air for those that have only more folders under them and no actual files).

I ran into something where RnySmile or someone speaking for him bragged about that misbehavior.

[Rentahero]

Firstly, BitComet cheats:

...

3.BitComet disconnects and reconnects to download more than is fair via optimistic unchoke -- (which is meant to give new arrivals something to share. Sadly, Azereus is reported to do this too. Automatically droping working connections is hostile activity -- it creates lots of churn which costs extra bandwidth for trackers and peers alike.

There are certain cases when I need to disconnect/reconnect on μTorrent to get things working right again. Lately I seem to be getting a lot of bad .torrent files that have bad block hashes in them. μTorrent's default behavior is to ban the sender, which is Bad, because the sender actually has done nothing wrong. Therefore, eventually I get disconnected from everyone because everyone tries to send me a "bad" block. This is bad for me and the rest of the swarm, because I also stop uploading to those that are banned. It isn't our clients' fault, it's the torrent file's. No one is cheating. In any case, the only way to resolve this issue is to disconnect and reconnect and hope it doesn't try to grab that bad block for awhile. Eventually someone will realize that the torrent has gone bad (or I find another source for it), but it's a real pain. In any case, even though it is "bad behavior" by your definition, it is unavoidable in some cases, but it really is a value judgment, because if I don't stop it and reconnect, no one is going to be able to get anything from me, either.

[Firon]

most clients ban after receiving a few bad blocks. if you're receiving lots of bad blocks, then it probably means it's a poisoned torrent. it's not the torrent files fault, it's the fault of either a) the peers with bad data or the people trying to poison the torrent (common with movie torrents).

and reconnecting every once in a while is not the same as reconnecting every few SECONDS automatically, over and over.

[Rentahero]

Oh, I know. I'm not saying it's a problem with μTorrent - it's doing what it's supposed to. I was just saying that that was the problem I was having. I also agree that there is a difference between constant disconnect/reconnect and manually doing it, but I was just trying to point out that disconnect/reconnect isn't always malicious in all cases.

Actually, it was a D-Link router issue, which I never would have thought of (DMZ host setting causes data modification). I'm downloading Linux distros for people currently, so I doubt big companies would go out of their way to poison them, unless it's Bill's...

But in any case, it is theoretically possible to have a bad torrent file. A bad byte in the torrent file is just as likely (if not more so, since HTTP doesn't do too many checks) as one in the downloaded file(s) themselves, so if the block hash data itself was bad, then it could lead to issues. (Though torrent files have the full hash as well as the block hashes to correct for these issues, I don't know how many clients actually check both against each other.) That's all I was trying to say there. But in this case I was wrong, as it wasn't the torrent file's fault in my particular case.

[Firon]

A corrupted .torrent file wouldn't even load...

[Rentahero]

Even one that just had a corrupted bit somewhere in the hash check data? Is there some internal hash check for the torrent file itself?

Admittedly, I've never looked at the torrent file itself, only interpretations of the data it contains. I sort of figured that if a bit got whacked (not dropped, but the value of it changed), the client wouldn't know. But if there's a check of the torrent file going on, then I guess I am more relieved.

[Klaus_1250]

A bad byte in the torrent file is just as likely (if not more so, since HTTP doesn't do too many checks)

You'd need an undetected error in a TCP-packet for that. Not impossible, but very unlikely. HTTP can do checks, you can add a MD5-hash in the header for the content. But it never gained much adoptation, played with it a few times and IE became noticably slower.

[Firon]

Try it yourself, hex edit some random bytes in a torrent and see if it loads.

[Rentahero]

Yeah, I thought what was happening with mine seemed ridiculously unlikely myself. I know that HTTP performs some checks of the packet, but I also know that MD5 and SHA1 wouldn't really exist if it was flawless

Anyway, I'd just like to wrap up with what I was saying by saying that μTorrent is a great achievement. I've tried at least a dozen BT clients for various reasons, and μT is where it's at. Very well done.

[nightshifted]

I sort of figured that if a bit got whacked (not dropped, but the value of it changed), the client wouldn't know. But if there's a check of the torrent file going on, then I guess I am more relieved.

There certainly is. The piece hashes are in the info dictionary, so any change would alter the SHA1 hash of the info dictionary, which is what identifies the torrent.

[Switeck]

I also agree that there is a difference between constant disconnect/reconnect and manually doing it, but I was just trying to point out that disconnect/reconnect isn't always malicious in all cases.

If my idea is implimented, it'd be able to handle a few disconnect differences:

A client with nothing to share or has only what everyone else has shouldn't be ignored.

A client that's disconnected for more than 5 minues shouldn't be ignored -- that's likely someone rebooting or a short tracker outage.

A client that disconnects and reconnects in <15 seconds AND has data to share should be ignored.

BUT...the ignore condition goes away completely the moment that client uploads anything to your computer.

The ignore condition is also overridden temporarily by optimistic unchoke, which should encourage the ignored computer to upload back.

So even if it wrongfully ignored someone, it would be both temporary and it would even fix itself.

[Rentahero]

Switeck: I agree with what you're saying in theory, but you don't really have a case for a time period between 15 seconds and 5 minutes. I think frequency needs to be taken into account too, in reconnects/hr. or some such thing. It's also possible that someone just has a flaky connection, and you should not penalize someone for that. I think there probably needs to be an algorithm to detect someone who is simply being a leech as opposed to someone who is legitimately having issues. I like the temporary idea -- it'd be too harsh if it was addition to the ban list.

nightshifted: Is the SHA1 generated from the piece hashes upon loading though? I had the impression that both were just stored, and it was up to the client to use the SHA1 before finalizing the download. If the SHA1 is generated, then that should be pretty safe, and I guess it'd be pretty obvious to anyone that the torrent file has a problem.

[1c3d0g]

Yeah, I thought what was happening with mine seemed ridiculously unlikely myself. I know that HTTP performs some checks of the packet, but I also know that MD5 and SHA1 wouldn't really exist if it was flawless

...

And even those hashes can be (partially) cracked nowadays...so it's not looking too good.

[tumu]

The "crack" or rather duplication is possible if you create the data yourself. Method for doing it to existing data has not been found yet.

[1c3d0g]

...I stand corrected. Though it's only a matter of time before that one is cracked as well... :|

[Rentahero]

tumu: That's perhaps even worse. Think about companies like HBO that are known to assault torrents with bad data. If they can give bad data that piece checks OK, then you'd never know which part was bad, just that the entire file has problems somewhere.

Not exactly the best example (stop downloading pirate TV shows!), but malicious users could create data that screws everyone using such techniques. I can just imagine some virus writer wannabe scriptkiddie grinning as they read this sort of thing.

Edit: Anyway, there is another way to tighten the checks. If there are two sets of blocks that are checked (split in different places, maybe differently sized), the chances of fooling the client into thinking that the block is correct becomes exponentially harder. Either that, or you could just have multiple hash check algorithms running simultaneously (which might be even better).

[Firon]

SHA1 is still relatively difficult to do.

In academic cryptography, any attack that has less computational complexity than the expected time needed for brute force is considered a break [11]. This does not, however, necessarily mean that the attack can be practically exploited. It has been speculated that finding a collision for SHA-1 is within reach of massive distributed Internet search.

In terms of practical security, the major concern about this new attack is that it might pave the way to more efficient attacks. Whether this is the case has yet to be seen, but a migration to stronger hashes is believed to be prudent. A collision attack does not present the same kinds of risks that a preimage attack would. Many of the applications that use cryptographic hashes, such as password storage or document signing, are only minimally affected by a collision attack. In the case of document signing, for example, an attacker could not simply fake a signature from an existing document -- the attacker would have to fool the private key holder into signing a preselected document. Reversing password "encryption" (e.g. to obtain a password to try against a user's account elsewhere) is not made possible by the attacks. Constructing a password that works for a given account requires a preimage attack, and access to the hash of the original password (typically in the shadow file) which may or may not be trivial.

http://en.wikipedia.org/wiki/SHA1

[nightshifted]

nightshifted: Is the SHA1 generated from the piece hashes upon loading though? I had the impression that both were just stored ...

The info_hash is calculated from the entire info dictionary (which includes the piece hashes) upon loading. It is not stored inside the .torrent file.

[Rentahero]

Gotcha. Then it should be relatively safe -- the major risk is that a piece hash comes out as OK when it isn't. Can't do too much about that.

[Klaus_1250]

Most applications that use SHA1 are already shifting to SHA256. I guess BT will follow somewhere in the near future, though I think SHA1 will be quite safe for the next few years / decade.

But still, you can add multiple hashes to a torrent isn't it? If a torrent would be provided with a ed2k-hash and a (g2) tigertree (there is on torrentmaker that can do that), you'd need to create a collision multiple for multiple hashes. I think that is nearly impossible.

[nightshifted]

I believe I've seen .torrent files with ed2k hashes for the component files, and TorrentSpy includes an .md5 signature for each file. But they're for the files; for the pieces there are only the SHA1 hashes.

[Rentahero]

Yeah, if there was more than just SHA1 for the pieces then it would be pretty unbreakable by human things.