Jump to content

Utorrent Streaming


rodgerse

Recommended Posts

Technically speaking I think you could force your client to only DL pieces in order, and that would give you best streaming speed since you DL the next segment when it comes instead of devoting bandwidth to a different random piece.though that does mean the client will just wait for the next piece in line instead of DLing one available now, overall Dl speed will suck.

So if you make your client Dl in order, but if the next piece isn't available yet it doesn't waste time so it DLs the closest one while it waits, so later it will have that piece anyway, doesn't that mean DL speed and streaming speed would both be %100 0optimal?.

Any possible program or protocol, or general technique for this?, and how good does it sound.Any other real technicall issues that might make it bad?.

Link to comment
Share on other sites

it doesn't necessarily have to be 100% linear. Videos are usually 175 KB/s? Most people have more than 1.5mbit of download so you could probably set like a high priority on certain pieces, regular priority on others.

What im trying to say is after you download enough of the beginning to start watching something, all it would need to do to keep the stream going is download the next continuing section before you run out of data to watch.

The apparent avg download speed is 4.5 mbit in North America, higher in Western Europe. So there's more than enough room for random pieces to keep the swarm going.

I think this would be a good feature for utorrent especially with all the focus on streaming and youtube like media.

Link to comment
Share on other sites

I don't know if people understand what I said, but I mean wouldn't making utorrent DL the closest available pieces to the last one downloaded be in fact better than how it rnadomly does now?.

There's no trade off for speed or anything since It'll still download as fast, just as linearly as possibly.Are they're any other technicall issues though?.

for the record, mplayer is useless, vlc can play all kinds of incomplete, corrupt and downloading video files.

Link to comment
Share on other sites

IT"s not part of the bittorrent protocol to assume sequential retrieval of pieces.

And 'random' isn't the correct way to describe the piece distribution mechanism.

IT is closer to rarest-first-when-available. Or at least closest to it when not in super-seed mode.

The goal is to keep people able to seed stuff as soon as possible as joining the swarm.

Link to comment
Share on other sites

Technically you could force a client to DL pieces sequentially, in fact that would give you better ability to watch streaming since your only Dling in order.only Dl time itself would suck, since the client waste time idling while looking only for the exact next piece instead of available ones.

what I'm saying is that it could Dl random pieces while waiting for the next one in line, so speed and streaming would be otimal.

Link to comment
Share on other sites

>It's not part of the bittorrent protocol to assume sequential retrieval of pieces.

The bittorrent protocol doesn't say anything about which pieces to download first.

>And 'random' isn't the correct way to describe the piece distribution mechanism.

IT is closer to rarest-first-when-available. Or at least closest to it when not in super-seed mode.

No peer or seed knows what is rarest. What you are claiming has the same effect on the torrent as sequential piece requests. Assuming clients know what is rare and what isn't they will all be requesting the same pieces. This comparison relies also on ALL peers downloading sequential pieces.

>The goal is to keep people able to seed stuff as soon as possible as joining the swarm.

I think this is why clients requests pieces. Uploaders do not force pieces into other peers, it only sends what is requested.

Link to comment
Share on other sites

>It's not part of the bittorrent protocol to assume sequential retrieval of pieces.

The bittorrent protocol doesn't say anything about which pieces to download first.

Rarest first is recommended, IIRC. (I didn't recheck it, but I'm pretty sure about that).

No peer or seed knows what is rarest.

Peers know which piece is rarest among their peers. This is the point of the BITFIELD message in the handshake and HAVE messages during the download.

I'm thinking that the BitTorrent, Inc. people are doing something in this area related to BitTorrent DNA, but the stuff I've read to date is a bit stronger on the marketing verbage and lighter on the technical detail. I do know that it is server-assisted.

Rarest-first is about getting as many complete copies within the swarm as possible. Streaming is about getting the next needed parts of the content to the destination before it is needed. These are not necessarily mutually exclusive, but they are very different goals.

Keep this in mind: How many videos have you stopped and deleted after watching the first few minutes? If BitTorrent was to be heavily weighted to streaming, then the availability of the first fraction of the file would be extremely heavy when compared to the rest of the file.

BitComet has a function that provides a low-bitrate preview of the file you're downloading. It isn't weighted to the first fraction of the file, but enabling that feature does change the piece-selection algorithm somehow (I haven't asked).

Link to comment
Share on other sites

Yes but most people connect to 30 peers. Many torrents have much bigger swarms.

Assuming rarity of pieces do not change radically then most people are asking for the same pieces. Pure or as close to pure randomness is the best way of distributing anything and trying to force a uniform distribution is really hard if not impossible, it might work if everyone had the same speed and the same up/down speed.

Rarest first is not about getting as many copies out into the swarm. It is the hope that swarms have more access to everything but that is the goal of everyone in the swarm regardless of what type of system is used.

Does it achieve it, no. Proof is the bittyrant paper. I can hear all the people say this and that about game theory but it was empirically proven.

It makes no difference how you choose pieces as long as everyone is choosing the same way. The only way to achieve a uniform distribution is either through pure randomness or if you are using a system, through a lack of real information about the swarm. If everyone knows the same thing they will ask for the same thing.

The problem here is the pre-concieved idea of rarity being of the utmost importance when in reality it does nothing to improve the final objective of a torrent.

Link to comment
Share on other sites

Yes but most people connect to 30 peers. Many torrents have much bigger swarms.

Assuming rarity of pieces do not change radically then most people are asking for the same pieces. Pure or as close to pure randomness is the best way of distributing anything and trying to force a uniform distribution is really hard if not impossible, it might work if everyone had the same speed and the same up/down speed.

.

.

.

It makes no difference how you choose pieces as long as everyone is choosing the same way. The only way to achieve a uniform distribution is either through pure randomness or if you are using a system, through a lack of real information about the swarm. If everyone knows the same thing they will ask for the same thing.

The conclusion for that assumption is incorrect. Taking your example of 30 peers, and a torrent that has 700 pieces, a peer will have an average of 24 (pieces divided by peers) choices of rarest pieces. Said differently, in any particular corner of the swarm, there will be an average of 24 pieces that are equally rare. Clients should be choosing randomly from the pool of most-rare pieces.

Rarest first is not about getting as many copies out into the swarm. It is the hope that swarms have more access to everything

No argument, but I do not understand the distinction.

Does it achieve it, no. Proof is the bittyrant paper. I can hear all the people say this and that about game theory but it was empirically proven.

Which part was proven? I'm familiar with the paper, but it's been a long time since I read it. I'm a fan of the method (as long as the greedy client still gives the excess bandwidth optimistically). I'm not sure I'm in the majority of opinions in that matter, however.

My take-away from the BitTyrant paper was that generosity (altruism) in the swarm, beyond a certain point, was not helpful and could be exploited. That sounds terrible, but it is not. Those that read the BitTyrant paper have to put away their preconceptions about words like greed, exploitation, and altruism

Link to comment
Share on other sites

Well besides what you have stated about altruism, it shows that swarm health and speed can be improved on a client level. Protocol changes are not required to improve distribution.

The current bittorrent model is too dependent on everyone giving away free pieces, when incentives are necessary. There is little guarantee that every peer will do its part and share which is the exploitation the paper refers to.

Whether pieces are distributed through rare first is irrelevant. Maximizing download and upload throughput serves the swarm better. If completion or lack of data is a problem then yes rare first might help, but its benefit to the swarm is small.

If bittorrent was prioritized to maximize the health of the swarm piece distribution would not matter.

Link to comment
Share on other sites

http://torrentfreak.com/bittorrent-launches-ad-supported-streaming-071218/

Now be quiet. :P

Edit: To bandhammer those who don't know what that means... the same company which owns µTorrent owns the BitTorrent trademark and runs the bittorrent client "Mainline" which soon will be entirely based upon µTorrent come 6.0 stable is BitTorrent,Inc. ;)

Link to comment
Share on other sites

  • 3 years later...

Hi, just signed up to write here...

The point of "rarest piece first" is making sure that people are not depending purely on seeds. With enough peers a torrent with 0 seeds can still have the complete data - but this is obviously a rare occasion. But having 5 downloaders for a seeder is not rare, and it is not different from having 0 seed, because most of your download will have to be provided by peers.

For the sake of simplicity let us talk about a torrent with 10 pieces and 1 seeders + 3 peers just starting download

Seeder seeds piece 1- now peer 1 has piece 1 - piece 1 can be downloaded from 2 sources

Seeder seed piece 1 to peer 2, and peer 1 seeds to peer 3 -> Everyone has piece 1

Seeder seeds piece 2 to peer 1 (or whichever) - and everyone else waits for this entire time

This would not have happened if the seeder gave a different piece every time...

This will inevitably result in "download the piece in order and wait for the others to download the piece in order until there is upload potential for other pieces"

In principle this will not work any differently with 50.000 people or 5 people, the speed at which you can download parts that are late in the order will always be low because there will only be a few people who have and can upload those parts.

The solution with a lineer system is the seeder seeds piece 1, piece 2, piece 3 in that order TO the same peer - that way he could stream properly, and finish the video early. And that peer would give piece 1 to peer 2 while downloading piece 2 from seeder, upload potential would double, triple, and quadruple for each piece in order... But the problem here is that more than half the peers are assholes and if they complete earlier than everyone else, they will just leave, turn off their computer to save energy or something.....

So you either wait in line and watch others download and just hope that people are contributive, nice and sharing (gl with that) and with luck (or in a world where peers have white fluffy wings and a golden halo above their heads) have no distribution problems

Or you sit around waiting for people to the get the pieces you already have before the torrent can move on.

If there is a seeder for everyone, I don't think the order of the pieces really matter because rarity is not an issue - but in this case the DL speed is probably awesome anyway, so you can just wait till you download the damned video before watching instead of whining and proposing systems that would cripple the torrents, systems that rely on high numbers of seeders but make it difficult for people to reach %100 percent by forcing them to wait on pieces because the upload potential is HIGHEST for the pieces that most peers have (ergo where it is not needed) while the pieces that people dont have (and therefore need to be uploaded) can only be uploaded by seeders and peers with a high percentage, and peers with high percentages can ONLY download from seeders which slows their progress while everyone relies on their progress to make any progress

ok ok... I got a little repetitive.. I hope it is comprehensible

Sorry about any errors, ENG is not my native language

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...