Anoid Posted July 6, 2013 Report Posted July 6, 2013 what can be done to fix the torrent protocol to support larger torrents?is there any reason that a torrent cannot be made to support 100TB or even 18EB of files?I realize 99.99% of torrents do not need this much big data, but I can think of many.
MisterRemote Posted July 7, 2013 Report Posted July 7, 2013 I can't think of any single torrent that would need to be such a size. Torrents are for mass distributing, show me more than 10 torrent users worldwide which would even have that much of disk space to download it and a fast enough internet connection to share it within a year. Really there is no practical usage for this for the next 20 years at least.
Anoid Posted July 11, 2013 Author Report Posted July 11, 2013 I can't think of any single torrent that would need to be such a size. Torrents are for mass distributing, show me more than 10 torrent users worldwide which would even have that much of disk space to download it and a fast enough internet connection to share it within a year. Really there is no practical usage for this for the next 20 years at least.I can think of dozens of ideas, computer chess, astronomy and more.there is more to torrents than an MP3 or AVI file
ciaobaby Posted July 11, 2013 Report Posted July 11, 2013 The people and organisations that would need such transfer are smart enough to figure out ways other than BitTorrent.http://www.cs.unh.edu/~varki/publication/PDCS2011.pdf
vze2mp9g Posted July 11, 2013 Report Posted July 11, 2013 The people and organisations that would need such transfer are smart enough to figure out ways other than BitTorrent.http://www.cs.unh.edu/~varki/publication/PDCS2011.pdfI totally agree with this statement. You really haven't been around the torrent scene lately because I have never seen a torrent 1TB in size, even if you decided to do an ISO of a Blue Ray disk, that would only make the torrent approximately 50Gb!Let say you did make a torrent that was 1TB, who in their right mind will attempt to download it? I see people complain when the torrent is over 20Gb. It's best to keep the torrents as small as possible because most people use their computer for other things as well, and if you loose your power supply due to a storm, it would take a while to check the data and resume downloading.Now, if you were a business, then it's a totally different story, and I would think that they don't use BitTorrent, and they have a lot faster IntraInet connection.Like someone mentioned about, we (the common folk) won't need something like that until 20 years down the line.
ktetch Posted July 11, 2013 Report Posted July 11, 2013 I can't think of any single torrent that would need to be such a size. Torrents are for mass distributing' date=' show me more than 10 torrent users worldwide which would even have that much of disk space to download it and a fast enough internet connection to share it within a year. Really there is no practical usage for this for the next 20 years at least.[/quote']I can think of dozens of ideas, computer chess, astronomy and more.there is more to torrents than an MP3 or AVI fileOne of the other things I do, is work on a distributed computing project designing particle accelerators. The computer designs one, runs a simulation by running 80,000 particles through it, in 0.01nanosecond steps, and sees how it does. It then works from that design and ALL PAST DESIGNS to create the next one, working evolutionary, not brute force.10 years of this work, 80 million designs, simulating to the point where you can simulate a particle from here to pluto and back 5 times over, in 3mm steps, and all the data fits on a single terabyte hdd. Mainly because a single design and result is only 1kb.Massive data just doesn't work for these projects (same as radio astronomy either deals with the data there and then, on site, or breaks things into very small packets. And discards what's useless)Your 'need' has no need.
Anoid Posted March 15, 2014 Author Report Posted March 15, 2014 http://computer-chess.azurewebsites.net/download.php the torrent for 6 pieces was possible thanks to the team who noticed a defect with larger sets those involved historically used RAID but today a single disk is now able to hold the 6 pieces set Chess programs are free and there are others that are commercial. My site provides the free stuff, the EGTB work with chess engines to augment the endgame which is hard to program.
Anoid Posted March 28, 2014 Author Report Posted March 28, 2014 I posted a new torrent on the chess site, I decompressed the set, then used 7-zip to crush it down more. The compression used is designed to be opened while remaining compressed. EMD can achieve 4x compress of EGTB, 7-zip is better but it has to be unpacked to be used. There is overhead for the EMD compression, none for the NBB and NBW raw files. The NBB and NBW files are indexed into a table called distance to mate, the index is made from a hash of a given chess position. A lot of hard work has gone into EGTB, and Microsoft provided a server for the computation, a 4 socket Alpha server.
Anoid Posted April 28, 2014 Author Report Posted April 28, 2014 here is a backgrounder on computer chess where the end game work started all the way back to the 1970s and earlier http://computer-chess.azurewebsites.net/wp/egtb.php Computer chess is so extreme it literally attracts some of the brightest to work on the problems
Anoid Posted May 4, 2014 Author Report Posted May 4, 2014 I have updated my guide for EGTB distribution. Using a torrent is so brutal that it takes over 2 years with consumer internet connections.http://computer-chess.azurewebsites.net/wp/chess-torrents.php
Anoid Posted May 28, 2014 Author Report Posted May 28, 2014 I noticed a lot of gripes over the size of some recent game torrents. Games are getting a lot bigger now/
Recommended Posts
Archived
This topic is now archived and is closed to further replies.