Jump to content

Calculate (sha1) "pieces" when creating torrent file

macb6497

By macb6497
in Protocol Design Discussion

Recommended Posts

macb6497 Member

macb6497

Posted
Posted

I'm almost done with my torrent server completely. All I have to do now is find out how .torrent files populate the "pieces" key for the info key.

I currently have this info in my torrent file, encoded in utf-8 format. The hash you see at the end is a has of the file's TITLE, not even close to the bit length I specified before the colon. The bit length for pieces was calculated using (filesize / piece length) * 20, and rounded using round(): 

d8:announce64:http://www.valevarkasystems.com:40/torrents/tracker/announce.php8:encoding5:UTF-84:infod6:lengthi71711042e4:name25:Executive Meeting480p.flv12:piece lengthi128000e6:pieces11205:òOìcØq¬rkÑzÎõyjZee

How do I properly calculate a value for the "pieces" key at the end?

Everytime I try to open it in Transmission, it is read as corrupted.

Do I use fopen(), grab a piece of the file that is "piece length" and create a sha1 hash for each piece until I reach the end of the file?

Thanks,

Link to comment
Share on other sites
DreadWingKnight Advanced Member

DreadWingKnight

Posted
Posted

1> the pieces value isn't necessarily the last entry in the torrent, use a bencode library for your programming language instead.

2> it's the sha1 hash, in raw 20-byte format per piece.

3> "piece length" of 128,000 isn't a good number. The correct number should be a multiple of 16,384 (the request block size, which should be 131072 for 128k).

4> "Do I use fopen(), grab a piece of the file that is "piece length" and create a sha1 hash for each piece until I reach the end of the file?" Yes. And do NOT pad the final piece.

Reference an example torrent maker for how to.

Link to comment
Share on other sites
macb6497 Member

macb6497

Posted
  • Author
Posted
1> the pieces value isn't necessarily the last entry in the torrent, use a bencode library for your programming language instead.

2> it's the sha1 hash, in raw 20-byte format per piece.

3> "piece length" of 128,000 isn't a good number. The correct number should be a multiple of 16,384 (the request block size, which should be 131072 for 128k).

4> "Do I use fopen(), grab a piece of the file that is "piece length" and create a sha1 hash for each piece until I reach the end of the file?" Yes. And do NOT pad the final piece.

Reference an example torrent maker for how to.

Great. At least I'm getting the same statements from each forum. I actually tried #4 before you posted and the torrent file was no longer corrupted; I could open it with Transmission and uTorrent Server for linux, v3.0.

Here's the other forum I posted at to give other people help: http://stackoverflow.com/questions/9506667/calculate-sha1-pieces-when-creating-torrent-file

Now that I have the torrent working, I'll need a little more help to perfect it.

#4: "Do NOT pad the final piece";

To tell you what I did, they way I hashed the beginning 128k blocks is exactly the same way I hashed the last block that was more than likely truncated, i.e. not a full 128k. Is this the part you're talking about? It's not good practice to create a 20-byte hash of the last truncated part of the file?

#3: find references

I have downloaded many php scripts and rivettracker scripts for learning how they parse, encode and create .torrent files. I'm ok there.

Can you also put a direct link to the file being shared?

If this below question is off topic, I've created another post here: http://forum.utorrent.com/viewtopic.php?id=115028

Finally, I really need help with this. Normally I have to create a new torrent file using Transmission in order to "link" a .torrent file with another file, i.e. a .avi or .mkv. I can get uTorrent to automatically add files from a certain directory, which helps because I create .torrents automatically via a PHP script. But, how can uTorrent automatically check .torrents and link them to files already on my computer so that I can seed. These files are unique and I need to seed first, but uTorrent always wants to download first when the .mkv file is already downloaded and even in the same folder.

Thanks,

Link to comment
Share on other sites
DreadWingKnight Advanced Member

DreadWingKnight

Posted
Posted
To tell you what I did, they way I hashed the beginning 128k blocks is exactly the same way I hashed the last block that was more than likely truncated, i.e. not a full 128k. Is this the part you're talking about? It's not good practice to create a 20-byte hash of the last truncated part of the file?

What you did is correct. You are indeed supposed to hash the truncated piece.

Can you also put a direct link to the file being shared?

As in?

Link to comment
Share on other sites
macb6497 Member

macb6497

Posted
  • Author
Posted
Can you also put a direct link to the file being shared?

As in?

When I create .torrent files via the PHP script and add them to utorrent, utorrent wants to start downloading even though I have the completed .flv file in the same folder as the .torrent file. Can I provide a direct http link in the .torrent file for direct linking, or are hashes the only means of verifying a file?

Link to comment
Share on other sites
macb6497 Member

macb6497

Posted
  • Author
Posted

Ok, I'm using the url-list key.

I've added

8:url-list

$stringlen:http://www.url.com/tracker/seed.php?hash_info=$dichash&piece=3

It's still working so far!

Do you know how long my client will wait before actually trying the url-list script? I need to know if this is actually working without seeders.

Also, do I simply need to use commas between urls in the url-list to separate them?

$stringlen:http://www.url.com/tracker/seed.php?hash_info=$dichash&piece=3,http://www.url.com/tracker/seed.php?hash_info=$dichash&piece=2

Thanks,

Link to comment
Share on other sites
macb6497 Member

macb6497

Posted
  • Author
Posted
what bencode library are you using to create your torrents?

I'm not using any library...

I "utf8_encode" the pieces of the file before "sha1()" hashing them for the "pieces" key and I sha1() the info dictionary for the info_hash. That's the only encoding I do. Everything else is plain text.

Inside my .torrent file I have included the key "8:encoding5:UTF-8"

Link to comment
Share on other sites
macb6497 Member

macb6497

Posted
  • Author
Posted
you're setting yourself up for major torrent formatting problems by not using an existing bencode library (I recommend the one from DeHackEd's tracker package at http://dehacked.2y.net/BT/ )

It's a lot easier to deal with.

I have it.

I move url-list to the end of the file just before the list "e" for the first dictionary. I simply created one list with the direct file location and closed it off.

I now just have to make them match and I should be done...

Link to comment
Share on other sites
macb6497 Member

macb6497

Posted
  • Author
Posted
you're setting yourself up for major torrent formatting problems by not using an existing bencode library (I recommend the one from DeHackEd's tracker package at http://dehacked.2y.net/BT/ )

It's a lot easier to deal with.

I have it.

I move url-list to the end of the file just before the list "e" for the first dictionary. I simply created one list with the direct file location and closed it off.

I now just have to make them match and I should be done...

I have some great news!

Remember when I stated that I was utf8_encoding() the chunks before sha1() hashing them? That was actually causing a problem. I simply removed that, reload the PHP script that creates the .torrent file, downloaded the file, opened it up in utorrent and I downloaded!

My .torrent files work in uTorrent server for linux and kTorrent for linux when I comes to http seeding!

Thank's very much for url-list key and standard piece lengths.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

Go to topic listing
×
×
  • Create New...