Archived

This topic is now archived and is closed to further replies.

Ultima

BEncode Editor

Recommended Posts

BEncode Editor v0.6.0 WIP

v0.6.0 (2008-??-??)

+ Feature: Count the number of replacements made during Replace All

+ Feature: Cut/Copy/Paste items

+ Feature: Filter searches based on key

+ Feature: Holding Ctrl while reloading reloads data from disk

+ Feature: Move item up/down

+ Feature: On import, use filename as suggestion for key name (if empty)

+ Feature: Regular expressions search/replace (PCRE engine)

+ Feature: Select all Logger tab items with Ctrl+A

+ Feature: Sort keys (for dictionaries only)

~ Change: Add accelerators for dialog buttons

~ Change: Center popup dialogs on display

~ Change: Disable dialog buttons on action

~ Change: Display Edit dialog on double-click only if text double-clicked

~ Change: Don't collapse/expand treeview item on double-click

~ Change: Faster BDecoding (~20% improvement over v0.5.1 in tests)

~ Change: Faster BEncoding (~85% improvement over v0.5.1 in tests)

~ Change: Faster Find/Replace operation (much improved on larger treeviews)

~ Change: Faster treeview filling (~50% improvement over v0.5.1 in tests)

~ Change: Focus main window on drag-and-drop

~ Change: Increase input limit on edit controls

~ Change: Select all text in Value field each time Find/Replace dialog shown

~ Change: Unicode support on treeview and listview

~ Change: Use accelerators instead of global hotkeys

~ Change: Visual feedback when busy performing find (busy mouse cursor)

* Fix: Inaccurate window resize limiting under various Windows themes

* Fix: Sub-item count not displayed for dictionaries/lists added to a list

Download:

(WIP 1) [2008-04-17] http://www.savefile.com/files/1510612

(WIP 2) [2008-04-19] http://www.mediafire.com/?zy4jpzftv2f

(WIP 3) [2008-04-22] http://www.savefile.com/files/1516391

(WIP 4) [2008-04-27] http://www.savefile.com/files/1525881

(WIP 5) [2008-04-29] http://www.savefile.com/files/1530830

(WIP 6) [2008-05-26] http://www.savefile.com/files/1573691

It's the "I want it all, and I want it now" (WIP) release :) This knocks off a bunch of stuff from my todo list, which has been nagging at me for a while... Don't be gentle with it, and do try to break it. I've this feeling it's filled with bugs left and right. And uh, it's only the Unicode release (for now), but who's keeping count, right?

Share this post


Link to post
Share on other sites

o.O schweet. First question, your % gains... is this including testing with my resume.dat?

Secondly, How did you decrease load time? Third, thanks for the replace all change :D, except it doesn't work(I see the logfile message, nm).

[15:46:58] [bDecode] Start: "O:\µTorrent\resume.dat.old"

[15:47:42] [bDecode] End (44.030 seconds)

[15:47:42] [TreeView Fill] Start

[15:49:00] [TreeView Fill] End (76.788 seconds)

Confirmed ~25% decrease in loadtime :D

FRAK ME! Copy/Paste is AWESOME... Are there limitatios one should be aware of. I know only one sub-key can be copied at a time, but is there perhaps some limitation which pops up a message "too large to copy to buffer" or somesuch?

As a workaround for your mentioned Del behaviour, you can always Shift-End/Right Arrow, backspace.

Is there a keybinding for Move Up/Down ?

At first I was confused when I went to edit dictionaries to look for this sort feature, then I noticed the key would auto-expand... Can the double click event either be triggered to disable one or the other or both options?

Menus INDEED :)

Since I have just experienced the .... corruption faced by many who will use the tool I have created a rudimentary guide how to restore the data from backup. This is VERY easy since as uT saves data, its sorted [A-Za-z0-9] so you know where the last good key is "8:webseedslee", add an 'e' and you're at the end of good data. The guide is @ http://forum.utorrent.com/viewtopic.php?pid=322452#p322452 and I welcome suggestions how to improve it. Time to watch a guy lose a brain anurism :)

Share this post


Link to post
Share on other sites

It was just testing in general across multiple bencoded files (yes, your resume.dat file included). I disabled a global AutoIt option that automatically expands environment variables in all strings used (slowing things down by around 10%-15%), and manually expand the strings where necessary now. As I mentioned previously, I already had around a 10% improvement (overall) from upgrading AutoIt to the betas.

And yeah, Logger tab, just because a messagebox is ugly, and there's no room in the Find/Replace dialog to put the count.

Some notes I've added to the readme:

* Holding Ctrl while reloading reloads data from disk

* Holding Shift while pasting pastes item below the currently focused item

* Only dictionaries can be sorted (and without recursive key sorting)

Something else I should note... The accelerator table bug I notified the AutoIt devs about is still there, so you'll notice some wonky behavior, like the Delete button not working in the Add/Edit/Find/Replace dialogs (so you can't remove characters to the right of the input cursor by pressing Delete). Meh.

Edit: There are no limitations that I know of for copy/paste. Well, you can't cut the root item, but who would need to, right?

Edit: And yeah, I know Backspace works.

Edit:

Is there a keybinding for Move Up/Down ?

At first I was confused when I went to edit dictionaries to look for this sort feature ...

Do look in the menus. As far as double-clicking and auto-expanding goes, if I could do anything about it, I would.

Share this post


Link to post
Share on other sites

I tried to copy an empty announce-list list, only to be greeted by an error, same on the root key. Also, if it already isn't able to when working, can you add copying from one file, and pasting in another? And it only happens from the edit menu, not Ctrl+C. And pasting using Ctrl+P when editing a key seems to be broken, same with copying. And is sort keys supposed to sort the keys alphabetically? It doesn't seem to do anything.

AutoIt Error

Line -1:

Error: Variable used without being declared.

Share this post


Link to post
Share on other sites

@GTHK: Copying/pasting and all that jazz is b0rked when in child dialogs because of the aforementioned AutoIt bug with accelerators. Don't try to use any of the accelerators while in the dialogs. It's partly my fault for being lazy with the checking, but meh, it's a testing build that uses a testing build of AutoIt with a known bug that may get fixed some time.

Copying and pasting in general, though, works fine for me. Yes, I can copy/paste the root key. Yes, I can copy/paste empty lists/dictionaries/whatever.

When you're sorting keys, you need to select the root of the dictionary you're trying to sort. As for that variable not declared error, where do you see it (that is, how does it occur)?

Finally, regarding copying/pasting between files, that'd require that I bencode the copied data and store it in the clipboard... and then bdecode again when I paste. I already considered doing this for the purpose of allowing copy/paste between files, but frankly, I find it an ugly and slow alternative.

Share this post


Link to post
Share on other sites

Not really, no. Just a few minor tweaks/optimizations here and there, but nothing much to write home about. I was planning on doing a bit of code reorganization to (possibly) prepare for commandline operation, but I'm still trying to wrap my head around the more mundane/minute details, so I didn't really get to it.

Share this post


Link to post
Share on other sites

http://www.savefile.com/files/1516391

Benchmark away! I really want to see how much of an improvement this build makes in the Real World™ (AKA not on my computer). Before/after numbers are always nice :) And do compare the treeviews (you know, Alt-Tab between multiple versions) just to make sure I didn't screw the parsing/loading up.

Oh, and uh, the accelerator issue should be fixed now.

Share this post


Link to post
Share on other sites

<poke poke poke> More poking at it :D

Will report back in the morning after the stresstest.

With my 3 most common testers :D

50,228K-4226x36

[16:16:19] [bDecode] Start: "O:\µTorrent\resume.dat"

[16:17:02] [bDecode] End (43.297 seconds)

[16:17:02] [TreeView Fill] Start

[16:18:04] [TreeView Fill] End (61.008 seconds)

39,312K-1807x50

[16:23:18] [bDecode] Start: "O:\µTorrent\Old Versions\resume.dat"

[16:23:46] [bDecode] End (27.882 seconds)

[16:23:48] [TreeView Fill] Start

[16:24:24] [TreeView Fill] End (35.701 seconds)

24,368K-852x49

[16:25:33] [bDecode] Start: "O:\µTorrent\Beta\TAS\resume.dat"

[16:25:45] [bDecode] End (11.356 seconds)

[16:25:46] [TreeView Fill] Start

[16:26:04] [TreeView Fill] End (18.041 seconds)

0.5.1

[16:28:21] [bDecode] Start: "O:\µTorrent\resume.dat"

[16:29:23] [bDecode] End (62.145 seconds)

[16:29:23] [TreeView Fill] Start

[16:31:16] [TreeView Fill] End (112.137 seconds)

[16:33:38] [bDecode] Start: "O:\µTorrent\Old Versions\resume.dat"

[16:34:15] [bDecode] End (36.713 seconds)

[16:34:17] [TreeView Fill] Start

[16:35:23] [TreeView Fill] End (65.955 seconds)

[16:35:52] [bDecode] Start: "O:\µTorrent\Beta\TAS\resume.dat"

[16:36:07] [bDecode] End (15.477 seconds)

[16:36:08] [TreeView Fill] Start

[16:36:37] [TreeView Fill] End (29.225 seconds)

Share this post


Link to post
Share on other sites

Those are excellent gains, if I do say so myself... ;D Thanks for confirming the improvements. There wasn't anything funky/different with the output, right? I'm surprised there haven't been too many problems reported thus far.

The treeview double-click expand/collapse thing should hopefully be fixed in a future version, after the AutoIt devs fix the bug.

Share this post


Link to post
Share on other sites

If by "output" you mean testing export as binary or bencoded data.. I'm afraid I haven't tested that. I just went through the awesome scripting of loading each of my (moderately sized) resume.dat files 100 times.

I did notice... there appeared to be a limit to "value" @ size 30000 bytes? I used multiple right click->paste in previous versions (as Ctrl-V doesn't work) and was able to pump it up to 30228... Is this expected? I can import for example binary data up to 155KiB (as that's the first thing I did instead of bencoded data when I was coming up with a way to restore old torrents from a resume.dat backup)..... If you could explain this arbitrary limit :D I'd appreciate the understanding.

Share this post


Link to post
Share on other sites

The limit is quite arbitrary indeed. So arbitrary, in fact, that it wasn't even me who set it xD

I've now (manually/intentionally) set the limit to "so large BEncode Editor would probably die before the clipboard finished getting dumped onto the edit controls," so uh, look forward to that? (I'm not really sure why anyone would want to copy/paste such a huge block of text, but oh well :P)

Edit: Hm, keys that contain large amounts of data would not be edited property if the limit is too low... I suppose that's one upshot to increasing the limit.

Regarding output testing, nah, I simply meant output to screen, as in treeview. I ask because I sorta changed how treeview items got added, which explains the 50% speed improvement. I tested, and all seems well, but I wanted confirmation that I didn't break anything in the process.

Share this post


Link to post
Share on other sites

I have a theory the previous limit was 2^16 yes? With Unicode characters taking up 2 bytes per character that reduces it to 32768... In trying to figure out what else needs to be added, my only wish is you think of some way to be able to include both KEY and VALUE in Find/Replace ;)

As far as treeview... Nothing appears out of place, perhaps if you say HOW you changed it it would be easier to test that. * expansion as well as double click and add/remove keys doesn't cause any irregularities I can see. And nothing appears to be different between .51 (even with my triple-scrolling vertical bar) resume.dat... so I think you're safe.

Edit: More reverified fun stuff. c/o XIPH and lots of those lossless-philes http://orange.blender.org/blog/original-lossless-source-available/#comments

0.6.0-B3

[12:15:27] [bDecode] Start: "O:\µTorrent\ED-360-png.torrent"

[12:15:43] [bDecode] End (15.978 seconds)

[12:15:43] [TreeView Fill] Start

[12:16:41] [TreeView Fill] End (57.449 seconds)

[12:30:43] [bEncode] Start: "O:\µTorrent\ED-360-png.torrent"

[12:31:12] [bEncode] End (28.743 seconds)

That's encoding the file after adding an 3067 char comment to the (842,859 bytes) file. It has even more files than I've ever seen but still proves it's keys*subkeys which slows loading not the sheer depth of a specific dictionary or list. Consists of 15581 files. Average size=220.86KB of 3440896 KiB total dataset-size.

0.6.0-B1

[12:37:20] [bDecode] Start: "O:\µTorrent\ED-360-png.torrent"

[12:37:35] [bDecode] End (14.704 seconds)

[12:37:35] [TreeView Fill] Start

[12:38:43] [TreeView Fill] End (67.646 seconds)

0.5.1

[12:41:04] [bDecode] Start: "O:\µTorrent\ED-360-png.torrent"

[12:41:25] [bDecode] End (20.518 seconds)

[12:41:25] [TreeView Fill] Start

[12:42:48] [TreeView Fill] End (82.652 seconds)

0.5.2 WIP 2/10

[12:45:11] [bDecode] Start: "O:\µTorrent\ED-360-png.torrent"

[12:45:37] [bDecode] End (26.007 seconds)

[12:45:37] [TreeView Fill] Start

[12:46:55] [TreeView Fill] End (78.002 seconds)

0.5.0 WIP 1/20

[12:49:32] [bDecode] Start: "O:\µTorrent\ED-360-png.torrent"

[12:49:55] [bDecode] End (23.238 seconds)

[12:49:55] [TreeView Fill] Start

[12:52:03] [TreeView Fill] End (115.462 seconds)

0.4.1

[12:55:11] [bDecode] Start: "O:\µTorrent\ED-360-png.torrent"

[12:55:35] [bDecode] End (24.106 seconds)

[12:55:35] [TreeView Fill] Start

[12:57:19] [TreeView Fill] End (103.828 seconds)

0.4

[12:59:56] [bDecode] Start: "O:\µTorrent\ED-360-png.torrent"

[13:00:30] [bDecode] End (34.654 seconds)

[13:00:30] [TreeView Fill] Start

[13:02:14] [TreeView Fill] End (103.977 seconds)

0.3 Aqua due to no internal timer in this version

[13:06:15] [bDecode] Start: "O:\µTorrent\ED-360-png.torrent"

[13:06:43] [bDecode] End <28 seconds>

[13:06:43] [TreeView Fill] Start

[13:10:48] [TreeView Fill] End <243 seconds>

0.2

bfev2ed1090sloadeddonewy3.th.pngbfev2ed790sloadedbf0.th.pngbfev2ed610sload3yb5.th.pngbfev2ed455sload3tn0.th.png

bfev2ed305sload3ln4.th.pngbfev2ed160sload3qn5.th.pngbfev2ed160sload2xo2.th.pngbfev2ed105sloadxp8.th.png

Pictures because there is no logging mechanism for it: Started 13:30, ended 33:20... But I counted it seemed only 1090 seconds to fill the treeview. I will say that the OLD method was much slower, adding and displaying as-decoded.

It's also notable the memory usage of this old version is ~40% more at least for this test file. Barring inclusion of this version however the BFE has since increased efficiency by 100% :D

Share this post


Link to post
Share on other sites

The limit was simply a (default) 30000 byte limit -- nothing more, nothing less. That you were able to pass the limit by copy-paste, I'm unable to explain, but oh well. Thanks for the detailed tests!

http://www.savefile.com/files/1525881

CHANGELOG:

+ Feature: Filter searches based on key

~ Change: Increase input limit on edit controls

~ Change: Visual feedback when busy performing find (busy mouse cursor)

NOTE:

* Finding "Value by Key" uses an exact (non-partial) search for key names

I also made some wide-reaching, underlying changes in various places, so testing everything for breakage would be much appreciated :) I've already tried myself, but after staring at the code/program for forever, I'm pretty much at my limit :o

Test the search filtering heavily too. I don't feel confident about it (though I guess I never do feel confident about any release?). The changes I made for it to work were relatively minor (much simpler than I expected), but debugging it was a pain because of my lack of 1337 haxx0ring skillz. Well, actually, a stupid oversight had me running around doing cartwheels for a few hours hours just trying to find a dumb bug (with an easy fix) in the filtering. But it seems to work fine now -- I hope.

Share this post


Link to post
Share on other sites

Grr what did you change? Popped up in new posts >< This must be a mod option yes? That you make some changes and it doesn't update the thread, but others do.... pesky piksy pesternomi

Edit: Obligatory test values

[17:35:41] [bDecode] Start: "O:\µTorrent\Beta\TAS\resume.dat"

[17:35:53] [bDecode] End (12.009 seconds)

[17:35:53] [TreeView Fill] Start

[17:36:13] [TreeView Fill] End (18.845 seconds)

[17:36:38] [bDecode] Start: "O:\µTorrent\Old Versions\resume.dat"

[17:37:12] [bDecode] End (33.857 seconds)

[17:37:12] [TreeView Fill] Start

[17:37:52] [TreeView Fill] End (39.634 seconds)

[17:38:12] [bDecode] Start: "O:\µTorrent\resume.dat"

[17:39:06] [bDecode] End (54.290 seconds)

[17:39:07] [TreeView Fill] Start

[17:40:07] [TreeView Fill] End (58.455 seconds)

I have to say though I may have bitten off more than I can chew, I forgot about the size of the file >< Trying to use your "Value by Key" Find/Replace on 4226 items is... gonna take an hour by my count. It started at a speedy 42-48 replaces per second, then as the treeview grew (I assume) it's taking longer. It just slowed to 21 every 15 seconds after 12 minutes in. As an aside, I don't know what to do about this, but you can't cancel while "Replace All" is going on... so should the button be disabled? Or would you rather allow Cancel to interrupt the R.A. action?

OK so it's done[18:54:05] [Find/Replace] Replacements made: 4226 Could we change the Windows "Alert" sound or whatever it is that plays when R.A. finishes ? Are there any other places where a sound is played?

Dude, this is nifty, after any action which expands the treeview, sometimes it's just faster to reload the treeview instead of trying to use any other keybindings to mass change the display, because when it's loaded everything goes back to root display, nothing expanded.

Edit: 186 seconds [20:36:16] [Find/Replace] Replacements made: 847.

Share this post


Link to post
Share on other sites

Could you have importing of data to the root entry? It would be handy for merging two resume.dat's instead of cut-pasting them out of the dictionary entry created by trying to import them. I suggest this because of this thread.

Also, would it be possible to add copy-cut-paste of multiple items?

Good job on the other stuff :).

Share this post


Link to post
Share on other sites

:P I already thought of the "merging" idea back when adding data was implemented... but then how does BFE import it(if there's no beginning 'd' and ending 'e' for dictionary... so I thought I would let sleeping dogs lie. :P It works well enough with Ctrl-X, Up, Ctrl-V, Down, Down

Share this post


Link to post
Share on other sites

As it stands, when you have [Root] selected, it defaults to adding as a child, the add as could have a merge option, and at least work for the same types of data since list<->dictionary conversion has been rejected, and for the root entry disable the title field. Just an idea :P.

Share this post


Link to post
Share on other sites

I haven't mentioned how useful the hourglass is >:D That's how I came up with my "find/replace" figures, if you didn't catch on.

I still think it's to do with my limited RAM/CPU in this computer.. but if someone could come up with a function or curve demonstrating the performance times they see. My poor 1.5GHZ single core with 768 MB of DDR 400 doesn't cut it today apparently ><

Share this post


Link to post
Share on other sites

Do you mean a graph of the BDecode and TreeView Fill information you posted? If not, well, it still provides some information at a glance:

bdecodere6.th.pngtreeviewfillfe5.th.png

Share this post


Link to post
Share on other sites

I was talking more of... time to find/replace over # of replacements.... and dictionary/list # vs size :P

:D I like bar charts, so thanks for that hehe. Now make your own numbers... and compare. FTR: my comp is AMD XP 2000+ 768 MB DDR 400 if you want to start collecting stats.

You'll notice that the ED values while easy to baseline due to their easy availability and free nature... Don't quite test the full variability of the display mechanism of BFE over time.... For instance it may take forever to decode a 500 KiB bencoded file with a dictionary with 20,000 keys, but display in an instant (or small fraction of the time it takes to decode). Comparatively decoding a 500KiB bencoded file with 100 keys & 200 sub-values may take exponentially less time to decode but moreso to display due to the sub-nodes BFE renders in the background (For reference my numbers previously 50,228K-4226x36 RED=RAM usage, BLUE=sub-root keys, GREEN=values per sub-root key)

Share this post


Link to post
Share on other sites

I can honestly say I don't know half the stuff you're talking about. If you clearly point out some data to graph though I'll gladly do it. I can do line and pie charts too :P. I thought bar looked better though :/. As for stat collecting, do you have a public test file or something I can use?

Share this post


Link to post
Share on other sites