The dexvert from sembiance

MegaZeux file detection

Detecting MegaZeux worlds with magic bytes

The MegaZeux world file header (for all versions) is documented here. From there, we can see that the following logic is sufficient:

The file extension is always MZX (world file),
If byte 25 is zero, the bytes 26-28 (magic) are either MZX, MZ2, MZA, or M\x02?, where ? is any byte;
If byte 25 is non-zero, the bytes 41-43 (magic) are either MZX, MZ2, MZA, or M\x02?, where ? is any byte;

Bytes 0-24 can then be used to parse the world title (codepage 437, null-terminated).

Detecting MegaZeux boards with magic bytes

Like the instructions above:

The file extension is always MZB (board file),
Byte 0 is always 0xFF,
Bytes 1-3 (magic) are either MB2 or M\x02?, where ? is any byte;

Bytes 0-24 can then be used to parse the world title.

Detecting MegaZeux save files with magic bytes

Like the instructions above:

The file extension is always SAV,
The bytes 0-4 (magic) are either MZSV2, MZXSA, or MZS\x02?, where ? is any byte.

Detecting MegaZeux 2.00+ worlds with checkres

The most comprehensive to detect a MegaZeux 2.00+ file is to use checkres, a tool built as part of the MegaZeux distribution (source code, directory src/utils):

./checkres -AV [file]

If the error code is 1, the file is not a valid MegaZeux 2.00+ file. If the error code is 0, the file is a valid MegaZeux 2.00+ file - additionally, a CSV will be output containing the file's dependencies and whether or not they have been located, for example:

Required by,Expected file,Status,Found in
CAVERNS.MZX,CV_BALON.MOD,FOUND,.
CAVERNS.MZX,CV_BOSS.MOD,FOUND,.
CAVERNS.MZX,CV_HEVEN.MOD,FOUND,.
CAVERNS.MZX,CV_MAGIC.MOD,FOUND,.
CAVERNS.MZX,CV_STORY.MOD,FOUND,.
CAVERNS.MZX,CV_TECH.MOD,FOUND,.
CAVERNS.MZX,CV_TITLE.MOD,FOUND,.

Notice that I mentioned "2.00+" (this is also important, as for very modern files - we're talking the last few years - the format changed significantly, being effectively a ZIP file with a special file layout). For MegaZeux 1.xx worlds (the only MegaZeux worlds currently on DiscMaster), the output will be as follows:

ERROR: Worlds from MegaZeux 1.xx are not supported by this utility.

As with ZZT/Super ZZT, I am open to working on a dedicated tool to handle these (though PNG screenshot exports would be a little trickier).

Minor issues with MacJapanese

Some text files are not recognised

I can hunt more if that's useful?
example

Some text files displayed with wrong encoding

similar change needed as the recent HTML one?
some text files will be MacRoman
example
example

Known file types marked as unknown

http://discmaster.textfiles.com/browse/10348/MACPOWER-1998-08.ISO.7z/MACPOWER-1998-08.ISO/mac/Shareware%20Paradise/P_%20POCKET/P_%20POCKET(68k).sit/P_%20POCKET(68k)

Pickle'sPocket is a Macromedia Director executable (I think they called them projectors)
ReadMe is a SimpleText document

Are these "unknown" because they are inside a .sit archive?

ps: this "Pickle's Pocket" is my latest amazing find, using contents search 🥇

Additional file extension request

HTML

.hts

(Not checked: HTM, HTML, HTX, XHTML, SHTML, PHTML)

Never heard of this myself but there we go!

Examples

404 not found

http://discmaster.textfiles.com/view/1814/Clip%20Art%20(Weird%20Science)(1994).iso/img2/space/planet.img

probably something to do with the brackets?

via http://discmaster.textfiles.com/search?q=%22MARTIN+MELIN%22&qfields=filename&qfields=ext&qfields=content

Search by language?

I can't remember how you flag Japanese media like mine, but it would be useful to be able to limit the search similarly.

.BAS file detected by extension, but contents are unknown

This RBBS-PC.BAS file (presumably from BASICA) is detected by extension, but is ultimately determined to be "unknown".

https://discmaster.textfiles.com/view/10618/The%20PC-Sig%20Library%20-%20Shareware%20for%20the%20IBM%20PC%20and%20Compatibles%20(PC-SIG)(Tenth%20Edition%20Disks%201-2804)(1991).iso/PC_SIGCD/01/5/DISK0152.ZIP/RBBS-PC.BAS

Easier way to view all images in directory?

Example

http://discmaster.textfiles.com/browse/15974/Precos%204.97%20Computer%202000.iso/pc/info/3periphe/epson/printlib/library/shared.dir

This came on in my RSS for Thoru Yamamoto. A real nice find, totally unknown!

I wanted to view all three images full size one after the other.

Turns out these images are small anyway so it was a dead end.

But, currently, viewing multiple images in sequence is difficult as the prev/next buttons are small and separate than the images.

I'm wondering if there's a better way. Some kind of "light box", slideshow, coverflow, or other interface.

RSS for search results

I thought it would be very useful if search results could be returned as RSS.

This would let me check "automatically" (I'd set frequency to, say, once-per-day) for any matches of sought-after items/keywords in new DiscMaster uploads.

just like archive.org

I subscribe to a few searches on Internet Archive to alert me to new matching uploads.

Example:

I'd expect similar from DiscMaster:

http://discmaster.textfiles.com/search?q=花札&fuzzy=fuzzy
would (currently) return 3 items in the RSS feed

Colour of local/external HTML links?

I sent a DiscMaster link to somebody and they commented "this is just the same snapshot as Wayback Machine has"

It wasn't, but the reformatted Wayback links made them think it was.

So I wondered if it will be possible to differentiate local DiscMaster links and Wayback links?

The obvious way would be different colour for local/external links. But it's not so accessible. And maybe it would need a description on the page of the differences.

Maybe colour and an icon? but that would break the layout of some pages.

I'll leave the difficult decision up to you 😅

Point vintage URLs to Wayback Machine?

Almost all old links have rotted away.

How about automatically reformatting each URL in HTML files to point to Wayback Machine, with the timestamp of the file containing the link?

Example page containing dead links:

http://discmaster.textfiles.com/view/11593/db.tidbits.com.tar/db.tidbits.com/issue/406.html

😎

Typo in src/format/executable/dll.js

dexvert/src/format/executable/dll.js

Line 10 in 4077c64

name = "Microsoft Windows Dynmic Link Library";

It should say "Dynamic", not "Dynmic"

Directories are not showing up in search results

Search

Actual

(no results)

Expected

Support for other classic Macintosh text encodings

Currently dexvert has support for MacRoman, which is assumed for all HFS format media.

It would be great to get support for MacJapanese, and other classic Macintosh text encodings.

Personally, I've used Tcl to convert classic Macintosh encodings to Unicode, as that support was written by Apple themselves.

The tricky thing about this issue: there are no identifiers to signify which encoding a filename is using. It was assumed to be the same as the host OS.

Much like dexvert assumes MacRoman, my own tool assumes MacJapanese (which is a superset of MacRoman so handles any included filenames that use MacRoman fairly well). It's a tricky issue for sure.

Maybe dexvert could use the encoding that matches the language set in the internet archive metadata?

Example ISO: https://archive.org/details/macpeople-1998-no-2

Please ask if any further information is required, I can most likely help.

Thank you.

AIX Backup/restore (non)viability

I hate to be a Debbie Downer on this one, but AIX backup/restore will be very tricky for anyone to implement, as the format itself is tied to the closed-source AIX version of JFS.

There is a copy of the program's code floating around of questionable origin as a reference, and this too

http://www.theweeks.org/xcssa/files/sys-admin_V8/html/v04/i04/a5.htm

Pick up HTML encodings from meta tags?

This page uses ksc-5601 South Korean encoding as described in the meta tag of the HTML page.

http://discmaster.textfiles.com/view/4411/DOKAN13.iso/Webs/www.gainax.co.jp/akiba-k.html

Maybe these meta tags can be parsed for encodings? Might be a quick win for correcting the encoding of various exotic HTML.

Fixing ZZT/Super ZZT file detection (contributing file format support)

The repository lists ZZT and Super ZZT files as UNSUPPORTED, yet Discmaster is clearly detecting them - with about 80% false positives, unfortunately - and I can't possibly figure out how it is currently doing it from the codebase as-is, nor do I seem to be able get it to work enough to try locally on a set of sample files.

As a person familiar with the inner workings of ZZT, I'd like to contribute a more resilient detector, as I am very much interested in preserving and documenting every ZZT/Super ZZT/MegaZeux game world out there.

Searching with max file size limit is slow

Compare

with

http://discmaster.textfiles.com/search?q=%22nabu+network%22&qfields=filename&qfields=ext&qfields=content&sizeMax=1MB

without

http://discmaster.textfiles.com/search?q=%22nabu+network%22&qfields=filename&qfields=ext&qfields=content

News page numbering

I just sent somebody this link: http://discmaster.textfiles.com/news12.html

But it occurred to me... will the contents of this page change over time? Seems to me it will.

So I wondered if they older news pages could be named in such a way that their content will not change? Maybe they could be split by month/week/day?

DiscMaster news dating

May I humbly suggest that the date of publishing for the DiscMaster news blog entries is set to the end of the day, or the first minute of the following day.

Currently, when they appear in my RSS they appear "in the past" some way down my timeline.

Even though they're published "now" they're dated to the beginning of the day so appear many hours old.

Default search form is timing out

http://discmaster.textfiles.com/search

Never seems to stop loading.

Existing search queries display OK.

Hex viewer?

Not sure how feasible this is.

Thought about it when trying to figure out how to view these PRN files.

http://discmaster.textfiles.com/browse/15974/Precos%204.97%20Computer%202000.iso/mac/Info/3Peripherals/EPSON/LIBRARY_MAC_P/LIBRARY

sembiance / dexvert Goto Github PK

dexvert's People

Contributors

Stargazers

Watchers

Forkers

dexvert's Issues