Giter Site home page Giter Site logo

lclevy / adflib Goto Github PK

View Code? Open in Web Editor NEW
79.0 79.0 28.0 3.07 MB

A free, portable and open implementation of the Amiga filesystem

License: GNU General Public License v2.0

Shell 2.61% C 92.13% Batchfile 0.01% HTML 0.31% Makefile 1.73% M4 0.59% CMake 2.63%

adflib's People

Contributors

david-c14 avatar deplinenoise avatar emoon avatar gryf avatar kalamatee avatar kyz avatar lclevy avatar niteto avatar polluks avatar realnc avatar t-w avatar tonigi avatar tuomasjjrasanen avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

gryf uikit0 david-c14 engur emoon tuomasjjrasanen haukevb hometheateroperatingsystem cfuguet deplinenoise b0nefish rheit stanman29 polluks realnc gareth-smyth raboho bonki kalamatee christoph-fassbach trgswe robsmithdev henrikstengaard t-w mhage tedostrem

adflib's Issues

Creating floppy compatible with 1.3

Hi,

I wonder if it's possible to create a floppy that is Kickstart 1.3 compatible? I have the following code (with a bootblock from a floppy that is usable with 1.3) (error checks removed to reduce code size, but nothing fails in my test)

The following starts fine when I use 2.0, but won't work when I try with 1.3

#include <adflib.h>
#include <stdint.h>

extern uint8_t s_bootblock[1024];

void test_create_floppy() {
     adfEnvInitDefault();

    struct AdfDevice* device = adfCreateDumpDevice("dummy.adf", 80, 2, 11);
    adfCreateFlop(device, "empty", FSMASK_INTL);
    struct AdfVolume* volume = adfMount(device, 0, false);
    adfInstallBootBlock(volume, s_bootblock);
    adfUnMount(volume);
    adfUnMountDev(device);

    adfEnvCleanUp();
}

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

uint8_t s_bootblock[1024] = {
    0x44, 0x4F, 0x53, 0x00, 0xE3, 0x3D, 0x0E, 0x73, 0x00, 0x00, 0x03, 0x70, 0x43, 0xFA, 0x00, 0x3E,
    0x70, 0x25, 0x4E, 0xAE, 0xFD, 0xD8, 0x4A, 0x80, 0x67, 0x0C, 0x22, 0x40, 0x08, 0xE9, 0x00, 0x06,
    0x00, 0x22, 0x4E, 0xAE, 0xFE, 0x62, 0x43, 0xFA, 0x00, 0x18, 0x4E, 0xAE, 0xFF, 0xA0, 0x4A, 0x80,
    0x67, 0x0A, 0x20, 0x40, 0x20, 0x68, 0x00, 0x16, 0x70, 0x00, 0x4E, 0x75, 0x70, 0xFF, 0x4E, 0x75,
    0x64, 0x6F, 0x73, 0x2E, 0x6C, 0x69, 0x62, 0x72, 0x61, 0x72, 0x79, 0x00, 0x65, 0x78, 0x70, 0x61,
    0x6E, 0x73, 0x69, 0x6F, 0x6E, 0x2E, 0x6C, 0x69, 0x62, 0x72, 0x61, 0x72, 0x79, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
};

unadf improperly parses options with an argument

2 issues have been found in the unadf command line option parser:

  1. unadf segfaults on invalid parameters
$ unadf -v
unADF v1.2 : a unzip like for .ADF files, powered by ADFlib (v0.8.0 - 2023-06-26)

Segmentation fault

$ unadf -d
unADF v1.2 : a unzip like for .ADF files, powered by ADFlib (v0.8.0 - 2023-06-26)

Segmentation fault

(gdb) r
Starting program: [...]/ADFlib/build/debug/examples/unadf -v
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
unADF v1.2 : a unzip like for .ADF files, powered by ADFlib (v0.8.0 - 2023-06-26)


Program received signal SIGSEGV, Segmentation fault.
__strlen_sse2 () at ../sysdeps/x86_64/multiarch/strlen-sse2.S:142
142     ../sysdeps/x86_64/multiarch/strlen-sse2.S: No such file or directory.
(gdb) bt
#0  __strlen_sse2 () at ../sysdeps/x86_64/multiarch/strlen-sse2.S:142
#1  0x00005555555a1f8d in __interceptor_strlen.part.0 ()
#2  0x000055555565895f in parse_args (argc=2, argv=0x7fffffffde68)
    at [...]/ADFlib/examples/unadf.c:199
#3  0x00005555556580a4 in main (argc=2, argv=0x7fffffffde68)
    at [...]/ADFlib/examples/unadf.c:94
  1. The argument given after the two options requiring an argument is (unexpectedly) used for both:
$ ../unadf -dv 0 ../tests/arccsh.adf
unADF v1.2 : a unzip like for .ADF files, powered by ADFlib (v0.8.0 - 2023-06-26)

Device : Floppy DD. Cylinders = 80, Heads = 2, Sectors = 11. Volumes = 1
Volume : Floppy 880 KBytes, "cshell" between sectors [0-1759]. OFS. Filled at 75.2%.
x - 0/c/
x - 0/c/Format
x - 0/c/Mount
x - 0/c/Zip
x - 0/c/Type
x - 0/c/DMS
x - 0/c/cmd.txt
x - 0/c/UNLZX
x - 0/c/Relabel
x - 0/c/Info
x - 0/c/CPU
x - 0/c/Assign
x - 0/c/Deksid
x - 0/c/List
x - 0/c/transdisk
x - 0/c/LhA
x - 0/c/LoadWB
x - 0/c/LZX
x - 0/l/
x - 0/l/LZX.Keyfile
x - 0/devs/
x - 0/devs/DOSDrivers/
x - 0/devs/DOSDrivers/SD0.info
x - 0/devs/DOSDrivers/SD0
x - 0/devs/statram.device
x - 0/devs/SyStEm-CoNfIgUrAtIoN
x - 0/s/
x - 0/s/.cshrc
x - 0/s/aliases
x - 0/s/.login
x - 0/s/startup-sequence
x - 0/libs/
x - 0/libs/arp.library
x - 0/libs/asl.library
x - 0/libs/diskfont.library
x - 0/CSH
x - 0/LoadWB
x - 0/system-configuration

$ ls
0

$ ls 0
c  CSH  devs  l  libs  LoadWB  s  system-configuration
  • Both to fix.

API

There is a lot of discussions already about it, I'd expect more, so maybe we can make a place for it (to list, brainstorm and set directions).

Some ideas to (eventually) consider (please update as needed):

  1. file / directory APIs
    • a standard style to implement?
    • changes vs. wrappers
    • issues: #38, #40
  2. consistent naming (ie. submodule prefixes) across the library
  3. isolating client API from internal one (separated headers in include/?)
    • eg. for file API, expose only a wrapper over the low level functions(?)
  4. devices
    • allow more types of devices (not only one native and "dump"), also custom, user-implemented
    • add a memory/ramdisk device - useful ie. for testing (see the comment below
    • unify the device interface for all types
      • the type of the device is currently is set in adfOpenDev() and depend on the name (ie filename) of the device
        • instead: differentiate opening/creating depending on type (need rather different functions as different arguments are needed, eg. filename for dump, size for ramdisk device)
      • make the same device structure with interface also for dump devices
  5. review error codes (see the comment below) and their use
    • reduce amount of (or remove completely) string output to stdout/err (or leave if a "logger" defined/enabled)
    • replace output warnings with proper return codes
      • bitfield vs enumeration
        • bitfield would allow to aggregate more than one in single return code
        • that was the orig. idea (there is even a test hasRC() - but it is not used anywhere...)
    • user readable (string) error info: logger vs errno/last error info
      • errno/last error info
        • only static error string or dynamic (that would include more info like filename, block
          number or other data relevant for the error)
        • drawbacks: single, global (what can be problematic, even though the lib is not supporting multi-threading)
      • use logger for more detailed error info (if logger enabled/defined)?

Each of these might be a separated issue (or even issues), that could be linked here (so that it is easier to navigate).

Issue with adding files to adf (floppy) image

Hi,

I'm trying to use this lib to add a (executable) file to a floppy. The floppy is a standard DD with a standard Bootblock with S directory, Startup-Sequence file which has "test_exe" in it.

I use this code (roughly) to add the file:

int main(int argc, const char* argv[])
{
    struct Device* flop = 0;
    struct Volume* vol = 0;
    struct File* file = 0;
    size_t fileSize = 0;
    void* data = 0;

    const char* output = argv[1];
    const char* input = argv[2];

    if (!(data = loadToMemory(input, &fileSize)))
    {
        printf("Unable to load file \"%s\" to memory\n", input);
        return -1;
    }

    if (!(flop = adfMountDev((char*)output, 0)))
    {
        printf("%s : unable create newdev: %s\n", argv[0], output);
        return 1;
    }

    if (!(vol = adfMount(flop, 0, 0)))
    {
        printf("%s : unable to mount floppy\n", argv[0]);
        return 1;
    }

    if (!(file = adfOpenFile(vol, "test_exe", "w")))
    {
        printf("%s : unable to create test_exe\n", argv[0]);
        return 1;
    }

    printf("fileSize %d\n", (int)fileSize);

    if (adfWriteFile(file, fileSize, (uint8_t*)data) != fileSize)
    {
        printf("%s : unable to write test_exe\n", argv[0]);
        return 1;
    }

    adfFlushFile(file);
    adfUnMount(vol);
    adfUnMountDev(flop);

    return 0;
}

But when I try to boot it it simple will not boot the image. If run the executable manually it works, also if do install df0: it will work as expected. (Notice that I have already installed the floppy before adding the executable to it)

If I take a copy of the floppy and run install df0 on it again it works as expected. If I do a binary diff of the two images there are some places where they differ which is likely the cause of why the Amiga can't boot the image.

I compile this under MacOSX 64-bit (I have also made sure that LITT_ENDIAN is set to 1)

Any ideas of the cause of this?

Cheers!

Memory leaks in adf_dev_hd: not freeing vol after another allocation fails 

RETCODE adfMountHd ( struct AdfDevice * const dev )
{
[...]
    while( next!=-1 ) {
[...]
        vol = (struct AdfVolume *) malloc (sizeof(struct AdfVolume));
[...]
        vol->volName = (char*)malloc(len+1);
        if (!vol->volName) { 
            adfFreeTmpVolList(listRoot);
            (*adfEnv.eFct)("adfMount : malloc");
            return RC_ERROR;
        }
[...]
        /* stores temporaly the volumes in a linked list */
        if (listRoot==NULL)
            vList = listRoot = newCell(NULL, (void*)vol);
        else
            vList = newCell(vList, (void*)vol);
[...]

If allocation for vol->volName fails, vol (allocated earlier, above) will never be deallocated.

(A rare case, only when a mem. allocation fails - so not caught by sanitizers).

A couple of issues GCC10 brings up

Hi! while compiling this with GCC 10 I see the following 2 messages of note (the first seems a bit more important since it copies to a NULL pointer)...

/ADFlib/src/adf_link.c:48:5: warning: argument 1 null where non-null expected [-Wnonnull]
48 | memcpy(tmpPath,entryBlk.name,len);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/ADFlib/src/adf_disk.c:167:27: warning: self-comparison always evaluates to false [-Wtautological-compare]
167 | if (dev==NULL || nPart<nPart || nPart >= dev->nVol) {
|

(in)validity of block allocation bitmap

Reviewing some pieces of bitmap allocation code (for #63) revealed a couple of potential issues:

  1. (in)validity flag (of the bitmap) is not checked while mounting a volume

    • it rather should be checked and if the bitmap is not valid - it should be rebuilt/reconstructed from the data on the volume (so it should basically work like Amiga's DiskValidator); otherwise, if an invalid bitmap is just read (as it is now), eg. an already used block (but not marked so in bitmap) can be (again) allocated for another purpose; all in all it can lead to hard to detect data corruption

  2. the use of the (in)validity flag is limited to only adfUpdateBitmap (one function), while it seems is should be used to mark "dirty" bitmap while writing any block (data, directory, link) until the bitmap is updated accordingly, as the block can be (de)allocated by those operations (meaning the bitmap will be invalid if the operation gets interrupted for some reason)

(All this have to be checked further, this is just a brief note for now).

Improve adf_salv (salvage)

The adf_salv module, with the very useful undel functions, needs improvements in terms of:

  • memory management (ie. memory leaks)
  • error checking (eg. undeleting things not checking if the blocks are read correctly)

(Since in some cases possibly the error check should be avoided (even if a checksum is not correct, data could be recovered), the issue is created to take notes/comments on this).

Branching scheme

New rules regarding branching scheme:

The project will adapt the following model:
https://nvie.com/posts/a-successful-git-branching-model/

For now, this concerns branches: master and devel, the rest of existing branches are "feature" branches.

Briefly the rules are:

  • master gets updated only from devel, after thorough tests and acceptance (in fact, since 0.8.0, it will contain only the code from the latest release).
  • devel gets updated from feature branches, after tests and acceptance

Other things of the scheme may be used/adapted as needed (hotfixes for released code and such,
very likely release branch(es) will also appear).

Questions?

Broken feature in unADF

I believe I have detected a "new" bug in unADF. It is part of adflib-0.8.0-ubuntu.zip from June 26, 2023. I don't know when it was introduced but it was not present in version 0.7.11a-5 available from the Ubuntu package server.

The -d option for unADF no longer works. It's not just me, the patool archive tool can no longer successfully use unADF.

Issuing the command:
unadf '/home/dave/Desktop/Space.adf' -d '/home/dave/Desktop/DumpTarget'

Produces this result:
unADF v1.2 : a unzip like for .ADF files, powered by ADFlib (v0.8.0 - 2023-06-26)

Device : Floppy DD. Cylinders = 80, Heads = 2, Sectors = 11. Volumes = 1
Warning <adfReadBootBlock : incorrect checksum 0x444f5300, calculated 0xf4fbf2fb>
Volume : Floppy 880 KBytes, "Space" between sectors [0-1759]. OFS. Filled at 1.4%.
Warning <adfFileOpen : file "-d" not found.>
/home/dave/Desktop/Space.adf: can't find file -d in volume
/home/dave/Desktop/Space.adf: can't find directory /home in volume

This is not critical issue. Using unADF without the -d option will successfully extract the disk image into the current directory.

Any simple way to add in-memory files bytes/dirs and their attributes to a hdf?

Some WHDload sets are distributed as lha's (cause they're intended to be extracted on the amiga), but foreign emulators naturally want to do the same natively. I opened this issue showing the 'problem':

libretro/libretro-uae#60

But it would be cool if this library had a simple example of taking a lha, extracting the bytes and paths and attributes and recreating a reproducible hdf image everytime as a proof of concept.

From my understanding if the files don't touch the actual native filesystem before being written to the hdf they won't lose the amiga attributes or get forbidden characters the amiga allows but native doesn't interfering.

Support links for reading (eventually also writing?) files (and directories)

I understand from your explanation that the implementation of the links in Amiga's OS is better to avoid... However, I was thinking about adding the support at least in some part (like reading linked files).

I investigated this a bit, it seems that reading hardlinked files should not be difficult. I think, It is a matter of (optional) opening the block of with file entry in place of the hard link in the function adfOpenFile(), then the API would work as with regular file (and, if I understand well, in Amiga filesystems, a hardlink is just a pointer to real file/directory, so there is no metadata to update in it - but please correct me if I am wrong).

Would this be OK. to add something like this? (with some test I guess) If so, i prepare a PR.

Btw. For now, I am not sure about directories and the softlinks (maybe I resolve this on my side...) - but just let know if you'd eventually be willing to merge such changes as well (that is, if I figure out anything useful...).

Crash on adf_bitm.c:417 using adf_show_metadata

adf_show_metadata crashed on the first ADF I tried. Works on other ADFs, but consistently fails with this one.

#0  __memcpy_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:488
#1  0x00007f854da5212c in adfReadBitmapBlock (vol=0x55b1644fe910, nSect=948, bitm=0x0) at adf_bitm.c:417
#2  0x00007f854da51544 in adfReadBitmap (vol=0x55b1644fe910, nBlock=1758, root=0x7fff5eb1e710) at adf_bitm.c:134
#3  0x00007f854da60d9a in adfMount (dev=0x55b1644fd6e0, nPart=0, readOnly=1) at adf_vol.c:215
#4  0x000055b163b1f22c in main (argc=2, argv=0x7fff5eb1ee88) at adf_show_metadata.c:57

(different optimization level, shows memcpy call)

#0  0x00007fa987a344df in memcpy (__len=512, __src=0x7fffba1f6218, __dest=0x0) at /usr/include/bits/string_fortified.h:29
#1  adfReadBitmapBlock (vol=vol@entry=0x5612fff70910, nSect=nSect@entry=948, bitm=0x0) at adf_bitm.c:417
#2  0x00007fa987a34b5a in adfReadBitmap (vol=vol@entry=0x5612fff70910, nBlock=<optimized out>, root=root@entry=0x7fffba1f6698) at adf_bitm.c:134
#3  0x00007fa987a3c882 in adfMount (dev=<optimized out>, nPart=<optimized out>, readOnly=<optimized out>) at adf_vol.c:215
#4  0x00005612fef5d0b9 in ?? ()
#5  0x00007fa987627cd0 in __libc_start_call_main (main=main@entry=0x5612fef5d020, argc=argc@entry=2, argv=argv@entry=0x7fffba1f6de8) at ../sysdeps/nptl/libc_start_call_main.h:58
#6  0x00007fa987627d8a in __libc_start_main_impl (main=0x5612fef5d020, argc=2, argv=0x7fffba1f6de8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffba1f6dd8) at ../csu/libc-start.c:360
#7  0x00005612fef5d175 in ?? ()

This is the memcpy() at https://github.com/lclevy/ADFlib/blob/master/src/adf_bitm.c#L417

adfReadBitmapBlock() is called with NULL bitm by adfReadBitmap() at https://github.com/lclevy/ADFlib/blob/master/src/adf_bitm.c#L134

The ADF is an image of a MED 3.0 (later OctaMED) OFS floppy. It will be made available on demand.

Things to finish for 0.8.0

I guess we need to clarify what we do for the 0.8.0, prioritize a bit and close what really matters now, while leave other things for later.

I'd see the following priorities:

  • finish priority things with the build system (ie. autotools)
    • versioning, ie. soname
      • this should be done in #33
    • add things for the release to the GitHub Actions (ie. artifacts), #36
      • (this actually is something in addition, not really required - but nice to have)
  • clean-up API (only problematic things, ie. discussed #38)
    • remove confusing "a" from adfFileOpen() (edit: done in 75004da)
  • review/update API docs
  • review/update other files (readme.md), in particular statuses
  • update unadf (#39)
  • fix encountered bugs (esp. important ones)

No other major or breaking changes that will take time to implement/fix (ie. no bumping requirements regarding build system software versions).

Let me know what you think.

CI / GitHub's "Actions"

There is some initial version for "continuous" integration (or, at least, builds and some tests) that was added with #34. For now, it is set up only to start builds when there is an action (merge or push) on the citest branch. It is done so to prevent unnecessary overuse (with each small commit), as some of the tests might be a bit I/O expensive (maybe they can be a bit optimized later, eg. to check less cases, but for now I would leave it that way).

I have created citest branch in the lclevy/ADFlib repo - but it seems, the repo does not have the "actions" enabled (I was useing this with my fork so far). So, @lclevy, you may want to eventually enable it, if you want the builds executed in this repo.

For the configuration itself, there can be more things/details to discuss about it - so an issue for this might be useful.

Reading files with incorrect checksums on (data) blocks

Currently, the file read operation (adfFileRead) stops reading the file in case of an incorrect block checksum. And there is no indication for the client code about the error - reading just stops.

Such situation was already encountered on several ADF images, so it seems there can be more cases like this, and it is troublesome that the files cannot be read "just" because of the incorrect checksum (having in fact correct contents).

So - there is a need to have a mechanism that:

  • will allow to check the status of the last I/O operation (something like errno)
  • will allow to decide whether reading should be completed (or not) despite of the checksum errors

Both things can be done either on the level of volume (mount) or file (open), eventually set with some additional function.

SEGFAULT on file seeking more than 35135 bytes

As I have mentioned in the pull request, I ran on a problem with adfFileSeek(), which when used with an offset larger than 35135 bytes (from the beginning of the file), just crashes the program with a segfault.

A simple test (like the ones you already have in regtests/Test/) that triggers the issue:

#include <stdio.h>
#include"adflib.h"


int main ( int argc, char * argv[] )
{ 
    adfEnvInitDefault();

//	adfSetEnvFct(0,0,MyVer,0);

    /* mount an existing device : OFS */
    struct Device * const dev = adfMountDev ( argv[1], TRUE );
    struct Volume * const vol = adfMount ( dev, 0, TRUE );
    adfVolumeInfo ( vol );

    /* test file seek OK.*/
    //const unsigned int highest_not_failing_pos = 36864;
    const unsigned int highest_not_failing_pos = 35135;
    char large_file[] = "moon.gif";
    struct File * file = adfOpenFile ( vol, large_file, "r" );
    if ( ! file )  return 1;
    printf ( "Seeking to a non-failing position (%d)\n",
             highest_not_failing_pos );
    adfFileSeek ( file, highest_not_failing_pos );
    adfCloseFile ( file );

    // test file seek that (currently) fail
    file = adfOpenFile ( vol, large_file, "r" );
    if ( ! file )  return 1;
    printf ( "Seeking to the first failing position (%d)\n",
             highest_not_failing_pos + 1 );
    adfFileSeek ( file, highest_not_failing_pos + 1 ); // SEGFAULT(!)
    adfCloseFile ( file );

    // clean-up
    adfUnMount ( vol );
    adfUnMountDev ( dev );

    adfEnvCleanUp();

    return 0;
}

Just add something like:

echo "Executing file_seek_test..."
cp $OFSDUMP testofs_adf
file_seek_test testofs_adf
rm testofs_adf
echo "-----"

to regtests/Test/floppy.sh.

Got "RDSK id not found" when unzip on Ubuntu 20.04

Hi, I got an error when trying to unzip a .adf file on Ubuntu 20.04, this is the error

> unadf file.adf
unADF v1.0 : a unzip like for .ADF files, powered by ADFlib (v0.7.11a - January 20th, 2007)

Error <ReadRDSKblock : RDSK id not found>
Can't mount the dump device 'file.adf'.

From what I found on internet

the RDSK is a raw device path and it contains all drives that have not yet been formatted and are thus referred to as RAW

so I check /dev on my computer and see no /dev/rdsk there. I guess I have no RDSK on my computer so it can not mount the content. Can I do any work around from it or am I missing anything?

Thank you.

Opening devices in the current dev branch

The following code in. add_dev.c:

struct AdfDevice * adfDevOpen ( const char * const  name,
                                const AdfAccessMode mode )
{
    const struct AdfDeviceDriver * const driver = adfGetDeviceDriverByDevName ( name );
    if ( driver == NULL || driver->openDev == NULL )
        return NULL;
    return driver->openDev ( name, mode );
}

seems to use name as both name for the device name and the adf name.

Am I reading this correctly?

Over-filling a floppy doesn't report an error

Steps to reproduce:

    struct Device* device = adfCreateDumpDevice("test.adf", 80, 2, 11);
    adfCreateFlop(device, "Floppy2ADF", 0);
    struct Volume* vol = adfMount(device, 0, FALSE);
    struct File* output = adfOpenFile(vol, (char*)base_name, "w");

    char buf[1024] = {};
    for (int i = 0; i < 1024; ++i) {
        if (1024 != adfWriteFile(output, 1024, buf)) {
            printf("OK, we didn't write all the bytes\n);
        }
    }
    adfFlushFile(output);

This fails to print anything - it looks like block 879 is allocated over and over again when the floppy runs out of space, and the file written is garbage.

adfEndOfFile() never triggers and adfFileRead only gives "blank" data

Trying to write my own command line tool using this library and I'm having some weird issues.
Firstly adfEndOfFile() never triggers so the program gets stuck in the while loop and continues to make the output file larger.
Note: push() should add a file to the currently opened adf and pull() should extract a file from the adf.
Second issue is when trying to extract a file from the adf (manually setting the byte limit to get around the first issue) adfFileRead() seems to only output zeroed bytes.
There's a good chance I'm just doing something wrong but I can't figure out anything more from the seemingly outdated API.

void push()
{
    if (!adf_open)
    {
        printf("        No ADF open!\n");
        return;
    }
    char filename[INPUT_MAX];
    printf("Filename: ");
    scanf("%s", &filename);
    FILE *fp = fopen(&filename[0], "rb");
    if (!fp)
    {
        printf("        Failed to read file!\n");
        return;
    }
    struct AdfFile *afp = adfFileOpen(vol, &filename[0], 'w');
    if (!afp)
    {
        printf("        Failed to create file!\n");
        fclose(fp);
        return;
    }
    uint8_t byte;
    int byte_return = 0;
    while (1)
    {
        fread(&byte, 1, 1, fp);
        byte_return = adfFileWrite(afp, 1, &byte);
        if (byte_return != 1)
        {
            printf("        Disk Write Error.\n");
            break;
        }
        if (feof(fp))
        {
            break;
        }
    }
    adfFileClose(afp);
    fclose(fp);
}

void pull()
{
    if (!adf_open)
    {
        printf("        No ADF open!\n");
        return;
    }
    char filename[INPUT_MAX];
    printf("Filename: ");
    scanf("%s", &filename);
    FILE *fp = fopen(&filename[0], "wb");
    if (!fp)
    {
        printf("        Failed to open output file!\n");
        return;
    }
    struct AdfFile *afp = adfFileOpen(vol, &filename[0], 'r');
    if (!afp)
    {
        printf("        Failed to open file!\n");
        fclose(fp);
        return;
    }
    uint8_t byte;
    while (!adfEndOfFile(afp))
    {
        adfFileRead(afp, 1, &byte);
        fwrite(&byte, 1, 1, fp);
    }
    adfFileClose(afp);
    fclose(fp);
}

A generic API for all types of devices

The main ideas:

  • make a simple, generic and unified API for all types of devices
  • provide a simple way for creating other, used-implemented types of devices
    (one of such devices is ramdisk, proposed and implemented by @kyz, hooked so far as a "native" device)

adflib debian/

the package already exist but is called unadf. Tomasz do you want to maintain it for debian? I can sponsor your packaging…

Can't access empty native device

After writing my own native device handlers, it seems that the adfMountDev() returns nullptr if the disk (DD) doesn't already contain filesystem. (*adfEnv.wFct)("adfReadRootBlock : id not found");

Is there something I'm missing or why is this considered an error?

Fuzz testing

Google runs OSS-Fuzz project, where they spend a lot of CPU cycles fuzz-testing free software. They also offer bounties for getting projects to signing up (which makes me think of Ratbert's rat dance)

Laurent, would you consider requesting? Google may or may not accept.

Even if Google don't accept it, it would help us to have a fuzz test framework, i.e. develop a program like unadf, but without writing the extracted results to disk, and looks through all volumes on the device, and does traversal both with and without dircache if dircache is present. Then give it some disk images, and the fuzzer does the rest, mutating the input in random / directed ways to try and make the code go down different paths and crash. It watches the code execution flow and looks at what memory comparisons were made.

I'm not sure how we could fuzz-test writing files. I like to think that the only way that it would crash is by some sequence of operations, e.g. Open, Write, Seek, Read, Seek, Write..., or even have multiple files being written simultaneously. How would we turn a single input buffer into such a sequence, so the fuzzer can direct it? Perhaps some single-character string language, like "O" means open a new random-named file, "D" means create a directory, ... and other commands like enter a directory, go back to parent directory, switch to the next open file handle in a ring, write a block, read a block, seek forward a block, seek back a block, close a file handle, etc.

Updating existing files in an ADF image

I have started working on write support for fuseadf and have just found out that the library does not support updating existing files in ADF image. The write support modes (for adfOpenFile()) are:

  • "w" - only creating new files
  • "a" - only appending to existing files

This is probably not the simplest feature to implement but a pretty crucial one (eg. at the moment, there is no reasonable way to implement write support in a filesystem using ADFlib) so it would be good to add it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.