mochidev/codabledatastore

An ACID-compliant database written in pure-swift enabling on-disk persistence for apps and services.

codabledatastore's Issues

Improve ergonomics of using computed indexes

Parameter packs or macros should be able to help here, hopefully…

Investigate why an initial index file is created, but not tracked

When a persistence is created and a datastore is registered, an empty index is created, but is not tracked in any way.

Indexed should conditionally conform to Hashable

If T conforms to Hashable, so should Indexed.

In-process registry of open disk persistences

We should utilize an in-process registry to ensure that only one disk persistence for a given URL is used at once, logging to make sure a developer doesn't inadvertently create a second instance.

Convert between a read-write DiskPersistence to a read-only view of one

Add a proper way to convert between a read-write DiskPersistence to a read-only view of one, backed by the same instance.

Fix DiskPersistence/defaultURL so it doesn't throw

Specifically in cases when a bundle ID or app support directory is not present.

Rename load(_ range:)

We should rename load(_ range:) to load(_ idRange:) to make it clear to code completion results that this is not for the regular indexed fields.

Create a macro for @Indexed that automatically grabs the property names

Also rename the current Indexed to _Indexed to make the underlying bits work, and figure out how to make key play nice with Codable.

Add deleteIfPresent method that doesn't throw if the entry is missing

Error "Is a directory" when reopening existing store

I've been getting the following error any time I try to reinitialize a store that was previously created. Creating and using a store the first time always works, but if I try to reuse the same URL when rebuilding it always fails with this error:

CodableDatastoreTest/CodableDatastoreTestApp.swift:18: Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=NSCocoaErrorDomain Code=256 "The file “2023-07-13 20-58-15 29023E6541A4E39A.snapshot” couldn’t be opened." UserInfo={NSFilePath=/Users/_______/Library/Developer/CoreSimulator/Devices/B6EABA41-EBF0-46A4-8A33-0150DDA84DD2/data/Containers/Data/Application/505B0923-E569-441A-862C-351FA816B866/tmp/73B58D72-6462-4C31-B5B9-FF5DCF5C085F-19496-000005FDA764CEDA/Snapshots/2023/07-13/20-58/2023-07-13 20-58-15 29023E6541A4E39A.snapshot, NSUnderlyingError=0x600000c98660 {Error Domain=NSPOSIXErrorDomain Code=21 "Is a directory"}}

I even copied the DiskPersistenceDatastoreTests file and tried to run it twice with a static URL and get the same error

Example func that works the first time and successfully persists and then fails with above error when trying to build and run a second time

func write() {
        let url = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: "group.com.pnewelljr.CodableDatastoreTest")!.appending(path: "Store", directoryHint: .isDirectory)
        let persistence = try DiskPersistence(readWriteURL: url)
        try await persistence.createPersistenceIfNecessary()

        let datastore = Datastore.JSONStore(
            persistence: persistence,
            key: "test",
            version: Version.zero,
            migrations: [
                .zero: { data, decoder in
                    try decoder.decode(TestStruct.self, from: data)
                }
            ]
        )
       try await datastore.warm()
      
        
        try await datastore.persist(TestStruct(id: "3", value: "My name is Dimitri"))
        try await datastore.persist(TestStruct(id: "1", value: "Hello, World!"))
        try await datastore.persist(TestStruct(id: "2", value: "Twenty Three is Number One"))

        let count = try await datastore.count
}

Investigate adding configurable cache sizes to DiskPersistence

#158 introduced some defaults, but it would be ideal if they could be configured as well. Additionally, platforms like watchOS could use smaller defaults to have a smaller footprint out of the box.

Add contains variant to datastore

This is similar to a load for an identifier, but it instead skips decoding, just returning if an entry exists.

Load on an empty datastore fails

If you attempt to load a value when the datastore has no root object, a DatastoreInterfaceError.datastoreKeyNotFound error will be thrown instead of nil.

Use bookmark data for URL locations to support file moves

To better support file moves, use a bookmark as the primary storage in memory. This way a data store can be used as a file the user can move around.

Support static properties for default disk stores

It would be much nicer to get a single common instance when getting the default store:

extension DiskPersistence where AccessMode == ReadWrite {
    /// The default persistence for the read-write store of an app.
    public static var `default`: DiskPersistence<AccessMode> = ...
}

extension DiskPersistence where AccessMode == ReadOnly {
    /// The default persistence for the read-only store of an app.
    public static var readOnlyDefault: DiskPersistence<AccessMode> = self.default.readOnly
}

However, the default URL helper cannot through in that case, specifically on Linux, and we must have a way from converting from a read-write store to a read-only one.

Depends on:

• #15
• #16

Store indexed lookups in the primary datastore to aid in deletions

Create a dedicated actor for DiskIO

This should have a static method that takes a url, and returns a shared actor for the volume the URL resides on. This way, all synchronous IO can be isolated to a single actor. Additionally, when running with the 5.9 runtime, we should use a custom executor to grab a new thread for that IO rather than use the swift concurrency pool, since it would eat up a slot otherwise.