foretaginc/alchemy

Archived - High performance, realtime BaaS with RBAC, graphing, full text search, S3 / B2 Storage and GIS with a GraphQL, gRPC and REST API

Alchemy

Alchemy is an open source, batteries included data engine, it provides a GraphQL API for your ArangoDB database with support for authentication, webhooks, events, remote schemas among other features. It is written in Rust with performance and reliability in mind and to be enterprise ready.

Features

• Designed for Kubernetes and scalability
• Multi protocol CRUD for your database automatically
  • GraphQL
  • REST
  • gRPC
• Realtime updates (via GraphQL Subscriptions)
• First class support for Files via S3 and B2
• GIS capabilities for Maps & Navigation
  • Supports GeoJSON and GeoSpatial Queries
• Full text search (via ArangoSearch)
• Role based Authorization
• Eventing system (Actions & Triggers)
• Admin Console
• CLI
• Remote (GraphQL) Schemas
• Metadata APIs
• Schema Migrations
• Caching (frequent queries)
• Read-only replicas
• API limiting
• Collaborators (& roles)
• Single Sign On
• Alerting
• APM & Monitoring
• Backup and restore

and more planned / in the works.

Community

Get involved with Alchemy and our community on our matrix server:

alchemy's Issues

Feature Request: Relationship CRUD

Preliminary checklist

• I understand that all communication must be in English
• I read the Documentation
• I read the Contributing guide
• I agree to follow the Code of Conduct
• I searched the issues and discussions but couldn't find anything (or linked relevant results below)

Problem

Right now there's no way to actually create, read, update and delete relationships for existing entities.

Solution

The solution for this issue is implementing new dynamic queries and mutations based on the database structure. Depending on the relationship type different solutions are provided.

Progress

• One to one
• One to many
• Many to one
• Many to many

One to one

Example: on this example there's an entity being Department, and another being Manager, and a relationship one to one between those two entities called DepartmentManager.

Structure

• Entities:
  • Department
  • DepartmentManager
• Relationships:
  • Department to DepartmentManager | NOTE: With the current system the edge name resulting from the creation through the meta API would be department_department_managers which is not accurate. To avoid these names we would have to do some kind of special namings for edges, prompting it to the user, for example, he would set from and the to collections on the meta API, if there's overlapping on the name, we would remove the overlapped word, making the resulting edge name: department _department _manager → department_managers

Edge definitions

Manager property on Department entity

{
  "name": "manager",
  "edge": "department_managers",
  "from": "departments",
  "to": "managers",
  "type": "one_to_one",
  "direction": "inbound"
}

Department property on Manager entity

{
  "name": "department",
  "edge": "department_managers",
  "from": "managers",
  "to": "departments",
  "type": "one_to_one",
  "direction": "outbound"
}

New types

type DepartmentManagerRelationship {
    _from: Department!
    _to: DepartmentManager!
}

New queries

type Query {
    getDepartmentManagerRel(where: { from: DepartmentIndexFilter!, to: DepartmentManagerIndexFilter! }): DepartmentManagerRelationship!
    getDepartmentsManagersRels(where: { from: DepartmentBoolExp!, to: DepartmentManagerBoolExp! }): [DepartmentManagerRelationship!]!
}

New mutations

type Mutation {
    # This would throw error if one of the entities already has a relationship (this is handled by ArangoDB indices),
    # because it's only one to one
    createDepartmentManagerRel(object: { from: DepartmentIndexFilter!, to: DepartmentManagerIndexFilter! }): DepartmentManagerRelationship!
    # If from is empty on _set, then the where has to have a filter on from because we're only changing one part of the 
    # relationship, otherwise, if to is empty on _set, then the where has to have a filter on to. If both are filled at least one 
    # of the filters both of the filters has to exist as they cannot have duplicates. In case both are empty throw an exception.
    updateDepartmentManagerRel(_set: { from: DepartmentIndexFilter, to: DepartmentManagerIndexFilter }, where: { from: DepartmentIndexFilter, to: DepartmentManagerIndexFilter }): DepartmentManagerRelationship!
    # At least one of the params has to be filled, in case they're not, throw an exception.
    removeDepartmentManagerRel(where: { from: DepartmentBoolExp, to: DepartmentManagerBoolExp }): DepartmentManagerRelationship!
}

Notes

• All the unique field validation MUST be handled by ArangoDB.
• Add validation when adding or updating a relationship to check whether the entity on from or to exists

---

TODO: Expand this on the issue conversation with other relationship types procedurally when implementation gets completed.

Alternatives

No response

Contribution

Yes, I will work on a PR

Keywords

graphql, relationships, integration

Feature Request: Files and storage backend

Is your proposal related to a problem?

Currently there is no easy way to handle file uploads via Alchemy.

Describe the solution you'd like

Storage backends

A storage backend for Alchemy needs to be implemented both on B2 and S3 allowing for the user to select the appropriate backend, file path; It should be possible to add multiple storage backends (even of the same type) i.e. accounts collection storing a profile picture on a S3 platform and orders collection storing an invoice PDF on a B2 platform while deliveries collection storing a shipping image on a different S3 platform.

Files

A scalar value for Files should be added to Alchemy that acts like an Object with the following properties:

type File {
        id: ID!
	name: String!
	url: String!
	created_at: DateTime!
}

The file should be uploaded and represented by this Scalar.

Describe alternatives you've considered

Not relevant

If the feature is approved, would you be willing to submit a PR?

Not relevant

Bug: Filter available on unique relationships

Preliminary checklist

• I understand that all communication must be in English
• I read the Documentation
• I read the Contributing guide
• I agree to follow the Code of Conduct
• I searched the issues and discussions but couldn't find anything (or linked relevant results below)

Version

dev

Environment

N/A

Background

Whenever you create a relationship between two entities on which one of the arms of the relationship is a One, for example: ManyToOne, OneToMany, OneToOne, the Schema generated should create on the One arm a single-entity returning type without any filter as it doesn't make sense to filter something that is always one.

Current behavior

Right now the Schema generated adds filtering for both arms.

Steps To Reproduce

1. Create one entity, example: Account
2. Create another entity, example: Role
3. Create a relationship between these entities with a relationship property on both of them: AccountRole

Screenshot or Screencast

Logs and traces

No response

Solution

Disable filtering for single-entity returning fields

Debugging

No response

Contribution

Yes, I will work on a PR

Keywords

graphql, relationships

Feature Request: PostgreSQL?

Anything planned for PostgreSQL?

Feature Request: Split up alchemy-related collections to a meta-only database

Preliminary checklist

• I understand that all communication must be in English
• I read the Documentation
• I read the Contributing guide
• I agree to follow the Code of Conduct
• I searched the issues and discussions but couldn't find anything (or linked relevant results below)

Problem

Right now all the meta-data related to the Alchemy instance is on the same database as the real data is.

This is a problem in case of overlapping between table names and entities, for example, if someone creates an entity called AlchemyCollection (for some reason their project needs this name) there would be an error as the collection name generated after this would be alchemy_collections which is already reserved for the Alchemy instance meta-data.

Solution

The solution for this is to create another database with the name _{name}_meta, so, for example, if a database is called game, its meta-data database is gonna be _game_meta.

This new database is where all the Alchemy-related collections are gonna be stored. This fixes the overlapping issue between collection names and also isolates the real information from the Alchemy data which may help some projects to migrate their data or other things.

Alternatives

The problem could also be solved by instead of using ArangoDB as the medium to store meta-data-related information, using another storage system such as files. This could also work as the meta-data information Alchemy handles is not big, so there wouldn't be any issues with this system.

Contribution

Yes

Keywords

integration, meta

Feature Request: Schema (re)generation on runtime

Is your proposal related to a problem?

Currently there is no way to use Alchemy before making any changes via the Meta API or by manually altering the alchemy_collections, this requires a complete a rebuild / rerun of the project taking up valuable development time.

Describe the solution you'd like

We should abstract the schema generation allowing for on demand schema generation and then Alchemy should rely on Arango events to trigger a schema regeneration when changes are made to the alchemy_collections and other relevant collections.

Describe alternatives you've considered

References to projects that rely on HTTP Streams from Arango:
https://github.com/baslr/arangochair
https://github.com/baslr/arangochair-serversendevents-demo