spacecloud-io/space-cloud

Open source Firebase + Heroku to develop, scale and secure serverless apps on Kubernetes

SpaceCloud is currently in maintaiance mode. Checkout Hasura or Wundergraph instead.

Develop, Deploy and Secure Serverless Apps on Kubernetes.

Website • Docs • Support

Space Cloud is a Kubernetes based serverless platform that provides instant, realtime APIs on any database, with event triggers and unified APIs for your custom business logic.

Space Cloud helps you build modern applications without having to write any backend code in most cases.

It provides GraphQL and REST APIs which can be consumed directly by your frontend in a secure manner.

Features

View complete feature set here.

• Powerful CRUD: Flexible queries, transactions, aggregations and cross-database joins
• Realtime: Make live queries to your database
• File storage: Upload/download files to scalable file stores (e.g., Amazon S3, Google Cloud Storage)
• Extensible: Unified APIs for your custom HTTP services
• Event-driven: Trigger webhooks or serverless functions on database or file storage events
• Fine-grained access control: Dynamic access control that integrates with your auth system (e.g., auth0, firebase-auth)
• Scalable: Written in Golang, it follows cloud-native practices and scales horizontally
• Service Mesh: Get all the capabilities of a service mesh without having to learn any of that!
• Scale down to zero: Auto scale your http workloads including scaling down to zero

Supported databases ❤️:

• MongoDB
• PostgreSQL and PostgreSQL compatible databases (For eg. CockroachDB, Yugabyte, etc.)
• MySQL and MySQL compatible databases (For eg. TiDB, MariaDB, etc.)
• SQL Server

Table of Contents

• Quick Start
• Client-side tooling
• How it works
• Support & Troubleshooting
• Contributing
• License

Quick start

If you are new to Space Cloud, we strongly recommend following our step-by-step guide to get started

Other guides

View the installation guides for Docker and Kubernetes.

Client-side tooling

Space Cloud exposes GraphQL and REST APIs. See setting up project guide to choose a client and set it up.

GraphQL APIs

GraphQL is the recommended way to use Space cloud, and it works with any GraphQL client. However, we recommend using Apollo Client. See awesome-graphql for a list of clients.

REST APIs

You can use the REST APIs of Space Cloud if you are more comfortable with REST.

To make it easy to consume the REST APIs in web projects, we have created a Javascript SDK for you.

How it works

Space Cloud is meant to replace any backend php, nodejs, java code you may write to create your endpoints. Instead, it exposes your database over an external API that can be consumed directly from the frontend. In other words, it allows clients to fire database queries directly.

However, it's important to note that the client does not send database (SQL) queries to Space Cloud. Instead, it sends an object describing the query to be executed. This object is first validated by Space Cloud (using security rules). Once the client is authorized to make the request, a database query is dynamically generated and executed. The results are sent directly to the concerned client.

We understand that not every app can be built using only CRUD operations. Sometimes it's necessary to write business logic. For such cases, Space Cloud allows you to access your custom HTTP servers via the same consistent APIs of Space Cloud. In this scenario, Space Cloud acts merely as an API gateway between your services and the client. However, the cool part is that you can even perform joins on your microservices and database via the GraphQL API of Space Cloud.

Space Cloud integrates with Kubernetes and Istio natively to bring to you a highly scalable Serverless Platform. It encrypts all traffic by default and lets you describe communication policies to protect your microservices.

With that, it also provides autoscaling functionality out of the box including scaling down to zero.

Support & Troubleshooting

The documentation and community should help you troubleshoot most issues. If you have encountered a bug or need to get in touch with us, you can contact us using one of the following channels:

• Support & feedback: Discord
• Issue & bug tracking: GitHub issues
• Follow product updates: @spaceupcloudio

Contributing

Space Cloud is a young project. We'd love to have you onboard if you wish to contribute. To help you get started, here are a few areas you can help us with:

• Writing the documentation
• Making sample apps in React, Angular, Android, and any other frontend tech you can think of
• Deciding the road map of the project
• Creating issues for any bugs you find
• And of course, with code for bug fixes and new enhancements

License

Space Cloud is Apache 2.0 licensed.

space-cloud's Issues

Custom fields in signUp of user management module

error while creating config via Init command in Windows

Error thrown :-
/d:\codeformer\space-cloud\space-cloud.yaml: The filename, directory name, or volume label syntax is incorrect.

Improper error codes while signing up

Need syntax highlighting in docs

The docs styles isn't proper

I noticed some styling issues with the docs specifically for the code tags. Can we make the long code lines wrap automatically?

Also, it would be great if the width of the docs would be larger on the browser.

The security rules don't match the project name on the client to the project id in the config file

This is potentially a security bug for MongoDB clusters having multiple databases.

Need to solve this asap!

Can we publish internal Space Cloud events to a pluggable sink?

We should have a mechanism to publish internal events for functions to subscribe on. We can publish events like

• CRUD operation hooks
• File Upload

Need to create a language specific getting started in docs

We need to make a language specific getting started guide as well. It should show the baisc commands which can be used along with the steps to import and initialise the library!

AllRequest may be redundant

Currently, server.proto defines BatchRequest in the following way:

message AllRequest {
  string col = 1;
  bytes document = 2;
  string operation = 3;
  bytes find = 4;
  bytes update = 5;
  string type = 6;
}
message BatchRequest {
  repeated AllRequest batchrequest = 1;
  Meta meta = 2;
}

Instead of AllRequest we can directly use the already defined CreateRequest, ReadRequest, etc

message BatchRequest {
  repeated CreateRequest cr = 1;
  repeated ReadRequest rr = 2;
  repeated DeleteRequest dr = 3;
  repeated UpdateRequest ur = 4;
  repeated AggregateRequest ar = 5;
  Meta meta = 6;
}

To be considered:

1. This new method will remove the AllRequest message.
2. The predefined messages will be used. This will help to segregate and identify the specific components in the batch request. This may also make it easier to implement in the client modules.
3. The meta field may be repeated, but we can always omit it in the component CreateRequest, ReadRequest, etc.

Need to improve the docs and quick start

The docs are not good enough. People new to the project won't be able to understand its aim and where it fits. Also the quick start seems to be a bit scattered and doesn't paint a clear picture regarding what it does.

The production (distributed) version of Realtime is still pending

We need to add kafka for the distributed realtime feature. Also we need not rely just on kafka, we can use any message broker with an at least once semantics for this task. Something like the pluggable backend for the file store module.

Need to create gRPC endpoints for the filestorage module.

Do we even wanna do this? @Jayesh333 and @KhushalPShah we should select which function's we want in GRPC.

grpc is not realtime

The GRPC endpoints are not Realtime yet.

Ability to put raw MongoDB where query in the mongo client sdk !

We are already sending out the where clause as a MongoDB query to the backend. Why not let the user put the raw query directly?

I know this is a request for the client libraries. But lets put it here to keep track. The updates of it being done on individual clients can be done here.

Typo mistake in Readme

The main reason Space Cloud was born was to simplify the app / web development process. Right from making simple CURD CRUD operations to syncing data reliably in a distributed environment, everything must be as simple as a function call.

Can space-cloud download the example and run them directly?

It would be great if the space-cloud init command could setup the examples and start them as well.

Add joins to all supported databases

We need to add support for doing joins. In SQL databases it would be a native join while in Mongo it would be an application side join.

The concern I have here is, how will the where (find) clause be applied on the joints table.

Difficult to understand if space-cloud has actually started or not

No where in the docs have we mentioned whats the output to expect when space-cloud has started successfully. We should add something in the docs

insecure connection between server and client

SSL absent, no provision to transfer data in a secure manner

Space CLI isn't functional. Remove that from the docs completely

Running npm install -g space-cli installs an incorrect npm package. Let's completely remove the concept of space-cli.

Referring to issue on fixing docs #28

Leverage the implementation of grpc CRUD to the CRUD module, similar to FaaS

Just like Faas (using GRPC) has been implemented inside the FaaS module, with only a function call from the grpc server, we should have the same thing done for CRUD implementation using grpc as well.

The download config file steps in the quick start guide leads to errors

I've heard quite a few people having troubles with downloading the config file step. Apparently copying and pasting the config directly in a file doesn't work.

@KhushalPShah faced the same issue I guess.

Request to allow hosting static files.

It would be really useful if space-cloud allowed the user to host static files as well.
Adding this feature will definitely help space-cloud lift up above the others.

Support for search

Can we add support for search in the database we already support.

The config file used in the example can be more elaborate

I thing the config file provided in the example here should cover all the possible configuration. Maybe we can put the defaults of each module (enabled = false)?

Also it will be of great help if the config file has helpful comments. The config file should be self documenting.

Add interactive transaction support

We need to provide a unified transaction api for all the databases that support transactions.

Allow folder(s) to be auto created when uploading a file if the given path doesn't exists

Should there be an options struct {upsertFolder: true} or should it be the default behavior?

The stackoverflow, discord, and github links on docs is broken

Need to fix those links in the docs template we are using.

Add GraphQL support for database and realtime modules

Something similar to what we doing with gRPC

Need logging in Space Cloud

Currently space cloud doesn't provide any logging. We need a proper logging mechanism with configurable levels.

Typo mistake in Readme

The main reason Space Cloud was born was to simplify the app / web development process. Right from making simple CURD CRUD operations to syncing data reliably in a distributed environment, everything must be as simple as a function call.

Pub sub module

Problem faced right now

There's no way in SC to let users sync data easily without touching the Database module

How can we solve it

We can have a pub sub module built into Space Cloud which lets users to publish and subscribe data.

We need to make sure the security rules are enforced for each object published. Also the consistency level needs to be tunable.

For example, if a client looses the connection due to connectivity issues at the time a data point was published, should he be delivered the message or not?

Loads of things to consider to do this right!

Need to rename transaction to batch in http and gRPC endpoints.

The transaction support we have added works more like a batch. Let's make that more explicit. This will require us refactoring:

• The https endpoint (url)
• The gRPC rpc name.
• The CRUD interface in crud.go and operations.go
• the func name in sql and mongo packages

API for CRUD needs to be updated

The API for CRUD needs to be updated to make it short and more intuitive.

Here's the current API for javascript:

// Insert one / all:
db.insert(col).one(doc).then(res => ...).catch(ex => ...);
db.insert(col).all(docs).then(res => ...).catch(ex => ...);

// Get one / all:
db.get(col).where(whereClause).one().then(res => ...).catch(ex => ...);
db.get(col).where(whereClause).all().then(res => ...).catch(ex => ...);

// Update one / all:
db.update(col).where(whereClause).set(obj).one().then(res => ...).catch(ex => ...);
db.update(col).where(whereClause).set(obj).all().then(res => ...).catch(ex => ...);

// Delete one / all:
db.delete(col).where(whereClause).one().then(res => ...).catch(ex => ...);
db.delete(col).where(whereClause).all().then(res => ...).catch(ex => ...);

In the new API there should not be explicit one() and all() method. The API should take the default operation type to be all since one is used very rarely and also one is not available on SQL. However, the user should be able to change the op by calling an optional op method. The method apply will actually trigger the request.

This is the new proposal:

// Insert all:
db.insert(col).docs(docs).apply().then(res => ...).catch(ex => ...);
// Insert one:
db.insert(col).doc(doc).apply().then(res => ...).catch(ex => ...);

// Get all:
db.get(col).where(whereClause).apply().then(res => ...).catch(ex => ...);
// Get one:
db.getOne(col).where(whereClause).apply().then(res => ...).catch(ex => ...);
// Get count:
db.count(col).where(whereClause).apply().then(res => ...).catch(ex => ...);
// Get distinct:
db.distinct(col).where(whereClause).apply().then(res => ...).catch(ex => ...);

// Update all:
db.update(col).where(whereClause).set(obj).apply().then(res => ...).catch(ex => ...);
// Update one:
db.updateOne(col).where(whereClause).set(obj).apply().then(res => ...).catch(ex => ...);
// Upsert:
db.upsert(col).where(whereClause).set(obj).apply().then(res => ...).catch(ex => ...);

// Delete all:
db.delete(col).where(whereClause).apply().then(res => ...).catch(ex => ...);
// Delete one:
db.deleteOne(col).where(whereClause).apply().then(res => ...).catch(ex => ...);

unsafe password handling

actual plaintext passwords being sent in user management, really unsafe thing to do. Standard way is to salt the passwords and the compare the hashes, not plain text passwords

Use context in SQL

Use the context object in every SQL db operation similar to mongo

Filestore module doesn't let us delete a file in localstorage

Need to add the functions module endpoint in gRPC

Also, none of the function endpoints support streaming responses yet.

Need to create Transaction endpoint in grpc

A new transaction endpoint is in the works for http. We need to do the same for grpc.

What sample apps can we create as use cases for Space Cloud

We should definitely create some use cases. Need ideas on what we can do!

Here are some to get started

• Realtime chat app with "last seen" and "is typing" feature
• An IoT Fleet management system
• a markdown editor where multiple people can collaborate in real time (google docs for markdown
• an e-commerce portal
• a social media platform

Even the most craziest of ideas are welcome!

Authorisation should happen on the document level in case of insert many

Currently the user will have to deal with arrays in the security rules for insert many operations. Lets break it down to the document level.

The transactions function in GRPC is named as `HandleTransaction`. Let's rename it to transactions.

@KhushalPShah try to finish this off asap. Will need this to be resolved to merge #85.

JWT token is not parsed correctly

space-cloud doesn't removes the prefix - "Bearer " from the Authorization Header

Add mechanism to use most appropriate technologies for the given Cloud vendor

Using NATS on Gcloud and Amazon's internal Service on AWS

Schema validation

The problem faced right now

There's no way to enforce a schema for schemaless databases(like MongoDB) in SC.

How can it be solved

Allow SC to accept a schema via Mission control for various collections. This schema should have the following field level config optons:

• Data type (int, string, bool etc)
• Default value
• Required
• Optional encryption / hash / bcrypt operation

The file authentication has a bug

The prefix matching never works properly. @Jayesh26, can you have a look at it and fix.

Should we make a subreddit for Space Cloud?

It might be too early right not. But should we make a subreddit for Space Cloud?

The docs are not in sync with the readme

The docs do not have the motivation section present in the readme. Need to make sure they both are in sync.

Need to create a gRPC endpoint to subscribe / unsubscribe for realtime changes

This will be similar to what we have over websockets currently.

Transaction is still not supported in the js library

The http endpoint is ready. Lets add support to the js client as well