julienbayle/quix-library

Library samples repository of Quix. Explore and Deploy them easily on https://portal.platform.quix.ai

Quix library

Library samples repository for Quix platform.

The Quix library contains pre-built, open source code samples that you can use to quickly create a pipeline that transforms data in real time while it travels from source to destination.

The library includes three types of samples:

Sources: data connectors that you can plug and play. Or add an API gateway to push data. If you have a project in mind but the connector isn’t available, you can develop your own and share the code with Quix users by adding it to the library. You never know what kind of projects you might be supporting.

Transformations: code samples you can use as-is or modify to make your application more functional. Extract and load, merge, synchronize, enrich, predict or anything else you might want to do with your data — the options are endless since you can extend the samples with any class, method, package or library.

Destinations: the connectors to analytics dashboards, alerts, data warehouses and any other form of delivery. What’s the good of transforming data if you can’t easily apply it?

Explore, develop and deploy them directly on https://portal.platform.quix.ai.

You can create a Quix account for free here.

Contributing

Contributors are very welcome at Quix! Follow this guide to get a project into production on the platform.

Adding a new source, transformation or destination

1. Create a fork of the quix-library

2. Add a folder to the directory that best suits your project (e.g., Python>Sources>New Folder)

3. Add your source code to the folder

4. Complete a library.json with the relevant details for your project

   • ensure you follow the guidelines in the Empty folder for each directory

   • ensure you include a thorough README.md that follows the template

   • ensure you include a library.json that follows the template

5. Submit a PR for approval

Adding a new application

Applications store all the code to run an end-to-end app. They are made up of multiple projects.

1. Create a fork of the quix-library

2. Add a folder to the Applications directory and name it something short and obvious like "chat app with real-time sentiment scoring"

3. Add your source code to the folder

   • Create a sub-folder for each component project of your application, i.e., one for the chat app frontend and one for the real-time machine learning model on the backend

   • ensure you include a library.json with the relevant details

   • ensure you include a thorough README.md that follows the template

   • ensure you follow the guidelines in the Empty folder for each directory

4. Submit a PR for approval

Modifying a project

Follow the same pattern used when adding a new application

1. Create a fork of the quix-library

2. Modify your files

3. Check the readme.md and library.json files

4. Submit a PR for approval

Configuring library.json

Library.json contains the metadata that we use to provide a smooth developer experience in the Quix portal by powering the search and set-up features in the Library. Both features make your code more reusable for other developers.

"libraryItemId" A unique identifier for your project.

Create a GUID using an online generator like Free Online GUID Generator

"name" The display name for your project. Keep it short (<30 chars) and try to make it unique.

"language" The projects language (Python, C#, NodeJs, Shell Script, Javascript etc)

See here for a current list of languages.

"tags" Additional search filters that show-up in the library. Pick two or three of the most important tags like "Complexity," "SDK Usage" and "Pipeline State." See Quix Library for a current list of tags. Add new tags with discretion — they may not be accepted.

"shortDescription" A unique description of your project. Keep it less than 80 characters.

"DefaultFile" Defines the default file to show when loading the project in a Code Explorer and is the default file displayed in the code preview in the Quix Library.

"EntryPoint"

The build and deploy entry point (commonly a Dockerfile).

"RunEntryPoint"

The Run entry point (main code file to run).

"Variables"

Defines the external variables of the project.

Variables are used to configure the project during setup in the Quix Library or programatically from your application.

Each variable will create a unique setup field in the Quix Library.

"Name" The display name for your variable

"Type" The type of variable. Use Environ

"InputType"

• "HiddenText" for tokens and credentials

• "FreeText" for anything else

• "InputTopic" for consuming data from topics. Multiple topics permitted.

• "OutputTopic" for producing data to topics. Multiple topics permitted.

"Description" The public description of the variable

"Required" Boolean value

"DefaultValue" A default value for the variable

"SampleTemplate"

Defines whether the project is a Sample Template without ability to Save as Project or Explore the files (used for HTTP samples or similar templates using Placeholders)

"IconFile"

Defines an icon file to show in the project card. Optional.

Only required for recognised logos and technologies - source the official logo.

Required shape: square

Recommended format: .png

Recommended size: 48x48 pixels

"DeploySettings"

Defines the Instant Deploy settings for the project. Optional.

If you configure these properties, the project will have the "Setup & Deploy" button enabled in the Quix library. This provides users with the option to deploy the project without cloning it to their own repo. It’s particularly useful for connectors (source and destinations) and applications that don’t require customization.

If the settings are null, a Docker Image for this project will not be generated and the Deploy actions will not be available in the UI.

• "DeploymentType"

  Type of deployment. Can be a Service or a Job.

• "CpuMillicores"

  Maximum CPU millicores reserved for this deployment instance. 1 millicore = 1/1000th core

• "MemoryInMb"

  Maximum memory reserved for this deployment instance. 1mb = 1/1000th GB

• "Replicas"

  Number of duplicate instances of the deployment. Used for horizontal scale.

  Each instance will reserve the max CPU & Memory allocations.

• "PublicAccess"

  Whether the service has a publicly accessible url.

• "UrlPrefix"

  Prefix of the public url.

• "ValidateConnection"

  Whether the Deployment process should validate the connection of the project.

  Specific logic is needed in the code of the project to allow the UI to validate the connection.