shpiyu/ingen

InGen is a command line tool written on top of pandas and great_expectations to perform small scale data transformations and validations without writing code.

InGen

InGen is a command line tool written on top of pandas and great_expectations to perform small scale data transformations and validations without writing code. It is designed for developers and analysts to quickly transform data by specifying their requirements in a simple YAML file.

Table of Contents

• Installation
• Examples
• Contributing
• License

Installation

Note: This project has been tested on python version 3.7.9 and may not work on newer versions of python. There is an open issue to fix this problem.

To install the project locally follow the steps:

1. Make sure you have Python 3.7.9 installed on your system.
2. To be able to build the project locally, you will need to install the build package

   pip install build
   

3. Clone the repository

   git clone [email protected]:blackrock/interface-generator.git 
   

4. Build the project

   cd interface-generator
   python -m build 
   

5. Install the wheel

   pip install dist/interface_generator-*.whl
   

6. Run the project

   python -m ingen <metadata file path>
   

Examples

Checkout the sample metadata files in the examples directory to see how InterfaceGenerator can be used to solve common data problems.

Examples
Merge two CSV files

For a complete list of configuration options, see the metadata reference documentation here.

Contributing

All contributions are welcome, please see open issues or create a new issue to discuss your ideas. Please see our contributing guidelines for more information.

License

LICENSE