dca923/nipyapi

A convenient Python wrapper for Apache NiFi

NiPyApi

Nifi-Python-Api: A rich Apache NiFi Python Client SDK

Features

Three layers of Python support for working with Apache NiFi:

• High-level Demos and example scripts
• Mid-level Client SDK for typical complex tasks
• Low-level Client SDKs for the full API implementation of NiFi and selected sub-projects

Functionality Highlights:

• Detailed documentation of the full SDK at all levels
• CRUD wrappers for common task areas like Processor Groups, Processors, Templates, Registry Clients, Registry Buckets, Registry Flows, etc.
• Convenience functions for inventory tasks, such as recursively retrieving the entire canvas, or a flat list of all Process Groups
• Support for scheduling and purging flows, controller services, and connections
• Support for fetching and updating Variable Registries
• Support for import/export of Versioned Flows from NiFi-Registry
• Docker Compose configurations for testing and deployment
• A scripted deployment of an interactive environment, and a secured configuration, for testing and demonstration purposes

Please see the issue register for more information on current development.

Quick Start

There are several scripts to produce demo environments in nipyapi.demo.*

The mid-level functionality is in nipyapi.canvas / nipyapi.security / nipyapi.templates / nipyapi.versioning

You can access the entire API using the low-level SDKs in nipyapi.nifi / nipyapi.registry

The easiest way to install NiPyApi is with pip:

# in bash
pip install nipyapi

You can set the config for your endpoints in the central config file:

# in python
import nipyapi
nipyapi.config.nifi_config.host = 'http://localhost:8080/nifi-api'
nipyapi.config.registry_config.host = 'http://localhost:18080/nifi-registry-api'

Then import a module and execute tasks:

nipyapi.canvas.get_root_pg_id()
>'4d5dcf9a-015e-1000-097e-e505ed0f7fd2'

You can use the Docker demos to create a secured interactive console showing many features:

from nipyapi.demo.secured_console import *
from nipyapi.demo.console import *

You can also explore the scripts to get ideas for how NiPyAPi can be used to automate your environment.

Please check out the Contribution Guide if you are interested in contributing to the feature set.

Background and Documentation

For more information on Apache NiFi, please visit https://nifi.apache.org

For Documentation on this package please visit https://nipyapi.readthedocs.io.

NiFi Version Support

Currently we are testing against NiFi versions 1.1.2 - 1.10, and NiFi-Registry versions 0.1.0 - 0.5.0.

If you find a version compatibility problem please raise an issue

Python Requirements

Python 2.7 or 3.5-7 supported, though other versions may work. We will shortly stop supporting Python2

Tested on AL2 and OSX 10.14.x - Windows automated testing not attempted

Outside of the standard Python modules, we make use of lxml, DeepDiff and ruamel.yaml in processing, and Docker for demo/tests.