opsdroid/opsdroid

🤖 An open source chat-ops bot framework

An open source chat-ops bot framework

---

Quick Start • Documentation • Playground • Blog • Community

---

An open source chatbot framework written in Python. It is designed to be extendable, scalable and simple.

This framework is designed to take events from chat services and other sources and execute Python functions (skills) based on their contents. Those functions can be anything you like, from simple conversational responses to running complex tasks. The true power of this project is to act as a glue library to bring the multitude of natural language APIs, chat services and third-party APIs together.

See our full documentation to get started.

Contributors

This project exists thanks to all the people who contribute. [Contribute].

Backers

Thank you to all our backers! 🙏 [Become a backer]

Sponsors

Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [Become a sponsor]

opsdroid's Issues

Add webhook parser

It should be possible to trigger skills via a webhook. This could be implemented as a matcher and parser to allow users to directly decorate their skills to be called by a webhook.

To implement the RESTful API Flask could be incorporated into opsdroid and run at startup.

An example skill could look like

Skill

from opsdroid.matchers import match_webhook

@match_webhook("mywebhook", "POST")
async def webhook_skill(opsdroid, config, message):
  pass

Config

skills:
  - name: webhookskill

This skill could then be triggered with a POST request to http://host:port/skill/webhookskill/mywebhook.

Responding to the message could return the body to the webhook caller and the message passed into the skill could have a property of data which contains the POST or GET data if there is any.

Improve module installation

• Allow for updating or reinstalling of modules
• Test if url or ssh path exists before installing
• Handle GitHub authentication

Should mutation tests be added?

It could be interesting to add mutation tests.

Had a quick go with cosmic-ray but it doesn't play well with asynctest.

Change skill module imports

Currently when importing a skill opsdroid is looking for a file with the same name as the skill within the skill directory.

This should be changed so it just looks for the skill directory. This would allow a skill to be either a directory containing an __init__.py file or a simple python file.

Old structure

modules/skills/myskill/myskill.py
modules/skills/anotherskill/anotherskill.py

New structure

modules/skills/myskill/__init__.py
modules/skills/anotherskill.py

Queue messages

Messages should be sent via multiprocessing queues.

Currently messages are responded to by calling the respond method on them, which in turn calls the respond method on the connector that message came from. However that means messages can only respond via their own connector.

Instead of directly sending messages back to their connector they could be put in a queue which is accessible to all threads. Each connector should check the queue for messages applicable to it and send them to the appropriate place.

This could also be taken further so that messages are parsed by a worker which is separate to the connectors.

Copy message on respond

When a message responds it updates it's text value and passes itself to the connector. Due to pointers in Python the next rule to parse the message goes on to parse the response text.

The message respond method should create a shallow copy of itself to pass to the connector, instead of updating itself directly.

Manage logging properly

When a function calls subprocess.Popen() the logging seems to reset to default and print to stdout and stderror.

This is probably because logging hasn't been configured properly. The opsdroid object should probably handle this as it is accessible almost everywhere.

Automate version number

Current the version number is hard coded in the opsdroid/const.py file which causes problems if it is accidentally not updated as in v0.7.0.

This should be set automatically at runtime either from the previous production build or from the current working repository.

Add tests

Error when logging not specified in config

Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/jacob/Projects/opsdroid/opsdroid/opsdroid/__main__.py", line 109, in <module>
    main()
  File "/Users/jacob/Projects/opsdroid/opsdroid/opsdroid/__main__.py", line 102, in main
    configure_logging(opsdroid.config)
  File "/Users/jacob/Projects/opsdroid/opsdroid/opsdroid/__main__.py", line 23, in configure_logging
    logfile_path = config["logging"]["path"]
TypeError: string indices must be integers

Project aims

This chatbot aspires to these aims:

• Can connect to multiple messaging clients simultaneously
• Multiple rule formats (regex, intents, etc)
• Can use Natural Language Understanding (NLU) services
• Plugins should be installed as python modules
• Users should interact with the bot in natural language, not act like a CLI in a chat client
• Bots should maintain knowledge of conversational context
• Bots should function well in a group chat environment as well as 1-on-1.
• Documentation and examples should be plentiful
• Advanced messaging client events like user login should be available

Change module configuration layout

As suggested by @tpowellmeto connectors/databases/skills are all lists of modules. Therefore they should probably be a list in the configuration rather than a dictionary.

This makes empty module configurations easier to read as they are not empty key/value pairs.

Before

skills:
  hello:
    additionalconfigitem: "value"
  seen:
  dance:
  loudnoises:

After

skills:
  - name: hello
    additionalconfigitem: "value"
  - name: seen
  - name: dance
  - name: loudnoises

Connectors should fork

When a connector is started it should fork into its own process. This is because connectors block to accept messages from their source.

This requires #5 to enable persistent memory between connector processes.

Generate default config

It should be possible to generate some basic config with a command line flag to opsdroid. It should cause opsdroid to print out the config so that is can be piped into a file.

e.g

opsdroid --gen-config > configuration.yaml

Cron skill decorator

It should be possible to decorate a skill with a cron method, which causes the skill to run at specific time intervals.

Add average response time stat

Average response time should be recorded and added to the stats page.

Check modules for required opsdroid version

It should be possible in opsdroid skills, connectors and databases to specify a minimum version of opsdroid required to run them. Then when opsdroid starts it can check to ensure it is able to support all modules specified. If not it can skip loading them and log a warning.

Rename skills submodule

When writing a skill you need to import a match function to decorate your function with.

from opsdroid.skills import match_regex

@match_regex("what is your name?")
async def botname(opsdroid, message)
    await message.respond("My name is {0}".format(opsdroid.bot_name))

This could potentially be confusing as the function you're importing isn't a skill, its a helper function for creating skills.

It should probably be renamed to one of the following:

• opsdroid.matchers
• opsdroid.match
• opsdroid.skill-helpers
• opsdroid.skill-decorators
• opsdroid.triggers

Switch from Multiprocessing to Threading

Multiprocessing allows the application to take advantage of multiple cores, however it means that each process has a separate memory namespace.

Threading allows multiple code threads to be running at once, sharing a single cpu resource. These threads are in the same memory namespace and can therefore interact more freely.

As opsdroid should not be doing any heavy computation it would simplify the design to switch from Multiprocessing to Threading. It would solve problems such as the cron timer not being able to call methods on the connectors.

It would still be useful to use #49 as an event bus to avoid problems.

Inspiration for this change taken from Home Assistant

Add ability to disable regex parser

You may not want the regex parser to be enabled.

Config locations

Currently opsdroid looks for the configuration.yaml file in the current working directory. It should also look in ~/.opsdroid/configuration.yaml and /etc/opsdroid/configuration.yaml.

Add coverage

Install modules locally

Currently modules are installed using git. Therefore when developing a module all changes must be committed before it can be installed, even if installing from a local git repository.

It should be possible to pass in a path instead of a git repository and for that to be used instead.

Change logging settings

By default opsdroid should log to stdout/stderror. There should be configuration options to specify logging to a file.

Documentation for the shell connector should also be updated to recommend setting log to file to avoid issues when using the connector.

Gather all requirements before installing

Currently when installing modules each set of requirements is individually pip installed. It would be better to gather all requirements (including core opsdroid requirements) and do a single pip install. That way any collisions will be detected and potentially handled by pip, or errored on.

Can help towards #102

Specify module install directory

Currently modules are installed to a modules directory in the current working directory. This is fine when being run in a container as you definitely have write access to the WORKDIR within the container. However this could cause problems if installed without docker.

A config option should be added to specify where modules should be installed, this should also be added to the python path within the application.

Documentation and website

As of v0.2.0 the core project has the main features implemented, which means it is ready for general consumption.

Therefore the following needs to be done:

• Documentation
  • Getting started
  • Contributing to the core
  • Creating connectors
  • Creating database connectors
  • Writing skills
• Social media and graphics
  • Logo
  • Banners
  • Twitter
  • GitHub
  • Docker Hub
• Website
  • Create opsdroid.github.io jekyll site

Don't await parsing messages

Messages parsed in opsdroid.parse() should be added to the event loop as a task instead of being awaited. When the jobs are awaited it blocks opsdroid.parse() which in turn blocks the Connector.receive() method causing messages to stack up or be missed.

Add travis

Thinking delay and typing delay

Configuration options should be added to allow users to set delays for responses. This would allow a bot to appear more human.

Thinking delay

This delay would be added to every message and would simulate a user thinking about what to respond. This option should take the delay seconds as an integer or an array of two integers to determine the delay or the range for random delays respectively.

Typing delay

This delay would be added to the response of every message and would simulate the user typing out the message. This option should take an integer of "words per minute" to determine the length of the delay relative to the length of the response. It should also trigger the "user is typing" call during the delay, for connectors that support it.

Dependancy hell in modules

When installing modules the dependancies can conflict with the main project deps. This seems to work fine on first run but if you then restart the application it fails due to incorrect deps.

Loader static methods

There are some functions at the beginning of the loader file. These should be private static methods in the class.

Add metadata constants to skills

Skill developers should be able to write a couple of string constants describing their skill and it's usage, these should be stored somewhere in opsdroid when the modules are loaded. An official "help" skill should also be created to allow users to access these constants.

Config in ~/.opsdroid/configuration.yaml doesn't load

When storing the config file in ~/.opsdroid/configuration.yaml you still get an error.

Regex case sensitive

The regex match is currently case insensitive. It shouldn't be.

https://github.com/opsdroid/opsdroid/blob/master/opsdroid/helper.py#L30

Document official skills, connectors and databases

All skills, connectors and databases under the opsdroid GitHub org should at least have a comprehensive README.

Would also be good to have tests.

Missing configuration file error too verbose

$ opsdroid
--- Logging error ---
Traceback (most recent call last):
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 980, in emit
    msg = self.format(record)
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 830, in format
    return fmt.format(record)
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 567, in format
    record.message = record.getMessage()
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 330, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/opt/boxen/homebrew/bin/opsdroid", line 11, in <module>
    sys.exit(main())
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/__main__.py", line 47, in main
    opsdroid.load()
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/core.py", line 82, in load
    "/etc/opsdroid/configuration.yaml"
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/loader.py", line 100, in load_config_file
    " not found", 1)
Message: 'Config file ./configuration.yaml not found'
Arguments: (1,)
--- Logging error ---
Traceback (most recent call last):
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 980, in emit
    msg = self.format(record)
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 830, in format
    return fmt.format(record)
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 567, in format
    record.message = record.getMessage()
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 330, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/opt/boxen/homebrew/bin/opsdroid", line 11, in <module>
    sys.exit(main())
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/__main__.py", line 47, in main
    opsdroid.load()
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/core.py", line 82, in load
    "/etc/opsdroid/configuration.yaml"
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/loader.py", line 100, in load_config_file
    " not found", 1)
Message: 'Config file ~/.opsdroid/configuration.yaml not found'
Arguments: (1,)
--- Logging error ---
Traceback (most recent call last):
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 980, in emit
    msg = self.format(record)
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 830, in format
    return fmt.format(record)
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 567, in format
    record.message = record.getMessage()
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 330, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/opt/boxen/homebrew/bin/opsdroid", line 11, in <module>
    sys.exit(main())
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/__main__.py", line 47, in main
    opsdroid.load()
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/core.py", line 82, in load
    "/etc/opsdroid/configuration.yaml"
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/loader.py", line 100, in load_config_file
    " not found", 1)
Message: 'Config file /etc/opsdroid/configuration.yaml not found'
Arguments: (1,)
Error: No configuration files found

Skill setup should be optional

You might not want to have a setup function in your skill.

Add debug telemetry data

It would be useful for opsdroid to track some statistics such as messages parsed, average response time, etc.

This could be written to a text file periodically or even published via a web page like the nginx status page. A web status page would require #90 to implement Flask.

NameError when exiting with ctrl+c

opsdroid> ^C
Exception ignored in: <bound method Task.__del__ of <Task pending coro=<ConnectorShell.listen() running at /tmp/opsdroid/modules/modules/connector/shell/shell.py:63> wait_for=<Future pending cb=[Task._wakeup()]>>>
Traceback (most recent call last):
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/tasks.py", line 92, in __del__
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py", line 1119, in call_exception_handler
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1308, in error
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1415, in _log
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1425, in handle
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1487, in callHandlers
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 855, in handle
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1047, in emit
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1037, in _open
NameError: name 'open' is not defined
Exception ignored in: <bound method Task.__del__ of <Task pending coro=<parse_crontab() running at /Users/jacob/Projects/opsdroid/opsdroid/opsdroid/parsers/crontab.py:17> wait_for=<Future pending cb=[Task._wakeup()]>>>
Traceback (most recent call last):
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/tasks.py", line 92, in __del__
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py", line 1119, in call_exception_handler
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1308, in error
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1415, in _log
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1425, in handle
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1487, in callHandlers
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 855, in handle
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1047, in emit
  File "/opt/boxen/homebrew/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/logging/__init__.py", line 1037, in _open
NameError: name 'open' is not defined

aiohttp missing from requirements

$ opsdroid --gen-config > /tmp/opsconfig.yml
Traceback (most recent call last):
  File "/opt/boxen/homebrew/bin/opsdroid", line 7, in <module>
    from opsdroid.__main__ import main
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/__main__.py", line 8, in <module>
    from opsdroid.core import OpsDroid
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/core.py", line 13, in <module>
    from opsdroid.parsers.apiai import parse_apiai
  File "/opt/boxen/homebrew/lib/python3.5/site-packages/opsdroid/parsers/apiai.py", line 6, in <module>
    import aiohttp
ImportError: No module named 'aiohttp'

Specify requirements in a constant

It would be nice for skills to be able to specify their requirements in a constant as well as in a requirements.txt file.

Add "user is typing" support

For chat services which support displaying a "user is typing" message there should be a method available on the base connector class for triggering this.

This would be particularly useful when implemented with a typing delay (#65).

Check for base classes instead of name when importing connectors and databases

Currently you must give your connector or database class a specific name for them to be imported.

Now that these classes should inherit from the base classes we can test for isinstance(baseclass) instead.

Connector and Database base classes

Connector and Database module classes should inherit from some kind of base class with required methods defined to raise not implemented errors.

This would also mean that instead of checking the class name when importing the modules we can instead check whether the class inherits from the base class.

Change config reference

Config reference sections should be headers to make them readable in readthedocs.

http://opsdroid.readthedocs.io/en/dev/configuration-reference/#reference

Enable/Disable parsers on a skill by skill basis

Skills may be written with multiple parsers. Particularly when using an advances parser like api.ai, not all users will want to use this service so it is likely a regex fallback will be provided.

However this means for users who do enable api.ai this skill could potentially be called twice if the regex and api.ai both match.

It could be useful to enable/disable parsers on a skill by skill basis to avoid this.

Sync memory to database

When items are added to and retrieved from the memory they should be synced into a persistent database.

This requires a database module or two to be written and opsdroid.memory.Memory to be extended.

Default connector and default room

When writing a skill which originates from something other than a message (e.g cron #26) the response may need to know which room to post into.

Most chat clients have a default room, like #general in Slack. This could be available as a property in the connector so that skills can easily access it.

e.g

@non_message_decorator()
def myskill(opsdroid):
    for connector in opsdroid.connectors:
        message = Message("Message text", connector.default_room, None, connector)
        connector.respond(message)

It should also be possible to override the default room in the connector config.

connectors:
  slack:
    default-room: "#random"

Change default modules directory

Currently the default modules directory location is ./modules.

This makes a few assumptions:

• Current directory is in the python path
• There are no other python modules in the current directory
• There are no other modules named modules
• Current directory is writable

A better default location may be ~/.opsdroid/modules/opsdroid-modules. This would be created if it doesn't exist and ~/.opsdroid/modules could be added to the python path without fear of collision as opsdroid-modules is less generic. As it is in the home directory we can be fairly sure it is writable.

Also when a user specifies a custom modules directory it should still be suffixed with /opsdroid-modules and the custom directory should be added to the python path.