The purpose of this project is to create a virtual assistant to assist SPARC members. SPARC stands for Student Projects and Research Committee and is a student led organization in the Department of Electrical Engineering at Auburn University. The virtual assistant will be fully featured including abilities to help in the lab to automating a home. It will include a system that allows for easy to use management of lab equipment and various administrative features. The virtual assistant could also be integrated on a mobile robotic platform to allow more capabilities that a static computing platform could. Our current iteration of the virtual assistant is named Karen.
This project is programmed in Python. Python provides a simple yet powerful back end to run the assistant. The assistant can be run through several clients as outlined below to adapt to the various use cases. Suites are used to manage the intelligence of the assistant and can be turned on and off to allow flexibility. User commands are passed in from the clients through a suite manager python file. The suite managers loop through all of the suites to find the appropriate response and returns it back to the client. The suites communicate with a natural language processor named DialogFlow as described in more detail below.
- Voice Client - Uses a microphone to take input from the user and speaks response back to user.
- Text Client - Uses the terminal to allow user to type commands and receive responses.
- Slack Client - Uses Slack messaging app to allow SPARC members to type commands and receive responses.
- Device Client - Communicates with the other clients for device control on less powerful computing platforms.
- Default - A collection of basic responses.
- Lab - A collection of responses and functions pertaining to assisting users in the SPARC lab.
- Home - A collection of responses and functions pertaining to home automation and light control.
- Weather - A collection of responses and functions pertaining to processing weather commands.
- Personal - A collection of functions pertaining to personal commands like reminders, time, and alarms.
- Knowledge - A collection of functions to search Google, Bing, Wolfram Alpha, Wikipedia, or etc.
- Fallback - A collection of fallback responses if previous suites did not catch.
To help give the virtual assistant the ability to understand the English language, we are using a Google owned service called DialogFlow. It is a natural language processor in which it deconstructs user inputs into intents (meaning) and entities (specifics). Through customization and training, a machine learning algorithm can be tuned to allow the assistant to more intelligently communicate.
The DialogFlow documentation walks through an example of how it works: https://dialogflow.com/docs/getting-started/basics
PyCharm is a fully featured free Python IDE. It provides a powerful development studio for Python code. It can run Python programs in the same windows as the code is developed and integrates with git to allow a seemless experience. Another nice feature of PyCharm is the intelligent refactoring so that variables, classes, and even file names can be efficiently changed throughout the entire program.
- Install PyCharm Community Edition: https://www.jetbrains.com/pycharm/download/#section=windows
- Install Python 2.7.13: https://www.python.org/downloads/release/python-2713/. (Make sure that add to path is checked on Windows)
- Install git bash https://git-for-windows.github.io/.
- Clone directory into desired location using
git clone "https://github.com/SPARC-Auburn/Lab-Assistant"in git bash. - In PyCharm, press
File > New Project... - Press "..." button and navigate to cloned directory. Press "OK" and "Create"
- Press
File > Settings... > Project: Lab-Assistant > Project Interpreter - Change the Project Interpreter to be
Python27/python.exe - Press the green "+" button and add any Python libraries that are missing and press "OK".
- Plug in USB mic if computer does not have an integrated microphone.
- In the tree on the left, press
voice_karen.pyunder the assistant folder. - Start the voice client by running
python voice_client.pyby right clicking in the center of the screen and pressingRun voice_client.py - Karen should now respond to your requests. (How are you?, Tell me a joke., etc.)
- If you need to stop the program press the red square in the toolbar.
- Make sure installation is updated by running the following commands (This may take a long while):
sudo apt-get update sudo apt-get upgrade sudo -H pip install --upgrade pip - Clone directory into desired location using
git clone "https://github.com/SPARC-Auburn/Lab-Assistant" - Run installation script using:
chmod +x ./install.sh sudo -H ./install.sh - Change directory to
assistant. - Plug in USB mic if computer does not have an integrated microphone.
- Start the voice client by running
python voice_client.py - Karen should now respond to your requests. ("How are you?, Tell me a joke., etc.)
- If you need to stop the program use "CTRL-SHIFT-".
Please see the CONTRIBUTING.md document in this repository for recommendations on how to contribute.
Suites are the best way to add your own, custom functions into the Assistant framework. There are only a few requirements for suites to be imported and used:
- There must be a class that is the same as the file name in camel case notation. (ie. defaultsuite - class DefaultSuite)
- The class must have an init and checkcommand() function. Use defaultsuite.py as a template.
- Replace the "CLIENT_ACCESS_TOKEN" with the CLIENT_ACCESS_TOKEN from DialogFlow's corresponding agent.
- Add additional recognition under the elif statement.
To hook your method into the program, import the suite in manage_suites.py like
from suites.newsuite import *. Add your suite to the suites array to be included in the processing of commands.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent