interaction-lab / harmoni Goto Github PK

View Code? Open in Web Editor NEW

9.0 3.0 6.0 57.86 MB

Controller code for Human And Robot Modular OpeN Interactions

License: MIT License

Shell 1.13% CMake 14.44% Python 42.19% HTML 0.29% JavaScript 41.84% CSS 0.11%

harmoni hri ros social-robots controller

harmoni's People

Stargazers

Watchers

Forkers

riya-ranjan micolspitale93 icra-2021 mustafakurban

harmoni's Issues

Actionlib Issues

There are some general issues with our current ActionClient and ActionServer implementations. Wait arguments don't seem to work correctly (wait for goal submit, wait for client setup) and this is a common warning:

Your executeCallback did not set the goal to a terminal status. This is a bug in your ActionServer implementation. Fix your code! For now, the ActionServer will set this goal to aborted

Some of this is likely due to threading, and perhaps part of it is due to SimpleActionClient being only a partial implementation.

Remove test:=true references

Currently our dockerfile setup_script has a number of aliases for our old way of testing. These should be updated to reflect current rostest usage. In general anything that mentions the rosparam test (usually test:=true) should be updated/fixed.

HARMONI/dockerfiles/config/setup_script.sh

Line 24 in 609f52a

    
           alias rlmultiplechoice="roslaunch harmoni_decision harmoni_decision.launch test:=true"

Injecting Secrets Into Containers

I think there's a better way to inject secrets into containers, but this works for now. Might be good to make an issue to look into it later.

Originally posted by @RMichaelSwan in #24 (comment)

Harmoni_microphone stop not working

The stop() (OFF actionType) is not working in the harmoni_microphone.

Combine Face and Web Packages

Right now the packages are doing the same thing for server setup. The restructure involves making it so that the server can start whichever html is requested by an interaction and then work with it. This avoids having repeated code while allowing for an increasing number of custom html pages.

Pytorch for ARM/face_detect

There are no official binaries for pytorch on ARM processors. This may change in the future, but presently it means you have to build from source. The relevant face detection containers will need to updated and built for ARM once this is working. Note that if the deployment environment is a ARM device with CUDA (I.e. an Nvidia Jetson), then Nvidia does provide pytorch binaries for that platform.

Details on how to do this or how to get existing builds people have uploaded can be found here:
https://nmilosev.svbtle.com/compling-arm-stuff-without-an-arm-board-build-pytorch-for-the-raspberry-pi
pytorch/pytorch#23233
pytorch/pytorch#13130

Proper Secret Handling for MongoDB

Need to grab secret credentials in a similar way as GCS and AWS credentials are grabbed so that they don't get accidentally committed (also a possible security issue in the future).

Offending code:

HARMONI/harmoni_core/harmoni_recorder/src/harmoni_recorder/template_storedata.py

Line 14 in 609f52a

    
           mdb_client = MongoDBClient(client_uri="mongodb://172.18.3.3:27017/", username="root", password="example")

Originally posted by @RMichaelSwan in #41 (comment)

Quickstart build harmoni core packages roscpp cmake error

When attempting step 3 (build harmoni core packages) of desktop setup quickstart, this error happens for harmoni_recorder, harmoni_common_lib, harmoni_pattern, and harmoni_decision:

CMake Error at /home/multiparty/catkin_ws/devel/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
  Could not find a package configuration file provided by "roscpp" with any
  of the following names:

    roscppConfig.cmake
    roscpp-config.cmake

  Add the installation prefix of "roscpp" to CMAKE_PREFIX_PATH or set
  "roscpp_DIR" to a directory containing one of the above files.  If "roscpp"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  CMakeLists.txt:10 (find_package)

This error happens for harmoni_common_msgs:

CMake Error at /home/multiparty/catkin_ws/devel/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
  Could not find a package configuration file provided by "rospy" with any of
  the following names:

    rospyConfig.cmake
    rospy-config.cmake

  Add the installation prefix of "rospy" to CMAKE_PREFIX_PATH or set
  "rospy_DIR" to a directory containing one of the above files.  If "rospy"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  CMakeLists.txt:10 (find_package)

Here's the summary:

[build] Summary: 1 of 6 packages succeeded.                                    
[build]   Ignored:   10 packages were skipped or are blacklisted.              
[build]   Warnings:  None.                                                     
[build]   Abandoned: None.                                                     
[build]   Failed:    5 packages failed.                                        
[build] Runtime: 1.0 seconds total.

Wave2Letter (W2L) not responding to WAV file input with any output

The current test of the TTS package with W2L does not output any result with the test audio file. This test can currently be found on the 'feature/testing-framework' branch

Sequential Pattern Testing

The sequential pattern test only tests a specific pattern right now, not the ability to run sequential patterns in general.

Long term I would want to make sure it runs correctly for any possible sequential sequence, not just testing some specific sequence.

Suggestion: Make a unit test for the sequential testing code which mocks all external interfaces and just tests the ability to go through sequences
Ensure the user test includes most modules in the system and is well documented so users know what to expect.

Originally posted by @RMichaelSwan in #41 (comment)

Cleanup Misc Tutorials

Ensure the content is available elsewhere or turn it into a proper tutorial.

Cannot play audio on dockerfiles

Command: aplay tts.wav

Error message is: ALSA lib pcm_dmix.c:1029: (snd_pcm_dmix_open) unable to open slave aplay: main:722: audio open error: No such file or directory

Handle the delayed rendering of webpage

Remove the rospy.sleep() before the request in the harmoni_web package

Global rosparam/namespace refactor

We are using global rosparams in many instances where we should be using rosparams which are relative to some namespace (nodes and groups can have their own namespaces in launch files).

Names that start with a "/" are global -- they are considered fully resolved. Global names should be avoided as much as possible as they limit code portability.

Namespace Reference
Useful overview of roslaunch structure recommendations

For example:
When services are instantiated they often have an /instance_id rosparam in their launch file or in their code. The problem is, if two services somehow launch at the same time, they could overwrite each other's rosparam value (race condition) resulting in possible instancing bugs.

Similarly, we are often setting custom parameter names for each service instead of using the most obvious name and making it relative to the namespace of that service.

Fixing this issue will require a refactor of many of the main() functions in nodes and their associated launch files. A correct example is provided in the harmoni_face_detector service. In addition, the use of a constants file for Namespaces should be refactored. The Namespaces can be stored in launch files or possibly a global launch include.

Camera test passes even when it fails

Camera test passed even when issuing the following error:

  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_sensors/harmoni_camera/nodes/camera_service.py", line 69, in start
    self._read_stream_and_publish()  # Start the camera service at the INIT
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_sensors/harmoni_camera/nodes/camera_service.py", line 124, in _read_stream_and_publish
    raise RuntimeError("No camera frame available. Is the configured device accessible?")
RuntimeError: No camera frame available. Is the configured device accessible?```

Preemption refactoring

Preemption issue originally Fixed in 59ad148 . Issue was with blocking processes and no threads to unblock them.
May want to do a full refactor to avoid use of preempt_cb. Discussion so far:

The HarmoniActionServer has a main thread (thread 1) which creates two threads--the first we explicitly create for the executeCallback (thread 2) and another new thread that seems to be created for regular callbacks (thread 3--this thread creation must occur in the actionlib code somewhere).

The service_server and its associated service live inside of the executeCallback thread (thread 2).

Any while loops in your services which are called in response to start, do, request, etc. will block the service_server from responding to state transitions until the while loop breaks. As long as there is a sleep in your while loop, data (e.g. state) can be changed by a separate thread which can be used as a loop break signal.

I implemented a workaround for the issue caused by previous point without rewriting existing loops. I used a preempt callback in the service_server that the ActionServer (thread 3) can call at any time. This callback updates the State to PAUSE, which the while loop checks for and breaks out of when seen.

We should refactor code which contains long-running while loops. The preempt workaround I have provided requires extra knowledge of classes a service creator shouldn't care about too much. Possible solutions include putting the service_server in a separate thread, making an obvious requirement for service creators to thread their loops, or using a timer thread to poll for goal updates

The current start_sending_feedback() function found in the service_server is another a blocking function; in the microphone_service case it is called on the main thread (thread 1) after the HarmoniActionServer initializes. This deserves a refactor--feedback should either be reported when changes occur (e.g. state changes) or it should have its own thread, assuming we want it to be a longrunning monitor process.

I think this is an old discussion, but I still feel the title of the HarmoniServiceManager service template is confusing. It's being used as a service template usually, so why not just call it that? e.g. Serviceor AbstractService. I just checked, and there is no name overlap with built-ins.

Chris: Awesome write up! I completely agree that preempting is not a good long term solution, unless it is implemented under the hood as part of the action_server.py - so that no one needs to know what it is doing. Taking a step back, I think we may want to discuss if we want services to be state machines - with a state first approach and their own threads, or functions - which the action server is calling upon. I am excited to discuss on Wednesday. I'll try and wrap up the rostests before than so maybe we can close this branch before we get too far into the rewrite.

Originally posted by @RMichaelSwan in #36 (comment)

Cannot build harmoni_w2l

When I tried to build the docker-compose-harmoni (not dev), I got the following error:

ImportError: No module named 'setuptools'
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-hvmdev3f/numpy/
You are using pip version 8.1.1, however version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
ERROR: Service 'harmoni_w2l' failed to build

Build error on QT NUC

Running docker-compose -f docker-compose-qt-nuc.yml up leads to this error:

Reading package lists...
W: GPG error: http://packages.ros.org/ros/ubuntu xenial InRelease: The following signatures were invalid: KEYEXPIRED 1622248854
W: The repository 'http://packages.ros.org/ros/ubuntu xenial InRelease' is not signed.
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  ros-kinetic-joint-state-publisher-gui
0 upgraded, 1 newly installed, 0 to remove and 387 not upgraded.
Need to get 11.9 kB of archives.
After this operation, 55.3 kB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
  ros-kinetic-joint-state-publisher-gui
E: There were unauthenticated packages and -y was used without --allow-unauthenticated
The command '/bin/sh -c APT_INSTALL="apt-get install -y --no-install-recommends" &&     apt-get update &&     DEBIAN_FRONTEND=noninteractive $APT_INSTALL     ros-kinetic-joint-state-publisher-gui     && rm -rf -- /var/lib/apt/lists/*' returned a non-zero code: 100
ERROR: Service 'harmoni_full' failed to build : Build failed

Gesture Test Failing

[ROSUNIT] Outputting test results to /root/.ros/test_results/harmoni_gesture/rostest-test_gesture.xml
[INFO] [1621439158.279799]: Initializing the gesture service
[INFO] [1621439158.284864]: Setting up the gesture
[INFO] [1621439158.787637]: Initializing the gesture_qt service
[INFO] [1621439159.718419]: Initializing the gesture service
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
Could not initialize OpenGL for RasterGLSurface, reverting to RasterSurface.
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
Traceback (most recent call last):
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_actuators/harmoni_gesture/src/harmoni_gesture/gesture_service.py", line 239, in <module>
    main()
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_actuators/harmoni_gesture/src/harmoni_gesture/gesture_service.py", line 230, in main
    s = GestureService(service_name, params)
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_actuators/harmoni_gesture/src/harmoni_gesture/gesture_service.py", line 48, in __init__
    self.setup_gesture()
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_actuators/harmoni_gesture/src/harmoni_gesture/gesture_service.py", line 79, in setup_gesture
    rospy.sleep(0.1)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/timer.py", line 134, in sleep
    rospy.rostime.wallsleep(duration)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/rostime.py", line 277, in wallsleep
    time.sleep(duration) 
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/core.py", line 504, in _ros_signal
    prev_handler(sig, stackframe)
TypeError: 'Handlers' object is not callable
[Testcase: testtest_gesture] ... ok

[ROSTEST]-----------------------------------------------------------------------

[harmoni_gesture.rosunit-test_gesture/test_request_response][ERROR]-------------
init_node interrupted before it could complete
  File "/usr/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/lib/python3.6/unittest/case.py", line 601, in run
    self.setUp()
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_actuators/harmoni_gesture/test/rostest_gesture.py", line 28, in setUp
    rospy.init_node("test_gesture", log_level=rospy.INFO)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/client.py", line 323, in init_node
    raise rospy.exceptions.ROSInitException("init_node interrupted before it could complete")
--------------------------------------------------------------------------------


SUMMARY
 * RESULT: FAIL
 * TESTS: 1
 * ERRORS: 1
 * FAILURES: 0

rostest log file is in /root/.ros/log/rostest-harmoni_full-3306.log

Launching Services Tutorial

The service launch documentation is currently a bit outdated. The aliases setup_script.sh aliases also need a rework based on recent changes.

If we are including these in the docs they should be tested to make sure they still work (I think some are outdated)

Originally posted by @chrismbirmingham in #46 (comment)

Better param loading

Replace self.X = param["X"] with:
for key in my_dict:
setattr(self, key, my_dict[key])
For all classes.

Autogenerating web configs for web interface structure

We might want to add an issue for to make a generator for these configs. It looks like it would be a bit of a pain to do by hand.
e.g. harmoni_actuators/harmoni_web/web/src/config/config.json

Individual package requirements.txt

Every package should have its own install requirements. These should be listed in a requirements.txt file at the root of every package. There may be more than one if different methods require different solutions (e.g. dlib will have different requirements than yoloface or opencv AI kit or mediapipe).

A first solution should document the requirements for all existing packages and a replace the instillation.sh bash script in the root folder with a script that calls these requirements.

Docker Refactoring

These are a few things I have noted that could use work as I was working with the containers. Feel free to add more.

Lightweight containers should work well from bash with no default terminator/x11 forwarding
Too many mountpoints! It's easier to maintain and scale the fewer there are. Can copy most files/scripts needed into container builds instead.
In general the dockerfiles and compose config feels too complicated to work with/maintain reliably. Need to simplify more.
All containers should have apt-get install iputils-ping net-tools for network debug
Containers should be based off of official ROS images instead of Ubuntu. This would make them a bit cleaner.
Lightweight: only install the minimum libraries needed for HARMONI core. If you need more, make a variant container or put it into dev containers (dev containers can be a mess as long as they don't require much maintenance).

.asound device config as environmental variable

.asoundrc is currently a mounted config file, it should be controlled by an environmental variable.

Face configuration documentation on QT

The documentation for configuring the face on a QT says to run
roslaunch harmoni_face face.launch

However, it should be
roslaunch harmoni_face face_service.launch

Additionally, the rosrun lines on both NUC and PI seem to be unnecessary to launch the face.

Provide roadmap

Add the current roadmap to the documentation in the overview section.

State Handling

In the abstract class we should declare the state. We will set it in the init, and give an example for how it should be set by each function in the template class.

We should update the documentation with what each state should mean and how it will be used.

Bot Setup Tutorial

Now HARMONI looks a really well-documented repo!!
Should we add also some quick tutorial to create a bot with Lex with the corresponding value of the configuration.yaml file? The same also for other service (e.g., Dialogflow) which required the interface with external service.

Thanks!

That tutorial would be awesome and highly useful, especially if it touched on a little of the setup on the Lex side too. If my time opens up towards the end of the week, I can maybe look into it, but help would be much appreciated.

Originally posted by @RMichaelSwan in #46 (comment)

Automating Test

We have a number of rostests and unit tests working now. The next step is to get these running automatically through some script and possibly github action.

Speaker setup on QT

The setup documentation says to edit the .asoundrc file on the NUC but it should actually be changed on the QTRP

Cordial Face Error

Issue with dialog on the refactor branch #20

There is an issue with cordial being split into eyes and mouth while the configuration is for one node. This should either be split in the configuration or handled separately somehow.

Traceback:
[INFO] [1599532561.234088]: Initializing the dialogue service Traceback (most recent call last): File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_core/harmoni_pattern/scripts/harmoni_pattern/sequential_pattern.py", line 340, in <module> main() File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_core/harmoni_pattern/scripts/harmoni_pattern/sequential_pattern.py", line 327, in main dp = SequentialPattern(pattern_name, script) File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_core/harmoni_pattern/scripts/harmoni_pattern/sequential_pattern.py", line 38, in __init__ self._setup_clients() File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_core/harmoni_pattern/scripts/harmoni_pattern/sequential_pattern.py", line 61, in _setup_clients ), f"Scripted service: {service}, has not been configured" AssertionError: Scripted service: harmoni_face_mouth_default, has not been configured [harmoni_pattern-1] process has died [pid 3498, exit code 1, cmd /root/harmoni_catkin_ws/src/HARMONI/harmoni_core/harmoni_pattern/scripts/harmoni_pattern/sequential_pattern.py false Hey default __name:=harmoni_pattern __log:=/root/.ros/log/9f768010-f17b-11ea-a428-0242ac120304/harmoni_pattern-1.log]. log file: /root/.ros/log/9f768010-f17b-11ea-a428-0242ac120304/harmoni_pattern-1*.log

Relative path in configuration.yaml files

Should be using a relative path if possible (maybe something like $(find harmoni_pattern) to get the base path?), though this may depend on how this file is read in.
e.g. in harmoni_core/harmoni_pattern/config/configuration.yaml
pattern_scripting: /root/harmoni_catkin_ws/src/HARMONI/harmoni_pattern/pattern_scripting/dialogue.json

rviz not included in current builds

[INFO] [1603080446.855135]: Silence threshold set to 5
[INFO] [1603080446.858381]: Sucessfully reached microphone_default
[INFO] [1603080446.858739]: Service Server microphone_default has been successfully set up
[INFO] [1603080446.872074]: HardwareControlServer start continuously updating the feedback
Failed to load Python extension for LZ4 support. LZ4 compression will not be available.
2020-10-18 21:07:27-0700 [-] WebSocketServerFactory starting on 9090
2020-10-18 21:07:27-0700 [-] Starting factory <autobahn.twisted.websocket.WebSocketServerFactory object at 0x7fa548dfaa90>
2020-10-18 21:07:27-0700 [-] [INFO] [1603080447.035514]: Rosbridge WebSocket server started at ws://0.0.0.0:9090
[INFO] [1603080447.203762]: Rosapi started
process[harmoni_speaker_default-6]: started with pid [210]
[INFO] [1603080447.796149]: Initializing the speaker_default service
[INFO] [1603080447.798399]: Sucessfully reached speaker_default
[INFO] [1603080447.798603]: Service Server speaker_default has been successfully set up
[INFO] [1603080447.811907]: HardwareControlServer start continuously updating the feedback
process[audio/audio_play-7]: started with pid [224]

** (audio_play:224): CRITICAL **: file gstaudioringbuffer.c: line 1971 (gst_audio_ring_buffer_set_channel_positions): should not be reached
process[harmoni_gesture_default-8]: started with pid [241]
[INFO] [1603080448.835374]: Initializing the gesture_default service
[INFO] [1603080448.841293]: Setting up the gesture_default
process[harmoni_gesture_qt_default-9]: started with pid [254]
[INFO] [1603080449.417943]: Initializing the gesture_default_qt service
process[joint_state_publisher-10]: started with pid [268]
process[robot_joint_state_publisher_default-11]: started with pid [297]
[INFO] [1603080450.573871]: Initializing the gesture_default_qt_simulator service
process[robot_state_publisher-12]: started with pid [306]
process[rviz-13]: started with pid [328]
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
Could not initialize OpenGL for RasterGLSurface, reverting to RasterSurface.
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
================================================================================REQUIRED process [rviz-13] has died!
process has died [pid 328, exit code -11, cmd /opt/ros/kinetic/lib/rviz/rviz -d /opt/ros/kinetic/share/urdf_tutorial/rviz/urdf.rviz __name:=rviz __log:=/root/.ros/log/8e9eaeba-11c0-11eb-adbe-0242ac120304/rviz-13.log].
log file: /root/.ros/log/8e9eaeba-11c0-11eb-adbe-0242ac120304/rviz-13*.log
Initiating shutdown!
================================================================================
[rviz-13] killing on exit
[robot_state_publisher-12] killing on exit
[robot_joint_state_publisher_default-11] killing on exit
[joint_state_publisher-10] killing on exit
[harmoni_gesture_qt_default-9] killing on exit
[harmoni_gesture_default-8] killing on exit
[audio/audio_play-7] killing on exit
[harmoni_speaker_default-6] killing on exit
[bridge_face/rosapi-5] killing on exit
[bridge_face/rosbridge_websocket-4] killing on exit
[harmoni_face_default-3] killing on exit
2020-10-18 21:07:31-0700 [autobahn.twisted.websocket.WebSocketServerFactory] (TCP Port 9090 Closed)
2020-10-18 21:07:31-0700 [-] Stopping factory <autobahn.twisted.websocket.WebSocketServerFactory object at 0x7fa548dfaa90>
2020-10-18 21:07:31-0700 [-] Main loop terminated.
[face_http_server-2] killing on exit
http-server stopped.
[harmoni_microphone_default-1] killing on exit
[harmoni_gesture_default-8] escalating to SIGTERM
Traceback (most recent call last):
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_actuators/harmoni_gesture/scripts/harmoni_gesture/gesture_service.py", line 179, in <module>
    main()
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_actuators/harmoni_gesture/scripts/harmoni_gesture/gesture_service.py", line 164, in main
    s = GestureService(service, param)
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_actuators/harmoni_gesture/scripts/harmoni_gesture/gesture_service.py", line 49, in __init__
    self.setup_gesture()
  File "/root/harmoni_catkin_ws/src/HARMONI/harmoni_actuators/harmoni_gesture/scripts/harmoni_gesture/gesture_service.py", line 73, in setup_gesture
    rospy.sleep(0.1)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/timer.py", line 134, in sleep
    rospy.rostime.wallsleep(duration)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/rostime.py", line 277, in wallsleep
    time.sleep(duration) 
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/core.py", line 504, in _ros_signal
    prev_handler(sig, stackframe)
TypeError: 'Handlers' object is not callable
shutting down processing monitor...
... shutting down processing monitor complete
done```

Improve Multi-PC tutorial.

I think the approaches below might need further explanation. This is actually a super important section for most use cases, and so we want to make it really clear. One thing that might not be immediately clear is what needs to be set where, and what needs to be run on which machine.

Originally posted by @chrismbirmingham in #46 (comment)

_node title on nodes unnecessary

Appending "_node" to the names of our nodes provides no further information to the developer. You will only see nodes if you run rosnode list nodes and topics are clearly differentiated in rqt_graph, and this is not convention for ROS, so I don't see the benefit.

If this issue is accepted, all rospy.init_node() calls that append the "_node" string would need to be modified.

Develop architecture skeleton.

Provide controller code that satisfies the HARMONI architecture.

STT with google is broken

Updating the microphone to continuous streaming broke the google stt service. This needs to be updated to the continuous model, but there is an issue with knowing when to return the spoken text. The local deepspeech stt model uses a fixed time cutoff (.5s) of waiting for new text before assuming the user is done, which may be appropriate.

harmoni_bot roslaunch wants test args

roslaunch harmoni_bot harmoni_service.launch -- launching the service requires a test args which we no longer set/use

Apt key needs updating (all images)

(On branch robot/qt testing the gesture package)

While attempting to download a ros package I ran into the following:


root@harmoni_full# apt-get update
Get:1 http://packages.ros.org/ros/ubuntu focal InRelease [4676 B]
Err:1 http://packages.ros.org/ros/ubuntu focal InRelease                                                                                                           
  The following signatures were invalid: EXPKEYSIG F42ED6FBAB17C654 Open Robotics <[email protected]>
Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]                                                                                
Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease                                                                                                   
Get:4 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal InRelease [18.1 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]                    
Get:6 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [778 kB]            
Get:7 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal/main amd64 Packages [21.9 kB]                          
Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [368 kB]
Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [925 kB]     
Get:10 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]                                    
Get:11 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [27.6 kB]       
Get:12 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [32.0 kB]                                  
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1361 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1040 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [416 kB]
Fetched 5320 kB in 2s (2702 kB/s)                         
Reading package lists... Done
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://packages.ros.org/ros/ubuntu focal InRelease: The following signatures were invalid: EXPKEYSIG F42ED6FBAB17C654 Open Robotics <[email protected]>
W: Failed to fetch http://packages.ros.org/ros/ubuntu/dists/focal/InRelease  The following signatures were invalid: EXPKEYSIG F42ED6FBAB17C654 Open Robotics <[email protected]>
W: Some index files failed to download. They have been ignored, or old ones used instead.

The following describes the cause of the issue and a solution worked but the images probably need to be rebuilt and pushed to dockerhub.

https://answers.ros.org/question/379190/apt-update-signatures-were-invalid-f42ed6fbab17c654/

While updating the key, ros-noetic-robot-state-publisher and ros-noetic-urdf-tutorial should also be installed

Standardize Readmes

We should standardize the readme format and the included information. A suggestion would be the following:

[Package Name]

[One - two sentence description of included capabilities]

Usage

Parameters

Testing

References

[References to where the package came from]
[References to our documentation]

Cannot run speech-to-text service

When I tried to launch the stt service, testing it with the audio file recorded from the microphone (harmoni_stt/temp_data/test.wav) with the following command in the ros_w2l terminal (after running the bash get_w2l_model.sh as well):
roslaunch harmoni_stt stt_service.launch test:=true

I got this error:
'Started features model file loading ... \n' b"terminate called after throwing an instance of 'std::runtime_error'\n what(): failed to open feature file=/root/model/feature_extractor.bin for reading\nAborted (core dumped)\

Re-test swarm networking approach and provide instructions/compose files.

I have tried to use the swarm approach also for Misty, but I was not able to make it working (I don't know why), so I went for the host network approach. I think it is worth to try the swarm approach again instead.

Originally posted by @micolspitale93 in #46 (comment)