SPADE 3.0 Roadmap

[javipalanca]

SPADE 3.0 Roadmap

[javipalanca]

Dear Piao,
The documentation you refer to is about the next big release of SPADE (3.0) which is about to be released in a few days. Version 3.0 is still not published to PyPi since we are still fixing some minor bugs and adding some improvements.

If you can't wait to the release you can work with the "develop" branch, where the work in progress of version 3.0 is being committed.

As soon as version 3.0 is released I will notify you all about it.

Best,
Javi

[javipalanca]

Hi everyone,
I'm happy to announce that SPADE 3 is finally released :)

There is some work that it is still remaining (like transforming the deprecated modules to plugins) but there is at least a functional version.

My next goals are to create as much plugins as needed:

• BDI
• P2P
• Logic
• PubSub
• etc.

So your help and PRs are really welcomed 👍

Javi

[javipalanca]

Congratulations Javi!

Thanks a lot! Glad to read you!

I will try to contribute knowledge bases and organizational features as
soon as possible ;-) What would be a desired way to do that, using plug-ins?

Thanks again. Your contributions are so important. The idea is to have a core SPADE as small as possible, and then add the rest of features as plugins.
I'm still thinking about the best way to do it. I've written a vague guide where I try to define the ways to contribute plugins to SPADE (https://spade-mas.readthedocs.io/en/latest/extending.html), but I agree that there's a lot of room for improvement.

I'll try ASAP to develop some example plugins in order to validate this plugin mechanisms or not.
Of course, I'm glad to receive contributions and suggestions about this! :)

[mschatten]

Thanks Javi, great initiative!

I'm looking forward to contribute to a new version of SPADE, and switching to Py3 sounds fun.

Regarding the wishlist:

• not sure about FIPA, the initiative is pretty much dead in industry, but still I don't know what would be the alternative? I like the FIPA ACL standard (it has some important features if you ask me)...
• I like XMPP, but it would be nice if the underlaying transport protocol could be anything (e.g. web sockets, http, Zigbee, Z-Wave ...). To implement an abstract (high-level) agent protocol (like FIPA ACL, but not neccesarily) and add arbitrary low-level transport protocols (that might be abstracted by a class that has to be inherited and provide important methods like auth, send, recieve etc.). So, contributors can add any protocol they need...
• behavioral agent model is fine with me (maybe add a number of other generic behavior types, extend event behavior with other events than messages ...).
• consider to integrate normative agents
• better implementation of BDI (e.g. integration of knowledge bases regardless of underlaying platform, implementation of generic planning systems etc.)
• organizational aspects - not only structure, but process, strategy, culture, dynamics and interorganizational features would be great!

So much from me ;-)

M.

[psukys]

I would like to second on @mschatten's 2nd bullet point - moving away from a specific transport protocol and implement ACL or something similar.

[javipalanca]

I agree that abstracting the transport layer sounds interesting. However this would be a big change of direction in the SPADE development. Some years ago I chose XMPP because of its interesting communicating features: to make agents chat as humans do (great for Human-Agent interactions. e.g. I can chat with my agents from my instant messaging software or even from facebook). Also because of its Presence Notification protocol, which I think is amazing and completely innovative for agents, and all the XEPs (XMPP Extensions) that provide agents a rich set of features (e.g. SPADE currently uses presence notification, multi-user messaging, PubSub Events, XMLRPC service invocation, etc)
On the other hand, server-based protocols are a little bit tedious (although xmpp implements server2server like smtp).
SPADE currently supports other protocols (like http) by means of plugins.
The main decision should be to go to simple messaging protocols (no presence notification, no extensions) in order to be able to use any transport layer, or to take advantage of the xmpp features (and all its extensions) but conditioning the communication with other protocols to the use plugins and the use of a decentralized server-based protocol.

I think that both options have advantages and disadvantages. This is a big decision. Feel free to express your opinion.

[psukys]

Can't xmpp's features be as an optional extension?

[javipalanca]

They can, of course. That's why I ask which kind of platform do we want. A JADE clone? or something slightly different? (but even JADE has a default transport protocol, JMS, and a plugin based system).
We could select a default transport protocol too, and in the event of using others as plugins it would be optional to implement those features in that protocol if possible.

[javipalanca]

I would also like to have security in mind from the beginning.

[ebegoli]

Hi Javi,

Just for your consideration - there are some really nice networking/async libraries for Python that offer amazing peer-to-peer and peer-to-server performance. Examples are Twisted and Tornado.

Twisted:
https://twistedmatrix.com/trac/

Tornado:
http://www.tornadoweb.org/en/stable/

They could be useful as a foundations for the new networking layer for Spade. Amazing throughput and performance, yet easy to use as one would expect from well designed Python frameworks.

Regards,
Edmon

[filipesaraiva]

Hello, I just would like to say I want to help in this new release. For now I will follow the discuss. Thanks and good work for us!

[javipalanca]

Thanks @ebegoli, both are great async libraries with an event-loop (and more things), but having in mind to port to python 3, it seems reasonable to use the new asyncio implementation which looks promising.

[javipalanca]

@filipesaraiva Great! It's nice to have you in the team ;)

[mschatten]

Dear all,

great to see so much interest in SPADE ;-)

@javipalanca regarding your comments, I think most of these features (like presence notification, multi-user messaging and direct communication with the agent) can be implemented on a higher level. It would be more work of course, but the benefits of having an abstract transport layer are tremendeous in my opinion. Consider the various devices that are out there currently (buzzword Internet of Things) that could run local copies of spade agents on them (mobile phones, googles, home automation devices, UAVs, various gaming platforms, car computers etc.) that could be able to use their native protocols to implement advanced agents. Also, consider web and mobile apps that could act as frontends for agents with web sockets. There would be lots of new possibilities, all from one platform. Also, it opens a potentially big chance for community involvement and contribution from people who are interested in developing underlaying transport protocol classes.

Consider also a problem with using only XMPP which I faced multiple times: whenever there is a native protocol for some situation, you have to duplicate communication. E.g. in order for SPADE agents to communicate with other processes running on various devices they have to implement their native protocols anyway and then translate messages from one protocol to the other since XMPP cannot be used in most cases.

I would propose to use XMPP as the default protocol, but implement it in a way that would allow for simple adding of other protocols where needed. The idea would be to implement an higher-level (abstract) transport protocol class that gets as an argument a concrete transport protocol class (e.g. XMPP, HTTP, web sockets, etc.) with well defined interfaces (e.g. methods and atributes that have to be implemented to be used and optional features that can be implemented if possible). The higher-level class could then implement various advanced features which you mention or use underlaying services (from XMPP for example) if they are available.

A few other bullets I'd like to add to my wishlist is:

• better debuging capabilities (e.g. the current threading implementation does in most cases catch all errors somewhere and doesn't show any info about what happened, the agent just hangs).
• better documentation from the start (e.g. detailed API reference, use cases/tutorials etc.)

[ebegoli]

@javipalanca - absolutely. I did not even thing about asyncio.
@mschatten - Brilliant! I am all in to help with Internet of Things R&D aspects. Sign me up, coach :-)

[javipalanca]

Regarding protocols, layers and Internet of Things: we can separate the transport layer from the protocol. Being XMPP the protocol, it can be used via HTTP (see XEP-0124 http://xmpp.org/extensions/xep-0124.html). It can encapsulate SOAP (XEP-0072 http://xmpp.org/extensions/xep-0072.html) and it's even prepared for the Internet of Things (lots of XEPs: http://xmpp.org/extensions/xep-0347.html, http://xmpp.org/extensions/xep-0323.html, http://xmpp.org/extensions/xep-0324.html, http://xmpp.org/extensions/xep-0325.html and http://xmpp.org/extensions/xep-0326.html).
I think that other options that were previously commented (like using websockets) are transport layers, and there are lots of xmpp client libraries in lots of languages (http://xmpp.org/xmpp-software/libraries/) like javascript, node.js, python (of course), java, C, ruby, php, etc..., that can address xmpp over almost any transport layer.

Just like FIPA proposes HTTP, WAP or IIOP, XMPP has its own proposals. My proposal of using XMPP as a protocol is more oriented to substitute FIPA than HTTP or other abstract protocols (that could work as transport layers for xmpp). Of course, a compatibility layer for FIPA should be great, but I think that XMPP provides better features for agent communication than FIPA (which, btw, seems to be abandoned).

I do not want myself to be obsessed with this topic. It's just that I had the idea that SPADE was not going to be "just" a message sender/receiver, but having more interesting features. :)

Would suggest RabbitMQ (http://www.rabbitmq.com/getstarted.html) for this topic as a candidate to replace the XMPP server. It was suggested to me at a time when I mentioned the problems within SPADE, relatively to the XMPP server. I am unable to detail the platform for now because the platform is completely unknown to me.

[halflings]

Just another recommendation for a low-level transport layer: ZeroMQ (http://zeromq.org/) sounds like a good choice since it lets you do more than just pub/sub. Otherwise RabbitMQ could also work, but models everything as pub/sub.

[atiq-cs]

I don't know much about spade. But I wanted to run this spade with python3.4.3. What I basically tried to do is naive.

1. I ran python's 2to3.py on every python file in the source directory using a script
2. Running
   python setup.py installI
   I got indentation inconsistency errors in some files. So I manually fixed them.

But now, I'm getting

F:\Sourcecodes\python\spade\SPADE-2.2.1>python setup.py install
Could not import spade package!!! No module named 'debug'
Traceback (most recent call last):
  File "setup.py", line 8, in <module>
    from runspade import __version__
  File "F:\Sourcecodes\python\spade\SPADE-2.2.1\runspade.py", line 19, in <modul
e>
    from xmppd.xmppd import Server
  File "F:\Sourcecodes\python\spade\SPADE-2.2.1\xmppd\__init__.py", line 1, in <
module>
    from . import xmppd
  File "F:\Sourcecodes\python\spade\SPADE-2.2.1\xmppd\xmppd.py", line 32, in <mo
dule>
    from xmpp import *
  File "F:\Sourcecodes\python\spade\SPADE-2.2.1\xmpp\__init__.py", line 29, in <
module>
    from . import simplexml,protocol,debug,auth,transports,roster,dispatcher,fea
tures,browser,filetransfer,commands
  File "F:\Sourcecodes\python\spade\SPADE-2.2.1\xmpp\auth.py", line 23, in <modu
le>
    from .client import PlugIn
  File "F:\Sourcecodes\python\spade\SPADE-2.2.1\xmpp\client.py", line 25, in <mo
dule>
    import debug
ImportError: No module named 'debug'

Guys please help me out. Commit is here: https://github.com/sacsdu/spade/commit/91b26fc9817c7010080855b29e2b007069be82ae Any suggestion/fork/commit to fix this is welcome.

[unimagine]

Hi, I'm very interested in spade as I would like to incorporate it as a component in a personal project. The scope of the project is quite big (predictive demand locations based on finite resources using bayesian methods), and am developing in py3. Are there any current plans to release a py3 based version of Spade? I'm quite excited about the prospect of incorporating an agent-based aspect to my project, but I'm hoping that someday Spade will be released in py3, otherwise I will look to other options. Any idea how things are progressing, if at all?

[unimagine]

Still keen. I'm tempted to backtrack code to python 2, but as everything in Python is on the cusp at the moment, I feel that would be a mistake. Still holding out for this wonderful piece of software so I can play with some ideas. I know you guys are probably busy doing other life things, so I'll just keep waiting and working on other aspects. Thanks for putting the Spade2 out, if nothing else, I can work with that.

[Martschink]

I'm curious about how ontologies will work in practice. I understand the reason for their inclusion and the challenge for which they are offered as a remedy. At the outset, clearly defining the meaning of every possible word is something of a fool's errand, a fact that keeps lawyers wealthy and miserable. Also their implementation (required?) is not without burden or risk. I'm concerned that a hard rule that agents don't hear messages that don't have an exact match of the ontology identifier could actually impair the system.

[darithleng]

What do you think about improve the SPADE WUI as real-time monitoring on SPADE 3.0?
From agent, it read-time data from the system via RS-232 or CAN-Bus and that data can monitor on WUI from anywhere.

[mmealling]

The last comment from @javipalanca was from 2014 and the last commit to master was in 2013. General development of XMPP infrastructure essentially stopped a few years earlier than that. Its now 2017 and I'm trying to decide between JADEX, SPADE, and MESA and none are winning.

I'm seriously thinking of shoving SPADE's BDI library and HTTP transport into MESA and using that.

Anyone have any advice?

[mschatten]

Hello Michael, sorry for the slow response, but it was a busy week. What kind of applications are you trying to implement? Since MESA is a simulation platform, and SPADE and JADEX are development platforms... I do not know much about MESA's internals, so correct me if I'm wrong, but usually simulation platforms are not suited for development and vice versa. Regarding further development of SPADE, yeah, mostly the problem is to find time to work on it... We still use SPADE heavily in our projects and classes, but no one really has the time to rewrite the XMPP infrastructure... All the best, M.

…

-- Markus Schatten, PhD Assistant professor and head of Artificial Intelligence Lab University of Zagreb Faculty of Organization and Informatics Pavlinska 2, 42000 Varazdin, Croatia http://www.foi.hr/nastavnici/schatten.markus/index.html http://www.researchgate.net/profile/Markus_Schatten1 http://ai.foi.hr

On Sun, Apr 30, 2017 at 10:27 PM, Michael Mealling ***@***.*** > wrote: The last comment from @javipalanca <https://github.com/javipalanca> was from 2014 and the last commit to master was in 2013. General development of XMPP infrastructure essentially stopped a few years earlier than that. Its now 2017 and I'm trying to decide between JADEX, SPADE, and MESA and none are winning. I'm seriously thinking of shoving SPADE's BDI library and HTTP transport into MESA and using that. Anyone have any advice? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACzXpMyB71ryxd2Nyoi4OQXTx8LATYVjks5r1O6kgaJpZM4CzV0G> .

[mmealling]

I'm building an agent based simulation of a cislunar supply chain using concepts from here:
https://engineering.wayne.edu/faculty/govindu-_dissertation.pdf

It is increasingly difficult to find Java developers so I would like to move away from JADEX and its complexities to something like Python. I will be deploying this on AWS so my thought is to replace the XMPP infrastructure with SQS queues.

[javipalanca]

I had some really bad experience with queue-based protocols (AMQP) and my feelings were that this is not a really good solution for real-time instant messaging.
I'm currently reading about asyncio and having a look at https://github.com/horazont/aioxmpp which looks promising. However, there's still a lack of a server, something like aioxmppd would be really great.

[Bazmundi]

How about integration of Profeta as the BDI agent engine? Maybe morphing it towards ASTRA?

[Bazmundi]

Python version of EIS perhaps?

[Bazmundi]

So, having accidently come across Pulsar for Python 3, I am now, again, quizzical as to whether SPADE is Agent or Actor oriented. Sure enough, message passing using FIPA leans one towards Agents but the cognition side ... is Pulsar plus Profeta all we need?

[L-Piao]

Dear Javi,
Dear all,

I am a beginner with SPADE and I was confused with its compatibility with Python 3.

I started with several documentations of SPADE shown below. In such docs, it's mentioned that SPADE supports Python >=3.6.
https://spade-mas.readthedocs.io/en/feature-3.0/
https://media.readthedocs.org/pdf/spade-mas/feature-3.0/spade-mas.pdf

However, when I download and install SPADE (the latest version I've found is 2.3.3) it won't run in Python 3.6 but only in 2.7 environment. Have I understood it correctly, that SPADE 2.3.3 is still based on Python 2, and the new SPADE 3.0.0 is based on Python 3.6 and will appear soon? Thanks for clarifications!

Best Regards,
Piao

[L-Piao]

Thanks Javi! I downloaded and installed the "develop" branch. A small issue is the "pyasn1".

_removing 'c:\program files (x86)\python36-32\lib\site-packages\pyasn1_modules-0.2.2-py3.6.egg' (and everything under it)
Copying pyasn1_modules-0.2.2-py3.6.egg to c:\program files (x86)\python36-32\lib\site-packages
pyasn1-modules 0.2.2 is already the active version in easy-install.pth

Installed c:\program files (x86)\python36-32\lib\site-packages\pyasn1_modules-0.2.2-py3.6.egg
error: The 'pyasn1' distribution was not found and is required by aioxmpp_

Anyway it's easily solved by "pip install pyasn1". I will report issues while playing with the new SPADE 3.0.0. Thanks for your effort in developing ; )

Best,
Piao

[filipesaraiva]

@javipalanca on #15 (comment):

Hi everyone,
I'm happy to announce that SPADE 3 is finally released :)

Congratulations Javi, it is an amazing checkpoint for Spade development!

There is some work that it is still remaining (like transforming the deprecated modules to plugins) but there is at least a functional version.

My next goals are to create as much plugins as needed:

Are you plan create new modules or re-use some existent projects?

[javipalanca]

Hi,
of course the most desirable thing is not to reinvent the wheel and to use what is already done.
The extensibility of SPADE 3 is designed to be as easy as posible to add new plugins.
Next big thing I want to do is to create some of them.

[mschatten]

Congratulations Javi! I will try to contribute knowledge bases and organizational features as soon as possible ;-) What would be a desired way to do that, using plug-ins? All the best, M. -- Markus Schatten, PhD Associate professor and head of Artificial Intelligence Lab University of Zagreb Faculty of Organization and Informatics Pavlinska 2, 42000 Varazdin, Croatia http://www.foi.hr/nastavnici/schatten.markus/index.html http://www.researchgate.net/profile/Markus_Schatten1 http://ai.foi.hr

…

On Mon, Sep 10, 2018 at 12:36 PM Javi Palanca ***@***.***> wrote: Hi, of course the most desirable thing is not to reinvent the wheel and to use what is already done. The extensibility of SPADE 3 is designed to be as easy as posible to add new plugins. Next big thing I want to do is to create some of them. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACzXpMcH5jq1wAD59xOi4zP0OACQTju4ks5uZkCggaJpZM4CzV0G> .