Storing OPC UA Server Configuration

[freeopcua]

Storing OPC UA Server Configuration

[oroulet]

Interesting question. I always use opc-ua to represent data from an underlying system, thus I do not want to store anything. At start I query the underlying system to be sure to publish up-to date information. But I can imagine that for a very slow underlying system, it might be interesting to save state. InternalServer (available as self.iserver on Server class has a method dump_address_space and load_address_space. they may solve your issue directly. But they dump the entire address space. you may want to only dump selected nodes....
I do not know your application but I think I would prefer to store data internally in my application and then simply write a method to set value of opc-ua node at startup... Where is the drawback of this?

[zerox1212]

With your idea wouldn't you have to manage your nodes in two locations? Your application has to know about the nodes to set the values, and your OPC UA server has to have the nodes existing.

Or is your application always creating the OPC UA nodes every time it starts? Or if the OPC UA server restarts it also has to check for the nodes and then create them again?

I wanted to add/remove nodes (I've been calling them "tags") by only editing the XML file. When I finish my application I don't want to have to edit python code to add a new tag.

Currently my application works like this:
Server starts
Server creates all nodes from XML files
Server code creates python objects (class) with the same structure as OPC UA object
All the python objects which mirror the OPC UA objects are stored in a "tags" dict
Application code interacts with the python objects internally as it's much simpler than using all the built in python OPC UA methods to do what I need to

I have no clue if this is a bad way to do it, I'm not much of a programmer. Ultimately I just wanted to add or remove tags/nodes by just editing the XML file. I suppose my python object class which mirrors the OPC UA object node could save the data, but I don't know how to do that in python yet.

I'm also adding a built in "simulation driver" into the OPC UA server like TOP Server has, in which saving the nodes with the server itself makes sense. I suppose this goes into another topic on how to make a "driver" for the OPC UA server, for example with Python Modbus-tk to bridge modbus to OPC, etc.

[zerox1212]

Maybe if I show you what I made it will be helpful. I'm open to any advice as I'm new to Python and OPC UA.

My OPC UA object structure: Every "tag" has these variables:

This is my class which matches the OPC UA configuration. For every object imported by the XML an instance of this class is instantiated. My OPC UA server then creates a thread after it starts which calls the update.() method on every tag in the dictionary in a forever loop.

class DataObject(object):
    """
    Definition of OPC UA object which will hold data
    """
    def __init__(self, child_list):
        self.Value = 0.0
        self.Mode = 0
        self.DataSource = ""
        self.DataAddress = ""
        self.SimLowLimit = 0.0
        self.SimHighLimit = 0.0

        for _child in child_list:
            _child_name = _child.get_browse_name()
            if _child_name.Name == "Value":
                self.ValueN = _child
            elif _child_name.Name == "Mode":
                self.ModeN = _child
            elif _child_name.Name == "DataSource":
                self.DataSourceN = _child
            elif _child_name.Name == "DataAddress":
                self.DataAddressN = _child
            elif _child_name.Name == "SimLowLimit":
                self.SimLowLimitN = _child
            elif _child_name.Name == "SimHighLimit":
                self.SimHighLimitN = _child
            else:
                pass  # should throw error that a child node for the object didn't get handled

        self.Value = self.ValueN.get_value()
        self.Mode = self.ModeN.get_value()
        self.DataSource = self.DataSourceN.get_value()
        self.DataAddress = self.DataAddressN.get_value()
        self.SimLowLimit = self.SimLowLimitN.get_value()
        self.SimHighLimit = self.SimHighLimitN.get_value()

    def update(self):
        self.Mode = self.ModeN.get_value()
        self.DataSource = self.DataSourceN.get_value()
        self.DataAddress = self.DataAddressN.get_value()
        self.SimLowLimit = self.SimLowLimitN.get_value()
        self.SimHighLimit = self.SimHighLimitN.get_value()

        # Mode=0 is disabled/readonly
        # Mode=1 is normal: get data from configured external data source such as Modbus RTU, etc.
        # Mode=2 is simulation: the tag.Value will be set by the OPC UA server based on the configured limits
        if self.Mode == 0:  # use this mode as "read only" for now; may need to add a separate mode later
            pass
        elif self.Mode == 1:
            if self.DataSource is not None:
                self.ValueN.set_value(99.1)  # value gets set from configured data source here, such as a Modbus Slave
        elif self.Mode == 2:
            self.ValueN.set_value(random.uniform(self.SimLowLimit, self.SimHighLimit))

        # at the end of update() call update the objects .Value tag with value from OPC UA
        self.Value = self.ValueN.get_value()

My analytics application only ever reads the data from .Value. All the parameters for the "tag" are edited from outside via the SCADA (or other OPC UA client to the server).

As you can see my "Mode 1" part isn't finished, but I want to create drivers in this spot, so my OPC UA server can get data for the object from modbus or even another OPC server via a client. In a way it lets me configure the OPC UA server from an OPC UA client.

[oroulet]

I am not sure I understand everything but let's try ...
Anyway in an opc-ua app you will not escape the need to map internal data or underlying data to opc-ua structure. this is by design.
In your case it looks like you you have many values set by external application through opc-ua. I think I would use the built-in subscribe system, then you do not even need a thread.

class DataObject:
    def __init__(self, child_list):
        self.Value = 0.0
        self.Mode = 0
        self.DataSource = ""
        self.DataAddress = ""
        self.SimLowLimit = 0.0
        self.SimHighLimit = 0.0

       class Handler:
             # do not do any network call here!!!
            def data_change(node, val, xxxxxx):
                     if node == alue_node: # or child_list["Value"] or whaterver, I do not know you app
                              self.ValueN = val
                   elif _node_name == "Mode":
                         self.ModeN = val

      sub = uaserver.create_subscription(handler)
      uaserver.subscribe_data_change(child_list)

[zerox1212]

Thank you very much for taking the time to help me.

I will take a look at the method you describe. However I think I will still need a thread to "bridge" data from my Modbus RTU interface to the OPC UA server. Looks like node.set_value(modbus_value).

I will also take a look at how the dump_address_space and load_address_space code works so that I can save the state of my server when it shuts down / starts up.

[zerox1212]

I just updated to the latest release to test what we talked about in this issue.

Now I notice the server.dump_aspace method has been removed. Why?

[zerox1212]

I finally tested the address space dump/load. Unfortunately it doesn't work.

Calling dump_address_space only works BEFORE calling server.start(). Even if you call server.stop() you can't dump the address space due to TypeErrors.

TypeError: cannot serialize '_io.TextIOWrapper' object

This makes the function rather useless for saving the address space, since you can't do it after the nodes changed during run time.

EDIT: Correction, dump_address_space works after server.start() ONLY if there are no clients connected to the server. How can I make sure the server has no client connections (or force close the connections) to make sure I can back up my address space?

[oroulet]

How can I make sure the server has no client connections

close the socket.. .but server.stop() should close the socket. maybe add a sleep and try again?

[zerox1212]

I tried a 2 second sleep, same error. Let me try again with a longer sleep.

It seems like whatever prints

WARNING:opcua.binary_server_asyncio:processor returned False, we close connection from ('127.0.0.1', 56554)
INFO:opcua.binary_server_asyncio:Lost connection from ('127.0.0.1', 56554), None
Cleanup client connection:  ('127.0.0.1', 56554)

lets it work.

[zerox1212]

I tried sleeping, but it doesn't work either.

    server.stop()
    time.sleep(20)
    server.dump_aspace("address_space.bin")

Still get:
TypeError: cannot serialize '_io.TextIOWrapper' object

[zerox1212]

Do you have any advice for troubleshooting this? I would really like to use this function, but I don't know how to debug it because the _nodes object is too big for the pycharm debugger. I can't see where this _io.TextIOWrapper object is.

[oroulet]

Will try to test dump method. By I really do no get why you need such a thing, see earlier post

[zerox1212]

I guess I need it because I built my application wrong... I suppose. My application get's all it's variables from the OPC UA server (via XML import) instead of the other way around. The approach you speak of for my application would make all the variables in some other way, then publish them to OPC UA.

Unfortunately I don't have time to rewrite the application at this time, so the dump method is a decent fix to make all the variables persistent.

Thanks very much for your help, and sorry about the confusion. I'm still very new to object oriented programming and OPC UA.

[oroulet]

I looked at this socket issue and found out we were not closing socket, only shutting it down. Maybe this helps? a11e0ab

[zerox1212]

I will test it. Thank you very much.

[zerox1212]

So I tried to dig into this more... maybe you guys can teach me about asyncio. I would really appreciate it.

I see that the asyncio server is being closed explicitly in binary_server_asyncio.py:

    def stop(self):
        self.logger.info("Closing asyncio socket server")
        self.loop.call_soon(self._server.close)
        self.loop.run_coro_and_wait(self._server.wait_closed())

However, in the python documents, it also says that you should close any transports associated with asyncio explicitly.
https://docs.python.org/3/library/asyncio-dev.html#close-transports-and-event-loops

It seems like stopping the binary server should also close the transport explicitly, but I'm not really sure how to do this. This seems to be the reason why a_space_dump fails. I've tried every way I can think of to call OPCUAProtocol.transport.close(), but it never works. It looks like it can't be referenced from the stop function, because OPCUAProtocol is local to start.

Secondly, I have to say I don't really understand this at all:

            def connection_lost(self, ex):
                self.logger.info('Lost connection from %s, %s', self.peername, ex)
                self.transport.close()
                self.processor.close()

Function connection_lost is overriding the asyncio BaseProtocol here, but the documents say that calling transport.close() also calls connection_lost, and it does, I tried it. Python must be saving us from getting stuck in here somehow...
https://docs.python.org/3/library/asyncio-protocol.html#basetransport
How is that working?

Sorry again, I really don't know much at all about asyncio. Thanks for all the help.

[zerox1212]

So I got a_space_dump to work after calling server.stop() through a hack. I added this to the OPCUAProtocol class.

            def _process_data(self, data):
                buf = ua.utils.Buffer(data)
                while True:
                    if self.iserver.close_transport is True:
                        print("force close OPCUA Protocol transport object")
                        self.transport.close()

Then I added this to the stop() function.

    def stop(self):
        self.logger.info("Closing asyncio socket server")
        self.iserver.close_transport = True

This means if you call server.stop() and clients are connected, it will force them close and call connection_lost. Which calls close() on the uaprocessor. After that you can dump address space no problem.

What is the best way to close OPCUAProtocol transport explicitly @oroulet ?

[oroulet]

I do not know what is the best practice here. The asyncio socket examples do not seem to close transport.
Send pull request for your change. Maybe without option close_transport option. then we will test it a bit more.

[zerox1212]

OPCUAProtocol class is local to start() function. If I remove close_transport there is no way for stop() to do anything with OPCUAProtocol.

Do you have a link to the asyncio socket examples?

[oroulet]

https://docs.python.org/3/library/asyncio-stream.html#tcp-echo-server-using-streams

nothing more

[zerox1212]

I'm going to close this issue and open a new one about address space dumping for clarity.

[su600]

WARNING:opcua.binary_server_asyncio:processor returned False, we close connection from ('127.0.0.1', 56554)
INFO:opcua.binary_server_asyncio:Lost connection from ('127.0.0.1', 56554), None
Cleanup client connection:  ('127.0.0.1', 56554)

@zerox1212 Hi, I am facing the same error INFO as you mentioned here

opcua.binary_server_asyncio:processor returned False, we close connection ...

This happens when I use Prosys OPC UA Client try to browse the information model by NoSecurity mode(server-minimal.py) . By using Sign & SignAndEncrypt mode is fine, and UaExpert is fine without problems.

Could you please help me figure out what does this mean and what should I do to fix this?

Thank you!