pothosware/pothos-library

Framework library used by all Pothos projects

pothos-library's Issues

Proxy Objects could use a set/getattr for dealing with fields.

This is an enhancement to make the proxy abstraction more complete to support languages that have class fields. C++, python, and java have both static and non-static fields, so this should be valuable.

Places this could help:

• Currently there is code that calls the python built-in getattr to handle field access
• Some python Block class wrappers convert a field-based api to managed function calls -- but in this case the original C++ class used fields.

api to query NUMA information

managed class register baseclass calls

Currently, the managed class registrations dont allow base class members to be registered because the function arg0 type is not a match (its the base class and not the derived class).

Consider the following:

• specify the baseclass in the registration and automatically suck-in base class methods
• specify that the method is on the baseclass, and let the registration template foo handle it
• make the template registration foo always provide a cast or conversion or wrapper (this will nicely not compile if the cast isnt possible)
• others?

scheduler runtime JIT compilation support

Phase out the parametrized scheduler runtime, for JIT compiled runtime that is customized per thread for a specific configuration of ports, blocks, and kernels.

remote topology support

Just like blocks, topology should support connecting remote topologies (used in hierarchies of sub topologies).

server spawning + connection is slow

Something unknown is making spawning a process, running a server, and connecting to it take a noticeable amount of time, on the order of 100s of ms.

msvc use precompiled headers in client module builds

API to configure an upstream block's managed buffer

This feature enables scheduler DMA for the input side of a block.

topology connect functional markup

Consider supporting a markup for the connect arguments:
topology->connect(src, "src port name", dst, "dst port name")

The string arguments represent port names, but they could represent ranges or grouping of ports to be more useful.

• a range of ports 0 to max
• all signals, all slots
• a list of port names
• others?

Further, consider hierarchies of topologies. A connection markup that simplifies the burden of connecting all available ports could give us the following automatic behaviours: A block internal to the hierarchy gets a new signal or slot, or perhaps as variable number of IO ports... this would be automatically connected to the boundaries of the container topology without any additional code intervention, just a one time markup.

log errors throw by calls to work()

• we should not be relying on print

Move tests/* into library for builtin self test

There are only 3 or 4 tests in this directory. This removes boost unit test semi-dependency.

api to configure thread pools

hangup issue with self test and log level setting

FIXME - POTHOS_LOG_LEVEL=information bin/PothosUtil --self-test1 something seems to have a hangup/freeze issue

module loader safe load should cache result

This would speed up module loading as there is no need to retry the safe load if we have a previous confirmation.

• probably store the result in a config file
• use the module path as a key, and the timestamps on the library and module as a check

managed class registration can throw at init time

We should be catching and logging these errors. Cannot throw at static init time..

ProxyEnviroment factory - return Proxy vs ProxyEnviroment?

Think about proxy find environment returning a special proxy with the same method as ProxyEnvironment. This would make the abstraction more complete and simplify some implementation and user code.

Dont configure ports with a type, elements are dyanmic --

Incoming buffer chunks are dynamic from run to run based on the dtype associated with them. The output port type is defined by the dtype that the user posts to the output.

Stats collection API - JSON stats dump through Topology

• Block API call to query WorkStats (thread safe copy)
• improve work stats to include per port info, anything relevant
• Work stats for hierarchies?

earlier automatic port creation

In order to support domain resolution and automatic ports, consider a different way to automatically create indexed ports. As of now, the domain checking would access an automatic port that does not exist.

Also, this enhancement would suffer in a similar way: #41

create block yield call

Yield tells the scheduler that we (a block) are yielding the thread context, but we are not done processing and expect to be called again.

• Use this in blocks that for example, did not produce/consume within the time limit

proxy ambiguous overload

    static std::string superStringFcn(const std::string &s)
    {
        return s;
    }

     superFooProxy.call<std::string>("superStringFcn", std::string("test"));

Its ambiguous with the call that has a return type vs the one with a void return

error: call of overloaded ‘call(const char [15], std::string)’ is ambiguous

InputPort API optional policy for message queue depth

For example, drop oldest or newest message when the queue reaches max size. The queue can be:

• POLICY_INFINITE
• POLICY_DROP_OLDEST
• POLICY_DROP_NEWEST

block notification overload for port change

When the ports are changed at runtime, the author of a block with variable ports will overload this call to to make sure they are using viable port pointers. This will also be the callback that we recommend users use to cache/store port pointers.

question the efficacy of RemoteNode

Is there any value of RemoteNode class and static node table outside of the gui? If not, probably just move the functionality into the gui.

InputPort API to optional clip upper bounds of numElements()

The intention is to allow fine grained control over how many elements a worker can process without yielding output.

test name resolution when the network is down

test network resolution when the network manager (ubuntu) is down

• specifically in the case of the proxy unit tests
• if problem try passive resolution

Build doxygen documentation for include/*

InputPort API element preload option

Element preload enables:

• History stuffing
• Feedback loops

signals and slots for blocks in a topology

Signals and slots will allow for complex control interactions between blocks, built on top on a simple subscriber mechanism.

• Blocks already have a "slots" registry, that is simply the call registry.
• Blocks need to register "signals" that can be emitted
• Topology needs a connect for signals to slots
  • this must work across nodes, so topology will infer sockets
• Signaler WorkerActor needs a subscription list of actor addresses and slot names

late creation of the block's actor

blocks should be more lightweight, we dont need the actor to be active during constructor stuff or initialization/configuration calls from the user. The actor can be allocated in the appropriate thread pool before activate() is called.

• One motivation for this is that blocks are allocated simply to inspect their ports and data types in the GUI.
• Another is based around theron frameworks, where once an actor is made it cant be migrated to a different thread pool. So make the actor late, after thread pool affinities are determined.

Proxy over network needs a heartbeat

Proxy over network needs a heartbeat with timeout so we can clear expired objects on the server if a sudden disconnection occurs.

register workeractor message types in theron

• this should be a performance improvement because it uses a smaller table vs the RTTI system

modify the comment headers of hpp files for doxygen

The current comment headers for header files arent being picked up by doxgen. Look into an altered format to get the comments in doxygen.

pothos server stdio nonbuffered mode

The server can backup and die if the caller process isnt reading stdout/stderr.

• One solution is to set unbuffered mode in the server
• Other solution, remote/server.cpp needs to read

api calls to post stuff to block's input

post buffers, labels, messages to the input of a block in a threadsafe way from outside the topology. We should be able to post to the actor easily, just a question of usefulness.

pothos server redirect stdout from process to logger

node needs to redirect server stdio so it can be retrieved and printed

• redirect stdio in server to poco logging
• make this a configuration option

document block descriptor markup

• document block descriptor comment markup
• document block descriptor JSON schema

Put documentation on http://pothosware.github.io/pothos/index.html

push to gh-pages branch on pothos.git

post work bump() logic

When a work session completes, we need to choose when to bump the work handler again:

• always bump for source blocks
• bump after consumption/production
• bump after outputport popbuffer
• bump after inputport setreserve

a more complete CPU info query

sigar win32 cpu info broken - use GetLogicalProcessorInformationEx

We need to drop sigar all together and make an abstraction for a general system processor (multicore/numa) query. Sigar is overkill for what we need, and its too buggy.

Block API to set an useful string to identify the block

Block.hpp needs an API to set and get name to identify the block in runtime error messages.

call serialization for a type without containerizing it

serialization - figure out how to call a serialization for a type within the serialization call. This way we can nest without exporting anything. So far, we can just put stuff to serialize in a Pothos::Object

challenge: pass a remote proxy as an agument to a proxy function expecting a proxy

That's a mouthful, but this kind of meta-shit it nice when it works.

So suppose we have a remote proxy pRemote, and a local proxy pLocal. pLocal has a method that accepts a proxy object void foo(const Pothos::Proxy &); We want to pass pRemote into PLocal without screwing anything up. Currently, this does not work, some kind of conversion may be trying to take place. Investigate...

• Also consider when pLocal isn’t local, we we establish a new proxy handle in an arbitrary environment for this same object?

use NUMA allocation for specific affinity conditions

When the source and destination are affinitized to the same NUMA node,
use the buffer manager args to specify the node number.

block registry topology support

• register topology factories into the block registry
• retrieve and instantiate topologies from the registry

numerics to and from string formats

• formats for complex numbers
• formats for vectors or even matrices and numbers

think about re-entrant call interface for block

This saves a lockup when a handled call calls back into the call interface.
In this case, same thread context, dont need to message the actor,
just call the call handler directly,

worker actor port subscribe - use ptr for subscription

should just give the upstream/downstream block the pointer itself to forgo the lookup based on index/name (particularly name)

init code development environment sanity check

In the Pothos::Init(), confirm the existence of library files and headers (actual hpp files)

Object intermediate conversions

Object intermediate conversions. To help with the NxM problem for integers and containers of integers:

• create a registry that can recommend an intermediate type. TypeA -> I -> TypeB
• --OR-- allow intermediate conversions when a limited number of steps make that final conversion possible