aika-algorithm/aika

AIKA is a new type of artificial neural network designed to more closely mimic the behavior of a biological brain and to bridge the gap to classical AI. A key design decision in the Aika network is to conceptually separate the activations from their neurons, meaning that there are two separate graphs. One graph consisting of neurons and synapses representing the knowledge the network has already acquired and another graph consisting of activations and links describing the information the network was able to infer about a concrete input data set. There is a one-to-many relation between the neurons and the activations. For example, there might be a neuron representing a word or a specific meaning of a word, but there might be several activations of this neuron, each representing an occurrence of this word within the input data set. A consequence of this decision is that we have to give up on the idea of a fixed layered topology for the network, since the sequence in which the activations are fired depends on the input data set. Within the activation network, each activation is grounded within the input data set, even if there are several activations in between. This means links between activations serve two purposes. On the one hand, they are used to sum up the synapse weights and, on the other hand they propagate the identity to higher level activations.

Project status

During the past one and a half years there have been many changes to the architecture of the core algorithm. Most notably the introduction of binding-signals as the underlying mechanism for the linking process and the introduction of updatable fields which allow for a more descriptive way to implement the mathematics within the project. These changes are still work-in-progress, but will hopefully soon converge to a more stable state.

About the AIKA Neural Network

AIKA (Artificial Intelligence for Knowledge Acquisition) is a new type of artificial neural network designed to mimic the behavior of a biological brain more closely and to bridge the gap to classical AI. A key design decision in the AIKA network is to conceptually separate the activations from their neurons, meaning that there are two separate graphs. One graph consisting of neurons and synapses representing the knowledge the network has already acquired and another graph consisting of activations and links describing the information the network was able to infer about a concrete input data set. There is a one-to-many relation between the neurons and the activations. For example, there might be a neuron representing a word or a specific meaning of a word, but there might be several activations of this neuron, each representing an occurrence of this word within the input data set. A consequence of this decision is that we must give up on the idea of a fixed layered topology for the network, since the sequence in which the activations are fired depends on the input data set. Within the activation network, each activation is grounded within the input data set, even if there are several activations in between. This means links between activations server multiple purposes:

• They propagate the activation value.
• They propagate the binding-signal, that is used for the linking process.
• They establish an approximate causal relation through the fired timestamps of their input and output activations.
• They allow the training gradient to be propagated backwards.
• Negative feedback links create mutually exclusive branches within the activations network.
• Positive feedback links allow the binding neurons of a pattern neuron ensemble to support each other, by feeding the activation value of the patten neuron back to its input binding-neurons.

The AIKA network uses four different types of neurons:

• Pattern-Neurons (PN)
• Binding-Neurons (BN)
• Inhibitory-Neurons (IN)
• Category-Neurons (CN)

The pattern-neurons and the binding-neurons are both conjunctive in nature while the inhibitory-neurons and the category-neurons are disjunctive. The binding-neurons are kind of the glue code of the whole network. On the one hand, they bind the input-features of a pattern to the pattern-neuron and on the other hand receive negative feedback synapses from the inhibitory neurons which allow them to either be suppressed by an opposing pattern or allow themselves to suppress another conflicting pattern. Like the neuron types there are also several types of synapses, depending on which types of neurons they connect. For example, the input synapses of an inhibitory neuron are always linked to binding-neurons, while the input synapses of category-neurons are always linked to pattern-neurons.

The following types of synapses exist within the AIKA network:

• PrimaryInputSynapse ((PN|CN) -> BN)
• RelatedInputSynapse (BN -> BN)
• SamePatternSynapse (BN -> BN)
• PositiveFeedbackSynapse (PN -> BN)
• NegativeFeedbackSynapse (IN -> BN)
• ReversePatternSynapse (BN -> BN)
• PatternSynapse (BN -> PN)
• CategorySynapse (PN -> CN)
• InhibitorySynapse (BN -> IN)

The binding-signal that is propagated along linked synapses carries a state consisting of either of these three values: SAME, INPUT, BRANCH

SAME indicates that the binding signal has not yet left its originating neuron pattern ensemble and is used to bind the binding-activations to the current pattern. INPUT indicates that the binding signal was propagated to a dependant pattern neuron ensemble, for instance through the PrimaryInputSynapse or the RelatedInputSynapse. BRANCH indicates, that the binding signal originated from a binding activation instead of a pattern activation.

As already mentioned, the binding-neurons of a pattern neuron ensemble are used to bind this pattern to its input features. To verify that all the input-features occurred in the correct relation to each other the SamePatternSynapse is used. The SamePatternSynapse connects two binding-neurons within the same pattern neuron ensemble. The SamePatternSynapse connects two binding-neurons within the same pattern neuron ensemble and is only linked both ends of the synapse have been reached by the same binding-signal. Therefore, the SamePatternSynapse is used to avoid what is called the superposition catastrophe.

Since the category-neuron passes on the binding-signal of its input pattern-neuron, it can act as a category slot, therefore allowing the network great flexibility in abstracting concepts.

Initially, the network starts out empty and is then gradually populated during training. The induction of new neurons and synapses is guided by a network of template neurons and synapses.

aika's Issues

How to use

Quick question how do you use this? I cant seem to figure it out and theres no documentation coming with the code.

Build tests failing with mvn clean test

Hi,

As commented on pull request #1, my environment seems to have problems running the tests. I am on branch master.

commit b7ab9fdbc6f1955fc9d6246cce49916bae51b527
Author: Lukas Molzberger <[email protected]>
Date:   Fri Jun 16 13:01:00 2017 +0200

    Organize imports

Running: mvn clean test with:

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T05:41:47+13:00)
Maven home: /opt/maven
Java version: 1.8.0_131, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-oracle/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-79-generic", arch: "amd64", family: "unix"

Which is basically Ubuntu LTS with Oracle JDK 1.8. It fails with the following errors:

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.096 sec - in org.aika.network.SimplePatternMatchingTest

Results :

Failed tests: 
  PublicRangeTest.testPublicRange:101 null
  RecurrentNodeTest.testE:131 null
  RecurrentNodeTest.testFork:65 null
  RecurrentNodeTest.testOutputNode:200 null
  RecurrentNodeTest.testReverseDirection:166 null
Tests in error: 
  RecurrentNodeTest.testOutputNodeRD:272 NullPointer
  RecurrentNodeTest.testOverlappingClockAndTermSignals:341 NullPointer
  RecurrentNodeTest.testStartNode:396 NullPointer

Tests run: 61, Failures: 5, Errors: 3, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.672 s
[INFO] Finished at: 2017-06-16T23:38:45+12:00
[INFO] Final Memory: 19M/251M

There are a few places that inspecting the code in Eclipse, the IDE had a warning about possible NPE. It seems to happen indeed in some places in the log output.

testOverlappingClockAndTermSignals(org.aika.network.RecurrentNodeTest)  Time elapsed: 0 sec  <<< ERROR!
java.lang.NullPointerException: null
	at org.aika.network.RecurrentNodeTest.testOverlappingClockAndTermSignals(RecurrentNodeTest.java:341)

Optimizing Node.linkNeuronRelations

The Node.linkNeuronRelations method is called on every newly activated neuron to create the activation links to other activated neurons. This is done in both directions input and output. The problem here is that the number of incoming and outgoing synapses can be very large and the method will have to iterate over all of them to see if the linked neuron is activated. This works very well for a small number of synapses, but if the number is very large, it might be better to iterate over all activated neurons and see if there is a synapse between them. There is already a list of activated neurons in the Iteration object, but I don't think that it can be used here. It only contains activated neurons that are part of the final interpretation. This optimization, however, needs to iterate over all activated neurons. Therefore, another set of activated neurons is needed here.

Python

Please implement the code also in the python language.