hmcalister/hopfield-network-go

An implementation of the Hopfield Network in Golang.

hopfield-network-go's Issues

Add logging to network relaxation methods

Ensure that quitting the program (e.g. CTRL+C) saves the data files

Remove trials from simulation

This is usually handled by bash scripts now anyway

Add functionality for multi-unit updates

Currently, the Hopfield network updates one unit at a time (the default behavior). It would be interesting to add functionality to update more than one unit per matrix multiplication.

Delta learning rule takes a long time

Profiling shows that the delta learning rule is taking a very long time - mostly due to serial relaxation of states! This should be shifted to concurrent relaxation.

Fix activation timing

Currently the activation function is applied before setting the unit in the state vector. This means activations that use the state vector will be incorrectly calculated.

Remove State Aggregation Data Collector

This collector doesn't serve much purpose with the new networkSummary collector...

Add command line flag to set asymmetric weight matrix

Change random distributions in main to be seeded with values

Fix chunking function to actually chunk correctly

Currently the chunking function does not chunk correctly, as it iterates up one element at a time. This results in many chunks having the same elements, which is not ideal!

The fix here is to change the chunkingFunction for loop increment from i++ to i+=chunkSize

Flag Cleanup

Currently a lot of network behavior is tied up in messy constant definitions. It would be better to make everything into a flag for command line argument parsing.

Abstract functions to add data handlers

Data handlers currently all define a new function on a DataCollector to add an instance of the handler. It would be better to have an abstract AddDataHandler function that takes an instantiated instance of a type.

Clean up documentation

Fix chunking implementation to only matrix multiply / step network on each chunk, not each index

A new step is still taken with each new index, not each chunk as is expected.

Revert Continuous Activation Fn Changes

Let's add these when we need to, and avoid getting too silly too quickly...

Refactor data collector

Fix bug where NaNs appear in states and unit energies

It appears that occasionally (actually very often) NaNs will appear in the states and unit energies of relaxing states when using ConcurrentRelaxStates. This should be considered a major bug, and needs to be addressed very quickly.

Remove network domain dependence

Network domain can be altered in the source code itself in future, as it would be a sufficiently large enough change. I am making a model not an enterprise solution!

Add code profiling

Add data collection for network summary

Fix bug in concurrent updates where unstable states return true

Add new activation function

Including, say, the norm activation for both Euclidian and Infinite norm

Allow for data loading, saving

Change random matrix initialization to use small values

Currently the random matrix initialization uses a standard normalization. This means the random starting values could be significant (e.g. >1) . Changing the sigma in this call to something smaller would have the same effect but less impact on learning.

Implement a better data collection framework

Currently, data collection occurs in the main method and is changed as the experiment requires. It would be nice to have an easier method to select data collection. For example, having a DataCollector struct that can be built to required specifics (and hence easily specified) and have callbacks in the HopfieldNetwork methods.

Data writer often corrupts file

For some reason, the data file tends to sometimes corrupt data - either writing incorrect number of rows into the file or missing data in some columns compared to others.

Remove Trial Index from data collector structs

Trial index is currently left over in the data collector structs. Because trials are no longer supported in the simulation directly, it would be best to remove trial index from these structs

Improve Docstrings

Currently, Docstrings use an ugly syntax looking like

• argName: ...

It would be much nicer to simplify these. Perhaps to something like:

argName: ...

Activation functions currently do not enforce symmetric units

The Hopfield network works best when states are equal to their inverse. This can be implemented by reversing states with a low-value first unit (forcing the first unit to always be the high value) and altering all other units to match, or by breaking the symmetry with a bias unit.

Add methods to collect relaxation histories

Allow easy change between chunking and non-chunking updates

Chunking may result in poor performance for, e.g., chunkSize = 1. In this case we don't even need to chunk, just iterate over the original list!!

It would be nice to have a special case for chunkSize = 1 that does not require the (relatively) expensive chunking function.

Allow for multiple noise application method implementations and easy selection

Add epochs for learning rules

Allow learning rules to run for multiple epochs

Fix bug where state history array length is off by one for states that do not stabilize before maximum iterations

Add abstraction to data collection and shift specific implementation to main method

Make number of trials a command line arg

Make activation functions faster

Keyboard Interrupt not working as intended.

Currently, a keyboard interrupt is only detected at the start of a new trial. It would be nice to have the interrupt occur at ANY point throughout the program

Implement chunking function

Implement a chunking function that converts a slice into multiple slices of a specified length.

This will be required by #7

Add Learning Rules

Implement the learning rules, like Hebbian and Delta

Have a flag or function to produce only unique states

Currently the generated state function does not care if the generated state is unique with respect to previously generated states. It would be nice to ensure the generated states are unique.

Use compressed data format rather than csv

Make noise proportional to network weights

Makes threads used for relaxation a command line arg

Data Collector does not work in threads

Shift dataCollector to use threads and channels to avoid go concurrent map accesses

Add noise options to network

Data file is often truncated and some data left out

This effectively corrupts the data file.

Implement logging framework

Logging can and will be useful for this project. It is the easiest way to identify and diagnose bugs.

It appears that the log package is very intuitive...

Implement DataHandler for target states

Collect data on target states directly, rather than relying on finding states during relaxation

Give state relaxation a more descriptive return value

Currently, the returned type of the relaxation functions is a simple bool (stable or not). It would be better to measure other features (such as number of iterations to stabilize).