ogallagher/marketsense

Leverage the human neural network to predict market asset values

marketsense's Issues

Controls to show/hide sample market data graph

Depending on the training theory, one may or may not want to be able to see the sample’s market data (ex. prices) graph during training, in addition to the sound and color.

Support for alternative training procedures

Currently there’s only one type of training session, which selects fully random samples, and the user guesses the mapped color after listening to the sample however many times they wish. The procedure that a training session follows should be an option in the dashboard’s new session form.

An example of an alternative training procedure:

• select all random samples and calculate their future movements beforehand
• sort the samples by their future movements
• play back each of the samples in order, with the true color displayed, and without the user having to guess what the color should be

This future-movement-sorted-samples training procedure would reinforce the notion of certain sounds seeming more red and others seeming more green, by grouping like colored sounds together.

Create audio synthesizer

This tutorial uses Java Sound to synthesize via custom sound waves (as opposed to MIDI with predefined instruments).

Line anything through which a stream of audio data passes.

Mixer a collection of input and output lines.

Clip as opposed to an audio stream (SourceDataLine), a clip contains a fully buffered sound.

timbre the voicing/instrument/shape of a pitch.

sample unit of audio, mono or stereo.

sample rate number of samples per second (how exactly a timbre can be defined).

Enforce session sample size

As mentioned here, it is becoming a problem that the estimated interval size needed to fetch the desired number of trade bars for a sample during training is not enough, as it doesn’t account for closed exchange hours or weekends.

MarketSample sound doesn’t include first data point

I’ve discovered the generated sound is not including the first data point when creating training market samples.

Update config.properties at runtime to use latest session params

Fetch market data for training session market samples

This involves being able to detect whether the market data universe for the given training session, and fetching from the twelvedata API to insert it if not.

For any bar width, there will be no holes within an interval once the market data is retrieved from the web, so the missing data will be all prior to what the database has, all after, or (rarely) both.

Use Java Persistence API for db storage

Java Persistence API (JPA), via javax.persistence.

Derby database for Java via Java Database Connectivity (JDBC).

Include people, lessons, and saved market data. Audio will be stored in files.

Market sample data viewer, 2d line graph

Create a market sample data viewer (price/time graph) to help validate program accuracy and provide additional info about the sample. Include the start and end datetimes.

Start by attempting to use fxgraph as a canvas-based graphing library.

View performance over time charts

The only view of performance over time as of now is the training session history in the dashboard, which shows all sessions for the given user. A separate view for showing session scores history over time with filters for time, accounts, symbols, etc would graph different performance metrics over time for easier analysis and comparison.

Show future movement in visual market data graph

Show the future movement in the visual market data graph when the color guess is submitted and the answer revealed.

Create test session, share superclass w train session

I’m not sure how important a separate testing session would be, as it would still be backtesting like training, only that it would be used to create update your definitive score for that security overall, and maybe require a higher sample count.

Select whether to save generated sounds in dashboard

Convert initial thoughts

My initial thoughts are written on printer paper, and should be converted into proper documentation.

Expand market data to timbre mapping formula

Now that marketsense has a basic synthesizer I’ve been able to listen to some samples generated so far and am not yet satisfied with how distinguishable each generated timbre is, despite visible changes in waveform. I believe this could be improved by using one or both of the following:

1. Normalized market data (prices) are used as multipliers with sections of the amplitude within a period, and there’s no interpolation between each section within the period. Perhaps smooth interpolation between price multipliers will create less identically artificial-sounding tones.
2. Part of timbre is influenced by overtones, which are defined as higher frequency component waves within the fundamental frequency, having lesser amplitude. I could try composing a unique timbre by creating overtones for each point in the market data, perhaps constraining them to be within harmonics, using Fourier synthesis.

Update future movement formula

The current delta range formula normalizes the future's difference with the last price against the range of consecutive differences in the sample. This could be considered flawed because the sign (<0.5 vs >0.5) of futureMovement no longer necessarily reflects the sign of future - last, if the direction within the sample is very uniform.

See below for an example of a misleading future movement with the current formula:

Training session sample size not persisting in config.properties

Improve layout

Rework the dashboard and training session scenes so that everything is cleaner.

Console window toolbar button in main window

Reset true color tile on next training sample

Force use of own api key

Use dummy properties file in git, and real properties file in local env.

Create writable config.properties from config_dummy.properties if necessary

This will prevent having to modify the contents of the jar file to include a config.properties renamed from config_dummy.properties.

SecurityId composite key column not working with persistence

I believe I unwittingly solved this issue when adding default no-arg constructors to the entity classes involved.

Create live session

A live session, as opposed to a training session, uses only the latest possible sample with a generated sound. The color guess widget then is mapped to the estimated future price, but the true future is not known so no score is calculated.

Show training session history in dashboard

There is currently an empty list view for showing training session history, but it’s not populated with the trainingsession db table yet.

Catch api key errors when fetching market data

I believe this is code 401 in Twelvedata.

Create db persistence tests for securities, trade bars, and training sessions

So far the database stores people, but I’ll be creating tests to ensure Security, TradeBar, and TrainingSession entities are properly stored and able to be retrieved directly and with joins.

Ensure bar size != 1day works

I’ve only tested with 1-day trade bars for market samples so far, but want to support other bar sizes.

Performance graph controls and canvas interaction

• show/hide intervals
• show/hide mean line
• show score on hover
• disable scroll zoom on graph canvas

Perform and save a basic training session

As of now, the gui dashboard screen has a New Training Session form to select a config (ex. stock symbol, prices per sample, number of samples, etc) and then launch a new training session. The corresponding persist-compatible TrainingSession is then instantiated, but there is no training interface yet, and no update to the database.

Save performance view rows on view exit

Don’t refresh the performance view interface every time the user returns to it.

Calculate session score confidence interval

Currently, a TrainingSession has a single score, which is the average of accuracy scores for each market sample guessed vs actual future movement. This score would be more accurately represented as a confidence interval, given the mean score, standard deviation, and a desired confidence level to display.

API key input within program gui

• #24
• when handling an incorrect/unspecified market data api key, display an api key input field
• use the entered api key to update the TwelvedataClient instance

Add instructions for use and development to readme

• written user instructions
• create a demonstration video and link to readme
• write some basic development directions