philass/harkdb

Multi-backend GPU query engine written with Futhark

harkdb's Issues

Pre-Load table into GPU memory

Currently we Load the data from tables into the futhark context only after processing a query. This may be inefficient if loading this table into context is a lot more expensive than loading the columns to be selected and other things.

This needs to be tested to see if their are performance improvements if the implementation is changed to address the problem with these considerations in mind.

Files that need to be changed in this case

• CLI.cpp
• db_gpu_load.cpp

Join last line is slow

The last line of futhark/join.fut is inefficient

in transpose (concat (transpose t1s) (transpose t2s))

This likely should be an easy fix with a map of maps.

Join Sort does redundant work

the join implementation in futhark/join.fut does redudant sorting.

Ideally we would sort once when we combine the tables. In this implementation we would simply need to first sort by the column value being merged on and than by table.

Currently we use partition in the generate_pair function. This would not be neccesary if the lists were pre sorted. However this extra work may be insignificant. (need to test this).