Performance Slowdowns

[malloydelacroix]

Performance Slowdowns

[MalloyDelacroix]

The performance difference between those two commits is because of the introduction of rate limiters for extractors. I was beginning to see a lot of rate limit error responses from servers, so this was intended to fix this. I will look at reworking this system.

A SQLite storage system is in the plan for the future. It will be introduced in version 3.0. I have some other projects going at the moment and that will be a very large update to the platform, so it is likely not going to happen until 2019. Any current interactions that will be handled by sqlite are now handled by python objects in memory. The sqlite backend will likely not see any performance improvements. It will mostly be to support new features.

[MalloyDelacroix]

I have removed the rate limiter. Since there were no other reports of rate limit errors I decided I shouldn't try to fix problems that didn't exist. That should have been the only thing affecting performance in the newer commits.