anthonymorast/datatables

A tabular data structure for C++

Anthony Morast

About Me

• 💼 I am C++ Software Engineer at Sterling Trading Tech
• 📖 I have a blog on my personal website and Medium
• 📗 I'm reading An Introduction to Mathematical Finance with Applications by Arlie O. Petters and Xiaying Dong.
• 💬 Ich lerne Deutsch.
• 👷 I'm currently working on:
  • A fluid dynamics simulation playground in Rust using the Lattice Boltzmann method
  • Improving my website by working through the submitted bugs
  • Option pricing via binomial models in Rust

Education

• 📃 Applied and Compuational Mathematics, B.S.
• 📃 Computer Science, B.S.
• 📃 Computational Sciences and Robotics, M.S.

---

Things I Really Like to Use

Some More Things I Can Use

Connect With Me

📧 Email 💻 Consulting

---

datatables's Issues

add vscode files to .gitignore

Restructure DataTable declaration and implementation

Across multiple files

Create unit tests

Modernize code base

Per c++11, 14, 17, 20 standards

Improve cURL data reads by avoiding clumsy file write->read->delete

This can be done with lambda functions, e.g.

{
    CURL *curl;
    curl = curl_easy_init();
    std::string readBuffer {};
    if (curl)
    {
        curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, [&](void* contents, size_t size, size_t nmemb, void* userp){
            reinterpret_cast<std::string*>(userp)->append(reinterpret_cast<char*>(contents), size * nmemb);
            return size*nmemb;
        });
        curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
        curl_easy_perform(curl);

        /* always cleanup */
        curl_easy_cleanup(curl);
    }

    std::cout << "read from cURL: " << readBuffer << std::endl;

    return readBuffer;
}

Immediately useful funcitonality

• Ability to create DataTables from other DataTables (datatable::DataTable dt();
• Ability to remove single column
• Ability to append rows/columns to the dataset.

Improve performance of file reading

Slow on large files, e.g. MNIST

Enforce numeric types for template parameter

https://stackoverflow.com/questions/14294267/class-template-for-numeric-types

implement min and max functions

For convenience. Could do average and variance too.

Add ability to append rows/columns to the dataset.

Test build on other systems

• Win10
• MacOS
• Non-Debian distros

Add single column select

For example, datatable::DataTable.col() which returns a single column datatable (no response variable) so these can be chained:

datatable::DataTable.col("colname").max()
datatable::DataTable.col("colname").min()

etc.

Deep copy data when creating dt with 2-d array

Enhance min, max, sum, mean to use column names or numbers

Accommodate Date/DateTime in columns

Could convert back and forth between Unix epoch to be able to store in the existing double array. Would need to keep an array/vector of the column numbers that are datetimes and print and operate on these columns accordingly

Select Columns is probably broken

DataTables/src/DataTable/DataTable.cc

Line 344 in e3a7e23

for(int j = 0; j < _cols; j++) // swap loops to minimize number of k loops.

The loop variable k is prone to overstepping array bounds. This should be re-written like select_rows(...) anyways