eecheng87/esca

Effective System Call Aggregation

esca's Issues

Didn't support batch_start()

We need to support batch_start() to prevent unintended batching.

It would be like:

write(...);
batch_start();
write(...);
...
batch_flush();

Can not build for lighttpd

• OS: Ubuntu Desktop 20.04.1
• How to reproduction:

make config TARGET=lighttpd
make lighttpd

• Error messages:

./autogen.sh: 20: autoreconf: not found
make: *** [Makefile:62: lighttpd] Error 127

• How to fix it: Install autoconf tools

sudo apt install autoconf automake libtool

• Result: It works

Enforce code consistency

ESCA was just a hack at the very beginning. However, it has been public for evaluation purpose, and we should enforce source code consistency. Here are the general rules:

1. No camel case is allowed: Always follow the similar style used in Linux kernel source.
2. Add comments in system call wrapper implementation: Explain the reason why we would perform Interpositioning.
3. Minimize the number of symbol exposure. Declare variables/functions with static modifier for inner symbols. In addition, we might make use of GCC Visibility to hide unintended symbols.
4. Add high level descriptions in README.md.

Support lwan web server

lwan is a high-performance and scalable web server implemented with coroutine technique. There is a potential to extend the applicability of ESCA to adapt lwan after this patch was merged.

Following is proposed changes:

1. Modify Makefile, support automatically build lwan
2. Support wrapper for lwan. (lwan use different system call to handle the file with varied size)
3. Current lwan will spawn threads as many as the number of host cores. Since ESCA is for single thread applications, lwanshould be configured with a single thread.

Support multi-threaded applications

In current ESCA, it is only available in single-thread application. It is time to extend ESCA to support multi-thread applications. The expected implementation would be similar with SAIO. Following is proposed changes:

1. init_esca should allocate a table and pin to a proper physical page by customized system call. (refer to init_worker in SAIO)
2. sys_register should take responsible for the task mentioned in step 1. Also, kernel need to maintain a global table which contain all the allocated table from user processes. (refer to sys_esca_register in SAIO)
3. The table indexing of sys_batch should be changed.

To verify the correctness, a unit test program is also needed. The proposed testing program would like (simplified):

void worker() {
    batch_start();
    write(1, "foo", 3);
    write(1, "bar", 3);
    batch_flush();
}
int main() {
    ...
    pthread_create(..., &worker, ...);
    ...
    return 0;
}

Support multi-worker lighttpd

As stated in #21 , multi-worker lighttpd is not available. We should extend the applicability by changing the patch file. (option server.max-worker in config file determine the number of workers)

Propose better way to download testing file

In the current version, we just provide html files in repo.
This is not good, we need to propose another method to download them.

Support Lighttpd v1.4.64 or later

lighttpd v1.4.64 was released on Jan 19, 2022. It is time to migrate the new codebase for evaluation purpose.

Activate GitHub Actions to automate builds and tests

We can follow LKMPG, which activates GitHub Actions to automate the CI. Therefore, we can ensure that source code is always buildable and ready for general use.

Replace late alloc to pre-alloc memory pool

malloc might decrease performance for copying string into cpybuf.
It's necessary to implement memory pool for copying string.

Support Nginx v1.21.4 or later

Since kTLS might be considered in ESCA later, it is time to upgrade the version of Nginx that support kTLS feature.
Nginx should be upgrade to v1.21.4 or later.

Avoid override lighttpd/nginx installation in /usr/local

The current build system will attempt to build lighttpd and nginx from source. However, it will soon override the existing files by previous nginx/lighttpd installations. Since we tend to demonstrate the ability of ESCA, it is not necessary to override the files in /usr/local. In addition, sudo should be avoided while building from source.

Write kernel-doc comments for esca.c

File esca.c should follow kernel-doc comments, which allows the generation of API documentation extracted from the comments.

Support TLS in Nginx

The current implementation doesn't consider TLS processing. However, SSL/TLS have been employed (typically in the form
of HTTPS) by 64.3% of the 137,502 most popular websites on the Internet as reported in November 2018.

Expected modifications:

1. rewrite Makefile, Nginx should be configured with SSL module (--with-http_ssl_module).
2. Since user-land encryption, sendfile is replaced with write. Thus, wrapper of write should be added in ngx.c.
3. Add script to generate RSA/ECDSA key for TLS server.

Similar changes can be referenced in SAIO in the coming weeks.

Support multi-worker Nginx

As stated in #21 , multi-worker Nginx is not available. We should extend the applicability by changing the patch file. (option worker_processes in config file determine the number of workers)

Contention in `curindex` when thread is applied

Currently, variable curindex is shared by threads. We need to make several curindexs used by each thread to prevent contention.

Code cleanup

At present, the source files of both Linux kernel module and wrapper contain several dead code, which would reflect the prototyping. However, we should clean up and make the source more consistent.

Expected results:

1. Remove all dead code;
2. Add proper comments which explain why we do need such changes;
3. Instead of maintaining different files for Nginx and Lighttpd, we should consolidate the common parts.

Avoid redundant calling batch_flush

Current implementation didn't check if it's necessary to execute batch_flush.
In some case, this will be performance bottleneck.