hnsl/librcd

A library for Rapid {C|Concurrency|Correct} Development

librcd's Issues

Fix polarssl segfault

https://polarssl.org/tech-updates/security-advisories/polarssl-security-advisory-2014-02
#0: 0x549643 gcm_auth_decrypt in gcm.c:317
#1: 0x4ceeff ? in stack.s:85
#2: 0x53f5e0 ssl_decrypt_buf in ssl_tls.c:1275
#3: 0x4ceeff ? in stack.s:85
#4: 0x53f09f ssl_read_record in ssl_tls.c:2006
#5: 0x4ceeff ? in stack.s:85
#6: 0x5425c7 ssl_read in ssl_tls.c:3703
#7: 0x52dcc2 __librcd_ifc_join__tls_read in polar.c:118
#8: 0x4ceeff ? in stack.s:85
#9: 0x52dc32 tls_read in polar.c:112
#10: 0x52f45a tls_rio_read in polar.c:323
#11: 0x4ceeff ? in stack.s:85
#12: 0x4d8f15 rio_read_direct in ?:?
#13: 0x4d9753 rio_peek in rio.c:1403
#14: 0x490b7c http_session in wsr.c:360
#15: 0x4ceeff ? in stack.s:85
#16: 0x48ef22 _http_connection_fiber in wsr.c:821
#17: 0x4ceeff ? in stack.s:85

Don't mix GPL2 and GPL3

Currently librcd depends on a mix of (L)GPLv2 and GPLv3-licensed libraries, making distribution of binaries impossible. We need to replace those by either newly written code distributed under MPL2 or replace GPL2 code by GPL3 code or vice versa.

More concretely:

• /include/linux.h and /src/sort.c are from Linux and so are GPL2 only. It might be possible to remove comments, and rewrite macros+inline functions, to claim that the result is either non-copyrightable APIs, or fair use. Bionic does something similar. Sorting routines are simple to replace, of course.
• /libs/asmlib/* from http://www.agner.org/optimize is GPL3 only. I haven't checked how much of the library is used, but replacing it feels viable.

I think all the other GPL-licensed libraries all say "version 2 or later", so they'd work with both options.

Rewrite the post linker in librcd/C - decrease build time with 50%+

The post linker is currently a major time stealer when building. It's written in python to make use of an ELF/Dwarf library that doesn't suck (pyelftools). Unfortunately it's slow, even with pypy due to the large number of dynamic objects it needs to create. I belive that it would be much faster to have this written in C instead. The major obstacle is finding a simple elf manipulation and dwarf parsing library.

Rewrite the post pre processor in a better language than PHP

It's currently written in PHP because it was fastest for me to do. It would be nice if it was written in something else. Suggestions?

The standard library currently does not provide an automatically expanding buffer

I have never had use for an automatically expanding buffer, e.g. a classic 2^n buffer that reallocates and copies its content if it needs to expand, because I always max allocate instead as I prefer code that is optimised for the worst case instead of the expected or best case. This doesn't mean that someone could find this useful and it's such a wildly adopted pattern that librcd should provide it.

Dynamic linking

This would need explicit support by segmented stack code (lwthreads.c, stack.s, maybe clang fixes).

Backtraces create accidental GitHub issue references

See #19, which (hilariously) made all of issues 1-17 say they were referenced by that issue. It could be worth changing stack trace format to avoid saying #<number> to work around this. Or we could just say that any such list ought to have been `-quoted in the first place, and wontfix this.

Add high level tcp connection establish function that takes host, port and support for A record redundancy

Suggested name: rio_tcp_establish(). It should take a host name and a port. The host name is resolved to a list of A records. It should try to connect to each A record in order (randomly) until all records have been exhausted. The function should not return unless a tcp connection has been established (using rio_tcp_conn_wait).

Portability

Librcd currently only supports x86_64 Linux. Porting to other platforms (esp. Windows) would be a large undertaking and probably break some abstractions, and is not at the moment planned. Still, opening this as a tracking issue.

The following things are Linux-specific:

• Low-level code, such as lwthreads, signal handling and the code for segmented stacks (all .s files).
• I/O (files, pipes, network, executing programs, etc.)
• The musl libc compatibility layer.
• Backtraces depend on details of ELF (reflect.c).

64-bit specific:

• all assembly code
• memory allocation?
• integer types?

Clang would need certainly patching also. (@hannes-landeholm, plz edit this comment and add some details)

Librcd does not yet have a version scheme

Suggested version scheme: MAJOR(+rcX). The first version will be "0".

Librcd will not have minor version numbers or use the terms "beta" and "alpha" as they have become ambiguous and reenforce the idea that stability is black and white and you can pick a version and stick with instead of having a continuous upgrade cycle.

Release candidates are for testing before rolling out a new release. The final release has the "rc" suffix dropped. New releases always increment the MAJOR version number no matter how little changes has been made. A release candidate and a subsequent release may point to the exact same commit if the release was acceptable.

Example: v0, v1+rc0, v1+rc1, v1+rc2, v1, v2, v3+rc0, v3+rc1, v3, v4+rc0, v4

The point where releases are made are initially undefined but should take place at a point where librcd is stable enough to be used. Librcd will never "support" older versions, programmers should keep up to date. Newer versions may break back compatibility in interfaces and behaviour if the long term benefits outweigh the costs.

fstr_t stack allocations is horrible and cannot be done in a single statment (FSTR_STACK_DECL)

The FSTR_STACK_DECL macro is too long and makes my eyes bleed. It also a multiple assignment pattern (cancer™).

Suggested solution:

#define fstr_stack_alloc(n) ({ uint8_t _mem[n]; FSTR_PACK(_mem); })

Is the stack of the extended GCC context valid after it returns?

Convert io exceptions to eio

Librcd should use more specific eio exception classes instead of exception_io everywhere. E.g. eof_eio or fs_eio seem particularly useful.

Serialization of doubles is currently ridiculous

200 is currently serialized as 2e2. This is correct but unexpected and pointless. It's not usually what people expect when they just want to dump a floating point value in a quick and dirty way. We should reserve scientific notation for the big numbers.

lwt/scheduler: better performance in concurrency ring test

The overhead for concurrency is currently quite large. This is mainly caused by contention from global locks that can be eliminated with hardware transactional memory when it becomes widely available. Another solution is more fine graned locking but I've only encountered dead ends so far in R&D of this.

Unit test JSON library

Particularly escape codes and UTF-16 stuff.

Librcd currently depends on dig for dns lookups

It would be nice to replace this with some embedded dns library. In a perfect world the operating system takes care of this...

Add a vector datatype

Lists don't support direct indexing and are somewhat slower; arrays are fixed-size.

lwt/scheduler: better core cache locality for fibers by introducing inertia that prevents switching core

http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf

research __morestack_fx fix, it is likely not needed and has a negative effect on performance

SSE state saving was implemented after misunderstanding the x86_64 systemv abi and is likely not needed. Remove it from librcd and revert the related llvm patches.

rio: throw special eio exception for eof conditions

Sometimes eof is expected and since eio's was introduced we can use them to distinguish the condition. This line should be changed in rio.c:

 throw("read() failed: end of stream reached", exception_io);

This will greatly simplify the implementation of rio_read_to_end.

io exceptions with custom type

define_error(json, int32_t line, int32_t col, fstr_t token);

emitosis(json_error) {
    err.line = line;
    err.col = col;
    err.token = fsc(token);
    throw_em("json parsing failed, invalid token", err);
    throw_em_fwd("json parsing failed, invalid token", err, e);
}

define_errore(protocol)

throw_io(e_msg, protocol_error);

throw("", exception_arg)

catch (exception_io, e) {
    if_error(e, json_error) {
        err.line;
    } else if_error(e, parse_error) {

    } else {
        throw_fwd("fwd ", e->type, e);
    }
    if (e->error_type == parse_error) {
        parse_error_t* err = e->error_data;

    }
    lwt_throw_exception(e);
}