kmsmith137/ch_frb_l1

Toplevel software module for the CHIMEFRB L1 node

ch_frb_l1

For an incomplete (but improving) manual, see https://kmsmith137.github.io/ch_frb_l1_docs/.

ch_frb_l1's Issues

Runtime error caused by full pipe to child process when lots of L1 nodes are started at once

From the logs (as shared in #frb-ops):

Aug 29 19:55:55 cf1n0 ch-frb-l1-dispatch.sh: ch_frb_io: listening for packets (ip_addr=10.6.201.10, udp_port=1313)
Aug 29 19:55:55 cf1n0 ch-frb-l1-dispatch.sh: [Assembler-2] Found frame0_nano: 1566915994999999960
Aug 29 19:55:55 cf1n0 ch-frb-l1-dispatch.sh: [Assembler-1] Retrieving frame0_ctime from http://carillon.chime:54321/get-frame0-time
Aug 29 19:55:55 cf1n0 ch-frb-l1-dispatch.sh: [Assembler-1] Fetching frame0_time from http://carillon.chime:54321/get-frame0-time
Aug 29 19:55:56 cf1n0 ch-frb-l1-dispatch.sh: [Assembler-1] Found frame0_nano: 1566915994999999960
Aug 29 19:56:07 cf1n0 ch-frb-l1-dispatch.sh: terminate called after throwing an instance of 'std::runtime_error'
Aug 29 19:56:07 cf1n0 ch-frb-l1-dispatch.sh: what():  bonsai::trigger_pipe::write(): pipe is full and timeout expired, looks like child process isn't reading quickly enough from the PipedDedisperser
Aug 29 19:56:08 cf1n0 ch-frb-l1-dispatch.sh: /home/l1operator/ch-frb-l1-dispatch.sh: line 104:  3722 Aborted                 ./ch-frb-l1 ${L1_ARGS} ${L1_CONFIG} ../ch_frb_rfi/json_files/rfi_16k/${RFI_CONFIG} /data/bonsai_configs/${BONSAI_CONFIG} ${L1B_CONFIG}
Aug 29 19:56:08 cf1n0 systemd: ch-frb-l1.service: main process exited, code=exited, status=134/n/a
Aug 29 19:56:09 cf1n0 systemd: Unit ch-frb-l1.service entered failed state.

Compilation errors: undefined references to rf_pipelines

Hi Kendrick,

I am building this in a docker: https://github.com/MilesCranmer/docker-chime. I actually have everything compiling up to this point, which you can test by running ./build_all.sh (after doing git submodule init --update to load all the CHIME dependencies).

I am at the last step, using this dockerfile: https://github.com/MilesCranmer/docker-chime/blob/master/ch_frb_l1/Dockerfile, and this local Makefile: https://github.com/MilesCranmer/docker-chime/blob/master/ch_frb_l1/Makefile.local.

I compile some of the files, but it breaks here:

g++ -std=c++11 -Wall -O3 -march=native -ffast-math -funroll-loops -pthread -fPIC -L. -I. -I/usr/include/python2.7 -I/usr/local/lib/python2.7/dist-packages/numpy/core/include/ -I/usr/include/jsoncpp -L/usr/local/lib -I/usr/local/include -L/usr/local/lib -o ch-frb-l1 ch-frb-l1.o yaml_paramfile.o l1-rpc.o -L. -L/usr/local/lib -I/usr/local/include -L/usr/local/lib -lrf_pipelines -lbonsai -lch_frb_io -lzmq -lyaml-cpp -ljsoncpp
ch-frb-l1.o: In function `dedispersion_thread_main(l1_params const&, std::shared_ptr<ch_frb_io::intensity_network_stream> const&, std::shared_ptr<bonsai::trigger_pipe> const&, int)':
ch-frb-l1.cpp:(.text+0x72f): undefined reference to `rf_pipelines::make_bonsai_dedisperser(std::shared_ptr<bonsai::dedisperser> const&)'
collect2: error: ld returned 1 exit status
Makefile:70: recipe for target 'ch-frb-l1' failed
make: *** [ch-frb-l1] Error 1

These are my local libraries, which contain rf_pipelines:

➜  ls /usr/local/lib 
libbonsai.so      libhdf5.so.10.3.0      libhdf5_hl.la         libhdf5_hl_cpp.so.11
libch_frb_io.so   libhdf5_cpp.a          libhdf5_hl.so         libhdf5_hl_cpp.so.11.1.0
libhdf5.a         libhdf5_cpp.la         libhdf5_hl.so.10      librf_pipelines.so
libhdf5.la        libhdf5_cpp.so         libhdf5_hl.so.10.2.0  libsimpulse.so
libhdf5.settings  libhdf5_cpp.so.14      libhdf5_hl_cpp.a      pkgconfig
libhdf5.so        libhdf5_cpp.so.14.0.0  libhdf5_hl_cpp.la     python2.7
libhdf5.so.10     libhdf5_hl.a           libhdf5_hl_cpp.so

Is this a question of mixed dependencies? Is there some canonical git commit hash which I should check out for rf_pipelines?

Thanks,
Miles