pietern/goestools

Tools to work with signals and files from GOES satellites

goestools's Issues

What can goestools do exactly?

Having no issues but I would like to know what this program does exactly besides produce images and text files from the NWS. Not sure if there's more but it's only giving me JPG files from GOES and GIF and TXT files from the NWS.
I was wondering about the following...
Geostationary Lightning Mapper (GLM),
Data Collection System (DCS),
Emergency Managers Weather Information Network (EMWIN),
Search and Rescue Satellite Aided Tracking (SARSAT),
GOES Rebroadcast (GRB)? Would that be the files in the GOES-xx directory?

Source:
https://en.wikipedia.org/wiki/GOES-16

Also, is there a GUI frontend for goesrecv?

Ctrl-C doesn't terminate goesrecv when compiled in debug mode

Looks like rtlsdr_cancel_async doesn't have any effect in debug mode. Release mode is fine.

Handler for text products

NWS text files are not handled yet.

Not getting a signal

I have been trying a couple of weeks to try to get this working and I've had no luck, nor any luck from Reddit to get any help.

My issue is that i'm not getting packets or a good vit signal (above 2000)

But what confuses me is that, on windows, I have used Satcom-demodulator and have gotten an 80% signal avg and have downloaded several pictures with it. , And the signal is perfectly visible viewing it on SDR#

So, I'm not sure what the issue here is... Maybe something is wrong with the config, or corrupted?
Thanks!

(Btw I'm pointed at GOES 16, not 17)

Build failure of goesrecv. Rpi 3B+ Ubuntu Server 18.04 64 bit

Building of goestools keeps failing on my Raspberry pi 3 B+ running Ubuntu Server 18.04 64 bit.

After running make the compiler hangs and crashes on.
'[90%] Building CXX object src/goesrecv/CMakeFiles/goesrecv.dir/monitor.cc.o'

Here is a pic of where the crash occurs

This is the first part of the cmake output

This is the second part of the cmake output

If you want i can email an archive of my build directory. Its too large for github.

I have tried rebooting. I have also tried re-downloading and rebuilding 2-3 times. The result is always the same. I recently built the same software yesterday on Linux Mint (on amd64 not arm) and had no notable issues.

I have been following these instructions. https://gist.github.com/lxe/c1756ca659c3b78414149a3ea723eae2#file-goes16-rtlsdr-md

Check config file is a file

It silently fails to load the config if it is not a file.

Prevent goesproc file overwrite

Give a cli option to goesproc to prevent overwriting existing files in the output directory

Detect nanomsg drops

Right now nanomsg drops are silent. If packets are missing from the stream it is not possible to tell whether this is happening on the downlink or on the network. Detection for this should be added.

proj initialization error: major axis or radius = 0 or not given

I'm seeing this quite frequently while I am receiving data from GOES-17.

terminate called after throwing an instance of 'std::runtime_error'
  what():  proj initialization error: major axis or radius = 0 or not given

I'll try and hook gdb up to get some more context for the crash since I seem to be able to recreate it without too much issue.

Web Interface

Hi All,

This past weekend I tossed together a quick web interface for GOES Tools:
https://github.com/doubleohwhatever/goes-tools-panel/

Nothing fancy and the code is a mess but it's better than nothing. I'll clean it up when I have time.

True Color

Is it possible to get true color images from GOES-16? Something like this https://engelsjk.com/posts/goes-16-true-color-animation-using-python/images/true-color.png

Raspberry Pi Zero W

Sorry if I am putting this in the wrong place. New at participating in this. I really enjoyed getting this to work on a Pi3B+ successfully running under Raspian Stretch Lite.

I tried to duplicate the process of making it work on a Raspberry Pi Zero W. But, I keep on getting "Illegal instruction" error, when I execute the 'goesrecv' command as 'goesrecv -v -i 1 -c ~/goesrecv.conf'. Works fine on the Pi3B+.

I was wondering if this will ever be made (or is even possible to be made) to work on a Raspberry Pi Zero W?

Thank you.

Is there a way to see snow on the ground?

I would like to know if there is a way to see if I can see who has snow on the ground.
Or is there a custom LUT that I can use to extract it from one of the other bands?
This is my configuration file.

# Example goesproc configuration file to process GOES-R series products.
#

# Store all original GOES-16 products.
[[handler]]
type = "image"
origin = "goes16"
directory = "/home/pi/MyNAS/GOES/goes16/{region:short|lower}/{channel:short|lower}/{time:%Y-%m-%d}"
filename = "GOES16_{region:short}_{channel:short}_{time:%Y%m%dT%H%M%SZ}"
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# Store all original GOES-17 products.
[[handler]]
type = "image"
origin = "goes17"
directory = "/home/pi/MyNAS/GOES/goes17/{region:short|lower}/{channel:short|lower}/{time:%Y-%m-%d}"
filename = "GOES17_{region:short}_{channel:short}_{time:%Y%m%dT%H%M%SZ}"
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# GOES-16 ABI false color.
[[handler]]
type = "image"
origin = "goes16"
regions = [ "fd", "m1", "m2" ]
channels = [ "ch02", "ch13" ]
directory = "/home/pi/MyNAS/GOES/goes16/{region:short|lower}/fc/{time:%Y-%m-%d}"
filename = "GOES16_{region:short}_FC_{time:%Y%m%dT%H%M%SZ}"
format = "jpg"
json = false

  [handler.remap.ch02]
  path = "/usr/local/share/goestools/wxstar/wxstar_goes16_ch02_curve.png"

  [handler.lut]
  path = "/usr/local/share/goestools/wxstar/wxstar_goes16_lut.png"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# GOES-16 ABI RGB-enhanced
[[handler]]
type = "image"
origin = "goes16"
regions = [ "fd", "m1", "m2" ]
channels = [ "ch07", "ch08", "ch09", "ch13", "ch14", "ch15" ]
directory = "/home/pi/MyNAS/GOES/goes16/{region:short|lower}/{channel:short|lower}_enhanced/{time:%Y-%m-%d}"
filename = "GOES16_{region:short}_{channel:short}_enhanced_{time:%Y%m%dT%H%M%SZ}"
format = "jpg"
json = false

  ## The following gradients are rough approximations of the 
  ## McIDAS RGB enhancements used by NOAA/NESDIS/STAR on their site..
  ##
  ## For more info:
  ##
  ##   https://www.star.nesdis.noaa.gov/GOES/GOES16_FullDisk.php 
  ##   http://cimss.ssec.wisc.edu/goes/visit/water_vapor_enhancement.html
  ##   http://cimss.ssec.wisc.edu/goes/visit/enhanced_v_enhancements.html

  ## Shortwave IR (Channel 7)
  [handler.gradient.ch07]
  points = [
    { units = 400, color = "#000000" },
    { units = 250, color = "#b9b9b9" },
    { units = 249.999, color = "#00ffff" },
    { units = 240, color = "#000080" },
    { units = 230, color = "#00ff00" },
    { units = 220, color = "#ffff00" },
    { units = 210, color = "#ff0000" },
    { units = 200, color = "#000000" },
    { units = 190, color = "#ffffff" }
  ]

  ## Water Vapor (Channels 8 and 9)
  [handler.gradient.ch08]
  points = [
    { units = 276, color = "#000000" },
    { units = 275.9, color = "#ff0000" },
    { units = 258, color = "#ffff00" },
    { units = 250, color = "#000070" },
    { units = 233, color = "#ffffff" },
    { units = 195, color = "#408020" },
    { units = 178, color = "#00ffff" }
  ]
  [handler.gradient.ch09]
  points = [
    { units = 276, color = "#000000" },
    { units = 275.9, color = "#ff0000" },
    { units = 258, color = "#ffff00" },
    { units = 250, color = "#000070" },
    { units = 233, color = "#ffffff" },
    { units = 195, color = "#408020" },
    { units = 178, color = "#00ffff" }
  ]

  ## Longwave IR (Channels 13, 14, and 15)
  [handler.gradient.ch13]
  points = [
    { units = 333, color = "#000000" },
    { units = 238, color = "#b9b9b9" },
    { units = 237.999, color = "#00ffff" },
    { units = 228, color = "#000080" },
    { units = 218, color = "#00ff00" },
    { units = 208, color = "#ffff00" },
    { units = 198, color = "#ff0000" },
    { units = 188, color = "#000000" },
    { units = 178, color = "#ffffff" }
  ]
  [handler.gradient.ch14]
  points = [
    { units = 333, color = "#000000" },
    { units = 238, color = "#b9b9b9" },
    { units = 237.999, color = "#00ffff" },
    { units = 228, color = "#000080" },
    { units = 218, color = "#00ff00" },
    { units = 208, color = "#ffff00" },
    { units = 198, color = "#ff0000" },
    { units = 188, color = "#000000" },
    { units = 178, color = "#ffffff" }
  ]
  [handler.gradient.ch15]
  points = [
    { units = 333, color = "#000000" },
    { units = 238, color = "#b9b9b9" },
    { units = 237.999, color = "#00ffff" },
    { units = 228, color = "#000080" },
    { units = 218, color = "#00ff00" },
    { units = 208, color = "#ffff00" },
    { units = 198, color = "#ff0000" },
    { units = 188, color = "#000000" },
    { units = 178, color = "#ffffff" }
  ]

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# GOES-17 ABI false color.
[[handler]]
type = "image"
origin = "goes17"
regions = [ "fd", "m1", "m2" ]
channels = [ "ch02", "ch13" ]
directory = "/home/pi/MyNAS/GOES/goes17/{region:short|lower}/fc/{time:%Y-%m-%d}"
filename = "GOES17_{region:short}_FC_{time:%Y%m%dT%H%M%SZ}"
format = "jpg"
json = false

  # This reuses the GOES-16 contrast curve assuming it is identical
  [handler.remap.ch02]
  path = "/usr/local/share/goestools/wxstar/wxstar_goes16_ch02_curve.png"

  # This reuses the GOES-16 LUT assuming it is identical
  [handler.lut]
  path = "/usr/local/share/goestools/wxstar/wxstar_goes16_lut.png"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# Images relayed from Himawari-8.
[[handler]]
type = "image"
origin = "himawari8"
directory = "/home/pi/MyNAS/GOES/himawari8/{region:short|lower}/{time:%Y-%m-%d}"
filename = "Himawari8_{region:short}_{channel:short}_{time:%Y%m%dT%H%M%SZ}"
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# NWS text (weather reports).
[[handler]]
type = "text"
origin = "nws"
directory = "/home/pi/MyNAS/GOES/nws/{time:%Y-%m-%d}"
filename = "{time:%Y%m%dT%H%M%SZ}_{awips:nnn}{awips:xxx}"
json = false

# NWS images.
[[handler]]
type = "image"
origin = "nws"
directory = "/home/pi/MyNAS/GOES/nws/{time:%Y-%m-%d}"
filename = "{time:%Y%m%dT%H%M%SZ}_{filename}"
format = "png"
json = false

# Miscellaneous text.
[[handler]]
type = "text"
origin = "other"
directory = "/home/pi/MyNAS/GOES/text/{time:%Y-%m-%d}"
filename = "{time:%Y%m%dT%H%M%SZ}_{filename}"
json = false

# Store relayed GOES-15 full disks
[[handler]]
type = "image"
origin = "goes15"
regions = [ "fd" ]
directory = "/home/pi/MyNAS/GOES/goes15/{region:short|lower}/{time:%Y-%m-%d}"
filename = "GOES15_{region:short}_{channel:short}_{time:%Y%m%dT%H%M%SZ}"
crop = [ -2374, 2371, -1357, 1347 ]
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# Store relayed GOES-15 northern hemisphere region
[[handler]]
type = "image"
origin = "goes15"
regions = [ "nh" ]
directory = "/home/pi/MyNAS/GOES/goes15/{region:short|lower}/{time:%Y-%m-%d}"
filename = "GOES15_{region:short}_{channel:short}_{time:%Y%m%dT%H%M%SZ}"
crop = [ -1864, 1447, -1357, -3 ]
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# Store relayed GOES-15 southern hemisphere region
[[handler]]
type = "image"
origin = "goes15"
regions = [ "sh" ]
directory = "/home/pi/MyNAS/GOES/goes15/{region:short|lower}/{time:%Y-%m-%d}"
filename = "GOES15_{region:short}_{channel:short}_{time:%Y%m%dT%H%M%SZ}"
crop = [ -1864, 896, -19, 1043 ]
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# Store relayed GOES-15 US region
[[handler]]
type = "image"
origin = "goes15"
regions = [ "us" ]
directory = "/home/pi/MyNAS/GOES/goes15/{region:short|lower}/{time:%Y-%m-%d}"
filename = "GOES15_{region:short}_{channel:short}_{time:%Y%m%dT%H%M%SZ}"
crop = [ -1312, 1542, -1327, -345 ]
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# Store relayed GOES-15 special regions
# No crop specified because it is expected to move around
[[handler]]
type = "image"
origin = "goes15"
regions = [ "si00", "si01", "si02", "si03", "si04" ]
directory = "/home/pi/MyNAS/GOES/goes15/{region:short|lower}/{time:%Y-%m-%d}"
filename = "GOES15_{region:short}_{channel:short}_{time:%Y%m%dT%H%M%SZ}"
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# The following handler takes the same crop from the FD, NH, and US
# products to get more frequent imagery of a smaller area on the
# northern hemisphere. The crop region is a combination of the NH and
# US crop regions.
[[handler]]
type = "image"
origin = "goes15"
regions = [ "fd", "nh", "us" ]
crop = [ -1312, 1447, -1327, -345 ]
directory = "/home/pi/MyNAS/GOES/goes15/combine-north/{time:%Y-%m-%d}"
filename = "GOES15_{channel:short}_{time:%Y%m%dT%H%M%SZ}_{region:short}"
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

# The following handler takes the same crop from the FD and SH
# products to get more frequent imagery of a smaller area on the
# southern hemisphere.
[[handler]]
type = "image"
origin = "goes15"
regions = [ "fd", "sh" ]
crop = [ -1864, 896, -19, 1043 ]
directory = "/home/pi/MyNAS/GOES/goes15/combine-south/{time:%Y-%m-%d}"
filename = "GOES15_{channel:short}_{time:%Y%m%dT%H%M%SZ}_{region:short}"
format = "jpg"
json = false

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/local/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

Can't finish my Circonus Intallation

I have been running my goestools setup for quite some time capturing images. However, I recently happened to come across the monitoring goesrecv with Circonus on the Github. I have successfully installed the NAS service. It is active and running. I have also have setup an account to have access to my dashboard.

However, when I run the curl command with my key I get a prompt stating "*** Previous NAS installation detected *** Please use the following crconus-installation......."
The command seems to stop at that prompt. When I go to the website: https://github.com/circonus-labs/circonus-agent#quick-start

It simply states to execute the curl command. Has the COSI one step install been updated to where the User does not need to install NAD? Any help wold be much appreciated. Thanks!

Goesproc false color filenames

Filenames for false color images made with C02 and C13 seem to take the filename of the C13 source image. To avoid confusion, I propose that the false color filename be changed to reflect that it is false color instead of C13 e.g. "OR_ABI-L2-CMIPF-M3CFC_G16..." or "OR_ABI-L2-CMIPF-M3C00_G16..." This way the C13 source won't get overwritten if users try to put the false color in the same directory.

Multiple false color regions in a single GOES-16 handler

Right now it assumes a single region for the false color handler, but it should be possible to generate them for all regions, without duplicating the handler for every region.

channel numbering issues

Following up to the comments in 96dc554 with an issue for better visibility.

num can be zero, which is tripping up the assertion. Here's the whole nlh structure:

(gdb) print num
$1 = 0
(gdb) print nlh
$2 = {static CODE = 129, headerType = 129 '\201', headerLength = 14, agencySignature = "NOAA", productID = 16, productSubID = 0, parameter = 0,
  noaaSpecificCompression = 1 '\001'}
(gdb) print text
$3 = "Time of frame start = 2019-05-16T04:54:38.4Z;Satellite = G16;Instrument = FM1;Channel = N/A;Imaging Mode = ABI Mode 6;Region = Mesoscale;Resolution = 0.5km at nadir;Segmented = no"

Statsd monitoring for goesproc

Would be useful to have some statsd output from goesproc similar to goesrecv for monitoring purposes. I'm currently monitoring goesrecv statsd using telegraf/influxdb/grafana to plot all receiver stats and be alerted at anomalies in drops/vit errors.

Currently there is no good way to monitor if goesproc stops working or has errors other than lack of image output or checking the console.

Support for the SDRplay SDR's?

Is there anyway I can get this to work with my SDRplay RSP2? In Linux, it does support Soapy and there is a module that is written for it. I think it's called 'SoapySDRPlay'.
Or is this tool only for the RTL dongles?

Output directory argument for goesproc and goeslrit

Complete signal loss after a few minutes of use.

When i turn it on after not using it for a bit, i get perfect signal (300 vit, 0 dropped packets). After around 5 minutes though, the vit goes up to 2200 and i get all dropped packets. Is this something normal for GOES or is this an issue with my setup/goestools? Im using Raspberry pi 2, RTLSDR (80 gain), and a GOES SAWBird powered via bias tee.

GOES 17 M1 time stamps

Not sure if there is a fix for this. M1 time stamps for channel 13 and channel 2 are now off by one minute and I'm not sure if it is coming from the satellite or the software. As a result, full color images are no longer generated. This is for M1 only, FD and everything else is fine. It was OK up until March 5 and briefly OK on March 13. For instance

Writing ./goes17/m1/ch13/2019-03-19/GOES17_M1_CH13_20190319T135126Z.jpg (took 0.050s)
Writing: ./goes17/m1/ch02/2019-03-19/GOES17_M1_CH02_20190319T135226Z.jpg (took 0.076s)

Is there a workaround?

Expose M1 location info

As I understand it, the M1 payloads contain info about the area that GOES-16 is looking at. This info is how goesproc is able to correctly project the map overlay. Can this data be exposed to the user somehow? (Perhaps via exif info in the images or something). This would be useful for an end-user who wants to scan a bunch of images and find ones that are looking at particular areas of the globe.

new raspian version make command error

For some reason I'm getting the following error after using the make command and following the instructions line by line on raspbian lite and/or raspbian desktop.

src/goesrecv/CMakeFiles/rtlsdr_source.dir/build.make:62: recipe for target 'src/goesrecv/CMakeFiles/rtlsdr_source.dir/rtlsdr_source.cc.o' failed
make[2]: *** [src/goesrecv/CMakeFiles/rtlsdr_source.dir/rtlsdr_source.cc.o] Error 1
CMakeFiles/Makefile2:2246: recipe for target 'src/goesrecv/CMakeFiles/rtlsdr_source.dir/all' failed
make[1]: *** [src/goesrecv/CMakeFiles/rtlsdr_source.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2

SoapySDR support

I'm using SoapySDR with a bladeRF xA4 on Ubuntu and it would be wonderful to be able to specify the source as a SoapySDR device in the goesrecv configuration file.

Right now I'm looking at ways to pipe the IQ data through nanomsg.

How do I change the color?

How do I change the color of my full disk image to look like the one from https://www.star.nesdis.noaa.gov ? Both are 5424 x 5424.
I find the color could be better.

Add configuration options to goeslrit

goeslrit should have similar configuration options to goesproc for specifying output directory structure based on time, satellite, sector, spectral band, etc

compile error - "proj version 4 required"

I just pulled the latest version of the sources of goestools. When I try to compile goesproc (I believe) I get an error stating that "proj version 4 [is] required". Let me know what additional information you'll need, I'm a little new to this.

Here is the output from when I make the project, I can't seem to bold face the errors, but they start at around the 89% mark or so.

[ 91%] Building CXX object src/goesrecv/CMakeFiles/goesrecv.dir/goesrecv.cc.o
In file included from /home/isaac/.cache/yay/goestools-git/src/goestools/src/goesproc/map_drawer.h:5,
                 from /home/isaac/.cache/yay/goestools-git/src/goestools/src/goesproc/handler_goesr.cc:14:
/home/isaac/.cache/yay/goestools-git/src/goestools/src/goesproc/proj.h:4:2: error: #error "proj version 4 required"
 #error "proj version 4 required"
  ^~~~~

Missing black background for visible channel mesoscale images

Visible channel mesoscale imagery showing the edge of the Earth disk is not given a black background as with full disks imagery. Example: https://www.dropbox.com/s/q8vbqglyg0xeeeb/OR_ABI-L2-CMIPM2-M3CFC_G16_s20181362046598_e20181362047067_c20181362047103.png?dl=0

CONUS data now getting processed by goesproc

Looking at https://www.ssec.wisc.edu/datacenter/satellite-schedules/goes-16-schedules/

It appears that every five minutes there should be a CONUS content downloaded, however I am not seeing that.

I am seeing the M1 / M2 / FD regoins, but nothing more.

Goesproc info on stdout

Goesproc currently only prints "Writing" messages to stdout when processing GOES-15 full disks using the default config file e.g.
"Writing ./out/goes15/fd/GOES15_FD_IR_20180307-000600.png
Writing ./out/goes15/fd/GOES15_FD_IR_20180307-030600.png"

Goesproc should print these messages for whatever decoded files it outputs.

GoesProc fails to interpret frame dates correctly

On certain (64bit?) Linux systems the function parseTime(std:string& in, struct timespec* ts) in time.cc fails to correctly interpret the input string to a valid time_t, and therefore produces an invalid timespec result (value -1, resulting in 1969-12-31T23:59:59 dates for everything).

The string is parsed into a tm struct (containing separate fields for year, month, day etc), which then calls mktime() to turn this into a time_t value.

The call to mktime() produces -1 on my Ubuntu 19 64bit system however, meaning none of the frame start timestamps are interpreted correctly.
On Raspberry Pi (latest Raspbian) this works fine.

I tried to separate the date conversion code into a standalone program, and interestingly here it works fine with the exact same code of parseTime(), with the exact same input that was observed to produce -1 in goesproc.

Somewhere along the execution of everything goesproc does, the behavior of mktime changes and no longer produces valid values.

goespackets example

goespackets --subscribe tcp://$IP:5004 --record --filename '%Y-%m-%d/packets-%FT%H:%M:00Z.raw'

GOES 15 map is off

GOES 15 has moved to 128.57W and the map is off. I am receiving the GOES 15 images from GOES 17 relay. Everything else is working perfectly and I'm extremely pleased, thanks a bunch, I haven't had this much fun since I first tried wxtoimg several years ago!

terminate called after throwing an instance of 'std::runtime_error'

Hi this is a error. I del all and reinstall it, and the same,
try new goesproc-goesr.conf config reading here, adding and removing lines in goesproc-goesr.conf, and many times say console, invalid image handler product. someone have a "goesproc-goesr.conf" final please?
I try all, the process crash after minutes
Thanks

i@GOES16:~ $ sudo goesproc --config /home/pi/goestools/goesproc-goesr.conf --mode packet --subscribe tcp://127.0.0.1:5004
Writing: ./nws/2019-07-31/20190731T230700Z_2019072122307548-pac48_latestBW.gif
Writing: ./nws/2019-07-31/20190731T230700Z_201907212230707-pacsfc72_latestBW.gif
terminate called after throwing an instance of 'std::runtime_error'
what(): Assertion rv == 2 failed at /home/pi/goestools/goestools/src/goesproc/handler_goesr.cc:29, Expected to extract mode and channel from file name (OR_ABI-L2-SSTF-M6_G16_s20192122200487_e20192122300196_c20192122306390.lrit)

Map overlay

Can use this issue to track relevant info. I think it would be simplest to not do shapefile parsing and instead use something like shp2json to turn it into JSON and reuse the existing JSON parser that is already bundled. Then use cairo to draw lines. Would be nice to ditch opencv as well while working on this, as it is only used for its jpg/png/etc write capability.

• Shapefile sources: https://www.naturalearthdata.com/downloads/
• Set of d3 articles: https://medium.com/@mbostock/command-line-cartography-part-1-897aa8f8ca2c
• GEOS projection math: https://www.cgms-info.org/documents/pdf_cgms_03.pdf

VCDU and TP_PDU drops?

Hi there,
So once and awhile I am seeing a 'VC 14: VCDU drop' and a VC 14: Detected TP_PDU drop' in the terminal window that goesproc is running in.
The command I am using is....
goesproc -c /usr/local/share/goestools/goesproc-goesr.conf -m packet --subscribe tcp://127.0.0.1:5004

Goesproc-goesr.conf is saving the images to my NAS over a wired connection.

So, what is causing this error?

Both goesproc and goesrecv are running on my Raspberry Pi 3 B.

Handle non-CMIP products from GOES-R series

Looks like there are some experiments going on sending products other than the Cloud and Moisture Image Product on the HRIT stream. The goesproc handler only deals with the CMIP product and expects details to be embedded in the LRIT ancillary text header and this is missing from the new files.

There should be a separate handler for non-CMIP products.

Is GOES HRIT active?

I don't understand if the HRIT service is the same as EMWIN.

According to NOAA's website, at https://www.nws.noaa.gov/emwin/ the HRIT/EMWIN services are "Not Operational".

Does this mean I can't get images from GOES satellites now? Or is EMWIN unrelated to images that can be decoded by this software?

goesproc: std::invalid_argument -- what(): stoi

Running goesproc for the last year in docker containers [ Ubuntu trusty, docker-engine 1.11.2-0trusty -- 3.19.0-25-generic #2614.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux -- libc6:amd64 2.19-0ubuntu6.7 -- gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04)].

Today, I noticed this error in the docker logs:

{"log":"terminate called after throwing an instance of 'std::invalid_argument'\n","stream":"stderr","time":"2019-12-23T05:36:16.062325837Z"}
{"log":" what(): stoi\n","stream":"stderr","time":"2019-12-23T05:36:16.06236406Z"}

This is in dmesg:

[46703.734555] traps: goesproc[30816] general protection ip:7f1b3be75196 sp:7ffed5bc3f90 error:0 in libc-2.23.so[7f1b3be3e000+1c0000]

I am running two independent instances of goesproc: one for goes16 and one for goes17 goesrecv instances. These goesrecv instances are running on rpi hosts which are connected directly to their respective antenna.

This is the key section for the docker image creation:

RUN
apt-get install -y
build-essential
cmake
git-core
libopencv-dev
zlib1g-dev

download, compile, and install librtlsdr

RUN
cd /opt &&
apt-get install -y libusb-1.0-0-dev &&
git clone https://github.com/steve-m/librtlsdr.git &&
cd librtlsdr &&
mkdir build &&
cd build &&
cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr -DINSTALL_UDEV_RULES=ON .. &&
make -j2 install

RUN
cd /opt &&
git clone --recursive https://github.com/pietern/goestools &&
mkdir -p /opt/goestools/build &&
cd /opt/goestools/build &&
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local &&
make &&
make install

Both docker instances crash within minutes of each other which is odd and rarely longer than 30 minutes.

I am running this git hash version: commit 9ad66af which has an stoi call in "if (key == "Channel")" control block. The other instance of stoi is in a try/catch.

The latest master version has omitted the stoi function in the "if (key == "Channel")" control block.

I am curious if the data has changed from the satellite transmission that is causing a fault in goesproc which could be causing this failure or could it be something else.

how to add text date to image?

how to add text date to each image goes?
and scales?
I add scale image crop with "IMDrops ImageTools" in this sequence images https://www.facebook.com/lw7dst/videos/370021817021417/
thank's

Detect buffer overruns in goesrecv

On a slow machine it is still possible for the DSP to run too slow and for the SDR to drop samples.

If this happens it should be logged instead of the current behavior where it is completely ignored.

The artifact of dropping samples is seeing a bunch of hard to explain packet drops and irregular number of packets processed per second (when running in verbose mode).

DCS processing

This could be nice. Using this issue to collect some links.

• http://eddn.usgs.gov/dcpformat.html
• https://acwi.gov/hydrology/stiwg/index.html

Handler for EMWIN products

EMWIN files are not handled yet. The LRIT data is always zipped for EMWIN.

Add install statements to CMakeLists.txt

make install does nothing right now.

make -j2 install fails

I've been following the very clear installation instructions for goestools installation, however when I run "sudo make -j2 install' it gives the following errors:

[ 65%] Linking CXX executable areadump
[ 65%] Built target unzip
[ 65%] Linking CXX executable lritdump
CMakeFiles/areadump.dir/areadump.cc.o: In function lrit::SegmentIdentificationHeader lrit::getHeader<lrit::SegmentIdentificationHeader>(std::vector<unsigned char, std::allocator<unsigned char> > const&, std::map<int, int, std::less<int>, std::allocator<std::pair<int const, int> > > const&)': /home/pi/goestools/src/lrit/lrit.h:166: undefined reference to lrit::SegmentIdentificationHeader lrit::getHeaderlrit::SegmentIdentificationHeader(std::vector<unsigned char, std::allocator > const&, int)'

....(end of error message):

collect2: error: ld returned 1 exit status
src/lrit/CMakeFiles/lritdump.dir/build.make:96: recipe for target 'src/lrit/lritdump' failed
make[2]: *** [src/lrit/lritdump] Error 1
CMakeFiles/Makefile2:1836: recipe for target 'src/lrit/CMakeFiles/lritdump.dir/all' failed
make[1]: *** [src/lrit/CMakeFiles/lritdump.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2

Issue with Assertion..

Hi my issue is when I try to run goesrecv on command with my Airspy Mini & the usual command I'd ran it before: goesrecv -v -i 1 -c ~/goesrecv.conf it throws out this instance/error at me: 'std::runtime_error what(): Assertion 'rv >= 0' failed at /home/blake/goestools/src/goesrecv/airspy_source.cc:62 (I checked the line but didn't know how to fix it). Could somebody please tell me why this is & possibly how to fix it. Thanks for your time too hey..

NOT ISSUE QUESTION ONLY

Hi and thank's for development this code, my question is: is possible run in windows 7 goesproc ? or how compile it, thanks !

assertion failure: Expected to extract mode and channel from file name

I'll try to poke at this later, but just making a record of this here for now

terminate called after throwing an instance of 'std::runtime_error'
  what():  Assertion `rv == 2` failed at /nas/achin/devel/goestools/src/goesproc/handler_goesr.cc:29, Expected to extract mode and channel from file name (OR_ABI-L2-RRQPEF-M6_G16_s20191981650398_e20191981700106_c20191981700191.lrit)
Aborted

Missing: Himawari-8, GOES-15 and GOES-17 relays

I know that goestools does capture the images from these three satellites that do get relayed through GOES-16 because it's in the config file and I did get it once, a long time ago. How come they are not more constant?
Also, shouldn't I get an admin.txt file?