nasa-ammos/ait-core

ait-core's Issues

BLISS to AIT Name Change

Parent epic for tracking work on BLISS to AIT name changes

Update README to match wiki

Get Core docs built and publicly viewable

Update deprecated method call in dmc.py

This is a fun one.

https://github.com/NASA-AMMOS/AIT-Core/blob/master/bliss/core/dmc.py#L124 is where we throw the deprecation WARNING message for the new GPS Leap Seconds method, but then 9 lines above it we call that deprecated method :-) (https://github.com/NASA-AMMOS/AIT-Core/blob/master/bliss/core/dmc.py#L115)

Add example script directory config to config.yaml

We're missing a script.directory config setting in the example config.yaml

Add text and email notification support

Add functionality for triggering email and text notifications so components can fire off alerts for critical situations (E.g., out of limit events).

Install errors when using a non-cloned version of the code

setup.py overwrites the install handlers to add a githook when installing the repo. If the repo isn't cloned (E.g., someone downloaded a zip/tarball of the code) this part of the install causes an error. It doesn't appear to affect the install but it's not terribly user-friendly.

Update API Wait documentation to reference packet names from example tlm.yaml

The wait() documentation references packets that don't exist in the default tlm.yaml included in bliss-core. We should update this to match with the example telemetry used in the documentation and included in the repo.

# Using a string
wait('my_instrument.tlm.BLISS_EHS.CmdsRcvd == 2')

# Using a lambda expression
wait(lambda: my_instrument.tlm.BLISS_EHS.CmdsRcvd == 2)

# Using a function
def twoCmdsRcvd(): return my_instrument.tlm.BLISS_EHS.CmdsRcvd == 2
wait(twoCmdsRcvd)

bliss_cmd_send hardcoded to 127.0.0.1

In bliss.core.bin bliss_cmd_send.py the host is hard-coded as 127.0.0.1 (variable is unused, default destination of bliss.core.api.CmdAPI is '127.0.0.1' is used):

AIT-Core/bliss/core/bin/bliss_cmd_send.py

Line 71 in ddbae5b

host = "127.0.0.1"

Users have requested being able to send to different IP addresses/hosts.

Add log rotation support to CmdAPI Command History file

Add log rotation support to the command history pcap writer in CmdAPI. This should be able to use the built in PCap rotation support.

Add (better) documentation for sequence functionality

The sequence functionality is largely undocumented. We should update the documentation to have a sequence-specific section so users have a clear understanding of what the toolkit supports

Cmd validation ignores CCSDS app id when reporting duplicate op codes

The Cmd validation needs to consider the CCSDS support added in a previous ticket when reporting duplicate commands or opcodes.

Snippet from ticket where support was implemented is below. Changes were added in dac2500.

---

We need to elevate CCSDS header support in bliss.core to allow (optional) specification of CCSDS headers and expected values when defining telemetry packets:

- !Packet
  name:  EHS
  desc:  Engineering Health and Status
  ccsds:
    apid: 42
  fields:
    ...

and commands:

- !Command
  name:      NO_OP
  opcode:    0x00
  ccsds:     { apid: 0x06, type: 1, shflag: 1, seqflags: 0b11 }
  subsystem: CORE
  title:     No Operation
  desc:      No Operation

Add TravisCI Build

Command submission default host/port should be configurable

Command submission destination (via bliss-cmd-send and via the GUI) should be configurable via config.yaml to support submission to non-standard hosts/ports by default.

This is somewhat related to #25 as well

Search doesn't work on docs hosted on ReadTheDocs

Get docs built and publicly viewable

ReadTheDocs seems like it would be ideal. If not we can fall back to the gh-pages approach we used before.

Update release procedures

Update release procedures on the wiki for all the repositories.

• Be sure to pay close attention to additional steps required only on 1/some of the repos (E.g., building the static files for GUI).
• Include steps for signing tags and release artifacts as necessary
• Include info on building the changelog per repository with https://github.com/github-changelog-generator/github-changelog-generator and update changelog generation script as needed

Add mandatory config values

We should determine what configuration values are 100% required for core functionality, report errors appropriately if those values aren't set as expected, and update documentation appropriately. This grew out of a conversation on our handling of the default port and telemetry values that are often hard coded magic values in various scripts/modules.

Update bliss-tlm-csv to allow for raw DN telemetry

Update bliss-tlm-csv to allow for raw DN telemetry

Switch AIT-Core BLISS naming over to AIT

Update repo so all past BLISS naming references are updated to AIT

• Package name and structure rework
• All documentation
• PyPi build

Update CHANGELOG

The CHANGELOG changes for the 1.0.0 release is missing previous release changes. Need to call it with the --since-tag version from the docs.

Config strftime expansions are always applied to paths

Config file loading attempts to expand out all strftime format strings in paths. There's no way to turn this off at the moment for a given string/path.

The PCap rotation being used by #10 requires that a format string be passed to open() with strftime format strings so the log can be rotated. Currently this is using command.history.filename as the value but any strftime format strings in the name are expanded out on load. This results in all the logs being rotated to the same file name ...

We need to add a way to mark a string as not requiring the strftime expansion in the config file.

Command silent failure when providing a number when string is expected

Per @lorsposto

bliss-cmd-send fails silently when a command is provided a number argument when a string is expected. The command fails validation but no messages are returned, so the user is not informed about what went wrong.

Sample command with one string argument:

- !Command
  name:      TEST_CMD
  opcode:    0x0000
  subsystem: CORE
  title: Test cmd
  desc:  Test cmd

  arguments:
    - !Argument
      name:  arg1
      desc:  Arg 1
      units: none
      type:  S16
      bytes: [0,15]

Sending the command:

>> bliss-cmd-send TEST_CMD 1
2018-03-22T09:02:19.995 | PROGRAM  | /Users/lsposto/.virtualenvs/ait/bin/bliss-cmd-send TEST_CMD 1
2018-03-22T09:02:19.998 | PROGRAM  | done.

Demonstrating the lack of validation messages. The problem is apparent after trying to encode the command anyway:

>>> import bliss.core.cmd
>>> cmddict = bliss.core.cmd.getDefaultCmdDict()
>>> cmdobj = cmddict.create('TEST_CMD', 1)
>>> messages = []
>>> cmdobj.validate(messages)
False
>>> messages
[]
>>> cmdobj.encode()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/lsposto/.virtualenvs/ait/lib/python2.7/site-packages/bliss/core/cmd.py", line 252, in encode
    encoded[defn.slice(offset)] = defn.encode(value)
  File "/Users/lsposto/.virtualenvs/ait/lib/python2.7/site-packages/bliss/core/cmd.py", line 115, in encode
    return self.type.encode(value) if self.type else bytearray()
  File "/Users/lsposto/.virtualenvs/ait/lib/python2.7/site-packages/bliss/core/dtype.py", line 238, in encode
    return bytearray(struct.pack(self.format, value))
struct.error: argument for 's' must be a string

Command arguments specified "out of order" in cmd.tlm has unexpected results

A user reported that if you specify arguments in cmd.yaml out of order relative to what the byte ordering indicates that it can cause you some problems. Not really something worth immediately "fixing" or exploring but probably something that should be made (more) explicit in the docs to avoid confusion in the future.

Push GUI build to PyPi

Add support for derived telemetry values

EGSE users have specified that they would like to be able to define derived telemetry points, especially when scripting instrument functionality with bliss.core.api, so they can track various stats that might not be returned in telemetry. For instance, they might track the number of times that a sensor's power state has been toggled on/off.

Whether this would be injected into telemetry packets in the api or implemented and accessed some other way is still TBD

Drop deprecated functions

All code marked as deprecated should be removed for 1.0.0

Improve dmc.py leap second handling

The current leap second handling in dmc.py requires manual updating whenever a new leap second it added. We should update this to pull the latest leap second data from a reliable source and update the store as needed to ensure we always have the most up to date data.

Push Core build to PyPi

Add CODEOWNERS file

Add decorator for marking functions as deprecated

Add a decorator for marking functions as deprecated that will display a warning when the function is called informing the user that the function will be removed in the future.

Update README

The README is super out of date, especially compared to the docs that we build. Let clean it up a bit.

• Rebrand with AIT instead of BLISS
• Change the intro paragraph to the one form the docs and remove the references to projects. E.g,:

The AMMOS Instrument Toolkit (Formerly the Bespoke Links to Instruments for Surface and Space (BLISS)) is a Python-based software suite developed to handle Ground Data System (GDS), Electronic Ground Support Equipment (EGSE), commanding, telemetry uplink/downlink, and sequencing for JPL International Space Station and CubeSat Missions. It is a generalization and expansion of tools developed for a number of JPL ISS missions.

• Fix all the broken URLs. They all point to old internal links
• Drop the File and Directory Structure and Authors sections

Default GUI telemetry stream config points to invalid packet name

The default GUI telemetry stream config should point to a valid packet name as defined in the example tlm.yaml

Add example of all config options in default config.yaml

It would be helpful to specify example values for all of our valid configuration options in config.yaml to help new people when they're exploring the toolkit. For example, we don't show how to set the script directory via script.directory or the HTML/static directory for the GUI via gui.html.directory

Audit installation instructions for inconsistencies with new PyPi builds

We should audit the docs and make sure our installation instructions are accurate now that we've moved packages out to PyPi.

Remove Developer Documentation from docs

The docs contain "Developer Documentation" describing the old release process, which should be removed.

Update wiki with project documentation

Update the wiki with project documentation including

• Contributor guide
• Community contact information
• Project structure and team information

It would be helpful to make the landing page on the wiki the same as the README so we can keep stuff easily up to date.

Add documentation for monitoring / notification software

The telemetry limit monitoring + notification functionality needs documented and the base settings included in the default config example (if not already). This could also fall under the GUI documentation since the code currently sits in the backend.

Get builds onto PyPi

Table tests aren't cleaning up files

test_table_table00.txt is left behind after executing tests

DN-to-EU functionality is undefined for Array types

We should figure out exactly how we want DN-to-EUs to behave for ArrayType fields. It would seem to make the most sense that we apply the DN-to-EU for each value in the array if we're going to support DN-to-EUs for this Type at all.

Remove Contributing Guide from Sphinx docs

Push DSN build to PyPi

Update tlm to csv script to allow for queries against ground receipt time

Update bliss_tlm_csv.py so users can query against ground receipt time instead of only against a field present in the telemetry.

Add docs build tag to README

Drop documentation upgrade scripts now that ReadTheDocs builds are up

• Drop ait-core/build/update_docs_release.sh
• Drop ait-gui/build/update_docs_release.sh
• Drop ait-dsn/build/update_docs_release.sh

Typos in install documentation

The PyPI installation header From AIT PyPi should be From PyPI

The environment configuration section provides info on where to find the virtualenvwrapper files as ~/.virtualenvs/postactive which should be ~/.virtualenvs/postactivate

Default leapseconds file in DMC is not consistent

The leapseconds default file name is not consistent when referenced through the UTCLeapSeconds object. Pretty sure this is why constants exist =)

Convert README from md to rst for PyPi compatibility

Badges in .md break the formatting of the README on pypi. Converting to .rst fixes the formatting.

LimitDefinition doesn't guard against missing limit levels

LimitDefinition has warn() and error() methods for checking if a value violates the defined limits. However, these helpers don't guard against missing checks.

For example,

def test_check_missing_value_warn():
     """
     # test_check_missing_value_warn

     - !Limit
        source: CCSDS_HEADER.secondary_header_flag
        desc: tbd
        lower:
          error: Not Present
      """
      ldict = limits.LimitsDict(test_check_missing_value_warn.__doc__)
      ldict['CCSDS_HEADER.secondary_header_flag'].warn('Foo')

This results in an AttributeError being raised from Threshold()

ERROR: # test_check_missing_value_warn
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/mjjoyce/.virtualenvs/oco3mos/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/mjjoyce/Coding/OCO3/oco3mos/bliss-core/bliss/core/test/test_limits.py", line 89, in test_check_missing_value_warn
    ldict['CCSDS_HEADER.secondary_header_flag'].warn('Foo')
  File "/Users/mjjoyce/Coding/OCO3/oco3mos/bliss-core/bliss/core/limits.py", line 148, in warn
    check = check or value < self.lower.warn
  File "/Users/mjjoyce/Coding/OCO3/oco3mos/bliss-core/bliss/core/limits.py", line 82, in __getattr__
    raise AttributeError("Limit has no such threshold '%s'" % name)
AttributeError: Limit has no such threshold 'warn'

We ought to guard against this issue in the warn and error helpers