magicant/flesh

POSIX shell emulation library under construction

flesh's Issues

Refining alias handling monad

#75 made AliasT a true monad, and we can refactor it even more.

• Test: Don't reparse substituted token.
• Make most parsers in Flesh.Language.Parser.Syntax just monads rather than AliasT. #79
• Consider hiding getAliasT. #79
• Consider renaming AliasT to ReparseT. #80
• Make substituteAlias a monad class method. (#74 (comment))
  • Move MonadParser and ParserT to Flesh.Language.Parser.Class`.
  • Replace MonadReader with MonadAlias as a superclass of MonadParser.

Test failure

791212b

Failures:

  hspec-src/Flesh/Language/Parser/TestUtil.hs:101:
  1) Flesh.Language.Parser.Lex.anyOperator, parses control operator, (..., returns expected result and state
       Falsifiable (after 57 tests and 4 shrinks):
       "\\\n"
       Right ("(","") /= Right ("(","\\\n")

  hspec-src/Flesh/Language/Parser/TestUtil.hs:101:
  2) Flesh.Language.Parser.Lex.anyOperator, parses control operator, )..., returns expected result and state
       Falsifiable (after 57 tests and 4 shrinks):
       "\\\n"
       Right (")","") /= Right (")","\\\n")

  hspec-src/Flesh/Language/Parser/TestUtil.hs:101:
  3) Flesh.Language.Parser.Lex.operator, parses argument control operator, (..., returns expected result and state
       Falsifiable (after 57 tests and 4 shrinks):
       "\\\n"
       Right ("(","") /= Right ("(","\\\n")

  hspec-src/Flesh/Language/Parser/TestUtil.hs:101:
  4) Flesh.Language.Parser.Lex.operator, parses argument control operator, )..., returns expected result and state
       Falsifiable (after 57 tests and 4 shrinks):
       "\\\n"
       Right (")","") /= Right (")","\\\n")

Randomized with seed 1980947583

Pretty printing

• #31 Main implementation
• #39 Error message
• #35 Implement pushChars (Seems the InputPosition type needs to be redefined.)
• #38 Bug: A redundant input line is being required after each input of completeLine

AliasT is not actually applicative

The current instance definition for the AliasT applicative and monad is invalid because the composition law doesn't always hold.
lookahead b is reparsed in a *> (lookahead b <* c) but not in (a *> lookahead b) <* c where alias substitution occurs on c.

Alias substitution following blank-ending alias

As defined in POSIX, any word token is subject to alias substitution if it follows another alias substitution that ends with a blank. To support this feature, the parser monad needs to be redefined.

• Redefine MonadInput to remember if an upcoming token should be subjected to alias substitution. #56, #57
• Apply alias substitution according to the new MonadInput definition. #58
• Forget the blank-ending alias on a newline token.
• Use the new alias/word parser for parsing syntax constructs. #58

Rare test failure

  hspec-src/Flesh/Language/Parser/LexSpec.hs:32:
  1) Flesh.Language.Parser.Lex.comment fails if input does not start with #
       Falsifiable (after 26 tests and 3 shrinks):
       "\\\n"
       Left (Soft,Error {reason = UnknownReason, position = Position {fragment =
 Fragment {code = "\\\n", situation = StandardInput, lineNo = 0}, index = 2}}) /
= Left (Soft,Error {reason = UnknownReason, position = Position {fragment = Frag
ment {code = "\\\n", situation = StandardInput, lineNo = 0}, index = 0}})

Randomized with seed 1355872402

Restructuring parser monads

Continued form #77 (comment)

• Split MonadInput into MonadBuffer and MonadReparse.
• Introduce PushBackT.
• Introduce new MonadInput.
• Introduce CursorT.
• Rename MonadInputRecord to MonadRecord.
• Make MonadRecord independent of MonadInput.
• Remove ContextT.
• Add MonadReader DefinitionSet to MonadParser dependencies.
• Redefine ParserT to combine base monads.
  • ParserT should be the only instance of MonadParser.
• Define OneShotInputT.
• Define LineInputT.
• Test: Don't reparse substituted token.

class Monad m => MonadInputIO m where
  readNext :: m (Seq (Positioned Char))

class Monad m => MonadInput c m where
  readAt :: c -> m (Either Position (c, Positioned Char))
  positionAt :: c -> m Position

class Monad m => MonadBuffer m where -- currently known as MonadInput
  popChar :: m (Either Position (Positioned Char))
  lookahead :: m a -> m a
  peekChar :: m (Either Position (Positioned Char))
  currentPosition :: m Position

class Monad m => MonadReparse where
  maybeReparse :: m (Maybe [Positioned Char], a) -> m a

class MonadBuffer m => MonadRecord where -- currently known as MonadInputRecord
  reverseConsumedChars :: m [Positioned Char]

class (
  MonadReparse m,
  MonadRecord m,
  MonadReader DefinitionSet m,
  MonadError Failure m,
  MonadPlus m)
    => MonadParser m

instance MonadInputIO ...

newtype OneShotInputT m a = OneShotInputT (m a)
instance MonadInput PositionedString (OneShotInputT m)
newtype LineInputT m a = LineInputT (StateT (Fragment, Seq (Positioned Char), Bool) m a) -- remembers if EOF is reached
instance MonadInput Int  (LineInputT ...)

newtype ExceptT e m a = ExceptT (m (Either e a))
instance MonadInput      (ExceptT ...)
instance MonadError ...  (ExceptT ...)
m (Either F a)

newtype CursorT c m a = CursorT (StateT c m a)
instance MonadError ...  (CursorT ...)
instance MonadBuffer     (CursorT ...)
c -> m (Either F (a, c))

newtype PushBackT m a = PushBackT (StateT [Positioned Char] m a)
instance MonadError ...  (PushBackT ...)
instance MonadBuffer     (PushBackT ...)
instance MonadReparse    (PushBackT ...)
[PC] -> c -> m (Either F (a, [PC], c))

newtype ReparseT m a = ReparseT (m (Maybe (Maybe (ReparseT m a), a)))
instance MonadError ...  (ReparseT ...)
instance MonadBuffer     (ReparseT ...)
instance MonadReparse    (ReparseT ...)
[PC] -> c -> m (Either F (Maybe (Maybe ..., a), [PC], c))

newtype RecordT m a = RecordT (StateT [Positioned Char] m a)
instance MonadError ...  (RecordT ...)
instance MonadBuffer     (RecordT ...)
instance MonadReparse    (RecordT ...)
-- instance MonadReader ... (RecordT ...)
instance MonadRecord     (RecordT ...)
[PC] -> [PC] -> c -> m (Either F (Maybe (Maybe ..., a, [PC]), [PC], c))

newtype ReaderT r m a = ReaderT (r -> m a)
instance MonadError ...  (ReaderT ...)
instance MonadBuffer     (ReaderT ...)
instance MonadReparse    (ReaderT ...)
instance MonadReader ... (ReaderT ...)
instance MonadRecord ... (ReaderT ...)
DS -> [PC] -> [PC] -> c -> m (Either F (Maybe (Maybe ..., a, [PC]), [PC], c))

newtype ParserT m a = ParserT (... m a)
instance MonadError ...  (ParserT ...)
instance MonadBuffer     (ParserT ...)
instance MonadReparse    (ParserT ...)
instance MonadReader ... (ParserT ...)
instance MonadRecord     (ParserT ...)
instance MonadPlus       (ParserT ...)
instance MonadParser     (ParserT ...)

Parser implementation WBS

Token

• Single quotation #9
• Double quotation #7
• (Tilde expansion)
• Parameter expansion
  • Non-braced expansion #96
  • Braced expansion
  • Prefix
  • Index (non-POSIX)
  • Modifier
    • - + = ?
    • # %
    • Substitution (non-POSIX)
• Command substitution
  • Backquote syntax #63
  • $() syntax #48
• Arithmetic expansion #66
• (Brace expansion (non-POSIX))
• Alias
  • Simple substitution #13
  • Preventing recursion #25
  • Substituting next token after blank-ending alias #54
  • Global alias (non-POSIX)

Syntax

• Complete line #24
• Compound list #42
• And-or list #22
• Pipeline #21
• Command
  • Simple command
    • Command word #11 #42
    • Redirection #19 #42
    • Assignment #71
  • Compound command
    • Grouping #42
    • Subshell #46
    • For #73
    • Case #95
    • If #69
    • While & until #59
    • Redirection #45
  • Function definition
    • Bourne-style (without keyword) #92
    • Ksh-style (with keyword, non-POSIX)
• Redirect
  • < <> > >> >| <& >& #51
  • Here document
    • Operator #19
    • Expanded content #27
    • Literal content #28
  • Here string (non-POSIX)

Miscellaneous

• Converting UnknownReason to human-friendly error message

Simple command component reordering error

1>foo do is printed as do 1>foo, which is clearly a syntax error.

Define fundamental monads for syntax parser