dgets/docshell

Damo's version of the DOC (Citadel-esque) shell, in JavaScript, for use with Synchronet 3.14+ systems

DOCshell/dDoc

DOCshell (or dDoc [Damo's version of DOC]) is a shell for the Synchronet BBS package meant to emulate the experience of vDOC (or DOC in general), implemented in the BBS family starting with ISCA BBS in the 80s, at the University of Iowa.

Goal and Mission

The general goals of this interface are to provide as little overhead, for posting and reading messages, as possible. I'm actually deferring a little bit to the judgment of some others that've been utilizing this interface for longer than I. I prefer the old school BBS days, where more functionality, and more options, meant more things to explore; this goal is decidedly different, it is to implement as little distraction from message text/bases and instant messages to other users as possible. As such, many Synchronet features will not be implemented until I have a polished beta, and begin tucking them away where only people comfortable with the interface will be able to stumble across them.

Current State

DOCshell/dDoc has hit 'release' state 0.1b, ready for beta testing, as of 29 Jul 15. Although there are still some minor display issues, I believe that all usability features are in place. Of course, part of beta releasing this software is testing that hypothesis out. ;)

Checking it Out for Yourself

If you would like to see what this kind of system is all about, you can find slightly differing versions of the DOC software operating at the following two sites:

* ISCA BBS (telnet://bbs.isca.uiowa.edu/)
* eSchwa BBS (telnet://bbs.eschwa.com/)

Please be aware that, at the very least, eSchwa BBS is run by fascists.

dDoc's Home

My own BBS (still pending implementation of this DOCish interface [which will be coming very soon]) can be found at the following addresses:

* Tinfoil Tetrahedron BBS (telnet://tinfoil.synchro.net/)
* Tinfoil Tetrahedron BBS (secure via SSH: ssh://tinfoil.synchro.net:8022/)

docshell's Issues

Message Entry Display Bug

Upon entering the routine to add a new message, entry starts a line ahead of where it should, leaving a blank between the #/author header and the subsequent text of the message to be entered.

Debugging code displaying 'lNdx' is showing it incrementing after the first line bringing it to 1 by increments of 2 (just hitting the odd numbers). However, when the message entry routine is completed, and the debugging dump happens, it shows the current mTxt index as (lNdx - 1). Only the very last line is shown on the debugging dump (not sure if that's how it's supposed to be right now or not).

So I guess that's 3 small bugs, a portion of which are actually in the debugging dump routine at this point.

Message Base Scanning Modes Work

This doesn't really fall under message handling, I guess, so it's time for another issue. . .

There were multiple bugs in the logic for doing a 'new message scan' from the primary menu/prompt); I worked out a bunch of these today. The issues were primarily OO accessing done incorrectly, due to not knowing the Synchronet API inside and out yet. Once this was fixed, it became apparent that the new message scan was not updating pointers correctly. Simple incrementation adjustments got this fixed up all right.

Upon completion of this, other issues have become apparent. Right now they seem mostly aesthetic, consisting primarily of console text inserts, from debugging displays, ending up breaking up the screen continuity.

Next up on the agenda, after that turd is polished, will be handling reverse message read; this should definitely go hand-in-hand with the work on modularizing the message display code. There is too much cut 'n paste going on here, and that's what made the original DOC code so horrific to read in the very first place. 'Course some of that wasn't even cut 'n pasting of horrible code, but horrible code rewritten in multiple places, some even more horrific than the Freddy Krueger original.

Any other agendas with message base scanning features/enhancements/displaying should be added underneath.

Lacking Stub

In implementation at the actually shell level for a user there needs to be a baja stub created. /sbbs/exec/ddoc.bin

NOTE: After this is handled, in order to avoid a user being stranded in the shell, there needs to be a hook back to changing default shell, at the very least.

Message Entry Not Saving to Message Base

This seems to be an artifact such as what was occurring with the previous debugging code (in ddebug.js), when the initial message saving test was implemented. Echicken had come up with a solution that I still do not fully understand. I will be consulting with him in the future; the header for the message seems to be the key. Bug is located in dmbase.js.

dperuser.js should be renamed or non-user specific routines moved

This should be done because having the 'room info' functionality, even just for file IO, in the same location as the 'per-user' routines is ridiculous. Perhaps all file IO should be moved out; seems like this might be the best way to handle things for now.

Docs need to be caught up

Documentation on the objects, methods, and properties need to be revamped since a bit of the functionality has changed. Inline documentation and the wiki page is also out of date now due to the fact that the modular subsystem has been implemented and the depreciated crap removed. FWIW, the unimessageread or whatever it is, at the end of dmbase.js seems to be completely vestigial and without purpose, also.

Move supportive files to $SBBS/mods/load/

For clarity's sake, since $SBBS/mods/ is a shared directory, it would be best to try to move the other supportive files/libraries that are loaded later into the /load/ subdirectory of this tree. Not sure how much code work should be involved; I believe it automagically looks there, but if not adding a few paths shouldn't be that tough.

Looking for Other Message Handling Issues

Need to go through the message entry routines in order to get this online ASAP. Need to verify that blank line handling works, tab handling works (or is disabled), backspaces have already been verified... Also, wordwrap needs to be [presumably] debugged, and it'd be great to find out that this can be replaced simply by the wordwrap() code already inherent in Synchronet, though it doesn't appear to work in my situation with custom message input, currently. There may be an easy way to work around this, though.

My guess is most of the time will be spent working on wordwrap, if it can't be simply replaced.

Reverse message scan is broken again

Almost having it working right again, but not quite:

[Neutron Star> msg #2 (6 remaining)] Read cmd -> 
End of scanSub() main loop
tmpPtr: 3       inc: 1  fuggit: false
In main scanSub() loop  tmpPtr: 3
ptr: 3  base.last_msg: 8
Wed, 19 Nov 2014 09:11:47 -0800 from Khelair
jfaklsdjglk;

[Neutron Star> msg #3 (5 remaining)] Read cmd -> Back (change direction)...
End of scanSub() main loop
tmpPtr: 2       inc: -1 fuggit: false
In main scanSub() loop  tmpPtr: 2
ptr: 2  base.last_msg: 8
Tue, 18 Nov 2014 16:08:49 -0800 from Khelair
ouah

[Neutron Star> msg #2 (6 remaining)] Read cmd -> 
End of scanSub() main loop
tmpPtr: 1       inc: -1 fuggit: false
In main scanSub() loop  tmpPtr: 1
No preceeding messages
Exception: verifyBoundsException        Msg: No preceding messages      #: 3
ecode: 0
ptr: 1  base.last_msg: 8
Got exception name: dispMsgException    Msg: Invalid message slot
Num: 1
In main scanSub() loop  tmpPtr: 0
No preceeding messages
Exception: verifyBoundsException        Msg: No preceding messages      #: 3
ecode: 1
ptr: 0  base.last_msg: 8
Tue, 18 Nov 2014 16:08:49 -0800 from Khelair
ouah

[Neutron Star> msg #0 (8 remaining)] Read cmd -> In main scanSub() loop tmpPtr: -1
No preceeding messages
Exception: verifyBoundsException        Msg: No preceding messages      #: 3
ecode: 1
ptr: -1 base.last_msg: 8
Got exception name: dispMsgException    Msg: At start of messages
Num: 3

End of scanSub() main loop
tmpPtr: -2      inc: -1 fuggit: true
Closed mBase: topegrpneutrons
Ename: scanSubException Msg: Done with message scan     #: 4
Got topegrpneutrons in user.cursub
Neutron Star>

The output was produced using only spacebar to page forward or backward through the messages, and 'b' when the inc variable is changed to reflect a different direction in scanning. Hell, without debugging output this would be pretty close to perfect as it is.

hitting space while reading in a room with no new messages = infinite loop?

This behavior is best experienced in Lobby> on the test system currently. I don't know if it has anything to do with the deleted messages in Lobby> or not. However, once I start reading the room by hitting space at the room prompt, I see the sole non-deleted message, "msg #0 (11 remaining)" at the read prompt. I then hit space again. Currently on the test system, there are no more (undeleted) messages in Lobby>.

WHAT ACTUALLY HAPPENS

1. I see a bunch of debugging output, mainly to do with skipping the deleted messages.
2. If I hit space again, I see the same message again, but the read prompt now says "msg #10 (1 remaining)".
3. If I hit space yet again, I see some debug output and a room prompt, and then the same message again as the system appears to have automatically started reading the room again?

[Lobby> msg #10 (1 remaining)] Read cmd ->  
End of scanSub() main loop
tmpPtr: 11  inc: 1  fuggit: false
In main scanSub() loop  tmpPtr: 11
ptr: 11 base.last_msg: 11
Got exception name: dispMsgException    Msg: Invalid message slot
Num: 1
In main scanSub() loop  tmpPtr: 12
tp: 12  inc: 1  mBase.last_msg: 11  mBase.code: undefined   mBase.is_open: true
Hit last message, returning 1
Exception: verifyBoundsException    Msg: Last message pointer   #: 1
ecode: 1
Got topegrplobby in user.cursub
Lobby> Entered scanSub(); forward = true  user.cursub: topegrplobby
sBoard.code: topegrplobby
Opened: topegrplobby allegedly . . .
sBoard.scan_ptr = 0
mBase.first_msg = 10
mBase.total_msgs = 2
mBase.last_msg = 11
Inc: 1  based on 'forward'
In main scanSub() loop  tmpPtr: 0
ptr: 0  base.last_msg: 11
Wed, 21 Jan 2015 06:21:58 -0800 from REDACTED
  I'm in the process of working out the last of the bugs here.  As soon as the
reverse reading and 's'kip functionality is implemented this base will be
active and utilized by the new DOC shell.

[Lobby> msg #0 (11 remaining)] Read cmd -> Stop

WHAT I EXPECT TO HAPPEN

Upon hitting space at the read prompt on the last readable message in a room, I should be dropped to the room prompt.

System logging needs to be enhanced greatly for debugging and eventual security

Logging of debugging information, as well as standard important flow control, crashes, and bug/exception detection needs to be added via provided Synchronet API log functionality. This will assist greatly in debugging that crashes the entire shell, or infinite looping structures that remove important information from the terminal too quickly.

After alpha stage and beta rollout is completed, this will assist and be quite essential for security; I'd like to implement logging a bit more detailed than the type that Synchronet relies on (ie keystroke logging solely) through the default shell at this point.

Debugging granularity

For full beta development, as well as the next issue that I'm going to create that will allow more testing to take place by more individuals, it will be necessary to implement much more granular debugging. IE: debugging of just xpress messaging functionality, or message base (or even down to just posting), or navigation features, etc.

Xpress Text Entry Optimization

The current express text entry method is using console.getkey(), without any special form of input processing. I began using these methods before I had become familiar enough with the Synchronet API to understand how to use console.getstr(), which handles all of the 'special keys' that I need to worry about on its own. No worrying about ctrl keys, tab processing, or anything of the sort. So all that needs to be handled here is gutting the internals, throwing out that crap, and putting in the appropriate console.getstr() calls, which will need no debugging. Nice 'n easy.

ddebug depreciated

The debugging routines that I was going to implement in ddebug.js have fallen out of favor; I'm strongly considering moving this to a separate project for any other, fullscreen, JavaScript projects for Synchronet in the future. I've wanted to speed up development, and I'm not sure if it's worked by skipping working on these specialized heads-up routines or not, but I'm at a point now where it's really going to slow me down to bother with that. It's still a great concept for it's own project, or inclusion when I decide to stat messing with frame.js, though.

Message scans are not showing the last message in any one sub

When doing a forward scan the last_msg in the object is never displayed, only the ones prior to it. My first place to peek on this issue would be where @ntwitch was mentioning that there is a disparity between pointers being utilized that start at a 0 index and those that start at 1 as the base. Identify and compare the logic.

Object 'Prototypes' Need to be Documented

I'm starting to forget different method prototypes and parameters that are to be passed to them. Due to the fact that I'm working on this code from remote and having to use vim currently over a terminal link (that isn't supporting my console movement keys correctly [home, end, pgup, and pgdown]), scrolling around in order to relocate each method and review the parameters to be passed is proving to be a real pain in the ass.

It is definitely time to go back through and document each of the prototypes like I was doing with JavaDoc previously. :P Get it done.

'n'ext message functionality is not properly reading from the current sub/room

Something small is broken again that is keeping 'n'ext message functionality (the same routine as the 'spacebar scroll') from properly reading new messages in the current sub-board (room in dDoc parlance). At this point it is always going back to reading from the Lobby, the default sub/room.

As I am changing the functionality/schema for the methods utilized in issue #40, this is not exactly the same issue. There are still problems with that particular issue, as well, but I am reimplementing where exactly that control structure is going to take place. All that I know for now is that it'll be handled outside of scanSub(); keeping the functionality of scanSub() at just scanning one particular sub seems to be a good default behavior (thank you, @gordon-morehouse, for pointing this out).

Message Entry Simplification (stop rebuilding the wheel)

Upon browsing the Synchronet API today, it became apparent that the word wrapping issue does not have to be coded personally here; console.getstr() should have all of the functionality that is necessary in order to avoid duplicating code.

This may otherwise simplify other areas within the code; message handling routines will be trimmed down considerably.

yell to sysop

This should probably be tied in correctly before sending things out as an alpha, I'd expect.

No way to break out of unresponsive flow loops or general infinite loop bug flow

According to echicken, there is an option to turn off ctrl-key passthrough and handle them through the outside calling structure. We need to implement ^C breaking throughout the entire shell while debugging, then selective implementation afterwards to break out of certain control flows.

Jump functionality

Non-existant. Write it.

mHdr issue when doing a forward scan?

A new issue has appeared wherein hitting spacebar from the read menu (at this point it was induced after checking to logout, choosing no, jumping to a new sub, then starting a scan, after the first successful read) wherein dmbase.js is perishing due to mHdr being null.

Implement logout from the message read menu, also

Just need to be able to logout straight from the reading sub-menu, as well as from the primary menu.

Need to talk to Utopia's former sysop regarding continuity of implementation of features really soon here, also.

zap room

This should be the last of the ones to finish up before this sucker is ready to go alpha on the command shells listing, provided the other shit makes a good pass through.

Telegrams arrive asynch

Telegrams seem to be arriving asynchronously, though this was not checked with any serious degree of investigation at this point. When the first test was made, the situation was quite atypical; a test message was sent from my account to my account, which is atypical in itself. Then, I was also logged in on another node at the same time, due to testing I'd previously been handling.

It came in immediately after sending it, arriving before the room name prompt had even popped back up. Another issue with it was that it was in the 'normal' attributes for text, which does not mesh with the proper DOC flow standards. It had no header, footer, etc, obviously.

There needs to be a polling routine set up for this; if not possible through telegrams then we'll have to be setting up a polling routine and a JSON database for usage with it. If it's not now, for this reason, it'll be something else soon enough, anyway. Such as X message logging for history, etc...

After loading debuggers JSON, code falls through in dperuser.js to entering info then breaks

As the following screenshot shows:

Xpress message reception needs some envelope work

At the current time, express messages that are coming in from another user are just dumping text into their socket. This needs to be tidied up a bit. I assumed internal polling would wrap it, but we'll have to attach the glitter 'n sparkles beforehand in the message text block.

Generalize exceptions to a global-level exception method

As per @ntwitch:

(10:16:00) ntwitch: Why don't you make one general exception class, and include it everywhere (in javascript "load" I guess?) and set the name, message, and number on creation.
(10:17:06) ntwitch: throw new dDocException("dispMsg exception", "Out of messages in current sub", 2);
(10:17:31) ntwitch: that would avoid scope errors through your whole project.
(10:17:52) DamoGets: I was thinking about that :'D  I'll do it, you're right it's ridiculous to have all of that

scanSub() and dispMsg() are not properly wrapping to the next sub upon completion of one

@ntwitch had a good suggestion on this one about iterating the sub-board loop outside of dispMsg(), or maybe it was all the way outside of scanSub(), though that kind of takes the point out of scanSub()'s mission. I'm pretty sure it was the former that he was suggesting, though.

Here's the debugging output on the bug as it stands now:

Lobby> xxxx DEBUGING  xxxx   ddoc2.js    prior to log_str
xxxx DEBUGING  xxxx   ddoc2.js    between log_str and log_key
xxxx DEBUGING  xxxx   ddoc2.js    after log_key
In scanSub(); forward = true    user.cursub: Lobby
in while--> tmpPtr: 0
ptr: 0  base.last_msg: 10
Wed, 21 Jan 2015 06:21:58 -0800 from Khelair
  I'm in the process of working out the last of the bugs here.  As soon as the
reverse reading and 's'kip functionality is implemented this base will be
active and utilized by the new DOC shell.

[Lobby> msg #0 (10 remaining)] Read cmd -> ecode2: 0
xxxx DEBUGING  xxxx   dmbase.js    prior to   docIface.log_str_n_char
xxxx DEBUGING  xxxx   ddoc2.js    prior to log_str
xxxx DEBUGING  xxxx   ddoc2.js    between log_str and log_key
xxxx DEBUGING  xxxx   ddoc2.js    after log_key
xxxx DEBUGING  xxxx   dmbase.js    after docIface.log_str_n_char

tmpPtr += 1 = 1
fuggit: false
in while--> tmpPtr: 1
ptr: 1  base.last_msg: 10
ecode2: -2
ptr: 2  base.last_msg: 10
ptr: 3  base.last_msg: 10
ptr: 4  base.last_msg: 10
ptr: 5  base.last_msg: 10
ptr: 6  base.last_msg: 10
ptr: 7  base.last_msg: 10
ptr: 8  base.last_msg: 10
ptr: 9  base.last_msg: 10
Skipping
Working with sub list: [object MsgSub],[object MsgSub],[object MsgSub],[object MsgSub],[object MsgSub]
Entered docIface.nav.skip(), looking for: Lobby
Working with list:
[object MsgSub],[object MsgSub],[object MsgSub],[object MsgSub],[object MsgSub]0: Lobby
Found current sub Lobby in the list
Setting user.cursub to Lobby
Entered try/catch for opening new messages basexxxx DEBUGING  xxxx   dmbase.js    prior to   docIface.log_str_n_char
xxxx DEBUGING  xxxx   ddoc2.js    prior to log_str
xxxx DEBUGING  xxxx   ddoc2.js    between log_str and log_key
xxxx DEBUGING  xxxx   ddoc2.js    after log_key
xxxx DEBUGING  xxxx   dmbase.js    after docIface.log_str_n_char

tmpPtr += 1 = NaN
fuggit: false
in while--> tmpPtr: NaN
ptr: NaN        base.last_msg: 0
ecode2: -2
ptr: NaN        base.last_msg: 0

...after which point the previous line become an infinite loop.

Xpress Not Finding Online Users

As mentioned in the title, the express message subroutine is currently not locating users that are logged in. I'm not sure if this is an error introduced by reorganizing the OO structural layout of the files or if there is a logic error somewhere else. Shouldn't take too long to find and figure out, though.

Message scans must be set off by ddoc, for ddoc only users

In case users select things wrong in the initial configuration, it is imperative that ddoc, when selected as a shell, needs to save the previous settings and set new message scan and new un-read messages to user scan off, as they will not call routines from the appropriate ddoc shell interface.

skip room

needs to be implemented, pending the current room being saved, etc etc etc.

Loop structures at the end of message entry never exit

So yeah, the exterior flow control in the message entry handling structure doesn't exit when you've saved or continued a message... Time to look at that a little more carefully. :P At least abort works so that we're not stuck in an infinite loop any moar. That was an easy fix. :|

Rewrite scanSub()

Get this rewritten to be able to handle its previous behavior of just scanning the one sub; implement a flow control loop outside of it later on to be able to handle jumping to the next sub with unread messages.

At this point I feel a complete rewrite is the best option, as my code here had gotten really nasty from all of the debugging causing me to insert and delete chunks of code until I didn't really have my own head wrapped around it any more.

skip() (and presumably other docIface.nav methods) are not working properly from other objs

A problem has arisen; it is the final one holding this code back from going alpha/beta. It seems that when docIface.nav.skip() is called, it is not able to utilize docIface.util.getRoomList() successfully. It is getting a null list back, which is preventing dmbase.js right now from full implementation of doing a successful message scan. To be precise, it is at the part of the code where it hits the end of the messages in the current room (when doing a forward scan), and it to skip() to the next room. I'm not sure why this is happening, except that it is almost certainly a matter of scope. I'm open for anybody's advice on this one; no doubt my OO understanding of scope is flawed a bit in this matter.

Of course, the docIface object is locate in the top-level ddoc2.js.

Define object separately from functions

This was mentioned by 'tracker1' in Synchronet IRC. I'll ask him for more details soon as I've had a chance to get through the mHdr issue. :)

Modularize dperuser.js

Working on breaking apart the different routines that are often repeated, such as opening files, etc, in order to avoid rewriting so many try/catch blocks. Get rid of the redundant code!

Changeover to new JSON user record format

Pretty much self-explanatory. The original format was foolish; moving to a per-user record structure that makes a lot more sense for parsing and for comprehension. This will be a quick issue to solve, then it'll be back to Issue #39 and then to #40 again.

Back functionality

...from the main menu, that is; not completed, get it done. This one needs to be done after the saving & restoration of the current sub-board entry point and all, probably, as well.

verifyBounds() returns bogus valid code

The aforementioned, yeah.

I'm going to implement exception throws in here to clarify this up a bit, also.

Active base/sub-board preservation/restoration

Need to set up init/exit routines for saving and restoring the currently active message group and sub-board so that these aren't lost upon exit from the ddoc shell.

Posts are leaking into the non-ddoc message groups

For some reason, some of the test messages that I was working with seem to have leaked out from being posted in the correct group (I was trying to put them in DystopianUtopia:Neutron Star for testing) into the Main:Notices area. Obviously this needs to be fixed before the message handling alpha is completed, as well.

'Softcoded' debugging volunteers need to be held out of static code

Current debugging functionality relies on 3 volunteers that are hardcoded into ddoc2.js in order to activate debugging. Along with the implementation of granular debugging functionality (which will need to be stored on a per-user basis), volunteers need to be removed from hardcode and placed into a textfile in the proper location (simplest), or (in line with implementing other per-user fields just for ddoc) in a JSON file or database, in order to set granular features, etc.

Ditch gross multiple user records/debugging and use a per-user JSON object

This will require a rewrite of dperuser.js. If I continue down the path that I'm going, the code is going to end up horribly cluttering the user directory at some point, and I really don't want that, or relying on a sub-directory (though that may become necessary). At this point, adding these JSON per-user records instead will take care of the data handling for multiple issues:

• Issue #37
• Issue #33
• Issue #28 -- at least the 'user' portion of the info
• Issue #34
• Issue #17 -- as far as saving what rooms the user is subscribed to (I'm deciding against going with setting Synchronet's default message scan; we're just gonna keep things separated for reasons that I'll go into later)

More on this later.

Fully Modularize Message Entry

Get that shit broken up. Seriously.

Create a new object to base the handling routines in, separate, and apply properly. Got to get out of this habit of writing linearly.

'Room Info' needs implementation

Users need to be able to hit 'i' at primary menu prompt and potentially the message read prompt, as well, in order to pull up the information such as the room/sub-board's long name, creator/moderator, and blurb about the room (this might end up in an arbitrary directory), etc. A la vdoc style, of course.

dmbase.js organized illogically

While scanning through dmbase.js and other files today in an effort to bring the documentation more up-to-date, it came to my attention that during a late night coding session a lot of the methods in that file, specifically under the msg_base.entry_level sub-object do not belong there. This will require a myriad of small changes in different areas of the code, but won't be tough to implement. Going to try to get this done, as other developers may be coming in to help me polish this up shortly now that the brass tacks are pretty much done.

Shell switch ability

There needs to be a way to select a different shell, or at least set user defaults back to the Synchronet Classic shell (where they will be able to select a different one, if necessary-- puke) in order to get out of the DOCshell once they've entered it for testing, or need to access features that are currently not available in the shell itself.

poast.addMsg() needs recip/sub-board passing

It became apparent, in looking into implementing yell for sysop, that addMsg() would have to be expanded in order to keep from rebuilding the wheel or straight-up cut 'n pasting code. The new 3rd parameter is the message base and the 4th will be the recipient (or null/All). While I'm only thinking about it from the yell for sysop functionality right now, having the capabilities of that method increased will make more configurable additions easier later on.

Figure out a solution for custom user & sub info

There needs to be either a small JSON file deposited in some directories (per-user & per-sub), or a JSON DB subsystem utilized in order to make sure that user info that has no place to be stored in the Synchronet BBS user object can be dealt with in order to emulate DOC more fully. Items that need to be dealt with are:

• user info, for profile usage when it is implemented
• sub-board/room 'i'nfo, for a detailed description of what rooms are for
• user 'd'oing field, again for more full emulation of DOC experience
• user settings integration that fall outside of what is provided by Synchronet

Any additional items will be added in comments below.