The praefectus from fornever

Get rid of IPositionedSequence<'a>.AllowedToInsertAtArbitraryPlaces

We have the a TODO in the following code:

praefectus/Praefectus.Core/Diff/Types.fs

Lines 8 to 13 in 9970a26

    
           type IPositionedSequence<'a> = 
        
               // TODO: Drop this flag, since it is the only mode we use in production code 
        
               abstract AllowedToInsertAtArbitraryPlaces: bool 
        
               abstract MaxCoord: int 
        
               abstract GetItem: coord: int -> 'a option 
        
               abstract AcceptsOn: coord: int * item: 'a -> bool

It should be fixed.

Integration tests

As discussed in #18, we'll need integration tests that directly run our binary under various conditions and test it from the outside.

Improvement for the diff algorithm

As explained in the corresponding article, we currently have a problem in the diff algorithm implementation: it won't step over several empty columns in one step, which leads to some transformations not being as efficient as they could (I believe there're a couple of examples in the current tests even).

This should be fixed.

Taskwarrior integration node

There should be a fully-fledged integration with Taskwarrior:

we should be able to read all the changes from Taskwarrior instance
we should be able to export changes from Praefectus back to the Taskwarrior
merge changes on import/export properly
search for data conflicts of any kind, and report them to the user

Depends on:

#10 (since we need configuration to seek for Taskwarrior instance and to know whether the integration is enabled at all)
#9 (since we need to store data a lot)
#6 (since we need to know what to store)
#8 (since Taskwarrior integration is a separate node by design)
#7 (since we need to know what/how to merge)

Foundational concept design: version control

RFC link: Version Control

Praefectus should implement a simple version control system (VCS) for the information it contains. Every task should have a fully persisted change history with information about the nodes that were involved in changes.

The history should be a DAG (the same as in Git), since multiple tasks created within different "branches" could then be merged by the user into one entity.

Should every node maintain its own branch? How should import/export process cooperate with that? We'll need to solve these questions in this task.

RFC is ready, so this may be closed after finishing the initial implementation:

Configuration system

RFC link: Configuration

There should be a configuration system for Praefectus: the config file it reads the data location/integrations/settings from. Probably may be modelled after the Taskwarrior config (~/.task or whatever it uses).

Depends on:

#5

Simple data storage

Praefectus should store its data somehow. We'll need to decide, how to implement the data storage. Points of interest:

create a storage abstraction (since eventually I want to support different storage kinds for different tasks, e.g. a server instance of Praefectus may use PostgreSQL while a (mobile) client may use SQLite)
do we need SQL at all? Could we use NoSQL? Are there any reliable NoSQL solutions?
seek for the best managed storage solution (LiteDB? SQLite.Managed? What else?) since I prefer not to rely on any native code unless absolutely necessary

Depends on:

#5

Enable Markdown linter for the documentation

Our documentation occasionally contains stale links; we should automate the cleanup process.

Use case: import GitHub issues and export to Taskwarrior format

I want Praefectus to be able to import GitHub issues (from some scope, say, several repositories), and then export it to Taskwarrior format.

Simple UI/UX to navigate the database

RFC link: Command Line Interface

While the UX ideas aren't finalized yet, we'll need a simple UI already on this work stage.

So, I propose to introduce some basic command line parsing and command set similar to the one of Taskwarrior, which will work with our database and will help to perform some basic task maintenance set.

It'll need to be properly documented, but I impose no requirements to finalize it right now.

Depends on:

#5 (we'll need a basic app to work with)
#9 (we'll need a database to operate on)
#6
#7
#8 (we'll need all the basic design tasks to be solved before we can start adding any terminal UI)

Praefectus Prime

I've coined the minimal viable implementation that's required for me to start using Praefectus. This will be called "Praefectus Prime", as the first and initial implementation, core of everything else.

It will manage tasks according to the following list of requirements:

all tasks are stored in a list of Markdown files
every task may include a structure of checklists
each task may have several tags, they're prioritized based on tags
some tasks have constraints ("do this before that"), should affect order

The primary directives of Praefectus Prime are to:

keep list of what should be done next
reorder the tasks according to changing priorities, for me to stay focused
get rid of finished tasks (archive them somehow, usually by tagging them in a special way and removing after a grace period)
keep a record of backups (in a Git repo)

The user interaction should be based on ~~project [EXPUNGED]~~ declassified: project Fabricator.

All other priorities suspended.

Plan:

Taskwarrior integration tests

I'd like to thoroughly test our Taskwarrior integration capabilities via the integration tests (as opposed to the unit tests). We'll still write tests in scope of #11, but there should be a separate integrational test suite.

Depends on:

#11

Implement basic task management

Implement basic task management as described in the tasks RFC (#6).

Depends on:

#39

Basic F# infrastructure

The corresponding PR, #16, contains an updated task list for this issue. The list below is preserved as a historical artifact/initial task definition.

console app project template
multi-platform CI
unit tests (some dummy test for now)
basic packaging (runtime-dependent deployment to run without .NET Core SDK, but with runtime only)
basic configuration (at least some static log location)
logging framework

Publish trimmed binaries

Currently, our builds are pretty heavy: for example, size of unpacked "self-contained" variant of Praefectus for Windows is about 71 MiB (and about 30 MiB if zipped with "Ultra" compression level).

One easy way to significantly decrease the package size is to publish the binaries trimmed by dotnet linker (also known as Mono linker). To do that, all we'll need is to add /p:PublishTrimmed=true to our publish command. In my experiment, dotnet linker have decreased the unpacked application size to 30 MiB (16 MiB zipped), which is great.

It comes at a price though. Dotnet linker has its constraints, and it, by specification, may make mistakes (i.e. remove the code that is accessed at runtime, causing runtime errors when this code is called), because some of the classes/methods may be accessed in unusual way (with reflection and such). There's additional configuration that should be applied to dotnet linker in such cases.

The common recommendation for such case is to perform integration testing of resulting binaries: if these integration tests pass, then it means the binaries are good. That means, this task depends on integration test availability; we cannot publish trimmed binaries until we're sure they work well.

Good news are if we'll have the integration tests, this unblocks other significant improvements we may do with our publishing process: we may use any AOT engine, such as CoreRT, or even rewrite Praefectus in any programming language that does unit testing differently, and we'll still be able to verify the resulting binaries using our integration test suite.

Depends on:

#20 (because we need to set up the integration test suite first)

Foundational concept design: task attributes

RFC link: Tasks and Attributes

We'll need a set of the most basic task management concepts: tasks and their attributes, including:

basic attributes: task title, creation date
extended attributes (various Taskwarrior stuff, including Taskwarrior task UUID, priority, tags etc.)
volatile attributes (Taskwarrior)
cross-task dependencies
~~an algorithm to detect dependency loops (Taskwarrior disallows them, but they still may be introduced if a conflict occurs)~~

Depends on:

#22
#39

Establish project RFC process

As this is a project to get things done, I'd like to establish more or less formal process of new fundamental feature development.

TL;DR: this project will use tailored GitHub flow for branch/merge policy, and a simple RFC system for fundamental features in development. This issue will establish the first RFC and, successively, the first fundamental piece of documentation.

Utilize RestoreLockedMode

To properly utilize the NuGet lock files, we'll need to set the RestoreLockedMode key in our build system. See the details here.

Currently, other set of NuGet packages may be resolved for building platform-dependent packages; we may choose to enforce --no-restore on publish, and add the platform-dependent packages to the lock files.

Version Control: merge algorithm implementation

Implement the object merge algorithm according to the Version Control RFC.

Depends on:

#32

Scheduling: allow to schedule tasks for the last day of month

There's a small feature I'd like to file to not forget about it. Probably it could be implemented in some bigger scope, when designing the scheduling system, but for now let it to just be here.

Taskwarrior periodic/recurrent tasks lack flexibility, and one particular feature I need for my schedule is to be able to schedule tasks to be performed in the last day of month (e.g. some rent payments or whatever).

When we'll be designing our scheduling system, we'll need to add this feature beyond the "basic" Taskwarrior-like feature set.

This task depends on the following:

#6 (since task schedule will likely be an attribute)

The dependency list isn't closed, since the scheduling feature isn't planned yet.

Version Control: data type implementation

Implement the data types from the Version Control RFC, and integrate them with the rest of the task/attribute data model.

Depends on:

#22

Add property-based tests for diff

Currently, diff algorithm has some manual tests, but it could still be susceptible to various kinds of problems.

I believe that a good step forward would be to add property-based tests in addition to the current diff test suite.

Minimum Viable Product

This issue is a milestone. It groups other issues and gives a bit of motivation for the grouping and prioritizing of them. It is expected that no work will be done in scope of the issue itself; all the work will be done in the issues it depends on.

I didn't wrote anything on this kind of issues in the process document since I expect to use them only on the start of the project, for few first releases, and they aren't important part of the RFC framework.

I want Praefectus to first interop with, and then – eventualy – surpass, the Taskwarrior in my daily routine. Taskwarrior will be the first integration Praefectus gets, and the main/flagship one (for a while, at least).

So, this release will be mostly dedicated to providing the basic infrastructure/fundamentals, and then clean interoperation mode with Taskwarrior.

Our main goals on this stage:

#5: provide the infrastructure: project template, build, CI, logging
#6: design the basic concepts that will be required for integration (tasks, task attributes)
#8: design the most basic concepts for multi-node support (I thought we could live without that for this milestone, but it looks like we won't able to)
#9: establish the data storage (the simplest one for further maintainership/migration)
#11: provide a Taskwarrior node (either via standard import/export mechanism or via the direct database read/write, preferably the former one)
#12: establish integration test process for the Taskwarrior import/export
#13: provide a simple UX (probably mimic the Taskwarrior one)
#14: full data backup and restore

Essentially, I want to start using Praefectus instead of Taskwarrior at least sometimes during implementation of this milestone.

Here's a list of non-goals to clarify a bit:

full multi-node synchronization
scripting
AI
even a little bit finalized UX
scheduling (outside of features that are already in the Taskwarrior)

Version Control: validation

Implement the Version Control validation algorithms as described in the Version Control RFC, and integrate them with the diagnostic framework created in #22.

Depends on:

#22
#32

Data backup and restore

One of the points of Praefectus is simple data backup and restore. We'll need a simple command that performs a backup, and another one that restores a backup.

When restoring a backup, it should try to merge the data from the backup with the actual data, and also there should be an option to completely destroy the current data and replace it with the backup.

We should take precautions to not lose everything due to the user accidentally performing a descructive backup operation: maybe disallow that on a non-empty database, or ask for additional confirmation, or perform a forced backup before destructive restore.

Depends on:

#13 (since we need a UI to add the commands to)
#9 (since we need the data to work on)
#7 (since we need a data merge algorithm)

Publish framework-dependent packages

Currently we're only publishing the framework-independent packages for supported operating systems, which mean that the only external requirements for our packages are that of .NET Core itself.

It could be good for most cases, but is problematic in terms of package size: even if we apply recommendations from #18, the compressed package is still about 14 MiB, which is very significant for an application that only prints its version to the logs and does nothing else. Compressed framework-dependent package may only take 1.8 MiB.

Thus, I suggest we also publish framework-dependent package for people who have an external runtime already installed.

To do that, it's enough to pass --self-contained false to our publish command.

We'll still publish separate packages for all the supported platforms, because the set of native libraries is different, which could create some troubles e.g. if Windows binaries are in macOS app package and require special signing procedure.

Foundational concept design: multi-node support

Praefectus should support working with multiple nodes. There'll be multiple node from the very beginning: there'll be a Taskwarrior node (with import/export capabilities), and a Praefectus user-facing node (so the user may customize/change the tasks imported from Taskwarrior, with the changes eventually being synchronized back to Taskwarrior).

Here're the questions we'll need to solve while designing this stuff:

should task attributes belong to individual nodes and/or to node types and/or to the global namespace?
how the attributes should be merged if multiple nodes have changed the same attribute?
should we provide a pluggable/configurable merge algorithms?

Depends on:

#6 (since we need to decide on the sorts of information we store in Praefectus first)
#7 (since we need to know how the branching/merging will work before applying it to the multi-node practice)

Simplify the task management system

Current task management with everything packed into attributes is too heavy to implement. I think we should scale it down to user-defined attributes, and define embedded attributes in less agnostic way. Otherwise, it seems like I develop a general-purpose database API with no actual purpose whatsoever.

It's not true that Taskwarrior has no scopes

In the README section “Why not Taskwarrior?”, point 9 says:

Taskwarrior has no "scopes" (e.g. for home and office work).

This isn't true: Taskwarrior has contexts, which are essentially filters that are applied automatically and that you can switch between.

	type IPositionedSequence<'a> =
	// TODO: Drop this flag, since it is the only mode we use in production code
	abstract AllowedToInsertAtArbitraryPlaces: bool
	abstract MaxCoord: int
	abstract GetItem: coord: int -> 'a option
	abstract AcceptsOn: coord: int * item: 'a -> bool

fornever / praefectus Goto Github PK

praefectus's People

Contributors

Stargazers

Watchers

praefectus's Issues

Recommend Projects

Recommend Topics

Recommend Org