lscalese/journal-file-indexer

IRIS Journal file indexer

journal-file-indexer's Issues

Storage optimization

Check if it's possible to optimize the storage.
ex : the class dc.journalindexer.data.SetKillRecord as a property GlobalName. If a global is present 100 000 times in the table, the data is copied 100 000 times. It could be better to save the global name in another table and just keep a reference in dc.journalindexer.data.SetKillRecord.

Create queries to search in indexed journal files

Create pre-defined queries to search into indexed journal files.

Create REST Services

Create REST services from the OAS 3 specification.

Documentation Version 0.2.0

Write the documentation for the features developed in version 0.2.0.

Unit tests for data restore feature.

UI - Table to show journal file records

Create Stats

Create statistique
Count Set\kill operation, size by database, global, processid

Create OpenAPI Specification 3

Pause - Resume - Stop Features

The index process could be very long with big journal file.
Add features to pause\resume or stop the process.

Review version 0.1.0

Before starting development of version 0.2.0, get a review from the staff.

Create Web Socket Server

Index a journal could be long if it's a large journal file.
Create a socket server to notify the client about the progression.

Create tools to search and show results.

Create a tool using pre-defined queries to search and show results.
Keep in mind result must be exposed in a REST service later.

UI - Card to show journal record entry details

Documentation for version 0.4.0

Unit Tests for journal indexer

Create unit tests for the indexer process.

UI - Index Journal

Create a web interface to index an existing journal file on the server.

Documentation about the version 0.3.0

Gather command line tool in ^JRNINDEXER

Currently command line tools are spread in different classmethod.
Gather in one routine ^JRNINDEXER would be more convenient.

Load table stats fail at install

The loading of sql-stats.xml.renamed fail, the system cannot create the directory ${libdir}journal-indexer

add the following entry in the module.xml to fix

<Invoke Class="%File" Method="CreateDirectory" Phase="Compile" When="Before" CheckStatus="true">
     <Arg>${libdir}journal-indexer</Arg>
</Invoke>

Report by Dmitry Maslennikov.

Filter entries to index.

Add a filter system to index only matching journal entries (like ^JRNFILT ).

Add a button to show stats after indexing with success.

UI - List indexed journal

Progresssion page still on the previous journal

When we index a second journal file the progress page show the previous indexing journal status.
The reproduction scenario is not precise yet.

Unit Tests for search feature

Implement unit tests for features developed in version 0.2.0.

Create Index to search by global subscripts

Create an index to search easily by subscripts.

Currently, there is a property GlobalNode (value example : ^%SYS("WQM","Repeat","%CheckGroups") ).

Provide an Initial Tune Stats

For performance reaseon it could be better to provide tune stats for the schema dc_journalindexer_data.

Tool to restore data from indexed journal file

Create Index to search in OldValue and NewValue properties

Create an efficient index to search on OldValue and NewValue properties.
Check if something exists wit iFind or develop a functional index.

These properties could be datatype or Listbuild. I could be very interesting to perform a search on each element of a listbuild.

Wizard to create a filter for command line tools

Currently we have to put in terminal a string with the filter in json format.
It isn't convenient, so we have to develop a wizard to create the json string automatically.

Secure /jrnindexer/api with jwt auth

Secure the api with JWT auth and create a login page (angular app).

UI - Upload interface

Add interface to upload a journal file with drag and drop feature.

Design tables for journal

Design tables to store journal file information.
The indexer needs to temporarily store journal file information in a table.

Create all needed tables to store all records of a journal file.
It could follow the structure of %SYS.Journal classes.

Bad libel in tab stats PID

The tab PID in stats page as a column name "Database" but it must be "PID".

ui - Form to restore a old value or new value

UI - Show a loading image instead of text "no data" when stats page is opened

Currently, the screen with indexed journal information shows "no data" even if a request is in progress to load data.
It could be better to show loading icons when a request is pending.

Documentation for indexer process

Write the readme file for the version 0.1.0.
Must contain:

• installation details (OEX, Docker)
• how to index a journal file programmatically
• how to search in the journal file with SQL queries

Journal Indexer Process

In order to perform a high-speed search in a journal, develop a process dedicated to parsing de journal file and indexing the content in temporary tables.

The input could be a stream or a file path.

UI - Advanced filter to search journal data