Create a deployable executable on windows for fTbus

[marouanehassanioptimistik]

Create a deployable executable on windows for fTbus

[kukukk]

Based on the package description yes, it should be achievable.

This command line interface enables you to package your Node.js project into an executable that can be run even on devices without Node.js installed.

But I will say for sure only when I tested it.

[kukukk]

I made a quick test with pkg (https://www.npmjs.com/package/pkg) to see if it works well with SQLite, but I only tested it on Linux (Ubuntu).
Theoretically it is able to generate executable for Linux/Windows/macOS, on the same platform, but because SQLite uses native binaries it may be required to generate the executable for every platform separated.

I will check it.

[marouanehassanioptimistik]

Ok i do not see any issue to create separate executable per platform as long as we can automate in jenkins the creation of the distribution (which shouldn't be too much of a problem I think)
Ideally also it should have no external dependency even on nodejs, do you think it is achievable?

[kukukk]

One more thing... Do you have a list which Windows versions and Linux distributions/versions do you want/have to support?

[kukukk]

Native binaries are supported, but packaging them inside the executable is not resolved yet. It means that we have to put the native SQLite binary to the same folder as the executable.

Ubuntu 18.04
I packaged the application on Ubuntu 18.04 where Node.js was installed and moved to an another Ubuntu 18.04 installation without Node.js installed. The folder contained:

• fTbus executable
• node_sqlite3.node native binary built for Linux
• fTbus.json config file

The application was working.

Windows 10
I packaged the application on Windows 10 where node was installed and moved to an another Windows 10 installation without Node.js installed. The folder contained:

• fTbus.exe executable
• node_sqlite3.node native binary built for Windows
• fTbus.json config file

The application was working

Summary
It means that when distributing the application we will have to put together the executable and the native SQLite binary. E.g: zipping them together.

Note:
If you want to have only the executable we will have to look for another packager (I don't know yet if there is any other that works as we need) or to switch from SQLite to an another fully Javascript based database implementation, like https://www.npmjs.com/package/lowdb. It has almost 1.5 million downloads a week, so it seems to be quite popular. However, they say that it is not recommended with big databases (~200 MB).

[marouanehassanioptimistik]

One more thing... Do you have a list which Windows versions and Linux distributions/versions do you want/have to support?

For windows I would say from windows 7
For linux no specific requirement for now as there is no real licensing issue

[marouanehassanioptimistik]

Summary
It means that when distributing the application we will have to put together the executable and the native SQLite binary. E.g: zipping them together.

Fair enough it should be ok to zip them together and distribute the zip i do not necessarily require a single file as long as for the client it is as easy as unzipping a distribution.

Two questions however :

• where are the data on file system stored ?
• can the package be registered as windows service?

[kukukk]

• where are the data on file system stored ?

Both for Windows and Linux I used the following setting to define the caching location:
"cacheFolder": "./cache"
This way the cache folder was created in the same folder where the executable was located. With an absolute path, I think, we can store data anywhere on the file-system with the condition that we have write access to it.

• can the package be registered as windows service?

I have never tried to create and register a Windows service, so I can't tell you. But I can try to do it, if you want.

Fair enough it should be ok to zip them together and distribute the zip i do not necessarily require a single file as long as for the client it is as easy as unzipping a distribution.

Another option on Windows would be creating an installer, instead of using a zip. Maybe with an installer we can also register it as a Windows service.

[marouanehassanioptimistik]

Both for Windows and Linux I used the following setting to define the caching location:
"cacheFolder": "./cache"
This way the cache folder was created in the same folder where the executable was located. With an absolute path, I think, we can store data anywhere on the file-system with the condition that we have write access to it.

ok that is good

I have never tried to create and register a Windows service, so I can't tell you. But I can try to do it, if you want.

From my knowledge (but I could very well be wrong) registering an exe as a service cannot be done with any exe, the exe should implement something special for windows to recognise it. I would be interested if you can look into it as it eases greatly the deployment and management by the client.

Another option on Windows would be creating an installer, instead of using a zip. Maybe with an installer we can also register it as a Windows service.

Installer or zip is for me just a question of taste for unless that changes something regarding service registration

[kukukk]

Unfortunately, it seems that you were right about registering an executable as a service. After a very basic research I see 3 different approaches:

• create a native Windows service application and register it with sc.exe (not sure with Node.js we can do this)
• use some tools to convert an executable to a Windows service, like http://nssm.cc or https://github.com/kflu/serman
• use https://www.npmjs.com/package/node-windows, but it requires node. Maybe we could pack the standalone node executable into our installer and use it. But it's crazy and complicated...

Some link I found (for further documenting):

• https://stackoverflow.com/questions/3582108/create-windows-service-from-executable
• https://www.paddingleft.com/2018/05/29/install-nodejs-program-as-windows-service-by-nssm/
• http://www.rioki.org/2016/02/01/windows-services-nodejs.html
• https://svenweller.wordpress.com/2016/07/01/howto-node-windows-service/

At this point I would do some further research with nssm and serman. If we can install our application as a Windows service with any of them without having to install anything else, probably we can create an installer which includes every files we need, put them in Program Files/fTbus and install fTbus.exe as Windows service.

[kukukk]

Using nssm I managed to install a Windows service and be able to start/stop it.
To test it I enabled caching and both TimescaleDB and InfluxDB north app and checked the cache folder. The SQLite databases for both applications were generated.

I also was able to install/start/stop/remove the service from command-line without any user intervention.

If we can use an installer where we can execute custom commands we can install the application (fTbus.exe, node_sql3.node and nssm.exe) into Program Files, register an "fTbus" service and start it.
Other option is to provide as a zip file with an additional install.bat which will simply execute the neccessary command.

[marouanehassanioptimistik]

I also was able to install/start/stop/remove the service from command-line without any user intervention.

Good that is the goal!

If we can use an installer where we can execute custom commands we can install the application (fTbus.exe, node_sql3.node and nhssm.exe) into Program Files, register an "fTbus" service and start it.
Other option is to provide as a zip file with an additional install.bat which will simply execute the neccessary command.

I do not mind using an installer or install.bat bundled in zip as long as all it requires from the IT user responsible of the install is double click :-) . Maybe the install.bat approach is easier and more readable? @kukukk what do you think?

[marouanehassanioptimistik]

Using nssm I managed to install a Windows service and be able to start/stop it.

I have one question though nssm is bundled with ftbus distribution or should be present on target server?

[kukukk]

Using an install.bat is much more easier. These are the commands I had to execute to install/start and stop/remove the service:

Install
nssm.exe install fTbus "C:\Users\Szabolcs Balogh\Downloads\fTbus\fTbus.exe"
nssm.exe start fTbus

Uninstall
nssm.exe stop fTbus
nssm.exe remove fTbus confirm

We will have to work on it a little bit to dynamically get the current directory instead of hard-coding it, but it should not be a problem even in a batch file. Another complication can be caused by the platform, if we want to support both 32bit and 64bit versions (nssm has different executable for 32bit and 64bit).

This way, probably, we can also create the zip file during the build process.

Note:
I still have to test it on Windows 7.

Update
nssm is a standalone executable which can be bundled together with fTbus. Doesn't have to be installed on the target server.

[marouanehassanioptimistik]

Using an install.bat is much more easier. These are the commands I had to execute to install/start and stop/remove the service:

Ok we should go for that then as i do not see any big drawback in this approach

if we want to support both 32bit and 64bit versions (nssm has different executable for 32bit and 64bit).

We can limit to 64bits version in this v0 this doesn't seem too problematic to me as well

I still have to test it on Windows 7.

If it can work on windows 7 that would be great as some of our client base still have windows 7 licences only.

nssm is a standalone executable which can be bundled together with fTbus. Doesn't have to be installed on the target server.

Ok that is clear

[kukukk]

I was able to install fTbus as a Windows service with nssm on Windows 7. The test was the same: check if the SQLite databases are created in the cache folder.

Should I test every south driver and north application to be sure that everything works as expected?

Note:
If fTbus is running as a Windows service there will be no visual feedback if the application crashes. It means that if something is not right with the config file, it won't start, and the only thing the client will see is that the service is Paused instead of Running.

[jfhenon]

The web server integrated in the engine will be the way to monitor the health of fTbus. We will define the right parameters to display but at least the current information displays the version number meaning it’s running. In the future, we can display informations about the size of the cache(s), the number of addValue could be tracked, the status of North/South applications etc… JF Le 31 janv. 2019 à 14:57, Szabolcs Balogh <[email protected]<mailto:[email protected]>> a écrit : I was able to install fTbus as a Windows service with nssm on Windows 7. The test was the same: check if the SQLite databases are created in the cache folder. Should I test every south driver and north application to be sure that everything works as expected? Note: If fTbus is running as a Windows service there will be no visual feedback if the application crashes. It means that if something is not right with the config file, it won't start, and the only thing the client will see is that the service is Paused instead of Running. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#91 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ATdSnb3sWVNvBd6Vxl6xyXckNe-YYCy7ks5vIvY-gaJpZM4aXoGY>.

[marouanehassanioptimistik]

If fTbus is running as a Windows service there will be no visual feedback if the application crashes. It means that if something is not right with the config file, it won't start, and the only thing the client will see is that the service is Paused instead of Running.

In addition to what @jfhenon suggested which will work only if ftbus has not crashed to provide admin feedback Winston should help us log crash to disk for debug purposes right? Maybe if it is not already done we should catch any exception, log it to log file before throwing it back? What do you think?

[marouanehassanioptimistik]

Should I test every south driver and north application to be sure that everything works as expected?

@kukukk Do you anticipate any strange behaviour for specific app? In my opinion we do not really rely on the OS features except for some file writing/reading which are abstracted by nodejs. I would say for now the testing you have done is sufficient. I will create a separate issue if we feel we require further testing

[kukukk]

When I spcified the cache folder in fTbus.json I used "\" instead of "/" and it was not able to handle the path and it crashed. There was no web-server to monitor the health of fTbus. The only way to see that something is wrong was to check the status of the service, which was "Paused" instead of "Running".

[jfhenon]

Well, if the console is not working, it’s also an indication that something went wrong Le 31 janv. 2019 à 15:34, Szabolcs Balogh <[email protected]<mailto:[email protected]>> a écrit : When I spcified the cache folder in fTbus.json I used "\" instead of "/" and it was not able to handle the path and it crashed. There was no web-server to monitor the health of fTbus. The only way to see that something is wrong was to check the status of the service, which was "Paused" instead of "Running". — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#91 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ATdSnZ_NaO4GLBgCZHdg0NDP0k9Cv17Mks5vIv8OgaJpZM4aXoGY>.

[jfhenon]

Pkg has been set for the issue #92 but for the NT service part do we need to change something in the code or is it enough to document in the readme?

[kukukk]

I managed to write an install.bat to install fTbus as Windows service and start it, and uninstall.bat to stop the fTbus service and remove it.

install.bat

@echo off

title Install fTbus as Windows service

echo Administrator permissions required. Detecting permission...

net session >nul 2>&1
if %errorLevel% == 0 (
    echo Installing fTbus as Windows service...
    START nssm.exe stop fTbus
    START nssm.exe install fTbus "%cd%\ftbus-win.exe"
    START nssm.exe set fTbus AppDirectory "%cd%"

    echo Starting fTbus service...
    START nssm.exe start fTbus
) else (
    echo No Administrator persmission. Please run Command Prompt as Administrator
)

uninstall.bat

@echo off

title Remove fTbus from Windows service

echo Administrator permissions required. Detecting permission...

net session >nul 2>&1
if %errorLevel% == 0 (
    echo Stopping fTbus service...
    START nssm.exe stop fTbus

    echo Removing fTbus service...
    START nssm.exe remove fTbus confirm
) else (
    echo No Administrator persmission. Please run Command Prompt as Administrator
)

To work, the Command Prompt must be started as Administrator. Both scripts check if they have Administrator permissions. If yes, they will add/start and stop/remove fTbus to/from Windows service. If Command Prompt was not started as Administrator they will just echo a message that they need Adminsitrator permission.

For them to work we have to put the following files in the same folder (and zip together):

• nssm.exe
• ftbus-win.exe
• node_sqlite3.node
• install.bat
• uninstall.bat

Is it enough to make fTbus available or do we need a better installer?

[jfhenon]

I believe it’s sufficient. JF On 13 Mar 2019, at 13:01, Szabolcs Balogh <[email protected]<mailto:[email protected]>> wrote: I managed to write an install.bat to install fTbus as Windows service and start it, and uninstall.bat to stop the fTbus service and remove it. install.bat @echo off title Install fTbus as Windows service echo Administrator permissions required. Detecting permission... net session >nul 2>&1 if %errorLevel% == 0 ( echo Installing fTbus as Windows service... START nssm.exe stop fTbus START nssm.exe install fTbus "%cd%\ftbus-win.exe" START nssm.exe set fTbus AppDirectory "%cd%" echo Starting fTbus service... START nssm.exe start fTbus ) else ( echo No Administrator persmission. Please run Command Prompt as Administrator ) uninstall.bat @echo off title Remove fTbus from Windows service echo Administrator permissions required. Detecting permission... net session >nul 2>&1 if %errorLevel% == 0 ( echo Stopping fTbus service... START nssm.exe stop fTbus echo Removing fTbus service... START nssm.exe remove fTbus confirm ) else ( echo No Administrator persmission. Please run Command Prompt as Administrator ) To work, the Command Prompt must be started as Administrator. Both scripts check if they have Administrator permissions. If yes, they will add/start and stop/remove fTbus to/from Windows service. If Command Prompt was not started as Administrator they will just echo a message that they need Adminsitrator permission. For them to work we have to put the following files in the same folder (and zip together): * nssm.exe * ftbus-win.exe * node_sqlite3.node * install.bat * uninstall.bat Is it enough to make fTbus available or do we need a better installer? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#91 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ATdSndkMCrhGEZGyM0DjhCVCvQAvnfNTks5vWOiBgaJpZM4aXoGY>.