Add Windows Support

[rotki]

Add Windows Support

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

The funding of 250.0 DAI (250.0 USD @ $1.0/DAI) attached to this issue has been approved & issued to @cryptomental.

• Learn more on the Gitcoin Issue Details page
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $19,221.43 more funded OSS Work available on the Gitcoin Issue Explorer

[cryptomental]

After going through dozens of Microsoft-related errors and googling I discovered something very important. A prerequisite that installs Visual Studio and Microsoft SDK + python 2.7

npm install --global --production windows-build-tools --add-python-to-path

that has to be installed from an elevated shell (run as admin), has a bug. It returns with a 0 code from the installer when only part of the binaries were installed (visible in the screenshots below). I discovered the process is still running after Windows reboots and installs lots and lots of packages, currently I am on a bit slow connection and it takes a while to download them all. I am pasting below a few screenshots that may be useful in the future.

I suspect that AppVeyor's VM images have all the tools already built-in.

I will wait until the windows build toolchain install completes on my VM. To be continued tomorrow.

[cryptomental]

HI @LefterisJP All right it makes sense to leave it open. I will experiment a bit today but I suspect that it will take me longer than that. Let's see how it goes.

[cryptomental]

I will now press 'stop work' since I will be unavailable for the next three weeks to allow anyone to pick up the bounty and continue working on this. @LefterisJP feel free to continue after you are back from holidays and/or redistribute the work as I am not sure if I will have enough time after I come back.

The needed Windows software on AppVeyor's VM are:

• Visual Studio 2015
• OpenSSL 1.0.2o
• TCL/TK 8.5.18
• Python 3.6
• Python 2.7 for Electron rebuilt
• ZeroMQ 4.0.4 http://zeromq.org/distro:microsoft-windows (also missing in Chocolatey, there will be definitely issues to install it there)
• NodeJS/NPM

OpenSSL include files and libs must be on VisualStudio search patch to be able to build sqlcipher (encrypted sqlite) and pysqlcipher3 (Python3 wrapper around it).

OpenSSL DLLs must be manually copied from OpenSSL installation folder to c:\windows\system32 (or corresponding folder for x64 Windows), otherwise pysqlcipher3 will silently fail (this took me a lot of time to figure out).

Lots of clues and required path settings can be seen in the screenshots that I pasted above.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 250.0 DAI (250.0 USD @ $1.0/DAI) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $28,734.53 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

@mumuss Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• warning (3 days)
• escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[gitcoinbot]

@mumuss Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• warning (3 days)
• escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[gitcoinbot]

@mumuss Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• warning (3 days)
• escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[gitcoinbot]

@mumuss Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• warning (3 days)
• escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[cryptomental]

I last ran Windows a couple years back, but was curious and wanted give it a try.

I spent around 1 hour on installing fresh Windows 7 Virtualbox VM from scratch, then installed

• VSCode
• GIT
• nodejs and npm
• Python27
• Visual Studio redistributable 2008
• Windows SDK for Windows 7

And currently blocking thing is msgpack not being able to compile with .NET 4.0

C:\Users\IEUser\rotkehlchen\node_modules\msgpack>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
C:\Program Files\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppBuild.targets(297,5): warning MSB8003: Could not find WindowsSDKDir variable from the registry
.  TargetFrameworkVersion or PlatformToolset may be set to an invalid version number. [C:\Users\IEUser\rotkehlchen\node_modules\msgpack\build\deps\msgp
ack\libmsgpack.vcxproj]
C:\Program Files\MSBuild\Microsoft.Cpp\v4.0\Platforms\Win32\Microsoft.Cpp.Win32.Targets(147,5): error MSB6006: "CL.exe" exited with code -1073741515. [
C:\Users\IEUser\rotkehlchen\node_modules\msgpack\build\deps\msgpack\libmsgpack.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:258:23)

Unfortunately msgpack requires the Windows SDK with .NET 3.5 but it is no longer available for download https://www.microsoft.com/en-us/download/details.aspx?id=24826

Perhaps AppVeyor provides this out-of-the-box but it is required to run on Windows anyway.

I will try to go back to this on Monday and google a bit, perhaps I will find a solution.

[cryptomental]

I found a hint there nodejs/node-gyp#679 and https://github.com/nodejs/node-gyp#installation.

[LefterisJP]

Hey @cryptomental nice to see you here. If you want to tackle this it would be great! Thanks for all the hints.

[vs77bb]

@cryptomental You've been approved!

[cryptomental]

Thank you! I am starting today with a fresh VM and will start from scratch.

[cryptomental]

Today's update: windows build toolchain with Python 2.7 installed fully, I installed Python 3.6 on top, all dependendencies, was able to build msgpack that was previously failing.

Although all Python dependencies seem to install correctly, there is a problem with pysqlcipher3 which is an encrypted sqlite3. I was able to install the npm version of sqlcipher but the Python3 package installs without building and will likely need to be installed manually from source. The instructions are there https://github.com/rigglemania/pysqlcipher3 but it will be a bit cumbersome.

After pysqlcipher I suspect the next big part will be to automate everything in AppVeyor. To be continued tomorrow...

[LefterisJP]

Ah thank you @cryptomental. Yes I also suspect pysqlcipher would be the hardest thing to do as when I checked it out windows seemed a bit more hard to support.

[cryptomental]

I spent a few hours fighting with sqlcipher, installed all dependencies, followed tutorials, currently stuck at

c:\Users\IEUser\pysqlcipher3-win32\sqlcipher>nmake /f Makefile.msc sqlite3.c

Microsoft (R) Program Maintenance Utility Version 14.00.24210.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        copy /Y tsrc\sqlite3ext.h sqlite3ext.h
        1 file(s) copied.
        tclsh .\tool\mksqlite3c.tcl
        copy tsrc\shell.c .
        1 file(s) copied.
        copy .\ext\session\sqlite3session.h .
        1 file(s) copied.

c:\Users\IEUser\pysqlcipher3-win32\sqlcipher>nmake /f Makefile.msc

Microsoft (R) Program Maintenance Utility Version 14.00.24210.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        copy /Y tsrc\sqlite3ext.h sqlite3ext.h
        1 file(s) copied.
        tclsh .\tool\mksqlite3c.tcl
        copy tsrc\shell.c .
        1 file(s) copied.
        copy .\ext\session\sqlite3session.h .
        1 file(s) copied.
        cl -nologo -W4 -DINCLUDE_MSVC_H=1   -DSQLITE_OS_WIN=1 -I. -I. -I.\src -fp:precise -MT -DNDEBUG -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -DSQLITE_THREADSAFE=1 -DSQLITE_THREAD_OVERRIDE_LOCK=-1 -DSQLITE_TEMP_STORE=2 -DSQLITE_HAS_CODEC  -DSQLITE_MAX_TRIGGER_DEPTH=100  -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_COLUMN_METADATA=1   -O2 -Zi -Fosqlite3.lo  -c sqlite3.c
sqlite3.c
sqlite3.c(18783): error C2065: 'OP_String8': undeclared identifier
sqlite3.c(18784): error C2065: 'OP_ResultRow': undeclared identifier
sqlite3.c(20735): warning C4456: declaration of 'rc' hides previous local declaration
sqlite3.c(20695): note: see declaration of 'rc'
sqlite3.c(22000): error C2065: 'OPFLG_INITIALIZER': undeclared identifier
sqlite3.c(22000): error C2099: initializer is not a constant
sqlite3.c(41873): error C2059: syntax error: 'type'
sqlite3.c(41904): error C2061: syntax error: identifier 'winShm'
sqlite3.c(44200): error C2143: syntax error: missing ')' before '*'
sqlite3.c(44200): error C2143: syntax error: missing '{' before '*'
sqlite3.c(44200): warning C4431: missing type specifier - int assumed. Note: C no longer supports default-int
sqlite3.c(44200): error C2371: 'sqlite3_int64': redefinition; different basic types
sqlite3.c(1303): note: see declaration of 'sqlite3_int64'
sqlite3.c(44200): error C2146: syntax error: missing ';' before identifier 'iOffset'
sqlite3.c(44200): warning C4218: nonstandard extension used: must specify at least a storage class or a type
sqlite3.c(44200): error C2059: syntax error: ')'
sqlite3.c(44200): error C2054: expected '(' to follow 'iOffset'
sqlite3.c(44258): error C2143: syntax error: missing ')' before '*'
sqlite3.c(44258): error C2143: syntax error: missing '{' before '*'
sqlite3.c(44258): error C2059: syntax error: ','
sqlite3.c(44258): error C2059: syntax error: ')'
sqlite3.c(44259): error C2143: syntax error: missing ')' before '*'
sqlite3.c(44259): error C2143: syntax error: missing '{' before '*'
sqlite3.c(44259): error C2059: syntax error: ')'
sqlite3.c(44259): error C2059: syntax error: ';'
sqlite3.c(44275): error C2065: 'winFile': undeclared identifier
sqlite3.c(44275): error C2297: '*': illegal, right operand has type 'int *'
sqlite3.c(44275): error C2059: syntax error: ')'
sqlite3.c(44286): warning C4013: 'winUnmapfile' undefined; assuming extern returning int
sqlite3.c(44290): error C2223: left of '->h' must point to struct/union
sqlite3.c(44290): error C2198: 'function through pointer': too few arguments for call
sqlite3.c(44314): error C2223: left of '->h' must point to struct/union
sqlite3.c(44321): error C2223: left of '->zPath' must point to struct/union
sqlite3.c(44321): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
sqlite3.c(44321): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 4
sqlite3.c(44321): error C2198: 'winLogErrorAtLine': too few arguments for call
sqlite3.c(44338): error C2065: 'winFile': undeclared identifier
sqlite3.c(44338): error C2297: '*': illegal, right operand has type 'int *'
sqlite3.c(44338): error C2059: syntax error: ')'
sqlite3.c(44353): error C2223: left of '->mmapSize' must point to struct/union
sqlite3.c(44354): error C2223: left of '->mmapSize' must point to struct/union
sqlite3.c(44355): error C2223: left of '->pMapRegion' must point to struct/union
sqlite3.c(44355): warning C4022: 'memcpy': pointer mismatch for actual parameter 2
sqlite3.c(44355): error C2168: 'memcpy': too few actual parameters for intrinsic function
sqlite3.c(44360): error C2223: left of '->mmapSize' must point to struct/union
sqlite3.c(44361): error C2223: left of '->pMapRegion' must point to struct/union
sqlite3.c(44361): warning C4022: 'memcpy': pointer mismatch for actual parameter 2
sqlite3.c(44361): error C2168: 'memcpy': too few actual parameters for intrinsic function
sqlite3.c(44380): error C2223: left of '->h' must point to struct/union
sqlite3.c(44380): warning C4022: 'function through pointer': pointer mismatch for actual parameter 2
sqlite3.c(44380): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'DWORD *'
sqlite3.c(44380): warning C4024: 'function through pointer': different types for formal and actual parameter 3
sqlite3.c(44380): warning C4133: 'function': incompatible types - from 'OVERLAPPED *' to 'LPDWORD'
sqlite3.c(44380): error C2198: 'function through pointer': too few arguments for call
sqlite3.c(44385): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44389): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44389): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'char [8]'
sqlite3.c(44389): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 2
sqlite3.c(44389): error C2223: left of '->zPath' must point to struct/union
sqlite3.c(44389): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
sqlite3.c(44389): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 3
sqlite3.c(44389): error C2198: 'winLogErrorAtLine': too few arguments for call
sqlite3.c(44416): error C2065: 'winFile': undeclared identifier
sqlite3.c(44416): error C2297: '*': illegal, right operand has type 'int *'
sqlite3.c(44416): error C2059: syntax error: ')'
sqlite3.c(44471): error C2223: left of '->h' must point to struct/union
sqlite3.c(44471): warning C4022: 'function through pointer': pointer mismatch for actual parameter 2
sqlite3.c(44471): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'DWORD *'
sqlite3.c(44471): warning C4024: 'function through pointer': different types for formal and actual parameter 3
sqlite3.c(44471): warning C4133: 'function': incompatible types - from 'OVERLAPPED *' to 'LPDWORD'
sqlite3.c(44471): error C2198: 'function through pointer': too few arguments for call
sqlite3.c(44490): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44496): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44497): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44501): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44501): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'char [10]'
sqlite3.c(44501): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 2
sqlite3.c(44501): error C2223: left of '->zPath' must point to struct/union
sqlite3.c(44501): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
sqlite3.c(44501): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 3
sqlite3.c(44501): error C2198: 'winLogErrorAtLine': too few arguments for call
sqlite3.c(44506): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44506): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'char [10]'
sqlite3.c(44506): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 2
sqlite3.c(44506): error C2223: left of '->zPath' must point to struct/union
sqlite3.c(44506): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
sqlite3.c(44506): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 3
sqlite3.c(44506): error C2198: 'winLogErrorAtLine': too few arguments for call
sqlite3.c(44519): error C2065: 'winFile': undeclared identifier
sqlite3.c(44519): error C2297: '*': illegal, right operand has type 'int *'
sqlite3.c(44519): error C2059: syntax error: ')'
sqlite3.c(44533): error C2223: left of '->szChunk' must point to struct/union
sqlite3.c(44534): error C2223: left of '->szChunk' must point to struct/union
sqlite3.c(44538): warning C4013: 'winSeekFile' undefined; assuming extern returning int
sqlite3.c(44540): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44540): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'char [13]'
sqlite3.c(44540): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 2
sqlite3.c(44540): error C2223: left of '->zPath' must point to struct/union
sqlite3.c(44540): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
sqlite3.c(44540): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 3
sqlite3.c(44540): error C2198: 'winLogErrorAtLine': too few arguments for call
sqlite3.c(44541): error C2223: left of '->h' must point to struct/union
sqlite3.c(44541): error C2198: 'function through pointer': too few arguments for call
sqlite3.c(44543): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44545): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44545): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'char [13]'
sqlite3.c(44545): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 2
sqlite3.c(44545): error C2223: left of '->zPath' must point to struct/union
sqlite3.c(44545): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
sqlite3.c(44545): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 3
sqlite3.c(44545): error C2198: 'winLogErrorAtLine': too few arguments for call
sqlite3.c(44553): error C2223: left of '->pMapRegion' must point to struct/union
sqlite3.c(44553): error C2223: left of '->mmapSize' must point to struct/union
sqlite3.c(44554): error C2223: left of '->mmapSize' must point to struct/union
sqlite3.c(44588): error C2065: 'winFile': undeclared identifier
sqlite3.c(44588): error C2297: '*': illegal, right operand has type 'int *'
sqlite3.c(44588): error C2059: syntax error: ')'
sqlite3.c(44626): error C2223: left of '->pMapRegion' must point to struct/union
sqlite3.c(44627): error C2223: left of '->pMapRegion' must point to struct/union
sqlite3.c(44627): error C2198: 'function through pointer': too few arguments for call
sqlite3.c(44632): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44637): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44637): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'char [9]'
sqlite3.c(44637): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 2
sqlite3.c(44637): error C2223: left of '->zPath' must point to struct/union
sqlite3.c(44637): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
sqlite3.c(44637): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 3
sqlite3.c(44637): error C2198: 'winLogErrorAtLine': too few arguments for call
sqlite3.c(44641): error C2223: left of '->h' must point to struct/union
sqlite3.c(44641): error C2198: 'function through pointer': too few arguments for call
sqlite3.c(44648): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44652): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44652): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'char [9]'
sqlite3.c(44652): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 2
sqlite3.c(44652): error C2223: left of '->zPath' must point to struct/union
sqlite3.c(44652): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
sqlite3.c(44652): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 3
sqlite3.c(44652): error C2198: 'winLogErrorAtLine': too few arguments for call
sqlite3.c(44661): error C2065: 'winFile': undeclared identifier
sqlite3.c(44661): error C2297: '*': illegal, right operand has type 'int *'
sqlite3.c(44661): error C2059: syntax error: ')'
sqlite3.c(44687): error C2223: left of '->h' must point to struct/union
sqlite3.c(44687): error C2198: 'function through pointer': too few arguments for call
sqlite3.c(44691): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44693): error C2223: left of '->lastErrno' must point to struct/union
sqlite3.c(44693): warning C4047: 'function': 'DWORD' differs in levels of indirection from 'char [12]'
sqlite3.c(44693): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 2
sqlite3.c(44693): error C2223: left of '->zPath' must point to struct/union
sqlite3.c(44693): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
sqlite3.c(44693): warning C4024: 'winLogErrorAtLine': different types for formal and actual parameter 3
sqlite3.c(44693): error C2198: 'winLogErrorAtLine': too few arguments for call
sqlite3.c(44738): error C2143: syntax error: missing ')' before '*'
sqlite3.c(44738): fatal error C1003: error count exceeds 100; stopping compilation
NMAKE : fatal error U1077: '"C:\Program Files\Microsoft Visual Studio 14.0\VC\BIN\cl.EXE"' : return code '0x2'
Stop.

c:\Users\IEUser\pysqlcipher3-win32\sqlcipher>

to be continued tomorrow...

[cryptomental]

@LefterisJP I am quite proud of myself. Rotkehlen is running on Windows.

Obviously I did not anticipate that this bounty will consume so much time. Making pysqlcipher3 built and install correctly was beyond insane itself. Unfortunately this package alone requires lots of third party dependencies, like installing old TCL/TK (with latest version does not compile as seen above), and copying OpenSSL packages to c:\windows\system32 folder (requires Admin privilidges). Not to make things easy, with the latest OpenSSL this package also does not build and an older one is needed. Fortunately I was finally able to build everything with Visual Studio 2015 and install package with Python 3 support.

Later, I needed to rebuild electron due to ZMQ errors and this requires Python 2.7, so I needed to install this as well. As for the rotkehlen, it fails at history.py:230 due to Windows escape chars but I was able to enable logging, and find the root cause.

Packaging and building pysqlcipher3 itself is way beyond this bounty, but I could provide a readme on how to enable this all correctly. From what I have seen this week, it would be much easier to prepare a very simple VirtualBox Linux image with preinstalled rotkehlchenio and make Windows users to run it than to support Windows version of the tool.

Unfortunately, I am heading for a long awaited 3-week vacation this weekend and I will be most of the time completely offline and since I am not quite sure how to proceed with this bounty. I would be grateful if you let me know your decision and whether to close it or not. I will paste screenshots now.

[cryptomental]

As seen above, Windows version is up and running.

[cryptomental]

And last but not least

C:\Users\IEUser\rotkehlchen>npm test

> [email protected] test C:\Users\IEUser\rotkehlchen
> mocha ui/tests/



  Application launch
    √ assert we got 1 window running
    √ make sure we get the loging popup (54ms)


  2 passing (20s)


C:\Users\IEUser\rotkehlchen>

[LefterisJP]

Amazing work @cryptomental. Now if only you can do appveyor automation that would be complete and I can use my wife's computer (has Windows) to test it all works.

Really great.

I am also going to head on a 1.5 week vacation from Sunday and on so we can keep it open untils it's finalized?

Can you also document the installation process for Windows in the docs so others can follow it? I need to follow it and verify it works too.

Essentially what the issue says in the description

[cryptomental]

I encountered already a blocking issue. TCL/TK 8.5.18 version has to be installed in AppVeyor but Chocolatey package is missing. I tried the latest available 8.6 version but it is broken as well due to package not available anymore on ActiveState server.

See:

https://ci.appveyor.com/project/cryptomental/rotkehlchen

There were questions on update 10 months back
https://chocolatey.org/packages/ActiveTcl

And the available releases: http://downloads.activestate.com/ActiveTcl/releases/

I could try to replicate this / fix this but will take time.

[cryptomental]

After numerous failed attempts I managed to get appveyor up and running, including test run and electron deployment stage https://ci.appveyor.com/project/cryptomental/rotkehlchen/build/1.0.63

It took me around 60 builds of trial and error, I cleaned the code up and will open a pull request soon.

I borrowed part of the appveyor setup from the excellent Appveyor integration example from VIM for Windows, and also added references. @LefterisJP later you may want to have a look at their bintray integration, perhaps it will be also useful for releasing Windows binaries.

I managed to avoid TCL/TK, though at some point thanks to VIM for Windows I was able to use it. TCL is needed to generate sqlcipher version of sqlite amalgamation (combined sqlite source files into a single sqlite.c). After I discovered that the amalgamation is already bundled in node-sqlcipher package I decided it will save a lot of hassle to reuse their version and after some tweaks I was able to use the node-sqlcipher amalgamation to build and install pysqlcipher3 on windows.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 1 month, 1 week ago.
Please review their action plans below:

1. cryptomental has been approved to start work.

   I figured out the missing Appveyor part, pull request opened.

Learn more on the Gitcoin Issue Details page.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work for 250.0 DAI (250.0 USD @ $1.0/DAI) has been submitted by:

1. @cryptomental

@vs77bb please take a look at the submitted work:

• PR by @cryptomental

---

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $20,585.85 more funded OSS Work available on the Gitcoin Issue Explorer

[LefterisJP]

Hey @cryptomental welcome back!

Do you need me to authorize anything in the Appveyor side to have the tests run in appveyor for every PR?

[cryptomental]

Hi @LefterisJP yes, you will need to sign in to AppVeyor with your GitHub account and then add this repository to your AppVeyor projects. I was able to configure AppVeyor on my fork only.

[LefterisJP]

They got the most idiotic configuration for github organizations. But after merging your PR and playing a lot with their settings and github authorizations I got it running for PRs:

I used an old abandoned PR to test it: #128

Seem to be failing due to npm problems though atm.

I get the same problem in the Travis job of the same build ... so hopefully it's temporary.

request to https://registry.npmjs.org/tail failed, reason: Hostname/IP doesn't match certificate's altnames: "Host: registry.npmjs.org. is not in the cert's altnames: DNS:a.sni.fastly.net, DNS:a.sni.global-ssl.fastly.net"

EDIT: It's a global npm issue: https://status.npmjs.org/incidents/v22ffls5cd6h

[cryptomental]

The npm issue was resolved. I am glad that you managed to get it working! As for the Appveyor itself I remember that ocasionally for other projects I had to wait for the job up to 1 hour to kick in but recently it seems to have improved.

[LefterisJP]

Yeah seems to be working but the new UI tests seem to fail: https://ci.appveyor.com/project/rotkehlchenio/rotkehlchen/build/1.0.9

Granted ... the PR is not yet merged and could be any number of things, but I wonder if you have an idea:

Error: element (".jconfirm-box.jconfirm-type-green.jconfirm-type-animated") still not existing after 5000ms

That means that the popup modal that greets the user after signin never shows

[cryptomental]

@LefterisJP I will try to find time and have a look tomorrow. I suspect that the problem may be located on Python code side.

[cryptomental]

Hi @LefterisJP I created a branch in my fork off the PR #136 and spent around an hour playing around with loglevels and exposing artifacts.

I added rotkehlchen.log to artifacts and I learnt that AppVeyor exposes artifacts only if the build is green, so I forced it to be green on my forked PR branch. There are no Python errors or exceptions in the log, just a few debug messages:

https://ci.appveyor.com/project/cryptomental/rotkehlchen/build/1.0.79

https://ci.appveyor.com/project/cryptomental/rotkehlchen/build/1.0.79/artifacts

direct link to the log: https://ci.appveyor.com/api/buildjobs/v8m5mvl2b1q334bt/artifacts/rotkehlchen.log

I noticed in the PR #136 comments that the tests are flaky and I hope that the PR author will be able to fix his new tests also on Windows. Other than than I hope that the 'Windows support' ticket can be closed as-is, as the appveyor seems to work and the new Windows related tickets can be opened if needed.

[LefterisJP]

@cryptomental so you have no local Windows machine to play with on this? It seems that there is no problem on the python side from the logs you showed.

[cryptomental]

I did not have one yesterday, I will try to give it a shot today.

[mykeels]

@LefterisJP I think you'll need to add the environment variables to the AppVeyor environment too, concerning this comment.

[cryptomental]

@LefterisJP I found the root cause and fixed the issue.

There is a problem with regular expressions parsing on Windows when it comes to paths.
On *nix we have /a/path and on Windows C:\A\Path and the backslashes are treated as escape sequences.

A few screenshots below may help to understand the problem:

And with the fix 4/6 tests of the new tests are passing: https://ci.appveyor.com/project/cryptomental/rotkehlchen/build/1.0.80

I was able to see test execution in real time on Windows as well. The failing ones fail with ECONNRESET (I will paste backtrace later) and I am not sure how to handle those, perhaps the tests are a bit flaky.

I will submit soon the pull request with the fix and I hope that we can finally close this ticket ;-)

[cryptomental]

The backtrace I mentioned above:

     Error: write ECONNRESET
      at new RuntimeError (node_modules\webdriverio\build\lib\utils\ErrorHandler.js:143:12)
      at Request._callback (node_modules\webdriverio\build\lib\utils\RequestHandler.js:330:43)
      at self.callback (node_modules\request\request.js:185:22)
      at Request.onRequestError (node_modules\request\request.js:877:8)
      at Socket.socketErrorListener (_http_client.js:387:9)
      at onwriteError (_stream_writable.js:418:12)
      at onwrite (_stream_writable.js:440:5)
      at _destroy (internal/streams/destroy.js:39:7)
      at Socket._destroy (net.js:564:3)
      at Socket.destroy (internal/streams/destroy.js:32:8)
      at Socket._writeGeneric (net.js:764:17)
      at Socket.connect (net.js:719:12)
      at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1173:10)

and the PR with the regex fix will be opened within 5 minutes.

[cryptomental]

#145 opened.

I guess after the PR is merged we should be able to close this ticket.

[LefterisJP]

Hello @cryptomental for this:

(I will paste backtrace later) and I am not sure how to handle those, perhaps the tests are a bit flaky.

The backtrace you pasted is from what? Locally running the tests? Because the backtrace does not look like the one shown for the UI tests in the appveyor build you linked above. The ones that fail there seem to be the flaky UI tests that @mykeels is trying to finalize in his PR, so nothing windows specific.

[LefterisJP]

As for the #145 PR along with the work you have done all around windows research and appveyor I want to thank you and consider your part of the work here done. I will open other github tickets for specific windows issues that will popup when I test it myself too in Windows and when the binary generation is tackled.

@vs77bb As far as the bounty is concerned this should be considered complete.

[cryptomental]

Thank you a lot @LefterisJP . I really enjoyed it.

[cryptomental]

And as for the backtrace, yes, I was referring to backtrace when running tests locally on Windows. I was observing them in rotkehlchen.log in realtime. I realized that the file is wiped with each test case, therefore I had not seen backtraces in AppVeyor before as only the last test case's log was exposed as AppVeyor artifact that I set up on my branch e.g. https://ci.appveyor.com/project/cryptomental/rotkehlchen/build/1.0.80/artifacts