[SOLVED] Custom matcher - not working in waves-client, without HTTPS.

[username1565]

[SOLVED] Custom matcher - not working in waves-client, without HTTPS.

[bitSecurities]

Thx @username1565 a lot!

Many tips above to check more on my side, it is very helpful.

I will check all these elements and also thank you for possibility to check how is your matcher working now on the new client of Waves.

Have a good weekend!

[bitSecurities]

@username1565
Thx a lot!

Regarding 1-2 - I will try to explore it during weekend.

Reagrding 3 - yes, this is reasin (a fee for a miner), what we have taken false during our previous discussion about HFT and market making

Reagrding 4- yes, I tested your matcher several times, it looks that sometimes DDNS makes errors in connections between a client and the matcher.

My matcher is https://matcher.blockcore.fund/ and also http://matcher.blockcore.fund/
Yes, you can configure a free acount in Cloudflare for your domain and you get a free SSL certificate there (you can also buy more professional types of services there, but for many cases this free account is OK). Then in DNS management of your domain in Cloudflare - you can define record A which is directed to your server, After getting the connection from Cloudflare to your servert, you will be able to run your matcher on the domain in the Internet with https with valid certificate.

PS - I was offline during last 30 hours and I cann see that my Windows machine was restarted. So it was a time that my test-matcher wasn't active.

[tsigel]

Hello, the client only works with the https protocol for better user security.
To work with your matcher, you need to contact him via https.

[username1565]

As I said earlier, after changing port to 80, Matcher REST API still binded to port 6886.
So, Is there any option in config, to enable HTTPS?
Or I need to do this manually? If yes, then is there any guide
how to setup HTTPS/OpenSSL for Windows,
and is there any proxy to make the forced redirect
from http://external_IP:6886/ to https://localhost:433/ ?

If yes, I think will be better to add this inside the code for waves full node,
or inside the code for matcher extension.

[username1565]

When I added my local matcher in waves-lite-client, I didn't saw the orders and markets...
And, as I understand, here: wavesplatform/Waves#1354
Matcher is a semi-centralised part of the Waves network.

---

---

---

Hm...
I see, all network has only two matchers...
Maybe second matcher is for the fork: https://client.turtlenetwork.eu
or this fork: https://turtlenetwork.blackturtle.eu/
UPD:
No, no...
According this data
I see exchanges between TUSD, TETH, TBTC, etc...
In descriptions of those tokens, I see this is tokenomica tokens,
so this second matcher is a matcher for the project https://tokenomica.com

---

So can this all network be crashed, if this matchers will be disabled or banned???

And is there any sense to enable the matcher, on the node, if using this - so hard??

---

Also...
Do you have the list of existing matchers?
Can you ping this all, and sorting this by uptime
and select the better matcher to make transactions faster?
Can you programming the synchronization of the settings, markets, and orders for the network of existing matchers?

[username1565]

I did setup nginx, as https proxy, and get success then.
Just downloaded NGINX Windows-1.4.1, and according this guide
added the following strings in config:

    # HTTPS proxy
    #
    server {
        listen       0.0.0.0:6879;
        server_name  localhost;

        ssl                  on;
        ssl_certificate      ssl/cert.pem;
        ssl_certificate_key  ssl/key.pem;

        ssl_session_timeout  5m;

        ssl_protocols  SSLv2 SSLv3 TLSv1;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers   on;

        location / {
            proxy_pass  http://127.0.0.1:6869;
        }
    }

After this, https://127.0.0.1:6879 contains Mather REST API, and this opened over HTTPS.
SSL sertificate is self-signed, so I did add an exception in browser.
After port-forwaring, this opened by link: https://my_external_IP:6879/
and after put the link https://my_external_IP:6879/matcher in the settings here: https://client.wavesplatform.com/#!/dex-demo
I did connect to my matcher successfully.
There is no any orders, but I opened test order, and fill this myself.
I see transaction in WAVES blockchain, and my matcher address got the comission 0.003 WAVES from this exchange.

Also, NODE REST API can working too through HTTPS on the same machine, on another port, through nginx HTTPS-proxy. But need add separate section in config too.
This working slow, because redirects, and your node and matcher working faster.

This issue is solved. Just some small questins is unanswered.
Have a nice day.

[bitSecurities]

@username1565
I have one question more.
Have you used 1.4.7 Windows version of nginx with the:

• conf file and certificates from (http://kosiara87.blogspot.com/2013/06/windows-set-up-sslhttps-reverse-proxy.html)?
• added these lines for HTTPS proxy as you are proposing?

Becasue I cannot see 1.4.1 Windows version on http://nginx.org/en/download.html, only 1.4.7 version is possible.

[username1565]

@bitSecurities, I just used Windows-1.4.1 version from that guide - for tests, on Windows XP, on my notebook.

nginx.conf

worker_processes  1;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        off;
    keepalive_timeout  65;

    #           HTTPS proxy - node
    server {
        listen       0.0.0.0:6869;
        server_name  localhost;

        ssl                  on;
        ssl_certificate      ssl/cert.pem;
        ssl_certificate_key  ssl/key.pem;

        ssl_session_timeout  5m;

        ssl_protocols  SSLv2 SSLv3 TLSv1;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers   on;

        location / {
            proxy_pass  http://192.168.0.107:6869;
        }
    }

    #           HTTPS proxy - matcher
    server {
        listen       0.0.0.0:6886;
        server_name  localhost;

        ssl                  on;
        ssl_certificate      ssl/cert.pem;
        ssl_certificate_key  ssl/key.pem;

        ssl_session_timeout  5m;

        ssl_protocols  SSLv2 SSLv3 TLSv1;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers   on;

        location / {
            proxy_pass  http://192.168.0.107:6886;
        }
    }
}

Then, NODE REST API and MATCHER REST API is opened on notebook by links:
https://127.0.0.1:6869/
https://127.0.0.1:6886/
and if I do port-forwarding for notebook's ports 6869 and 6886, then this working over HTTPS, with self-signed SSL secrificate cert.pem, and private key key.pem in the folder /conf/ssl folder of nginx main folder.

Also, as was been described in that blog,
I did create two bat-files, to start and stop nginx.exe:
start_nginx_as_https_proxy.bat:

start nginx

stop_nginx_as_https_proxy.bat:

nginx -s stop

to start and stop just by double click on this files.

Also, to don't play with ports, can be used just HTTPS port (443),
only one nginx server in config:

    #           HTTPS proxy - matcher
    server {
        listen       0.0.0.0:443;
        server_name  localhost;

        ssl                  on;
        ssl_certificate      ssl/cert.pem;
        ssl_certificate_key  ssl/key.pem;

        ssl_session_timeout  5m;

        ssl_protocols  SSLv2 SSLv3 TLSv1;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers   on;

        location / {
            proxy_pass  http://192.168.0.107:6886;
        }
    }

In this case, need to do port-forwarding just for 443 port for MATHER REST API over HTTPS,
and 6869 with 6886 ports still opened over HTTP.
Then, in waves-client, just specified the default node: https://nodes.wavesplatform.com/ and custom matcher: https://external_IP:6886/matcher

I did try newer versions of nginx, but this is not working on Windows XP.
Also, I did not have success with this config for latest stable version Windows-1.16.1, on Windows 8.1.

P.S.: I did try nginx 1.4.7 with first config, and this working on Windows XP.
But I cann't run this successfully on Windows 8.1, with second config.
In this case, I tried:

...
listen       0.0.0.0:443;
...
proxy_pass  http://127.0.0.1:6886;
...

Maybe just need to read documentation of newer version nginx,
and maybe there is better versions of HTTPS proxies for newer versions of Windows,
just need to search it.

[bitSecurities]

@username1565
Thx a lot for all these structured information, it is very helpful for me, also it will be very helpful for alnother readers of your manual "how to do Waves matcher on Windows"!
I was trying with nginx today, but I got errors with the certificates are provided in that guide, despite in your example is it clear that should run well.

So I have used IIS and URL rewrite to launch finally matcher REST API on https://my_external_IP with an exception in browser (using unvalid SSL certificate).

So I went to Cloudflare to install suddomain of existing website (for using SSL dertificate is provided by Cloudflare) for this REST API through setting-up record A pointing to the IP of my above windows server. And I don't know how to make any succesful connection from Cloudflare via this pointing. Currently is such result that after opening the subdomain in the web-browser I can see an error of Cloudflare of connection to the host (=my computer).
I tried DNS settings (Active Directory)on my Windows server and install there this subdomain, but still I'm getting the error from Cloudflare on the subdomain in Internet. I read today many manuals how to install a domain on Active Directory in Windows, but I haven't got guide how prepare Windows system to be a host for a record A of a domain is provide on external DNS zone (like in Cloudflare).

Maybe a new tip from your side will be very helpful for me, very appreciated.

P.S. Maybe it is aslo possible to get an access to your website of your matcher to see how it works in practise via Waves client (after setting it there for this trial)?

[username1565]

@bitSecurities,

I was trying with nginx today, but I got errors with the certificates are provided in that guide, despite in your example is it clear that should run well.

I just used this commands, to make generate self-signed SSL certificate with openssl.exe, after consecutive commenting and uncommenting of this commands.

generate_SSL.bat:

::__________________________________________________________________________________________
:: Generate RSA-key, using random file with entropy.
:: input: 	rand.rnd 	- just the random file to make entropy.
:: output: 	key.pem 	- 2048-bit RSA PRIVATE-key.
::__________________________________________________________________________________________
::openssl genrsa -rand rand.rnd -out key.pem 2048



::__________________________________________________________________________________________
::Generate SSL-sertificate request by existing key.pem
:: input:	key.pem 	- RSA key.
:: output:	req.pem 	- certificate request
::__________________________________________________________________________________________
::openssl req -new -key key.pem -out req.pem



::__________________________________________________________________________________________
::You will now have a private key in key.pem and a CSR in req.pem
::Then send req.pem to CA.
::Upon reception of a signed certificate from CA, put it to cert.pem.
::The certificate should look like:
::-----BEGIN CERTIFICATE-----
::MIICLjCCAZcCAQEwDQYJKoZIhvcNAQEEBQAwWzELMAkGA1UEBhMCQVUxEzARBgNV
::BAgTClF1ZWVuc2xhbmQxGjAYBgNVBAoTEUNyeXB0U29mdCBQdHkgTHRkMRswGQYD
::dp7jnmWZwKZ9cXsNUS2o4OL07qOk2HOywC0YsNZQsOBu1CBTYYkIefDiKFL1zQHh
::8lwwNd4NP+OE3NzUNkCfh4DnFfg9WHkXUlD5UpxNRJ4gJA==
::-----END CERTIFICATE-----
::__________________________________________________________________________________________


::__________________________________________________________________________________________
::You can also generate a temporary but ready to use untrusted test certificate by running:
::openssl req -new -key key.pem -out cert.pem -x509 -config "bin\openssl.cnf"
::__________________________________________________________________________________________
::cert.pem - this is that certificate.

pause

Then, after got key.pem and cert.pem, I just put this files in ../nginx/conf/ssl/cert.pem, and ../nginx/conf/ssl/key.pem,
and then configure nginx, using this strings:

        ssl_certificate      ssl/cert.pem;
        ssl_certificate_key  ssl/key.pem;

Also, there is possible to get verified and trusted ssl sertificates for free, on some services.

Maybe a new tip from your side will be very helpful for me, very appreciated.

I don't know why your cloudflare domain not working. Maybe something need to setting up.

Maybe it is aslo possible to get an access to your website of your matcher to see how it works in practise via Waves client (after setting it there for this trial)?

Yes, you can. But this is on http, not HTTPS now, and this is working in old waves-lite-client.
http://dyndns-node.duckdns.org:6886/ - you can see my matcher rest api there.
Then, download waves-lite-client, as zip, from Releases,
unpack this zip, open /js/waves-lite-client-mainnet-0.5.18.js,
and rewrite matcher:

118255:             //MATCHER_ADDRESS: 'https://matcher.wavesnodes.com',				//default matcher - HTTPS
118256:            MATCHER_ADDRESS: 'http://dyndns-node.duckdns.org:6886',				//custom matcher - HTTP
118257:            //MATCHER_ADDRESS: 'https://dyndns-node.duckdns.org:6886',				//custom matcher - HTTPS

Then save, and open index.html.

After you connect to the matcher, you will'nt see any orders, but you can open this, and open another order, and fill your order by your another order. And this working.
If you have any tokens, just specify their Asset ID in two places on DEX-tab, to exchange this using waves-lite-client, if you have enough WAVES to pay comissions.
Remember, the matchers got comissions of 0.003 WAVES - for each exchange order, by default.

Also, nginx HTTPS proxy is lagging, because there is used two redirect between two my machines.
So I just disabled nginx, after successfully tests, and matcher not working on HTTPS, only over HTTP.
HTTPS need just to specify matcher API URL in waves-client (see the settings there).
Then, matcher working there, but after add exception in browser for this SSL sertificate, because this is self-signed certificate.

P.S.: Now, my test-matcher is temporary working on https://dyndns-node.duckdns.org
So you can using

https://dyndns-node.duckdns.org/matcher

for tests in the settings of waves-client too.

UPD:
I just did add redirect to https:// from http://

/nginx/html/redirect-to-https.html code:

<!DOCTYPE html>
<html>
<head>
   <!-- HTML meta refresh URL redirection -->
   <!-- Redirect from HTTP to HTTPS --> 
   <meta http-equiv="refresh" content="3; url=https://dyndns-node.duckdns.org/">
</head>
<body>
   <p style="display: inline-block;">Redirect <a href="https://dyndns-node.duckdns.org/">to HTTPS</a> - after&nbsp;<div id="seconds" style="display:inline-block;">3</div> seconds...</p>
   <script type="text/javascript">
		var div = document.getElementById('seconds');
		setInterval(function(){div.innerHTML = parseInt(div.innerHTML, 10)-1;}, 1000);
   </script>
</body>
</html>

My /nginx/conf/nginx.conf code now:

worker_processes  1;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        off;
    keepalive_timeout  65;

	
    #           HTTP redirect to HTTPS
    server {
        listen       0.0.0.0:80;
        server_name  localhost;

        location = / {
            root   html;
            index  redirect-to-https.html;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

    #           HTTPS proxy - matcher on https://localhost:443/
    server {
        listen       0.0.0.0:443;
        server_name  localhost;

        ssl                  on;
        ssl_certificate      ssl/cert.pem;
        ssl_certificate_key  ssl/key.pem;

        ssl_session_timeout  5m;

        ssl_protocols  SSLv2 SSLv3 TLSv1;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers   on;

        location / {
            proxy_pass  http://192.168.0.107:6886;
        }
    }
}

and now, you can see this here: http://dyndns-node.duckdns.org/
The version of nginx is v1.4.1, and I have runned this on Windows XP now,
and you can see this by some not found page, like this:
http://dyndns-node.duckdns.org/version.html

---

Also, as you can see, after connect to the custom matcher,
there is incorrect order history, because, by default,
using https://marketdata.wavesplatform.com/
To set up your own market-datafeed API,
you need to set up WAVES DATAFEED API,
then you can see your own candles, charts, prices, and order history there.

[username1565]

Hey, @bitSecurities!
I see, you say about you want to add your IP to A-record for cloudflare domain name,
but you have some problems with resolving this domain...
Maybe you just need to wait, within 24-48 hours to update DNS info everywhere.
Also, if you have dynamic IP, you can using DDNS-services, like https://www.duckdns.org/
take there long domain name, and just add this domain to your cloudflare-domain-name
as CNAME-record (alias, another domain name).
And, maybe, in this case, your API's can be opened by cloudflare domain name,
and then you can using verified SSL-certificate from cloudflare,
not self-signed certificate with adding the exception in browser.

[bitSecurities]

@username1565
I got the matcher is working on subdomain through Cloudflare SSL certificate on https without any error of validation of the SSL certificate.
Simply I had to configure IIS with URL redirection with giving the name of subdomain back - what Cloudflare started to accept as connection.
Also http works OK with Swagger (obviously it doesn't work in the new Waves client)

I have two other questions, if possible.

Is this part of conf file executed by the matcher?
I suppose it cannot be, because all various setting there make the same result of the new client, in my case it always presents the same pairs at starting the client, despite various configurations like this below:

    # Predefined ordering of base assets
    predefined-pairs = [
     {amountAsset = "WAVES", priceAsset = "Fmg13HEHJHuZYbtJq8Da8wifJENq8uBxDuWoP9pVe2Qe"},
     {amountAsset = "WAVES", priceAsset = "HyFJ3rrq5m7FxdkWtQXkZrDat1F7LjVVGfpSkUuEXQHj"},
     {amountAsset = "WAVES", priceAsset = "2xnE3EdpqXtFgCP156qt1AbyjpqdZ5jGjWo3CwTawcux"},
     {amountAsset = "WAVES", priceAsset = "6pmDivReTLikwYqQtJTv6dTcE59knriaodB3AK8T9cF8"},
     {amountAsset = "Fmg13HEHJHuZYbtJq8Da8wifJENq8uBxDuWoP9pVe2Qe", priceAsset = "2aSqCbvCTgvCpwkGsk4mea4tCLG4Zgp69aQDhHNvRUZv"},
     {amountAsset = "Fmg13HEHJHuZYbtJq8Da8wifJENq8uBxDuWoP9pVe2Qe", priceAsset = "8zEZuJcKPQmFuYgVe5ZMpxgiPLu5zBhjA6xgdGomQDaP"},
     {amountAsset = "Fmg13HEHJHuZYbtJq8Da8wifJENq8uBxDuWoP9pVe2Qe", priceAsset = "D2MNuUyA38pSKoV7F7vpS15Uhw9nw5qfbrGUfCLRNuRo"},
     {amountAsset = "2xnE3EdpqXtFgCP156qt1AbyjpqdZ5jGjWo3CwTawcux", priceAsset = "HyFJ3rrq5m7FxdkWtQXkZrDat1F7LjVVGfpSkUuEXQHj"}
   ]

2. I also configured different fee (let's say 0.00125 Waves) in my conf file, but ... still the execution is with a fee of 0.003 Waves, as presented below:
   https://w8io.ru/3P5mtgN9CKyY9uFRKToB1oXMKnUurkF4msv
   Why isn't taken a fee which is configured in the conf file of the matcher, but a typical fee?

And I also don't understand statistics of the single executed order which I made through this matcher - the seller fee was 0.003 Waves, the buyer fee was 0.003 Waves, but the matcher got only 0.003 Waves - where is rest of 0.003 Waves going?
https://wavesexplorer.com/tx/9Co9t7ibdLmJTMRX8n4v72qfaVfv8zpzh3PimsoG1K2x
Fee 0.003 WAVES
Buy Matcher Fee 0.003 WAVES
Sell Matcher Fee 0.003 WAVES
The summary of 0..003+0.003 should be 0..006 fee for the matcher, but the fee is 0.003.

Thank you in advance for your comment at the above things.

[username1565]

Hi, @bitSecurities.

1. About such strings, in configuration, you can read this:

Using parameters "price-assets" and "predefined-pairs" it is possible to define rules of assets pairs creation for the matcher. Inside the matcher a trading pair consists of an "amount" asset and a "price" asset. The "amount" asset is used to specify the amount in order. The "price" asset is used to specify the price in order.

Parameter "price-assets" is the list of assets IDs that will be used as price asset (second one) in a pair with any unknown asset. If both assets in the pair are "price" assets you have to use parameter "predefined-pairs" to define the correct order of assets in the pair.

So, this need to correct display prices, for example: 200 WAVES/ETH is equal 0.005 ETH/WAVES, and you can set this as WAVES/ETH, or ETH/WAVES,
and to do this for all new and unknown assets.
But you say you don't see any changes, and maybe this configuration need for blockchain,
but the interpretting of this and converting the values can be customized client-side,
inside the source code of client.
I did not bother with this. But if you're interested, you can look inside the source code.

---

2. I don't know why your fees was not been updated.
   Maybe you need to restart your node, or this value need to specify in another place, or this is overrided by another specified or hardcoded value inside the node source code.
   As I can see here
   this value must be just > 0...
   matcherFee be > 0 and < MaxAmount

---

3. About comissions, I see some fees are paying by matchers for the miners,
   and as you can see here, there is three values, and matchers paying to the miners too:

buyMatcherFee - Amount fee for matching from buyOrder. Transferred to Matcher's account balance.
sellMatcherFee - Amount fee for matching from sellOrder. Transferred to Matcher's account balance.
fee - Fee for the transaction to be included in a block by a miner. It is paid from the Matcher's account

So, 0.003 WAVES (buyMatcherFee) + 0.003 WAVES (sellMatcherFee) = 0.006 WAVES - received the matcher.
0.006 WAVES (total) - 0.003 WAVES (fee sent to the miner, who generated the block) = 0.003 WAVES (Matcher balance).

So, if you will be one single miner, in your own sub-network of decentralized network,
you will not loss no one WAVELETS (minimal part of 1 WAVES).

---

4. Also, I have some questions for you:
   Did you test my matcher? Is this working on http/https? Can I disable this?
   Can you tell your link to tests, and maybe using if default matcher will be crushed, for example?
   When you got SSL certificate from cloudflare, did you generate the keys yourself,
   or this was been generated there, and just signed there? Is this free?
   Can I generate and got the free SSL sertificate somewhere for DDNS-domain,
   without some service actions for this, in future?