Simex API Scraping

[diadata-org]

Simex API Scraping

[PumpkingWok]

Hi @frranck
Thank you for the info, I'm trying to implement it without using external packages.I should have done it.
I'm going to open a PR in 3/4 hours at most.
Have a nice day, thanks in advance !!

[kaythxbye]

We would like to receive trades containing the following crypto assets from the exchange (if available):

• BTC
• ETH
• XRP
• BCH
• EOS
• LTC
• XLM
• ADA
• MIOTA
• USDT
• TRX
• NEO
• XMR
• DSH
• ETC
• BNB
• XEM
• XTZ
• VET

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 0.345 ETH (100.91 USD @ $292.5/ETH) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $20,189.87 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 12 months from now.
Please review their action plans below:

1. PumpkingWok has started work.

   Hi,
   I can look in it this afternoon, i should conclude this in a very few days.
   Thanks in advance

Learn more on the Gitcoin Issue Details page.

[PumpkingWok]

Hi @diamw,
I have looked into official SIMEX documentation and it seems they offer for now only REST API, no subscriptions to websocket channels as i think you would like to implement it.
I have searched on the web too but nothing.
Thanks !!

[kaythxbye]

Hi @PumpkingWok thanks for the update! True, there seems to be no websocket API. Are you confident you can do a scraper for REST? It is still better than having no data.

[PumpkingWok]

Hi @kaythxbye, Yes i could develop it without hard problems, i think, but the logic should be little different. The REST API calls are limited, more or less 6/10 per second, depend on exchange.
There is a risk that some trading orders will be lost.Opposite, It needs to a control on orders to discard some that could be replicate.

[kaythxbye]

Do they have pagination so that you can sync the progress?

[PumpkingWok]

It is the structure of json returns from get https://simex.global/api/trades/?pair_id=5
pair_id=5 is the ETH id.The trades are unique based on id, i think.
It is the call for retrieve the list of all currencies -> https://simex.global/api/currencies
Unfortunately some return an empty array or worst a redirect to SIMEX home webpage.

Edit: It is the pair https://simex.global/api/pairs, I'm going to check if it is could work.

[PumpkingWok]

Hi @kaythxbye,
I could implement a semi scraper that makes about 5 calls per second, it is simex API limit.
In theory i have just checked and in this way there should be no trading data loss.At most it could hold less crypto pair.

[frranck]

FYI the kraken wrapper is also using a rest API.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work for 0.345 ETH (96.03 USD @ $278.33/ETH) has been submitted by:

1. @PumpkingWok

@frranck please take a look at the submitted work:

• (Link Not Provided) by @PumpkingWok

---

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $19,326.17 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

The funding of 0.345 ETH (99.08 USD @ $287.18/ETH) attached to this issue has been approved & issued to @PumpkingWok.

• Learn more on the Gitcoin Issue Details page
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $21,719.70 more funded OSS Work available on the Gitcoin Issue Explorer

[frranck]

We got some super weird data from Simex,

> select * from trades WHERE symbol='ETH' and estimatedUSDPrice>300.00
name: trades
time                estimatedUSDPrice  exchange foreignTradeID pair   price      symbol volume
----                -----------------  -------- -------------- ----   -----      ------ ------
1539895195000000000 4615.114444810785  Simex    eca69f         ETHBTC 0.72014361 ETH    2.05369618
1539895207000000000 4616.15757310874   Simex    eca6a7         ETHBTC 0.72030638 ETH    1.60556773
1539895213000000000 4615.3772616167025 Simex    eca6b2         ETHBTC 0.72018462 ETH    -1.19331346
1539895219000000000 4614.556511840215  Simex    eca6b6         ETHBTC 0.72005655 ETH    1.19501631
1539895224000000000 4615.407061620299  Simex    eca6ba         ETHBTC 0.72018927 ETH    -2.22487649
1539895233000000000 4614.871046071728  Simex    eca6c3         ETHBTC 0.72010563 ETH    0.71491889
1539895240000000000 4616.9262850294845 Simex    eca6c7         ETHBTC 0.72042633 ETH    -1.44230553
1539895255000000000 4617.859826217436  Simex    eca6cd         ETHBTC 0.720572   ETH    -2.57568792
1539895262000000000 4615.9971657775495 Simex    eca6d3         ETHBTC 0.72028135 ETH    -1.29163809
1539895266000000000 4615.552729164764  Simex    eca6d6         ETHBTC 0.720212   ETH    0.68162108
1539895275000000000 4618.531383717852  Simex    eca6e1         ETHBTC 0.72067679 ETH    -1.89289692
1539895280000000000 4618.806889557559  Simex    eca6e3         ETHBTC 0.72071978 ETH    -2.06269205

Any clue where it could be coming from @PumpkingWok ?

[PumpkingWok]

Hi @frranck,
How are you ? I have just received an email with your new comment regard this issue but I don't see it in the page.Thanks.