The rsab from jacksmithinsulander

IF NEEDED The webscrapping functions for the FA script is made with selenium, which is really resource heavy, might need to make it somehow else in the future if that becomes a problem

Rework DB returns to be more functional

Rather than returning an array like:

[(70, 'eth', 'eth', 'ether', '0x1F98431c8aD98523631AE4a59f267346ea31F984', '0xA30EBe412e17aE91E80F36DED94B0AB09f99ca89', 1698514091, '0x53c6A49e8B07D7b05B81d65CaB43e360B8fbeA43', 'dev', '0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2', 'WETH', 'uniswap3', 18450185)]

It would be better to process that list within DB and return a Dict or something similar to this:

{
    id: 70,
    net: 'eth',
    pool_main_contract: '0x1F98431c8aD98523631AE4a59f267346ea31F984',
    pool_address: '0xA30EBe412e17aE91E80F36DED94B0AB09f99ca89',
    time_created: 1698514091,
    token1_address: '0x53c6A49e8B07D7b05B81d65CaB43e360B8fbeA43',
    token1_symbol: 'dev',
    token2_address: '0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2',
    token2_symbol: 'WETH',
    dex: 'uniswap3',
    creation_block: 18450185
}

Then the information can be easily accessed by doing pool.creation_block instead of the harder to read and debug pool[12]

fix a venv file

Balancer - Check that node is online before return

Puzzling piece by piece together so that everything communicates with eachouter

load balancing for price fetch

use different free node rpcs, switch between them

Add logger config

Structuring up folders and shit

Add check for venv

Add check in main.py to check if script in being run in a virtual env

Fix naming for chains across the project

Example: in conf.json, they chains are referred to by their short names, i.e "eth" while in older code such as rpc_list/swap_list they are referred to as their full names, i.e "ethereum"

Intermittent error during scanning - TypeError: 'NoneType' object is not iterable`

Intermittently when trying to iterate over any found pools, we get this error:

Traceback (most recent call last):
  File "C:\Users\nok13\Documents\GitHub\WAGMI\main.py", line 100, in <module>
    start_scanner()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\main.py", line 37, in start_scanner
    scanner.start()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\modules\scanner\scanner_main.py", line 60, in start
    for pool in latest_pools:
TypeError: 'NoneType' object is not iterable

Part where it originates in our code seems to be for pool in latest_pools:, line 60, in scanner_main.py

Change the tokenfetch function from checking the last blocks, to checking "live" new blocks / new pools

Sometimes cloudflare (ssl provider) asks for captcha on tokensniffer, we might have to handcode some workaround for that

Making the bot listen to more dexes on more chains (open for full list)

We'll need:
Uniswap on :
Arbitrum
Optimism
Polygon
Mainnet

Sushiswap on:
Arbitrum
Optimism
Polygon
Avax
Fantom
BSC
Mainnet

Pancakeswap on:
BSC

Quickswap on:
Polygon

Pangolin on:
Avax

Trader Joe on:
Avax
Arbitrum

Spookyswap on:
Fantom

Beethoven X on:
Fantom
Optimism

DODO on:
Mainnet
Polygon
Arbitrum
Optimism
Avalance
BSC
BOBA

Balancer on:
Mainnet

Apeswap on:
Mainnet
Polygon
BSC

Kyberswap on:
Mainnet
Polygon
Avalanche
Optimism
Arbitrum
BSC
Fantom

Rewrite get_price with new Balancer

Rewrite get_price with new Balancer and update any place it's been called to match it.

Add logger config to setup

Add individual FA results when adding to DB

New db implementation copies pool from 'pool_found'
Needs new db function to edit/add information, like the new 'passed_fa' columns

Redesign balancer to handle HTTP errors

If there's any error after the w3 is returned, it's not caught.
Redesign would handle w3 requests within balancer somehow so that errors can be caught and a new w3 can be fetched.

Intermittent error during scanning - Could not decode (...)

Intermittently when trying to get the symbol from ERC20 contracts, we get these errors:

Traceback (most recent call last):
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3\contract\utils.py", line 111, in call_contract_function
    output_data = w3.codec.decode(output_types, return_data)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\eth_abi\codec.py", line 145, in decode
    return decoder(stream)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\eth_abi\decoding.py", line 129, in __call__
    return self.decode(stream)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\eth_utils\functional.py", line 45, in inner
    return callback(fn(*args, **kwargs))
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\eth_abi\decoding.py", line 175, in decode
    yield decoder(stream)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\eth_abi\decoding.py", line 129, in __call__
    return self.decode(stream)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\eth_abi\decoding.py", line 144, in decode
    start_pos = decode_uint_256(stream)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\eth_abi\decoding.py", line 129, in __call__
    return self.decode(stream)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\eth_abi\decoding.py", line 199, in decode
    raw_data = self.read_data_from_stream(stream)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\eth_abi\decoding.py", line 306, in read_data_from_stream
    raise InsufficientDataBytes(
eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes.  Only got 0 bytes

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\nok13\Documents\GitHub\WAGMI\main.py", line 100, in <module>
    start_scanner()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\main.py", line 37, in start_scanner
    scanner.start()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\modules\scanner\scanner_main.py", line 75, in start
    token2_symbol = token2.functions.symbol().call()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3\contract\contract.py", line 461, in call
    return call_contract_function(
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3\contract\utils.py", line 129, in call_contract_function
    raise BadFunctionCallOutput(msg) from e
web3.exceptions.BadFunctionCallOutput: Could not decode contract function call to symbol with return data: b'', output_types: ['string']

Function where it originates in our code seems to be token2.functions.symbol().call(), line 75, in scanner_main.py

Scanner gets error 429 from RPCs sometimes

Add more extensive Loguru options to conf.json

Needs to be added to main.py as well so that the settings are read and set for the logger

Fix a configuration file system

Add creation block for found pools

Pricefetch save data as sql table

Intermittent error during scanning - json.decoder.JSONDecodeError (...)

Intermittently when trying to get the symbol from ERC20 contracts, we get these errors:

Traceback (most recent call last):
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3\_utils\encoding.py", line 226, in json_decode
    decoded = json.loads(json_str)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\nok13\Documents\GitHub\WAGMI\main.py", line 100, in <module>
    start_scanner()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\main.py", line 37, in start_scanner
    scanner.start()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\modules\scanner\scanner_main.py", line 75, in start
    token2_symbol = token2.functions.symbol().call()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3_balancer\main.py", line 96, in __call__
    return self._call_w3_func(func_call)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3_balancer\main.py", line 100, in _call_w3_func
    self._w3 = self.get_w3(self.net)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3_balancer\main.py", line 56, in get_w3
    return self.get_w3(network)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3_balancer\main.py", line 47, in get_w3
    if (self.is_contract.w3.is_connected()):
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3\main.py", line 445, in is_connected
    return self.provider.is_connected()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3\providers\base.py", line 110, in is_connected
    response = self.make_request(RPCEndpoint("web3_clientVersion"), [])
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3\providers\rpc.py", line 93, in make_request
    response = self.decode_rpc_response(raw_response)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3\providers\base.py", line 96, in decode_rpc_response
    return cast(RPCResponse, FriendlyJsonSerde().json_decode(text_response))
  File "C:\Users\nok13\Documents\GitHub\WAGMI\venv\lib\site-packages\web3\_utils\encoding.py", line 232, in json_decode
    raise json.decoder.JSONDecodeError(err_msg, exc.doc, exc.pos)
json.decoder.JSONDecodeError: Could not decode '<!doctype html>\n<html data-adblockkey="MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBANDrp2lz7AOmADaN8tA50LsWcjLFyQFcb/P2Txc58oYOeILb3vBw7J6f4pamkAQVSQuqYsKx3YzdUHCvbVZvFUsCAwEAAQ==_VgFwaC3qkh+DE2wImu1/lytWPYNOB31eyXnb+uIZRKRYVlBnImTzuGyM1epbMiBpu6GzoiTZH/BDSh0OrD9aUQ==" lang="en">\n<head>\n    <meta charset="utf-8">\n    <meta name="viewport" content="width=device-width, initial-scale=1">\n    <link rel="icon" href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVQI12P4//8/AAX+Av7czFnnAAAAAElFTkSuQmCC">\n    <link rel="preconnect" href="https://www.google.com" crossorigin>\n</head>\n<body>\n<div id="target" style="opacity: 0"></div>\n<script>window.park = "eyJ1dWlkIjoiOTMwMWM3Y2UtY2UwMy00NGY0LTg5MjgtNDIwZDY5MzM1NTc3IiwicGFnZV90aW1lIjoxNzAwMDgwNDA2LCJwYWdlX3VybCI6Imh0dHBzOi8vbm9kZXMudmVmaW5ldHdvcmsub3JnL3NtYXJ0Y2hhaW4iLCJwYWdlX21ldGhvZCI6IlBPU1QiLCJwYWdlX3JlcXVlc3QiOnt9LCJwYWdlX2hlYWRlcnMiOnt9LCJob3N0Ijoibm9kZXMudmVmaW5ldHdvcmsub3JnIiwiaXAiOiIxODUuNTYuODMuODMifQo=";</script>\n<script src="/brzrCPdBN.js"></script>\n</body>\n</html>\n' because of Expecting value: line 1 column 1 (char 0).: line 1 column 1 (char 0)

Function where it originates in our code seems to be token2.functions.symbol().call(), line 75, in scanner_main.py

Script to scan for known malicious packages before live testing

Implement config into scanner

Scanner currently scans all chains, regardless of what's set in conf.json

If It is not there yet, we need to add what dex a token is listed on in the database

Make a stable / eth balancing module

Main RPC balancer

Class with functions.
Basically an API to interface with a list of RPC nodes for different networks

Fine tune FA parametes

Create a program that calculates what price we want to buy the token at

Onionroute / vpn route the web3 connection

I think it could be worth it in order to load balancing the bots even further

Balancer - Take care of middleware before return

Figure out how to iterate fa.py over foundPools.db, pass the passed FA params (which ones are passed) to the db file

Draft for main file

Some kind of draft for main file which runs and manages the different parts

Fix a Code to actually buy the token

Intermittent issue in FA scan - 'NoneType' object has no attribute 'get'

Intermittently during FA scans, we get these errors:

Traceback (most recent call last):
  File "C:\Users\nok13\Documents\GitHub\WAGMI\main.py", line 103, in <module>
    iterate_fa()
  File "C:\Users\nok13\Documents\GitHub\WAGMI\modules\fa\fa_iterate.py", line 51, in iterate
    result = full_fa(
  File "C:\Users\nok13\Documents\GitHub\WAGMI\modules\fa\fa.py", line 129, in full_fa
    goplus_analysis = goplus_fa(chain_name_arr[0], token)
  File "C:\Users\nok13\Documents\GitHub\WAGMI\modules\fa\fa.py", line 24, in goplus_fa
    token_data = res_json.get("result", {}).get(token.lower(), {})
AttributeError: 'NoneType' object has no attribute 'get'

Function where it originates in our code seems to be res_json.get("result", {}).get(token.lower(), {}), line 24, in fa.py

jacksmithinsulander / rsab Goto Github PK

rsab's People

Contributors

Stargazers

Watchers

rsab's Issues

Recommend Projects

Recommend Topics

Recommend Org