# Note: Install Python 3
# Note: install Poetry for Linux
$: curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
# Note: install Poetry for Windows
$: (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python
$: python get-poetry.py --uninstall
$: poetry install # install all dependencies
$: pip install dist/eve-tools-0.0.1-py3-none.any.whl
$: poetry-template
$: poetry shell
$: cd docs
# Note: review source/conf.py and source/index.rst
$: make html
# Note: see docs in docs/build/apidocs/index.html
$: poetry run python ./eve_tools/chat/runner.py # analyze collections of chat logs
$: poetry run python ./eve_tools/runner.py
$: poetry run pytest --durations=0
$: poetry run pytest --cov=eve_tools --cov-report=html -m "not integration_test" tests
#: Note: see unit test coverage report in htmlcov/index.html
Dependencies, Python version and the virtual environment are managed by Poetry.
$: poetry search Package-Name
$: poetry add Package-Name[==Package-Version]
Define project entry point and metadata.
Configure Python libraries.
$: poetry run black .
$: poetry config pypi-token.pypi PyPI-API-Access-Token
$: poetry publish --build
https://pypi.org/project/eve-tools/
Use swagger-editor to generate a client library for ESI.
To receive back the response headers as well as the data change:
def __call_api(self, ...):inapi_client.pyto return the desired valuekwargs['_return_http_data_only'] = Truetokwargs['_return_http_data_only'] = Falseto change the default behaviour
Download the latest SDE. SQLite's sqlite-latest.sqlite.bz2 is recommended.
N: 100 000
Batch: 0.5s
Individual: 46.2s
Choice:
- pyswagger: doesn't cache easily into a file; implements swagger api !!! hasn't been updated for a very long time!
- diskcache: can cache files
Alternatives:
- esipy: the author abandoned an adjacent project, has poor documentation and short lived cache; easy EVE API access, implemented security, implements swagger api, retries, ETag, ...
- swagger-codegen: generates an enormous code overhead; creates a whole python project, tests, data models and api function
- Differential caching
- sort BPCs
- what happens when a contract expires? update cache? ignore (current, but bad)?
- longer caching (30 days)
- account for errors like the API returning ([], 200)
- SDE is required - analyze it
- do more research
- Update Prices - select all items and click Download Prices
- Blueprints - select one - tick calculate build/buy - see if you should buy or build a component
- File - add character - login - set as default
- Manufacturing list - compare all owned blueprints to figure out which one you should build first
- SVR stands for Sales Volume Ratio
- Python <-> SQLite (sqlite3 lib)
- Describe EVE SDE
- list SDE locations, headers and description
- explore the SDE
- Get data from the SQLite DB
- Blueprint cost? Example, Vigilant
https://developers.eveonline.com/ - register a token
Algo:
- sort BPOs by ISK per hour
- get price for blueprint from esi
- price is the lowest individual contract forge blueprint price
- account for efficiency research
- get time for blueprint from sde
PS D:\Repositories\EVE-Tools> poetry run python .\eve_tools\runner.py
2021-08-09 14:18:16.880 | INFO | eve_tools.esi.esi_file_cache:get_response:30 - who=API Function:get_contracts_public_region_id, Expire Timedelta:0:30:00, Args:(10000002,), Kwargs:{'p
age': 1}, what=('get_contracts_public_region_id', (10000002,), {'page': 1}), why=Hit cache, how=Access cache
Traceback (most recent call last):
File ".\eve_tools\runner.py", line 51, in <module>
run()
File ".\eve_tools\runner.py", line 46, in run
sys.exit(main(sys.argv[1:]))
File ".\eve_tools\runner.py", line 27, in main
item_exchanges = datastore.get_contracts_public_region_id(10000002)
File "d:\repositories\eve-tools\eve_tools\helper\eve_datastore.py", line 57, in get_contracts_public_region_id
contracts = self.get_paged_data(self.contract_api.get_contracts_public_region_id, timedelta(minutes=30),
File "d:\repositories\eve-tools\eve_tools\helper\eve_datastore.py", line 66, in get_paged_data
response = self.cache.get_response(ApiPackage(api_function, expire_timedelta, *args, page=1))
File "d:\repositories\eve-tools\eve_tools\esi\esi_file_cache.py", line 31, in get_response
File "d:\repositories\eve-tools\eve_tools\esi\esi_file_cache.py", line 45, in get_from_cache
response = self.cache.get(key)
File "C:\Users\MislavJ\AppData\Local\pypoetry\Cache\virtualenvs\eve-tools-NPmAMQhe-py3.8\lib\site-packages\rapdevpy\file_cache.py", line 23, in get
return self.cache.get(digest)
File "C:\Users\MislavJ\AppData\Local\pypoetry\Cache\virtualenvs\eve-tools-NPmAMQhe-py3.8\lib\site-packages\diskcache\core.py", line 1176, in get
value = self._disk.fetch(mode, filename, db_value, read)
File "C:\Users\MislavJ\AppData\Local\pypoetry\Cache\virtualenvs\eve-tools-NPmAMQhe-py3.8\lib\site-packages\diskcache\core.py", line 280, in fetch
return pickle.load(reader)
ModuleNotFoundError: No module named 'eve_tools.data_layer.esi_response'
Warning! It seems that the cache is extremely sensitive to where the pickled/cached Class is located!
Where do I store the ESi data as opposed to SDE data?
Two databases:
- SDE database can be swapped with the new updated one without a problem
- you cannot query both with a SQL SELECT -> need to process data on the application layer, SDE and ESI side
- ESI database can be moved independently of the SDE database
- ESI database will be inserted and updated quite a lot while the SDE won't -> optimize them for different things
- cannot foreign key link entities
Went with two databases so I can update the SDE databse more easily when a new oe arrives.
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'public_contracts.issuer_id' could not find table 'characters' with which to generate a foreign key to target col
umn 'character_id'
Error appears if you did not import the associated Class.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent