hoffie/dkb-visa

DKB transaction download and conversion (CSV, QIF)

dkb-visa's Issues

IndexError: list index out of range

I get the following trace when starting:

Traceback (most recent call last):
File "/home/uwe/dkb-banking/dkb-visa/dkb.py", line 413, in
fetcher.login(args.userid, pin)
File "/home/uwe/dkb-banking/dkb-visa/dkb.py", line 61, in login
br.form = list(br.forms())[0]
IndexError: list index out of range

html included in csv file

Note that sometimes the original csv-file includes information, formatted via html.
I solved it like this:

    # check that unparseable lines are not included as transactions
    # dkb should really fix their csv export, they should have the
    # money for that, right?
    csv_text = csv_text.splitlines()
    csv_text_filtered = []
    for line in csv_text:
        if "<br />" in line:
            continue
        else:
            csv_text_filtered.append(line)
    csv_text = "\n".join(csv_text_filtered)

This will exclude transactions that have <br /> in their description.

from_date greater than 3 years ago gives initially displayed transactions and does not create an error.

I tested this via dkbweb.py which calls dkb.py, so this should apply to dkb.py as well.

If you specify a from_date like 1.1.1970, you won't get an error from mechanize. But it triggers another issue: The csv export will only contain the initially visible transactions because the date is not submittable. This means that the page won't update and the csv-file won't be generated with the new dates.

I "fixed" this for myself by fetching all the transactions from the last year every time and later I filter the results via sets. See: https://github.com/inktrap/dkb-export/blob/master/dkbweb.py#L215. Btw.: I also checked that BASEURL is an https-url. :)

Edit: Just to clarify: The allowed from_date is set server side and constant.

to/from date takes value date and not booking date

The command line download start date (--to-date) takes the value dates and not the booking dates. Not really sure what the intention is but from a use case perspective, it might not make sense:

1. I run this script automatically every day to download all card transactions that happened yesterday
2. These new transactions are added automatically to GnuCash so GnuCash is updated every night.

Problem:
If a value date is two days ago or from last week or last month (because a supplier books very late or international transactions or whatever) or there is a refund (which is usually the value date of the original booking), I would have to download all transactions from the last X months.

Only solution I can think of, is do download all transactions from last year and then filter the file by booking date. Maybe this behaviour can be enabled by an additional flag.

Let me know what you think. Am happy to provide the fix.

Unable to find search period radio box

Running dkb.py as usual results in:

Navigating to 'Umsätze'...
Selecting transactions in time frame 03.03.2017 - 28.05.2017...
Traceback (most recent call last):
  File "./dkb.py", line 427, in <module>
    fetcher.select_transactions(args.cardid, from_date, args.to_date)
  File "./dkb.py", line 190, in select_transactions
    self._select_all_transactions_from(form, from_date, to_date)
  File "./dkb.py", line 115, in _select_all_transactions_from
    raise RuntimeError("Unable to find search period radio box")
RuntimeError: Unable to find search period radio box

valuta_date empty for soll/haben

An entry like the following one has an empty COL_VALUTA_DATE, because it does not show a
transaction, only the Soll or Haben (that you currently owe or have). So this
should not be one of the transactions.

"Nein";"01.01.2015";"";"Soll";"-00,01";"";

However filtering and matching transactions to accounts should happen later, so
I used this simple but ugly fix:

if len(line[self.COL_VALUTA_DATE].strip()) == 0:
    return re.sub('.*?(\d{1,2})\.(\d{1,2})\.(\d{2,4}).*?', r'\2/\1/\3',
        line[self.COL_DATE])
else:
    return re.sub('.*?(\d{1,2})\.(\d{1,2})\.(\d{2,4}).*?', r'\2/\1/\3',
        line[self.COL_VALUTA_DATE])

Otherwise the generated qif-file won't be importabable

Add Github topics to the repository

Hi, can you add the following topics to this repository?

• https://github.com/topics/dkb
• https://github.com/topics/banking
• https://github.com/topics/finance

Thanks!

option "--qif-account" is not implemented.

It seems the option "qif-account" is never used. I tried to use it to be able to distinguish between two different accounts when merging both into a common gnucash document and saw no effect. Also, I could confirm in the source code that the value written by the ArgumentParser is never used.

Anyway, thanks for the great rewrite after "web_bank.py" does not work anymore. As a workaround I proposed a feature to web_bank.py and used it myself just doing the conversion from manually downloaded csv files to qif: okurz/web_bank@4175542

Script emits Warning: No parser was explicitly specified

Everytime I run this tool it emits the following warning:

/usr/lib/python2.7/dist-packages/bs4/__init__.py:166: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("lxml"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

To get rid of this warning, change this:

 BeautifulSoup([your markup])

to this:

 BeautifulSoup([your markup], "lxml")

  markup_type=markup_type))

Does anyone else get this too?

Unable to find transaction selection form

Since yesterday, I get this error message:

Start time: 25.08.2017  
PIN: 
Starting login as user xxx...
Navigating to 'Umsätze'...
Selecting transactions in time frame 25.08.2017 - 01.09.2017...
Traceback (most recent call last):
  File "/usr/bin/dkb.py", line 416, in <module>
    fetcher.select_transactions(args.cardid, from_date, args.to_date)
  File "/usr/bin/dkb.py", line 173, in select_transactions
    br.form = form = self._get_transaction_selection_form()
  File "/usr/bin/dkb.py", line 99, in _get_transaction_selection_form
    raise RuntimeError("Unable to find transaction selection form")
RuntimeError: Unable to find transaction selection form

Did not find the time to look into this in detail, but maybe DKB changed the HTML structure.

no data entries can be found at all

Since some days (did not use dkb-visa for some months in before) I can not get any data entries at all, anymore. Maybe webpage form changed again.
Running dkb-visa with pdb debugger and checking the form input IMHO the request URL is fine but the response has an empty block like

…
<input name="$$$event_search" type="hidden" value="search"/>
</form>
</div>
</div>
</div>
</div>
</div>
<div class="clearfix richfooter_wrap" id="richfooter_wrap">
…

where I would actually expect some data as I can see them with the firefox inspector. Does anyone still have success on dkb webpage?

Unable to find transaction selection form

Last week and today (19.9.2019) I tried to obtain my transactions and got the following error message and stack trace:

richi@yang:~/Coding/dkb-visa$ ./dkb.py --userid XXXX --cardid YYYY --output dkb_visa_card_umsaetze.qif
Start time: 21.06.2019
PIN: 
Starting login as user XXXX...
Navigating to 'Umsätze'...
Selecting transactions in time frame 21.06.2019 - 18.09.2019...
Traceback (most recent call last):
  File "./dkb.py", line 493, in <module>
    fetcher.select_transactions(args.cardid, from_date, args.to_date)
  File "./dkb.py", line 242, in select_transactions
    br.form = form = self._get_transaction_selection_form()
  File "./dkb.py", line 168, in _get_transaction_selection_form
    raise RuntimeError("Unable to find transaction selection form")
RuntimeError: Unable to find transaction selection form

Python 3 support

Hello,

it looks like mechanize is not available for python3.

I'm currently trying to integrate dkb-visa into my own script, which unfortunately is written in python3. Do you have any plans to migrate to python3? How could I help with the migration?

The following StackOverflow comment mentions some mechanize alternatives for python3, which might be helpful: https://stackoverflow.com/a/31774959

Anway, thank you for the awesome script!