siddhantgoel/beancount-dkb

Beancount Importers for DKB (Deutsche Kredit Bank) CSV Exports

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

beancount-dkb's Issues

expected keys for credit card have changed for me

Hi,

just today used this for the first time (thanks for sharing!). I cannot use poetry here right now, so I cannot run tests, hence I am unable to provide a pull request. Are you seeing the change to the CSV too?

•        expected_keys = set(['Von:', 'Bis:', 'Saldo:', 'Datum:'])
  

•        expected_keys = set(['Saldo:', 'Datum:', 'Zeitraum:'])
  

Hth,
Psychoticbeef

CSV changed again

It looks like the CSV changed yet again!

The first line is empty now. It doesn't mention the type of account anymore.

And I get:

    return line["Betrag"].rstrip(" €")
           ~~~~^^^^^^^^^^
KeyError: 'Betrag'

It appears to be called Betrag (€) now.

ERROR:root:Importer DKB CreditImporter.extract() raised an unexpected error: 'Zeitraum:'

I made this bug report in case someone stumbles upon the same problem as I did. I'm new to beancount and after my initial setup of my accounts I stumbled upon beancount-dkb extractor.

I downloaded my credit card and account transactions from DKB and tried the DKB extractor which failed with following error:

$ bean-extract dkb.config ./incoming
ERROR:root:Importer DKB CreditImporter.extract() raised an unexpected error: 'Zeitraum:'
Traceback (most recent call last):
  File "/home/max/.local/pipx/venvs/beancount/lib/python3.9/site-packages/beancount/ingest/extract.py", line 182, in extract
    new_entries = extract_from_file(
  File "/home/max/.local/pipx/venvs/beancount/lib/python3.9/site-packages/beancount/ingest/extract.py", line 67, in extract_from_file
    new_entries = importer.extract(file, **kwargs)
  File "/home/max/.local/pipx/venvs/beancount/lib/python3.9/site-packages/beancount_dkb/credit.py", line 128, in extract
    expected_keys.remove(key)
KeyError: 'Zeitraum:'

After multiple exports from the DKB side I noticed that the file content changes based on the used filters on the DKB web-page. The beancount-dkb extractor only works if I used a "von"-"bis" filter. Any other filters (like "last 30 days") exported a file with the line "Zeitraum" instead of "Von:" and "Bis:", which apparently confuses the extractor.

Workaround: set your filter on the DKB web-page to a Von-Bis date and bean-extract works fine.

Here a short feedback which could improve your project. I had no idea how to work with beancount extractors. The "Usage" chapter could be improved to add minimal context how to use beancount-dkb. Or add a link to Beancount - Importing External Data to provide the context.

Btw - thanks for the work to make the DKB extractor, it saved me the time to make one.

New onlinebanking renamed CSV fields

DKBs new online banking renamed the fields in the CSV:

Checking

Old columns

"Buchungstag";"Wertstellung";"Buchungstext";"Auftraggeber / Beg�nstigter";"Verwendungszweck";"Kontonummer";"BLZ";"Betrag (EUR)";"Gl�ubiger-ID";"Mandatsreferenz";"Kundenreferenz";

New columns

"Buchungsdatum";"Wertstellung";"Status";"Zahlungspflichtige*r";"Zahlungsempfänger*in";"Verwendungszweck";"Umsatztyp";"Betrag";"Gläubiger-ID";"Mandatsreferenz";"Kundenreferenz"

Tagesgeld

Old columns

"Umsatz abgerechnet und nicht im Saldo enthalten";"Wertstellung";"Belegdatum";"Beschreibung";"Betrag (EUR)";"Urspr�nglicher Betrag";

New columns

"Belegdatum";"Wertstellung";"Status";"Beschreibung";"Umsatztyp";"Betrag";"Fremdwährungsbetrag"

Add Github topics to the repository

Hi, can you add the following topic to this repository?

• https://github.com/topics/dkb

Thanks!

[Feature Request] Support `description_patterns` for `ECImporter`

Thanks for the great work.

It would be very useful, if you'd add support for the description_patterns to the ECImporter as well. If you send money for different purposes to the same receiver (e.g. rent and costs for utilities in separate transactions), then you can't distinguish the transactions. Same holds true if send money to yourself (transfering money to different saving accounts, etc).

Using the more unique transaction description could help here.

As this might add ambiguity for the importer how to categorize I'd suggest that the order is: if description_pattern and payee_pattern matches, then payee_pattern gets disregarded as it is more general.

[Feature Request] Support amount in payee_patterns for ECImporter

Another nice to have option would be a third optional argument for the payee_patterns: the amount.
I've multiple transactions from the same person, but they differ in description and amount.

Evaluating the description_pattern introduced in #108 works in this case also. But it would be easier in some cases with repeating transactions to differentiate the transactions by the transaction amount.

Credit Card Saldo number format

Hi,

when I export Credit Card stuff to CSV at my DKB account, the "Saldo" information (in the header) seems to have a different format, than your importer expects it to be.

beancount-dkb/beancount_dkb/credit.py

Lines 117 to 120 in 469d195

	elif key.startswith('Saldo'):
	self._balance_amount = Amount(
	fmt_number_de(value.rstrip(' EUR')), self.currency
	)

For me, the number in the CSV is already in a machine-readable format (e.g. 4664.87), and does not have to be converted. Could you please check this?

Thanks and Regards
Fred

Encoding Issue?

First time using the importer, so this may be user error or specific to my system...

The csv downloaded from DKB is encoded with ISO 8859-1, so I configured the import.config as:
file_encoding='ISO-8859-1',

My .beancount file is encoded with utf-8, defined by its first line:
;; -*- mode: beancount; coding: utf-8; fill-column: 400; -*-

When running the importer with bean-extract path_to_config path_to_csv >> path_to_beancount, the content of my csv is parsed and appended, however there seems to be some sort of encoding mismatch, see attached screenshot. Visual Studio Code guesses that the generated encoding is UTF-16 LE (opening with that encoding, the space and line breaks appear correct, however Umlaute are incorrect, so something is off there as well)

Please support Tagesgeld

DKB also has a savings product called "Tagesgeld".

The first line is:

"Konto";"Tagesgeld DE2xxxxxxxxxxxxxxxxxxx"

Everything else is the same.

If I replace Tagesgeld with Girokonto in the CSV, the importer works.

Has DKB's CSV file format changed for credit card transactions?

Following a multi-year hiatus I am trying to resurrect my accounting system. I'm trying to import credit card transactions, and I know this has worked in the past, but somehow now it doesn't. I suspect that DKB has silently changed the format of the exported CSV file, but I have no old records to prove it.

Here's what the new file format looks like:

"Karte";"Visa Kreditkarte";"5678 •••• •••• 1234"
""
"Saldo vom 20.04.2024:";"--1234,56 EUR"
""
"Belegdatum";"Wertstellung";"Status";"Beschreibung";"Umsatztyp";"Betrag (€)";"Fremdwährungsbetrag"
"18.04.24";"";"Vorgemerkt";"Supermarkt";"Im Geschäft";"-21,8";""
"17.04.24";"";"Vorgemerkt";"TANKSTELLE";"Im Geschäft";"-62,05";""
"17.04.24";"18.04.24";"Gebucht";"DAS FUTTERHAUS";"Im Geschäft";"-52,97";""
"17.04.24";"18.04.24";"Gebucht";"Restaurant XYZ";"Im Geschäft";"-56,4";""
"16.04.24";"17.04.24";"Gebucht";"Baeckerei ABC";"Im Geschäft";"-7,19";""
...

Running bean-extract with configuration

from beancount_dkb import ECImporter, CreditImporter

CONFIG = [
    CreditImporter(
        "5678 •••• •••• 1234",
        'Assets:DKB:Credit',
        currency='EUR',
    )
]

on this file produces an empty output (execpt for the emacs mode-line) for me.

Looking at test files, my suspicion is that the first line has an unexpected format; If I add a hyphen to Visa-Kreditkarte and remove the subsequent ";" then I receive the error message

ERROR:root:Importer DKB CreditImporter.extract() raised an unexpected error: '"Belegdatum";"Wertstellung";"Status";"Beschreibung";"Umsatztyp";"Betrag";"Fremdwährungsbetrag"' is not in list
Traceback (most recent call last):
  File "/path/venv-3.12/lib/python3.12/site-packages/beancount/ingest/extract.py", line 182, in extract
    new_entries = extract_from_file(
                  ^^^^^^^^^^^^^^^^^^
  File "/path/venv-3.12/lib/python3.12/site-packages/beancount/ingest/extract.py", line 67, in extract_from_file
    new_entries = importer.extract(file, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/venv-3.12/lib/python3.12/site-packages/beancount_dkb/credit.py", line 100, in extract
    return self._extract(file, extractor)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/venv-3.12/lib/python3.12/site-packages/beancount_dkb/credit.py", line 109, in _extract
    header_index = lines.index(extractor.HEADER)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: '"Belegdatum";"Wertstellung";"Status";"Beschreibung";"Umsatztyp";"Betrag";"Fremdwährungsbetrag"' is not in list
;; -*- mode: beancount -*-

I am using

beancount==2.3.6
beancount-dkb==0.19.0

with python v3.12.2. Thoughts and pointers will be appreciated, thank you.

Importer picks Belegdatum instead of Wertstellung, which may result in failing balance assertions

Sometimes, if the Saldo Datum of a CSV export is between the Wertstellung and the Belegdatum (for credit cards) or Wertstellung and Buchungstag (for checking accounts) of one or more transactions you will have a failing balance statement.

Note that in the following example the Datum is on 02.07.2021 and the two transaction Wertstellung are on 05.07.2021 and 01.07.2021:

"Kreditkarte:";"xxxxxxxxxxxxxxxxx";

"Zeitraum:";"seit der letzten Abrechnung";
"Saldo:";"-18.63 EUR";
"Datum:";"02.07.2021";

"Umsatz abgerechnet und nicht im Saldo enthalten";"Wertstellung";"Belegdatum";"Beschreibung";"Betrag (EUR)";"Urspr�nglicher Betrag";
"Nein";"05.07.2021";"01.07.2021";"KAUFLAND";"-9,56";"";
"Nein";"01.07.2021";"01.07.2021";"ALDI";"-18,63";"";

this becomes the following ledger

2021-07-01 * "ALDI"
  Assets:DE:DKB:Creditcard                   -18.63 EUR
  Expenses:Food:Groceries

2021-07-01 * "KAUFLAND"
  Assets:DE:DKB:Creditcard                    -9.56 EUR
  Expenses:Food:Groceries

2021-07-02 balance Assets:DE:DKB:Creditcard     -18.63 EUR

Which will fail balance checks.

I believe the correct export should be to pick the Werstellung field for a transaction date instead, which results in

2021-07-01 * "ALDI"
  Assets:DE:DKB:Creditcard                   -18.63 EUR
  Expenses:Food:Groceries

2021-07-02 balance Assets:DE:DKB:Creditcard     -18.63 EUR

2021-07-05 * "KAUFLAND"
  Assets:DE:DKB:Creditcard                    -9.56 EUR
  Expenses:Food:Groceries

which is correct.

New CSV export format seems to use `€` instead of `EUR`

A current CSV export from DKB uses the character € instead of EUR and a non-breaking-space. The current code just fails because of the missing EUR.

""
"Kontostand vom 02.02.2024:";"123,45 €"
""

Missig build dependency for setuptools

The dependency for setuptools is missing and so a local install fails:

$ pip install -e /some_local_path/beancount-dkb/
Obtaining file:///some_local_path/beancount-dkb
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Installing collected packages: beancount-dkb
  Running setup.py develop for beancount-dkb
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [1 lines of output]
        ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
    ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
rc: 1

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.