Data importer confuses thousands separator for decimal separator when decimal separator is missing.

[nonsavant]

Data importer confuses thousands separator for decimal separator when decimal separator is missing.

[JC5]

Could you share the logs of the data importer when it runs into this amount? It should be able to deal with this.

[nonsavant]

@JC5 Backend (docker) logs, or the output that is shown in the browser during the import?

[JC5]

Docker logs.

https://docs.firefly-iii.org/how-to/general/debug/

[nonsavant]

OK. First glance in the logs I couldn't see anything even vaguely relevant, so I did the import again. Here we go:

[2024-01-16 17:31:22] local.DEBUG: Now submitting transaction 3/3  
[2024-01-16 17:31:22] local.DEBUG: Duplicate detection method is "none", so this method is skipped (return true).  
[2024-01-16 17:31:22] local.DEBUG: Transaction #3 is not checked beforehand on uniqueness.  
[2024-01-16 17:31:22] local.DEBUG: Going to map data for this line.  
[2024-01-16 17:31:22] local.DEBUG: Submitting to Firefly III: {"group_title":null,"error_if_duplicate_hash":false,"transactions":[{"type":"withdrawal","date":"XXXX","currency_id":1,"currency_code":null,"amount":"1.000000000000","description":"XXXX","source_id":1,"source_name":null,"destination_id":null,"destination_name":"XXXX","original_source":"jc5-data-import-v1.4.1","tags":[],"source_iban":null,"source_number":null,"source_bic":null,"destination_iban":null,"destination_number":null,"destination_bic":null}]}  
[2024-01-16 17:31:23] local.DEBUG: isReadyForFileStep("submit")  
[2024-01-16 17:31:23] local.DEBUG: isReadyForFileStep: Return true  
[2024-01-16 17:31:23] local.DEBUG: Now at App\Http\Controllers\Import\SubmitController::status(submission-hjUj2s9LHFjV)  
[2024-01-16 17:31:23] local.DEBUG: Now in startOrFindJob(submission-hjUj2s9LHFjV)  
[2024-01-16 17:31:23] local.DEBUG: Try to see if file exists for job submission-hjUj2s9LHFjV.  
[2024-01-16 17:31:23] local.DEBUG: Status file exists for job submission-hjUj2s9LHFjV.  
[2024-01-16 17:31:23] local.DEBUG: Status: submission_running  
[2024-01-16 17:31:23] local.DEBUG: Messages: 1, warnings: 0, errors: 1  
172.18.0.1 - - [16/Jan/2024:17:31:23 +0100] "GET /import/submit/status?identifier=submission-hjUj2s9LHFjV HTTP/1.1" 200 1477 "http://XXXX/import/submit" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0"
[2024-01-16 17:31:23] local.INFO: Add message to index #2: Created withdrawal <a target="_blank" href="XXXX">XXXX</a> (EUR 1)  
[2024-01-16 17:31:23] local.DEBUG: Add message on index #2 (line no. 3): Created withdrawal <a target="_blank" href="XXXX>#XXXX"</a> (EUR 1)  
[2024-01-16 17:31:23] local.DEBUG: Now in App\Services\Shared\Import\Status\SubmissionStatusManager::storeSubmissionStatus(submission-hjUj2s9LHFjV): submission_running  
[2024-01-16 17:31:23] local.DEBUG: Messages: 2, warnings: 0, errors: 1  
[2024-01-16 17:31:23] local.INFO: Created withdrawal <a target="_blank" href="XXXX">#XXXX</a> (EUR 1)  
[2024-01-16 17:31:23] local.DEBUG: Going to add import tag to transaction group #8359  
[2024-01-16 17:31:24] local.DEBUG: Added import tag to transaction group #8359  
[2024-01-16 17:31:24] local.INFO: Done submitting 3 transactions to your Firefly III instance.

Don't be distracted by the errors: 1 message, there was a zero amount transaction in the data.

[nonsavant]

Sorry, just checked the link - should I rerun with further debugging enabled, or is the above enough?

[JC5]

Search for these kinds of line:

Decimal character in "%s" seems to be a dot.
Converted amount from "%s" to "%s".
No decimal character found. Converted amount from "%s" to "%s".

Probably earlier in the process, and the %s things are just placeholders.

[nonsavant]

How does this look?

[2024-01-16 17:31:03] local.DEBUG: Start with amount "-1.000"  
[2024-01-16 17:31:03] local.DEBUG: Stripped "-1.000" to "-1.000"  
[2024-01-16 17:31:03] local.DEBUG: Decimal is still NULL, probably number with >2 decimals. Search for a dot.  
[2024-01-16 17:31:03] local.DEBUG: Searched from the left for "." in amount "-1.000", assume this is the decimal sign.  
[2024-01-16 17:31:03] local.DEBUG: Converted amount from "-1.000" to "-1.000".  
[2024-01-16 17:31:03] local.DEBUG: Final NUMERIC value is: "-1.000"  
[2024-01-16 17:31:03] local.DEBUG: Stored column #4 with value "-1.000" and role "amount" in field "amount"  

Wish my developers would write logging like this :)

[JC5]

Perfect. I'll pick it up.

And thanks for the compliments 😁

[github-actions]

Hi there! This is an automatic reply. Share and enjoy

This issue is now 🔒 closed. Please be aware that closed issues are not monitored by the developer of Firefly III.

• If the original bug is not actually fixed, please open a new issue. Refer to this issue for clarity.
• Follow-up questions must be posted in a new discussion
• Further replies to this issue may get no response.

If there is more to discuss, please open a new issue or discussion.

Thank you for your contributions.