cybermaggedon/gnucash-uk-vat

A bridge between GnuCash and UK HMRC VAT return MTD APIs for automating VAT returns

gnucash-uk-vat

Introduction

This is a utility which manages HMRC (UK) VAT returns in accordance with HMRC MTD directives for users of the GnuCash accounting system. It can study your GnuCash accounts, compare this with your HMRC VAT obligations, and produce the VAT return to meet your obligations. As an optional step, once the VAT return is filed, a bill can be posted describing the VAT owing, and posted to an Accounts Payable account.

Discuss

There's a #gnucash-uk-vat channel on our Discord server if you want to discuss... https://discord.gg/3cAvPASS6p

A word from our sponsors :)

If you need low cost, automated accounts filing for small businesses, Accounts Machine has a commercial offering which incorporates the functionality you see here in a web service.

The proposition is simple. Accounts Machine does not offer an accounting system. You manage your accounts using GnuCash or anything which produces CSV output. You manage your accounts, manage your invoices, manage receipts, and record everything in your application under your control. When it comes to filing returns, you upload your accounts to accountsmachine.io and use the system to file VAT returns, corporation tax returns, and company accounts with Companies House. The production of returns is as automated as possible. Accounts production conforms to iXBRL specifications, and HMRC VAT MTD.

This service is able to simplify accounts filing for simple businesses. Your company should be a Limited Company, conform to micro-entity requirements, and have a single trade. This service is aimed at making life easier for startups and small businesses.

The roadmap is for launch in 2Q2022. VAT filing is complete, just going through the final validation process with HMRC. Demo is here

Visit https://accountsmachine.io. Accounts Machine is registered with HMRC and Companies House for filing purposes and have successfully filed VAT, corp tax and company accounts.

Join our discord service to keep on top of latest progress: https://discord.gg/3cAvPASS6p

Enthusiastic early adopters will find free deal links on the Discord server. Looking forward to filing for you. :)

GnuCash backends

Two ways of interacting with your GnuCash accounts are supported:

• The gnucash module is bundled only with Linux GnuCash packages, and can only by used on Linux.
• The piecash module is pure Python and can be obtained from package repositories. This module only supports GnuCash files saved in Sqlite files or a Postgres database. You can convert a GnuCash XML file into Sqlite by using the "Save As..." option in GnuCash.

Status

This is a command-line utility, with a GTK-based dialog mode which removes the need to know about configuration files or command-line options. I have used this to submit my own VAT returns. If you want to join the party come to the #gnucash-uk-vat channel on Discord server https://discord.gg/3cAvPASS6p and I'll try to help you through the process.

Credentials

In order to use this, you need production credentials (client ID and secret) for the VAT submission API. HMRC does not permit these credentials to be shared publicly:

We have checked with our colleagues who look after HMRC’s API Platform. They have advised that this is not allowed and would be likely to result in your Developer Hub application being blocked. We recommend that instead of sharing these credentials that you inform your users how they can register for their own Developer Hub application and use its credentials with your code.

You can either apply for production credentials using the HMRC developer portal (you need to register), or contact me as I can share a few credentials privately.

Developer hub: https://developer.service.hmrc.gov.uk/api-documentation/docs/using-the-hub

You click 'Get production credentials' and enter details about the application. I call this application gnucash-uk-vat, you should call it something different. When you apply for credentials, HMRC will contact you to fill in an application. As I understand it, this involves going through a full acceptance test which involves testing against the sandbox.

Once you have credentials, you should put them in the config.json file in the application section placeholders.

Installing

To install directly from a git repository:

pip3 install git+https://github.com/cybermaggedon/gnucash-uk-vat

To install/update from the local checkout, a wrapper script can be used:

[git-bash-prompt]> ./setup.sh

See Installing.

Usage

There are two modes:

• Assist mode, which is a GTK-based dialog-driven. This automates everything including configuration file setup.
• CLI mode, which is purely CLI-based and has no dependency on PyGTK.

GnuCash accounts structure

See Accounts.

Local test service

There is a local test service which allows you to emulate the HMRC VAT service with test data under your control.

See local test.

Sample accounts

A sample account file is included at accounts/accounts.gnucash. This account file contains some transactions dated in the 1Q17 quarter which match the test data in HMRC's Sandbox. There are also some transactions in 2020 which match the obligations in the dummy-vat-service data.

Licences, Compliance, etc.

Privacy

gnucash-uk-vat is hosted by you. It runs on your computer, accesses information from your accounts, and forwards data using the HMRC APIs. Everything is within your control. No other network systems are used, and no information is transmitted to other parties.

Additional data (configuration and credentials) is stored on your filesystem under your control and you should manage the credential files as you would any password or other secret.

Licence

Copyright (c) 2020-2021, Cyberapocalypse Limited Copyright (c) 2021-2022, Accounts Machine Limited

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

gnucash-uk-vat's Issues

Exception: module 'gnucash' has no attribute 'SessionOpenMode'

Hi cybermaggedon, thank you very much for this library!

My setup:
-Debian 10 Buster
-gnucash/oldstable,now 1:3.4-1+b10
(Version: 3.4, Build ID: 3.4+ (2018-12-30))

I'm trying to get the sandbox test working, and I'm hitting an exception when gnucash-uk-vat is trying to get the session mode:
(see accounts_gnucash.py#l42)
mode = gnucash.SessionOpenMode.SESSION_READ_ONLY

Exception: module 'gnucash' has no attribute 'SessionOpenMode'

From a different issue (MatzeB/pygnucash#12), I tried a different strategy
I imported "gnucash" in interactive shell, and indeed there was no SessionOpenMode object.
Saw no enums of similar name either. Doing print(gnucash.__file__) did point me to the lib file, here:
/usr/lib/python3/dist-packages/gnucash
And grep'ing for SESSION_READ_ONLY, found no matches.
Checking the file headers, gnucash_core_c.py is of the Version 3.0.10, and latest copyright date I saw was 2010.
So I'm guessing I don't have the right lib, or my gnucash is too old.

Also tried installing pygnucash from pypi, however that lib is completely different (at least the one I have seems to be the right one, but perhaps outdated).

Have you had a similar issue before?

Again, thanks for your efforts (it's amazing to not have to proprietary software for tax).

Need to document GnuCash accounts assets vs liabilities

People have reported negative numbers turning up in VAT returns, which then fail to file (negative numbers are invalid). Caused by using the 'wrong' asset vs liability account for recording VAT.

The documentation should explain the impact of using asset vs liability accounts which cause these negative values.

tabulate is a dependency, not included in package

Add tabulate to setup.py

Running on Windows produce event loop errors

Once the --init-config problem is worked around (#13), get event loop problems on Windows. Looks like all operations work (fetching obligations, fetching VAT returns, computing data), but the event loop throws an exception on exit. This is using Python 3.10 as installed from VS on Win 10.

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x0000029F6404F0A0>
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\asyncio\[proactor_events.py](http://proactor_events.py/)", line 116, in __del__
    self.close()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\asyncio\[proactor_events.py](http://proactor_events.py/)", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\asyncio\[base_events.py](http://base_events.py/)", line 750, in call_soon
    self._check_closed()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\asyncio\[base_events.py](http://base_events.py/)", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

asyncio.run issue using older Python

My 2 pence

Some time ago I started on the self employed part of MTD, then gave up. It's at https://github.com/EvansMike/hmrc_self_submit if any of that code is useful, if it's still compatible with the API , then please use it for integration into GnuCash. I don't pay VAT so I never got into the VAT part.

Catch errors which indicate a broken VAT return

e..g spot negative values in returns, or numbers that look massively out by comparing sales with VAT.

Windows / needs better packaging for pip install

pip3 install of gnucash-uk-vat doesn't leave a working gnucash-uk-vat command

Error trying to run gui

I'm getting the following error when running with the --assist resulting in the next button being disabled after selecting a gnucash file.
Does anyone know how to resolve this?

$ gnucash-uk-vat --assist
Traceback (most recent call last):
File "/.local/lib/python3.10/site-packages/gnucash_uk_vat/assist.py", line 708, in prepare
self.check_file()
File "/.local/lib/python3.10/site-packages/gnucash_uk_vat/assist.py", line 614, in check_file
raise e
File "~/.local/lib/python3.10/site-packages/gnucash_uk_vat/assist.py", line 609, in check_file
accts = accounts.Accounts(self.vat.config)
AttributeError: module 'gnucash_uk_vat.accounts' has no attribute 'Accounts'

Current state

Thanks for this, looks like just what I need now I'm being forced onto MTD.

Is the software still working with the latest changes from HMRC? Is it being actively kept up to date?

Cannot authenticate

I just found some time to look at this again so I have resumed attempting to get this working in the sandbox so I can request a production id, but I am unable to proceed past the authenticate step.

When I go to the provided url (I have omitted the client id):

https://test-www.tax.service.gov.uk/oauth/authorize?response_type=code&client_id=********&scope=read%3Avat%2Bwrite%3Avat&redirect_uri=http%3A%2F%2Flocalhost%3A9876%2Fauth

I get the response:

{"error":"invalid_request","error_description":"redirect_uri is invalid"}

Do you have any idea how I can proceed?

Loopback IP instead of local IP used on Linux

Found an unexpected bug, the "Gov-Client-Local-IPs" header is populated with the loopback address ("127.x") on Linux, and not the local IP address.
This is because Python's socket.getbyhostname() returns the value in /etc/hosts, which usually has the loopback address.

On Win and Mac OS the behaviour seems to be correct, but I haven't tested for myself.

After a bit of searching, I found a solution like this (which doesn't add extra dep's):

def get_ip_address():
    with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
    s.connect(("192.168.1.1", 80)) 
    return s.getsockname()[0]

Unfortunately I don't know how to get the gateway IP, so the code above uses my default (the example used 8.8.8.8, but I don't want to open connections externally if it's not needed).

gnucash-uk-vat on Windows / MacOS, --init-config not working

Due to the use of dmidecode in device.py, --init-config only works on Linux. Once the config.json file has been setup (which involves taking a valid config file and modifying device information manually), most other functionality seems to work: fetching obligations.

device.py can be modified to use Windows / MacOS native equivalents to dmidecode.