jonnoftw/rpi-can-logger

Project to log CAN bus data from a PiCAN2 and a GPS module

Raspberry PI CAN Bus Logger

This project provides code for logging CAN Bus data with a Raspberry Pi. It additionally also logs GPS data. All of this data is stored on an SD card and can then be easily uploaded to a server for easy viewing.

Features

• Logs and interprets CAN bus data from:
  • OBD2
  • Tesla vehicles
  • Bus and Truck with FMS
  • Outlander PHEV
• Logs GPS
• Can operate in querying and sniffing mode
• Stores data on SD card. Can be configured to automatically upload via web API when connected to WiFi or 4G internet.
• Can be powered entirely from power provided by the OBD port in your vehicle! You can also wire it into your fuse box or cigarette lighter to prevent it being powered permanently and draining your battery.
• Accompanying Bluetooth App to:
  • Visualise your data in realtime
  • Fetch and upload stored data
• Web based data visualiser

Parts

The following parts are used:

• Raspberry Pi 3 Model B or Raspberry Pi Zero W
• PiCAN CAN-Bus board or equivalent PiCAN product with 1 or 2 CAN buses. Any CAN receiver compatible with python-can should work though.
• GPS Receiver
• DC-DC Converter

If you want WiFi to work with the PiCAN2 shield attached, you'll need to unsolder the GPIO pins and drop them to the bottom and reattach the shield.

If you are using the DC-DC converter, you will need to plug its outputs into the GPIO pins of the Raspberry Pi at pins 2 (5v power) and 6 (ground):

To make your own OBD connector, you will need:

• OBD Male connector
• Crimps

You'll only need to connect pins:

• 4 Chassis Ground
• 6 CAN High
• 14 CAN Low
• 16 12V Power

Tesla

Please follow steps 1 and 2 from this instructable to access the Tesla CAN bus http://www.instructables.com/id/Exploring-the-Tesla-Model-S-CAN-Bus/:

You will need the following parts:

• Tesla port
• Crimps

Full Setup

1. Download the latest raspbian lite image from here: https://raspberrypi.org/downloads/raspbian
2. Insert your SD card into your computer
3. Use your preferred method to put the rasbpian image onto your machine. On linux:

wget https://downloads.raspberrypi.org/raspbian_lite_latest
tar -xvf  raspbian_lite_latest
# the if argument might be different
dd if=2017-09-07-raspbian_stretch-lite.img of=/dev/sdb bs=4M conv=fsync status=progress

4. Unmount your SD card, and plug it into your raspberry pi
5. Run the following commands after logging in (default username is pi, password is raspberry) and configuring wifi by putting your settings in /etc/wpa_supplicant/wpa_supplicant.conf (you will need to restart the wifi to have the settings take effect by running sudo service networking restart):

sudo apt update
sudo apt install git make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev xz-utils bluez python-bluez pi-bluetooth python3-yaml python-yaml
curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash
env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.6.2
git clone https://github.com/JonnoFTW/rpi-can-logger.git

Install Rpi-Logger

1. Determine the configuration file you want to use or roll your own.
2. To install the dependencies and system services, run:

pip3 install -r requirements.txt
sudo python3 setup.py config_file.yaml

3. Enable UART on your RPI (for the GPS) and CAN (skip the second dtoverlay line if your CAN shield only has 1 input) for the CAN shield by adding these lines to /boot/config.txt:

enable_uart=1
dtparam=spi=on
dtoverlay=mcp2515-can0,oscillator=16000000,interrupt=25
dtoverlay=mcp2515-can1,oscillator=16000000,interrupt=24
dtoverlay=spi-bcm2835

4. In order to stop the RPI from asking your serial ports /ttyS0 to log on, change /boot/cmdline.txt and remove:

console=serial0,baudrate=115200

5. Add these lines to your /etc/network/interfaces file (set it to 250000 if you are using FMS and skip can1 if you only have 1 CAN port):

auto can0
iface can0 inet manual
    pre-up /sbin/ip link set can0 type can bitrate 500000 triple-sampling on restart-ms 100
    up /sbin/ifconfig can0 up
    down /sbin/ifconfig can0 down

auto can1
iface can1 inet manual
    pre-up /sbin/ip link set can1 type can bitrate 500000 triple-sampling on restart-ms 100
    up /sbin/ifconfig can1 up
    down /sbin/ifconfig can1 down

6. The logging and file upload service will now run on startup. By default it will use: example_fms_logging.yaml.
7. To setup uploading of files, you will need to create a mongo_conf.yaml file in the project directory:

log_dir: ~/log/can-log/
keys:
 - vid_key_1
 - vid_key_2
api_url: http://url.to/api/ # the api on the end is important

Configuration

RPI-CAN-Logger is highly configurable and supports nearly all standard OBD-2 PIDs and the currently understood frames from Tesla as described in this document.

Configuring CAN Logging

We currently support 4 forms of logging:

• Sniffing OBD
• Querying OBD
• Sniffing Tesla
• Sniffing FMS

Here we will examine the various configuration options:

interface: socketcan # can bus driver
channel: can1 # which can bus to use
log-messages: /home/pi/log/can-log/messages/ # location of debug messages
log-folder: /home/pi/log/can-log/ # location of log files
log-size: 32 # maximum size in MB before log file is rotated 
log-pids: # the pids we want to log, refer to rpi_can_logger/logger/obd_pids.py,  tesla_pids.py fms_pids.py outlander_pid
  - PID_ENGINE_LOAD
  - PID_THROTTLE
  - PID_INTAKE_MAP
  - PID_RPM
  - PID_SPEED
  - PID_MAF_FLOW
log-trigger: PID_MAF_FLOW # when this PID is seen, return the buffer in current state (only works in sniffing mode)
disable-gps: false # enable/disable GPS logging
gps-port: /dev/ttyS0 # serial port for GPS device
tesla: false # indicates whether or not we are logging a tesla vehicle
sniffing: false # indicates that we are sniffing
log-bluetooth: true # whether or not we log to bluetooth
bluetooth-pass: super_secret_password # the password required to stream the bluetooth data
log-level: warning # log level (warning or debug)
fms: false # are we logging FMS? (Bus and Truck only)
verbose: true # give verbose message output on stdout

Configuring Data Upload

In the root directory of this project create a file called: mongo_conf.yaml, it should look like this:

log_dir: /home/pi/log/can-log/
pid-file: ~/log/can-log.pid
api_url: 'https://url.to.server.com/api/'
keys:
    - vid_key_1
    - vid_key_2

The keys are the API keys for each vehicle that this logger will log for.

Cloning SD Cards

Because we're deploying to a lot of these devices, you'll need to make an image after setting everything up on your SD card. Once you're done, plug your SD card into another computer and run:

dd of=logger.img if=/dev/sdb bs=4M conv=fsync status=progress

Once that's finished you'll have a file called logger.img on your machine, insert a new card and run:

dd if=logger.img of=/dev/sdb bs=4M conv=fsync status=progress

This should clone the SD card assuming they're exactly the same. If the cards are different sizes:

Larger Card

• Run raspi-config and resize the partition OR
• Remount the SD card and use your favourite partitioning tool to expand the 2nd partition

Smaller Card

Assuming the amount of data used on the image is less than the target SD card size, then you will need to shrink the data partition before you make the clone SD card. You can do this on Linux with the following (from this tutorial):

sudo modprobe loop
sudo losetup -f
sudo losetup /dev/loop0 logger.img
sudo partprobe /dev/loop0
gksu gparted /dev/loop0

Use gparted to resize the 2nd partition so that it fits within the size of your target SD card. Then hit apply. Now we will truncate the image.

sudo losetup -d /dev/loop0
fdisk -l logger.img

You should get something like:

$ fdisk -l logger.img 

Disk fmslogger.img: 15.9 GB, 15931539456 bytes
255 heads, 63 sectors/track, 1936 cylinders, total 31116288 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x96bbdd32

        Device Boot      Start         End      Blocks   Id  System
logger.img1            8192       93813       42811    c  W95 FAT32 (LBA)
logger.img2           94208    31116287    15511040   83  Linux

Record the sector size (on the 2nd line "Units = ...", 512 bytes here) and end for the 2nd partition (31116287 here), now you can run:

truncate --size=$[(31116287+1)*512] logger.img

You can now write the shrunken image.

Configuring Clones

After you've done all that set a new hostname (with no hyphens after rpi-logger-) for your device by running:

sudo python3 ./systemd/pariable.py rpi-logger-12345
sudo reboot

Where 12345 is the vehicle identifier. In order to connect via the bluetooth app, the device hostname must start with rpi-logger-

You'll also probably need to pair the bluetooth with your phone, run:

sudo bluetoothctl
discoverable on
pairable on

Initiate pairing on your phone. Then run:

discoverable off
pairable off
quit

Reboot your device and everything should be good to go.

Testing

There's a bunch of different tests provided the tests folder:

• can_spam.py will transmit CAN frames with the last two of eight bytes as increasing integers from 0 to 0xff
• gps_test.py will dump output of the GPS
• gps_sniff_test.py will dump raw CAN messages to display
• gpio_led_test.py will test the LEDs
• can_dump.py will dump the CAN data to a CSV file
• query_single_pid.py will query every OBD PID and check for a response
• phev_query will query data from the battery control unit on a Mistubishi PHEV Outlander

rpi-can-logger's Issues

Compatible with ELM327 II Bluetooth Can-Bus dongle

Hi,

I'm using scanmytesla app with ELM327 II Bluetooth dongle to monitor battery temperature.
Is your project compatible with this bluetooth donge, so I can log can-messages to the sd-card?

Problem during logging

Hi,

First of all, thank you for your work.
I would like to ask for a little help. I've installed the repository on a Raspberry Pi Zero 2 W, but I'm stuck. I tried testing the logging feature, but it was unsuccessful, or I may have missed something. I sent messages via a PCAN USB adapter from PCAN-View to an MCP2515 CAN bus adapter (

which I modified for 3.3V supply and 16 MHz)

Here is the insalled .yaml configuration file:

channel: can0
disable-gps: true
fms: true
interface: socketcan
log-bluetooth: false
log-folder: ~/log/can-log/
log-level: warning
log-messages: ~/log/can-log/messages/
log-pids: [FMS_ENGINE_HOURS, FMS_FUEL_CONSUMPTION, FMS_ENGINE_TEMP, FMS_FUEL_ECONOMY]
log-size: 32
log-trigger: FMS_ENGINE_HOURS
pid-file: ~/log/can-log.pid
sniffing: false
tesla: false
vehicle-id: scania
verbose: true

Changing "sniffing: false" to "true" did not solve the problem.

I checked the status of the rpi-logger.service:

• Loaded: loaded (/lib/systemd/system/rpi-logger.service; enabled; vendor present: enabled)

• Active: active (running)

After checking the .json file I see this:
{"vid": "scania", "trip_id": "20240630_150437_P1aGqk_scania", "trip-sequence": 83}

If I run the can_sniff_test.py I see the messages.

What could be wrong?
Thanks!

Adding new J1939 PID:s to FMS snif

If I add new PID:s to fms_pids.py, can i use them in logging?
I have some additional standard J 1939 pid's (PGN's) needed for logging.
Additionally I am new to rpi-can-logger. How to change configuration in yaml files?
Especially meaning of log-size and log-trigger?

Invalid CAN Bus Type - socketcan_native

Hi,

Sorry i'm french and my English it'snt good.
I test you project with a rapsberry 4 and pican2.
I have an error of communication.
can you help me? How can I contact you outside of Gitub

Thank you very much.

Best regards

Clarification of README comment about WiFi performance issues with mounted PiCAN

Hello! This question is about some information in the README, and isn't really about any issue with the code :). Anyway, concerning this quote from the README..

If you want WiFi to work with the PiCAN2 shield attached, you'll need to unsolder the GPIO pins and drop them to the bottom and reattach the shield

Could you elaborate a bit on the WiFi performance issues with PiCAN2? I take it the PiCAN2 blocked the WiFi antenna such that WiFi was unusable. And it sounds like your solution was to move the GPIO pins to the other side of the Pi, effectively mounting PiCAN2 on the opposite side of the Pi, leaving the WiFi antenna more exposed.

Please confirm if my interpretation of the quote is correct, cheers!

Running: pip3 install -r requirements.txt gives following error

pi@raspberrypi:~ $ pip3 install -r ~/rpi-can-logger/requirements.txt
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Obtaining Package from git+https://github.com/hardbyte/python-can.git#egg=Package (from -r /home/pi/rpi-can-logger/requirements.txt (line 1))
Cloning https://github.com/hardbyte/python-can.git to ./src/package
Running command git clone -q https://github.com/hardbyte/python-can.git /home/pi/src/package
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
WARNING: Generating metadata for package Package produced metadata for project name python-can. Fix your #egg=Package fragments.
WARNING: Discarding git+https://github.com/hardbyte/python-can.git#egg=Package. Requested python-can from git+https://github.com/hardbyte/python-can.git#egg=Package (from -r /home/pi/rpi-can-logger/requirements.txt (line 1)) has inconsistent name: filename has 'package', but metadata has 'python-can'
ERROR: Could not find a version that satisfies the requirement package (unavailable)
ERROR: No matching distribution found for package (unavailable)