phuedx/pinc

A network connection conditioner for a handful of low-end devices in my home office

Pi Network Conditioner

What?

Pi Network Conditioner (PiNC) is an application I made to help me quickly throttle the network connections of a number of mobile devices that're all connected to a Raspberry Pi wireless access point. The application runs on the Raspberry Pi, creating and deleting traffic control (TC) queuing disciplines and filters to manage round-trip time (RTT) and bandwidth for each device.

Why?

I work with the Reading Web née Mobile Web team at the Wikimedia Foundation (WMF). The Reading deparment recently had an internal discussion about around how we might better understand users with low-end devices connected to low-bandwidth, unstable mobile networks, somewhat inspired by Cade Metz's "Facebook Workers Ditch iPhones in Push for World Conquest". During the discussion @dr0ptp4kt mentioned that at the WMF offices in San Francisco there's a wireless network that throttles the network connections of all connected devices to a 2G connection, or thereabouts. I, however, nearly always work in London.

How?

PiNC works by assigning traffic from the device to one of a set of pre-defined "network throttling profiles". Each profile is a combination of two TC queuing disciplines (qdiscs): a Token Bucket Filter qdisc that limits the rate at which packets are sent; and a netem qdisc that delays each packet before it is sent.

When a device has its network throttling profile changed, a TC filter is created that matches packets from the device's IPv4 address and assigns them to a "flow" that corresponds to the profiles's first qdisc.

Getting Started

First and foremost you'll need a Raspberry Pi wireless access point. Download a Raspbian Jessie Lite image and write it to an SD card; then follow Lady Ada's "Setting up a Raspberry Pi as a WiFi access point" tutorial, which not only tells you how but how to debug common issues as well.

Both the PiNC service and UI are written in JavaScript. You'll need to download and install Node.js. Fortunately, the latest stable release of Node.js (LTS) is available to download as pre-built ARMv6, ARMv7, or ARMv8 binaries. If you're using a Raspberry Pi 1 Model A, Model A+, Model B, or Raspberry Pi Zero, then you should grab the ARMv6 binaries; whereas if, like me, you're using a Raspberry Pi 2 Model B, then you should download the ARMv7 binaries.

Now that you've got your Raspberry Pi all set up, you can install, setup, and run PiNC on it with the following commands:

git clone https://github.com/phuedx/pinc.git
cd pinc
./script/bootstrap
./script/setup
./script/server

Resources

I've referred to the following while piecing together PiNC:

• tc(8)
• Linux Advanced Routing & Traffic Control HOWTO
• netem
• tc Packet Filtering and netem, which was indispensible when I was fixing #4: Existing filters aren't deleted

Feedback

Your thoughts and comments are, of course, always welcome. In descending order of responsiveness, you can:

• PM me on IRC,
• create an issue on GitHub, or
• tweet at me on Twitter

Shout-outs

I wouldn't have built PiNC without @joakin's support and couldn't have without his initial review.

License

PiNC is MIT-licensed.

pinc's Issues

Conditioning not taking effect

I have a Raspberry Pi 3 (built-in wifi), and I've gotten as far as configuring it as an access point, running the PiNC service, and looking at the web interface.

However, when I actually enable throttling for a connected device, it doesn't seem to have any effect. The connection quality from the target device appears the same. In the PiNC web interface, the throttling icon is shown next to the device, but when I refresh the page, the icon goes away and the device is no longer shown as throttled.

Remove exec abstraction

Currently, src/server/exec.js is relied upon to abstract away whether or not to shell out the tc commands from the profiles service. When in dev mode, require('/path/to/exec').exec( '...' ) is a NOP.

If we were to separate out the caching part of the profiles service from the tc part, define an interface and its null implementation, then we can safely remove the abstraction by composing these new services:

const profilesService = require('...')
const cachingProfilesService = require('...')
const nullProfilesService = require('...')

const profileService = cachingProfilesService( IS_PROD ? profilesService : nullProfilesService )

Determine device profile state from tc output

Currently, the profile service keeps track of state via an internal store. The store is mistakenly referred to as a cache and while there's definitely a write-through mechanism in place, there's no corresponding read from tc when the cache goes stale. Indeed, the "cache" is never considered stale!

It should be fairly easy to parse out the profiles assigned to devices from the output of tc filter show dev <interface>, given that the flow IDs of those profiles are known ahead of time and are mapped easily (see getFlowID in src/server/profiles.js).

Configure download and upload bandwidth separately

As I commented on #5:

Google Chrome/Chromium's network throttling profiles now have download and upload bandwidth configured separately, thus µDL should configure download and upload bandwidth separately too.

Document limitations of emulation

Provided that the profiles are based on documentation – from carriers, hopefully – or research, then, like Google Chrome's Network Conditioner, this can be a useful emulation. However, it remains an emulation.

Add a /profiles endpoint

I think a /profiles endpoint that returned the profiles with the network info would be very useful for the UI too. Example response:

  [
    {
      name: "2G",
      bandwidth: "50",
      rtt: "500"
    },
    // ...
  ]

Research and document other network throttling profiles

• Latency and Jitter in Mobile Networks from Ilya Grigork's High Performance Browser Networking.
• WebPageTest's connectivity.ini.sample

Existing filters aren't deleted

If a profile has previously been set, then the existing filter (created in profileService#setProfile) needs to be removed.

Rename project to Pi Network Conditioner

Ronseal.

Naming things is difficult. When this project was first started, I thought "Hey! I have a small device lab in my office! That's a micro device lab!" Alas, this project isn't that. It's a Raspberry Pi Network Conditioner… or PiNC!

Share flow ids between the node process and the setup script

From irc review:

I think it is worth sharing the profile ids between the node process and the bash script somehow https://github.com/phuedx/micro-device-lab/blob/dev/src/profiles.js#L8-L13 so that you don't have to implicitly keep them in sync the ids with the setup script ones.

I think the easiest way would be a script that exports the profiles and ids as environment variables, such script would get called at the beginning of script/setup and in package.json when npm starting:

  "scripts": {
    "start": "./scripts/env && node index.js"
  }

 cd "$(dirname "$0")/.."

+script/env
 script/bootstrap

Not exactly sure about that script/env would set. I know there's bash dictionaries but I'm not sure how those would be read in node with process.env.

Is there an easy way to load a JSON file into a bash script?

Any opinions?

Add descriptions to profiles

Ronseal.

… and expose them in the user interface.