tohojo/sqm-scripts

SQM scripts traffic shaper

The sqm-scripts traffic shaper

This repository contains the sqm-scripts traffic shaper from the CeroWrt project. See: http://www.bufferbloat.net/projects/cerowrt/wiki/Smart_Queue_Management

How does sqm-scripts set up traffic shaping?

sqm-scripts uses the Linux qdisc mechanism to configure traffic shaping and scheduling. Either by a combination of the HTB qdisc (for shaping) and fq_codel (for packet scheduling), or using the CAKE integrated scheduler to do both at once.

Because qdiscs only see traffic as it is leaving an interface, for ingress shaping sqm-scripts sets up an IFB device. Incoming packets are redirected to this device where a regular qdisc can handle them. The IFB device is named for the egress interface it is paired with, so the IFB for 'eth0' will be called 'ifb4eth0'.

This is all illustrated on the following diagram:

Diagram contributed by Matt Taggart (@taggart). Source in qdisc-diagram.dia.

Requirements

To run sqm-scripts you just need a Linux machine with a kernel from the last decade or so (any of the longterm releases on kernel.org should work - please don't run a kernel that is not a currently released version, longterm or current). The only exception is that if you want to use the cake qdisc, you either need kernel 4.19 or newer (with matching iproute2 version), or you need to build CAKE yourself from the out-of-tree repository at https://github.com/dtaht/sch_cake.

Installing

sudo make install should install things on a regular Linux box. For OpenWrt, there are packages available in the distribution, so just install the sqm-scripts package, and optionally luci-app-sqm for GUI support.

Running on regular Linux distributions

After installing using make install, do the following to enable sqm-scripts:

1. Copy /etc/sqm/default.conf to /etc/sqm/<dev>.iface.conf where <dev> is the name of the interface you wish to run sqm-scripts on. Then adjust the values in the file to your environment, setting at least UPLINK, DOWNLINK, and possibly SCRIPT.

2. If you're on a Debian-derived distribution that uses old-style network config in /etc/network/, the Makefile should detect this and drop in appropriate hotplug scripts, so sqm-script should automatically run on the next 'ifup'.

3. If you're on a systemd-enabled distro, just enable the sqm@<dev> service corresponding to your interface name. E.g., for eth0, issue systemctl enable sqm@eth0 (and run systemctl start sqm@eth0 to start sqm-scripts immediately).

"Installing" the current development version from git on OpenWrt

Run the steps below on your own computer (not on the router) to retrieve the newest script version from this repository, create the scripts, then copy those new scripts to your router.

1. Make a local clone of the git repository (if you have not already):

   git clone https://github.com/tohojo/sqm-scripts

2. Change into the new directory:

   cd ./sqm-scripts

3. Make sure the source is updated:

   git pull

4. Create the scripts for your platform (PLATFORM is either linux or openwrt) and output them to a local current_sqm_base directory:

   make install PLATFORM=openwrt DESTDIR=./current_sqm_base

5. Change to ./current_sqm_base:

   cd ./current_sqm_base

6. Optional for OpenWrt: The final step will overwrite your router's current sqm configuration file (at /etc/config/sqm). If you want to preserve the current configuration, delete the newly created config file from the local etc/config:

   rm -r etc/config

7. Now, use scp to copy the new scripts to the router. Change $YOUR.SQM.HOSTNAME to the address/DNS name for your computer - probably 192.168.1.1 or on cerowrt gw.hom.lan. If your account on the router is not "root", change "root" to your account:

   scp -r ./* root@$YOUR.SQM.HOSTNAME:/

Note this method relies on the presence of the required qdiscs on the router/destination host. On openwrt, you should first install the "normal" sqm-scripts package to take care of all the dependencies, then use this procedure to update to the newest sqm-scripts.

Run-time debugging

SQM_VERBOSITY_MAX controls the verbosity of sqm's output to the shell and syslog (0: no logging; 8: full debug output). SQM_DEBUG controls whether sqm will log the output of the last invocation of start-sqm into var/run/sqm/${interface_name}.start-sqm.log and the ouput of the last invocation of stop-sqm into var/run/sqm/${interface_name}.stop-sqm.log e.g. for pppoe-wan /var/run/sqm/pppoe-wan.start-sqm.log and /var/run/sqm/pppoe-wan.stop-sqm.log.

Examples

• Log only the binary invocations and their output:

  /etc/init.d/sqm stop ; SQM_DEBUG=1 SQM_VERBOSITY_MAX=0 /etc/init.d/sqm start

• Log verbose debug output and all the binary invocations and their output:

  /etc/init.d/sqm stop ; SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm start

• Log both start and stop:

  SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm stop ; SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm start

Note: Both the start and stop log are re-written on every sqm instance start and stop and are logging all output independent of the value of SQM_VERBOSITY_MAX. They will not grow indefintely, but they are written repeatedly. On reliably rewritable media like hard disk, ssd, flash with wear-leveling, or ram-disk, SQM_DEBUG can be safely set to 1 in defaults.sh, but on media like NOR flash that do only allow few write-cycles, keeping the default at 0 and using the above invocations to run a single instance with SQM_DEBUG=1 is recommended.

sqm-scripts's Issues

Time for a new release?

Since the last release in February 2017 we accumulated a few fixes and changes, that should be ready for a wider user group. So how about making a decision on what to do with the bufs that @guidosarducci identified and created fixes for in the current code base and organize a new release?

Evaluation of the available qdiscs: explanation in GUI is wrong

I tested yesterday building SQM with cake for Openwrt trunk, and while doing that I ran into two documentation / clarity issues. This is the first one of them.

I installed the cake kmod to the router and restarted sqm, but cake qdisc was still not available in SQM. It took me a while to figure out why, although SQM Luci GUI claims that a new browser session is enough ;-)
https://github.com/tohojo/sqm-scripts/blob/master/luci/sqm-cbi.lua#L98

Queuing disciplines useable on this system; 
instantiated only after first successful start of SQM. 
You need to start a new GUI session to see updates

In reality, the available qdiscs are only evaluated at the router reboot (or when SQM is installed & first run). A manual restart of SQM does not initiate qdisc evaluation, as a separate status directory /var/run/sqm/available_qdiscs/ is used to store the information:
https://github.com/tohojo/sqm-scripts/blob/master/src/run-openwrt.sh#L17

[ -d "${SQM_QDISC_STATE_DIR}" ] || ${SQM_LIB_DIR}/update-available-qdiscs

User can force the evaluation by deleting the directory /var/run/sqm/available_qdiscs/ , if hhe reads the SQM source code and finds out about that.

I feel that the evaluation might be good to do also when the user initiates a restart of SQM. However, the qdisc sniffing code is part of the same run.sh that is run also for the hotplug actions, so we should avoid .

I propose that

• the wrong explanation about "new GUI session" is corrected. E.g. the last sentence is either deleted or changed to "After installing new qdiscs, please reboot the router to make them available."
• as a more minor point, please consider making a SQM restart to also force a new evaluation of qdiscs. However, I understand that users rarely do install new qdiscs, so this whole item is a really minor one.

sqm-scripts: change to no longer start up at boot, slash a few seconds off of boot-up time

I noticed that sqm-scripts starts up at boot time even though interfaces that have configuration sections for it have not yet been fully initialized or made available. (Yes, the script has checks in place, but this only adds to the boot time).

Even more so, these interfaces that sqm-scripts tries to setup will still need to be re-initialized by it down the line, as more hotplug events are initiated during the router initialization phase.

This spams syslog quite a bit, because at the sequence sqm is started, for every sqm line that is logged, an immediate copy of the line is also logged because of user.emerg log levels. This is magnified as more interfaces are defined in the sqm config file (I'm using a dual-wan setup, and have sqm for each WAN interface).

This also has the side effect of increasing boot-time because sqm tries to setup these interfaces, only to be told to reconfigure its interfaces as hotplug events are triggered.

So I edited /etc/init.d/sqm to include:

boot()
{
return 0
}

In this way, I find it most efficient to NOT have sqm setup interfaces at boot, but instead to have it setup interfaces via its hotplug script, which is already properly triggered. I feel that this is the right place to have sqm do its magic. ;-) This also cuts the router startup time by a couple of seconds.

I wonder if we could have this by default, or is there something else that I'm missing to not do this by default?

Question: Full interface and VLAN Tag

Hi T0ke / all,

Let my scenario. A single interface system (A Raspberry Pi) where:

• eth0 -> public IP address;
• eth0.{1..8} -> Other internal VLANs.

Does SQM acts only in regard to eth0 public IP traffic? Or will it SQM the whole traffic, encompassing the eth0 and the tagged traffic?

TL;DR

I used to have SQM (happilly) running in eth0.2 (was then my public interface). I had a 10 Mbps internet and that worked just. Fine.
Well, I got a upgrade to a 120 Mbps internet. And I got a max effective throughput of 60 Mbps.
After investigating the bottleneck, I found a interface-bound ktrhead eating almost 100% CPU.

In an effort to mitigate this impact, I found that if I put the public traffic in eth0 untagged, I can get ~75-80 Mbps effective traffic.

And I like SQM. Really. Will SQM work with the eth0 IP traffic only, or will it act over the whole eth0 traffic?

Thanks! :-)

New cmd_wrapper code considered slightly too verbose

So we introduced the nice new command wrapper to get more verbose error reports for failing calls to ip and tc. But this new verbosity also triggers for those calls that we tried to silence by redirecting the output to /dev/null (2> /dev/null)

Example the call in layer_cake.qos
$TC qdisc del dev $IFACE root 2> /dev/null

actually ends up in tc_wrapper as

Mon Jan 7 14:50:38 2019 user.notice SQM: ERROR: tc_wrapper: qdisc del dev pppoe-wan root

So the redirection has seemingly been chopped off.

By calling
$TC qdisc del dev $IFACE root "2> /dev/null"

we can push the chopping off action from layer_cake.qos into the tc_wrapper function in functions.sh, but this still causes our error detection to trigger in cmd_wrapper.

Not sure how to fix this cleanly. My hunch is that we might need to create a function that properly tests the existence of qdisc" components) before trying to delete them.
Or we use an additional variable to signal to cmd_wrapper the desired log level of the error output...

Sun Jan  6 17:39:45 2019 user.notice SQM: Stopping SQM on pppoe-wan
Sun Jan  6 17:39:45 2019 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc qdisc del dev pppoe-wan ingress
Sun Jan  6 17:39:45 2019 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
Sun Jan  6 17:39:45 2019 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc qdisc del dev pppoe-wan root
Sun Jan  6 17:39:45 2019 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
Sun Jan  6 17:39:46 2019 user.notice SQM: Starting SQM script: layer_cake.qos on pppoe-wan, in: 46246 Kbps, out: 9890 Kbps
Sun Jan  6 17:39:47 2019 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc qdisc del dev pppoe-wan root
Sun Jan  6 17:39:47 2019 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
Sun Jan  6 17:39:47 2019 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc qdisc del dev pppoe-wan handle ffff: ingress
Sun Jan  6 17:39:47 2019 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
Sun Jan  6 17:39:47 2019 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc qdisc del dev ifb4pppoe-wan root
Sun Jan  6 17:39:47 2019 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
Sun Jan  6 17:39:47 2019 user.notice SQM: layer_cake.qos was started on pppoe-wan successfully
...

As far as I can see once we fix this we might be able to cut a new release for openwrt....

ipq806x / Netgear R7800: difference in download speeds with cake and simple/fq_codel (possibly due to HTB )

I have a new router Netgear R7800 that is an 1,7 GHz dual-core IPQ8065 device. I have been building and testing new LEDE builds on it and I have stumbled upon strange behaviour with SQM using simple/fq_codel:

• There is a significant speed gap between the set download speed limit and the realised download speed, as measured by flent.
• With cake the same speed limit produces significantly higher download speed.
• My old WNDR3800 (ar71xx) produces the same speed with cake and simple as R7800 with only cake.
• Increasing the download speed limit by 10 Mb increases the actual download speed by maybe 7 Mbit/s, so the router is not hitting any real CPU or I/O performance limit.

Data with wired LAN connection:

• Without any SQM, my connection with R7800 produces 100 Mbit/s down, 12 Mb/s up, 35 ms latency
• With speed limit down 90, up 8, the results are:
  • strategy / down / up / latency
  • R7800 simple: 67 / 7 /20
  • R7800 cake: 84 / 7 / 19
  • WNDR3800 simple: 83 / 5 / 18
  • WNDR3800 cake: 83 / 5 / 19
• With speed limit down 100, up 8:
  • R7800 simple: 73 / 7 / 20 , So a 6-7 Mbit/s speed increase without negative impact on upload speed or latency.
• With R7800 there was no significant difference between wired / wifi
• I tried bumping kernel HZ from default 100 to 250, but that had no effect.

It is almost like the simple/fq_codel would produce in ipq806x a realised download speed of some 75% of the set limit. With the old trustry ar71xx/WNDR3800 the speed gets higher upto the hardware limits.

This almost looks like there is some kind of calculation bug in the fq_codel code when compiled for IPQ806x (code base is arm_cortex-a15_neon-vfpv4 )

Any clues to what could cause this? How to debug?

• Can router's automatic CPU frequency scaling (between 384-1700 MHz according to load) cause trouble for codel? But not for cake?

Using insmod vs modprobe

What is the reason for using insmod currently? On openwrt modprobe and
insmod seem to be equivalent, and on other systems insmod seems to
expect a filename not a module name. So wouldn't it be more correct to
use modprobe?

sqm with cake , unexpected operator

sudo sqm reload eth0
Stopping SQM on eth0
Starting SQM script: layer_cake.qos on eth0, in: 11603 Kbps, out: 810 Kbps
/usr/bin/sqm: 291: [: TRUE: unexpected operator
/usr/bin/sqm: 291: [: TRUE: unexpected operator
layer_cake.qos was started on eth0 successfully

Reduce too much bandwidth

I am using 100Mbps LAN.
If I start sqm, I can achieve up to 6MB/s download speed.

sqm pollutes /etc/config/ucitrack with multiple entries

SQM adds a new line to the ucitrack file at each new firmware flash (when the uci-defaults are run). The lines accumulate steadily to the "firewall" section if the user does not clear settings at some point. I re-compile and flash my community firmware almost daily, so this sqm behaviour shows clearly in ucitrack. See below.

The reason is this line in sqm uci-defaults:
https://github.com/tohojo/sqm-scripts/blob/master/luci/uci-defaults-sqm#L7

add_list ucitrack.@firewall[0].affects=sqm

There should probably either

• be a check for the existence of a previous entry before adding a new line, or
• instead of uci-defaults, the action would be done once in a postinst script (after the package install).

Evidence from my router:

root@OpenWrt:~# cat /etc/config/ucitrack

config network
        option init 'network'
        list affects 'dhcp'
        list affects 'radvd'

config wireless
        list affects 'network'

config firewall
        option init 'firewall'
        list affects 'luci-splash'
        list affects 'qos'
        list affects 'miniupnpd'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'
        list affects 'sqm'

config olsr
        option init 'olsrd'

config dhcp
        option init 'dnsmasq'
        list affects 'odhcpd'
...

Too large maxpacket , some errror line ,, etc

11月 10 19:27:18 ippan-pc sudo[10442]: ippan : TTY=pts/0 ; PWD=/home/ippan ; USER=root ; COMMAND=/usr/bin/sqm reload eno1
11月 10 19:27:18 ippan-pc sudo[10442]: pam_unix(sudo:session): session opened for user root by (uid=0)
11月 10 19:27:19 ippan-pc NetworkManager[376]: [1541845639.1836] manager: (TMP_IFB_4_SQM): new Generic device (/org/f>
11月 10 19:27:19 ippan-pc systemd-udevd[10570]: link_config: autonegotiation is unset or enabled, the speed and duplex are no>
11月 10 19:27:19 ippan-pc systemd-udevd[10570]: Could not generate persistent MAC address for TMP_IFB_4_SQM: No such file or >
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.202+0000: 1912: error : virFileReadAll:1434 : ファイル '/sys/cla>
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.202+0000: 1912: error : virNetDevGetLinkInfo:2485 : unable to re>
11月 10 19:27:19 ippan-pc NetworkManager[376]: [1541845639.2172] manager: (TMP_IFB_4_SQM): new Generic device (/org/f>
11月 10 19:27:19 ippan-pc systemd-udevd[10585]: link_config: autonegotiation is unset or enabled, the speed and duplex are no>
11月 10 19:27:19 ippan-pc systemd-udevd[10585]: Could not generate persistent MAC address for TMP_IFB_4_SQM: No such file or >
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.236+0000: 1912: error : virFileReadAll:1434 : ファイル '/sys/cla>
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.236+0000: 1912: error : virNetDevGetLinkInfo:2485 : unable to re>
11月 10 19:27:19 ippan-pc systemd-udevd[10607]: link_config: autonegotiation is unset or enabled, the speed and duplex are no>
11月 10 19:27:19 ippan-pc systemd-udevd[10607]: Could not generate persistent MAC address for ifb4eno1: No such file or direc>
11月 10 19:27:19 ippan-pc NetworkManager[376]: [1541845639.2592] manager: (ifb4eno1): new Generic device (/org/freede>
11月 10 19:27:19 ippan-pc NetworkManager[376]: [1541845639.7298] manager: (TMP_IFB_4_SQM): new Generic device (/org/f>
11月 10 19:27:19 ippan-pc systemd-udevd[10850]: link_config: autonegotiation is unset or enabled, the speed and duplex are no>
11月 10 19:27:19 ippan-pc systemd-udevd[10850]: Could not generate persistent MAC address for TMP_IFB_4_SQM: No such file or >
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.749+0000: 1912: error : virNetDevSendEthtoolIoctl:2995 : ethtool>
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.751+0000: 1912: error : virNetDevSendEthtoolIoctl:2995 : ethtool>
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.753+0000: 1912: error : virNetDevSendEthtoolIoctl:2995 : ethtool>
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.755+0000: 1912: error : virNetDevSendEthtoolIoctl:2995 : ethtool>
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.757+0000: 1912: error : virNetDevSendEthtoolIoctl:2995 : ethtool>
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.759+0000: 1912: error : virNetDevSendEthtoolIoctl:2995 : ethtool>
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.760+0000: 1912: error : virNetDevSendEthtoolIoctl:2995 : ethtool>
11月 10 19:27:19 ippan-pc libvirtd[603]: 2018-11-10 10:27:19.762+0000: 1912: error : virNetDevSendEthtoolIoctl:2995 : ethtool>
11月 10 19:27:19 ippan-pc sudo[10442]: pam_unix(sudo:session): session closed for user root

tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev eno1 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 1000
Sent 216664 bytes 1206 pkt (dropped 43, overlimits 1858 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 120: dev eno1 parent 1:12 limit 1001p flows 1024 quantum 300 target 17.7ms interval 112.7ms memory_limit 32Mb
Sent 215021 bytes 1194 pkt (dropped 43, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 1643 drop_overlimit 0 new_flow_count 366 ecn_mark 0
new_flows_len 1 old_flows_len 4
qdisc fq_codel 130: dev eno1 parent 1:13 limit 1001p flows 1024 quantum 300 target 17.7ms interval 112.7ms memory_limit 32Mb
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 110: dev eno1 parent 1:11 limit 1001p flows 1024 quantum 300 target 17.7ms interval 112.7ms memory_limit 32Mb
Sent 1643 bytes 12 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 159 drop_overlimit 0 new_flow_count 12 ecn_mark 0
new_flows_len 1 old_flows_len 0
qdisc ingress ffff: dev eno1 parent ffff:fff1 ----------------
Sent 1160321 bytes 1533 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev ifb4eno1 root refcnt 2 r2q 10 default 10 direct_packets_stat 0 direct_qlen 32
Sent 1357542 bytes 1533 pkt (dropped 0, overlimits 420 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 110: dev ifb4eno1 parent 1:10 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
Sent 1357542 bytes 1533 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 4876 drop_overlimit 0 new_flow_count 686 ecn_mark 0
new_flows_len 1 old_flows_len 6

tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc htb 1: dev eno1 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 ver 3.17 direct_qlen 1000
linklayer atm mtu 2047 tsize 512
qdisc fq_codel 120: dev eno1 parent 1:12 limit 1001p flows 1024 quantum 300 target 17.7ms interval 112.7ms memory_limit 32Mb
qdisc fq_codel 130: dev eno1 parent 1:13 limit 1001p flows 1024 quantum 300 target 17.7ms interval 112.7ms memory_limit 32Mb
qdisc fq_codel 110: dev eno1 parent 1:11 limit 1001p flows 1024 quantum 300 target 17.7ms interval 112.7ms memory_limit 32Mb
qdisc ingress ffff: dev eno1 parent ffff:fff1 ----------------
qdisc htb 1: dev ifb4eno1 root refcnt 2 r2q 10 default 10 direct_packets_stat 0 ver 3.17 direct_qlen 32
linklayer atm mtu 2047 tsize 512
qdisc fq_codel 110: dev ifb4eno1 parent 1:10 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 32Mb ecn

How to set download/upload limit per IP/MAC of device?

Im using sqm-scripts over Openwrt, the general limits on my guest interface work fine, but how I can set another limit per host and not only general limits?

Something like:

config queue
option enabled '1'
option interface 'br-guest'
option download '3000'
option upload '10000'
....

config devices
option enabled '1'
option mac-addr 'xx:xx:xx:xx:xx'
option download '800'
option upload '1500'
....
config devices
option enabled '1'
option mac-addr 'all'
option download '500'
option upload '1000'
....

Thanks

SQM scripts unused classes

Hello all,

in src/simple.qos class 1:10 is created for ingress and egress but never used. The class also never gets a qdisc, e.g. fq_codel, assigned like the other classes, e.g. 1:20. The only exception is on ingress if ["$SQUASH_INGRESS" = "1"]. Is there a reason to add the 1:10 classes anyway? Or can they be removed?

Regards,
Sebastian

Nuking one of the "simplest" scripts for LEDE 17.01?

Right now simplest.qos and simplest_tbf.qos are basically identical in functionality, except that the latter uses TBF in place of HTB. Is there any reason we need to ship both by default? We could demote one of them to sqm-scripts-extra. I'm thinking simplest_tbf is the one to keep, since TBF seems to be a win CPU-wise. On the other hand, simplest.qos has received significantly more testing.

Thoughts? @moeller0 @EricLuehrsen @dtaht @hnyman

/usr/lib/sqm/run.sh: eval: line 1: sqm_logger: not found

Hi,

Whenever sqm-scripts is run in the foreground, I observe this error:

/usr/lib/sqm/run.sh: eval: line 1: sqm_logger: not found
SQM: /usr/lib/sqm/stop-sqm: Stopping eth0.2
[...]

The file /usr/lib/sqm/run.sh needs a little modification as follows:

--- a/run.sh    2015-08-16 09:47:38.000000000 -0000
+++ b/run.sh    2015-08-16 09:42:25.000000000 -0000
@@ -60,6 +60,7 @@
    #sm: only stop-sqm if there is something running
    CUR_STATE_FILE="${SQM_STATE_DIR}/${IFACE}.state"
    if [ -f "${CUR_STATE_FILE}" ]; then
+       . ${SQM_LIB_DIR}/functions.sh
        sqm_logger "SQM active on ${IFACE}, stopping it"
        "${SQM_LIB_DIR}/stop-sqm"
    fi

After the change, it properly logs the line now:

SQM: SQM active on eth0.2, stopping it
SQM: /usr/lib/sqm/stop-sqm: Stopping eth0.2
[...]

The issue is observed in the package version sqm-scripts - 1.0-3.

PPPOE helpers remain in functions.sh

The pppoe flavored scripts were removed for lack of maintenance, but the helpers still remain in functions.sh. Is it desirable to remove those? Or, are they hanging around for something else?

Time for a 1.0.8 release label?

I don't know how much more testing is desired on any of the recent tweaks. I don't see bugs popping up for the scripts at least (scratch that, I made one). Kernel seems to have a few recent integration issues, but anyway. Just thinking maybe a label is due.

Failing gracefully on errors

Currently we mostly don't check commands for errors. We probably should,
and then fail gracefully if something critical fails...

This includes in particular insmod, (see #3), but also other things like
iptables setup and chains of tc commands...

Options for low bandwidth

Hi everyone;

This weekend I deployed a OpenWRT @ Parent's, where they have a entry-level internet access: 2 Mbps down, 500 kbps up. Well, the default fq_codel didn't play really nicely with such low bandwidth, being HTB the algorithm that performed best (to be 100% honest, I just played with fq_codel and HTB).

Given your experience, what would be the most suitable queue discipline for such low bandwidths? My aim here is add something in the description, so that will help other users in the same situation.

For what is worth: the tuned sqm-scripts 2 Mbps connection, with OpenWRT yielded a better web experience to my parents than the former 10 Mbps DSL + carrier modem doing all the job (modem, NAT, wifi), paying a quarter for the service. That was a WIN.

Ah, last but not least; what would be the minimum recommended BW for fq_codel?

Thanks and have a great week.

'/etc/init.d/sqm stop' results in "SQM script not found!"

Here is the error when I tried to stop sqm:

~# /etc/init.d/sqm stop
SQM: /usr/lib/sqm/stop-sqm: Stopping eth0.2
SQM: ifb associated with interface eth0.2: 
SQM: SQM script  not found!
SQM: /usr/lib/sqm/stop-sqm: Stopping eth1
SQM: ifb associated with interface eth1: 
SQM: SQM script  not found!

I made a couple of modifications in /usr/lib/sqm/run.sh so that /etc/init.d/sqm stop works correctly:

~# /etc/init.d/sqm stop
SQM: /usr/lib/sqm/stop-sqm: Stopping eth0.2
SQM: ifb associated with interface eth0.2: 
SQM: ifb associated with interface eth0.2: 
RTNETLINK answers: File exists
SQM: /usr/lib/sqm/stop-sqm: ifb4eth0.2 shaper deleted
SQM: /usr/lib/sqm/stop-sqm: ifb4eth0.2 interface deleted
SQM: /usr/lib/sqm/stop-sqm: Stopping eth1
SQM: ifb associated with interface eth1: 
SQM: ifb associated with interface eth1: 
RTNETLINK answers: File exists
SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 shaper deleted
SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 interface deleted

I am not sure why

CUR_IFB=$( get_ifb_associated_with_if ${IFACE} )
[ -z "$CUR_IFB" ] && CUR_IFB=$( ifb_name ${IFACE} )

in /usr/lib/sqm/stop-sqm both return empty strings, as you can see the result from above. I haven't taken a deeper look into it, so I thought you might wanna take a look.

Running /etc/init.d/sqm stop a second time results in nothing happening, so everything's fine.

I also found that some lines do not indent with a tab, so I changed that as well:

--- run.sh.orig 2015-08-25 22:58:16.000000000 -0800
+++ run.sh  2015-08-26 00:05:47.000000000 -0800
@@ -18,8 +18,12 @@

 # Stopping all active interfaces
 if [ "$ACTION" = "stop" -a -z "$RUN_IFACE" ]; then
-    for f in ${SQM_STATE_DIR}/*.state; do
-        [ -f "$f" ] && IFACE=$(basename $f .state) ${SQM_LIB_DIR}/stop-sqm
+    for f in ${SQM_STATE_DIR}/*.state ; do
+        [ -f "$f" ] && {
+            export IFACE=$(basename $f .state)
+            export SCRIPT=$(awk -F\" '($1 ~ /^SCRIPT=$/) {print $2}' $f)
+            "${SQM_LIB_DIR}/stop-sqm"
+        }
     done
     exit 0
 fi
@@ -63,7 +67,7 @@
        "${SQM_LIB_DIR}/stop-sqm"
    fi

-        [ "$ACTION" = "start" ] && "${SQM_LIB_DIR}/start-sqm"
+   [ "$ACTION" = "start" ] && "${SQM_LIB_DIR}/start-sqm"

ifb associated with interface IFACE is still empty

In sqm-scripts - 1.0.2-1, the function get_ifb_associated_with_if() in /usr/lib/sqm/functions.sh does not properly match the associated ifb.

Before the fix:

~# /etc/init.d/sqm stop
SQM: /usr/lib/sqm/stop-sqm: Stopping eth0.2
SQM: ifb associated with interface eth0.2: 
SQM: /usr/lib/sqm/stop-sqm: ifb4eth0.2 shaper deleted
SQM: /usr/lib/sqm/stop-sqm: ifb4eth0.2 interface deleted
SQM: /usr/lib/sqm/stop-sqm: Stopping eth1
SQM: ifb associated with interface eth1: 
SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 shaper deleted
SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 interface deleted

The fix is simple:

--- functions.sh.orig   2015-08-29 03:31:10.000000000 -0800
+++ functions.sh    2015-08-29 03:31:27.000000000 -0800
@@ -65,7 +65,7 @@
 get_ifb_associated_with_if() {
     CUR_IF=$1
     # CUR_IFB=$( tc -p filter show parent ffff: dev ${CUR_IF} | grep -o -e ifb'[[:digit:]]\+' )
-    CUR_IFB=$( tc -p filter show parent ffff: dev ${CUR_IF} | grep -o -e ifb'[^)]\+' )    # my editor's syntax coloration is limitied so I need a single quote in this line (between eiditor and s)
+    CUR_IFB=$( tc -p filter show parent ffff: dev ${CUR_IF} | grep -o -e ifb'[^)]*' )    # my editor's syntax coloration is limitied so I need a single quote in this line (between eiditor and s)
     sqm_logger "ifb associated with interface ${CUR_IF}: ${CUR_IFB}"
     echo ${CUR_IFB}
 }

The + attempts to match the preceeding token ONE OR MORE times. The problem here is that the preceeding token evaluates to a zero match (empty) because we are using the negated character class [^)]. While this may be the case, it is "important to remember that a negated character class still must match a character." [1] In other words, the negated character class must still match a character () in this case), but the resulting token evaluates to zero (empty). Sounds a bit confusing, no? ;) The appropriate repetition to use here, given the preceeding token, is the *, which matches ZERO OR MORE times. [2]

After the fix:

~# /etc/init.d/sqm stop
SQM: /usr/lib/sqm/stop-sqm: Stopping eth0.2
SQM: ifb associated with interface eth0.2: ifb4eth0.2
SQM: /usr/lib/sqm/stop-sqm: ifb4eth0.2 shaper deleted
SQM: /usr/lib/sqm/stop-sqm: ifb4eth0.2 interface deleted
SQM: /usr/lib/sqm/stop-sqm: Stopping eth1
SQM: ifb associated with interface eth1: ifb4eth1
SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 shaper deleted
SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 interface deleted

Cheers,
ianp

[1] http://www.regular-expressions.info/charclass.html#negated
[2] http://www.regular-expressions.info/repeat.html

Version numbers?

Currently, the packages use date-based versioning (in the ceropackages
repo). Would it be better to do a 'proper' versioning (which we can then
tag in this repo)? If so, what versioning scheme? 0.xx.yy?

SQM script not found!

systemd 226

I've installed these scripts on a Gentoo Linux with systemd but it doesn't work, systemd doesn't read the /etc/sqm/eth0.iface.conf file:

$ cat /run/sqm/eth0.debug.log
ERROR: SQM script  not found!

systemctl status [email protected]
● [email protected] - SQM scripts for iface eth0
   Loaded: loaded (/usr/lib64/systemd/system/[email protected]; enabled; vendor preset: enabled)
   Active: active (exited) since mié 2015-12-16 01:13:10 CET; 38min ago
 Main PID: 3596 (code=exited, status=0/SUCCESS)

dic 16 01:13:10 ns310428.ip-188-165-201.eu systemd[1]: Starting SQM scripts for iface eth0...
dic 16 01:13:10 ns310428.ip-188-165-201.eu start-sqm[3596]: Starting SQM script: simple.qos on eth0, in: 250000 Kbps, out: 250000 Kbps
dic 16 01:13:10 ns310428.ip-188-165-201.eu systemd[1]: Started SQM scripts for iface eth0.
dic 16 01:27:31 ns310428.ip-188-165-201.eu systemd[1]: Started SQM scripts for iface eth0.
dic 16 01:27:34 ns310428.ip-188-165-201.eu systemd[1]: Started SQM scripts for iface eth0.
dic 16 01:28:51 ns310428.ip-188-165-201.eu systemd[1]: Started SQM scripts for iface eth0.
dic 16 01:34:54 ns310428.ip-188-165-201.eu systemd[1]: Started SQM scripts for iface eth0.

Making sqm-scripts compatible with other packages (e.g. mwan3)

Hi.

I saw post: https://lists.bufferbloat.net/pipermail/cerowrt-devel/2014-April/002793.html
Recently I install mwan3 so disabled sqm. Could anybody confirmed that current version is compatible with mwan3?

limit steam, battle.net, windows updates etc.

hey,
im running sqm qos on a openwrt router and some downloads like steam, battle.net, windows updates etc. still cause bufferbloat.

pie has it's default target overwritten when used in openwrt

even if you specify it to be empty, or try to override it, it still gets set to 5ms. (the default is 16ms)

config queue 'eth0'
option download '85000'
option linklayer 'none'
option enabled '1'
option interface 'eth0'
option upload '5500'
option debug_logging '0'
option verbosity '5'
option qdisc 'pie'
option script 'simplest.qos'
option itarget '16ms'
option etarget '16ms'
option qdisc_advanced '1'
option squash_dscp '1'
option squash_ingress '1'
option ingress_ecn 'ECN'
option egress_ecn 'ECN'
option qdisc_really_really_advanced '1'

Ingress and DSCP Squashing (Ingress Shaping in General)

Just a thought that came back from lots of trials with this... I think that ingress shaping is most if not all of the scripts is broken.

Some scripts include the following to Squash Ingress:

if [ "$SQUASH_DSCP" = "1" ]
then
    ipt -t mangle -I PREROUTING -i $IFACE -m dscp ! --dscp 0 -j DSCP --set-dscp-class be
...

But my understanding is that the Ingress Qdisc is happening BEFORE mangle/PREROUTING as shown in this chart: http://l7-filter.sourceforge.net/PacketFlow.png . Therefore the Squashing or change of DSCP is not taken into account in the shaping.

I have implemented two counter measures for this:

1. Not using Iptables for ingress and going with TC u32 filters alone (port or DSCP based).
2. Using "action connmark" to restore the connection mark previously set on egress. Action connmark is not mainline and is currently missing from 4.1 but is present in 3.18. This is very flexible and allows the use of iptables, although you need to first mark on egress and save that mark.

Maybe Im not doing something right here.

functions.sh: sqm_stop does not delete some iptables rules

These lines seem to fail with "iptables: No chain/target/match by that name.":

ipt -t mangle -D POSTROUTING -o $IFACE -m mark --mark 0x00 -g QOS_MARK_${IFACE}
ipt -t mangle -D PREROUTING -i vtun+ -p tcp -j MARK --set-mark 0x2

I think ${IPT_MASK_STRING} needs to be used on the marks to match the initial setup.

Also, simple.qos: ipt_setup may setup some PREROUTING rules that should also be reversed by sqm_stop:

if [ "$SQUASH_DSCP" = "1" ]
...
    ipt -t mangle -I PREROUTING -i $IFACE -m dscp ! --dscp 0 -j DSCP --set-dscp-class be
...
    ipt -t mangle -A PREROUTING -i $IFACE -m mark --mark 0x00${IPT_MASK_STRING} -g QOS_MARK_${IFACE}

Thanks.

policing attempt. no policing support in openwrt?

in desperation, watching fast.com clobber my link, I wanted to try a two tier system, with cake trying to shape to a 85mbit rate, and then policing slightly below my measured inbound rate (110mbit).

There doesn't appear to be an act_police module in openwrt packages anymore...

tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: prio 9 u32 match u2 0 0 police rate 100mbit burst 16k

prio 10 then sends the redirect to ifb4

RTNETLINK answers: No such file or directory

splitting inbound out

has anyone benchmarked tbf + fq_codel vs cake yet? I think it's going to be a lot faster, but there's not a good way to express it in sqm...

still getting maxpacket > 1514 even after turning off gro

This is how my WAN interfaces' offload features look like BEFORE running sqm, on a fresh boot:

~# ethtool -k eth1
Features for eth1:
rx-checksumming: off [fixed]
tx-checksumming: off
    tx-checksum-ipv4: off [fixed]
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: off [fixed]
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: off [fixed]
scatter-gather: off
    tx-scatter-gather: off [fixed]
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
    tx-tcp-segmentation: off [fixed]
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]
hw-switch-offload: off [fixed]

~# ethtool -k eth0.2
Features for eth0.2:
rx-checksumming: off [fixed]
tx-checksumming: off
    tx-checksum-ipv4: off
    tx-checksum-ip-generic: off
    tx-checksum-ipv6: off
    tx-checksum-fcoe-crc: off
    tx-checksum-sctp: off
scatter-gather: off
    tx-scatter-gather: off
    tx-scatter-gather-fraglist: off
tcp-segmentation-offload: off
    tx-tcp-segmentation: off [requested on]
    tx-tcp-ecn-segmentation: off [requested on]
    tx-tcp6-segmentation: off [requested on]
udp-fragmentation-offload: off [requested on]
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [requested on]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: on [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
fcoe-mtu: off
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]
hw-switch-offload: off [fixed]

As you can see, generic-receive-offload is enabled for both interfaces. After running sqm-scripts, it is still enabled. Shouldn't sqm-scripts have disabled those via:

eth_setup() {
    ethtool -K $IFACE gso off
    ethtool -K $IFACE tso off
    ethtool -K $IFACE ufo off
    ethtool -K $IFACE gro off

    if [ -e /sys/class/net/$IFACE/queues/tx-0/byte_queue_limits ]
    then
       for i in /sys/class/net/$IFACE/queues/tx-*/byte_queue_limits
       do
          echo $(( 4 * $( get_mtu ${IFACE} ) )) > $i/limit_max
       done
    fi
}

in /usr/lib/sqm/functions.sh? I'm not sure what's going on here.

Even after manually disabling gro via ethtool -k DEV gro off then running sqm-scripts, I'm still getting maxpacket > 1514:

~# tc -s qdisc show
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn 
 Sent 8899619 bytes 10245 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc htb 1: dev eth1 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 1000
 Sent 1997199 bytes 2629 pkt (dropped 81, overlimits 2968 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 110: dev eth1 parent 1:11 limit 1001p flows 1024 quantum 300 target 24.6ms interval 119.6ms 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 120: dev eth1 parent 1:12 limit 1001p flows 1024 quantum 300 target 24.6ms interval 119.6ms 
 Sent 1991952 bytes 2596 pkt (dropped 81, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 1749 drop_overlimit 0 new_flow_count 569 ecn_mark 0
  new_flows_len 0 old_flows_len 1
qdisc fq_codel 130: dev eth1 parent 1:13 limit 1001p flows 1024 quantum 300 target 24.6ms interval 119.6ms 
 Sent 5247 bytes 33 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 159 drop_overlimit 0 new_flow_count 1 ecn_mark 0
  new_flows_len 0 old_flows_len 1
qdisc ingress ffff: dev eth1 parent ffff:fff1 ---------------- 
 Sent 3414805 bytes 3346 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 1: dev eth0.2 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 2
 Sent 2108287 bytes 2991 pkt (dropped 138, overlimits 2866 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 110: dev eth0.2 parent 1:11 limit 1001p flows 1024 quantum 300 target 24.6ms interval 119.6ms 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 120: dev eth0.2 parent 1:12 limit 1001p flows 1024 quantum 300 target 24.6ms interval 119.6ms 
 Sent 2103199 bytes 2959 pkt (dropped 138, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 1749 drop_overlimit 0 new_flow_count 967 ecn_mark 0
  new_flows_len 0 old_flows_len 4
qdisc fq_codel 130: dev eth0.2 parent 1:13 limit 1001p flows 1024 quantum 300 target 24.6ms interval 119.6ms 
 Sent 5088 bytes 32 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 159 drop_overlimit 0 new_flow_count 1 ecn_mark 0
  new_flows_len 0 old_flows_len 1
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ---------------- 
 Sent 3694926 bytes 3555 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev wlan0 root 
 Sent 38867 bytes 98 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn 
 Sent 38867 bytes 98 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev wlan0-1 root 
 Sent 38867 bytes 98 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 0: dev wlan0-1 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0-1 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0-1 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn 
 Sent 38867 bytes 98 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0-1 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc htb 1: dev ifb4eth1 root refcnt 2 r2q 10 default 10 direct_packets_stat 0 direct_qlen 32
 Sent 4091706 bytes 3346 pkt (dropped 0, overlimits 2529 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 110: dev ifb4eth1 parent 1:10 limit 1001p flows 1024 quantum 300 target 9.9ms interval 104.9ms ecn 
 Sent 4091706 bytes 3346 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 1749 drop_overlimit 0 new_flow_count 706 ecn_mark 166
  new_flows_len 1 old_flows_len 2
qdisc htb 1: dev ifb4eth0.2 root refcnt 2 r2q 10 default 10 direct_packets_stat 0 direct_qlen 32
 Sent 4346530 bytes 3513 pkt (dropped 42, overlimits 2887 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 110: dev ifb4eth0.2 parent 1:10 limit 1001p flows 1024 quantum 300 target 9.9ms interval 104.9ms ecn 
 Sent 4346530 bytes 3513 pkt (dropped 42, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 1749 drop_overlimit 0 new_flow_count 1038 ecn_mark 201
  new_flows_len 1 old_flows_len 4

Should this be a concern? Reading [1], it states that:

q=1514      # HTB Quantum = 1500bytes IP + 14 bytes ethernet.
            # Higher bandwidths may require a higher htb quantum. MEASURE.
            # Some ADSL devices might require a stab setting.
ethtool -K $ext tso off gso off gro off # Also turn of gro on ALL interfaces 
                                    # e.g ethtool -K eth1 gro off if you have eth1
                                    # some devices you may need to run these 
                                    # commands independently

and

 If you see a maxpacket greater than 1514, you have some tso/gso/gro offload on on some interface.

[1] https://wiki.gentoo.org/wiki/Traffic_shaping

SQM drops packages for Twitch streams.

My brother has Twitch streams running and with SQM enabled they are being throttled a lot causing massive frames drop. Can you guys fix it, please?

It seems Quamtum size is too large in simplest.qos

tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc htb 1: dev eth0 root refcnt 2 r2q 10 default 16 direct_packets_stat 4 ver 3.17 direct_qlen 3
linklayer atm overhead -4 mtu 2047 tsize 512
qdisc fq_codel 110: dev eth0 parent 1:10 limit 1001p flows 1024 quantum 1526 target 17.7ms interval 112.7ms memory_limit 32Mb
qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
qdisc htb 1: dev ifb4eth0 root refcnt 2 r2q 10 default 16 direct_packets_stat 0 ver 3.17 direct_qlen 32
linklayer atm overhead -4 mtu 2047 tsize 512
qdisc fq_codel 110: dev ifb4eth0 parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn

tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev eth0 root refcnt 2 r2q 10 default 16 direct_packets_stat 4 direct_qlen 3
Sent 11639966 bytes 9027 pkt (dropped 1414, overlimits 17052 requeues 0)
backlog 8003b 6p requeues 0
qdisc fq_codel 110: dev eth0 parent 1:10 limit 1001p flows 1024 quantum 1526 target 17.7ms interval 112.7ms memory_limit 32Mb
Sent 11632016 bytes 9021 pkt (dropped 1414, overlimits 0 requeues 0)
backlog 8003b 6p requeues 0
maxpacket 1643 drop_overlimit 0 new_flow_count 3979 ecn_mark 0 memory_used 11202
new_flows_len 2 old_flows_len 5
qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
Sent 1116151 bytes 8637 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev ifb4eth0 root refcnt 2 r2q 10 default 16 direct_packets_stat 0 direct_qlen 32
Sent 1648565 bytes 8637 pkt (dropped 0, overlimits 259 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 110: dev ifb4eth0 parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
Sent 1648565 bytes 8637 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 1696 drop_overlimit 0 new_flow_count 4972 ecn_mark 0
new_flows_len 1 old_flows_len 5

Small Improvements

1. Tuning for 100+ Mbps wan link connections (FTTC/FTTH).
2. Setting of up/dw rate unit of measurements (Mpbs/Kbps).
3. Automate for Link Layer setting.

sqm scripts don't work with NetworkManager 1.10.2

NetworkManager can't make connection after executing sqm start wlan0 command. It worked even with 1.10.1 but not with 1.10.2 ( https://bbs.archlinux.org/viewtopic.php?id=232651 )

NM notification says:

Device can not be configured

nmcli con up WifiName gives output:

Error: Connection activation failed: The base network connection was interrupted

My /etc/sqm/wlan0.iface.conf

UPLINK=6000
DOWNLINK=10240
SCRIPT=simple.qos

Is there way to make it working again? Thanks.

Hotplug script problem with ACTION=ifdown and DEVICE="" (i.e. empty)

See OpenWRT FS#1741 for details and logs.

It appears in OpenWRT-18.06 that hotplug events called on the scripts in /etc/hotplug.d/iface/ with ACTION=ifdown ALWAYS set DEVICE= (no device specified).

When /etc/hotplug.d/iface/11-sqm is executed it has the effect of running:

/usr/lib/sqm/run.sh stop

Which results in ALL configured sqm instances defined in /etc/config/sqm being deconfigured.
Hotplug events with ACTION=ifup always set DEVICE to a valid physical interface as expected, and sqm for that device is configured. However, any other interface(s) deconfigured during ifdown are left unconfigured.

The 11-sqm hotplug script in OpenWRT-17.01 ignored ifdown events so this was not an issue.

The OpenWRT wiki on hotplug events imply that DEVICE is always set, hence my bugreport on OpenWRT. A simple workaround I'm using for now is to ignore ifdown hotplug events in 11-sqm.

Could SQM log verbosity be reduced?

After the recent updates, the log spam from SQM grew to new proportions. After a reboot of the router, there are 122 lines from SQM in the system log (without counting the 24 insmod lines):

root@OpenWrt:/tmp# logread | grep SQM | wc
      122      1466      9682

(It was ~90 lines before the recent commit to make SQM also start at boot, which added one more stop/start round generating ~30 additional lines)

I hope that some of those log items could be pushed to be behind a "verbosity" or "debugging" option.

Most of the lines in the log do not offer much value for an average end-user, so they should be somehow squashed.

STOPPING:

Sat Nov 21 11:29:38 2015 user.notice SQM: /usr/lib/sqm/stop-sqm: Stopping eth1
Sat Nov 21 11:29:38 2015 user.notice SQM: ifb associated with interface eth1:
Sat Nov 21 11:29:38 2015 user.notice SQM: Currently no ifb is associated with eth1, this is normal during starting of the sqm system.
Sat Nov 21 11:59:53 2015 user.notice SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 shaper deleted
Sat Nov 21 11:59:53 2015 user.notice SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 interface deleted

STARTING AGAIN:

Sat Nov 21 11:59:53 2015 user.notice SQM: /usr/lib/sqm/start-sqm: Starting eth1
Sat Nov 21 11:59:53 2015 user.notice SQM: /usr/lib/sqm/start-sqm: Queue Setup Script: simple.qos
Sat Nov 21 11:59:53 2015 daemon.err insmod: module is already loaded - act_ipt
Sat Nov 21 11:59:53 2015 daemon.err insmod: module is already loaded - sch_ingress
Sat Nov 21 11:59:53 2015 daemon.err insmod: module is already loaded - act_mirred
Sat Nov 21 11:59:54 2015 daemon.err insmod: module is already loaded - cls_fw
Sat Nov 21 11:59:54 2015 daemon.err insmod: module is already loaded - cls_flow
Sat Nov 21 11:59:54 2015 daemon.err insmod: module is already loaded - cls_u32
Sat Nov 21 11:59:54 2015 daemon.err insmod: module is already loaded - sch_htb
Sat Nov 21 11:59:54 2015 daemon.err insmod: module is already loaded - sch_hfsc
Sat Nov 21 11:59:54 2015 user.notice SQM: QDISC htb is useable.
Sat Nov 21 11:59:54 2015 user.notice SQM: QDISC fq_codel is useable.
Sat Nov 21 11:59:54 2015 user.notice SQM: Starting simple.qos
Sat Nov 21 11:59:54 2015 user.notice SQM: ifb associated with interface eth1:
Sat Nov 21 11:59:54 2015 user.notice SQM: Currently no ifb is associated with eth1, this is normal during starting of the sqm system.
Sat Nov 21 11:59:54 2015 user.notice SQM: Squashing differentiated services code points (DSCP) from ingress.
Sat Nov 21 11:59:55 2015 user.notice SQM: get_limit:  CURLIMIT: 1001
Sat Nov 21 11:59:55 2015 user.notice SQM: cur_target: auto cur_bandwidth: 10000
Sat Nov 21 11:59:55 2015 user.notice SQM: get_target defaulting to auto.
Sat Nov 21 11:59:55 2015 user.notice SQM: get_limit:  CURLIMIT: 1001
Sat Nov 21 11:59:55 2015 user.notice SQM: cur_target: auto cur_bandwidth: 10000
Sat Nov 21 11:59:55 2015 user.notice SQM: get_target defaulting to auto.
Sat Nov 21 11:59:55 2015 user.notice SQM: get_limit:  CURLIMIT: 1001
Sat Nov 21 11:59:55 2015 user.notice SQM: cur_target: auto cur_bandwidth: 10000
Sat Nov 21 11:59:55 2015 user.notice SQM: get_target defaulting to auto.
Sat Nov 21 11:59:55 2015 user.notice SQM: egress shaping activated
Sat Nov 21 11:59:55 2015 user.notice SQM: QDISC ingress is useable.
Sat Nov 21 11:59:55 2015 user.notice SQM: Do not perform DSCP based filtering on ingress. (1-tier classification)
Sat Nov 21 11:59:55 2015 user.notice SQM: get_limit:  CURLIMIT: 1001
Sat Nov 21 11:59:55 2015 user.notice SQM: cur_target: auto cur_bandwidth: 85000
Sat Nov 21 11:59:55 2015 user.notice SQM: get_target defaulting to auto.
Sat Nov 21 11:59:55 2015 user.notice SQM: ingress shaping activated

Some ideas:

• There are always 3 lines for each of the 3 queues, Really needed?
• There are two lines for ifb, although usually it always end up with the "no ifb associated.." line. Is the first line really needed unless there really was an ifb association? Line 85 could be moved to line 101 behind an "else", so that it would only be printed if there was an ifb. https://github.com/tohojo/sqm-scripts/blob/master/src/functions.sh#L85

I guess that in the case of a normal startup without errors, the end-user is probably interested to see something short like:

SQM starting for interface X using simple.qos (fq_codel) with limits 85000/10000
SQM started ok.

Or something like that.

Ps.
One issue seems to have popped up with the recent changes:
Stopping SQM generates one funny log line about "ifb association". First SQM claims that "no ifb is associated with eth1", then it happily deletes 'ifb4eth1'.

Sat Nov 21 11:29:38 2015 user.notice SQM: /usr/lib/sqm/stop-sqm: Stopping eth1
Sat Nov 21 11:29:38 2015 user.notice SQM: ifb associated with interface eth1:
Sat Nov 21 11:29:38 2015 user.notice SQM: Currently no ifb is associated with eth1, this is normal during starting of the sqm system.
Sat Nov 21 11:59:53 2015 user.notice SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 shaper deleted
Sat Nov 21 11:59:53 2015 user.notice SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 interface deleted

I don't remember seeing that log line earlier. (stopping messages used to take 4 lines, now it is 5 lines).

ipv6 netfiltering always included

I ask this issue but I don't know if any openwrt/lede os supports any kind of ipv6 protocol version as default enabled without any disabling choice

running sqm scripts, tc filter protocol ipv6 & ip6tables, in addiction with standard ipv4 protocol rules, are always included into netfiltering tables (functions.sh, .qos scripts)

in case of any ipv6 support enabled (if this choice is available) the ipv6 netfilter table is uselessly filled:
these rules may be omitted saving memory, improving some speedup, & making netfilter heavyless with an appropriate setting flag

e.g. functions.sh, ipt()

ipt() {
    d=$(echo $* | sed s/-A/-D/g)
    [ "$d" != "$*" ] && {
        sqm_trace "iptables ${d}"
        iptables $d >> ${OUTPUT_TARGET} 2>&1
[ "$IP6MODE" != "0" ] && {
        sqm_trace "ip6tables ${d}"
        ip6tables $d >> ${OUTPUT_TARGET} 2>&1
}
    }
    d=$(echo $* | sed s/-I/-D/g)
    [ "$d" != "$*" ] && {
        sqm_trace "iptables ${d}"
        iptables $d >> ${OUTPUT_TARGET} 2>&1
[ "$IP6MODE" != "0" ] && {
        sqm_trace "ip6tables ${d}"
        ip6tables $d >> ${OUTPUT_TARGET} 2>&1
}
    }
    sqm_trace "iptables $*"
    iptables $* >> ${OUTPUT_TARGET} 2>&1
[ "$IP6MODE" != "0" ] && {
    sqm_trace "ip6tables $*"
    ip6tables $* >> ${OUTPUT_TARGET} 2>&1
}
}

with IP6MODE=0|1 as well

To prepare for full downloading during full upload.

It is getting a half rate download rate during full upload with sqm.
I think it needs priorize (layer).
For example, port 80,443 and so on.

How to control win10 updated

Windows10 systems forced circulation system, large file download will affect ADSL bandwidth saturation

do_modules: module - existence checking and loading

In function do_modules() in /usr/lib/sqm/functions.sh, there is a TODO note to check first whether the modules exist and only load then. How does this implementation look? Is it good enough?

do_modules() {
    KVER=$(uname -r)
    for i in act_ipt sch_$QDISC sch_ingress act_mirred cls_fw cls_u32 sch_htb sch_hfsc
    do
        [ -f /lib/modules/$KVER/$i.ko ] && insmod $i
    done
}

In case of a built-in module, then there's no need to insmod such module, because it is already part of the running kernel. This also avoids errors such as failed to find a module named sch_fq_codel because sch_fq_codel is a built-in module, and insmod sch_fq_codel results in such error.

Cheers,
ianp

More details from tc statistics

Hello Toke, Sebastian, everyone;

I would like to write something to parse tc -s qdisc show output.

However, the output is somewhat cryptic to me.

Would you be willing to provide a quick introduction on what are the most important values to watch and a brief explanation?

For instance, I have this output in my system:

[root@rasp-rodhome ~]#  tc -s qdisc show 
qdisc htb 1: dev eth0.2 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 2
 Sent 195450545 bytes 2252952 pkt (dropped 213, overlimits 65672 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 110: dev eth0.2 parent 1:11 limit 1001p flows 1024 quantum 300 target 14.0ms interval 109.0ms ecn 
 Sent 13233828 bytes 27281 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 942 drop_overlimit 0 new_flow_count 14417 ecn_mark 0
  new_flows_len 1 old_flows_len 1
qdisc fq_codel 120: dev eth0.2 parent 1:12 limit 1001p flows 1024 quantum 300 target 14.0ms interval 109.0ms ecn 
 Sent 182216717 bytes 2225671 pkt (dropped 213, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 3028 drop_overlimit 0 new_flow_count 232568 ecn_mark 1
  new_flows_len 1 old_flows_len 7
qdisc fq_codel 130: dev eth0.2 parent 1:13 limit 1001p flows 1024 quantum 300 target 14.0ms interval 109.0ms ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ---------------- 
 Sent 6787452644 bytes 7604188 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc htb 1: dev ifb4eth0.2 root refcnt 2 r2q 10 default 10 direct_packets_stat 0 direct_qlen 32
 Sent 6859538380 bytes 7581405 pkt (dropped 22782, overlimits 8056300 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc fq_codel 110: dev ifb4eth0.2 parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn 
 Sent 6859538380 bytes 7581405 pkt (dropped 22782, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  maxpacket 1514 drop_overlimit 0 new_flow_count 358042 ecn_mark 0
  new_flows_len 0 old_flows_len 1

Thanks!

• Rodrigo.

Enabling SQM on OpenWrt affects IPv6 speed (through tunnelbroker) badly

I have set up an OpenWrt box, with HE.net TunnelBroker as a 6in4 tunnel. When SQM is disabled, the IPv6 speed test reports that the bandwidth of my IPv6 connection is on par with IPv4 (in my case, 100Mbps). But if I enabled SQM, the speed of IPv6 suddenly drops to a few hundred kbps.

My SQM settings (eth1 is the WAN interface, connected directly to an ISP home ethernet port and gets its IP address directly through DHCP):

config queue 'eth1'
	option interface 'eth1'
	option qdisc_advanced '0'
	option debug_logging '0'
	option verbosity '5'
	option qdisc 'cake'
	option script 'piece_of_cake.qos'
	option download '97280'
	option upload '97280'
	option linklayer 'none'
	option enabled '0'

Relevant network interfaces:

3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether
15: 6in4-wan6@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1280 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/sit

Adding HFSC + SFQ qdisc.

How about hfsc + sfq?
hfsc + fq_codel occurs errors.
but hfsc + sfq is not bad.
sample
https://github.com/robinsmidsrod/SuperShaper-SOHO

sqm-script for ATM calculation .

I think

1. Speed nob should be set ( 53 * 8 )bits * n. Now , kbit per seconds.( tc htb uses 1k=1000 )
2. Fq_codel Target calculation should depends on MTU includes whole cell size . For example , Certain fq_codel target 1696 ( MTU = 1500 )* 8 / Speed Not 1500 * 8 / Speed. Now ? I don't know.
3. quantum number and so on.

Because if we set linklayer=atm , all packet calculation under root qdisc depends on power of 53bytes.

If already did , excuse me.

scaling sqm past 300mbit to as much as 900mbit

making htb's cburst and burst parameters 64k gets the APU2 up to
where it can shape 900mbits (up from ~400mbits) with fq_codel. 3 ksoftirq handlers start getting cpu
time, and we end up 54% idle to achieve that.

I should really go around running my own old code. I was deeply
involved in sqm when we still had to run at sub 200mbit levels. since
then it's been mostly tbf (burst 64k) + fq_codel or cake, and me ignoring various bug
reports about it not scaling well enough at higher rates.

my guess is that burst and cburst should scale roughly as a function
of the bytes that can fit into 1ms.

I also increased the quantum to 32k and 64k without noticing much of an effect.

Sniffing the correct 'ip' using 'which ip' fails during boot

I noticed today strange log messages after flashing the newest Openwrt trunk. There were changes to busybox & iproute2 yesterday, which has caused this.

netlink: 11 bytes leftover after parsing attributes in process `ip'

One of the Openwrt devs determined them to be due to to sqm using erroneously the busybox 'ip' altough the real one from iproute2 should be preferred. He fixed busybox to behave better (https://dev.openwrt.org/changeset/46833/), but I did some further debugging.

I added some debug logging statements to the two places in sqm scripts where 'ip' is sniffed using 'which ip'. And the result is interesting:

• During boot, the "update-available-qdiscs" and "defaults.sh" both sniff 'ip' to be /sbin/ip, i.e. the busybox applet, and using that generated the errors.
  The $PATH variable is "/bin:/sbin:/usr/bin:/usr/sbin"
• But later when I manually restarted SQM, the same "defaults.sh" script sniffs 'which ip' to be /usr/sbin/ip, the full application.
  The $PATH variable is now "/usr/bin:/usr/sbin:/bin:/sbin"

Apparently something changes in the environment in between.

The core issue from sqm perspective is that 'which ip' gives suboptimal results during boot, as path is wrong for some reason. The environment is probably not yet fully set up when the startup script runs. It might be better to manually first look the existence of /usr/sbin/ip, the full ip, before trying 'which ip' as the fallback.

I am not sure if the 'ip' implementations have that much difference, but wanted to document the unreliability of the current sniffing method.

https://dev.openwrt.org/ticket/20470#comment:3

To document my findings, below are log excerpts showing the 'path' variation.

root@OpenWrt:~# logread | grep -i -E "sqm|netlink"
Wed Sep  9 12:54:01 2015 user.notice root: sqm update-avail-qdisc: IP = /sbin/ip, path = /bin:/sbin:/usr/bin:/usr/sbin
Wed Sep  9 12:54:01 2015 user.notice SQM: QDISC fq_codel is useable.
Wed Sep  9 12:54:01 2015 kern.warn kernel: [   28.435431] netlink: 11 bytes leftover after parsing attributes in process `ip'.
...
Wed Sep  9 12:54:12 2015 user.notice root: SQM defaults sniffed IP=/sbin/ip
Wed Sep  9 12:54:12 2015 user.notice root: SQM defaults IP=/sbin/ip, PATH=/bin:/sbin:/usr/bin:/usr/sbin
Wed Sep  9 12:54:12 2015 user.notice SQM: /usr/lib/sqm/start-sqm: Starting eth1
Wed Sep  9 12:54:12 2015 user.notice SQM: /usr/lib/sqm/start-sqm: Queue Setup Script: simple.qos
Wed Sep  9 12:54:13 2015 user.notice SQM: QDISC htb is useable.
Wed Sep  9 12:54:13 2015 kern.warn kernel: [   39.935923] netlink: 11 bytes leftover after parsing attributes in process `ip'.
...
Wed Sep  9 12:54:17 2015 user.notice root: SQM defaults sniffed IP=/sbin/ip
Wed Sep  9 12:54:17 2015 user.notice root: SQM defaults IP=/sbin/ip, PATH=/bin:/sbin:/usr/bin:/usr/sbin
...
...
Wed Sep  9 15:37:52 2015 user.notice root: *** after boot: SQM RESTART ***
...
Wed Sep  9 15:38:09 2015 user.notice SQM: /usr/lib/sqm/stop-sqm: Stopping eth1
Wed Sep  9 15:38:09 2015 user.notice SQM: ifb associated with interface eth1: ifb4eth1
Wed Sep  9 15:38:09 2015 user.notice root: SQM defaults sniffed IP=/usr/sbin/ip
Wed Sep  9 15:38:09 2015 user.notice root: SQM defaults IP=/usr/sbin/ip, PATH=/usr/bin:/usr/sbin:/bin:/sbin
Wed Sep  9 15:38:09 2015 user.notice SQM: /usr/lib/sqm/stop-sqm: ifb4eth1 shaper deleted

Sorry, total SQM noob, but how do you know if SQM is running?

I install sqm-scripts on a linux router that I created myself running ubuntu.

When I did the make install:

install -m 0755 -d /etc/sqm /usr/lib/sqm
install -m 0644 platform/linux/sqm.conf /etc/sqm/sqm.conf
install -m 0644  src/functions.sh src/defaults.sh \
        src/*.qos src/*.help /usr/lib/sqm
install -m 0744  src/start-sqm src/stop-sqm src/update-available-qdiscs \
        /usr/lib/sqm
install -m 0755 -d /usr/lib/systemd/system \
        /usr/lib/tmpfiles.d /usr/bin
install -m 0644  platform/linux/eth0.iface.conf.example /etc/sqm
install -m 0644  platform/linux/[email protected] \
        /usr/lib/systemd/system
install -m 0644  platform/linux/sqm-tmpfiles.conf \
        /usr/lib/tmpfiles.d/sqm.conf
install -m 0755 platform/linux/sqm-bin /usr/bin/sqm
test -d /etc/network/if-up.d && install -m 0755 platform/linux/sqm-ifup \
        /etc/network/if-up.d/sqm || exit 0

I created a file: /etc/sqm/wan.iface.conf

I see the the file: /etc/network/if-up.d/sqm, does that mean that on network up, sqm should auto-start?

The problem is, how do I know that sqm is running?

I tried the normal ways I know:
ps -ef | grep sqm [nothing]
systemctl status sqm [Unit sqm.service could not be found.]

In the make install I see:

install -m 0644  platform/linux/[email protected] \
        /usr/lib/systemd/system

...but when I look in /lib/systemd/system I see nothing related to sqm. Is that correct? This system doesn't use init.d, and I don't use systemd for networking (I use ifupdown for that).

When I manually run: sudo sqm reload wan

Stopping SQM on wan
Starting SQM script: simple.qos on wan, in: 305000 Kbps, out: 24000 Kbps
simple.qos was started on wan successfully

tc -s qdisc show

qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc htb 1: dev wan root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 1000
 Sent 83584 bytes 451 pkt (dropped 0, overlimits 5 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 120: dev wan parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 82257 bytes 437 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 3028 drop_overlimit 0 new_flow_count 141 ecn_mark 0
  new_flows_len 1 old_flows_len 4
qdisc fq_codel 130: dev wan parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 110: dev wan parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 1327 bytes 14 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 226 drop_overlimit 0 new_flow_count 14 ecn_mark 0
  new_flows_len 1 old_flows_len 0
qdisc ingress ffff: dev wan parent ffff:fff1 ----------------
 Sent 266241 bytes 3625 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev lan0 root
 Sent 1358332112 bytes 1060056 pkt (dropped 1157, overlimits 0 requeues 926)
 backlog 0b 0p requeues 926
qdisc fq_codel 0: dev lan0 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 285431946 bytes 230704 pkt (dropped 94, overlimits 0 requeues 201)
 backlog 0b 0p requeues 201
  maxpacket 27252 drop_overlimit 0 new_flow_count 620 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev lan0 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 302118505 bytes 233112 pkt (dropped 187, overlimits 0 requeues 204)
 backlog 0b 0p requeues 204
  maxpacket 27252 drop_overlimit 0 new_flow_count 791 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev lan0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 425334011 bytes 328346 pkt (dropped 537, overlimits 0 requeues 235)
 backlog 0b 0p requeues 235
  maxpacket 27252 drop_overlimit 0 new_flow_count 977 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev lan0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 345447650 bytes 267894 pkt (dropped 339, overlimits 0 requeues 286)
 backlog 0b 0p requeues 286
  maxpacket 27252 drop_overlimit 0 new_flow_count 1043 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev lan1 root
 Sent 1789053 bytes 8162 pkt (dropped 0, overlimits 0 requeues 1)
 backlog 0b 0p requeues 1
qdisc fq_codel 0: dev lan1 parent :4 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 65652 bytes 357 pkt (dropped 0, overlimits 0 requeues 1)
 backlog 0b 0p requeues 1
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev lan1 parent :3 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 22316 bytes 176 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev lan1 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 1482765 bytes 4845 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev lan1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 218320 bytes 2784 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br0 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum 1500 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 236 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc htb 1: dev ifb4wan root refcnt 2 r2q 10 default 10 direct_packets_stat 0 direct_qlen 32
 Sent 317471 bytes 3625 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 110: dev ifb4wan parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
 Sent 317471 bytes 3625 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 4318 drop_overlimit 0 new_flow_count 228 ecn_mark 0
  new_flows_len 0 old_flows_len 1

When I reboot I see the same thing, so I THINK that it's running, but how do I know?

THANKS!

bumping up "limit" at higher bandwidths

For some traffic, at higher speeds than the original scenarios sqm-scripts was targetted for, the default limit can be too low. As to what it should be set to, I dunno, but I can get it to crack 1000 packets outstanding with the right forms of traffic at >200mbps. I have seen 2000 at a gbit, or more.

having a larger limit generally does not hurt if you have sufficient memory.

This is also in relation to the recent bug discussed and fixed in the thread:

https://www.mail-archive.com/[email protected]/msg108623.html

I do not think we need to expose this new parameter. Do need to backport it to openwrt tho.

Including the luci app in this repository?

Would it make sense to keep the Luci app in this repository as well? Might make it easier to keep things in sync, and we could still have separate packages in the openwrt repos...