Unrealisticly high upload measurements

[openspeedtest]

Unrealisticly high upload measurements

[openspeedtest]

try
labels:

• "traefik.http.middlewares.limit.buffering.maxRequestBodyBytes=10000000000"

The error was when we trying to test upload speed which is sending sever url with 30Mb of data but traefik for some reason preventing or doing something wrong. After adding maxRequestBodyBytes to 10000000000. Issue resolved for me.

I tested this on my local machine by running Nginx

http:
  routers:
    router1:
      service: myService
      middlewares:
        - "maxRequest-fix"
      rule: "Host(`192.168.1.170`)"
      tls: true
    
   

  middlewares:
   maxRequest-fix:
      limit:
      buffering:
        maxRequestBodyBytes: 10000000000

  services:
    myService:
      loadBalancer:
        servers:
          - url: "http://127.0.0.1:4000"

I will test docker and post updates.

Docker version tested. working fine.

version: "3.3"

services:
  traefik:
    image: "traefik:v2.5.5"
    command:
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --providers.docker
      - --api
      - --api.insecure
      #Enale Let's Encrypt
      #- --certificatesresolvers.leresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory
      #- [email protected]
      #- --certificatesresolvers.leresolver.acme.storage=/acme.json
      #- --certificatesresolvers.leresolver.acme.tlschallenge=true
   
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      #- "./acme.json:/acme.json"
    
    labels:
      # Dashboard
      - "traefik.http.routers.traefik.rule=Host(`traefik.docker.localhost`)"
      - "traefik.http.routers.traefik.service=api@internal"
      - "traefik.http.routers.traefik.tls.certresolver=leresolver"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.middlewares=authtraefik"
      - "traefik.http.middlewares.authtraefik.basicauth.users=user:$$apr1$$q8eZFHjF$$Fvmkk//V6Btlaf2i/ju5n/" # user/password
      
      # global redirect to https
      - "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
      - "traefik.http.routers.http-catchall.entrypoints=web"
      - "traefik.http.routers.http-catchall.middlewares=redirect-to-https"

      # middleware redirect
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"

 
  
  openspeedtest:
    image: openspeedtest/latest
    labels:
      - "traefik.http.routers.openspeedtest.rule=Host(`speedtest.docker.localhost`)"
      - "traefik.http.routers.openspeedtest.entrypoints=websecure"
      - "traefik.http.middlewares.limit.buffering.maxRequestBodyBytes=10000000000"
      - "traefik.http.routers.openspeedtest.middlewares=limit"
      - "traefik.http.routers.openspeedtest.tls=true"
      - "traefik.http.routers.openspeedtest.tls.certresolver=leresolver"
      

   

# Traefik Dash http://traefik.docker.localhost/dashboard/#/
# Basic auth (login: user | password: password)
# OpenSpeedTest (http://speedtest.docker.localhost/)
# touch acme.json; chmod 600 acme.json

[openspeedtest]

try

version: "3.9"
services:
traefik:
image: traefik:v2.8.0
container_name: traefik
command:
- --log.level=INFO
- --api.insecure=true
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
ports:
- 3000:80
- 8080:8080
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
openspeedtest:
image: openspeedtest/latest
container_name: speedtest-service
labels:
traefik.enable: 'true'
traefik.http.routers.openspeedtest.rule: Host(`192.168.1.128`)
traefik.http.routers.openspeedtest.entrypoints: web
traefik.http.middlewares.limit.buffering.maxRequestBodyBytes: 10000000000
traefik.http.routers.openspeedtest.middlewares: limit

[milindpatel63]

Ok so, adding all these below 4 options in traefik middleware seems to solve my issue...

  middlewares:
    openspeedtest-limit:
      buffering:
        maxRequestBodyBytes: 10000000000
        memRequestBodyBytes: 10000000000
        memResponseBodyBytes: 10000000000
        maxResponseBodyBytes: 10000000000

Do note, only adding the maxRequestBodyBytes like below

  middlewares:
    openspeedtest-limit:
      buffering:
        maxRequestBodyBytes: 10000000000

causes the insane upload speeds issue to be solved, but then I was having very slow download speeds....like 7 Mbps...it seems like the download test was being timed out intermittently....
adding all the 4 options mentioned above have solved it to some extent.

Still, I can get only about
Download: 633Mbps
Upload: 833Mbps
with traefik.

While, accessing directly via IP:PORT, I can get
Download: 924Mbps
Upload: 972Mbps

I guess, that's as high as I can go with traefik.
I am running openspeedtest and traefik on an old laptop with i5-3230m (4 threads) and 8GB ram.
Thanks for quick reply @openspeedtest 🤗

[Cebrain]

Only for documentation, i have the same problem with traefik and openspeedtest

[Cebrain]

@openspeedtest
Thanks, this limit setting fixed the problem. Tested it on my instance 👍

[openspeedtest]

@milindpatel63 For optimal performance, it's recommended to use it without a reverse proxy, if feasible.

[openspeedtest]

What was the download speed reported? That was expected result?
Definately upload data is getting compressed.
Try docker image without any modification.
We are not using any server side language to avoid data compression. We are doing it via Nginx config. So Nginx behaviour is critical. Avoid using any other server infront of Nginx. Otherwise make similar configuration on your server like turn off Gzip etc. Look at Nginx config for more information.

[openspeedtest]

You can use your own certificate in docker itself.

https://hub.docker.com/u/openspeedtest

How to use your own SSL Certificate?

You can mount a folder with your own SSL certificate to this docker container by adding the following line to the above command.

-v /PATH-TO-YOUR-OWN-SSL-CERTIFICATE**:/etc/ssl/**

I am adding a folder with nginx.crt and nginx.key from my desktop by using the following command.

docker run -v /Users/vishnu/Desktop/docker/:/etc/ssl/ --restart=unless-stopped --name openspeedtest -d -p 3000:3000 -p 3001:3001 openspeedtest/latest

[apiening]

Yes, the reported download speed had the expected result. The value is very close if not exactly the download speed that LibreSpeed reported as well.

Unfortunately I don't have the option to run OpenSpeedtest on it's own IP. I have over 30 services running on the system, all behind the SSL / reverse proxy.

Can you tell how the data that is posted during the upload speed test is generated? If it contains just zeroes or a very simple repeated pattern that would explain why it could be compressed at a very high rate.

I have opened a support request at the community forum for Traefik, asking if there is an option available to tell the client / browser to not compress the post data (see https://community.traefik.io/t/modify-header-to-disallow-compression-per-docker-service/12582).

[openspeedtest]

Edit: After 2 months i rewrite, garbage data generation function. now we use incompressible data, Little CPU Heavy Task but worth it.

Yes, it is easily compressible data, i don't think it is a good idea to generate hard to compress data at client side (Think about devices with poor performance or a device from 2008 or 10) so we disable compression option via Nginx.

In Nginx all you need to do is add "gzip off;" to your config.

Take a look at our Nginx config.
https://github.com/openspeedtest/Nginx-Configuration/blob/main/OpenSpeedTest-Server.conf

You don't have to run it on it's own ip address. You can add your own SSL Certificate to the docker image. That is explained in my second comment.

I don't know much about Traefik.

Ask then about turn off or add header to specific location

like yourdomain.com/upload

"upload" is a static file. So you need to enable post to static file.

In Nginx we do "error_page 405 =200 $uri;" to enable POST to static file.

Take a look at browser console and network tab while upload test is running and see what is the error message.

if there is no error message you need to disable compression algorithms like Gzip from server side.

[apiening]

Unfortunately, I have no answer on my post in the Traefik forum.

I tried to catch the difference between the Nginx config and Traefik, but I can't figure out the relevant setting.

I still think this issue might be solvable with a different configuration for the Traefik proxy, but I wonder why I don't observe a similar issue with the LibreSpeed speedtest (https://github.com/librespeed/speedtest).
I'm running LibreSpeed with exactly the same configuration and the upload speed is not affected / reported correctly.

You don't have to run it on it's own ip address. You can add your own SSL Certificate to the docker image. That is explained in my second comment.

I have one single IP address on the web server and the ports 80 and 443 are occupied. Since I want to have all services running on HTTPs standard port 443, I can't see any way other than using the Domain / SNI based reverse proxy that serves all the other HTTPs based services as well.

[Cebrain]

I also checked the traefik documentation, till now i couldn't find anything to resolve this problem.
For me also i want to use traefik because the used ports

[openspeedtest]

Turn off Compression https://doc.traefik.io/traefik/v2.0/middlewares/compress/

@apiening They are using php to handle this and we use Nginx then turn off Gzip.

You can mount SSL certificate via docker.

You need to write a custom configuration in traefik to handle compression and SSL.

I am not familiar with traefik. Still I like to give it a try. And I will post an update if I got it working.

You guys are using Let's Encrypt or commercial SSL ?

And post the configuration if possible.

Thanks 👍

[Cebrain]

@openspeedtest thanks for your response and pointing in the correct direction.

After adding the label to my container, i only get 10mbit download speed (before 1000mbit on a Gbit Link) and upload is less but still much to high.

"traefik.http.middlewares.test-compress.compress=true"

At the moment i use a local https cert, that isnt from Lets Encrypt.

[openspeedtest]

I setup Traefik on my local machine.
While uploading. from Chrome console I can see Connection reset error message.

Can you post your docker compose file with labels you used?

Also take a look at browser console.

If possible post logs from Traefik after a test.

[openspeedtest]

running without error after adding
traefik.http.routers.openspeedtest.tls=true
i was running without ssl.

This is from a local machine.

version: '2'

services:
  openspeedtest:
    image: openspeedtest/latest
    container_name: openspeedtest
    labels:
      - traefik.enable=true
      - traefik.http.routers.openspeedtest.rule=Host(`speed.localhost`)
      - traefik.http.routers.openspeedtest.entrypoints=web
      - traefik.docker.network=web
      - traefik.http.services.openspeedtest.loadbalancer.server.port=3000
      - traefik.http.routers.openspeedtest.tls=true
      
    restart: unless-stopped
    networks:
      - web
    ports:
      - 3000:3000

networks:
  web:
    external:
      name: web

OST.mp4

[Cebrain]

I have the same setting but still the same problem, but i found something out.

When i use the Internet Explorer (on WIndows 10) it works fine.

On Firefox:

On Edge Browser (Chromium Based) is the absulut Winner

[openspeedtest]

Take a look at your browser console and network tab while upload test is going on.
And post a screenshot if you see any error.
Also post logs / full config including compose file for openspeedtest and traefik
It will be useful for everyone.

[openspeedtest]

k8s people check this

[openspeedtest]

Nginx reverse proxy

[openspeedtest]

IIS SERVER
Nginx Server

[openspeedtest]

Caddy #67

[openspeedtest]

Reverse Proxy is Expensive. Your server will consume more CPU & RAM. Do not run it behind a reverse proxy unless it's absolutely necessary.

• If your server is very powerful, you may not notice a performance drop while running this behind a reverse proxy.
• If you use this on a raspberry pi, you will see a huge performance drop when running behind a reverse proxy.
• Bad configuration, such as not using hardware for encryption and decryption, may also lead to poor performance.
• Run tests with and without reverse proxy.
• If the performance is adequate, deploy it.

[milindpatel63]

I am still having this issue, but only in chromium based browsers.
I am running openspeedtest in Docker on Ubuntu with traefik in front of it pointing to http of openspeedtest with no middleware.

On my local network on another windows PC via gigabit ethernet,
on Chrome
Download: 865 Mbps
Upload: 18254 Mbps

on Firefox
Download: 798 Mbps
Upload: 670 Mbps

After adding maxRequestBodyBytes: 10000000000 in middleware,
on Chrome
Download: 7 Mbps
Upload: 603 Mbps

on Firefox
Download: 365 Mbps
Upload: 458 Mbps

I have tried 3 chromium based browsers (Chrome, Edge, and Thorium), all have same result.
On firefox it's all fine.
No error in console on chrome.
Also, no error in openspeedtest docker logs.

Since, it's specific to chromium browsers, I think its not related to traefik.

[openspeedtest]

Add
traefik.http.routers.openspeedtest.middlewares: limit
@milindpatel63

[milindpatel63]

Add traefik.http.routers.openspeedtest.middlewares: limit @milindpatel63

yes, ofcourse, i have added that.
I don't use docker labels for traefik config. Instead I use traefik's file based config.
Here's my relevant config.yml

http:
  routers:
    openspeedtest:
      entryPoints:
        - "https"
      rule: "Host(`openspeedtest.example.com`)"
      middlewares:
        - openspeedtest-limit
      tls: {}
      service: openspeedtest
  services:
    openspeedtest:
      loadBalancer:
        servers:
          - url: "http://127.0.0.1:3000"
        passHostHeader: true
  middlewares:
    openspeedtest-limit:
      buffering:
        maxRequestBodyBytes: 10000000000

[samdakap]

Hi, I'm not using traefik but a simple apache server with a configuration doing the ReverseProxy thing.
How can I configure the apache server?
Note that it's not running in a container. It's not even on the same physical machine as the server that runs openspeedtest.

I have tried adding LimitRequestBody in the virtual host configuration as follows:

<Location "/">
      RequestHeader set X-Forwarded-Proto 'https'
      LimitRequestBody  0
      LimitXMLRequestBody 0

      ProxyPreserveHost On
      ProxyPass http://$local_ip:$local_port/
      ProxyPassReverse http://$local_ip:$local_port/
    </Location>

But I'm still getting very low speeds compared to other openspeedtest instances hosted outside my local network.
Just to be clearer, when I test the connection from my computer (on local network) to OST on local network too, I get something like 70Mbp down and up. Yet, from the same computer on the same network, but testing using OST on external network (either own VPS or official OST webpage), I get >400Mbps down and >150Mbps up...

EDIT: note that I tested http://local_ip:local_port, https://local_ip:local_ssl_port, https://domain.to.local.com to test the OST instance hosted on my local network... All give the same low speed.

[openspeedtest]

Limit should be greater than 35 Mb.
Using HTTP 1.1 ?

[debiantippse]

If you are lazy like me you can simply use this sample compose config and use middleware chaining if you use other middlewares.

version: '3.8'

services:
  openspeedtest:
    image: openspeedtest/latest
    networks:
      - proxy
    container_name: openspeedtest
    labels:
      traefik.docker.network: proxy
      traefik.enable: "true"
      traefik.http.routers.openspeedtest-secure.entrypoints: websecure
      traefik.http.routers.openspeedtest-secure.middlewares: default@file, limit@docker
      traefik.http.middlewares.limit.buffering.maxRequestBodyBytes: "10000000000"
      traefik.http.routers.openspeedtest.middlewares: limit
      traefik.http.routers.openspeedtest-secure.rule: Host(`speedtest.example.com`)
      traefik.http.routers.openspeedtest-secure.service: openspeedtest
      traefik.http.routers.openspeedtest-secure.tls: "true"
      traefik.http.routers.openspeedtest-secure.tls.certresolver: http_resolver
      traefik.http.routers.openspeedtest.entrypoints: web
      traefik.http.routers.openspeedtest.rule: Host(`speedtest.example.com`)
      traefik.http.services.openspeedtest.loadbalancer.server.port: "3000"
networks:
  proxy:
    external: true

[Anonym-tsk]

How to change post-body content length on Synology DSM Reverse Proxy?

[openspeedtest]

@Anonym-tsk try this instead #94

[Anonym-tsk]

@openspeedtest disabling http/2 is not solution, just ugly workaround.

[openspeedtest]

@Anonym-tsk did you try #94 (comment) ? Disabling http2 is posted as another option.

Synology uses Nginx. If you can apply the mentioned configuration (in the Nginx Config) to handle HTTP/2 or HTTP/3, OST will work without any issues. However, I don't know how to make persistent changes on DSM Nginx.

[Anonym-tsk]

@openspeedtest yes, I tries these workarounds.

1. Disabling http2 works (but I need http2)
2. Using http instead of https potentially works, but I need hsts and this workaround doesn't work for me
3. Changing nginx config doesn't work on DSM 7.2 because config is autogenerated and overwrites every restart

For now there is no working solutions for Synology

[openspeedtest]

Yes, we need to find a way to modify the DSM Nginx configuration, try Nginx Proxy Manager, or explore other proxy servers that work. Moving OST to a separate IP address may help. HTTP/2 only establishes a single TCP connection to the server, while HTTP/1.1 can make up to six. For SpeedTest, HTTP/1.1 is generally better.