gitspartv/luajit-benchmarks

LuaJIT Benchmark tests

Spar

• 6 years in Lua
• 3 years in C++
• 1 year of commercial development
• Email: [email protected]
• LinkedIn: https://www.linkedin.com/in/linkedsavva/ (working experience is explained here, but you can also request my CV)

• С++ Portfolio
• Lua Portfolio

Languages, technologies, stuff, frameworks and tools I've used in some way:

• C++, Lua, LuaJIT, C++17, C++20, Python, Plain HTML + CSS + JavaScript, Groovy
• Boost, telecom (LTE), 3GPP, PostgreSQL, Docker, MySQL, fmt, SQLite, protobuf, webhook, coroutines, async, regex
• CalDAV, Diameter protocol, WebSocket, SOAP API, JSON, REST API, SVG, XML
• Compiler development, web scraping (University sites, Cian.ru, various property developer sites, Notion, hh.ru), C++ reverse-engineering/decompilation (x86-64, ARMv7-M) using IDA and Ghidra, DSL, Bot development (Discord, Telegram, VK, Pachca), logging, game server frontend/backend development, optimizations, gamedev, analytics, game UI development
• Source Engine, Luvit, Raspberry Pi, Nelua, Linux, LÖVE, NodeMCU, Arduino
• CMake, Conan, Grafana, Prometheus

My Projects:

• Hanley bot — Successfull pet-project bot written in C++ with over 200 users made for psychology specialists.
• Lua Patterns — A tool to learn and analyze Lua Patterns.
• Lua Infographics — Collection of infographics dedicated to Lua (Some serious and just for fun).
• LuaJIT Benchmarks — A page about optimizing Lua by benchmarking several variants of the same code and finding the best one (among them).
• LLLua — My language I plan to develop. See the concept for the future version (0.1 version can be already used).
• luamemprof — Memory profiler module for Lua and LuaJIT.
• My Discordia Libraries — The open-sourced part of Toolgun bot developed for the Garry's Mod official Discord server. The server has over 80,000 members as yet replacing the need for additional human moderation for more than 3 years. Includes Lua minidump reader library.
• discordia-slash — Discordia's extension for slash commands.
• Love2D Framework — Framework with GUI, timers and texture loading libraries based on GMod API.
• GodSentTools — Series of GMod addons, mainly for art and moviemakers.
• RemPos — GMod C++ module that connects iOS app with GMod and brings acceleration and gyroscope data.
• My GMod Libraries — Public libraries I made for my GMod server. Server is dead, but libraries can be used/refactored.
• University admission advisor — Bot written in Lua with VK API that downloads lists of applicants from different universities with my specialization, parses them and calculates the chance to pass.
• UniBot — Bot written in Lua with VK API that gives information about the schedule, next lesson and other things.
• LuaCUE — Corsair Utility Engine module for Lua written in C++.
• SparTask — Windows taskbar replacement for focusing on specific task, allows putting only necessary icons.

See all repositories...

Important

Since 2017 I've contributed a lot to Garry's Mod. Check out this list as well.

Projects I participated in:

• VSCode GLua Extension — Helped with making bytecode heatmap, uses LLLua bytecode reader rewritten in JS.

Small projects:

• Advent Of Code 2020 — My solutions for Advent Of Code 2020 written in Lua.
• Lua Error Statistics — Statistics of the most common errors reported by users on Facepunch forum.
• globalize — Simple POSIX program to install programs to /usr/local/bin/.
• Langston's Ant Art — Texture generator using Langston's Ant method, written in Love2D.
• Bruteforcer for a task from School 21 — Task I failed to do by hand. Wrote a bruteforcer for it. Written in Love2D.

GitHub Stats:

• 17 PRs for open source projects. Most noticeable contributions is the Garry's Mod codebase.
• 29 bug reports and suggestions only for Garry's Mod and same amount for other open source projects.
• 

luajit-benchmarks's Issues

Do benchmark with JIT'ed next() and ipairs()

Title :

Using LuaJIT/LuaJIT#275 and lua coded pairs()

Benchmark 3 doesn't give the same result on my computer.

Code 1

-- 91.679 ms on my computer

local get = function(self, idx)
    if type(idx) == "number" then
        idx = idx + 1
    end
    return rawget(self, idx)
end

local set = function(self, idx, val)
    if type(idx) == "number" then
        idx = idx + 1
    end
    rawset(self, idx, val)
end

local utils = { get = get, set = set }

local array = {}

for i = 1, 5000, 1 do
    array[i] = math.random(1000000)
end

len = #array;

for i = 0, len - 2, 1 do
    for j = 0, len - 2 - i, 1 do
        a = utils.get(array, j)
        b = utils.get(array, j + 1)
        if a > b then
            utils.set(array, j, b)
            utils.set(array, j + 1, a)
        end
    end
end

Code 2

-- 100.08 ms on my computer
local get = function(self, idx)
    if type(idx) == "number" then
        idx = idx + 1
    end
    return rawget(self, idx)
end

local set = function(self, idx, val)
    if type(idx) == "number" then
        idx = idx + 1
    end
    rawset(self, idx, val)
end

local utils = { get = get, set = set }

local array = {}

for i = 1, 5000, 1 do
    array[i] = math.random(1000000)
end

len = #array;

for i = 0, len - 2, 1 do
    for j = 0, len - 2 - i, 1 do
        a = get(array, j)
        b = get(array, j + 1)
        if a > b then
            set(array, j, b)
            set(array, j + 1, a)
        end
    end
end

Check out my repo for more details.

Reproducibility

Hello,

It would be sweet if people could see the script used to generate the results and run that script themselves. Manually editing the benchmark script is a slow and error-prone process.

[test2] Instructions outside of the hot loop are counted

https://gitspartv.github.io/LuaJIT-Benchmarks/#test2

The comparison tells the following:

Global table indexing: 38 instructions total.
Local: 18 instructions total.

Diff

  mov dword [0x24660410], 0x1
  movsd xmm7, [rdx+0x40]
  cvttsd2si eax, xmm7
  xorps xmm6, xmm6
  cvtsi2sd xmm6, eax
  ucomisd xmm7, xmm6
  jnz 0x7ffa543c0010        ->0
  jpe 0x7ffa543c0010        ->0
  cmp eax, 0x7ffffffe
  jg 0x7ffa543c0010 ->0
  cvttsd2si edi, [rdx+0x38]
  cmp dword [rdx+0x14], -0x09
- jnz 0x7ffa543c0010        ->0
- cmp dword [rdx+0x10], 0x2467f788
- jnz 0x7ffa543c0010        ->0
- mov ebp, [0x2467f790]
- cmp dword [rbp+0x1c], +0x3f
- jnz 0x7ffa543c0010        ->0
- mov ebx, [rbp+0x14]
- mov rsi, 0xfffffffb24665fd8
- cmp rsi, [rbx+0x518]
- jnz 0x7ffa543c0010        ->0
- cmp dword [rbx+0x514], -0x0c
  jnz 0x7ffa543c0010        ->0
+ cmp dword [rdx+0x18], 0x2466aca0
- mov edx, [rbx+0x510]
- cmp dword [rdx+0x1c], +0x1f
- jnz 0x7ffa543c0010        ->0
- mov ecx, [rdx+0x14]
- mov rsi, 0xfffffffb24666548
- cmp rsi, [rcx+0x230]
- jnz 0x7ffa543c0010        ->0
- cmp dword [rcx+0x22c], -0x09
- jnz 0x7ffa543c0010        ->0
- cmp dword [rcx+0x228], 0x24666520
  jnz 0x7ffa543c0010        ->0
  add edi, +0x01
  cmp edi, eax
  jg 0x7ffa543c0014 ->1

The benchmark counts overall instructions, not only the hot loop. The real performance is the same for both variants (at least for this particular test).

-- luajit-bench-2-v1.lua
local function test()
    local acc = 0
    for i = 1, 10^8 do
        acc = acc + math.sin(3.14)
    end
    return acc
end

print(test())

-- luajit-bench-2-v1.lua
local s = math.sin

local function test()
    local acc = 0
    for i = 1, 10^8 do
        acc = acc + s(3.14)
    end
    return acc
end

print(test())

Curious fact: if I change sin(3.14) to sin(i), the loops in both code variants (and so timings) are still equivalent.

You can easily look at both variants of IR/asm using luajit.me (just replace 10^8 with 10^5 or something like so).

Side note: It is hard to spot that the diff on the web page shows the code outside of the hot loop. I worth to show more context in the diff at least.

Is there a link to authority regarding the content of the statement?

LuaJIT stated that globals and locals now has the same performance unlike in plain Lua.

Is there an authoritative source of information? Like official links or something