Giter Site home page Giter Site logo

go-sqlancer's Introduction

Go-sqlancer

Test Go Report Card

Inspired by Manuel Rigger's paper Testing Database Engines via Pivoted Query Synthesis.

Testing approaches

Go-sqlancer has supported Pivoted Query Synthesis (PQS), Non-optimizing Reference Engine Construction (NoREC) and Ternary Logic Partitioning (TLP). You can use -mode to specify the testing approach.

Quickstart

make
bin/go-sqlancer -dsn "root:@tcp(127.0.0.1:4000)/"

And other flags you can set:

Usage of go-sqlancer:
  -approach string
        use NoRec or PQS method or both, split by vertical bar (default "pqs|norec|tlp")
  -depth int
        sql depth (default 1)
  -dsn string
        dsn of target db for testing
  -duration duration
        fuzz duration (default 5h0m0s)
  -enable-expr-idx
        enable create expression index
  -enable-hint
        enable sql hint for TiDB
  -log-level string
        set log level: info, warn, error, debug [default: info] (default "info")
  -silent
        silent when verify failed
  -view-count int
        count of views to be created (default 10)

Supported Statement

Functions & Operators

XOR, AND, OR, NOT, GT, LT, NE, EQ, GE, LE, IF, CASE, IN, BETWEEN, etc.

create table t(a float);
insert t values(NULL);
select * from t where (!(a and a)) is null;

---
tidb> select * from t where (!(a and a)) is null;
Empty set (0.00 sec)
----
mysql> select * from t where (!(a and a)) is null;
+------+
| a    |
+------+
| NULL |
+------+
1 row in set (0.00 sec)
---
create table t0(c0 int);
insert into t0 values(null);

---
tidb> select * from t0 where ((!(1.5071004017670217e-01=t0.c0))) IS NULL;
Empty set (0.00 sec)

tidb> select ((!(1.5071004017670217e-01=null))) IS NULL;
+--------------------------------------------+
| ((!(1.5071004017670217e-01=null))) IS NULL |
+--------------------------------------------+
|                                          1 |
+--------------------------------------------+
1 row in set (0.00 sec)
create table t(c int);
insert into t values(1), (NULL);

---
tidb> select c, c = 0.5 from t;
+------+---------+
| c    | c = 0.5 |
+------+---------+
|    1 |       0 |
| NULL |       0 |
+------+---------+
2 rows in set (0.01 sec)
---
mysql> select c, c = 0.5 from t;
+------+---------+
| c    | c = 0.5 |
+------+---------+
|    1 |       0 |
| NULL |    NULL |
+------+---------+
2 rows in set (0.00 sec)
mysql> desc table_int_float;
+-----------+---------+------+------+---------+----------------+
| Field     | Type    | Null | Key  | Default | Extra          |
+-----------+---------+------+------+---------+----------------+
| id        | int(16) | NO   | PRI  | NULL    | auto_increment |
| col_int   | int(16) | YES  |      | NULL    |                |
| col_float | float   | YES  | MUL  | NULL    |                |
+-----------+---------+------+------+---------+----------------+
3 rows in set (0.00 sec)
mysql> select col_float from table_varchar_float;
+-----------+
| col_float |
+-----------+
|      NULL |
+-----------+

---
tidb> SELECT * FROM table_varchar_float WHERE !(table_varchar_float.col_float and 1) IS NULL;
Empty set (0.00 sec)

View

Table partition

create table t(id int not null auto_increment, col_int int not null, col_float float, primary key(id, col_int)) partition by range(col_int) (partition p0 values less than (100), partition pn values less than (MAXVALUE));
insert into t values(1, 10, 1), (101, 100, 101);

---
tidb> SELECT /*+ use_cascades(TRUE)*/ * from t;
Empty set (0.00 sec)

tidb> SELECT * from t;
+-----+---------+-----------+
| id  | col_int | col_float |
+-----+---------+-----------+
| 101 |     100 |       101 |
|   1 |      10 |         1 |
+-----+---------+-----------+
2 rows in set (0.00 sec)

SQL Hint

  • hash_agg
  • stream_agg
  • agg_to_cop
  • read_consistent_replica
  • no_index_merge
  • use_toja
  • enable_plan_cache
  • use_cascades
  • hash_join
  • merge_join
  • inl_join
  • memory_quota
  • max_execution_time
  • use_index
  • ignore_index
  • use_index_merge
  • qb_name
  • time_range
  • read_from_storage
  • query_type
  • inl_hash_join
  • inl_merge_join

Issues found by go-sqlancer

Fuzz Issues

Notes

For experimental features in tidb, you need add some configs on tiup startup

[experimental]
allow-expression-index = true
allow-auto-random = true
tiup playground nightly --db.config path/to/config/file

go-sqlancer's People

Contributors

hexilee avatar illyrix avatar mahjonp avatar you06 avatar zhouqiang-cl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-sqlancer's Issues

View support

Generate views while testing.

  • common view
  • view based on another view(there can be multi layers)

Automatically analyzes bugs

Analyzing fuzz-bugs manually is boring and ineffective, we need some tools to find the minimally reproducible SQL automatically.

For example, we find the following tow queries behave differently in TiDB:

--Origin
SELECT
    tmp1.id_3 AS col_1,
    tmp1.col_int_3 AS col_2,
    tmp1.col_float_3 AS col_3,
    tmp2.id_0 AS col_4,
    tmp2.col_int_0 AS col_5,
    tmp3.id_2 AS col_6,
    tmp3.col_varchar_2 AS col_7,
    tmp4.id_1 AS col_8,
    tmp4.col_float_1 AS col_9
FROM
    (
        (
            table_int_float AS tmp1
            RIGHT JOIN table_int AS tmp2 ON !("N" <= tmp1.id_3)
        )
        JOIN table_varchar AS tmp3 ON !(
            ""
            AND 1
        )
    )
    RIGHT JOIN table_float AS tmp4 ON !(tmp1.col_float_3 IS NULL)
WHERE
    (8.835640563878916e-01 != "Keu+~E") IS NOT NULL;

--TLP Transformed
SELECT
    tmp1.id_3 AS col_1,
    tmp1.col_int_3 AS col_2,
    tmp1.col_float_3 AS col_3,
    tmp2.id_0 AS col_4,
    tmp2.col_int_0 AS col_5,
    tmp3.id_2 AS col_6,
    tmp3.col_varchar_2 AS col_7,
    tmp4.id_1 AS col_8,
    tmp4.col_float_1 AS col_9
FROM
    (
        (
            table_int_float AS tmp1
            RIGHT JOIN table_int AS tmp2 ON !("N" <= tmp1.id_3)
        )
        JOIN table_varchar AS tmp3 ON !(
            ""
            AND 1
        )
    )
    RIGHT JOIN table_float AS tmp4 ON !(tmp1.col_float_3 IS NULL)
WHERE
    (8.835640563878916e-01 != "Keu+~E") IS NOT NULL
    AND (
        !(
            (NULLIF("<'T", NULL))
            XOR (
                CASE
                    WHEN "2024-01-31 01:40:35" THEN tmp3.col_varchar_2
                END
            )
        )
    ) IS TRUE
UNION
ALL
SELECT
    tmp1.id_3 AS col_1,
    tmp1.col_int_3 AS col_2,
    tmp1.col_float_3 AS col_3,
    tmp2.id_0 AS col_4,
    tmp2.col_int_0 AS col_5,
    tmp3.id_2 AS col_6,
    tmp3.col_varchar_2 AS col_7,
    tmp4.id_1 AS col_8,
    tmp4.col_float_1 AS col_9
FROM
    (
        (
            table_int_float AS tmp1
            RIGHT JOIN table_int AS tmp2 ON !("N" <= tmp1.id_3)
        )
        JOIN table_varchar AS tmp3 ON !(
            ""
            AND 1
        )
    )
    RIGHT JOIN table_float AS tmp4 ON !(tmp1.col_float_3 IS NULL)
WHERE
    (8.835640563878916e-01 != "Keu+~E") IS NOT NULL
    AND (
        !(
            (NULLIF("<'T", NULL))
            XOR (
                CASE
                    WHEN "2024-01-31 01:40:35" THEN tmp3.col_varchar_2
                END
            )
        )
    ) IS FALSE
UNION
ALL
SELECT
    tmp1.id_3 AS col_1,
    tmp1.col_int_3 AS col_2,
    tmp1.col_float_3 AS col_3,
    tmp2.id_0 AS col_4,
    tmp2.col_int_0 AS col_5,
    tmp3.id_2 AS col_6,
    tmp3.col_varchar_2 AS col_7,
    tmp4.id_1 AS col_8,
    tmp4.col_float_1 AS col_9
FROM
    (
        (
            table_int_float AS tmp1
            RIGHT JOIN table_int AS tmp2 ON !("N" <= tmp1.id_3)
        )
        JOIN table_varchar AS tmp3 ON !(
            ""
            AND 1
        )
    )
    RIGHT JOIN table_float AS tmp4 ON !(tmp1.col_float_3 IS NULL)
WHERE
    (8.835640563878916e-01 != "Keu+~E") IS NOT NULL
    AND (
        !(
            (NULLIF("<'T", NULL))
            XOR (
                CASE
                    WHEN "2024-01-31 01:40:35" THEN tmp3.col_varchar_2
                END
            )
        )
    ) IS NULL;

How to find the minimally reproducible query?

TODO: Memory storage for INSERT supporting

make a local storage copy of target DB, to check data consistency
each insert/update/delete operations will cause data changes in memory storage. Pivot row was chosen from the storage instead of from DB

codahale/hdrhistogram repo url has been transferred under the github HdrHstogram umbrella

Problem

The codahale/hdrhistogram repo has been transferred under the github HdrHstogram umbrella with the help from the original author in Sept 2020 (new repo url https://github.com/HdrHistogram/hdrhistogram-go). The main reasons are to group all implementations under the same roof and to provide more active contribution from the community as the original repository was archived several years ago.

The dependency URL should be modified to point to the new repository URL. The tag "v0.9.0" was applied at the point of transfer and will reflect the exact code that was frozen in the original repository.

If you are using Go modules, you can update to the exact point of transfer using the @v0.9.0 tag in your go get command.

go mod edit -replace github.com/codahale/hdrhistogram=github.com/HdrHistogram/[email protected]

Performance Improvements

From the point of transfer, up until now (mon 16 aug 2021), we've released 3 versions that aim support the standard HdrHistogram serialization/exposition formats, and deeply improve READ performance.
We recommend to update to the latest version.

Bug: TLP should not support on-condition partition after outer join

Description

Assumes there are two tables:

> SELECT * FROM t0;
+------+
| id   |
+------+
|    0 |
+------+

> SELECT * FROM t1;
+------+
| id   |
+------+
|    1 |
+------+

Execute the origin query:

SELECT t0.id as id_0, t1.id as id_1 FROM t0 LEFT JOIN t1 ON TRUE;
+------+------+
| id_0 | id_1 |
+------+------+
|    0 |    1 |
+------+------+

Execute the transformed query:

SELECT t0.id as id_0, t1.id as id_1 
FROM t0 LEFT JOIN t1 
ON TRUE AND t0.id=t1.id IS TRUE 
UNION ALL 
SELECT t0.id as id_0, t1.id as id_1 
FROM t0 LEFT JOIN t1 
ON TRUE AND t0.id=t1.id IS FALSE 
UNION ALL 
SELECT t0.id as id_0, t1.id as id_1 
FROM t0 LEFT JOIN t1 
ON TRUE AND t0.id=t1.id IS NULL;
+------+------+
| id_0 | id_1 |
+------+------+
|    0 | NULL |
|    0 |    1 |
|    0 | NULL |
+------+------+

Fix

#56 has fixed part of this bug by scan outer join at the root of join-tree.

We need to scan the whole tree to fix this bug completely.

TODO: Add unit tests of generator.go

a problem to be solved is many functions in generator are related to rand() so it's difficult to give an assertion
how to verify an AST is another problem

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.