Giter Site home page Giter Site logo

arana-db / arana Goto Github PK

View Code? Open in Web Editor NEW
289.0 289.0 92.0 3.73 MB

Arana is a Cloud Native Database Proxy. It can also be deployed as a Database mesh sidecar.

Home Page: http://arana-docs.rtfd.io/

License: Apache License 2.0

Go 99.75% Dockerfile 0.09% Makefile 0.11% Just 0.05%
clould database databasemesh mysql

arana's Issues

user list

Welcome to use arana. To know who is using it, pls append your org info as follow,

  • Organization：Alibaba (Required)
  • Location: Hangzhou, Zhejiang, China（Required）
  • Contact: [email protected] (Optional)
  • Purpose：used as our fundamental network transport lib (Required)
  • Registry: zookeeper（Required）
  • Version: 1.5.5（Required）
  • Com/Org Logo：(Optional)
    Thanks again for your participation!

欢迎使用 arana，首先感谢你的使用，其次您可以参考下面的样例来提供您的信息以收集下使用场景：

* 组织：阿里巴巴（Required）
* 地点：**杭州（Required）
* 联系方式：[email protected]（Optional）
* 场景：阿里内部服务化改造（Required）
* 注册中心: zookeeper（Optional）
* 版本: 1.5.5（Optional）
* 公司图标：（Optional）

再次感谢你的参与!!!

filter mechanism

常规的请求应答模型，filter 机制只需要在代理层实现一个 filter chain，类似于高速公路上的收费站，每条通路上设置一个收费站，实现对请求的拦截。由于 DB 路由有 sharding 的需求，它的 filter 机制应该类似于水站，水源经过水站拦截后，分散成无数的水管，每个水管上安装有水阀。
db proxy 001
需要两种类型的 filter，一种是 executor filter，一种是 connection filter。

Default tag not work when parse config

What happened:
default tag not work when parse config
What you expected to happen:
default tag should work when var is empty
How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Start arana server, execute TestSelect method in integration_test.go will case panic.

What happened:

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

2021-12-30T16:22:00.085+0800    DEBUG   executor/redirect.go:137        ComQuery: SET NAMES utf8mb4
2021-12-30T16:30:24.726+0800    DEBUG   executor/redirect.go:137        ComQuery: SET NAMES utf8mb4
^[[20~2021-12-30T16:30:37.444+0800      DEBUG   executor/redirect.go:137        ComQuery: SET NAMES utf8mb4
2021-12-30T16:30:46.986+0800    DEBUG   executor/redirect.go:137        ComQuery: SET NAMES utf8mb4
2021-12-30T16:31:01.774+0800    DEBUG   executor/redirect.go:137        ComQuery: SET NAMES utf8mb4
2021-12-30T16:31:04.551+0800    DEBUG   executor/redirect.go:137        ComQuery: SET NAMES utf8mb4
2021-12-30T16:31:04.553+0800    INFO    executor/redirect.go:193        SELECT emp_no, birth_date, first_name, last_name, gender, hire_date FROM employees 
                WHERE emp_no = 100001
2021-12-30T16:31:04.554+0800    ERROR   mysql/server.go:156     mysql_server caught panic:
trying to call recycleReadPacket while currentEphemeralPolicy is 0
github.com/dubbogo/arana/pkg/mysql.(*Listener).handle.func1
        /Users/dongzonglei/source_code/Github/arana/pkg/mysql/server.go:156
runtime.gopanic
        /usr/local/go/src/runtime/panic.go:971
github.com/dubbogo/arana/pkg/mysql.(*Conn).recycleReadPacket
        /Users/dongzonglei/source_code/Github/arana/pkg/mysql/conn.go:345
github.com/dubbogo/arana/pkg/mysql.(*Listener).handle
        /Users/dongzonglei/source_code/Github/arana/pkg/mysql/server.go:186

readme

What happened:

no readme

What you expected to happen:

readme

Anything else we need to know?:

tell us what it is and how to use it.

Support table metadata

What would you like to be added:
Support table metadata for star expand, in case:
select * from person
->
select id, name from person

Why is this needed:

'unknown error: no such logical database \x02employees'

What happened:

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

  1. Start server refer this PR comment: #90 .
  2. Connect the server with mycli: mycli -h127.0.0.1 -P13306 -udksl employees -p123456
  3. show databases; is OK.
  4. use employees; is OK.
  5. show tables; is not OK. error message: (1105, 'unknown error: no such logical database \x02employees')

image

Implement 'SHOW TABLES' statement

What would you like to be added:
The statement of SHOW TABLES should only list those logical tables and non-sharding tables.
Please see the official manual, https://dev.mysql.com/doc/refman/5.7/en/show-tables.html

Why is this needed:
The tables in ARANA means 'logical tables' instead of 'physical tables', we need to handle it. For example, we have student_0000..student0031, the actual result should be student.

一些编程细节

What happened:

dubbogo 积累的一些历史旧账

What you expected to happen:

多注意细节，多看看历史问题，多翻翻旧账，照亮来时路，少犯错。

Handling login without database

What would you like to be added:
Handling login without database, for example:

$ mycli -h127.0.0.1 -P13306 -uarana -p123456

Why is this needed:

表中有值，也给客户端返回结果了，但是客户端没解析出来 

var v = &struct {
	EmpNo     int       `gorm:"emp_no"`
	BirthDate time.Time `gorm:"birth_date"`
	FirstName string    `gorm:"first_name"`
	LastName  string    `gorm:"last_name"`
	Gender    string    `gorm:"gender"`
	HireDate  time.Time `gorm:"hire_date"`
}{}

db, _ := sql.Open("mysql", "dksl:123456@tcp(127.0.0.1:13306)/employees?timeout=1s&readTimeout=1s&writeTimeout=1s&parseTime=true&loc=Local&charset=utf8mb4,utf8") //连接数据库
defer db.Close()

row := db.QueryRow(`SELECT emp_no FROM employees`)
if err := row.Scan(&v.EmpNo); err != nil {
	panic(err)
}
fmt.Println(v.EmpNo)

运行结果：
panic: sql: no rows in result set

image

What happened:
表中有值，也给客户端返回结果了，但是客户端没解析出来

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Fixes project file License header format.

Now, the project file License header format is:

//
// Licensed to Apache Software Foundation (ASF) under one or more contributor
// license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright
// ownership. Apache Software Foundation (ASF) licenses this file to you under
// the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.
//

we want to fix it:

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

Feature：etcd as configuration center

将etcd 作为配置中心:
配置获取策略：
默认拉取etcd中的数据，当etcd 中不存在数据时，将配置文件的数据同步到etcd当中。
已实现功能：
etcd作为配置存储，更新，初始化配置文件到etcd 中。
还需要实现功能点：
存储到etcd中的key value 键值对的粒度根据情况再划分小一些；
接入etcd 的watch 机制 和加入对应的 key value 的version 进行相关版本匹配；
将此方案的配置替换现有实现中的配置加载策略

Simple sequence implementation.

What would you like to be added:
Generate pk when executing INSERT:

  1. register entrance
  2. impls: snowflake/sequence table based/...

Why is this needed:

启动加载yaml配置文件，role、type字段支持类型判断 

func (t *DataSourceType) UnmarshalText(text []byte) error {
	if t == nil {
		return errors.New("can't unmarshal a nil *DataSourceRole")
	}
	if t.unmarshalText(bytes.ToLower(text)) {
		return fmt.Errorf("unrecognized protocal type: %q", text)
	}
	return nil
}

What happened:
支持的角色类型、协议类型，返回了error的错误
What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

if t.unmarshalText(bytes.ToLower(text)) {
		return fmt.Errorf("unrecognized protocal type: %q", text)
	}

改为

if !t.unmarshalText(bytes.ToLower(text)) {
		return fmt.Errorf("unrecognized protocal type: %q", text)
	}

Anything else we need to know?:

Enhance observability

What would you like to be added:

  • Use Prometheus go client to add more metrics.
  • Use OpenTelemetry to add more trace.
  • Add config.

Use Prometheus client-go and OpenTelemetry is because they are the most used now. :) Correct me if I am wrong, or if you have better choice, just comment.

Why is this needed:
Observability is very important for basic software architecture.There are three sections in this field.
image

跑集成测试报错了

What happened:

当执行集成测试查询时，aranan会报错：
mysql_server caught panic:
trying to call recycleReadPacket while currentEphemeralPolicy is 0

image

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):
启动main函数，然后执行 test 包下面的 integration_test.go 文件的 TestSelect方法，即可复现问题

Anything else we need to know?:

Support simple DELETE operation.

What would you like to be added:
implement simple DELETE statement, for example: delete from student where uid = 1 limit 1.

Why is this needed:
It is a basic feature.

When execute "select * from student" mysql/server.go:161 mysql_server caught panic:

What happened:

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Start server refer this PR comment: #90 .

execute:select * from student.

2022-03-25T11:43:14.767+0800    DEBUG   executor/redirect.go:135        ComQuery: select connection_id()
2022-03-25T11:43:14.773+0800    DEBUG   executor/redirect.go:135        ComQuery: select * from student
2022-03-25T11:43:14.773+0800    DEBUG   optimize/optimizer.go:149       compute shards: result=["*"], isFullScan=false
2022-03-25T11:43:14.773+0800    ERROR   mysql/server.go:161     mysql_server caught panic:
implement me
github.com/arana-db/arana/pkg/mysql.(*Listener).handle.func1
        /Users/dongzonglei/source_code/Github/arana/pkg/mysql/server.go:161
runtime.gopanic
        /usr/local/go/src/runtime/panic.go:965
github.com/arana-db/arana/pkg/runtime/plan.UnionPlan.ExecIn
        /Users/dongzonglei/source_code/Github/arana/pkg/runtime/plan/union.go:39
github.com/arana-db/arana/pkg/runtime.(*defaultRuntime).Execute
        /Users/dongzonglei/source_code/Github/arana/pkg/runtime/runtime.go:545
github.com/arana-db/arana/pkg/executor.(*RedirectExecutor).ExecutorComQuery
        /Users/dongzonglei/source_code/Github/arana/pkg/executor/redirect.go:180
github.com/arana-db/arana/pkg/mysql.(*Listener).ExecuteCommand.func1
        /Users/dongzonglei/source_code/Github/arana/pkg/mysql/server.go:558
github.com/arana-db/arana/pkg/mysql.(*Listener).ExecuteCommand
        /Users/dongzonglei/source_code/Github/arana/pkg/mysql/server.go:589
github.com/arana-db/arana/pkg/mysql.(*Listener).handle
        /Users/dongzonglei/source_code/Github/arana/pkg/mysql/server.go:206
2022-03-25T11:43:14.775+0800    DEBUG   executor/redirect.go:135        ComQuery: select connection_id()

Cannot connect with mycli

Cannot connect the arana server over mycli

The mycli is a very popular command-line tool, we should support it.

What happened:

  1. start server
  2. connect with mycli -h127.0.0.1 -udksl employees -p123456 -P13306

What you expected to happen: Should connect successfully

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.