Comments (4)
I try this issue now, it is key point how replace from pq.CopyIn
I think it can replace into LOAD DATA LOCAL INFILE...
e.g.
func CopyInMySQL(table string, columns ...string) string {
stmt := "LOAD DATA LOCAL INFILE ‘/dev/stdin’ " + QuoteIdentifier(table) + " ("
for i, col := range columns {
if i != 0 {
stmt += ", "
}
stmt += quoteIdentifier(col)
}
stmt += ") TERMINATED BY ‘\n’ ENCLOSED BY ‘\“‘ ESCAPED BY ‘\“‘"
return stmt
func quoteIdentifier(name string) string {
end := strings.IndexRune(name, 0)
if end > -1 {
name = name[:end]
}
return `"` + strings.Replace(name, `"`, `""`, -1) + `"`
}
https://dev.mysql.com/doc/refman/5.6/ja/load-data.html
from skogul.
@u5surf I think you can get away with "INSERT INTO" within a transaction also. I did that with postgres in an earlier perl-based PoC and the performance was OK enough.
from skogul.
I've added a MySQL sender now that will most likely be renamed and made to support multiple databases. It's worth noting that this new sender is NOT in direct contest with the one relevant to this issue. The one that now exists only handles specific schemas/metrics, which is an important use case, but entirely distinct from this issue that deals with arbitrary input data.
from skogul.
Closing this as the actual sender works.
from skogul.
Related Issues (20)
- Prometheus parser
- USP protobuf support HOT 1
- General protocol buffer approach
- Add negative config tests
- Add SQLite driver to sql sender
- Prometheus endpoint for internal stats HOT 4
- RabbitMQ receiver
- RabbitMQ Receiver - delay mechanism when send fails HOT 1
- Systematically go over all existing config-related unit tests and move them to testdata
- Add config-file tests for all modules with at least one test per module
- Migrate docker image building to Github Actions HOT 2
- Migrate automated release to Github Actions HOT 1
- Introduce run-time loaded modules
- WholeFile receiver differs in behavior from other file-receivers
- File writer/sender doesn't fsync
- How Do I Rename Metric Fields? HOT 1
- Influx sender will send invalid, empty tags HOT 1
- Juniper Telemetry parse failed HOT 4
- Debug mode
- Transformers are not visible in -show because there isn't an unmarshal for them
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from skogul.