moov-io / base Goto Github PK
View Code? Open in Web Editor NEWcore libraries used in Moov projects
License: Apache License 2.0
core libraries used in Moov projects
License: Apache License 2.0
The Context
interface and related methods should accept an interface{}
over a string
.
type Context interface {
Context() map[string]string // -> map[string]interface{}
}
since writeValue will be called by go-kit/log
to properly encode the string.
Expose the bank time functionality as a utility as part of our rest endpoint.
cc: @alovak
The database package currently supports running a different set of migrations based on the database specified in the config (sqlite or mysql currently). We should support something similar, but have one set of migrations for prod and another set for staging. You can see the sqlite vs mysql migrations at work here:
Line 20 in b5da069
For development, it's really helpful to inspect the database tables after a test run.
There will be a need to re-use Parse Error / ErrorList for moov-io api's, packages, etc.
Copy ParseError / ErrorList from moov-io / ach into base library for re-use.
Put in root of github.com/moov-io/base and the it can be imported from other packages: base.ErrorList
PayGate Version: ``
What were you trying to do?
use database.InMemorySqliteConfig with database.NewAndMigrate
What did you expect to see?
connection to database created and database migrated
What did you see?
err:"no such table: items"
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates are pending. To force PRs open, click the checkbox below.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
docker-compose.yml
mysql 9-oracle
mysql/Dockerfile
mysql 9.0-oracle
.github/workflows/cgo.yml
actions/setup-go v4
actions/checkout v2
.github/workflows/codeql.yaml
actions/checkout v2
github/codeql-action v2
github/codeql-action v2
.github/workflows/mysql-volumeless.yml
actions/checkout v2
.github/workflows/nocgo.yml
actions/setup-go v4
actions/checkout v2
go.mod
go 1.21
cloud.google.com/go/spanner v1.65.0
github.com/go-kit/kit v0.13.0
github.com/go-kit/log v0.2.1
github.com/go-sql-driver/mysql v1.8.1
github.com/golang-migrate/migrate/v4 v4.17.1
github.com/google/uuid v1.6.0
github.com/googleapis/gax-go/v2 v2.13.0
github.com/googleapis/go-sql-spanner v1.6.0
github.com/gorilla/mux v1.8.1
github.com/madflojo/testcerts v1.2.0
github.com/markbates/pkger v0.17.1
github.com/mitchellh/mapstructure v1.5.0
github.com/prometheus/client_golang v1.19.1
github.com/rickar/cal/v2 v2.1.17
github.com/spf13/viper v1.19.0
github.com/stretchr/testify v1.9.0
go.opentelemetry.io/otel v1.28.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.28.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.28.0
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.28.0
go.opentelemetry.io/otel/sdk v1.28.0
go.opentelemetry.io/otel/trace v1.28.0
google.golang.org/grpc v1.65.0
Over in moov-io/ach#298 (and slack) we decided to have timestamp values in our API be ISO 8601. This is a common format usable by lots of languages, but Go doesn't seem to have support for it.
There are a few stale implementations laying around Github. Let's fine one, dust it off and incorporate it into our base
repository.
Here's the structure: moov-io/ach#298 (comment)
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.
We've got a fairly good sqlite setup working for our apps and consolidating that would help keep apps uniform. I'm proposing a few new packages.
github.com/moov-io/base/database
// migrate runs our database migrations (defined at the top of this file)
// over a sqlite database it creates first.
// To configure where on disk the sqlite db is set SQLITE_DB_PATH.
//
// You use db like any other database/sql driver.
//
// https://github.com/mattn/go-sqlite3/blob/master/_example/simple/simple.go
// https://astaxie.gitbooks.io/build-web-application-with-golang/en/05.3.html
func Migrate(db *sql.DB, logger log.Logger) error {
logger.Log("sqlite", "starting sqlite migrations...")
for i := range migrations {
row := migrations[i]
res, err := db.Exec(row)
if err != nil {
return fmt.Errorf("migration #%d [%s...] had problem: %v", i, row[:40], err)
}
n, err := res.RowsAffected()
if err == nil {
logger.Log("sqlite", fmt.Sprintf("migration #%d [%s...] changed %d rows", i, row[:40], n))
}
}
logger.Log("sqlite", "finished migrations")
return nil
}
Perhaps one day we'll adopt flyway or a version table. For now this works.
github.com/moov-io/base/database/sqlite
func DBPath() string {
path := os.Getenv("SQLITE_DB_PATH")
if path == "" || strings.Contains(path, "..") {
// set default if empty or trying to escape
// don't filepath.ABS to avoid full-fs reads
path = "paygate.db"
}
return path
}
func Connection(path string) (*sql.DB, error) {
db, err := sql.Open("sqlite3", path)
if err != nil {
err = fmt.Errorf("problem opening sqlite3 file: %v", err)
logger.Log("sqlite", err)
return nil, err
}
return db, nil
}
func Version() string {
sqliteVersion, _, _ := sqlite3.Version()
return sqliteVersion
}
github.com/moov-io/base/database/sqlite/testsqlite
type TestSqliteDB struct {
db *sql.DB
dir string // temp dir created for sqlite files
}
func (r *TestSqliteDB) Close() error {
if err := r.db.Close(); err != nil {
return err
}
return os.RemoveAll(r.dir)
}
// Connection returns a TestSqliteDB which can be used in tests
// as a clean sqlite database. All migrations are ran on the db before.
//
// Callers should call close on the returned *TestSqliteDB.
func Connection(migrator func (db *sql.DB, logger log.Logger) error) (*TestSqliteDB, error) {
dir, err := ioutil.TempDir("", "test-sqlite")
if err != nil {
return nil, err
}
db, err := createSqliteConnection(filepath.Join(dir, "database.db"))
if err != nil {
return nil, err
}
logger := log.NewLogfmtLogger(ioutil.Discard)
if err := migrator(db, logger); err != nil {
return nil, err
}
return &TestSqliteDB{db, dir}, nil
}
We're starting to see a fair amount of drift between http.Server
's used in Moov. This is intentional as overtime we've adapted servers, but we should consolidate somewhat. There's a lot of re-used features that should be in a common library.
X-Request-Id
/ X-Idempotency-Key
/ X-User-Id
extractionGET /ping
routeruntime.Caller(n)
here too.http.ResponseWriter
)Refactor CheckRoutingNumber validator from ach to base for reuse in wire, and icl.
calculateCheckDigit (checksum) and roundUp10 would need to be refactored in as well.
It's not clear how base/log
package should be used without looking at the implementation.
There should be a README
to guide how users should use the logger.
From the X-Idempotency-Key
issue (moov-io/paygate#15 (comment)) we want to implement a redis implementation of idempotent.Recorder
.
There's https://github.com/rafaeljusto/redigomock which looks like it works with https://github.com/gomodule/redigo. It's GPL'd though.
@wadearnold That project is GPL v2, can we use that? Probably not?
https://github.com/rafaeljusto/redigomock/blob/master/LICENSE
See: moov-io/paygate#21
Replace megacheck with staticcheck
Our server should be able to handle unexpected panics by:
500
We should add a timestamp to all logs using
logger = log.With(logger, "ts", log.DefaultTimestampUTC)
logger.Log("msg", "hello")
// Output:
// ts=2016-01-01T12:34:56Z msg=hello
The existing implementation of ID() string
only creates random hex strings. There's nothing to ensure uniqueness. We should migrate to something which does.
Currently our stack cares these strings are 40 characters long and of hex strings to strip them from Prometheus paths. (e.g. GET /users/fffffffffffffff/accounts
to get-users-accounts
)
Candidates
We can do
logger.Set("orgIDs", log.String(fmt.Sprintf("[%s]",strings.Join(organizationIDs, ","))))
but a string array method would be helpful:
logger.Set("orgIDs", log.Strings(organizationIDs))
Both would output orgIDs=["id-1", "id-2", "id-3"]
Type signature for the valuer would be:
func Strings(vals []string) Valuer
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.