goadesign / gorma Goto Github PK
View Code? Open in Web Editor NEWStorage generation plugin for Goa
Home Page: http://goa.design
License: MIT License
Storage generation plugin for Goa
Home Page: http://goa.design
License: MIT License
We have a few validations now to verify the consistency of the design, more are needed.
I'll submit a PR to fix this issue, but I wanted to explain the rationale here first.
I have the following construct in my controllers:
// EmployeeController implements the employee resource.
type EmployeeController struct {
goa.Controller
storage models.EmployeeStorage
}
// NewEmployeeController creates a employee controller.
func NewEmployeeController(service goa.Service, db gorm.DB) app.EmployeeController {
return &EmployeeController{
Controller: service.NewController("EmployeeController"),
storage: models.NewEmployeeDB(db),
}
}
This construct with gorma works really well except when I have some convoluted flaming rings I have to jump through to answer a request. In those situations, I need access to the gorm.DB
object, but the underlying {{typeName}}DB
object is hidden behind the {{typeName}}Storage
interface. For example:
type EmployeeStorage interface {
List(ctx context.Context, tableName string) []Employee
One(ctx context.Context, tableName string, id int) (Employee, error)
Add(ctx context.Context, tableName string, o Employee) (Employee, error)
Update(ctx context.Context, tableName string, o Employee) error
Delete(ctx context.Context, tableName string, id int) error
}
type EmployeeDB struct {
DB gorm.DB
}
The only way around this issue right now is to create a separate {{typeName}}DB
instance in the controller.
We're expecting in one of our payloads and object that has basic attributes and array of objects in it. . However, I get these errors when we use gorma.
**models/clc.go:178: cannot use payload.Containers (type string) as type []Container in assignment
models/clc.go:203: cannot use payload.Containers (type string) as type []Container in assignment
This is how the payload is defined
`var ClcPayload = Type("ClcPayload", func() {
Attribute("name", String, func() {
MinLength(1)
MaxLength(1024)
})
Attribute("labels", func(){
ArrayOf(LabelPayload)
})
Attribute("description", String, func(){
MinLength(0)
MaxLength(1024)
})
Attribute("replicas", Integer, func(){
Minimum(1)
Maximum(10)
})
Attribute("containers", func(){
ArrayOf(ContainerPayload)
})
})`
var ContainerPayload = Type("ContainerPayload", func() {
Attribute("name", String, func() {
MinLength(1)
MaxLength(1024)
})
Attribute("labels", func(){
ArrayOf(LabelPayload)
})
Attribute("description", String, func(){
MinLength(0)
MaxLength(1024)
})
Attribute("image_url", String)
Attribute("ports", func(){
ArrayOf(PortPayload)
})
Attribute("envs", func(){
ArrayOf(EnvPayload)
})
Attribute("commands", func(){
ArrayOf(String)
})
Attribute("icus", Integer)
Attribute("memory", Integer)
Attribute("replicas", Integer, func(){
Minimum(1)
Maximum(10)
})
//Attribute("volumes", ArrayOf(VolumePayload))
})`
This is the definition of our data model
Model("Clc", func() {
BuildsFrom(func() {
Payload("clc", "create")
Payload("clc", "update")
})
RendersTo(Clc)
HasMany("Containers", "Container")
Description("Colocated Container Model Description")
Field("id", gorma.UUID, func() {
PrimaryKey()
Description("This is the Clc Model PK field")
})
BelongsTo("Service")
})
remove "like" for ID
remove "like" for bools
remove "equal" for ID
Bryce Reitano It seems the gorma doesn’t work with timestamps with the mysql backend. At least on my machine, I haven’t dug into it to provide more information in an issue.
[4:42 PM]
Derek Perkins that’s probably a driver issue
[4:43 PM]
user:password@tcp(mysql:3306)/?parseTime=true
(edited)
[4:43 PM]
with the go-mysql-driver
, you have to specify it in the connection string
[4:44 PM]
Bryce Reitano Ah, that makes sense. Disregard then, I’m not used to working with mysql in any fashion and been spoiled by Postgres. :simple_smile:
[6:52 PM]
Brian Ketelsen Maybe we should document that somewhere
[6:52 PM]
Note to self
Fix gorma to render the proper links for each view.
Calling generated method Update
with a non-existent ID will result in Update
's call to Get
returning nil, nil
. This means the Update
method's error check passes and it subsequently passes a zero value to Gorm, resulting in a panic: reflect: call of reflect.Value.MethodByName on zero Value
Get
should surface Gorm's ErrRecordNotFound
in the same way that the model helper OneFoo
does. That means altering the generated code here to return the original error instead of resetting it to nil: https://github.com/goadesign/gorma/blob/master/writers.go#L474
From a slack chat with @bketelsen
gorma doesn’t make you enter all the fields
RendersTo() and BuildsFrom() read the fields in the payload and media types
and add those to your model
but if you already have an “id” field, maybe gorma doesn’t mark it as a PK
I can see how this would happen
BuildsFrom() is the only one that adds fields
RendersTo() causes conversion funcs to be created, but doesn’t add fields
so if there’s no “id” in your Payload you wouldn’t get an id in the model, and you’[d have to add it manually
it’s a bug
Ensure that all the generated code respects MapsTo and MapsFrom
It looks like gorma’s generated delete method doesn’t take into account any parent relationships. There should be a Delete helper method which can also take parent identifier(s) so that delete operations can be scoped to parents who own it.
I got the following usecase:
I got books and Shelfs, Books can be assigned to Shelfs.
There is the case a book isnt assigned to a shelf and there for the shelf_id should be nil
I tried the Nullable command on a Field, but does not work.
Model("Shelf", func() {
Field("id", gorma.Integer, func() {
PrimaryKey()
})
HasMany("Books", "Book")
)}
Model("Book", func() {
Field("id", gorma.Integer, func() {
PrimaryKey()
})
//Hack not working
Field("shelf_id", gorma.Integer, func() {
Nullable()
})
})
This will generate a model like:
type Book struct {
ID int `gorm:"primary_key"` // primary key
ShelfID int // has many Book
}
type Shelf struct {
ID int `gorm:"primary_key"` // primary key
Books []Book // has many Books
}
But what i actualy want is (note the pointer to *int)
type Book struct {
ID int `gorm:"primary_key"` // primary key
ShelfID *int // has many Book
}
How can i achieve this? or is this even possible.
Generate a constant for every Resource & Action:
USERCREATE = "usercreate"
for role based authorization.
Supplier Model
-- HasOne(Account)
Should not modify supplier model, but should create a supplier_id field in the Accounts table
Helpers should be modified to do this
UUID (driver specific or with a string and some function callbacks?)
String
Others?
Gorma's automatic insertion of a primary-key field named "ID" is an excellent way to DRY out my model definitions:
https://github.com/goadesign/gorma/blob/master/dsl/relationalmodel.go#L43
Alas, although my models have a field definition for ID, the generated Storage interface is lacking an "ID" parameter, making the auto-ID a bit useless unless I evade the storage layer and interact with gorm directly.
According to @bketelsen:
in a refactor that code never gets called in the order that would make it useful.
I would be very appreciative of a PR that solves this
I'm opening this issue to promote awareness/understanding in case anyone else runs into the problem, and perhaps eventually to contribute a fix.
See slack conversation.
gorma.Decimal data type generates float data type, not float32.
I think it should be generating float32.
For the gorma.BigDecimal, it should be float64.
add tests, make any changes necessary.
Sending a record with only one of many fields set updates the rest of the fields to nil.
Types that aren't required now are created as pointers in Goa. Fix Gorma to follow suit.
Make them constants for ease of testing.
When Gorm interprets the name of a column w/ multiple words, it will do lower snake case for them:
type AwesomeCandy struct {
ID int
CandyTypeID int
}
In this case, Gorm will expect to deal with a column called candy_type_id
.
However, creating a Gorma model with a multi-word association of the form BelongsTo("CandyType")
will generate helper methods with incorrect designation for the foreign key in the Where
call:
func AwesomeCandyFilterByCandyType(candyTypeID int, originaldb *gorm.DB) func(db *gorm.DB) *gorm.DB {
if candyTypeID > 0 {
return func(db *gorm.DB) *gorm.DB {
return db.Where("candytype_id = ?", candyTypeID)
}
}
return func(db *gorm.DB) *gorm.DB { return db }
}
The generated call to Where
should contain a foreign key column matching what Gorm produces, in this case it should be candy_type_id
I propose to simply copy *RelationalFieldDefinition
's Underscore
method to *RelationalModelDefinition
and then use it in the function definition in Writers.go
For example:
func (m *Organization) OrganizationToAppOrganization() *app.Organization {
organization := &app.Organization{}
organization.TimeZone = &m.TimeZone
organization.Name = &m.Name
organization.ID = &m.ID
return organization
}
The order of TimeZone
, Name
, and ID
assignment lines is randomized with each generation.
Currently, using Alias
in the Model
context doesn't do anything but was meant to do the same thing that TableName
does.
Move the TableName
functionality to Alias
for the Model context.
Given:
Model("Identity", func() {
Description("")
Field("id", gorma.UUID, func() {
PrimaryKey()
SQLTag("type:uuid")
Description("This is the ID PK field")
})
Field("full_name", gorma.String, func() {
Description("The fullname of the Identity")
Alias("fullName")
})
Field("image_url", gorma.String, func() {
Description("The image URL for this Identity")
Alias("imageURL")
})
})
Model("User", func() {
Description("")
Field("id", gorma.UUID, func() {
PrimaryKey()
SQLTag("type:uuid")
Description("This is the ID PK field")
})
Field("email", gorma.String, func() {
Description("This is the unique email field")
})
BelongsTo("Identity")
})
Expected:
type Identity struct {
ID uuid.UUID `sql:"type:uuid" gorm:"primary_key"` // This is the ID PK field
...
FullName string `gorm:"column:fullName"` // The fullname of the Identity
ImageURL string `gorm:"column:imageURL"` // The image URL for this Identity
}
type User struct {
ID uuid.UUID `sql:"type:uuid" gorm:"primary_key"` // This is the ID PK field
...
IdentityID uuid.UUID // Belongs To Identity
Identity Identity
}
But got:
type User struct {
ID uuid.UUID `sql:"type:uuid" gorm:"primary_key"` // This is the ID PK field
...
IdentityID int // Belongs To Identity // INT ??
Identity Identity
}
use goa's logging pref
Build fails due to removed versioning support in goa (772bca5)
See: goa (772bca5)
Failed to compile gorma:
generator.go:120: undefined: design.APIVersionDefinition
generator.go:123: undefined: codegen.VersionPackage
generator.go:131: api.IterateVersions undefined (type *design.APIDefinition has no field or method IterateVersions)
generator.go:131: undefined: design.APIVersionDefinition
generator.go:199: api.IterateVersions undefined (type *design.APIDefinition has no field or method IterateVersions)
generator.go:199: undefined: design.APIVersionDefinition
writers.go:328: undefined: codegen.VersionPackage
writers.go:356: undefined: codegen.VersionPackage
Ginkgo ran 1 suite in 2.612667464s
Test Suite Failed
make: *** [test] Error 1
With a recent update on this PR, Gorma is generating relational functions with errors. For example, lets say we Surveys
and those have many SurveyItems
; SurveyItems
belonging to a Survey
. When generating code, we will get a function like
// ListSurveyItem returns an array of view: default.
func (m *SurveyItemDB) ListSurveyItem(ctx context.Context, surveyID int) []*app.SurveyItem {
defer goa.MeasureSince([]string{"goa", "db", "surveyItem", "listsurveyItem"}, time.Now())
var native []*SurveyItem
var objs []*app.SurveyItem
err := m.Db.Scopes(SurveyItemFilterBySurvey(surveyID, &m.Db)).Table(m.TableName()).Find(&native).Error
if err != nil {
goa.LogError(ctx, "error listing SurveyItem", "error", err.Error())
return objs
}
for _, t := range native {
objs = append(objs, t.SurveyItemToSurveyItem())
}
return objs
}
Where errors are thrown as such:
models/surveyitem_helper.go:31: cannot use &m.Db (type **gorm.DB) as type *gorm.DB in argument to SurveyItemFilterBySurvey
models/surveyitem_helper.go:67: cannot use &m.Db (type **gorm.DB) as type *gorm.DB in argument to SurveyItemFilterBySurvey
models/surveyitem_helper.go:86: cannot use &m.Db (type **gorm.DB) as type *gorm.DB in argument to SurveyItemFilterBySurvey
models/surveyitem_helper.go:113: cannot use &m.Db (type **gorm.DB) as type *gorm.DB in argument to SurveyItemFilterBySurvey
With the update the passing by reference needs to be removed.
Seems like it would be easier to read and understand tags if they used underscores or camelcase for their names. I feel this would make it easier for new users to understand and remember the tags and what they do. Instead of looking like a non-word, it will obviously be a group of words put together, which makes it easier to remember. For example:
Metadata("github.com/bketelsen/gorma#dyntablename", "true")
becomes
Metadata("github.com/bketelsen/gorma#dyn_table_name", "true")
or
Metadata("github.com/bketelsen/gorma#dynTableName", "true")
If using camelcase, nothing would have to change except for a ToLower
call on the tag itself before processing.
Thoughts?
I couldn't find it in the documentation but having support for resource expansion is the biggest feature I'd be looking for. https://stormpath.com/blog/linking-and-resource-expansion-rest-api-tips
If a Type Attribute has Format("date-time")
defined, store a *time.Time
field in the Model struct.
PR was submitted making breaking changes when using the UUID type within Gorma and Goa.
When I was creating my resource, inside my user.go
I get the following:
cannot use m.ID (type "github.com/satori/go.uuid".UUID) as type "github.com/goadesign/goa/uuid".UUID in assignment
To* should render the RenderTo model(s)
From* should render the BuiltFrom model(s)
X m2m Y
helper generated XtoX() doesn't include Y, only includes belongsto/hasmany tables.
var _ = MediaType("application/vnd.test+json", func() {
Attributes(func() {
Attribute("points", ArrayOf(Integer)),
})
})
Current result:
goagen --design github.com/moorereason/myapp/design app
exit status 1
[../../../github.com/goadesign/gorma/relationalmodel.go:289] Unsupported type: 0x7 array in unnamed BuildSource
[../../../github.com/goadesign/gorma/relationalmodel.go:257] Unsupported type: 0x7 array in unnamed BuildSource
[../../../github.com/goadesign/gorma/relationalmodel.go:289] Unsupported type: 0x7 array in unnamed BuildSource
make: *** [genapp] Error 1
For reference: relationalmodel.go:257
Iterate actions so we don't generate code for actions that don't exist
Some issues that I found peculiar, which slightly detracted from gorma's awesome usability:
List()
methods don't return an error (they log it) whereas other methods do
Whereas gorm composition methods return a *gorm.DB
, gorma data-access objects always want a gorm.DB
(and internally store a DB
). This requires me to add a lot of extra *
when building queries. Granted, there is no performance or safety difference between using DB
and *DB
; still, it seems like gorma should be consistent with the framework it wraps.
As discussed in #goa @ gophers.slack.com, these both seem worthy of fixing and I hope to whip up a PR sometime soon.
I have a use case that requires a non-built-in type to read geo data from a database and output it into GeoJSON, but this extends to any user type. I think that support can be restricted to custom types that match one of the following.
sql.Valuer
AND sql.Scanner
https://godoc.org/github.com/paulmach/go.geo#Path
CREATE TABLE roads ( id int4, geom geometry(LINESTRING,4326), name varchar(25) );
BEGIN;
INSERT INTO roads (id, geom, name)
VALUES (1,ST_GeomFromText('LINESTRING(191232 243118,191108 243242)',-1),'Jeff Rd');
INSERT INTO roads (id, geom, name)
VALUES (2,ST_GeomFromText('LINESTRING(189141 244158,189265 244817)',-1),'Geordie Rd');
INSERT INTO roads (id, geom, name)
VALUES (3,ST_GeomFromText('LINESTRING(192783 228138,192612 229814)',-1),'Paul St');
INSERT INTO roads (id, geom, name)
VALUES (4,ST_GeomFromText('LINESTRING(189412 252431,189631 259122)',-1),'Graeme Ave');
INSERT INTO roads (id, geom, name)
VALUES (5,ST_GeomFromText('LINESTRING(190131 224148,190871 228134)',-1),'Phil Tce');
INSERT INTO roads (id, geom, name)
VALUES (6,ST_GeomFromText('LINESTRING(198231 263418,198213 268322)',-1),'Dave Cres');
COMMIT;
type Road struct {
ID int
Name *string
Geom *geo.Path // import "github.com/paulmach/go.geo"
}
When creating a new model, Gorma will automatically create an ID
field on your model struct and set a gorm:"primary_key"
struct tag. This is great.
What is confusing however is that the generated methods for retrieving a single item from the database won't take an ID parameter without an additional Field
declaration in your model for a primary key:
Field("id", gorma.Integer, func() {
PrimaryKey()
}
Without the above, the Gorm Where
method in both the generated OneFoo
model helper and the Get
model method only have an empty string as an argument, rendering them effectively useless.
make gorma smarter about mixing camel & snake case in the 'goa' side of the design.
choose a provider?
use the --app flag instead
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.