imdrasil / jennifer.cr Goto Github PK
View Code? Open in Web Editor NEWCrystal ORM using ActiveRecord pattern with flexible query DSL
License: MIT License
Crystal ORM using ActiveRecord pattern with flexible query DSL
License: MIT License
Fix #set_attribute
method to mark given attribute as changed
Add measurement how much time request to db takes. Result should be added to log text. Only request time (w/o object building or packet reading process) should be measured.
Hi!
Since I've got bigint
for id
column in users
table
Im trying to use User
model like that:
class User < ApplicationRecord
with_timestamps
mapping(
id: Primary64, # same as {type: Int32, primary: true}
email: {type: String?},
created_at: {type: Time, null: false, default: Time.now},
updated_at: {type: Time, null: false, default: Time.now}
)
end
Then
When launch the app
Im getting errors like that:
expanding macro
in macro '__field_declaration' /Users/admin/prj/crystal/amber_backend/lib/jennifer/src/jennifer/model/mapping.cr:18, line 53:
...
50. # Inits primary field
51. def init_primary_field(value)
52. raise ::Jennifer::AlreadyInitialized.new(@id, value) if @id
> 53. @id = value.as(Primary64)
54. end
55.
...
can't cast Int32 to Int64
03:48:16 Watcher | (INFO) Compile time errors detected. Shutting down...
Whats wrong with it and why Im not able to use Primary64
in this case? 🤔
When I was using Primary32
I got:
2018-06-15 15:49:52 +03:00: TRANSACTION START
2018-06-15 15:49:52 +03:00: 594 µs INSERT INTO users(email, created_at, updated_at) VALUES ($1, $2, $3) RETURNING id | ["[email protected]", 2018-06-15 12:49:52 UTC, 2018-06-15 12:49:52 UTC]
2018-06-15 15:49:52 +03:00: TRANSACTION ROLLBACK
ERROR: cast from Int64 to Int32 failed
UPD
integer
works perfectly with Int32
mapping type 🤔
\d+ users
Table "public.users"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
------------+-----------------------------+-----------+----------+---------+---------+--------------+-------------
created_at | timestamp without time zone | | | | plain | |
updated_at | timestamp without time zone | | | | plain | |
id | integer | | | | plain | |
but not for bigint
\d+ users
Table "public.users"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------------+-----------------------------+-----------+----------+-----------------------------------+----------+--------------+-------------
id | bigint | | not null | nextval('users_id_seq'::regclass) | plain | |
shard.yml dependencies
dependencies:
kemal:
github: kemalcr/kemal
jennifer:
github: imdrasil/jennifer.cr
pg:
github: will/crystal-pg
sam:
github: imdrasil/sam.cr
sam.cr
require "jennifer/adapter/postgres"
require "jennifer"
require "sam"
Jennifer::Config.read("./api2/config/database.yaml", :development )
load_dependencies "jennifer"
Sam.help
database.yaml
defaults : &defaults
host: localhost
adapter: postgres
user: postgres
password: password
migration_files_path: ./../db/migrations
development:
db: api2_develop
<<: *defaults
test:
db: api2_test
<<: *defaults
production:
db: api2_production
<<: *defaults
When running crystal src/sam.cr -- db:create
Error in src/sam.cr:2: while requiring "jennifer"
require "jennifer"
^
in lib/jennifer/src/jennifer.cr:20: while requiring "./jennifer/query_builder/logic_operator"
require "./jennifer/query_builder/logic_operator"
^
in lib/jennifer/src/jennifer/query_builder/logic_operator.cr:6: superclass mismatch for class Jennifer::QueryBuilder::Criteria (Jennifer::QueryBuilder::SQLNode for Reference)
class Criteria < SQLNode
Hi @imdrasil I'm trying to create a Jennifer Recipe and I just realze there is something wrong with tag v0.5.1
Can you re-release/re-publish/re-create tag v0.5.1 or v0.5.2 because it doesn't include v0.5.1 on shard.yml, see:
https://github.com/imdrasil/jennifer.cr/blob/v0.5.1/shard.yml#L2
This causes problems with shards install
because dependencies will be always broken: (notice at
)
...
Installing jennifer (0.5.0 at 0.5.1)
Installing accord (1.0.0 at 1.1.0)
Using ifrit (0.1.2)
Installing time_zone (0.1.0 at 0.1.1)
...
(Other shards may have same issue as well)
Is adding support for INSERT ... ON CONFICT ...
something being considered? I can maybe take a stab at it at some point if it's a wanted feature
#none
methods allows to continue chaining query methods but at the end will just return empty array without hitting db.
Add storing all important information (class, default value, etc.) to metadata container. This will extend dynamical functionality of library.
I'm just getting started with jennifer.cr
and was trying to get my database up and running for my application. After running db:create
, I was ready to create a migration, so I looked at the README for instructions. It says to run
$ crystal sam.cr -- migration:generate your_migration_name
but doing so returns a "Task migration:generate was not found" exception.
Looking at the source, it looks like the task name is actually generate:migration
, and running that command seems to work:
$ crystal sam.cr -- generate:migration create_users
Migration CreateUsers20170512222213819 was generated
$
Where is new definition called from this method?
jennifer.cr/src/jennifer/model/base.cr
Line 97 in b9194fe
Getting this error when I try to build an empty object: (Company is my Model)
in lib/jennifer/src/jennifer/model/base.cr:97: wrong number of arguments for 'Company.new' (given 0, expected 1..2)
Overloads are:
- Company.new(values : Hash | NamedTuple, new_record)
- Company.new(__temp_88 : DB::ResultSet)
- Company.new(values : Hash(Symbol, ::Jennifer::DBAny) | NamedTuple)
- Company.new(values : Hash(String, ::Jennifer::DBAny))
The very first line in app.cr
is require "jennifer"
where build fails with the following:
> crystal build src/app.cr
Error in src/app.cr:1: while requiring "jennifer"
require "jennifer"
^
in lib/jennifer/src/jennifer.cr:9: while requiring "./jennifer/adapter"
require "./jennifer/adapter"
^
in lib/jennifer/src/jennifer/adapter.cr:1: while requiring "./adapter/base"
require "./adapter/base"
^
in lib/jennifer/src/jennifer/adapter/base.cr:1: while requiring "db": can't find file 'db' relative to '/my/project/dir/lib/jennifer/src/jennifer/adapter'
require "db"
^
In shards.yml
jennifer:
github: imdrasil/jennifer.cr
version: "~> 0.4.3"
I'm new to crystal and to jennifer.cr, so thanks for your patience if I'm missing something obvious.
Getting this error when running my code against the latest version.
require "jennifer"
^
in lib/jennifer/src/jennifer.cr:20: while requiring "./jennifer/model/*"
require "./jennifer/model/*"
^
Syntax error in lib/jennifer/src/jennifer/model/base.cr:155: unexpected token: self (parentheses are mandatory for macro arguments)
macro def self.models
^
I think it's related to this change. crystal-lang/crystal#5043
Hi!
It seems there is an issue when converting result sets from joined queries to Jennifer.cr objects. If relations from records from a result point to the same referenced record, it is correctly assigned only once. All subsequent occurences are nil.
Here's a test case which illustrates the problem:
Data (Postgres):
-- Users
CREATE TABLE test_users (
id bigint NOT NULL,
name character varying(50)
);
INSERT INTO test_users (id, name) VALUES (1, 'Anna');
INSERT INTO test_users (id, name) VALUES (2, 'Bert');
INSERT INTO test_users (id, name) VALUES (3, 'Charly');
-- Posts
CREATE TABLE test_posts (
id bigint NOT NULL,
user_id bigint NOT NULL,
content character varying NOT NULL
);
INSERT INTO test_posts (id, user_id, content) VALUES (1, 1, 'First Post');
INSERT INTO test_posts (id, user_id, content) VALUES (2, 1, 'Second Post');
INSERT INTO test_posts (id, user_id, content) VALUES (3, 2, 'Third Post');
INSERT INTO test_posts (id, user_id, content) VALUES (4, 1, 'Fourth Post');
Models:
class TestUser < Jennifer::Model::Base
mapping(
id: Primary64,
name: String
)
has_many :posts, TestPost, foreign: "user_id"
end
class TestPost < Jennifer::Model::Base
mapping(
id: Primary64,
user_id: Int64,
content: String
)
belongs_to :user, TestUser, foreign: "user_id"
end
First, I load all users with their posts -- everything looks good:
TestUser.all.relation("posts").with("posts").each do |u|
puts "User: #{u.name}"
puts "Posts:"
u.posts.each do |p|
puts "* #{p.content}"
end
puts "\n"
end
# 2018-04-12 19:44:43 +00:00: 854 µs SELECT test_users.*, test_posts.*
# FROM test_users
# LEFT JOIN test_posts ON test_posts.user_id = test_users.id
# User: Anna
# Posts:
# * First Post
# * Second Post
# * Fourth Post
# User: Bert
# Posts:
# * Third Post
# User: Charly
# Posts:
However, when loading all posts with their users TestPost#user
gets nil
, when the user was already referenced by a previous post:
TestPost.all.relation("user").with("user").each do |p|
puts "* #{p.content}"
if p.user
puts "by #{p.user!.name}"
else
puts "BOOOOOOM! Caught nil."
end
puts "\n"
end
# 2018-04-12 19:44:43 +00:00: 487 µs SELECT test_posts.*, test_users.*
# FROM test_posts
# LEFT JOIN test_users ON test_users.id = test_posts.user_id
# * First Post
# by Anna
# * Second Post
# BOOOOOOM! Caught nil.
# * Third Post
# by Bert
# * Fourth Post
# BOOOOOOM! Caught nil.
When I ommit the relation joins and go with just TestPost.all.each
, it works as expected.
I hope that's enough to explain the problem, let me know if I can add any more information.
Show benchmark execution results for:
If you not mention some field app will freeze.
# migration
create_table :some_models do |t|
t.string :f1
t.string :f2
end
# model definition
class SomeModel < Jennifer::Model::Base
mapping(
id: { type: Int32?, primary: true},
f1: String?
)
end
SomeModel.where { some_condition }.to_a
When adding a migration with a index I get a argument error. Running version 0.6.1
t.add_index(name: "nodes_uuid_index", field: :uuid, type: :uniq, order: :asc):
instantiating 'Jennifer::Adapter::SchemaProcessor+#build_create_table(Tuple(Symbol), NamedTuple())'
in db/migrations/20180919214132979_node.cr:3: instantiating 'create_table(Symbol)'
create_table(:nodes) do |t|
^~~~~~~~~~~~
in db/migrations/20180919214132979_node.cr:8: instantiating 'Jennifer::Migration::TableBuilder::CreateTable#add_index()'
t.add_index(name: "nodes_uuid_index", field: :uuid, type: :uniq, order: :asc)
^~~~~~~~~
in lib/jennifer/src/jennifer/migration/table_builder/create_table.cr:59: instantiating 'index(Tuple(), NamedTuple(name: String, field: Symbol, type: Symbol, order: Symbol))'
index(*args, **opts)
^~~~~
in lib/jennifer/src/jennifer/migration/table_builder/create_table.cr:48: instantiating 'index(String, Array(Symbol))'
index(
^~~~~
in lib/jennifer/src/jennifer/migration/table_builder/create_table.cr:43: wrong number of arguments for 'Jennifer::Migration::TableBuilder::CreateIndex.new' (given 6, expected 7)
Overloads are:
- Jennifer::Migration::TableBuilder::CreateIndex.new(adapter, table_name, index_name, fields, type, lengths, orders)
@commands << CreateIndex.new(@name, name, fields, type, lengths, orders)
^~~
t.add_index(name: "nodes_uuid_index", field: :uuid, type: :uniq, order: :asc, nil, nil):
Error in sam.cr:3: while requiring "./db/migrations/*"
require "./db/migrations/*"
^
Syntax error in db/migrations/20180919214132979_node.cr:8: expected named argument, not nil
t.add_index(name: "nodes_uuid_index", field: :uuid, type: :uniq, order: :asc, nil, nil)
^
For some reason whenever BadQuery
is raised I don't see the exception message. The query is valid and works most of the time but raises an error every so often (possibly under load when number of connections is > than connection pool), any ideas or thoughts are appreciated. Here is the code that generates the query
games = Game.where do
sql("start_time >= $1 and start_time < $2 and country = $3", [time.at_beginning_of_day, time.at_end_of_day, country])
end.to_a
Where @time
is a valid Time
object and country
is a valid string
And here is the error, all I see a a .
in the exception
2018-08-28T02:00:25.777683+00:00 app[web.1]: Exception: .
2018-08-28T02:00:25.777703+00:00 app[web.1]: Original query was:
2018-08-28T02:00:25.777706+00:00 app[web.1]: SELECT games.* FROM games WHERE (start_time >= $1 and start_time < $2 and country = $3) | [2018-05-20 04:00:00.0 UTC, 2018-05-21 03:59:59.999999999 UTC, "us"] (Jennifer::BadQuery)
2018-08-28T02:00:25.777708+00:00 app[web.1]: from /tmp/build_39342a2f73a9345162c09702695125ed/lib/jennifer/src/jennifer/query_builder/model_query.cr:66:28 in '*Jennifer::QueryBuilder::ModelQuery(HqBuff::Game)' at 0x8573ee
Thank you! If I manage to reproduce it in a reduced way I will post more details, sorry to be vague :)
EDIT: This is not in --release
mode
UPDATE users SET users.balance = users.balance + 1
WHERE users.id = 1
add row-level lock mechanism
add between operator-function
Hi,
I have the following db structure:
-- Table: links
CREATE TABLE links
(
id serial NOT NULL,
short_url character varying,
ios_url character varying,
web_url character varying,
android_url character varying,
created_at timestamp,
updated_at timestamp,
CONSTRAINT links_pkey PRIMARY KEY (id),
CONSTRAINT links_short_url_key UNIQUE (short_url)
);
-- Index: short_link_idx
CREATE UNIQUE INDEX short_link_idx
ON links
USING btree
(short_url);
And when trying to a create link with short_url
that is already created, request is hanging for ever without any timeout or exception being raised.
That is the last message in output:
...
2017-09-12 01:41:49 +0300: TRANSACTION START
Please have a look.
➔ crystal sam.cr -- generate:migration create_test
Error in sam.cr:4: expanding macro
load_dependencies "jennifer"
^~~~~~~~~~~~~~~~~
in sam.cr:4: expanding macro
load_dependencies "jennifer"
^
in macro 'load_dependencies' /Users/g33kidd/Sandbox/PRL/crival/lib/sam/src/sam.cr:75, line 2:
1.
> 2. require "jennifer/sam"
3.
while requiring "jennifer/sam"
in lib/jennifer/src/jennifer/sam.cr:1: instantiating 'Sam:Module#namespace(String)'
Sam.namespace "db" do
^~~~~~~~~
in lib/jennifer/src/jennifer/sam.cr:1: instantiating 'Sam:Module#namespace(String)'
Sam.namespace "db" do
^~~~~~~~~
in lib/jennifer/src/jennifer/sam.cr:4: instantiating 'Jennifer::Migration::Runner:Module#migrate()'
Jennifer::Migration::Runner.migrate
^~~~~~~
in lib/jennifer/src/jennifer/migration/runner.cr:46: instantiating 'migrate(Int32)'
migrate(-1)
^~~~~~~
in lib/jennifer/src/jennifer/migration/runner.cr:7: No migration defined
return if ::Jennifer::Migration::Base.migrations.empty?
Not really sure what's going on here. I have the database and configuration setup properly (i think.)
app_1 | I, [2017-08-12 15:49:48 +0000 #84] INFO -- : Server started in development.
app_1 | I, [2017-08-12 15:49:48 +0000 #84] INFO -- : Startup Time 00:00:00.0001665
app_1 |
app_1 |
app_1 | 2017-08-12 15:50:14 +0000:
app_1 | SELECT e.enumtypid
app_1 | FROM pg_type t, pg_enum e
app_1 | WHERE t.oid = e.enumtypid
app_1 | 2017-08-12 15:50:14 +0000:
app_1 | SELECT users.*
app_1 | FROM users
app_1 | LIMIT 1
app_1 |
app_1 | Unhandled exception on HTTP::Handler
app_1 | Index out of bounds.
app_1 | Original query was:
app_1 | SELECT users.*
app_1 | FROM users
app_1 | LIMIT 1
app_1 | (Jennifer::BadQuery)
app_1 | 0x545b17: *CallStack::unwind:Array(Pointer(Void)) at ??
app_1 | 0x6dfe29: to_a at /app/lib/jennifer/src/jennifer/query_builder/model_query.cr 77:9
app_1 | 0x6df6e4: first at /app/lib/jennifer/src/jennifer/query_builder/query.cr 146:13
app_1 | 0x6c7856: index at /app/src/controllers/home_controller.cr 3:13
app_1 | 0x6c25cb: call at /app/lib/amber/src/amber/router/route.cr 255:3
app_1 | 0x773546: process_request at /app/lib/amber/src/amber/router/context.cr 58:5
app_1 | 0x53e556: ~procProc(HTTP::Server::Context, String) at /opt/crystal/src/concurrent.cr 29:3
app_1 | 0x6bb984: call_next at /opt/crystal/src/http/server/handler.cr 255:3
app_1 | 0x6baebd: call at /app/lib/amber/src/amber/router/pipe/csrf.cr 15:11
app_1 | 0x6ba9b2: call_next at /opt/crystal/src/http/server/handler.cr 24:7
app_1 | 0x6ba560: call at /app/lib/amber/src/amber/router/pipe/session.cr 10:12
app_1 | 0x6b9f51: call_next at /opt/crystal/src/http/server/handler.cr 24:7
app_1 | 0x6b9bd0: call at /app/lib/amber/src/amber/router/pipe/flash.cr 11:9
app_1 | 0x6b9364: call_next at /opt/crystal/src/http/server/handler.cr 24:7
app_1 | 0x6b8905: call at /app/lib/amber/src/amber/router/pipe/logger.cr 12:9
app_1 | 0x6b65a6: call at /app/lib/amber/src/amber/router/pipe/pipeline.cr 21:11
app_1 | 0x77bc43: process at /opt/crystal/src/http/server/request_processor.cr 39:11
app_1 | 0x77b8f9: process at /opt/crystal/src/http/server/request_processor.cr 16:3
app_1 | 0x77b809: handle_client at /opt/crystal/src/http/server.cr 191:5
app_1 | 0x53eae3: ~procProc(Nil) at /opt/crystal/src/kernel.cr 167:1
app_1 | 0x560b24: run at /opt/crystal/src/fiber.cr 255:3
app_1 | 0x5284d6: ~proc2Proc(Fiber, (IO::FileDescriptor | Nil)) at /opt/crystal/src/concurrent.cr 61:3
app_1 | 0x0: ??? at ??
class HomeController < ApplicationController
def index
users = User.all.first
end
end
class User < Jennifer::Model::Base
mapping(
id: { type: Int32, primary: true },
email: String,
created_at: { type: Time, null: true },
updated_at: { type: Time, null: true }
)
end
Crystal version 0.23.1 and 0.22.
#includes
method to #preload
and remove last one.#extends
with old behavior of #includes
.This manipulations allows new upcomers from rails world easier switch to Jennifer - default behavior of ActiveRecord's #includes
is preloading records rather than joining =
Investigate and add (if possible) support of callable query object as %scope
argument
class SomeModel < Jennifer::Model::Base
# ...
scope :some_scope, SomeQueryObject
end
class SomeQueryObject < Jennifer::QueryObject
def call(arg1, arg2)
relation.where { (_id != arg1) & (time == arg2) }
end
end
I have found a typo on DropIndex on:
The error message I got:
in lib/jennifer/src/jennifer/migration/base.cr:79: undefined constant TableBuilder::DropInde (did you mean 'TableBuilder::DropIndex')
TableBuilder::DropInde.new(table_name, name).process
A fast correction would be good.
Allows model to have nillable field but restrict it to be nil before saving. In this new ::validate_presence_of
should help.
Jennifer v 0.4.2 supports crystal 0.23.1. Support of crystal 0.24.1 will be added in 0.5.0 version with some breaking changes by the end of 2017.
I found at least two:
has_many :stats, Stats
in Model gives me an error
in macro 'has_many' expanded macro: included:1, line 6: type must be NoReturn, not Stats:Class
While using sam
i.e. crystal sam.cr -- db:schema:load
Config.db returns "" nevetherless db is in config for sure, during def self.read
db is set properly, but is gone somewhere after.
Rolling back to 0.4.0 all the errors are gone.
Hi Roman!
I have simple query:
SELECT count(*) AS stat_count, date_trunc('year', created_at) AS period FROM stats GROUP BY period ORDER BY period DESC
But if i try to execute query with DSL:
Stats.all.select("count(*) AS stat_count, date_trunc('year', created_at) AS period FROM stats GROUP BY period ORDER BY period DESC")
Error with extra FROM
:
ERROR: syntax error at or near "FROM".
Original query was:
SELECT count(*) AS stat_count, date_trunc('year', created_at) AS period FROM stats GROUP BY period ORDER BY period DESC
FROM stats
(Jennifer::BadQuery)
Okay, rewrite with .from
and .group
:
Stats.all.select("count(*) AS stat_count, date_trunc('year', created_at) AS period").from("stats").group("period")
Noooooooooooo :)
ERROR: syntax error at or near "GROUP".
Original query was:
SELECT count(*) AS stat_count, date_trunc('year', created_at) AS period
FROM ( stats GROUP BY period ORDER BY period DESC ) (Jennifer::BadQuery)
This query can be executed with Jennifer DSL? If not than how to execute raw SQL?
Hi @imdrasil jennifer looks amazing!
We are using Jennifer at Amber too
This issue is a suggestion to add h1 title to he following pages:
I wrongly added a copyright message to bottom page, maybe I can delete this message or replace it with a new one 😅
Thanks you for jennifer!
Hi,
I've stumbled across a possible issue or maybe also just a side note when working with PostgreSQL.
Whenever you use patterns with percent signs in raw SQL queries and pass them as arguments, the query fails:
Foo.where { sql("bar LIKE '%baz%'") }.pluck(:bar)
Assuming those are valid tables / columns, the error is as follows:
ERROR: Too few arguments (ArgumentError)
from /usr/share/crystal/src/string/formatter.cr:596:8 in 'current_arg'
from /usr/share/crystal/src/string/formatter.cr:315:5 in 'next_arg'
from /usr/share/crystal/src/string/formatter.cr:213:5 in 'int'
from /usr/share/crystal/src/string/formatter.cr:185:7 in 'consume_type'
from /usr/share/crystal/src/string/formatter.cr:173:11 in 'consume_type'
from /usr/share/crystal/src/string/formatter.cr:39:7 in 'consume_percent'
from /usr/share/crystal/src/string/formatter.cr:21:9 in 'format'
from /usr/share/crystal/src/kernel.cr:294:5 in 'sprintf'
from /usr/share/crystal/src/string.cr:4063:5 in '%'
from lib/jennifer/src/jennifer/adapter/postgres/sql_generator.cr:117:10 in 'parse_query'
from lib/jennifer/src/jennifer/adapter/base.cr:133:9 in 'parse_query'
from lib/jennifer/src/jennifer/adapter/request_methods.cr:59:16 in 'pluck'
from lib/jennifer/src/jennifer/query_builder/executables.cr:46:9 in 'pluck'
???
Depending on the character after the %
it may also raise an ArgumentError: Malformed format string
.
It seems to me that this is because of the underlying call to Kernel#sprintf
in Jennifer::Postgres::SQLGenerator
(https://github.com/imdrasil/jennifer.cr/blob/master/src/jennifer/adapter/postgres/sql_generator.cr#L117) where each %
is then used for interpolation. When I use the pattern '%%baz%%'
, it works. Despite this being a valid pattern and despite the existence of a query DSL replacement it would probably still be useful to note this issue. Or am I missing some transformation that the query undergoes before reaching the SQLGenerator
?
This probably won't matter but the PostgreSQL version is 10.4, Jennifer on v0.6.0 and Crystal is v0.25.1.
Hi,
the database I am using Jennifer with has most of its floating point values stored as numeric
s. In order to make it work with Jennifer, I have to use PG::Numeric
as type in the mapping. However, it would be convenient if it was possible to use e.g. Float64
, to avoid having to use to_f
constantly. Also using Postgres-specific data type in the application seems odd.
Is there already a way to deal with Numerics. I hacked around in the _extract_attributes
method, with some success, but got the feeling ther could be a nicer solution.
Thanks for any help!
When you run crystal docs
a bunch of Jennifer internal code makes it into the html. Here is a screenshot showing what I mean.
I think it's just a matter of adding :nodoc:
on some of the internals
Hi there!
I've defined a model, and am simply trying to pull a full instance of it:
pp StarSystem.all.first!
However, this hangs forever with:
StarSystem.all.first! #=> 2017-07-07 23:25:49 -0400: SELECT e.enumtypid
FROM pg_type t, pg_enum e
WHERE t.oid = e.enumtypid
2017-07-07 23:25:49 -0400: SELECT star_systems.*
FROM star_systems
This however works fine:
pp StarSystem.all.count
StarSystem.all.count # => 2017-07-07 23:27:18 -0400: SELECT e.enumtypid
FROM pg_type t, pg_enum e
WHERE t.oid = e.enumtypid
2017-07-07 23:27:18 -0400: SELECT COUNT(*) FROM star_systems
1
This works too:
pp StarSystem.all.pluck(["name"])
StarSystem.all.pluck(["name"]) # => 2017-07-07 23:28:56 -0400: SELECT e.enumtypid
FROM pg_type t, pg_enum e
WHERE t.oid = e.enumtypid
2017-07-07 23:28:56 -0400: SELECT star_systems.name
FROM star_systems
[["Njikan"]]
I resolved this issue at one point, but I can't remember how I did it :( According to the specs, this should work fine..
The postgres access logs look normal. I'm unsure, but it looks like it isn't actually running the query.
Something I'm missing?
I love the documentation in this project much. What I like about it much is that it is up-to-date according to the code. When I used anything from the documentation it worked.
Currently the documentation is on wiki pages.
It is ideal to have a link to documentation on main README page. "Read documentation index here." linked to a page with main topic and content.
Documentation is best maintained if it is associated with the actual version of the source code. Wiki does not allow it, no clear how to create PR with improvements, ... What I do is docs/documentation_file.md
and make links from the README.md. Then it is available for enhancements from contributors by the same process as improving the code.
Im getting this error:
Error in lib/jennifer/src/jennifer/adapter/base_sql_generator.cr:137: BUG: no target defs
io << query._raw_select
When I run this command on a amber project:
crystal src/sam.cr -- generate:migration CreateCompanies
Any idea on whats the problem?
Config is currently stored in a number of class level variables and means that you can only be connected to one database in any app. It would be very useful to be able to have multiple configurations and use these on a per model basis.
First pass at this would be to
Second pass, (possibly as a separate issue) would be to allow runtime selection and configuration of adapters instead of compile time. Happy to make these changes if you're 👍
Issue:
query._order.join(", ", io) { |(k, v), _| io.print k.as_sql(self), " ", v.upcase }
causes the following error:
in lib/jennifer/src/jennifer/adapter/base_sql_generator.cr:202: undefined method 'upcase' for Nil (compile-time type is (String | Nil))
query._order.join(", ", io) { |(k, v), _| io.print k.as_sql(self), " ", v.upcase }
^~~~~~
Every query is built by calling self.body_section
in base_sql_generator.cr
.
This calls order_clause(io, query)
in the same file (line 199-204
).
The following line (202) is the problematic one:
query._order.join(", ", io) { |(k, v), _| io.print k.as_sql(self), " ", v.upcase }
Explanation:
query._order
is a CriteriaContainer
, which is an Enumerable
.
The Enumerable
class join
method calls each_with_index
, which uses each
underneath.
each
is overriden in CriteraContainer
to use @value_bucket[internal_key]
as the values for the iteration. These are not guaranteed to be non-nil.
This is fixed by changing the line to:
query._order.join(", ", io) { |(k, v), _| io.print k.as_sql(self), " ", v.upcase unless v.nil? }
As a user, if I preload association using includes
I want Jennifer to determine empty association and use the result when the association is accessed.
Currently, when there is an empty association that is included using includes
then Jennifer executes the query again when the association is accessed.
The following code,
def jumios
users = User.all
.includes(:jumio)
.where{_id.in [6909, 5143]}
.to_a
users.each do |user|
puts "User #{user.id}: #{user.jumio.nil?}"
end
end
class User < Jennifer::Model::Base
...
has_one :jumio, Jumio
end
produces the following output, provided the user 6909 has no Jumio records and user 5143 has at least one Jumio record,
2017-08-29 21:39:42 +0000:
SELECT users.*, jumios.*
FROM users
LEFT JOIN jumios ON jumios.user_id = users.id
WHERE users.id IN($1, $2)
| [6909, 5143]
User 5143: false
2017-08-29 21:39:42 +0000:
SELECT jumios.*
FROM jumios
WHERE jumios.user_id = $1
LIMIT 1
| [6909]
User 6909: true
Observe how user 6909 is queried again while 5143 is not.
The duplicate queries seriously bloat the processing time on even a simple logic. For 10 users the time increases by 50s and 100s milliseconds.
ATM I work on providing support of crystal 0.25.0
. This process has some challenges but the most interesting one has changed type fallback system and/or method return type eager checking. As an example
after running crystal spec/view/experimental_mapping_spec.cr
on the add_crystal_25_support
branch we will get:
Module validation failed: Stored value type does not match pointer operand type!
store %"Jennifer::QueryBuilder::MultiQueryRelationTree"* %1, %"Jennifer::QueryBuilder::NestedRelationTree"** %__temp_1884
%"Jennifer::QueryBuilder::NestedRelationTree"*Stored value type does not match pointer operand type!
store %"Jennifer::QueryBuilder::NestedRelationTree"* %4, %"Jennifer::QueryBuilder::MultiQueryRelationTree"** %5
%"Jennifer::QueryBuilder::MultiQueryRelationTree"*
???
...
This is actually isn't the only problem, but ATM at least we need to figure out how to fix this simplest example
I don't really understand what this error is trying to tell me. Here's the code and error:
instantiating 'OrmTestJennifer:Module#simple_update(Int32)'
in src/orm_test/jennifer/setup.cr:44: instantiating 'OrmTestJennifer::User:Class#where()'
u = User.where { _orm == "jennifer" & _idx == idx_value }.first.as(User)
^~~~~
in lib/jennifer/src/jennifer/model/resource.cr:89: no overload matches 'Jennifer::QueryBuilder::ModelQuery(OrmTestJennifer::User)#set_tree' with type (Bool | Jennifer::QueryBuilder::Condition)
Overloads are:
- Jennifer::QueryBuilder::Query#set_tree(other : LogicOperator | Condition)
- Jennifer::QueryBuilder::Query#set_tree(other : Query)
- Jennifer::QueryBuilder::Query#set_tree(other : Criteria)
- Jennifer::QueryBuilder::Query#set_tree(other : Nil)
ac.set_tree(tree)
^~~~~~~~
Using the master branch of jennifer and crystal 0.25.1
Problem: When specifying a custom port (like 3307), the connection string is decomposed and rebuilt without a semicolon separating the host and the port.
Effect: This causes a connection error. It tries to connect to localhost3307:3306
because, by default in the mysql driver, it can not find a port from localhost3307
connection string.
I have narrowed this down to line 145
in base.cr
.
It is fixed changing the line
from:
host_part += Config.port.to_s if Config.port && Config.port > 0
to:
host_part += (":" + Config.port.to_s) if Config.port && Config.port > 0
I'm not sure this is an issue itself, but I just found that by including jeniffer.cr
shard, and just require it in an empty project, the compile time with --release
takes forever.
When supplying a WHERE
query that does not have a natural lhs/rhs (it is one statement), the whole condition is considered a LHS.
In condition.cr
Line 78-94
, the RHS is asked for sql_args
but this is not performed on the LHS.
As a result, queries such as this fail to supply the proper arguments:
SELECT regions.*
FROM regions
WHERE (ST_CONTAINS(ST_GeometryN(ST_GeomFromGeoJSON(regions.bounds, 1, 0), 1), POINT(?, ?)))
Example of the query being 'built':
return Region.all.where {
sql(
"ST_CONTAINS(ST_GeometryN(ST_GeomFromGeoJSON(regions.bounds, 1, 0), 1), POINT(?, ?))",
[point.longitude, point.latitude]
)
}.to_a.first
The resulting error is:
Incorrect arguments to mysqld_stmt_execute.
Original query was:
SELECT regions.*
FROM regions
WHERE (ST_CONTAINS(ST_GeometryN(ST_GeomFromGeoJSON(regions.bounds, 1, 0), 1), POINT(?, ?)))
Jennifer::QueryBuilder::ModelQuery(Region+)@Jennifer::QueryBuilder::ModelQuery(T)#to_a:Array(Region+)
Region::find_point<Geo::Point>:Region+
As a user, I would like Jennifer to deserialize content of HStore column from Postgres database.
Column verification_score is expected to be a (String | Nil) but got Slice(UInt8). (Jennifer::DataTypeMismatch)
Column | Type | Modifiers
----------------------------+-----------------------------+----------------------------------------------------
verification_score | hstore |
The idea behind using crystal String
was that I can then parse the value into Hash. Is there a way that Jennifer can automate it? How would I do it by myself Slice(UInt8)
does not sound right for the value such as,
verification_score
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
"email"=>"5", "total"=>"35", "quotes"=>"5", "selfies"=>"0", "twitter"=>"5", "facebook"=>"5", "linkedin"=>"5", "instagram"=>"0", "strengths"=>"{\"facebook\"=>1, \"instagram\"=>0, \"linkedin\"=>1, \"twitter\"=>1}", "audio_tracks"=>"5", "phone_number"=>"5"
This a more of a question/request, but have you considered using a schema.cr
file instead of a mapping
macro inside of the model? I realize it would be a big change, but it would bring an important benefit, the ability to have the model definition update automatically. It also makes the model files look a little less cluttered.
Obviously, the one downside to this is that you don't have everything to do with the model all in one place, but I feel like it would be an ok tradeoff.
Thoughts?
Something broke with the 0.4.2 release, more specifically in regards to the sam.cr
integration or the migrations.
# sam.cr
require "jennifer/adapter/postgres"
require "jennifer"
require "sam"
Jennifer::Config.configure do |conf|
conf.host = "localhost"
conf.adapter = "postgres"
conf.db = "jennifer-bug"
conf.migration_files_path = "./migrations"
end
load_dependencies "jennifer"
Sam.help
Run crystal run sam.cr -- help
.
crystal23 run sam.cr -- help
Tasks:
--------------------:--------------------
help Prints description for all tasks
db:migrate Will call all pending migrations
db:step Invoke next migration. Usage: db:step [<count>]
db:rollback Rollback migration. Usage: db:rollback [v=<migration_exclusive_version> | <count_to_rollback>]
db:drop Drops database
db:create Creates database
db:setup Runs db:create and db:migrate
db:version Prints version of last runned migration
db:schema:load Loads database structure from structure.sql
generate:migration Generates migration template. Usage generate:migration <migration_name>
...
in lib/jennifer/src/jennifer/adapter/base_sql_generator.cr:34: instantiating 'select_clause(String::Builder, Jennifer::QueryBuilder::Query+, Array(String))'
select_clause(s, query, exact_fields)
^~~~~~~~~~~~~
in lib/jennifer/src/jennifer/adapter/base_sql_generator.cr:141: instantiating 'Jennifer::QueryBuilder::Query+#_select_fields()'
query._select_fields.not_nil!.join(", ", io) { |f| io << f.definition }
^~~~~~~~~~~~~~
in lib/jennifer/src/jennifer/query_builder/i_model_query.cr:30: instantiating 'Array(String)#each()'
@relations.each do |r|
^~~~
in /opt/crystal23/src/indexable.cr:148: instantiating 'each_index()'
each_index do |i|
^~~~~~~~~~
in /opt/crystal23/src/indexable.cr:148: instantiating 'each_index()'
each_index do |i|
^~~~~~~~~~
in lib/jennifer/src/jennifer/query_builder/i_model_query.cr:30: instantiating 'Array(String)#each()'
@relations.each do |r|
^~~~
in lib/jennifer/src/jennifer/query_builder/i_model_query.cr:31: undefined method 'Jennifer::QueryBuilder::IModelQuery#model_class()'
table_name = @table_aliases[r]? || model_class.relation(r).table_name
^~~~~~~~~~~
I've tried this both with the Crystal 0.24 pre-release as well as with 0.23 - can be reproduced on both.
I have a hunch it's the part about make IModelQuery class as new superclass of ModelQuery(T); move all methods no depending on T to the new class but I kind of gave up on debugging it.
Do you have any idea how to fix this?
Hi! First of all, thanks for the amazing effort put into this!
I went to build the docs for this project (v0.3.1
and master
) and it could not compile:
➜ jennifer.cr git:(master) cr docs
Error in line 1: while requiring "./src/**"
in src/jennifer.cr:12: while requiring "./jennifer/query_builder/*"
require "./jennifer/query_builder/*"
^
in src/jennifer/query_builder/condition.cr:1: while requiring "./criteria"
require "./criteria"
^
in src/jennifer/query_builder/criteria.cr:4: undefined constant DBAny
alias Rightable = Criteria | DBAny | Array(DBAny)
While the README is excellent, the terse and familiar searchable docs would be great to have.
I find time to look into fixing this, I'll open a PR.
Side note - is the TODO list README entirely up to date? It mentions PG::Array support not being present, however there are Array
types in the DBAny
union here: https://github.com/imdrasil/jennifer.cr/blob/master/src/jennifer/adapter/postgres.cr#L8-L12
If you wouldn't mind going in to more detail on the current support for this, that would be great - I do plan to make use of a PG::Array
column in my app.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.