jonase/learndatalogtoday

Interactive Datalog Tutorial

learndatalogtoday

An interactive Datalog tutorial.

Prerequisites

You will need Leiningen and java installed.

Run locally

$ lein uberjar
$ java -cp target/learndatalogtoday-standalone.jar clojure.main -m learndatalogtoday.handler

Server is now running on $PORT (http://localhost:8080 by default).

License

Copyright © 2013-2023 Jonas Enlund

Distributed under the Eclipse Public License, the same as Clojure.

learndatalogtoday's Issues

Missing :where in Ch8 example

I think this Ch 8 example

[:find ?name
 :in $ %
 (actor-movie ?name "The Terminator")]

is missing :where and should be

[:find ?name
 :in $ %
 :where
 (actor-movie ?name "The Terminator")]

Thanks for this great resource!

possible unmentioned syntax sugar

I struggled a bit with chapter 4 where it shows that if you put a variable in the attribute position it will become an attribute entity id rather than a keyword as I would of expected. Does that mean that this:

[:find ?p
 :where
 [?p :person/name "jake"]]

Is really just syntax sugar for this:

[:find ?p
 :where
 [?a :db/ident :person/name]
 [?p ?a "jake"]]

If so I think it would be worth a mention

Different results for same where-conditions in different order.

As declared in the beginning of tutorial, logical meaning of :where [ __a_clause__ ] [__b_clause__] is same as :where [ __b_clause__ ] [__a_clause__]. However, this rule doesn't work in following task:

http://www.learndatalogtoday.org/chapter/4

Task 2 : Find all available attributes, their type and their cardinality. ...

This is considered as correct query:

[:find ?attr ?type ?card
 :where
 [_ :db.install/attribute ?a]
 [?a :db/valueType ?t] 
 [?a :db/cardinality ?c]
 [?a :db/ident ?attr]
 [?t :db/ident ?type]
 [?c :db/ident ?card]]

But when simply changing the order of condition clauses - result changes and warning appears. Example:

[:find ?attr ?type ?card
 :where
 [?a :db/valueType ?t] 
 [?a :db/cardinality ?c]
 [?a :db/ident ?attr]
 [?t :db/ident ?type]
 [?c :db/ident ?card]
 [_ :db.install/attribute ?a]]

In my case it was very confusing - because I managed to compose the query like in last example, and finally had to "give up".

Uncaught ReferenceError: CodeMirror is not defined

All of these returned 404s:

/third-party/codemirror-3.14/lib/codemirror.css
/third-party/codemirror-3.14/lib/codemirror.js
/third-party/codemirror-3.14/mode/clojure/clojure.js

Here's my console output after following the instructions in the readme:

$ git clone [email protected]:jonase/learndatalogtoday.git
Cloning into 'learndatalogtoday'...
cd learemote: Counting objects: 238, done.
remote: Compressing objects: 100% (165/165), done.
remote: Total 238 (delta 91), reused 180 (delta 37)
Receiving objects: 100% (238/238), 46.12 KiB | 71.00 KiB/s, done.
Resolving deltas: 100% (91/91), done.
Checking connectivity... done

$ cd learndatalogtoday

$ ./fetch-js-deps.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 85732  100 85732    0     0  64493      0  0:00:01  0:00:01 --:--:--  105k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  771k  100  771k    0     0   217k      0  0:00:03  0:00:03 --:--:--  262k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 51841    0 51841    0     0  25220      0 --:--:--  0:00:02 --:--:-- 40217
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 93064  100 93064    0     0  82131      0  0:00:01  0:00:01 --:--:--  151k
Markdown.Converter.js -> resources/public/third-party/pagedown/Markdown.Converter.js
jquery-1.10.1.min.js -> resources/public/third-party/jquery/jquery-1.10.1.min.js
bootstrap.zip
codemirror.zip

$ lein cljsbuild once
Retrieving lein-ring/lein-ring/0.8.6/lein-ring-0.8.6.pom from clojars
Retrieving lein-ring/lein-ring/0.8.6/lein-ring-0.8.6.jar from clojars
Compiling ClojureScript.
Retrieving hylla/hylla/0.1.0/hylla-0.1.0.pom from clojars
Retrieving ring-mock/ring-mock/0.1.5/ring-mock-0.1.5.pom from clojars
Retrieving datomic-query-helpers/datomic-query-helpers/0.1.0/datomic-query-helpers-0.1.0.pom from clojars
Retrieving com/cemerick/clojurescript.test/0.0.4/clojurescript.test-0.0.4.pom from central
Retrieving markdown-clj/markdown-clj/0.9.29/markdown-clj-0.9.29.pom from clojars
Retrieving hiccups/hiccups/0.2.0/hiccups-0.2.0.pom from clojars
Retrieving com/datomic/datomic-free/0.8.4111/datomic-free-0.8.4111.pom from clojars
Retrieving com/cemerick/clojurescript.test/0.0.4/clojurescript.test-0.0.4.jar from central
Retrieving ring-mock/ring-mock/0.1.5/ring-mock-0.1.5.jar from clojars
Retrieving hylla/hylla/0.1.0/hylla-0.1.0.jar from clojars
Retrieving hiccups/hiccups/0.2.0/hiccups-0.2.0.jar from clojars
Retrieving datomic-query-helpers/datomic-query-helpers/0.1.0/datomic-query-helpers-0.1.0.jar from clojars
Retrieving com/datomic/datomic-free/0.8.4111/datomic-free-0.8.4111.jar from clojars
Retrieving markdown-clj/markdown-clj/0.9.29/markdown-clj-0.9.29.jar from clojars
Compiling "resources/public/app.js" from ["src/cljs"]...
Successfully compiled "resources/public/app.js" in 6.957538 seconds.

$ lein ring server
Started server on port 3000

Find movies older than a certain year (inclusive)

In chapter5=>http://www.learndatalogtoday.org/chapter/5 , first exercise...

It seems that the predicate 'older than' must be >= instead of <=, isn't it?

Expected solution:

[:find ?title
 :in $ ?year
 :where
 [?m :movie/title ?title]
 [?m :movie/year ?y]
 [(<= ?y ?year)]]

Supossed solution:

[:find ?title
 :in $ ?year
 :where
 [?m :movie/title ?title]
 [?m :movie/year ?y]
 [(>= ?y ?year)]]

Datalog performance

In http://localhost:3000/chapter/2: "The order of the data patterns does not matter (except possibly for performance)"

It definitely matters :-) Datalog has no query optimiser; it'll execute clauses in the order given and take as long as it needs to do that. You should place your most restrictive clauses at the top so that successive clauses have less data to operate on.

Depending on the shape of your data, you can see orders of magnitude difference in query performance!

Attempting to run any query results in "Oh snap! EOF while reading"

"whitelisted symbol" rules are too restrictive, not allowing building up rules from smaller rules.

Attempted this solution to chapter 8, ex. 1:

[
 [(prefriends ?p1 ?p2)
  [?m :movie/director ?p1]
  [?m :movie/cast ?p2]
 ]
 [(prefriends ?p1 ?p2)
  [?m :movie/cast ?p1]
  [?m :movie/director ?p2]
 ]
 [(prefriends ?p1 ?p2)
  [?m :movie/cast ?p1]
  [?m :movie/cast ?p2]
 ]
 [(friends ?p1 ?p2)
  (prefriends ?p1 ?p2)
  [(not= ?p1 ?p2)]
 ]
]

which was not accepted since prefriends is not a whitelisted symbol.

Chapter 3 Confusing text

In Chapter 3, there are two examples of code, followed by this text:

This query takes two arguments: $ is the database itself (implicit, if no :in clause is specified) and ?name which presumably will be the name of some actor.

The above query is executed like (q query db "Sylvester Stallone"), where query is the query we just saw, and db is a database value. You can have any number of inputs to a query.

In the above query, the input pattern variable ?name is bound to a scalar - a string in this case. There are four different kinds of input: scalars, tuples, collections and relations.

I find the wording here is confusing, requiring careful re-reading to parse between "The above query" and "In the above query" which refer to, it appears to me (please correct me if I am mistaken!) the two different queries.

I suggest either:

• labeling the examples "query A" and "query B" to be extra clear
• refer to examples specifically as "non-reusable query" and "parameterized query", respectively
• moving the descriptive text to be adjacent to their respective examples (and not interleaved, if this is the case)

Tried to use insecure HTTP repository without TLS

When I tried to do, lein uberjar

I got:

Tried to use insecure HTTP repository without TLS:
 Tried to use insecure HTTP repository without TLS:
project: http://mrep.s3-website-us-east-1.amazonaws.com
 org/hornetq/hornetq-core-client/2.4.7.Final/hornetq-core-client-2.4.7.Final.jar  project: http://mrep.s3-website-us-east-1.amazonaws.com
 com/datomic/datomic-lucene-core/3.3.0/datomic-lucene-core-3.3.0.jar
  Tried to use insecure HTTP repository without TLS:


This is almost certainly a mistake; for details see   project: http://mrep.s3-website-us-east-1.amazonaws.com
 org/hornetq/hornetq-server/2.4.7.Final/hornetq-server-2.4.7.Final.jar
This is almost certainly a mistake; for details see
https://codeberg.org/leiningen/leiningen/src/main/doc/FAQ.md  
 
https://codeberg.org/leiningen/leiningen/src/main/doc/FAQ.md
This is almost certainly a mistake; for details see
 
Tried to use insecure HTTP repository without TLS:
 project: http://mrep.s3-website-us-east-1.amazonaws.com
 com/h2database/h2/1.3.171/h2-1.3.171.jar 
https://codeberg.org/leiningen/leiningen/src/main/doc/FAQ.md
 

This is almost certainly a mistake; for details seeTried to use insecure HTTP repository without TLS:
 project: http://mrep.s3-website-us-east-1.amazonaws.com
 io/netty/netty-all/4.0.13.Final/netty-all-4.0.13.Final.jar 
 
This is almost certainly a mistake; for details see 
https://codeberg.org/leiningen/leiningen/src/main/doc/FAQ.md
 
https://codeberg.org/leiningen/leiningen/src/main/doc/FAQ.md

According to lein's FAQ:

 This means your project was configured to download dependencies from a repository that does not use TLS encryption. This is very insecure and exposes you to trivially-executed man-in-the-middle attacks. 
...
It's also possible you have a dependency which includes a reference to an insecure repository for retrieving its own dependencies. If this happens it is strongly recommended to add an :exclusion and report a bug with the dependency which does this.

I don't really do lein nor do I know how to do an exclusion, so it would be nice to get a fix for this.

Possible ambiguous syntax

The second question ("tab #1") in Chapter 4 has two ellipses in the starting code. How are we to know which if either or both are "fill in the blanks" versus "syntax that we are to leave be"? After viewing the answer, I see what was expected of me, but could this be made clearer by having a "strict" blank that doesn't look like valid syntax?

Suggestion 1: For example, in the Clojure Koans, they use the double underscore "__" as their "blanks" to fill in.
Suggestion 2: Add an annotation or hint that helps the learner know that they are only to touch X line(s) for a given problem.

Browsers like Firefox warn on invalid tls certificate

The hosting should probably leverage letsencrypt.org certificates to avoid scary-looking browser warnings:

https://letsencrypt.org/getting-started/

Correct answer marker

Would be lovely if the user got a "You answered this correctly" message, akin to how there is a "Sorry, these results are not correct" response on incorrect answers.

Chapter 8, Exercise 1: case of director & cast for the same movie not handled

In Chapter 8, Exercise 1, the suggested solution is as follows:

[[(friends ?p1 ?p2)
  [?m :movie/cast ?p1]
  [?m :movie/cast ?p2]
  [(not= ?p1 ?p2)]]
 [(friends ?p1 ?p2) [?m :movie/cast ?p1] [?m :movie/director ?p2]]
 [(friends ?p1 ?p2) (friends ?p2 ?p1)]]

I think the second rule definition should also have an inequality clause for ?p1 and ?p2, and read

[(friends ?p1 ?p2)
  [?m :movie/cast ?p1]
  [?m :movie/director ?p2]
  [(not= ?p1 ?p1)]]

Which certainly looks more verbose and doesn't fit on the same line in the exercise editor anymore, but it handles a previously unhandled case: When a director is also cast in their own movie (which doesn't seem to exist in this db, but generally isn't that unlikely).

Perhaps I'm missing something and the first rule definition already prevents this?

Can I query by attribute values?

I'm stuck at this chapter:
https://www.learndatalogtoday.org/chapter/4

So I do this:

[:find ?e
 :where 
 [?e :db/ident :person/name]
]

and it shows me 68, but I can't do this:

[:find ?e ?a
 :where 
 [?e ?a :person/name]
]

— meaning to see all the entities with their attributes that equal to :person/name. The sandbox returns just an empty result set:

Why is that?

Whitelisting rules

In http://www.learndatalogtoday.org/chapter/8 excercise 1, it would be nice to be able to use other rules. But it seems there's a whitelist that's prohibiting me in doing so:

Rules:
[[(friends ?p1 ?p2)
(associated-with ?p1 ?movie)
(associated-with ?p2 ?movie)]
[(associated-with ?person ?movie)
[?movie :movie/cast ?person]]
[(associated-with ?person ?movie)
[?movie :movie/director ?person]]]

Output:
Oh snap!
Non-whitelist symbol used in query/args: #{associated-with}. The symbol whitelist is #{sum sequels not= friends tutorial.fns/age .getMonth .getDate count < = > max >= <= movie-year min avg}

Would it be possible to avoid the whitelist when it comes to rules?

Is this a bug?

Find people by age. Use the function tutorial.fns/age to find the age given a birthday and a date representing "today".

http://www.learndatalogtoday.org/chapter/6

I think this should be "find the birthday given the age and ...".

Chapter 4 Exercise 2 Answer

Hi I am learning datalog with the website of this project. It is quite fun but I have a small question here.

In Chapter 4 Exercise 2, the instruction guide likes "To find all installed attributes you must use the :db.install/attribute attribute".

However, the answer without using ":db.install/attribute" will trigger the correct answer. Just got a little bit confusing about it.

Here is the answer which also trigger the results:

[:find ?attr ?type ?card
 :where

 [?a :db/cardinality ?c]
 [?c :db/ident ?card]

 [?a :db/valueType ?t]
 [?t :db/ident ?type]

 [?a :db/ident ?attr]]

Query issue

In chapter 6

I think the first query contains an error. It is given ?person but uses an undeclared ?name to look folks up.

Should it be the following?

[:find ?age
 :in $ ?name ?today
 :where
 [?p :person/name ?name]
 [?p :person/born ?born]
 [(tutorial.fns/age ?born ?today) ?age]]

On the page http://www.learndatalogtoday.org/chapter/0 the button "Run Query" doesn't do anything

In the console however this message appears:
Uncaught Invalid match arg:
/<(?!area|br|col|embed|hr|img|input|link|meta|param)(([\w:]+)[^>]*)/>/i

I am using Chromium 48 on Linux, although this shouldn't matter.

chapter 5 excercise 1 bug

"Find movies older than a certain year (inclusive)" but the following is accepted as the right answer :
[:find ?title :in $ ?year :where [?m :movie/title ?title] [?m :movie/year ?year]] while the answer with >= predicate is not accepted