Comments (17)
Thanks @j4m3sb0mb, the speed vs size tradeoffs aren't well understood for this library as far as I know.
The last time that I looked at this, I saw a similar improvement in compile-time for the speed
optimized version, which is great for CLI use and online-use (post-loading). However, the tradeoff is slower page loads for online-use and larger bundles when using concerto as a node dependency.
Perhaps this question helps?
Is an extra 100ms acceptable for Template Studio recompiles between edits in exchange for a 75% reduction in bundle size?
My previous assumption was that speed is more important.
Unless there is a clear consensus on this question, I suggest instead focussing on the removal of unused rules from this file (there are a lot of them!), https://github.com/accordproject/concerto/blob/master/packages/concerto-core/lib/introspect/parser.pegjs
That way we get a size reduction but still optimize for speed. 🏆
from concerto.
@mttrbrts
I've tried to remove all lines from 540 to 1260, it doesn't break the parser and it pass all tests.
parser.js size is then 356,2kb with speed optimization
from concerto.
I've tried to modify the script that creates the parser, with optimize option set to size the size of parser.js is 132kb instead of 558kb
"prepare": "pegjs --optimize size ./lib/introspect/parser.pegjs"
from concerto.
I've tried to modify the script that creates the parser, with optimize option set to size the size of parser.js is 132kb instead of 558kb
"prepare": "pegjs --optimize size ./lib/introspect/parser.pegjs"
@j4m3sb0mb Nice! Does that optimize version pass the tests?
from concerto.
@jeromesimeon yes it does
from concerto.
Thanks @j4m3sb0mb It's a relatively significant improvement. I don't see why we wouldn't do this for the time being. @dselman is traveling, but I can review a PR if you make one.
I would still leave this issue open for the other reasons listed (clean up of parser itself, investigate other parser generator technologies).
Would you like me to assign this issue to you for now?
from concerto.
yes
from concerto.
I have just seen this in the pegjs
documentation:
--optimize
Selects between optimizing the generated parser for parsing speed (speed) or code size (size) (default: speed)
Do you have any experience with performance related to this?
I haven't done much testing with pegjs
myself, so I'll probably want to get some feedback on this from either @dselman or @mttrbrts
from concerto.
@jeromesimeon no experience, sorry
from concerto.
@jeromesimeon
I'm testing size optimized version and speed optimized once against org.accordproject.trademark
but I don't find significant differences:
speed optimized version compiles in go in
748, 726, 798
while size optimized compiles in
838, 869, 842
from concerto.
I'll add two cents since @mttrbrts 's review really helped me understand what the questions are. I think I agree that speed during use matters more, even if that means the page initially loads a little slower (not sure how much slower really it's not that bad in Template Studio for instance). This is a one-time cost compared to the constant run-time cost when parsing.
from concerto.
To be a little more specific about parser.pegjs
. This file was originally a copy of a parser for JavaScript which we added new rules to capture the structure for Concerto (The new rules are here, https://github.com/accordproject/concerto/blob/master/packages/concerto-core/lib/introspect/parser.pegjs#L1555).
However, most of the original rules are still there unused (for example more of sections A.3
and A.4
, Lines 540-1260). However, simply removing these lines is likely to break other rules that depend on these lines.
So refactoring will require some incremental changes and lots of testing to remove unused code.
The good news is that we have excellent code coverage, so running unit tests should flag issues quicky.
from concerto.
Excellent! That's a nice surprise.
I suspect that there are lots of other rules that aren't used too. For example:
URI
definition, is much more advanced than we need. I'm sure that that could be simplified.- Tokens, L476
- Keywords, L210
Have some fun!
from concerto.
Excellent! That's a nice surprise.
I suspect that there are lots of other rules that aren't used too. For example:
URI
definition, is much more advanced than we need. I'm sure that that could be simplified.- Tokens, L476
- Keywords, L210
Have some fun!
Did a bit more in #154
The current parser looks pretty clean now. Tokens and Keywords have been cleaned up.
I'm unclear on the definition of what "more advanced than we need" is for URIs. Shouldn't we comply with some kind of URI spec? What would we remove?
Another area that might be simplified are Unicode character sets, but that's the same questions what do we believe the behaviour should be?
from concerto.
@dselman @jeromesimeon It appears that pegjs is no-longer maintained pegjs/pegjs#667
There is a fork (https://github.com/peggyjs/peggy), however it is fairly nascent.
It this the opportunity to switch to a new technology entirely, perhaps standardise with a parser lib that we use in ergo
or markdown-transform
from concerto.
@dselman @jeromesimeon It appears that pegjs is no-longer maintained pegjs/pegjs#667
There is a fork (https://github.com/peggyjs/peggy), however it is fairly nascent.
It this the opportunity to switch to a new technology entirely, perhaps standardise with a parser lib that we use in
ergo
ormarkdown-transform
Markdown transform uses: https://github.com/jneen/parsimmon
Ergo uses: http://gallium.inria.fr/~fpottier/menhir/
from concerto.
This issue is partially address by the introduction of the new concerto-cto
package which makes the parser an optional dependency. Closing this issue until there is new context for reviewing it again.
from concerto.
Related Issues (20)
- Provide an API to extend the default rules of the analyser
- Inconsistent ast types in introspect HOT 2
- Issue trying to validate declarations without properties
- Support Decorator Command Set validation independent of applying decorators
- Support max length for array and map
- "Install the Command Line Tool" link is broken in readme
- Vocabulary support for Map Types in Concerto HOT 3
- JSON Schema output is not working for Scalar Arrays
- Concerto CLI parser allows multiple application of the same Decorator, but not ModelManager HOT 1
- Should not be able to create a concept whose super type is itself
- Decorator Command with properties without a type adds decorator to declaration
- feat: Extract the vocabulary set and decorator command set from a model manager
- Remove Functional API
- VocabularyManager: support configurable fallback locales
- Migrate to ESM from CommonJS Modules HOT 1
- Map does not support polymorphic values HOT 1
- While validating a `ConceptDeclaration` with no properties, the validation fails ungracefully if the `properties` field of the AST is `null` HOT 2
- Errors when creating validators in concerto introspect only return a string in a error message , which makes programatic handling the errors dificult
- Vocabulary auto generated for $identifier HOT 5
- toAst and fromAst are not identity operation HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from concerto.