Comments (7)
Okay, installed new Rubinius version, now I get:
An exception occurred running test_adding_machine.rb
TokenData is not a module (TypeError)
Backtrace:
Rubinius.open_module_under at kernel/delta/rubinius.rb:82
Rubinius.open_module at kernel/delta/rubinius.rb:95
AddingMachine.__module_init__ (AddingMachine) at AddingMachineLexer.rb:66
main.__script__ at AddingMachineLexer.rb:60
Rubinius::CodeLoader.require at kernel/common/codeloader.rb:145
Kernel(Object)#require at kernel/common/kernel.rb:707
main.__script__ at test_adding_machine.rb:1
Rubinius::CodeLoader#load_script at kernel/delta/codeloader.rb:67
Rubinius::CodeLoader.load_script at kernel/delta/codeloader.rb:91
Rubinius::Loader#script at kernel/loader.rb:460
Rubinius::Loader#main at kernel/loader.rb:571
Rubinius::Loader.main at kernel/loader.rb:609
Object#__script__ at kernel/loader.rb:621
from antlr3.
Seems like this is rbx related problem, posted an issue at rbx as well.
Btw. seems to work fine at JRuby as well. :-)
from antlr3.
dbussink: NilsH: looks like it uses some stuff rbx doesn't have yet
dbussink: NilsH: like StopIteration
NilsH: dbussink, ah, okay, so this means it will not work in near future ?
dbussink: NilsH: well, from what i can see the usage of StopIteration is not really necessary
dbussink: NilsH: it can really simply be refactored out and that even simplifies the code
dbussink: NilsH: Enumerable#next / Continuations aren't currently supported in rbx
dbussink NilsH: some more information on why using #next can be problematic / slow: http://archive.codehaus.org/lists/org.codehaus.jruby.dev/msg/[email protected]
from antlr3.
Hello Nils,
Thanks for the compliment -- I appreciate it. Also, thanks for taking the time to research this issue so thoroughly, as you certainly saved me the extra effort involved in researching Rubinius' level of support.
Admittedly, I designed my implementation squarely to be compatible with vanilla Ruby 1.8.7, adding 1.9 compatibility later on. Thus, while I've toyed around with how the library works with Rubinius, I still haven't tested it extensively on that platform.
Thus, I was not aware that Rubinius does not support StopIteration, which, as you probably know, is an exception that simply signals the inner-most loop that's currently executing to stop. Reviewing the code, I see that it is used in the module ANTLR3::TokenSource, which is used to make lexers and certain token streams act Enumerable. While it's mostly there for convenience, it is used by the various TokenStream classes to harvest tokens from lexers.
From what I see in the code so far, it should be a fairly simple fix. I will try and resolve it when I have a little extra time later in the week. If you would like a quick workaround in the meantime, edit the file`lib/antlr3/token.rb'. Change the following code (around line 327) to the code listed below it:
OLD CODE:
def each
block_given? or return enum_for( :each )
loop { yield( self.next ) }
rescue StopIteration
return self
end
NEW CODE:
def each
block_given? or return enum_for( :each )
while token = next_token and not ( token.nil? || token.type == EOF )
yield( token )
end
return self
end
By the way, the comment made by the Rubinius developer concerning next' methods is generally correct, but actually does not apply in this situation. The default
next' method you get when you include Enumerable in a class uses Continuations to magically run the each' method in a step-by-step fashion instead of one single shot through the whole sequence. As a result, they do tend to be obnoxiously slow. However, in this situation, I am explicitly writing a sensible
next' method specifically to avoid the slow version of the same method you get with Enumerable. Thus, I wouldn't worry about the assessment.
Finally, as I haven't tested the code extensively with jruby yet, I'm glad to hear that it appears to be working on that platform. Thank you for the information. If you encounter any issues with it, feel free to let me know. I don't recall any jruby no-nos off the top of my head, though I wouldn't be surprised if there is somewhere in the code that uses ObjectSpace iteration.
I'll have to run the tests on these platforms over the weekend and see if any issues come up. Again, thanks for the report. I'll let you know when I've uploaded a fix for the issue.
Kyle
from antlr3.
Actually, reading back on my post, I see a bit of redundancy in the code I suggested. I would try replacing it with this instead;
def each
block_given? or return enum_for( :each )
while token = next_token and token.type != EOF
yield( token )
end
return self
end
from antlr3.
Hey there,
thx for the extensive answer and quickfix-tips. You dont have to thank me, I try to help good open source projects wherever I can. As I do not have time to dive into all of the codebases, I take it as my duty to fill good bug-reports if I recognize something is going wrong. :-)
Will report any JRuby-related problems coming through my way, as I plan on using your gem in a JRuby application first, and after this for another project on rbx.
But with the TokenData error we have to wait for the rbx-devs to fix the problem, as you can see here: http://github.com/evanphx/rubinius/issues/#issue/512 it seems to be a RBX-related thing. Yesterday dbussink told me in IRC he has an idea where this could be located and it should not be hard to fix.
By the way:
Perhaps we should suggest at the antlr-mailinglist, that they list your work as the recommended ruby-target, as the one they are linking to on the homepage seems to be abandoned,... Will do that this afternoon. :-)
Kind regards,
Nils
from antlr3.
Update:
The Rubinius-Crew fixed the issue with the model, and they strongly recommended never to subclass module. I promised I will tell you. :-) They handed in this link as well: http://www.engineyard.com/blog/2010/rubinius-wants-to-help-you-make-ruby-better/
Also evan says he will support the StopIteration thing soon, but he also recommended NOT to use it.
By the way, your quickfix works so far in the cases I tested it. Thx for your work man!
Yeah, and if you wonder why I need to run your gem on rbx:
We would like to have a pure-ruby parser for www.fancy-lang.org , that can run directly at rbx.
from antlr3.
Related Issues (20)
- DOT Tree Generation throws exception HOT 3
- Patch (v 1.8.11) for cygwin causes issues on (some) non-cygwin envs HOT 1
- Module Wrapper HOT 2
- Generated TreeParser seems to have wrong qualified base class HOT 1
- How to access the following[] tokens from state? HOT 1
- Tree Parser documentation is empty
- rake antlr doesn't work. it depends on missing 'isolate/installer' HOT 1
- Template output example
- Test rig failure
- bug in CommonTreeNodeStream
- License missing from gemspec HOT 2
- Exception when run "antlr4ruby" in Mac HOT 4
- Composite Grammar HOT 1
- Passing options to imports? HOT 1
- Importing Grammar causes Warning
- Contact info
- Ruby 1.9.1 + antlr3 1.7.5 - mismatched character "+"; expecting "+"? HOT 6
- Issues with combined grammars HOT 2
- FreeBSD does not define SOL_TCP, and so cannot remotely debug grammars HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from antlr3.