Comments (17)
I tried textmate-grammar-python
and looks way nicer with the tokenization (see #222 (comment)). Definitely makes it easier to deduce if it's method definition in an abstract class. Further, I really like the nested dictionaries, where I can just skip the body of a function, once I collected what I need.
It will require a lot of re-writing, but I'm quite tempted by it.
We can setup a call, but be warned I am by no means an expert in the cross-referencing. You can contact me at jorgen at cederberg dot be.
from matlabdomain.
If you want a starting point re: classes, I have now gotten most of a classdef parser written here https://github.com/apozharski/matlabdomain/blob/only-enums/sphinxcontrib/textmate_parser.py including argument blocks.
I am happy to continue work on it and submit a pr or you can pull out whatever is useful.
from matlabdomain.
@apozharski regarding priority of docstrings, they are as follows:
- properties, enums, (events): Comments before the property have higher precedence, than a trailing comment. However, there cannot be empty lines before the property.
- functions and classes: always after the function or classdef line.
from matlabdomain.
@apozharski I ran into an issue with class attributes and created an issue watermarkhu/textmate-grammar-python#67.
In the current parsing of classdef / method / property attributes I reuse the same method:
matlabdomain/sphinxcontrib/mat_types.py
Line 1474 in 4d890d7
from matlabdomain.
Hi @watermarkhu
This looks really interesting. At the moment I started tackling #44 and #222, and the pygments token output is just a mess to start parsing. I'll give a shot a see if it can replace pygments and then improve the functionality.
Regarding starting up a new auto-documenter, I can only tell how this domain was started. The original author basically built the documenter directly upon autodoc for Python. This gave them a good start and basis.
However, the code is not the easiest to work with in my opinion. We still run into features that needs to be reimplemented, for instance #180. Even after maintaining the package for many years now, I still struggle with the Sphinx internals of Documenter and Directives 😵.
A different approach for autodoc is done in https://github.com/mozilla/sphinx-js. I hope this helps.
from matlabdomain.
Good to hear!
I'm currently mostly struggling with setting up roles in a new domain in order to make cross-referencing possible eventually. Can we possibly setup a call?
from matlabdomain.
@watermarkhu Two comments to https://github.com/watermarkhu/textmate-grammar-python:
- Requirement for Python 3.11 is hard for
sphinxcontrib-matlabdomain
. I still support version 3.9. - Requirement for PyYAML >=7 is in conflict with
conan
version requirement ofPyYAML>=6.0, <7.0
.
Do you want me to add them as issues?
from matlabdomain.
Good to see! Adding the issues would be great.
Let's discuss about 3.9 support on the PR that you submitted.
from matlabdomain.
Hello, not to step on any toes here but I would like to know if this effort has stalled (understandably, time is always a valuable commodity)? The matlab library I am a maintainer of is currently going through a major documentation pass and to that end I have allocated some time to working on tooling. As such, I think this would be a good place to start as it will help in closing #52, #54, #212, and #222.
Those four issues are currently my target to get done (perhaps in one fell swoop along with this one) as they would be very useful for our documentation. I have started an attempt to implement classdef
class parsing working here and it seems like it should be doable to replace the current parsing code with something (at least marginally) better. Let me know if I have misread the situation.
from matlabdomain.
Hi. It was definitely stalled. I have had zero time to work on this project unfortunately. This week, I'll give it a shot. The most difficult issue to solve is still #222.
from matlabdomain.
As an aside there are definitely some bugs in the textmate parser (watermarkhu/textmate-grammar-python#66 (comment) for example), though after digging I suspect they are in the underlying grammar maintained by mathworks. I am currently looking at a possible fix for it though @watermarkhu may have the inner track on understanding the grammar format.
from matlabdomain.
If you want a starting point re: classes, I have now gotten most of a classdef parser written here https://github.com/apozharski/matlabdomain/blob/only-enums/sphinxcontrib/textmate_parser.py including argument blocks.
I am happy to continue work on it and submit a pr or you can pull out whatever is useful.
Thanks. I will take this as a starting point. It looks very useful already. If you have any PR's, I'll work on the development branch dev-textmate-grammar-for-parsing
.
from matlabdomain.
@apozharski regarding priority of docstrings, they are as follows:
* properties, enums, (events): Comments before the property have higher precedence, than a trailing comment. However, there cannot be empty lines before the property. * functions and classes: always after the function or classdef line.
Yep that is what I thought was the case. Thanks for clarifying. I will do some cleanup and get the routines to check for non-consecutive comments and submit a PR to your dev branch.
from matlabdomain.
@joeced After spending a few too many hours trying to fix the mathworks provided textmate grammar I am convinced that it is not worth continuing to force a square peg, a parsing system primarily designed for syntax highlighting, into the round hole that is using it for extracting structure. After doing some research there is a better alternative that is https://github.com/acristoffers/tree-sitter-matlab which is a matlab grammar for tree-sitter
which uses a "proper" LR parser and produces a much more usable AST. It also does not have the seeming performance downsides: watermarkhu/textmate-grammar-python#68.
Over the last couple days I quickly threw together a working prototype with support for I believe the full suite of matlab syntax (argument
blocks, enumeration
blocks, events
blocks etc.): https://github.com/apozharski/matlabdomain/blob/tree-sitter-dev/sphinxcontrib/mat_tree_sitter_parser.py
I think this is the direction this project should go in as it does not require us to fix yet more bugs in MATLAB-textmate-grammar
, and it already supports the full feature-set we need. The one concern is that while tree-sitter
is available on pyPI tree-sitter-matlab
is not yet. I have reached out to the developer via issue acristoffers/tree-sitter-matlab#12 and they seem receptive to packaging it for pyPI.
from matlabdomain.
Hi @apozharski, thank you very much for looking into this. Taking the time and effort. Much appreciated! I'm away from a computer at the moment, but will get back to you in 2 weeks.
from matlabdomain.
hi @apozharski - I'm back now!
Did you have any time to work on using tree-sitter-matlab and would you try to make a pull request?
The most important thing for me is not what parsing library is used, but that it is better than the existing, where better equals:
- Fewer lines of code.
- Easier to maintain.
- Easier to add fixes for missing features like abstract methods (#222) and related #44.
Again - thanks for looking into this!
from matlabdomain.
Hello @joeced,
yes I had been sidelined last week on this due to some urgent work.
I have a branch with a 90-95% working parser and I will open a separate PR with it. In general I think it has simplified the code significantly however I would love to hear your feedback.
The latest work I have done on this is slowly beginning to fix things to get the tests back in working order (and I have found a bug in tree-sitter-matlab
which I have a PR for there now.
Absolutely no problem!
from matlabdomain.
Related Issues (20)
- [Meta] Gathering interest in a "Framework :: Sphinx :: Domain" PyPi classifier HOT 1
- Feature request: show matlab class property dimensions, class, validators HOT 1
- Subfolders in class folders are inappropriately treated as class folders. HOT 10
- Parsing warning HOT 1
- Inquiry on features and documentation: scripts, citations, comments and parsing of existing docs HOT 1
- "Island" comments in the properties section causes parsing of a class to fail HOT 1
- Unexpected parse warning HOT 1
- Document abstract methods HOT 2
- Vague error when there is an empty @ClassFolder HOT 1
- Extension error when deplyoing with GitHub pages HOT 4
- Check if auto-linking feature works with classes in folders with the same name. HOT 3
- bug in auto-linking literal block detection
- documenter.generate() being called multiple times for module HOT 1
- can't link function short name if it matches module name HOT 4
- non-functional :func: and :class: references with identical section name HOT 4
- class folder with matlab_short_links causes fatal error HOT 2
- Napoleon reformulation does not get applied to `MatProperty` docstrings. HOT 1
- Some class property attributes seem to be missing
- Use `properties` (etc.) blocks as automatic sections for the `autoclass` directives.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from matlabdomain.