Writing a research paper on the subject

[konchunas]

Writing a research paper on the subject

[hegza]

Here's the link to the original published in SpringerLink, finally! The publishers are really slow with these things. This project is cited as number 8. It's behind a paywall but I'm looking into if I can parallel publish it via our university.

https://link.springer.com/chapter/10.1007%2F978-3-030-60939-9_9

I followed up on the research (due to availability of funding :)) and wrote a prototype AST-to-AST transpiler that can be found at https://github.com/hegza/serpent-rs. It's a whole lot more complex than the original pyrs project, which is not necessarily a good thing, but it can do some additional tricks. It's also less feature complete because of the complexity. Here's also a link to a command-line interface I'm using to work with the project https://github.com/hegza/serpent-cli/

[hegza]

Yes, as it happens, Python seldom has "Python only" libraries and usually relies on either C, C++ or Rust for native acceleration, which all have pretty good inter-op with Rust anyway. Generally, the only hurdle to pass is the availability of the API between Rust and the accelerated backend. Once an API exists, it's a matter of re-mapping transpiled py2rs calls into the API. Remapping is also simple enough, since semantics don't vary a lot between these languages: you pass in N parameters and return M values.

However, borrow, pointer and value semantics need attention and there are likely edge cases that don't map nicely. The Rust compiler seems to be helpful with those however. And then there are even more complex edge cases like lambdas / function pointers and OOP APIs in Python.

I haven't been working on it a lot recently, but I think I need to get back into it since it's a major part of my dissertation :D Transpiling APIs based on Python classes seems to be the biggest unresolved issue. I've an idea on how to resolve the rest.

[konchunas]

Hey! Sorry, with these Github UI updates I have somehow missed your message.

I am glad you found my transpiler useful! And I am even more stoked you managed to write a paper on its topic!

Yes, sure I would like to hear more about both your project and solutions you propose. Do you have draft of your paper published somewhere?

[hegza]

I don't think I can put it on open web due to conference publishers protections. Do you hang out at Rust Zulip server, Rust community discord, the Rust development discord? I'm using the same nick and real name respectively. If you message me I can reply with a link to the draft. Alternatively, the official publication is on 5.-9.7.2020 if accepted, but that's a long way off.

[dbsxdbsx]

@hegza, do you mean that it is feasible to translate pure python script into rust code, even with some 3rd party modules, like numpy, panda or something else?

[hegza]

I'd put it in the basket of "needs more testing", but it does work for e.g. NumPy since Rust has a library that mimics NumPy's interface (ndarray).

My original use case was 800 lines of NumPy linear algebra with differential equations in a 3D for-loop. konchunas/pyrs was actually quite helpful in translating that. If you get even close to Rust-syntax, the Rust compiler will guide you through the rest.

I figured many of those further transformations are obvious from the source code when you have access to AST-information, that's why I tried out that here https://github.com/hegza/serpent-rs.

[dbsxdbsx]

@hegza, thanks for your efforts in this field. I think the best program experienece woud be ---- First write code in python to check logic, then translate it to a both safe and fast native compilable and exexcutable language, which is Rust. And the latter one is better to be done automatically, that is why a python2Rust transpiler is important.

It seems that for those famous python modules, you are taking rust-crates out of the shelf for translation, but how to deal with others that have no corresponding crates in Rust? Say, I am writing reinforcement learning algorithm in python, the module popular used is called OpeaAI Gym for simulating environments, it is quite convenient to use it in python and build custom environment, but it can't take advantage of multi-thread due to the GIL issue in python. Is there a way to work around? (I've see a crate called gym-rs, but it still need a python interpreter).

By the way , as ndarray would help to translate numpy from python to rust, maybe it is time to take a look at polar, which is advocated as a faster panda.

[dbsxdbsx]

@hegza, actually, I would like the transpiled rust code be uncompilable---In this case, it would almost mean that the original python code is not robust enough, maybe more than 2 mutable refereance in python code or some other flaw code there.