Questions about mocsy use and packaging

[jamesorr]

Questions about mocsy use and packaging

[jamesorr]

Regarding packing Fortran in Python, I have used f2py for mocsy. More recently, i have switched to 'f90wrap' (also available on github) for a yet to be released, updated version of mocsy. Are you suggesting to use something else?

The new version mocsy will allow the user to compute the first derivatives of all output variables of the 'vars' routine (e.g., pCO2, pH, CO32-, saturation states) with respect to each input variable (DIC, Alkalinity, temperature, salinity, total phosphorus, total silicon). These first derivatives are also called sensitivities or buffer factors. In addition, the new version of mocsy will provide error propagation.

[emiliom]

@ocefpaf, thanks for your follow ups. I have no experience with f2py, so I have nothing to contribute directly, other than testing to confirm that the docstrings become accessible from Python. Regarding your two options:

1. We write a script that generates the single file from the original Fortran code.
2. I do a better job wrapping all the individual files. (Following more closely what is already done with the Makefile.

I also have no useful opinion there, since I can't help with option 2.

Regarding your intent to create unit tests, that sounds great. I'm happy to help, but not until the week after this one (I'll be traveling in meetings all week). I think we can plan on keeping the notebooks as additional user help and documentation, and your unit tests, well, for unit/code testing.

@jamesorr, great to hear about the upcoming enhancements to mocsy. Thanks also for your detailed responses. I don't have much to add to what Filipe has said. I'm really glad you liked the notebook. We can refine it later and add it to your github repository when it's finalized.

[ocefpaf]

Regarding packing Fortran in Python, I have used f2py for mocsy. More recently, i have switched to 'f90wrap' (also available on github) for a yet to be released, updated version of mocsy. Are you suggesting to use something else?

I use f2py in the #1. That might be hidden by the packaging boilerplate I am using, but when I import Extension and setup from numpy.distutils.core the Fortran extension will be compiled at install time using f2py.

I will take a look how to do this with f90wrap. I never used it before, but it seems to be used to pre-process the Fortran code before passing it to f2py. That might be what we need to avoid the copy-and-paste of the code into 1 source file.

The new version mocsy will allow the user to compute the first derivatives of all output variables of the 'vars' routine (e.g., pCO2, pH, CO32-, saturation states) with respect to each input variable (DIC, Alkalinity, temperature, salinity, total phosphorus, total silicon). These first derivatives are also called sensitivities or buffer factors. In addition, the new version of mocsy will provide error propagation.

Sounds awesome! I am not a CO2 guy. I am here just to help get it packaged, but some people around me are really exited about these additions.

When you say the new version of mocsy. Is that the master branch? Or should I be targeting this PR to another development branch?

[jamesorr]

Regarding the new version of mocsy, I haven't yet put in on github because we are still developing and testing it (with my colleague Jean-Marie Epitalon). It may be another month or two before it is available. es is with dual-number automatic differentiation, which is as accurate as the analytical solution. When it is updated, I'll probably put it on the master branch.

[ocefpaf]

Thanks for the info! That should not affect the efforts to package mocsy.

I just finished reading f90wrap page on GitHub and I do not believe it will help with packing. It does create the Python module automatically but, as a Python guy, I dislike all those auto-generated classes.

To package with f90wrap one would need to run f90wrap first, then f2py, and create a packaged based on the compiled module and python generated files. Those steps seems to add a layer of complexity that numpy.distutils.core was designed to remove. (BTW f90wrap uses numpy.distutils.core to install itself.)

With that said, from a technical (and packaging biased) point of view, I do not recommend using f90wrap. Unless mocsy needs some of the unique features of f90wrap.

I believe we have more flexibility to improve the user experience by writing some thin Python wrapper-functions to the Fortran functions. Or at least just provide a direct access like this.

Either way, the current form of copying all the Fortran code into 1 file is not acceptable. I will re-write the setup.py to be smarter about this.

[emiliom]

I believe we have more flexibility to improve the user experience by writing some thin Python wrapper-functions to the Fortran functions. Or at least just provide a direct access like this.

Either way, the current form of copying all the Fortran code into 1 file is not acceptable. I will re-write the setup.py to be smarter about this.

@ocefpaf, let me know how I can help with this. I think I should be able to pitch in a bit. Besides, that will help me learn more about the wrapping and how you create the conda recipes.

But, not until next week, when I'm back home from meetings ...

[ocefpaf]

@ocefpaf, let me know how I can help with this. I think I should be able to pitch in a bit. Besides, that will help me learn more about the wrapping and how you create the conda recipes.

Sure. I am a little bit swamped these days too. Lot of work in my day job and I am moving to a new place. I will be out for a while.

[emiliom]

Getting back to mocsy ... @ocefpaf, I noticed that you recently looked at options for better fortran wrapping. Have you been able to research this further? Earlier you wrote:

Either way, the current form of copying all the Fortran code into 1 file is not acceptable. I will re-write the setup.py to be smarter about this.

Do you anticipate us being able to make some progress on this in the near future? One thing I'd look forward to (and can offer to help, if you tell me how) is having access from Python to the rich doc strings for each function, as currently found in the Fortran modules. And of course, being able to work directly/automatically off of @jamesorr's master code, to accommodate updates easily.

One small update I can report is that I've edited my mocsy_examples ipython notebook to remove the incorrect statements about suspicious pCO2 results. In Example 3 I now use optgas='Pzero' to match the output csv file from the mocsy repository, which must have been run with that option rather than the default optgas='Pinsitu'. This change also exposes the optgas option, which is an important one that wasn't being used in any of the examples.

Thanks. Off to AGU on Monday, for the whole week. But I can work on this while at AGU, if we can coordinate!

[ocefpaf]

Do you anticipate us being able to make some progress on this in the near future?

It is relatively easy to do. I just could not find the time. (I have am important SECOORA deadline for January.)

I will be able to get back to this during the holidays.

[emiliom]

Great to hear! Good luck with your SECOORA work. No rush.