Allow function definition syntax

[Ricyteach]

Allow function definition syntax

[connorferster]

Hi @Ricyteach,

tldr;
I have released a new version that has an updated API for the decorator that should allow for the functionality you are describing. It uses standard Python function definition syntax. The README.md has been updated to document the new API for the decorator. Additionally, I have created a demo notebook to demonstrate the functionality. Rendering support for 1D arrays has also been added in v0.6.0.

History
This project started when I had the exact same thought: in engineering, we are always writing and re-writing the same equations over and over again in our personal programs in Excel or MathCAD or when we perform calculations by hand. What if we could create a library of standard calculations that we could piece together to quickly compose a coherent calculation process?

I had tried various kinds of classes to try and create an Equation class that could fit the purpose but I was not terribly successful with it or satisfied with any of the results.

Finishing my last year of engineering school, I started using Jupyter Notebook for my homework and tried to create customized Latex templates to produce PDFs that didn't look overly look like "computer code", which would be unacceptable to submit. I made heavy use of dynamically rendered markdown cells to mild success. It occurred to me that really I would need to learn Latex and find a way to convert my Python code to Latex for rendering to improve my output.

I borrowed a book on Latex from the library and set about writing some prototype functions that could inspect a Python function, performing some basic parsing, and run some conversion functions in a Jupyter Notebook. This had some success and I created the first version of handcalcs that was called CalcWriter.

CalcWriter was designed to read a single .py file which contained a single main() function. The calculation code of the main() function would would be an entire calculation process on the scale of, say, the bending capacity of a glue-lam beam. Another file might have a function to then calculate the shear capacity of a glue-lam beam. The CalcWriter class would "load" the .py file and wrap it so that the CalcWriter class was then callable as though it were the main() function in the file. Because the CalcWriter object was callable, I could use it in larger programs, combining several CalcWriter objects and making use of ipywidgets to create a customized calculation GUI with sliders and drop downs.

Once the inputs to all the CalcWriter objects were returning appropriate outputs, I would then call the .print2file() method on each object to render the function code in Latex as separate PDFs that I would then combine as one document. However, making these programs was still a lot of work and not practical to put together calculations on-the-fly for a "real project" that was happening "now". These programs were also quite rigid: sometimes, I would need to "tweak" the functions in the .py file for a particular purpose, which made them less general.

I decided that this approach was not flexible enough and that I really wanted the ability to just write a quick calc in a Jupyter cell and have it render out as Latex, showing all substitutions. This way, I could have that on-the-fly flexibility I wanted and, by composing a Jupyter Notebook well, I could create this "library of calculations" that I originally had with CalcWriter. Furthermore, using tools like papermill, I could parameterize and iterate over my calculations in a way that is similar to what I could do with CalcWriter.

Coming back full circle
So, when you asked about being able to create modularized functions that could display as Latex with handcalcs, I had to laugh at myself because I realized that, with the new structure of the handcalcs code, I could have easily implemented this capability with the new handcalcs BUT I HAD NOT DONE IT YET: the idea that was the very genesis of this whole project over the last two years, I had not carried through to completion :)

At this point, I really like how I can use handcalcs in an ad-hoc, unstructured way by typing up some calcs, throwing a couple of markdown notes around them and printing a great looking PDF with an nbconvert Latex template that suppresses inputs.

With the decorator, I like how it can be used in a similar manner to how I originally, and you, envision as a way of modularizing functions that can auto-render as Latex.

Let me know your thoughts on this. Does the decorator, as it works now, allow the functionality you are aiming for?

[Ricyteach]

Ohhhh nevermind don't bother with that: it's a choice that pint is making internally.

[Ricyteach]

Well that was easy. Just need to change the Quantity.default_format to ~.

EDIT: ah, actually it turns out you can specify default_format on the unit registry; don't have to modify the Quantity class field.

EDIT: and it also works with numpy arrays! But the commas are missing... can that be fixed?

[connorferster]

I am interested in hearing more because I don't think I quite understand yet :)

There is the @handcalc decorator that you can use on your own functions to generate the equivalent LaTeX code whenever the function is called: the parameters used to call the function will be the values substituted in the LaTeX code. This was originally designed for a non-Jupyter environment but the resulting LaTeX output can be surrounded by "\\[" ... "\\]" and it can be rendered with the Latex() function from IPython.display. See the section "Basic Usage 2" in the readme. If this is not quite how you mean, I guess I have two questions:

1. Are you thinking about a use case in a Jupyter environment or would this be for a non-Jupyter environment?
2. What would be rendered out? Would it be the LaTeX for the for the code in the function with the values used to call the function?

If the @handcalc decorator is not quite what you mean, tell me more about how it would work differently. Perhaps the @handcalc decorator can have an argument to adjust its behaviour to fit what you are describing.

Let me know!

[connorferster]

Hi @Ricyteach,

I have also just added a # Symbolic comment tag feature that may be a part of what you are thinking of. Have not heard back from you on this yet. Let me know your thoughts.

[Ricyteach]

Sorry for the delay. I haven't looked at the new tag yet; I will though.

Here's a very specific use case; hopefully it will open up some ideas about what it is I'm after. I'm a civil engineer and I work with building codes a lot, and I produce a lot of calculations attached to appendices of reports using equations right out of the building code. Right now I use Mathcad and Excel for this (Mathcad is a very nice WYSIWYG math editor/calculation tool; I use Excel because everyone has it and it's easy to send spreadsheets to people rather than Mathcad files).

There is almost an endless number of building code equations; for example, there is a load combination equation (ASCE 7 ASD #7) that looks like this:

(D is dead load, W is wind load)

I have a Mathcad sheet I currently use to check this load combination, with the equation defined (for reuse) like this:

The actual checks performed in the sheet-- later, after the load combination has been stated above-- look something like this:

(Where capacity is the capacity of the thing being checked, and FS is factor of safety; the =1 part on the end just uses 1 as a stand-in for True, and it means the load combination passed. It's just the result of the inequality statement.)

So the idea I have here is that you'd define a function that gets reused over and over, and rendered in the sheet at definition time and again each time it is used.

When you are in the context of expecting typed calculations that look similar to how they look when written "by hand", using lambda and the usual def function definition syntax is, frankly, ugly. It doesn't look remotely like hand calculations.

And this is of course a syntax error:

combo_7(D, W) = 0.6*D + 0.6*W

So, I have been experimenting with the idea below, which abuses __setitem__ for defining a function:

from sympy import tan, lambdify

class Equation():
    def __init__(self, name):
        self.name = name
    def __getitem__(self, values):
        return self.func(*values)
    def __setitem__(self, vars, expr):
        self.expr = expr
        self.vars = vars
        self.func = lambdify(self.vars, self.expr, "numpy")
    def __repr__(self):
        try:
            return f"{self.name}[{repr(self.vars)[1:-1]}] = {self.expr!r}"
        except AttributeError:
            return f"{type(self).__name__}({self.name!r})"

With the above idea, here's how you'd abuse the [ ] brackets to define a nice-looking, handcalcs kind of function:

from sympy.abc import *

# ASCE 7 ASD load combination #7
combo_7 = Equation("combo_7")
# nice looking function definition
combo_7[D, W] = 0.6 * D + 0.6 * W

Usage:

>>> import combo_7
>>> combo_7[100, 100]
120.0
>>> import numpy as np
>>> combo_7[np.array([100, 200]), np.array([100,200])]
array([120., 240.])

With some effort I could, of course, give the class a dynamically defined __call__ method so that this works, too:

>>> combo_7(D = 100, W = 100)
120.0

Anyway, just brainstorming here. I'd really love to replace Mathcad with Python some day. Your handcalcs tool is really nice and I'm hopeful it is the beginning of that possibility. But being able to define functions with parameters in a nice-looking way is essential.

[connorferster]

Hi @Ricyteach,

I think I have written that exact same Equation class at the very start of the journey that lead to handcalcs in its current form. This will require a much longer response which I will be able to get to in the next few days but I look forward to writing it. Stay tuned!

The new comment tag is not what you describing. However, the short answer is that the @handcalc decorator, I believe, will be able to achieve what you are describing. It would benefit from a couple of small and easy feature additions on my end to maximize it's utility in this way.

[Ricyteach]

Sounds great, look forward to learning from you!

EDIT: btw I did get a chance to look at the new # Symbolic tag and it's great!

[Ricyteach]

It turns out that the package works mostly as-is for the Equation class. Check it out.

However it's not without its problems. It's not unusual to want to use the same variable names for global parameters and for equation input parameters (see the error created by the last two cells).

Additionally, "calling" the function using 1 dimensional numpy arrays seems to work without much problem, except the output is missing a comma between values. The weird part is the comma shows up fine when it is a regular list, rather than a numpy array. Since it works fine for lists I'm wondering if the problem with numpy arrays could be fixed.

[Ricyteach]

Well I have to say, going by the example notebook (haven't tried the new API yet), this looks pretty much perfect-- exactly the kind of thing I was after. I especially like that the %%render magic isn't required anymore. Incidentally: my company has PE licenses in all 50 states and about 2/3 the Canadian provinces, so I'm very familiar with those NBCC load combinations.

I enjoyed the background history and have been on a very similar journey, though I have not had the time to devote to developing a solution. If I had been aware of jupyter and python in undergrad to the extent I am now, I probably would have (instead I relied heavily on Mathcad). But I am thrilled that you have, and cannot thank you enough for your efforts.

I'll give the updated API a try in the coming days/weeks and will be sure to provide feedback! A couple things that do come to mind immediately:

• I haven't delved too deeply into your units library yet, but I did notice it seems to only support SI units; so unfortunately, being on US building codes, this is a non-starter for me, and any other American engineer.
• Have you considered trying to support pint? I did try to use pint in handcalcs the other day but it didn't seem to work too well. I think if you supported pint, you'd see a lot more adoption... I think it's the most popular units library (at least, it's the one I have seen people use the most).
• Happy to see the example notebook shows an improved display for np arrays so it properly places commas between values; did you tweak that, or was my notebook doing something odd?
• Can you nudge in the right direction to learn how to use this nbconvert Latex template with suppressed inputs? I also need to learn to use papermill; I'm familiar somewhat with what it is, I think, but haven't used it yet.

[connorferster]

Hi @Ricyteach,

Some quick responses for you:

• The units library is simply SI units-based and so any units system that is defined by the SI units (such as US Customary units) are compatible. The 'structural' environment comes preloaded with the common SI and US units (e.g. ksi, kip, lb, etc.). However, I desperately need to update the documentation for this.

• I have tried using pint many times but found that, while it is well suited for scientific work (especially with its compatibility with uncertainties), it is not very convenient for fast engineering work. One of those reasons is that it does not auto-cancel units. That is a separate step one has to take. It is certainly the most popular units environment. It's not that I don't support it, per se, but that pint does not fully support IPython/Jupyter. Jupyter has defined an API for displaying rich representations of Python objects, such as HTML and Latex representations, through its methods _repr_html_ and _repr_latex_, respectively. Because handcalcs was developed to be used primarily in the Jupyter environment, it relies on an object's _repr_latex_ method. Pint has implemented its Latex representation through its string formatting interface, which means it won't print to PDF in Jupyter as well as it could. To be clear, pint is excellent. It's just not very well suited for the fast, 'at-my-fingertips' way of working that I would like. That being said, forallpeople is not as well suited for scientific work because it does not carry precision beyond six decimal places.

• For numpy, I had to add a few lines of code to the function that handles object conversions to str representations.

• Yes! This is the template I use. However, I have never been able to get it activated through the proper Jupyter config interface. Instead, I simply replace the default template in my Jupyter site-package directory, article.tlpx with classic.tlpx, renaming it to article.tlpx. To suppress input cells, change these lines

((* block maketitle *))
((* endblock maketitle *))

to this

((* block maketitle *))
((* endblock maketitle *))

((* block input_group *))
((* endblock input_group *))

This article has some good explanations.

[Ricyteach]

Thanks for the responses.

I understand about pint-- at one point I was messing around with modifying the source code to utilize the _repr_XXX_ jupyter expects (like you I was having problems with the way it was implemented), but like most of my ideas, it got put away somewhere and I don't even know where it is now. Probably wasn't much good anyway. :)

I'll look closer at the units library, then. I did only look at the documentation before. No problem on it not being updated, I totally understand.

Thanks for the article! That will help a lot.

I think I will still have come comments related to this original issue. I'll get back to you again on it soon.

[Ricyteach]

Actually after trying it again with your latest example code, pint does seem to do OK. I haven't dug into it for while (I'm sure you have) but I'll mess around with it to see if I can get it to display abbreviated units (as you know, in pint you usually do this with a formatting code,like f"{1*u.m:~}".

[connorferster]

Changes numpy array display? Weird... WHAT IS IT DOING?!

I will take a look at it :)

[connorferster]

Hi @Ricyteach,

I will close out this issue next week unless you have anything to add.

Thanks!

[Ricyteach]

Thanks Connor! I might come back and say a thing or two at a later date but for now I think we're good....

[BTWS2]

@Ricyteach Did you try pint with the 1.0.0 release of handcalcs?

#%%
import handcalcs.render
from pint import UnitRegistry
u = UnitRegistry()
u.default_format = "~"

#14 
%%render
length = 4 * u.km
width = 3.5 * u.mm
area = length * width
length

Is it still possible to use pint with forallpeople?

[Ricyteach]

@connorferster Just want to say: the 1.X versions of handcalcs are THE BOMB DOT COM. It has been a while since I've used this and BOY have you been busy with it since 2020.

With all these new features and the ability to control everything so well I might actually finally be able to kick my Mathcad habit to the curb!

[Ricyteach]

Hi @connorferster, quick question.

Is it possible to modify the functionality of displaying calcs upon import so that you can later adjust the handcalc() options? I'm talking about the override tags and precision, mostly.

Right now when you decorate a function for display on import, the only way I have found of specifying your precision and display options is to apply them at function definition time:

from handcalcs.decorator import handcalc

@handcalc(jupyter_display = True, precision = 1)
def NBCC2015LC(DL: float = 0, SDL: float = 0, SL: float = 0, LL: float = 0, WL: float= 0, EL: float = 0):
    LC1 = 1.4*DL
    LC2a = 1.25*DL + 1.5*LL
    LC2b = 1.25*DL + 1.5*LL + 0.5*SL
    LC3a = 1.25*DL + 1.5*SL
    LC3b = 1.25*DL + 1.5*SL + 0.5*LL
    return locals()

However it would be a lot better if you could adjust these in the notebook. I have tried it with the %%render magic, and it appears to work (the precision does change). However instead of rendering the imported math, it displays the LATEX.

https://github.com/Ricyteach/wysiwyg_math/blob/master/decorator_demo_modify_precision.ipynb

Is this a simple enough thing to fix?