vdesnoux/solex_ser_recon

Reconstruction disque solaire à partir de fichiers ser spectroheliographie

solex_ser_recon's Issues

Issue with edited SER files

Dear V, I noticed this edge case in this version and also the previous one. If one takes data with multiple scans, then the best way to process them in an automated way is to crop them into two (or more) pieces. The easiest is to use SER Player (but I assume PIPP will work also). The problem is in the editing process some header bytes must get lost and the program will crash because it can't find them. We were able to make your program work on edited files by commenting out 3 lines (see attached figure, Observer, Instrument, Telescope). In our version, we did not see the need for these segments of code so left them out. D&A.

yaml issues

Hi Valerie, I just downloaded both the Python code and the Windows version. There are some minor issues with the GitHub version.

-- The requirements should be updated to include 'pyyaml' (i.e. 'pip install pyyaml')
-- The README Requirements should be updated to include both 'serfilesreader' and 'pyyaml'
-- The included 'inti.yaml' file does not run correctly (it presumably has some missing data). In fact, if the file does not exist, the program creates a new one which does work. So best to not include the 'inti.yaml' in the repository.
-- Minor spelling corrections in the English version: in 'Inti_recon', 'Width and Height of SER frames' (instead of 'Wide and Heiht') and below that, 'Number of frames' (instead of 'Frames number').

By the way, I notice your version generally does better with H-alpha and my version works better on Calcium. I'm not sure why exactly.

Small bug

FYI, For a strongly tilted image I wanted to process, I needed to force the "crop" variable to 0. That allowed the processing to continue, but it then halted at line 912 of Inti_recon.py

The call to detect_edges needs to have
crop=crop
added to it there, as it is in other calls.

Downsized examples of the raw and clahe image are attached
Hope this helps. Any advice on using "crop" in these circumstances would be appreciated!
Matt

Improvement for prom image

Hi Vdesnoux/thelondonsmiths,

I'm seemingly not able to fork thelondonsmiths version of the main code, so that I can upload my own tweak. So I'm posting this here and apologize if there's a better way to approach this.

Bottom line : in creating the prom/protus image, I believe it makes sense to cut the disc of the sun out first, and then rebalance/process the result. I attach here the relevant change to thelondonsmiths SHG_MAIN.py, for lines 278-306. I've added comments (in English ... sorry!) that I hope make the idea clear.

Mconsidine
`
#image seuils protus
frame1=np.copy(frame)
#MattC cut out disk first
#MattC BTW given default setting of disk_display this was never executed
#if ratio_fixe==0 and disk_display==True:
#MattC for now, let's always do this
if 1==1 :
#MattC - demo simple processing of proms seperate from disc settings
clahe2 = cv2.createCLAHE(clipLimit=1.0, tileGridSize=(1,1))
cl2 = clahe2.apply(frame1)
x0=cercle[0]
y0=cercle[1]-1
r=int(cercle[2]0.5)+1
#MattC - turn disc to black, more aesthetically pleasing, IMHO
frame1=cv2.circle(cl2, (x0,y0),r,0,-1)
#MattC adjust scaling
Seuil_haut=np.percentile(frame1,99.9999)
Seuil_bas=0
print('seuil bas protu', Seuil_bas)
print('seuil haut protu', Seuil_haut)
frame1[frame1>Seuil_haut]=Seuil_haut
fc2=(frame1-Seuil_bas) (65000/(Seuil_haut-Seuil_bas))
fc2[fc2<0]=0
frame_contrasted3=np.array(fc2, dtype='uint16')
#MattC code that was at this line moved up above ...
if flag_result_show:
cv2.imshow('sun3',frame_contrasted3)
#cv2.waitKey(0)

`

poly error?

In running v3.2.2 from command line in Ubuntu, with python 3.8, I get

$ python3 Inti_ser_recon.py
Traceback (most recent call last):
File "Inti_ser_recon.py", line 344, in
poly.append(float(my_dictini['poly_slit_a']))
KeyError: 'poly_slit_a'

This comes about if the inti.yaml file is present from the github download. Ie, in a new install it doesn't get overwritten, so when the code looks to select poly_slit_a, it doesn't exist yet...

Matt

65000--> 64000

Hi Valerie, I am looking at a few ways to speed up the code. One question I have is why reset the value of IntensiteRaie to 64000 if it is above 65000 in the code below:

try:
IntensiteRaie[j]=(img[j,LineRecal+dx] (1-deci)+deciimg[j,LineRecal+dx+1])
if img[j,LineRecal+dx]>=65000:
IntensiteRaie[j]=64000

AVI support

I completely understand the reason/need to use 16-bit data for this sort of exercise. That said, some have taken spectra/slit images and used the AVI file format, which only handles 8-bit data.

I've taken a version of Solex_ser_recon (the latest version here from Valerie) and posted it to
https://github.com/mconsidine/Solex_ser_recon_AVI

If it would be easier to try to get pull requests to work, I can try that. Though I am a github-contributor newbie...

One thing to note is that I had problems - for reasons I don't quite understand - when the ellipse fitting code was reached. For a demo, I was using the 16_33_15.ser file on the Solex site, which was then successfully converted to AVI via PIPP.

If I set the desired ratio to 1 (thereby avoiding the ellipse code) then I get images, but they are obviously skewed. So there is still work to be done but the framework is all there, I believe.

Tilt correction not applied

Hi,
When I run the Solex_ser_recon code on the 16_33_15.ser file from the Solex site, the tilt correction is not applied. When I upload it to Inti, it is.

It turns out line 1073 here of Inti_recon.py has a two part condition that needs to be met, whereas the code in Inti (which has the correct output) looks like this:
if flag_nobords==False:

Perhaps an edit/update didn't get transferred to the Solex_ser_recon code?
Matt

Ellipse fitting question

Hi Valerie (and anyone else watching),
I have run into a "weirdness" that I can't correct for and thought you or someone else could shed some light on the problem.

I have a scan sequence that is "backwards" from the way the sun was projected on the slit and I figured that it would be easy enough to correct by flipping the "Disk" image and then continuing on with the processing. The reconstruction comes out well without this, but is just reversed.
I expected that flipping the input to the ellipse routine would generate essentially the same result but with the correct orientation. But that is not the case, it comes out distorted.

Do you have any suggestions as to why this would be the case? I've been wrestling all day with it and have gotten nowhere. I thought it might be some problem I introduced into my coce, but I just pulled down DJIBB's version, changed the one line to flip the Disk horizontally, and I get the exact same outcome.

Thanks for any thoughts,
Matt

UnboundLocalError: local variable 'r' referenced before assignment

Hi,

Thanks for the amazing work, this looks really cool :)

I have downloaded Inti from here, and running it on my SER file. After some initial quick processing I get the following error (sorry for the display, I've had to use a free desktop recorder to have access to the actual error, because the crash made the window to disapear)

I've also ran it on the observation 12_08_34 avilable here and it ran without problem. Seems like a bug in the code, but I don't understand why my obervation is bringing the code there while the example observation doesn't.

I'm not sure what to try next, would you have any idea? My observation is not really good, but I can post it somewhere if this can be of any help.

Best wishes!

AVI support

For what it is worth, I'll try to keep an AVI-friendly version of Valerie's "core" version of solex_ser_recon here
https://github.com/mconsidine/Solex_ser_recon_CORE_AVI

For this version, I won't make any changes to any of the functions/approaches taken in the "core" version, other than to try to make it AVI/8-bit compatible.

Also, I'm keeping this version on a separate github account from the AVI-friendly version of thelondonsmith's version, so as to not confuse anyone.

A version that will have some features I will be playing with, e.g bandwidth selection (which I already have working), turbulence adjustment with phase correlation (working in another app and needed to be ported) and slant estimates from solar positioning based on timestamps, but that others' may not find worthwhile or useful will be at
https://github.com/mconsidine/Digital_SHG

I hope this effort is useful and not confusing.
Matt

Bandwidth of the image

Hi Valerie,
How does the "bandwidth" work in your program? Is there a default pixel width used and can this be changed? For example, I can understand that at H-alpha, you probably want the minimum bandwidth which the SolEx can provide. But in Calcium, a much wider bandwidth is sufficient and may even be desirable.
Thanks.

Question about codeblock

Hi,
I have a question about the following block of code in Inti_recon.py:
`# creation du tableau d'indice des lignes a corriger
l_col=np.where(hcol!=0)
listcol=l_col[0]

# correction de lignes par filtrage median 13 lignes, empririque
for c in listcol:
    m=img[c-7:c+6,]
    s=np.median(m,0)
    img[c-1:c,]=s`

It is the loop I am wondering about. Let's say that "c" is 10 at some point. So "m" refers to img[3:16.] and then a median of this is calculated along axis=0, if I understand correctly.

Then, the section in img referred to by [9:10,] is replaced by this median.

When the loop advances to 11, "m" refers to img[4:17,] and a median is calculated again.

However, that median now includes in it the results of the median calculation when c was 10, as that result was copied back to "img", rather than a copy of "img".

Do I understand this correctly? It would seem to me that would not be an intended outcome, but if so I would be interested in the logic behind it.

Thank you,
Matt

A few issues with ellipse

The new ellipse feature is very nice but does fail sometimes. I have tracked down how it fails but did not find where in the code the issue is yet. First of all, I notice that the .txt file does not save the SY/SX ellipse parameter when in automatic mode. This is information that probably should be saved to the .txt file. But I could read the SY/SX data on the console.

When the ellipse function fails, for some reason it performs the Y/X function but then does not do the tilt angle function. But I could take the SY/SX parameter as printed on the console, rerun the program, and manually enter the SY/SX number. With this convoluted method, it would produce the correct circular image! See below for console screenshots in the "fail" in auto mode then the "success" with the manual input of the SY/SX parameter. The data is the one I mentioned in my post on solex-project (data from http://www.astrosurf.com/solex/download/16_33_15.zip).