Ten years challenge: fermions at unitarity

[ev-br]

Ten years challenge: fermions at unitarity

[rougier]

I think we're waiting for @vahtras review (gentle pressure :) )

[rougier]

Thanks for your (late) submission :) We'll assign an editor soon.

@pdebuyl Can you handle this submission (Physics/Python/Fortran) for the Ten Years Reproducibility Challenge (only # reviewer needed) ?

[pdebuyl]

As editor thus?

[rougier]

@pdebuyl Yes, sorry, I should have explained.

[pdebuyl]

ok

[pdebuyl]

I mean, "I will edit" :-)

[pdebuyl]

@jochym could you review the submission here? If this is your first review for ReScience, I will guide you through the process.

[pdebuyl]

@dombrno could you review the submission here? If this is your first review for ReScience, I will guide you through the process.

[pdebuyl]

@vahtras will review the paper. Thank you Olav!

[pdebuyl]

ping @vahtras

[pdebuyl]

@berquist would you review the submission "fermions at unitarity" for ReScience ? If this is your first review for ReScience, I will guide you through the process.

[vahtras]

Is there a howto for reviewers @pdebuyl ?

[pdebuyl]

Hi @vahtras

We have reviewer guidelines here: https://rescience.github.io/edit/

The most specific part of the review, in relation to ReScience, is to actually run the code and verify the claims of reproduction that are stated in the article.

Specifically for the ten-year challenge

• We only require one reviewer.
• The requirement to have a readable and reusable code are relaxed a bit. It would make little sense for the authors of this issue to perform a full modernization of their code.
• The author should provide in the article their reflection on the longevity, quality and retrievable character of their old code and of the corresponding environment (language, platform-specific code, proprietary tools, etc).

[ev-br]

In this specific case, rerunning the full set of calculations might not be very practical, since it's going to require a non-negligible cpu time on a cluster.
Not sure what are implications though.

[pdebuyl]

How many core-hours ? (or core-days / node-days depending on the hardware you have used)?

[ev-br]

The largest runs in the supplement repository are some 120 CPU hours on 24 cores. Multiply it by about 1.5-2 for thermalization.
Smaller system sizes are much faster, some 4-10 cpu hours each, if a partial verification is OK. (rerun small system sizes on the reviewer's machine, rerun the fits with a mix of reviewer's data and my data or somesuch).

[pdebuyl]

Well, this seems indeed costly. @vahtras do you have the resources for the small systems verification? (4-10 cpu hours each) ? I could execute that if necessary.

[pdebuyl]

Hi all,

I see only now that the review process is frozen here. @vahtras what do you think of the computational requirements?

[vahtras]

I apologize for the delay, will have some time for a look now.

…

On Thu, Jul 30, 2020 at 10:10 PM Pierre de Buyl ***@***.***> wrote: Hi all, I see only now that the review process is frozen here. @vahtras <https://github.com/vahtras> what do you think of the computational requirements? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLLJBN4U7N25L6JXIOCHODR6HHTHANCNFSM4M3DOYCA> .

[pdebuyl]

Hi @vahtras thank you for getting back to us :-) Other papers are still in the pipeline, so this should be ok.

[ev-br]

Anything needed from me at this stage?

[pdebuyl]

@vahtras do you still plan to review the article?

[vahtras]

Yes, finally. First question: the makefile has been hardcoded for Intel compilers. Does it build with GNU?

[ev-br]

It certainly does. ev-br/10yr_repro_challenge_35@837038f is the relevant makefile.

I then simply commented it out when transferring to cluster/intel compiler instead of adding platform detection (more brittle stuff to debug ten years down the line)

The change from gnu on a laptop to intel on cluster is here:
ev-br/10yr_repro_challenge_35@3603020

EDIT:the switch from gnu to intel is
ev-br/10yr_repro_challenge_35@acc2974

[rougier]

@pdebuyl Any progress?

[pdebuyl]

Hi @rougier sorry about this. @vahtras is this still doable for you? In the meantime, I will try to find another reviewer.

[vahtras]

please find another reviewer

…

On Tue, Jun 22, 2021 at 8:56 AM Pierre de Buyl ***@***.***> wrote: Hi @rougier <https://github.com/rougier> sorry about this. @vahtras <https://github.com/vahtras> is this still doable for you? In the meantime, I will try to find another reviewer. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLLJBL3JZMRVAZIML5KY3DTUAX27ANCNFSM4M3DOYCA> .

[rougier]

@pdebuyl I think you need to find a new reviewer. You can use the @ ReScience/reviewers notification if necessary.

[pdebuyl]

@rougier I have started to do so.

[rougier]

Good. Any progress?

[pdebuyl]

Yes, I have a candidate reviewer but am waiting for confirmation before putting it here.

[rougier]

Gentle reminder

[pdebuyl]

Hi @rougier unfortunately I "lost" my candidate reviewer. In all fairness (and despite the already long delay), I'd prefer to wait mid-august to look again the mid-summer is a really bad time to find reviewers!

PS: I'll also consider reviewing the paper and have Konrad or someone else be editor (august as well I am not in office)

[rougier]

Ok. Note that according to our process, you can serve as 2nd reviewer while being the editor.

[rougier]

Any progress?