neuroml/org.neuroml.export

Export from NeuroML & LEMS

org.neuroml.export's Issues

rate helper variables in mod files are declared with the wrong dimensionality

"rate" helper variables (e.g. rate_A for a variable called A) created when exporting to mod files are declared to be nondimensional, whereas they should have, for example, dimension 1/time for a nondimensional variable. In other words, if A is nondimensional, rate_A should have dimension 1/time.

This makes the created mod files fail the modlunit test.

q10 for Kinetic Scheme based channels

Neuron (mod) generation for KS based channels does not honour q10 corrections.
This is due to rateScale being generated correctly, but not being applied to alphas/betas.

[Bug] JSONSerializer always puts all sections into an `all` NEURON section list, even if the NeuroML cell morphology has a `all` segment group that does not include all segments/segment groups in it

Describe the bug
I have a cell morphology with sections in a myelin group. In the original NEURON model, this group does not seem to be part of the allsection list. So any parameters set for the all sectionlist are not set for sections in the myelin sections.

When testing out this morphology, I see that the generated hoc still adds the myelin sections to the all section list. In fact, it always seems to add all sections to the all section list irrespective of the all segment group's contents in the NeuroML representation.

So, as a result, there's a difference between the biophysics in the original NEURON hoc where mechanisms added to all did not end up the myelin sections and the NeuroML generated hoc where they do.

To Reproduce
Convert this NeuroML cell representation to NEURON and compare the all segment group and the all section list:

https://github.com/sanjayankur31/Human-L2-3-Cortical-Microcircuit/blob/feat/neuroml/NeuroML2/HL23SST.cell.nml

Expected behavior
I think if the NeuroML representation has a all group, the hoc file should use this information, not create a new all section list with all sections

Additional context

While looking at the code that handles this conversion, I noticed that foundAll is initialised to false but then never reassigned:

org.neuroml.export/src/main/java/org/neuroml/export/neuron/JSONCellSerializer.java

Line 300 in 766d52d

boolean foundAll = false;

which means that this block is always entered:

org.neuroml.export/src/main/java/org/neuroml/export/neuron/JSONCellSerializer.java

Line 371 in 766d52d

if(!foundAll)

That explains why the all section list always has all sections.

There's a check to see if the segment group is all:

org.neuroml.export/src/main/java/org/neuroml/export/neuron/JSONCellSerializer.java

Line 303 in 766d52d

if(!grp.getId().equals("all")) // I'll calculate this here...

I expect the idea is that an all segment group needs to be handled differently, but I don't see an else block there that does this.

A dirty hack to not let the lower code block always be executed seems to workaround this bug (and my biophysics now match exactly too):

5627e12

Neuron exporter generates code using non-parallel construct

Generated modfiles for synapses use state_discontinuity, nrnivmodl emits a warning:

Notice: Use of state_discontinuity is not thread safe
Notice: Use of state_discontinuity in a NET_RECEIVE block is unnecessary and prevents use of this mechanism in a multithreaded simulation.

[BUG]: `getPulseGenMethod` for Brian 2 doesn't properly account for the simulation time line

getPulseGenMethod fails when a pulse occurs beyond the simulation time line (steps < int("+(del_si+dur_si)+"/defaultclock.dt)). The getPulseGenMethod should be adjusted to account for this.

org.neuroml.export/src/main/java/org/neuroml/export/brian/BrianWriter.java

Line 435 in ce18e86

" np.repeat(0, (steps - int("+(del_si+dur_si)+"/defaultclock.dt))) ) ) * "+unit+" , \n"

Replace steps - int("+(del_si+dur_si)+"/defaultclock.dt) with max(0, steps - int("+(del_si+dur_si)+"/defaultclock.dt)).

[neuron-writer] how can one set the tolerance value?

I'm running some sims here and getting lots of nan values. Apparently, to workaround this, one has to reduce the tolerance value used in NEURON:

https://nrn.readthedocs.io/en/8.2.2/hoc/simctrl/cvode.html#CVode.atol

How would one do this while using NeuroML to generate NEURON code? (I don't think we have a way to do it yet?)

A workaround would be to generate the python script and then add a line to it, but this isn't ideal.

Edit: tasks:

• add Meta tag to LEMS schema (optional, since we don't always validate LEMS files against the schema)
• annotate Meta in jLEMS (https://docs.neuroml.org/Userdocs/LEMS_elements/Definingcomponenttypes.html#meta -> needs improvement)
• check that meta tag contents are exposed in writers
• add special case for handling cvode in neuron writer
• add example for testing, also in NEURON Showcase
• update LEMS_TEMPLATE in pyNeuroML to include Meta
• update LEMSSimulation to add method to include Meta
• regenerate docs.neuroml.org with update Meta annotation
• add special case for handling cvode in netpyne writer
• update lems simulation documentation to include netpyne options

Conversion to Neuron fails when there is no target component

When we try to convert a model which has no target component it fails throwing the following exception:

[2015-06-17 12:50:09.141] ERROR http-bio-8080-exec-47 System.err Caused by: org.lemsml.jlems.core.sim.ContentError: No runnable components have been specified
[2015-06-17 12:50:09.141] ERROR http-bio-8080-exec-47 System.err at org.lemsml.jlems.core.type.Lems.getTarget(Lems.java:334)
[2015-06-17 12:50:09.141] ERROR http-bio-8080-exec-47 System.err at org.neuroml.export.neuron.NeuronWriter.getMainScript(NeuronWriter.java:220)

If we want to convert the model in order to run a simulation this is not a problem as we will always have a target component. However when we just want to convert the model (and maybe download it from Geppetto) this becomes a problem.

@pgleeson Is there any reason why we should have a target component in order to convert a model (TBH I haven't look at the NeuronWriter code in detail)?

Things to be improved

• Proper exception handling
• Files in src/test/resources/tmp shouldn't be in the repo
• When a model is not supported return a report. (this was working but I have to remove it during the refactor task)
• All the files for testing should be contain in the export projects. At the moment some of them are in the project while some others are in neuroml.model
• Remove all main methods from writers. They should go either to Travis or to jUnit tests.
• These two lines

MinimalMessageHandler.setVeryMinimal(true);
 E.setDebug(false);

can be found in several writers. Is this sth we can generalize? Can we move them to some abstract class or utils?

• Validate if outputFolder exists and generate proper exception if this is not the case
• Refactor remaining hardcoded file extension to new Formats enum class (some of them have already been moved)
• org.neuroml.import has a dependency with org.neuroml.export just because a util method. Shall we remove this dependency and add a new dependency between org.neuroml.model and org.neuroml.export? Maybe this util method can be moved to org.neuroml.model
• move outputFileName to an abstract class
• add a unit test for jNeuroml
• Refactor neuron code
• Refactor infowriter code
• Improve and generalize logging system
• Documentation
• Improve junit tests. They should extend from a super class as functionality is similar in most of the cases or at least create a util class. Also we should improve what we are checking/validating at the moment. Most cases we just validate that the file exists but some extra validation would be nice
• Implement getSupportedOutputs functionality for a specific file. This method will be used from Geppetto to check which conversions can be done in neuroml.export lib.

NeuronWriter does not create unidirectional gap junctions?

I have a nml file with 1 GJ (between AVBR and DB1)

<gapJunction id="neuron_to_neuron_elec_syn_15conns" conductance="0.3378nS"/>
...
<electricalProjection id="NC_AVBR_DB1_Generic_GJ" presynapticPopulation="AVBR" postsynapticPopulation="DB1">
    <electricalConnectionInstance id="0" preCell="../AVBR/0/GenericNeuronCell" preSegment="0" preFractionAlong="0.5" postCell="../DB1/0/GenericNeuronCell" postSegment="0" postFractionAlong="0.5" synapse="neuron_to_neuron_elec_syn_15conns"/>
</electricalProjection>

pynml LEMS_c302_C2_AVB_DB.xml -neuron creates the ..._nrn.py file with following content:

# ######################   Electrical Projection: NC_AVBR_DB1_Generic_GJ
print("Adding electrical projection: NC_AVBR_DB1_Generic_GJ from AVBR to DB1, with 1 connection(s)")

h("objectvar syn_NC_AVBR_DB1_Generic_GJ_neuron_to_neuron_elec_syn_15conns_A[1]")
h("objectvar syn_NC_AVBR_DB1_Generic_GJ_neuron_to_neuron_elec_syn_15conns_B[1]")

# Elect Connection 0: cell 0, seg 0 (0.5) [0.5 on a_AVBR[0].soma] -> cell 0, seg 0 (0.5) [0.5 on a_DB1[0].soma], weight: 1.0
h("a_AVBR[0].soma { syn_NC_AVBR_DB1_Generic_GJ_neuron_to_neuron_elec_syn_15conns_A[0] = new neuron_to_neuron_elec_syn_15conns(0.5) }")
h("a_DB1[0].soma { syn_NC_AVBR_DB1_Generic_GJ_neuron_to_neuron_elec_syn_15conns_B[0] = new neuron_to_neuron_elec_syn_15conns(0.5) }")
h("setpointer syn_NC_AVBR_DB1_Generic_GJ_neuron_to_neuron_elec_syn_15conns_A[0].vpeer, a_DB1[0].soma.v(0.5)")
h("setpointer syn_NC_AVBR_DB1_Generic_GJ_neuron_to_neuron_elec_syn_15conns_B[0].vpeer, a_AVBR[0].soma.v(0.5)")

Am I right that the result is a bidirectional GJ?

I got the expected output without the 'A' part. I did try to comment out the 'A' part after executing pynml LEMS_c302_C2_AVB_DB.xml -neuron and then nrnivmodl && nrngui -python LEMS_c302_C2_AVB_DB_nrn.py to check the output.

Neuron mechanism fails with MacOS .dylib files[Bug]

In short, the command java -Xmx2G -Djava.awt.headless=true -jar "jNeuroML-0.11.1-jar-with-dependencies.jar" "LEMS_sim.xml" -neuron -run -compile -nogui -I '' fails because nrnivmodl generates a file named libnrnmech.dylib but neuroml expect a file with an extension .so or .la. The relevant part of the log:

 jNeuroML >>  (INFO) NMODL Compile >> >  -> Compiling leak_chan.c
 jNeuroML >>  (INFO) NMODL Compile >> >  -> Compiling pg_olm.c
 jNeuroML >>  (INFO) NMODL Compile >> >  -> Compiling pg_pyr.c
 jNeuroML >>  (INFO) NMODL Compile >> >  => LINKING shared library ./libnrnmech.dylib
 jNeuroML >>  (INFO) NMODL Compile >> >  => LINKING executable ./special LDFLAGS are:    
 jNeuroML >>  (INFO) NMODL Compile >> > Successfully created x86_64/special
 jNeuroML >>  (INFO) Executed command /Users/christian/opt/anaconda3/bin/nrnivmodl
 jNeuroML >>  (INFO) Verifying mod file compilation: looking for file: /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686/libnrnmech.la
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686/libnrnmech.la not found.
 jNeuroML >>  (INFO) Verifying mod file compilation: looking for file: /Users/christian/gdrive/publications_current/Journal/amygdala_model/x86_64/libnrnmech.la
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/x86_64/libnrnmech.la not found.
 jNeuroML >>  (INFO) Verifying mod file compilation: looking for file: /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686/libnrnmech.so
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686/libnrnmech.so not found.
 jNeuroML >>  (INFO) Verifying mod file compilation: looking for file: /Users/christian/gdrive/publications_current/Journal/amygdala_model/x86_64/libnrnmech.so
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/x86_64/libnrnmech.so not found.
 jNeuroML >>  (INFO) Verifying mod file compilation: looking for file: /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686/libnrnmech.la
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686/libnrnmech.la not found.
 jNeuroML >>  (INFO) Verifying mod file compilation: looking for file: /Users/christian/gdrive/publications_current/Journal/amygdala_model/umac/libnrnmech.la
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/umac/libnrnmech.la not found.
 jNeuroML >>  (INFO) Compilation failed. Unable to find necessary file(s). Please note that Neuron checks every *.mod file in this file's parent directory
 jNeuroML >>  (/Users/christian/gdrive/publications_current/Journal/amygdala_model).
 jNeuroML >>  For more information when this error occurs, enable logging at Settings -> General Properties & Project Defaults -> Logging
 jNeuroML >>  
 jNeuroML >>     NOTE: make sure you can compile NEURON mod files on your system!
 jNeuroML >>  
 jNeuroML >>  Often, extra packages (e.g. dev packages of ncurses & readline) need to be installed to successfully run nrnivmodl, which compiles mod files
 jNeuroML >>  Go to /Users/christian/gdrive/publications_current/Journal/amygdala_model and try running nrnivmodl
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686/libnrnmech.la
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/x86_64/libnrnmech.la
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686/libnrnmech.so
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/x86_64/libnrnmech.so
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686/libnrnmech.la
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/umac/libnrnmech.la
 jNeuroML >>  (INFO) Deleting generated dir(s): 
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/x86_64
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/x86_64
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/i686
 jNeuroML >>  (INFO) /Users/christian/gdrive/publications_current/Journal/amygdala_model/umac
 jNeuroML >>  (INFO) Success in compiling mods: false
 jNeuroML >>  org.neuroml.model.util.NeuroMLException: Error compiling mod files: /Users/christian/gdrive/publications_current/Journal/amygdala_model/leak_chan.mod; /Users/christian/gdrive/publications_current/Journal/amygdala_model/HCNolm.mod; /Users/christian/gdrive/publications_current/Journal/amygdala_model/Kdrfast.mod; /Users/christian/gdrive/publications_current/Journal/amygdala_model/KvAolm.mod; /Users/christian/gdrive/publications_current/Journal/amygdala_model/Nav.mod; /Users/christian/gdrive/publications_current/Journal/amygdala_model/pg_olm.mod; 
 jNeuroML >>  	at org.neuroml.export.neuron.NeuronWriter.generateAndRun(NeuronWriter.java:176)
 jNeuroML >>  	at org.neuroml.JNeuroML.main(JNeuroML.java:517)
 jNeuroML >>  
pyNeuroML >>> ERROR - execute_command_in_dir returned with output:  jNeuroML v0.11.1

GCC version:

% gcc -v      
Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: x86_64-apple-darwin21.6.0
Thread model: posix

NEURON -- VERSION 8.2.1 HEAD (c1590692) 2022-08-09

This problems comes up for example when using pynml.run_lems_with_jneuroml_neuron(...).

generated `time.dat` does not seem to put each time value on new line

The time.dat file generated from a NEURON backend run is a very long line instead of a value on each line:

0.0000000.0001000.0002000.0003000.0004000.0005000.0006000.0007000.0008000.0009000.0010000.0011000.0012000.0013000.0014000.0015000.0016000.0017000.0018000.0019000.0020000.0021000.0022000.0023000.0024000.0025000.0026000.0027000.0028000.0029000.0030000.0031000.0032000.0033000.0034000.0035000.0036000.0037000.0038000.0039000.0040000.0041000.0042000.0043000.0044000.0045000.0046000.0047000.0048000.0049000.0050000.0051000.0052000.0053000.0054000.0....

I expect this should be one value per line---something we need to tweak in NeuronWriter? Somewhere here?

org.neuroml.export/src/main/java/org/neuroml/export/neuron/NeuronWriter.java

Line 1357 in 766d52d

columnsPostTraces.get(timeRef).add(bIndent+" f_" + timeRef + "_f2.write('%f'% py_v_" + timeRef + "[i]) # Save in SI units...");

Add support for export of cell using pointCellCondBased/channelPopulation to NEURON

Currently, conductance based cell models expressed with cell/channelDensity is supported for mapping to Neuron (example) but not cell using pointCellCondBased/channelPopulation (example)

Support for this should be added in https://github.com/NeuroML/org.neuroml.export/blob/development/src/main/java/org/neuroml/export/neuron/NeuronWriter.java

Support for MinGW and newer versions of NEURON?

I am trying to compile an LEMS xml file with NEURON 8.0 installed on Windows 10 and am getting the following error message:

---

Unfortunately, jNeuroML doesn't currently support MinGW versions of NEURON. Try the Cygwin version of 7.3, e.g. nrn-7.3.i686-pc-cygwin-setup.exe

---

(LOG) at org.lemsml.jlems.core.logging.E.multiError(E.java:251)
(LOG) at org.lemsml.jlems.core.logging.E.error(E.java:217)
(LOG) at org.neuroml.export.neuron.ProcessManager.compileFileWithNeuron(ProcessManager.java:376)
(LOG) at org.neuroml.export.neuron.NeuronWriter.generateAndRun(NeuronWriter.java:163)
(LOG) at org.neuroml.JNeuroML.main(JNeuroML.java:517)
org.neuroml.model.util.NeuroMLException: Error testing: C:\Users\claus\source\repos\ow\OpenWorm\output\C2_FW_2022-02-22_18-50-54.
Is NEURON correctly installed?
NEURON home dir being used: C:\nrn

    at org.neuroml.export.neuron.ProcessManager.compileFileWithNeuron(ProcessManager.java:386)
    at org.neuroml.export.neuron.NeuronWriter.generateAndRun(NeuronWriter.java:163)
    at org.neuroml.JNeuroML.main(JNeuroML.java:517)

Caused by: org.neuroml.model.util.NeuroMLException: ****
Unfortunately, jNeuroML doesn't currently support MinGW versions of NEURON. Try the Cygwin version of 7.3, e.g. nrn-7.3.i686-pc-cygwin-setup.exe

---

    at org.neuroml.export.neuron.ProcessManager.compileFileWithNeuron(ProcessManager.java:152)
    ... 2 more

Is this going to be supported some day?

Use LEMS model language symbol with SED-ML

Currently, SED-ML documents are exported with model language symbol urn:sedml:language:neuroml2 for model sources which are LEMS files.

org.neuroml.export/src/main/java/org/lemsml/export/sedml/SEDMLWriter.java

Line 106 in ce18e86

startEndElement(main, "model", "id=" + netId, "language=urn:sedml:language:neuroml2", "source=" + inputFileName);

We recommend using the more accurate symbol urn:sedml:language:lems.

[Bug] sed-ml export: `id` attribute of `report` needs cleaning

See NeuroML/pyNeuroML#335

the id attribute of report contains a slash (/) there, which isn't permitted.

LEMS file and generated SEDML attached:

LEMS_model.sedml.txt
LEMS_model.xml.txt

Greater SWC support

Export of morphologies to SWC format should be straightforward to add

Import of SWC & conversion to NeuroML morphologies could be supported, but owuld probably be best done reusing some of the code here: https://github.com/pgleeson/Cvapp-NeuroMorpho.org

idealGasConstantDims units don't get exported to NEURON

idealGasConstantDims is defined in NeuroMLCoreDimensions.xml as J/(K *mol). In mod files, the relevant valid combination of units is mJ/K, so if for example I am defining a new componentType containing

<Constant name="idealGasConstant" dimension="idealGasConstantDims" value="8.3144621 J_per_K_per_mol"/>

this should end up in a mod file as

idealGasConstant = 8314.462 (millijoule / K)

but what I get instead is

idealGasConstant = 8.314462 ? Don't know units for : (idealGasConstantDims)

For an example of this, look at the default simulation configuration in my granule cell model.

List of LEMS/NML2 examples that can't be exported to neuron

Problems with .mod file generation/compilation:

• LEMS_NML2_Ex1_HH.xml
• LEMS_NML2_Ex3_Net.xml
• LEMS_NML2_Ex6_NMDA.xml
• LEMS_NML2_Ex7_STP.xml
• LEMS_NML2_Ex10_Q10.xml
• LEMS_NML2_Ex12_Net2.xml
• LEMS_NML2_Ex15_CaDynamics.xml
• LEMS_NML2_Ex17_Tissue.xml

Create a method to export a NeuroMLDocument into a LemsDocument

Graphwriter needs to replace "-" with "_" in the component ID

If the component name has "-" in them, they need to be replaced because the digraph ... line in dot cannot include "-".

MWE:

#!/usr/bin/env python3
"""
Create a simple network with two populations.
"""

from neuroml import NeuroMLDocument
from neuroml import Izhikevich2007Cell
from neuroml import Network
from neuroml import ExpOneSynapse
from neuroml import Population
from neuroml import PulseGenerator
from neuroml import ExplicitInput
from neuroml import SynapticConnection
import neuroml.writers as writers
import random
from pyneuroml import pynml
from pyneuroml.lems import LEMSSimulation
import numpy as np


nml_doc = NeuroMLDocument(id="IzNet")

iz0 = Izhikevich2007Cell(
    id="iz2007RS0", v0="-60mV", C="100pF", k="0.7nS_per_mV", vr="-60mV",
    vt="-40mV", vpeak="35mV", a="0.03per_ms", b="-2nS", c="-50.0mV", d="100pA")
nml_doc.izhikevich2007_cells.append(iz0)

syn0 = ExpOneSynapse(id="syn0", gbase="65nS", erev="0mV", tau_decay="3ms")
nml_doc.exp_one_synapses.append(syn0)

net = Network(id="IzNet")
nml_doc.networks.append(net)

size0 = 5
pop0 = Population(id="IzPop0", component=iz0.id, size=size0)
net.populations.append(pop0)

size1 = 5
pop1 = Population(id="IzPop1", component=iz0.id, size=size1)
net.populations.append(pop1)

random.seed(42)
prob_connection = 0.5
for pre in range(0, size0):
    pg = PulseGenerator(
        id="pulseGen_%i" % pre, delay="0ms", duration="10000ms",
        amplitude="%f nA" % (0.1 * random.random())
    )
    nml_doc.pulse_generators.append(pg)

    exp_input = ExplicitInput(target="%s/%i" % (pop0.id, pre), input=pg.id)
    net.explicit_inputs.append(exp_input)

    for post in range(0, size1):
        # 'from_' is used since 'from' is Python keyword
        if random.random() <= prob_connection:
            syn = SynapticConnection(from_="%s/%i" % (pop0.id, pre),
                                     synapse=syn0.id,
                                     to="%s/%i" % (pop1.id, post))
            net.synaptic_connections.append(syn)

nml_file = 'izhikevich2007_network.nml'
writers.NeuroMLWriter.write(nml_doc, nml_file)

print("Written network file to: " + nml_file)
pynml.validate_neuroml2(nml_file)

simulation_id = "example-izhikevich2007network-sim"
simulation = LEMSSimulation(sim_id=simulation_id,
                            duration=10000, dt=0.1, simulation_seed=123)
simulation.assign_simulation_target(net.id)
simulation.include_neuroml2_file(nml_file)

simulation.create_event_output_file(
    "pop0", "%s.spikes.dat" % simulation_id, format='ID_TIME'
)

for pre in range(0, size0):
    simulation.add_selection_to_event_output_file(
        "pop0", pre, 'IzPop0/{}'.format(pre), 'spike')

lems_simulation_file = simulation.save_to_file()

pynml.run_lems_with_jneuroml_neuron(
    lems_simulation_file, max_memory="20G", nogui=True, plot=False
)

# Load the data from the file and plot the spike times
# using the pynml generate_plot utility function.
data_array = np.loadtxt("%s.spikes.dat" % simulation_id)
pynml.generate_plot(
    [data_array[:, 1]], [data_array[:, 0]],
    "Spike times", show_plot_already=False,
    save_figure_to="%s-spikes.png" % simulation_id,
    xaxis="time (s)", yaxis="cell ID",
    linestyles='', linewidths='0', markers=['.'],
)

This generates this LEMS file:

<Lems>
    
    <!-- 

        This LEMS file has been automatically generated using PyNeuroML v0.5.6 (libNeuroML v0.2.52)

     -->
    
    <!-- Specify which component to run -->
    <Target component="example-izhikevich2007network-sim"/>

    <!-- Include core NeuroML2 ComponentType definitions -->
    <Include file="Cells.xml"/>
    <Include file="Networks.xml"/>
    <Include file="Simulation.xml"/>
    
    <Include file="izhikevich2007_network.nml"/>
   
    <Simulation id="example-izhikevich2007network-sim" length="10000ms" step="0.1ms" target="IzNet" seed="123">  <!-- Note seed: ensures same random numbers used every run -->
        
        <EventOutputFile id="pop0" fileName="example-izhikevich2007network-sim.spikes.dat" format="ID_TIME">
            <EventSelection id="0" select="IzPop0/0" eventPort="spike"/> 
            <EventSelection id="1" select="IzPop0/1" eventPort="spike"/> 
            <EventSelection id="2" select="IzPop0/2" eventPort="spike"/> 
            <EventSelection id="3" select="IzPop0/3" eventPort="spike"/> 
            <EventSelection id="4" select="IzPop0/4" eventPort="spike"/> 
        </EventOutputFile>
        
    </Simulation>

</Lems>

Exporting it to graphviz fails with this error:

jnml LEMS_example-izhikevich2007network-sim.xml -graph
 jNeuroML v0.10.1
(INFO) Reading from: /home/asinha/Documents/02_Code/00_mine/2020-OSB/NeuroML-Documentation/source/Userdocs/NML2_examples/LEMS_example-izhikevich2007network-sim.xml
(INFO) simCpt: Component(id=example-izhikevich2007network-sim type=Simulation)
Writing to: /home/asinha/Documents/02_Code/00_mine/2020-OSB/NeuroML-Documentation/source/Userdocs/NML2_examples/LEMS_example-izhikevich2007network-sim.gv
----Error: /home/asinha/Documents/02_Code/00_mine/2020-OSB/NeuroML-Documentation/source/Userdocs/NML2_examples/LEMS_example-izhikevich2007network-sim.gv: syntax error in line 3 near '-'

Line 3 is the digraph line where the component ID has "-" in it:

    # GraphViz compliant export for:IzNet (Type: network)

digraph example-izhikevich2007network-sim {
fontsize=10;


subgraph cluster_network {
    style=filled;
    color="#D6eeEA";
    node [style=filled,color=white];
    label = "Network to be simulated";

    node [shape=rectangle]; IzNet;
    #    Population IzPop0 contains components of: Component(id=iz2007RS0 type=izhikevich2007Cell) 
    node [shape=ellipse,color="white",fontcolor="black"]; IzPop0;    
    IzNet -> IzPop0 [len=1.00, arrowhead=diamond]

    #    Population IzPop1 contains components of: Component(id=iz2007RS0 type=izhikevich2007Cell) 
    node [shape=ellipse,color="white",fontcolor="black"]; IzPop1;    
    IzNet -> IzPop1 [len=1.00, arrowhead=diamond]

}

subgraph cluster_comps {
    style=filled;
    color="#CCFFCC";
    node [style=filled,color=white];
    label = "Components";

    node [shape=ellipse label=<<table border="0" cellborder="0"><tr><td>iz2007RS0 (izhikevich2007Cell)</td></tr><tr><td><font color="#666666">v0 = -0.06 V, k = 7.0E-7 kg^-2 m^-4 s^6 A^3, vr = -0.06 V, <br/>vt = -0.04 V, vpeak = 0.035 V, a = 30 s^-1, <br/>b = -2.0E-9 S, c = -0.05 V, d = 1.0E-10 A, <br/>C = 1.0E-10 F</font></td></tr></table>>]; "iz2007RS0 (izhikevich2007Cell)";

    "IzPop0" -> "iz2007RS0 (izhikevich2007Cell)" [label="5",len=1.00, arrowhead=diamond]
    node [shape=ellipse label=<<table border="0" cellborder="0"><tr><td>iz2007RS0 (izhikevich2007Cell)</td></tr><tr><td><font color="#666666">v0 = -0.06 V, k = 7.0E-7 kg^-2 m^-4 s^6 A^3, vr = -0.06 V, <br/>vt = -0.04 V, vpeak = 0.035 V, a = 30 s^-1, <br/>b = -2.0E-9 S, c = -0.05 V, d = 1.0E-10 A, <br/>C = 1.0E-10 F</font></td></tr></table>>]; "iz2007RS0 (izhikevich2007Cell)";

    "IzPop1" -> "iz2007RS0 (izhikevich2007Cell)" [label="5",len=1.00, arrowhead=diamond]
}

subgraph cluster_compTypes {
    style=filled;
    color="#D6E0EA";
    node [style="rounded, filled",color=white];
    label = "Component Types";
    node [style="rounded, filled",color=white];
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>izhikevich2007Cell</td></tr><tr><td><font color="#669999">Params: v0 (voltage), k (conductance_per_voltage), vr (voltage), <br/>vt (voltage), vpeak (voltage), a (per_time), <br/>b (conductance), c (voltage), d (current) </font></td></tr><tr><td><font color="#FF9966">State vars: v (voltage), u (current) </font></td></tr><tr><td><font color="#99CC00">iSyn = SUM OF: synapses[*]/i</font></td></tr><tr><td><font color="#99CC00">iMemb = k * (v-vr) * (v-vt) + iSyn - u</font></td></tr><tr><td><font color="#666633">v' = iMemb / C</font></td></tr><tr><td><font color="#666633">u' = a * (b * (v-vr) - u)</font></td></tr><tr><td><font color="#996633">IF v .gt. vpeak THEN </font></td></tr><tr><td><font color="#996633">(v = c)  <br/>AND (u = u + d)  <br/>AND (EVENT: spike)</font></td></tr><tr><td><font color="#666666">Exposures: u (current), iSyn (current), iMemb (current), <br/>v (voltage) </font></td></tr></table>>]; izhikevich2007Cell;
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseCellMembPotCap</td></tr><tr><td><font color="#669999">Params: C (capacitance) </font></td></tr><tr><td><font color="#666666">Exposures: iSyn (current), iMemb (current), v (voltage) </font></td></tr></table>>]; baseCellMembPotCap;
    izhikevich2007Cell -> baseCellMembPotCap [len=1.00, arrowhead=onormal]
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseCellMembPot</td></tr><tr><td><font color="#666666">Exposures: v (voltage) </font></td></tr></table>>]; baseCellMembPot;
    baseCellMembPotCap -> baseCellMembPot [len=1.00, arrowhead=onormal]
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseSpikingCell</td></tr></table>>]; baseSpikingCell;
    baseCellMembPot -> baseSpikingCell [len=1.00, arrowhead=onormal]
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseCell</td></tr></table>>]; baseCell;
    baseSpikingCell -> baseCell [len=1.00, arrowhead=onormal]
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseStandalone</td></tr></table>>]; baseStandalone;
    baseCell -> baseStandalone [len=1.00, arrowhead=onormal]
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>izhikevich2007Cell</td></tr><tr><td><font color="#669999">Params: v0 (voltage), k (conductance_per_voltage), vr (voltage), <br/>vt (voltage), vpeak (voltage), a (per_time), <br/>b (conductance), c (voltage), d (current) </font></td></tr><tr><td><font color="#FF9966">State vars: v (voltage), u (current) </font></td></tr><tr><td><font color="#99CC00">iSyn = SUM OF: synapses[*]/i</font></td></tr><tr><td><font color="#99CC00">iMemb = k * (v-vr) * (v-vt) + iSyn - u</font></td></tr><tr><td><font color="#666633">v' = iMemb / C</font></td></tr><tr><td><font color="#666633">u' = a * (b * (v-vr) - u)</font></td></tr><tr><td><font color="#996633">IF v .gt. vpeak THEN </font></td></tr><tr><td><font color="#996633">(v = c)  <br/>AND (u = u + d)  <br/>AND (EVENT: spike)</font></td></tr><tr><td><font color="#666666">Exposures: u (current), iSyn (current), iMemb (current), <br/>v (voltage) </font></td></tr></table>>]; izhikevich2007Cell;
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseCellMembPotCap</td></tr><tr><td><font color="#669999">Params: C (capacitance) </font></td></tr><tr><td><font color="#666666">Exposures: iSyn (current), iMemb (current), v (voltage) </font></td></tr></table>>]; baseCellMembPotCap;
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseCellMembPot</td></tr><tr><td><font color="#666666">Exposures: v (voltage) </font></td></tr></table>>]; baseCellMembPot;
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseSpikingCell</td></tr></table>>]; baseSpikingCell;
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseCell</td></tr></table>>]; baseCell;
    node [shape=box label=<<table border="0" cellborder="0"><tr><td>baseStandalone</td></tr></table>>]; baseStandalone;
}

    "iz2007RS0 (izhikevich2007Cell)" -> izhikevich2007Cell [len=1.00]
}

nmodl export: Passing events to child elements is not supported

This is needed for properly simulating synaptic plasticity, as in NeuroML a plastic synapse relays incoming spiking events to its (child) plasticityMechanism which uses spike timing information to calculate the plasticityFactor.

NEURON morphology fails with large morphologies

Due to limit on length of procedures in hoc.

Should break generated procedure in hoc file into multiple smaller procedures

What does the value of 318.91927 represent in abstract cell CM?

There are several spots where the value of 318.31927 is used for abstract cell CM. The cells have L and Diam = 10.

The area of a cell with L,D=10 would be 314.1592, which is slightly different...

Is there something special about this CM value? I did not find any other mention of this number elsewhere.

exported mod always uses cnexp solver

This solver only works for linear, diagonal odes. It produces nonsensical results in some cases, when used in other types of systems. It is fundamental that we either

• add logic to use cnexp only when possible and use derivimplict elsewhere
• default to derivimplicit which always works

see
https://www.neuron.yale.edu/phpBB/viewtopic.php?f=28&t=592
or quoting M. Hines:

The cnexp method is invalid for that derivative equation.
The cnexp method requires that each y' equation be linear and involve no other state than y.
That is, cnexp was intended for use only in HH style gating equations where the form looked like
m' = (minf(v) - m)/mtau(v)
or
m' = bm(v)_(1. - m) - am(v)_m
For general (nonlinear and coupled state ) equations, use derivimplicit

Investigate LEMS/NeuroML2 export to/import from DNSim

LEMS -> DNSim

Possible example to test: https://github.com/OpenSourceBrain/MorrisLecarModel/blob/master/NeuroML2/MorrisLecarSCell.xml

Could be based on the dLEMS to XPP converer:
https://github.com/NeuroML/org.neuroml.export/blob/development/src/main/resources/xpp/xpp.vm

DNSim -> LEMS

@jsherfey will look into making a DNSim -> XPP exporter & this can be the basis for DNSim -> LEMS

Based on a discussion with @scrook, @jsherfey, @pgleeson on 2/12/14

Why do exported SED-ML files use model language urn:sedml:language:neuroml2 rather than LEMS?

SED-ML files exported by pyNeuroML indicate language urn:sedml:language:neuroml2, although the sources are LEMS files.

<model id="net1" language="urn:sedml:language:neuroml2" source="LEMS_NML2_Ex5_DetCell.xml"/>

(a) The URN for NeuroML 2 should be urn:sedml:language:neuroml.version-2. See http://sed-ml.org/urns.html

(b) Why does this package indicate NeuroML 2 rather than LEMS? I can help add an URN for LEMS.

Workarounds for fractionAlong != (0 or 1)?

I have a NEURON model where the segments are connected at non 0 or 1 parent locations, resulting in NML that uses e.g. fractionAlong="0.8" or "0.517...". When I use jnml to convert LEMS file to neuron, it results in the following error:

Cannot yet handle fractionAlong being neither 0 or 1

If I set the fractionAlong to 0 or 1, it introduces small errors in the voltage during simulation.

My example NML is:

            <segment id="1" name="Seg0_priden">
                <parent fractionAlong="1" segment="0"/>
                <proximal y="-3.005222e+00" x="4.340876e+00" z="6.010444e+00" diameter="5.000000e-01"/>
                <distal y="-7.102542e+01" x="1.025923e+02" z="1.420508e+02" diameter="5.000000e-01"/>
            </segment>
            <segment id="2" name="Seg0_priden2_0">
                <parent ****fractionAlong="0.8"**** segment="1"/>
                <proximal y="-3.005222e+00" x="4.340876e+00" z="6.010444e+00" diameter="4.000000e-01"/>
                <distal y="-9.691841e+01" x="1.399933e+02" z="1.938368e+02" diameter="4.000000e-01"/>
            </segment>

Is the best workaround for this issue to split the parent segment into two segments along the non-integer fraction of the parent segment's length? If so, I'll include this in the error message text and submit a pull-request.

@pgleeson

Weighted synapses in Ex21 are not exported correctly to neuron

This is the reason LEMS_NML2_Ex21_CurrentBasedSynapses.xml does not work.

• generated alphaSyn.mod doesn't correctly expose "i" (try renaming state variable to sthing else)
• generated LEMS_NML2_Ex21_CurrentBasedSynapses_nrn.py does not correctly set the weight for the NetCon (last arg should be 5e-11 instead of 0).

Improve path related errors to help users figure out what is wrong with their paths

At the moment, while trying to get the correct path to record from a synapse, the errors are not very helpful to users. They're of this form:

Exception in thread "main" java.lang.NullPointerException: Cannot invoke "org.lemsml.jlems.core.type.Component.getComponentType()" because "c" is null
        at org.neuroml.export.neuron.LEMSQuantityPathNeuron.getExposure(LEMSQuantityPathNeuron.java:91)
        at org.neuroml.export.neuron.LEMSQuantityPathNeuron.getExposure(LEMSQuantityPathNeuron.java:117)
        at org.neuroml.export.neuron.LEMSQuantityPathNeuron.getDimension(LEMSQuantityPathNeuron.java:151)
        at org.neuroml.export.neuron.NeuronWriter.getMainScript(NeuronWriter.java:1415)
        at org.neuroml.export.neuron.NeuronWriter.generateMainScriptAndMods(NeuronWriter.java:288)
        at org.neuroml.export.neuron.NeuronWriter.generateAndRun(NeuronWriter.java:156)
        at org.neuroml.JNeuroML.main(JNeuroML.java:525)

I wonder if it is possible for us to print some more information, for example---what component was it trying to get the componentType of? That'll help users figure out where the error is in the path.

Incorrect generated mod for kinetic scheme initialisation

The generated INITIAL block, which invokes a (unnecessary) generated sequinitial routine should use SOLVE activation STEADYSTATE sparse instead. That is the cause of a discontinuous state jump from t=0 to t=dt.

[*nix] Mod file compilation only checks for statically linked libnrnmech file (libnrnmech.la), but should also check for shared objects (libnrnmech.so)

I'm working with the NEURON installation in NeuroFedora, and we build NEURON to use shared libraries, rather than static libraries. The rpm "spec file" to build NEURON is here (builds with mpich/openmpi and non MPI versions):

https://src.fedoraproject.org/rpms/neuron/blob/master/f/neuron.spec#_211

(The Fedora packaging guidelines discourage use of static libraries and libtool archives: https://docs.fedoraproject.org/en-US/packaging-guidelines/#packaging-static-libraries because that increases the size of binaries and prevents usage of system libraries)

The export code, however, even though nrnivmodl works correctly, indicates that mod file compilation failed because it is unable to find libnrnmech.la: but libnrnmech.so is correctly created:

$ ls
getting-started-izhikevich.py  izhikevich2007_network.nml                  LEMS_example-izhikevich2007cell-sim.xml  pulseGen_1.mod  pulseGen_3.mod  syn0.mod
iz2007RS0.mod                  LEMS_example-izhikevich2007cell-sim_nrn.py  pulseGen_0.mod                           pulseGen_2.mod  pulseGen_4.mod  x86_64
$ ls x86_64/
iz2007RS0.c    iz2007RS0.o    mod_func.c  pulseGen_0.c    pulseGen_0.o  pulseGen_1.mod  pulseGen_2.c    pulseGen_2.o  pulseGen_3.mod  pulseGen_4.c    pulseGen_4.o  syn0.c    syn0.o
iz2007RS0.mod  libnrnmech.so  mod_func.o  pulseGen_0.mod  pulseGen_1.c  pulseGen_1.o    pulseGen_2.mod  pulseGen_3.c  pulseGen_3.o    pulseGen_4.mod  special       syn0.mod  x86_64

So the code should also check for the *.so file, which works just fine too.

When displaying forward rate for a generic expression change, the generic expression needs to be displayed instead of a label saying genericexpression

Cerebellar Granule Cell Schutter > gateH

http://opensourcebrain.org/projects/grancelllayer?explorer=https%3A%2F%2Fraw.github.com%2FOpenSourceBrain%2FGranCellLayer%2Fmaster%2FneuroConstruct%2FgeneratedNeuroML2%2FGolgi_CaHVA_CML.channel.nml#

Speedup possible?

I did run following command to profile the program which gets called from a c302 script (openworm/CelegansNeuroML)

java -Xmx400M -agentlib:hprof=file=hprof_times.txt,cpu=times  -Djava.awt.headless=true -jar  "/usr/local/lib/python2.7/dist-packages/pyNeuroML-0.2.10-py2.7.egg/pyneuroml/lib/jNeuroML-0.8.0-jar-with-dependencies.jar"  "LEMS_c302_C2_AVB_VB_DB_VD_DD.xml"  -neuron -run -nogui

Only a very small part of the output:
...
CPU TIME (ms) BEGIN (total = 4181427) Fri Apr 21 17:12:36 2017
rank self accum count trace method
1 33.05% 33.05% 43 305405 java.lang.Object.wait
2 32.87% 65.92% 161205 305413 java.lang.ref.ReferenceQueue.remove
3 2.48% 68.40% 1888856 319750 java.io.DataOutputStream.writeUTF
4 1.74% 70.14% 74793210 319746 java.lang.String.charAt
5 0.89% 71.03% 238653 302248 java.io.UnixFileSystem.normalize
6 0.79% 71.82% 8651067 319701 java.util.zip.Inflater.inflate
7 0.67% 72.49% 28638360 302247 java.lang.String.charAt
8 0.47% 72.96% 8651067 319699 java.util.zip.Inflater.ensureOpen
9 0.44% 73.40% 3777712 319734 java.util.zip.InflaterInputStream.read
10 0.33% 73.73% 73670 319775 com.sun.xml.bind.v2.bytecode.ClassTailor.tailor
...

As you can see wait() and remove() needs 66% of the time.
Is it possible to speed up this program to make the c302 simulations faster?

Maybe add 'mvn install' to travis plan ?

Was looking at the travis plan of org.neuroml.export. Unless I'm mistaken, it seems that 'mvn install' is not called for org.neuroml.export. Wouldn't it make sense to do this ? This way also the tests would be run.

Improve SVG export for detailed morphologies

The class for generating SVG files from detailed NeuroML2 morphologies is here:
https://github.com/NeuroML/org.neuroml.export/blob/master/src/main/java/org/neuroml/export/svg/SVGWriter.java

This is used by jNeuroML to generate a scalable image file using:

jnml L5PC.cell.nml -svg

An example of the current output of this (for a L5 pyramidal cell) is:

This could be improved in numerous ways:

• Improving layout of 3 side views (dependent on orientation of cell)
• Add support for networks of cells, e.g. this
• Add colours a/c to whether segments are in soma_group, dendrite_group, axon_group
• Add option to export to png files using some standard image processing library (svg->png) available through maven
• Add off axis view (e.g. looking at 45deg down on cell)

Other detailed morphologies in NML2 are available here for testing: http://www.opensourcebrain.org/search_custom_field?f[]=43&op[43]=~&v[43][]=Detailed+cell+model

NeuronWriter needs to replace tokens that are reserved in Python

Here's an example LEMS simulation file:

<Lems>
    
    <!-- 

        This LEMS file has been automatically generated using PyNeuroML v0.5.11 (libNeuroML v0.2.55)

     -->
    
    <!-- Specify which component to run -->
    <Target component="olm_example_sim"/>

    <!-- Include core NeuroML2 ComponentType definitions -->
    <Include file="Cells.xml"/>
    <Include file="Networks.xml"/>
    <Include file="Simulation.xml"/>
    
    <Include file="olm_example_net.nml"/>
    <Include file="olm.cell.nml"/>
    <Include file="olm-example/leak_chan.channel.nml"/>
    <Include file="olm-example/HCNolm.channel.nml"/>
    <Include file="olm-example/Kdrfast.channel.nml"/>
    <Include file="olm-example/KvAolm.channel.nml"/>
    <Include file="olm-example/Nav.channel.nml"/>
   
    <Simulation id="olm_example_sim" length="300ms" step="0.01ms" target="single_olm_cell_network" seed="123">  <!-- Note seed: ensures same random numbers used every run -->
        
        <OutputFile id="output0" fileName="olm_example_sim.dat">
            <OutputColumn id="pop0[0]_v" quantity="pop0[0]/v"/> 
            <OutputColumn id="pop0[0]_iChannels" quantity="pop0[0]/iChannels"/> 
            <OutputColumn id="pop0[0]_Nav_dendrite_iDensity" quantity="pop0[0]/biophys/membraneProperties/Nav_dendrite/iDensity/"/> 
            <OutputColumn id="pop0[0]_Nav_soma_iDensity" quantity="pop0[0]/biophys/membraneProperties/Nav_soma/iDensity/"/> 
            <OutputColumn id="pop0[0]_Nav_axon_iDensity" quantity="pop0[0]/biophys/membraneProperties/Nav_axon/iDensity/"/> 
        </OutputFile>
        
    </Simulation>

</Lems>

The writer converts this to this sort of Python code:

        # ######################   File to save: olm_example_sim.dat (output0)
        py_v_pop0[0]_v_output0 = [ float(x  / 1000.0) for x in h.v_pop0[0]_v_output0.to_python() ]  # Convert to Python list for speed, variable has dim: voltage
        py_v_pop0[0]_iChannels_output0 = [ float(x  / 1.0E9) for x in h.v_pop0[0]_iChannels_output0.to_python() ]  # Convert to Python list for speed, variable has dim: current
        py_v_pop0[0]_Nav_dendrite_iDensity_output0 = [ float(x  / 0.001) for x in h.v_pop0[0]_Nav_dendrite_iDensity_output0.to_python() ]  # Convert to Python list for speed, variable has dim: currentDensity
        py_v_pop0[0]_Nav_soma_iDensity_output0 = [ float(x  / 0.001) for x in h.v_pop0[0]_Nav_soma_iDensity_output0.to_python() ]  # Convert to Python list for speed, variable has dim: currentDensity
        py_v_pop0[0]_Nav_axon_iDensity_output0 = [ float(x  / 0.001) for x in h.v_pop0[0]_Nav_axon_iDensity_output0.to_python() ]  # Convert to Python list for speed, variable has dim: currentDensity

here, since the [0] bits are included in the variable name, the script won't run. So, we need to figure out a way to replace these to come up with valid Python variable names. The simplest would to replace [xx] with _xx_ I guess, but we need to see how this may affect the running of the script.

Celsius degrees don't get properly translated to Kelvin

@pgleeson: If I have a neuroml file wich specifies a parameter in degC units (e.g. 35 degC), I would expect to see it converted in Kelvin in the mod file (308.15 K). What I get, instead, is a variable with dimensionality degC that gets plugged into the equations as if it were an absolute temperature.