see about upgrading our ExodusII/Nemesis versions

[libmesh]

see about upgrading our ExodusII/Nemesis versions

[benkirk]

I've got the latest exodusI/nemesis and netcdf packages, and am going to work on this this week while on travel...

@jwpeterson, I'll point you to my branch to try when it is working, you can let me know how bad this hurts MOOSE before I commit it to libMesh/master.

[benkirk]

This is going to be a big deal - in particular, the new exodus implementation requires netcdf-4.1 or greater, which in turn requires hdf5.

since our old exodusII/nemesis API has so few requirements, I'm inclined to leave it for a while with an --enable-old-exodusII or something.

@roystgnr, @friedmud, @jwpeterson -- thoughts?

-Ben

[friedmud]

Was there a particular reason you were upgrading? I'd say just leave it alone until there is a feature we need or something is broken...

Derek

Sent from my iPhone

On Feb 3, 2013, at 12:38 PM, "Benjamin S. Kirk" [email protected] wrote:

This is going to be a big deal - in particular, the new exodus implementation requires netcdf-4.1 or greater, which in turn requires hdf5.

since our old exodusII/nemesis API has so few requirements, I'm inclined to leave it for a while with an --enable-old-exodusII or something.

@roystgnr, @friedmud, @jwpeterson -- thoughts?

-Ben

—
Reply to this email directly or view it on GitHub.

[benkirk]

On Feb 3, 2013, at 1:45 PM, "Derek Gaston" [email protected] wrote:

Was there a particular reason you were upgrading? I'd say just leave it alone until there is a feature we need or something is broken...

Derek

Most importantly would be "large model support" - greater than 2Gb is my understanding. Also, looks like nemesis is now rolled into exodus, with simple shims providing the nemesis API, so this would be the right thing(TM) going forward.

-Ben

[friedmud]

I gotcha. Well, give it a whirl then :-)

I would appreciate a --use-old-exodus or whatever. But since you're doing
the work I'll leave that up to you :-)

Derek

Sent from my iPad

On Feb 3, 2013, at 1:06 PM, "Benjamin S. Kirk" [email protected]
wrote:

On Feb 3, 2013, at 1:45 PM, "Derek Gaston" [email protected]
wrote:

Was there a particular reason you were upgrading? I'd say just leave it
alone until there is a feature we need or something is broken...

Derek

Most importantly would be "large model support" - greater than 2Gb is my
understanding. Also, looks like nemesis is now rolled into exodus, with
simple shims providing the nemesis API, so this would be the right
thing(TM) going forward.

-Ben

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/35#issuecomment-13052891.

[jwpeterson]

On Feb 3, 2013, at 12:38 PM, "Benjamin S. Kirk" [email protected] wrote:

This is going to be a big deal - in particular, the new exodus implementation requires netcdf-4.1 or greater, which in turn requires hdf5.

This feels wrong. Like petsc requiring trilinos or openmpi requiring mpich. Nevertheless there is a possibility that we will need to write silo files, which are hdf5-based... So it could have additional benefits.

since our old exodusII/nemesis API has so few requirements, I'm inclined to leave it for a while with an --enable-old-exodusII or something.

@roystgnr, @friedmud, @jwpeterson -- thoughts?

Keep the old one as the default for a while?

Are you thinking of distributing hdf5 like we currently do with netcdf?

John

[benkirk]

I'd rather not distribute it (1) because configuring it takes as long (or longer) than libMesh, and (2) it is pretty widely available (more so than netcdf anyway). But I've got a fallback if we want to - since HDF5 is an autotools package it is easy to incorporate it, but at the cost of longer configures.

At any rate, I've got an exodus_refresh branch going at https://github.com/benkirk/libmesh.git

@jwpeterson, could you take a look at this and see how bad it is going to hurt you guys? By default it builds all the old stuff, you can force the new API by

$ ./configure ... --enable-netcdf=new --enable-exodus=new --enable-nemesis=new

It should find your HDF5 if it is there. If you don't have HDF5 you can still do

$ ./configure ... --enable-netcdf=new --enable-exodus=new --enable-nemesis=new --disable-hdf5

and that'll still build the new API, but my understanding is large files (>2Gb) won't work.

[friedmud]

I just pulled this down and I'm building it now...

[friedmud]

Looks like it's not quite working with in-tree building:

libtool: link: `/Users/gastdr/projects/libmesh/lib/libnetcdf.la' is not a valid libtool archive

I got that at the end of trying to compile MOOSE. We may have something wrong with our Makefile stuff...

I just "make install"ed and I'm rebuilding.

[benkirk]

Oh thanks, let me know either way and ill check the in tree stuff tomorrow.

I usually work out of tree until all is well to avoid unnecessary churn in the .gitignore file...

-Ben

On Feb 4, 2013, at 11:11 PM, "Derek Gaston" <[email protected]mailto:[email protected]> wrote:

Looks like it's not quite working with in-tree building:

libtool: link: `/Users/gastdr/projects/libmesh/lib/libnetcdf.la' is not a valid libtool archive

I got that at the end of trying to compile MOOSE. We may have something wrong with our Makefile stuff...

I just "make install"ed and I'm rebuilding.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/35#issuecomment-13117393.

[friedmud]

Surprisingly.... all is well with MOOSE when using:

../configure --prefix=$INSTALL_DIR --with-methods="opt dbg" --enable-legacy-include-paths --enable-legacy-using-namespace --enable-netcdf=new --enable-exodus=new --enable-nemesis=new --disable-hdf5

That was easy ;-)

Thanks for doing this Ben!

[benkirk]

Awesome, thanks for testing it.

I'm planning to tag v0.9.0 as soon as everything is green again, and afterwards I'll merge this. We can then decide when to make this the new default.

[benkirk]

This is merged as of 9d7303f, however it is not yet the default.

At present, you can configure using the new code via

    $ ./configure ... --enable-netcdf=new --enable-exodus=new --enable-nemesis=new

so @roystgnr it would be interesting to see what all those int64's do for you, which I think was the original motivation here.

[benkirk]

I'm going to make the 'new' flavors default, the 'old' will be optionally available.

[pbauman]

On Tue, Feb 12, 2013 at 9:37 AM, Benjamin S. Kirk
[email protected]:

I'm going to make the 'new' flavors default, the 'old' will be optionally
available

Thanks.

Has anyone tried 1. Reading in GRIDS with new ExodusII version that have
been around awhile or 2. Tried loading solutions with the new format into
ParaView? I'll find out for myself, but was hoping I could be warned if
there are any surprises.

[benkirk]

I've done both, with no surprises.

[pbauman]

On Tue, Feb 12, 2013 at 9:52 AM, Benjamin S. Kirk
[email protected]:

I've done both, with no surprises.

Awesome, thanks.

[jwpeterson]

As far as in-tree (uninstalled) builds are concerned, the new netcdf doesn't work out of the box, since its an automake package. Those who are interested in doing uninstalled builds can work around it by doing:

${LIBMESH_DIR}/libtool --mode=install install -c
${LIBMESH_DIR}/contrib/netcdf/4.2.1.1/liblib/libnetcdf.la ${LIBMESH_DIR}/lib

But this doesn't feel like a great solution since you have to know the intricate details of netcdf's location within the contrib hierarchy.

[roystgnr]

How do we get netcdf v4 to build correctly when hdf5 isn't in the compiler's default search path? Our own hdf5.m4 is autodetecting my hdf5 module based on the environment variables it sets, but we don't seem to be passing the resulting HDF5_CPPFLAGS/CXXLIBS information to contrib/netcdf/v4/configure, and looking through their configure.ac I don't even see how we'd pass that information in.

[benkirk]

Can you uncomment the end of configure.ac line 332 and see if that does it for you?

That seems a little heavy handed, passing all our optional includes and libs, but is the generic approach. Note though this can creep into our AC_OUTPUT more than I'd like, and to cherry-pick the options I'm considering something along these lines:

http://gnu-autoconf.7623.n7.nabble.com/AC-CONFIG-SUBDIRS-right-away-td9738.html

[roystgnr]

On Thu, 14 Feb 2013, Benjamin S. Kirk wrote:

Can you uncomment the end of configure.ac line 332

That fixes configure, thanks! I'll scream soon if compiling fails.

That seems a little heavy handed

Very heavy-handed... I don't actually see what problems it could cause
other than unnecessarily huge compilation command lines in
subpackages, though. I guess the extra directory paths could slow
down compiles over NFS? The extraneous entries in LIBS might break
some shared library linkers and would probably break static libraries?

http://gnu-autoconf.7623.n7.nabble.com/AC-CONFIG-SUBDIRS-right-away-td9738.html

So we'd do each subdir one at a time, allowing us to pass in only the
flags that we know that subpackage needs? I love that solution in
theory. In practice, I see a _AC_SRCDIRS call in that AX_CONFIG_DIR
macro, and if we're using autoconf internals then I'd worry about

future autotools releases breaking things out from under us.

Roy

[benkirk]

The immediate annoyance I noted was that sometimes AC_OUTPUT would get invoked on reconfigure and LIBS=... would get handed back to us, which of course is not a problem since they were ours to begin with, but made the perflog configure output look horrible. :-)

But yeah, the idea would be to do each subdir one at a time, exactly when needed, specifying only the arguments it needs. This shows up on the list as a highly desired feature for autoconf, so maybe if we do it right it'll go upstream and no longer be unsupported.

[benkirk]

@permcody requested we also build the Fortran bindings -- easy enough, but I'll defer until tomorrow or the next day when I have a couple hours to devote to "automakethink."

Just a note so I don't forget.

[benkirk]

I added the Fortran bindings, and things seem to be good with the new Nemesis/Exodus/Netcdf.