llnl / conduit Goto Github PK
View Code? Open in Web Editor NEWSimplified Data Exchange for HPC Simulations
Home Page: https://software.llnl.gov/conduit/
License: Other
Simplified Data Exchange for HPC Simulations
Home Page: https://software.llnl.gov/conduit/
License: Other
"scalar", "vector", etc
could be redundant with number_of_components, but may be needed in some cases to differentiate between a generic nd-array and say a vector or a tensor.
We are adding a path() function that returns a string with the full path to the given node
has_path and fetch path variants descend the tree, but, the new path() clashes a bit with a few other uses "path" in method names:
For example, in the Node and Schema classes:
In the NodeIterator class:
I propose changing the path variants in the Node and Schema classes to
And path() in NodeIterator to:
It's an API breaking change, but I think 0.2.0 is the right time to do this.
int intArray[2];
MPI_Status status;
int mpi_error = MPI_Recv(intArray, 2, MPI_INT, src, tag, comm, &status);
CONDUIT_CHECK_MPI_ERROR(mpi_error);
int schema_len = intArray[0];
int data_len = intArray[1];
char schema[schema_len + 1];
char data[data_len + 1];
the above allocs for schema and data are working -- but they aren't typical C++ allocs. We should replace them
in favor of only compact_to()
This is to stay consistent with the types of operations we want to support as const
and source examples, like we have in
http://software.llnl.gov/conduit/building.html#using-conduit-in-another-project
as currently setup, the generated lib has the file name prefix "liblibb64"
our docs are pointing to an out of date loc:
http://software.llnl.gov/spack/basic_usage.html#manual-compiler-configuration
in particular:
https://github.com/phosphorjs/phosphor-splitpanel
which will allow resizing of the viewer components.
hdf5_read(hid_t h5_id, const DataType &dt, Node &out)
this allows you to fetch using an offset, stride, etc and fetch as a different type than the hdf5 dataset
(for example, you can fetch an array of doubles from an offset of hdf5 dset that was created as bytes)
converting from external to owned, and vice versa.
We tried to attack this before, had a hard time with finding the right names.
From Kathleen B.:
Wow, I have this in my test :
proc0["cycle"] = 120;
worked fine on linux, but Visual studio complains that it is ambiguous:
15> error C2593: 'operator =' is ambiguous
15> Node.h(923): could be 'conduit::Node &conduit::Node::operator =(conduit::float64)'
15> Node.h(922): or 'conduit::Node &conduit::Node::operator =(conduit::float32)'
15> Node.h(920): or 'conduit::Node &conduit::Node::operator =(conduit::uint64)'
15> Node.h(919): or 'conduit::Node &conduit::Node::operator =(conduit::uint32)'
15> Node.h(918): or 'conduit::Node &conduit::Node::operator =(conduit::uint16)'
15> Node.h(917): or 'conduit::Node &conduit::Node::operator =(conduit::uint8)'
15> Node.h(915): or 'conduit::Node &conduit::Node::operator =(conduit::int64)'
15> Node.h(914): or 'conduit::Node &conduit::Node::operator =(conduit::int32)'
15> Node.h(913): or 'conduit::Node &conduit::Node::operator =(conduit::int16)'
15> Node.h(912): or 'conduit::Node &conduit::Node::operator =(conduit::int8)'
15> d): or 'conduit::Node &conduit::Node::operator =(conduit::bool8)'
15> while trying to match the argument list '(conduit::Node, int)'
for the object case,
add:
std::string child_name(index_t idx) const;
to complement:
index_t child_index(const std::string &path) const;
set_external(Node &) effectively allows you to create a shallow copy the passed node.
Since it does not modify the structure of the passed Node, I think we should also support a variant that allows the passed node to be const.
civetweb had logic for osx related to get_clocktime that fails on OSX 10.12.
This has been fixed on their master branch, so we need to incorporate:
https://github.com/civetweb/civetweb/pull/356/files
Into our copy of the source.
two use cases:
if they can set additional python paths in a host-config this could help folks configure python packages (eg: numpy) without env vars & we could do this per platform in an centralized way.
if we can figure out the best way to support PYTHONPATH env var for custom commands, this could also help us avoid the python module warning "you are installing your module to a place that isn't in your sys.path()" that shows up when we build the conduit python module
On windows we don't have access to mmap, we have to use a windows specific API.
http://msdn.microsoft.com/en-us/library/aa366556%28VS.85%29.aspx
For now, i ifdef calls to mmap out on windows, and added an exception if someone tries this path.
to make it easy to verify if json, hdf5 and hdf5 trees conform to blueprint protocols
for better compatibility with Sidre N-D Fortran array support and python ndarrays
start with the contiguous case:
+shape (list of ints)
+values (actually 1-d array with values)
right now we require an extra link for the fortran interfaces for each lib.
I think the symbols for the fortran interfaces can safely be put in the main libs.
If there is a need, we cold provide a way to discover if fortran support is enabled other than the existence of these libs. (The fortran modules files should fulfill this need)
reserve point as a shape, vertex is a better description for how the field is associated with the mesh (matches mfem nomenclature)
conduit::utils::create_directory(const std::string &path, bool recursive=true);
conduit::utils::remove_directory(const std::string &path, bool recursive=true);
It seems the compiler is undermining threaded builds:
Fatal Error: Can't rename module file '../../../fortran/f_conduit_node.mod0' to '../../../fortran/f_conduit_node.mod': No such file or directory
make[2]: *** [tests/conduit/fortran/CMakeFiles/f_conduit_node_obj.dir/f_conduit_node_obj.f.o] Error 1
make[1]: *** [tests/conduit/fortran/CMakeFiles/f_conduit_node_obj.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 76%] /private/var/folders/wj/jjhjt_0d3kg1vmk56d5bq_38001kr8/T/spack-stage/spack-stage-J_e5zR/conduit/src/tests/conduit/fortran/f_conduit_node.f:411.6:
use f_conduit_node
1
Fatal Error: Can't open module file 'f_conduit_node.mod' for reading at (1): No such file or directory
make[2]: *** [tests/conduit/fortran/CMakeFiles/f_conduit_node.dir/f_conduit_node.f.o] Error 1
http://stackoverflow.com/questions/29728867/cmake-parallel-build-for-fortran-90-with-modules
for output between machines with differing endianness, this will be such a common need that it should be the default.
related to #14, we should try to resolve the underlying issue with PRIVATE causing a transitive link dependency for imported targets
to avoid potential conflicts for system installs
ex:
change:
DataType.hpp
to
conduit_data_type.hpp
This is out of an abundance of caution to avoid conflicts -- since client codes need to "include/conduit" to their include paths
requires:
runtime fetch of install path
I'd like to use Blueprint to describe a mesh that has multiple materials in a single zone, where each material has its own value for a particular field (e.g. density). Does Blueprint have any mechanism for describing such a field? If not, what would be the best way to utilize Blueprint's existing schema to describe such a field? Thanks in advance!
surrounding the primary use case to support sparse representations for mixed material data (volume fractions and per-material state vars)
Since relay processes complex hdf5 trees, we need more context when things go wrong.
This pattern is wrong:
n = conduit_node_number_of_elements(cnode)
int32_c_ptr = c_conduit_node_fetch_path_as_int32_ptr(cnode,trim(path) // C_NULL_CHAR)
call c_f_pointer(int32_c_ptr, f_out, (/n/))
The size of f_out is set as the number of elements of the root, not the actually number of elements at the fetched path.
there should be a new spack release before SC16, eval if we should move uberenv to this version.
breathe (doxygen bridge) integration is the only thing that requires us to use a special configure step for the config.py passed to sphinx. Since that step leverages our CMake build system, it makes using read-the-docs out of reach.
We aren't really leveraging breathe -- it's work considering simplifying to a static config.py file
needed for strawman
to support installing and running w/ specific python instance w/o using python path
proof of concept works in the node viewer, but this should be a general feature of the relay::WebServer, move this support into this class.
We shouldn't need to link these in client code b/c these are listed PRIVATE dependencies and statically linked. However when conduit is included by another CMake build system via its exported cmake targets these become link dependencies.
The easiest way to resolve this is to simply install them again.
take a pass to resolve compiler warnings prior to 0.2.0
domain abutment (communication surfaces meshes)
arm nesting hierarchies.
A void_array (a DataArray<void*> instance) would be useful for the case were you want to generically stride though and access the bytes for any array w/o type checking.
The void_array will still have access to the full data type spec, so the parts of the DataArray inteface related to set, to_json, etc helpers could still work fine, but we need to look if it has implications on plumbing of the template use.
conduit::Node provides const access to data, but does not provide const to traverse existing children.
Given the semantics of fetch and the []operator -- the best solution seems like a const variants of
Node* Node::fetch_ptr() and Node* Node::child_ptr()
We should also provide a const iterator.
since we are doing a cartesian product, the length of the coords arrays can be different.
in favor of setters on the web server class
we have quite a few options to configure before starring a relay web server, passing empty strings in the correct order will be error prone.
pull out options into setter methods (set_document_root(), etc), add add a simple serve() method.
required values are checked when serve() is called.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.