alejandro-isaza / hdf5kit Goto Github PK
View Code? Open in Web Editor NEWHDF5 for iOS and OS X
License: MIT License
HDF5 for iOS and OS X
License: MIT License
Hi,
I'm trying to add HDF5Kit to a new project on a system running macOS Catalina beta 3 with Xcode 11 beta 3. In "Build Phases", under "Link Binary With Libraries", I tried adding the library with the "+" button, and then choosing "Add package dependency..." from the popup menu. This has worked for adding other projects for me, but for HDF5Kit, I get the error shown below:
I have verified that I do have hdf5 installed in /usr/local.
Any ideas why this might be happening?
Dataset sometimes includes multi-dimensional array. For example, a weights parameter of convolution layer has 4 axes (e.g. 3, 3, 1, 32). In h5py, group[parameter_name]
returns multi-dimensional array and group[parameter_name].shape
returns a tuple like (3, 3, 1, 32). To handle multi-dimentional operation, I need this support.
Regarding for read function that returns multi-dimensional array, related code in h5py might be found here: https://github.com/h5py/h5py/blob/master/h5py/h5d.pyx#L145
Regarding for shape property, it use get_space()
that uses SpaceID(H5Dget_space(self.id))
and get_simple_extent_dims()
: https://github.com/h5py/h5py/blob/master/h5py/h5d.pyx#L126
I really need these API. Please let me know if you have any other ways to get them. Thank you for your support!
I am trying to track down a crash that seems to happen sporadically for some people that use my application.
The crash only occurs in release and not in debug, which lead me to think that it has something to do with the applied optimisation level to HDF5Kit upon release?
In the screenshot below, you see which function failed. Line 96 is a precondition statement, precondition(data.count == size, "Data size doesn't match Dataspace dimensions")
I already created a local copy of the data before giving it to the function to avoid changes to the dataset by other threats while storage is happening. That did not solve the issue.
Now, I set optimisation of the HD5Kit framework to -O0. This seems to have solved the issue.
Please let me know if I am missing something!
(I am using the latest release, 0.2.1
With the new build, using swiftc 2.2-dev (the snapshot release) Dataset.swift
fails when a [Double], [Float], etc. is returned when the function return type definition is [AnyObject]. The good news is that other than that, HDF5Kit builds on Linux (Ubuntu 15.10)
I have a HDF5 file structured like the following:
file -- group --- dataSet
|- group |- dataSet
....
The file has an attribute named "group_names". Firstly, I'd like to get each group's name by using the string "group_names". I tried the following.
do {
let groupNamesStringAttribute = try file.openStringAttribute("group_names")
do {
let groupNames = try groupNamesStringAttribute?.read()
...
file.openStringAttribute
worked, but when I run groupNamesStringAttribute?.read()
, it returned the following errors.
.../Pods/HDF5Kit/dist/src/H5A.c line 695 in H5Aread(): unable to read attribute
major: Attribute
minor: Read failed
#001: .../Pods/HDF5Kit/dist/src/H5Aint.c line 524 in H5A__read(): unable to convert between src and dst datatypes
major: Attribute
minor: Feature is unsupported
#002: .../Pods/HDF5Kit/dist/src/H5T.c line 4560 in H5T_path_find(): no appropriate function for conversion path
major: Datatype
minor: Unable to initialize object
ioError
Please let me know if you have any idea to solve this. Also, I'd like to know the way how to get the attribute's name itself (in this case, "group_names").
I'm using xcode 11.2 beta 2 and Swift 5.
I'm reading in a relatively small hdf5 file, 650kb.
I'm not able to read in string attributes or string dataset
let a = group.openStringAttribute("category")
let category = try a?.read()
let list2 = listElementsGroup?.openStringDataset("materials_container_index_table")
let a2 = try list2?.read()
Both of these throw an ioError
I can open the file in HDFView. In this capture I can read in the floating point "value" but not the category or the unit.
I'm not sure the best way to debug what is happening here.
readFixedLength() once worked for a hdf5 file to get the name of group, but when I tried to use it for a different hdf5 file, it threw an error. When I checked what was returned, it seemed the last element was not the value it should be returned.
(lldb) po layerNamesStringAttribute.read()
▿ 10 elements
- 0 : "convolution1d_30"
- 1 : "activation_46"
- 2 : "reshape_12"
- 3 : "convolution2d_41"
- 4 : "activation_47"
- 5 : "maxpooling2d_11"
- 6 : "flatten_11"
- 7 : "dense_21"
- 8 : "activation_48"
- 9 : "dense_2@�\u{16}p\u{01}"
The last element should be "dense_22". I'll post if I find further information.
Hi pretty new to Swift so I just want to see if this package supports swift 5.5 or not. Thanks!
There should be support for reading and writing hyper slabs.
It's a continued topic from #21.
The output should be variable length, but it returns fixed length string (maximum: 11 characters) when I use read()
. Are there any ways to fix this?
The current output Optional(["convolution", "activation_", "convolution", "activation_", "maxpooling2", "dropout_1", "flatten_1", "dense_1", "activation_", "dropout_2", "dense_2", "activation_"])
should be
Optional(["convolution2d_1", "activation_1", "convolution2d_2", "activation_2", "maxpooling2d_1", "dropout_1", "flatten_1", "dense_1", "activation_3", "dropout_2", "dense_2", "activation_4"])
I'm not 100% sure, but in Source/StringAttribute.swift
, line49 somehow converts each string's data to maximum 11 characters length string. That invokes a wrong result mentioned here.
a6b4b86#commitcomment-19898264
I'm wondering if you guys have any ideas/suggestions about this. I am running a pretty simple piece of read code to extract from a dataset with just four Doubles in it, namely:
guard let file = File.open(path, mode: .ReadOnly) else { fatalError("Failed to open \(path)") } guard let mmmDS = file.openDataset("mmm", type: Double.self) else { fatalError("Failed to open MMM node") } let mmm = mmmDS.read() as! [Double]
Other than setting up some globals, this is all the code does. On running this, Xcode 7.1.1 on OS X 10.10.5 crashes on me. If I run from the command-line, I see:
fatal error: Don't know how to read Double: file /Users/tim/work/carthage-builds/Carthage/Checkouts/HDF5Kit/Source/Dataset.swift, line 98 Illegal instruction: 4
It seems the the Element.self == Double check is failing somehow. If I put the code into an HDF5Kit test, it works fine. I know the HDF5 is good. I can read it without problem using NumPy and it works in the test.
I suspect it is something to do with how I am building or adding the framework. I'm building HDF5Kit with Carthage. Only difference from master is that I have 'contains embedded swift code' = on, as without this the executable can't find the Swift libraries. I also use install_name_tool -change ...
to avoid rpath
issues by just using an absolute path:
otool -L LSDAlgo LSDAlgo: /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0) /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1252.0.0) /Users/tim/work/carthage-builds/Carthage/Build/Mac/HDF5Kit.framework/Versions/A/HDF5Kit (compatibility version 1.0.0, current version 1.0.0) /Users/tim/work/carthage-builds/Carthage/Build/Mac/Upsurge.framework/Versions/A/Upsurge (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1225.1.1) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1253.0.0) /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 600.0.0) /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
it would be good to know a way to use HDF5Kit which avoids this hard crash issue and I'd value any thoughts.
As far as I can tell, this library does not include support for reading/writing/deleting attributes ( https://www.hdfgroup.org/HDF5/doc1.6/UG/13_Attributes.html )
While it's not clean or complete, I was able to create attributes of type Double on groups by adding this code to Group.swift:
/// Create an attribute
public func createAttribute(name: String) -> hid_t {
let dataspace = Dataspace.init(dims: [1, 1])
let datatype = Datatype(type: Double.self)!
let attributeId = name.withCString{
return H5Acreate2(id, $0, datatype.id, dataspace.id, 0, 0)
}
return attributeId
}
/// Write an attribute
public func writeAttribute(attributeId: hid_t, value: Double) {
let datatype = Datatype(type: Double.self)!
var cValue = CDouble(value)
H5Awrite(attributeId, datatype.id, &cValue)
}
/// Close an attribute
public func closeAttribute(attributeId: hid_t) {
H5Aclose(attributeId)
}
I added .package(url: "https://github.com/aleph7/CHDF5.git", from: "1.0.0")
to my dependencies and "HDF5Kit"
to the target dependencies, but get that error.
Implement methods like those in HDF5 Lite
The write and read functions should (also) take pointers so that you can read/write data into custom buffer objects without having to copy it from/to an Array
.
Reading Int datasets in 32-bit architectures (like the iPad 2) triggers an assertion.
There should be support for unlimited dimensions to that new data can be appended to an existing file.
I'm trying to read a dataset from a group. This dataset is composed by 47033 rows x 2 columns. Using the read() functions I just get a 94066 array. How can I get the right object? Thanks.
Hello there,
I have hrtf.soap file with me, I want to read the content of that file using HDF5, one of my friend is doing using python.
Please help me know how can I do using this library.
Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.