Comments (2)
I don't really see anything wrong with your approach of folding inner dimensions.
Probably there should be a better way to do that that does not involve makeArray right?
Can you elaborate on this. You can't create an array without a function like makeArray
, be it an unsafe or mutable variant of thereof. Why do you need a nested array if you are folding the inner array anyways?
Array P Ix2 (Array P Ix2 Double)
Such an array is impossible in general, because an element of a primitive array must have a Prim
instance and no array can have such an instance, therefore with nested arrays an outer array must be either delayed, or boxed.
Here is an expanded version of your example that does the same thing:
densityDelayed :: Array P Ix4 Double -> Array D Ix2 (Array D Ix2 Double)
densityDelayed a = A.makeArray Seq (A.Sz2 100 100) $ \(A.Ix2 x y) -> a A.!> x A.!> y
density :: Array P Ix4 Double -> Array P Ix2 Double
density a = A.compute (A.sum <$> densityDelayed a)
Delayed array (D
) is not directly represented by an in memory buffer, instead it is just a function that does the slicing into the original array. So, in both versions of the density
function (the version above and the one in your question) there will be only two primitive arrays allocated in memory: the one that was supplied as an argument and the one for the result. Shape shifting/slicing is handled in massiv with delayed arrays. Maybe that's what you meant by shape shifting?
One note on parallelization in this example. If the outer size is 100x100 and the inner arrays are very small then it could be more efficient to compute the inner sum sequentially, eg. size a == 100 :> 100 :> 5 :. 5
:
density :: Array P Ix4 Double -> Array P Ix2 Double
density a = A.makeArray Par (A.Sz2 100 100) $ \(A.Ix2 x y) -> A.sum (A.setComp A.Seq (a A.!> x A.!> y))
Another note on the size of the resulting array, it is better to make it a bit safer by getting the size of the source array:
density :: Array P Ix4 Double -> Array P Ix2 Double
density a = A.makeArray Par (A.Sz2 m n) $ \(A.Ix2 x y) -> A.sum (A.setComp A.Seq (a A.!> x A.!> y))
where
A.Sz4 m n _ _ = A.size a
from massiv.
Thanks! For some reason I assumed that makeArray
explicit will be less efficient than composing array functions where some magic fusion techniques kick in, but if that’s not the case, even better!
Also thanks for the explanation about the delayed arrays.
And yes, I was hard-coding the 100 just while playing around. I’ll fix that once my simulation no longer behaves completely weird (https://mastodon.online/@nomeata/109942791776084644 …) :-)
from massiv.
Related Issues (20)
- unstreamIntoM improvements
- iZipWith4 HOT 7
- Breaking changes for massiv-1.0 release HOT 1
- Provide external file (mmaped) representation HOT 3
- Safe stencils aren't completely safe HOT 1
- It would be handy to have `minIndexBy`, `minIndex`, `maxIndexBy`, `maxIndex`.
- Lazier arrays? HOT 4
- Add vector to matrix (and similar operations) HOT 1
- odd behaviour of Sz1 when subtracting something HOT 2
- massiv-serialise/persist testsuites need lot of mem to build HOT 3
- Fix CPP usage so fourmolu doesn't choke
- Add fourmolu to CI
- `findIndex` should not be in `Data.Massiv.Vector` HOT 2
- `resize` should be called `reshape`
- Stand-alone package for Data.Massiv.Core.Index HOT 1
- Ease of Use Improvements
- Gitter link in question template 404s
- Convert `Array U Ix1 a` to `Unboxed.Vector a` without copying HOT 1
- massiv-test test suite failing to build on stackage HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from massiv.