Comments (4)
Hi,
It's not clear which code you are trying to run because you have the statisticians example and the example from the docs which is what apparently is causing the segfault. For the statisticians example I get an empty output. To make your example work I had to replace relative paths with absolute paths wherever you name a file. I also added a comma in front of drop line 26 to make it clearer to me but that seems to only remove a harmless warning. I am willing to consider supporting relative paths, but if hadoop dfs doesn't have cd and pwd commands, it seems to me we would be trying to add a feature to the hadoop fs which is not really the responsibility of rmr.
As far as the example from the docs, it works for me. Could you please report rmr2, hadoop and os versions? Segfaults shouldn't occur no matter what you type. Thanks
from rmr2.
Hi,
Sorry for not mentioning that the 1st piece of code was also giving the same error as the statisticians' example (same stderr logs for the mappers). Also, I must mention that I'm only getting this error on my VM (machine configurations below).
VM config:
R: v2.15.2
rmr: rmr2 v2.2.0
OS: CentOS 6.4
OS: CDH3u4
RAM: 8GB
I also replaced all relative paths with absolute ones, but continue to get the error.
P.S. I agree with the absolute path logic that you've put forward.
from rmr2.
Hi,
I tried your suggestion of absolute paths for my example on a cluster, and it worked fine.
Thank you!
P.S. I still keep getting the segfault error in my VM.
from rmr2.
I reinstalled rmr2 v2.2.0 and I'm not getting the error anymore. I'm closing this issue, as the problem has been solved. But according to me it still makes sense to figure out why it occurred in the first place.
from rmr2.
Related Issues (20)
- rmr.normalize.path wrong on "/"
- The system cannot find the path specified. HOT 9
- Hadoop job log4j:WARN Please initialize the log4j system properly HOT 5
- Mapreduce warning: namespace 'rmr2' is not available and has been replaced HOT 1
- Question with regards to lzo-ubuntu.sh
- HADOOP_CMD getting lost... HOT 2
- from.dfs() has Error in RStudio, but runs well in R console
- how can i find "kv.cmp = rmr2:::kv.cmp" in "pkg / tests / mapreduce.R"
- Changing number of mapper/ reducer HOT 2
- parsing XML douments in rhadoop
- what's the meaning of "in.memory.combine"
- rmr2 mapreduce does not produce any output HOT 6
- RMR2 Installaction question
- Recommended version of rmr2 for hadoop 2.7 and R 3.3.1
- Have some tutorials or references here about rmr2? HOT 1
- Reduce task fails with org.apache.hadoop.streaming.PipeMapRed: PipeMapRed failed! HOT 2
- rhadoop giving null key value output HOT 2
- path and to.dfs with R launched from two different directories
- from.dfs returns nothing
- output NULL of key and val
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rmr2.