Comments (2)
The feature is "deprecated" to be precise. The good news is that we didn't axe it in 2.1 so it's there, still deprecated. The way you are using it, that's what it's there for, and it wouldn't be deprecated if that's all that people did with it. But it's a hole in the abstraction layer and it's dangerous. Some people jumped in it and started changing separators, tampering with the format and all sort of other stuff that rmr uses internally to implement its features. It's like using a compiler and then tampering with the assembly code a bit. The first time the compiler changes, or if you even change an option in the compiler call, or maybe even a line in the program, and the thing breaks apart. So you may say once people are warned about it, they are grown ups, they should know better. Why penalize people who use the feature the way it's intended? Unfortunately it's not that simple. I've seen on line rmr programs where most of the lines were arguments to backend.parameters. Those things get posted and other people learn that that's the way to write programs and then they complain here that rmr doesn't work. And besides getting a lot of avoidable and difficult support work, one also gets the "rmr breaks a lot" reputation, instead of "rmr breaks a lot if you misuse it". But the plan is to replace backend.parameters with a more restricted set of options that will capture the safe and common uses of backend parameters. There was already an issue concerning the number of mapper and reducers open in the old repo, I will reopen it here. You can open additional issues if you have other uses of backend.parameters that you feel are important and safe. Thanks for your feedback.
from rmr2.
I opened issue #10 on the main topic raised here. If there are more important and acceptable uses of backend parameters we can open additional ones.
from rmr2.
Related Issues (20)
- rmr.normalize.path wrong on "/"
- The system cannot find the path specified. HOT 9
- Hadoop job log4j:WARN Please initialize the log4j system properly HOT 5
- Mapreduce warning: namespace 'rmr2' is not available and has been replaced HOT 1
- Question with regards to lzo-ubuntu.sh
- HADOOP_CMD getting lost... HOT 2
- from.dfs() has Error in RStudio, but runs well in R console
- how can i find "kv.cmp = rmr2:::kv.cmp" in "pkg / tests / mapreduce.R"
- Changing number of mapper/ reducer HOT 2
- parsing XML douments in rhadoop
- what's the meaning of "in.memory.combine"
- rmr2 mapreduce does not produce any output HOT 6
- RMR2 Installaction question
- Recommended version of rmr2 for hadoop 2.7 and R 3.3.1
- Have some tutorials or references here about rmr2? HOT 1
- Reduce task fails with org.apache.hadoop.streaming.PipeMapRed: PipeMapRed failed! HOT 2
- rhadoop giving null key value output HOT 2
- path and to.dfs with R launched from two different directories
- from.dfs returns nothing
- output NULL of key and val
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rmr2.