hadoop - How to resolve java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 2? -
i trying execute nltk in hadoop environment. following command used execution.
bin/hadoop jar $hadoop_home/contrib/streaming/hadoop-streaming-1.0.4.jar -input /user/nltk/input/ -output /user/nltk/output1/ -file /home/hduser/softwares/nltk/unsupervised_sentiment-master.zip -mapper /home/hduser/softwares/nltk/unsupervised_sentiment-master/sentiment.py
unsupervised_sentiment-master.zip --- contains dependent files required sentiment.py
i getting
java.lang.runtimeexception: pipemapred.waitoutputthreads(): subprocess failed code 2 @ org.apache.hadoop.streaming.pipemapred.waitoutputthreads(pipemapred.java:362) @ org.apache.hadoop.streaming.pipemapred.mapredfinished(pipemapred.java:576) @ org.apache.hadoop.streaming.pipemapper.close(pipemapper.java:135) @ org.apache.hadoop.mapred.maprunner.run(maprunner.java:57) @ org.apache.hadoop.streaming.pipemaprunner.run(pipemaprunner.java:36) @ org.apache.hadoop.mapred.maptask.runoldmapper(maptask.java:436) @ org.apache.hadoop.mapred.maptask.run(maptask.java:372) @ org.apache.hadoop.mapred.child$4.run(child.java:255) @ java.security.accesscontroller.doprivileged(native method) @ javax.security.auth.subject.doas(subject.java:415) @ org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation.java:1121) @ org.apache.hadoop.mapred.child.main(child.java:249)
any appreciated!!!
could pls post python files? guess that, need add #!/usr/bin/python top of py file. case when streaming using python.
best, hari
Comments
Post a Comment