[PowerGraph] can not load graph data from hdfs

User 2603 | 11/14/2015, 2:17:30 PM

I use --graph hdfs://hostname/path/to/file to load hdfs data. But it shows the error below

n in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) Can't construct instance of class org.apache.hadoop.conf.Configuration ERROR: hdfs.hpp(hdfs:111): Check failed: filesystem != __null

Comments

User 1592 | 11/14/2015, 3:47:37 PM

Most likely you need to set Hadoop Classpath. See for example: http://forum.dato.com/discussion/714/error-while-using-asynchronous-engine-with-multiple-machines


User 2603 | 11/15/2015, 4:15:25 AM

I use hadoop 2.7.1. Does powergraph use hadoop1.0.1?


User 19 | 11/17/2015, 2:13:38 AM

It appears that it uses Hadoop 1.2.1. See here: https://github.com/dato-code/PowerGraph/blob/4d48710427381e5c9590a91023ee00dd6ecda2f3/scripts/ec2_tools/setup-hadoop

Thanks, Chris