Edit
# FAQ This summarize all frequent asked questions in using DDF. --- * <a href="#ddf">DDF</a> * <a href="#ddf-yarn">DDF on Yarn</a> * <a href="#java">Java isses</a> * <a href="#R">R issues</a> --- <a name="ddf"></a> ### DDF 1. What DDF is not ? * DDF is not a database that run SQL. * DDF is not JUST a RDD. * DDF is not JUST a layer run on top of Spark. 2. What is good about DDF ? * DDF doesn't require users to write map(), reduce(), collect(). DDF value is a higher level of abstraction and any time you go one level higher, two things occur: (a) you can do more by saying less, and (b) you do give up arbitrary flexibility at the lower level. This is high productivity gains for developers. * DDF is more than just applying analytic functions to big data. It also enables data sharing (collaboration) via URIs. * DDF is also about providing a uniform API across different engines, of which Spark is only one. <a name="ddf-yarn"></a> ### DDF on Yarn 1. First you need to compile Spark with your Yarn version You can read instruction [here](http://spark.apache.org/docs/latest/building-with-maven.html) 2. Then build DDF on Spark that you just built. 3. To test the build you can follow ddf-shell in <a href="http://ddf.io/quickstart.html">tutorial</a> <a name="java"></a> ### Java issues 1.Q: How do I fix OutOfMemory while building DDF ? A: DDF uses maven to build project, you can set MAVEN memory to higher values, for example: ```sh export MAVEN_OPTS=-XX:MaxPermSize=128m ``` 2.Q: I don't have Java on my machine, how to install Java 7 on Mac ? A: Apple and Oracle do not support Java 1.7 out of the box, you need to install <a href="http://stackoverflow.com/questions/8826881/maven-install-on-mac-os-x" target="_blank">manually</a> 3.Q: I have this error JAVA_HOME is not set properly, how to fix that ? A: You need to set JAVA_HOME properly, for example: ```sh export JAVA_HOME=`/usr/libexec/java_home` ``` 4.Q: I saw this error message "You must build DDF's Java core and spark package firstly" A:You need to build ddf first. <a name="R"></a> ### R issues 1.Q: How to fix ClassNotFound exception while using R-DDF ? A: R-DDF use rJava to invoke DDF Java class, make sure you're using Java 1.7, R 3.x and you have installed rJava. This error happens mostly on MacOS X. - First you need to set JAVA PATH properly, in Mac it looks something like this ```sh JAVA_LD_LIBRARY_PATH=/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/server JAVA_LIBS='-L/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/server -ljvm' JAVA_CPPFLAGS='-I/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/include/darwin' ``` - Secondly ``` sh R CMD javareconf -e ``` - Then export ```sh DYLD_LIBRARY_PATH=/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/server/ ``` 2.Q: I have this error while running ```R CMD javareconf -e``` > make: llvm-gcc-4.2: No such file or directory A: This error occurs mostly on MacOSX. It happens because you can't install rJava. Please follow this <a href="http://stackoverflow.com/questions/19533220/cannot-install-r-package-from-source-in-mac-osx-maverick" target="_blank">instruction</a>