Basic Tutorial

User 121 | 3/28/2014, 5:39:23 PM

I am following basic tutorial: http://docs.graphlab.org/using_graphlab.html

I cannot see the results in the output1oufof_N files. My code is as follows:

int main(int argc, char **argv) {

graphlab::mpitools::init(argc, argv); graphlab::distributedcontrol dc;

dc.cout() << "Calculation of PageRank:\n";

graphtype graph(dc); graph.load("graph.txt", lineparser); //call back

graph.finalize();

graphlab::omniengine<pagerankprogram> engine(dc, graph, "synchronous"); engine.signal_all(); engine.start();

graph.save( "output", graph_writer(), false, // set to true if each output file is to be gzipped true, //whether vertices are saved false);// whether edges are saved

graphlab::mpi_tools::finalize(); }

For some reasons, I could not upload entire .cpp file, but I have implemented graphwriter, lineparser() as indicated in the tutorial

the output I see after executing the binary is :

INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph... INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph... INFO: synchronousengine.hpp(start:1299): Iteration counter will only output every 5 seconds. INFO: synchronousengine.hpp(start:1314): 0: Starting iteration: 0 INFO: synchronousengine.hpp(start:1363): Active vertices: 0 INFO: synchronousengine.hpp(start:1424): 0 iterations completed. Updates: 0 INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph...

Thank you for any help!

Comments

User 6 | 3/28/2014, 5:46:36 PM

It seems that something is wrong with your input file or its loading as the number of active vertices is 0, namely you have 0 vertices in your input graph. What do you have inside graph.txt ?

THanks


User 121 | 3/28/2014, 11:56:53 PM

$ cat graph.txt 1 a 4 10 4 b 10 10 c 11 1 11 d 10

I am reading first the vid then the page_name and then edges


User 6 | 3/29/2014, 4:22:50 AM

Please send me your code via a personal email I would be happy to take a look.


User 6 | 3/29/2014, 4:44:22 AM

p.s. It may be related to this issue: http://forum.graphlab.com/discussion/15/error-in-loading-graph-my-first-app-tutorial from some reason the website was not refreshed property. Can you verify you got "return true" on both cases inside the line_parser function?

Thanks


User 121 | 3/29/2014, 8:40:20 AM

Hi Danny, I have return true(); in both cases inside lineparse() function. Still, I can see no output in the output.* files.

Here is my line_parser() function:

bool lineparser(graphtype& graph, const std::string& filename, const std::string& textline) {

bool ret; std::cout << textline; std::cout << "\n";

std::stringstream strm(textline);

std::cout << "strm CTOR called";

graphlab::vertexidtype vid; std::string pagename;

//first entry in the line is a vertex ID strm >> vid; strm >> pagename;

std::cout << "Adding vertex..."; getchar(); // Insert this web page graph.add_vertex(vid, webpage(pagename));

// While there are elements in the line, continue to read until we are done while (1) { graphlab::vertexidtype othervid; strm >> othervid; if (strm.fail()) return true;

std::cout << "vid = " << vid << " and other_vid = " << other_vid << "\n";

ret = graph.add_edge(vid, other_vid);
if (ret) {
  std::cout << "edge added\n";
  getchar();
}

} return true; }

Thank you for your time


User 121 | 3/30/2014, 5:48:50 PM

Sorry, but this question is not yet answered (correctly ) . And I see that it is tagged as "Answered". Can somebody re-open the question ? thanks


User 6 | 3/30/2014, 5:52:56 PM

Hi, I was not able to reproduce your error using your code. Can you send us the full command line you are using? And the full output. Also please send us the output of ./pagerank graph.txt mpiexec -n 1 ./pagerank graph.txt mpiexec -n 2 ./pagerank graph.txt

Thanks


User 121 | 3/30/2014, 5:57:11 PM

unmesh@unmesh-laptop:~/programming/graphlab/release/apps/mypagerank$ ./mypagerank graph.txt GRAPHLABSUBNETID/GRAPHLABSUBNETMASK environment variables not defined. Using default values Subnet ID: 0.0.0.0 Subnet Mask: 0.0.0.0 Will find first IPv4 non-loopback address matching the subnet INFO: dc.cpp(init:573): Cluster of 1 instances created. Calculation of PageRank: INFO: distributedgraph.hpp(setingressmethod:3201): Automatically determine ingress method: grid INFO: distributedgraph.hpp(loadfromposixfs:2190): Loading graph from file: ./graph.txt Adding vertex...1 : a.comAdding vertex...4 : b.comAdding vertex...10 : c.comAdding vertex...11 : d.comINFO: distributedingressbase.hpp(finalize:185): Finalizing Graph...

vertices: 0#Edges: 0

INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph... INFO: synchronousengine.hpp(start:1299): Iteration counter will only output every 5 seconds. INFO: synchronousengine.hpp(start:1424): 0 iterations completed. Updates: 0 Finished running engine in -2.98023e-09 seconds. INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph... unmesh@unmesh-laptop:~/programming/graphlab/release/apps/my_pagerank$

I observed that somehow add_vertex was not successful. I checked the return value which was true.


User 121 | 3/30/2014, 5:57:45 PM

unmesh@unmesh-laptop:~/programming/graphlab/release/apps/mypagerank$ mpiexec -n 1 ./mypagerank graph.txt GRAPHLABSUBNETID/GRAPHLABSUBNETMASK environment variables not defined. Using default values Subnet ID: 0.0.0.0 Subnet Mask: 0.0.0.0 Will find first IPv4 non-loopback address matching the subnet INFO: dc.cpp(init:573): Cluster of 1 instances created. Calculation of PageRank: INFO: distributedgraph.hpp(setingressmethod:3201): Automatically determine ingress method: grid Adding vertex...1 : a.comAdding vertex...4 : b.comAdding vertex...10 : c.comAdding vertex...11 : d.comINFO: distributedgraph.hpp(loadfromposixfs:2190): Loading graph from file: ./graph.txt INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph...

vertices: 0#Edges: 0

INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph... INFO: synchronousengine.hpp(start:1299): Iteration counter will only output every 5 seconds. INFO: synchronousengine.hpp(start:1424): 0 iterations completed. Updates: 0 Finished running engine in -2.98023e-09 seconds. INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph... unmesh@unmesh-laptop:~/programming/graphlab/release/apps/my_pagerank$


User 6 | 3/30/2014, 6:04:55 PM

Are you using the latest version from github? I am using your code (after removing the getchar() you added) and getting

./pagerank2 graph.txt GRAPHLABSUBNETID/GRAPHLABSUBNETMASK environment variables not defined. Using default values Subnet ID: 0.0.0.0 Subnet Mask: 0.0.0.0 Will find first IPv4 non-loopback address matching the subnet INFO: dc.cpp(init:573): Cluster of 1 instances created. Calculation of PageRank: INFO: distributedgraph.hpp(setingressmethod:3201): Automatically determine ingress method: grid INFO: distributedgraph.hpp(loadfromposixfs:2190): Loading graph from file: ./graph.txt 1 a 4 10 strm CTOR calledAdding vertex...vid = 1 and othervid = 4 edge added vid = 1 and othervid = 10 edge added 4 b 10 strm CTOR calledAdding vertex...vid = 4 and othervid = 10 edge added 10 c 11 1 strm CTOR calledAdding vertex...vid = 10 and othervid = 11 edge added vid = 10 and othervid = 1 edge added 11 d 10 strm CTOR calledAdding vertex...vid = 11 and othervid = 10 edge added INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph... INFO: distributedingressbase.hpp(exchangeglobalinfo:532): Graph info: nverts: 4 nedges: 6 nreplicas: 4 replication factor: 1 INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph... INFO: synchronousengine.hpp(start:1299): Iteration counter will only output every 5 seconds. INFO: synchronousengine.hpp(start:1314): 0: Starting iteration: 0 INFO: synchronousengine.hpp(start:1363): Active vertices: 4 INFO: synchronousengine.hpp(start:1412): Running Aggregators INFO: synchronousengine.hpp(start:1424): 1 iterations completed. Updates: 4 INFO: distributedingress_base.hpp(finalize:185): Finalizing Graph... a 0.15 c 0.15 b 0.15 d 0.15


User 121 | 3/30/2014, 6:16:44 PM

Hello, git pull gives me "Already up to date" also git diff remote/origin/HEAD gives me nothing.

This is really going beyond me. Should I set up some local cluster env ? I suppose not.. What should I do ? I want to move on , but if basic computation is not running as I want, I am really feeling stuck !

PS: I had done the same last week without any difficulty and then my hard disk crashed.


User 6 | 3/30/2014, 6:18:40 PM

Do you mean that the same computation worked for you last week and now it does not? In that case you hard disk crash got some files in unstable state. I suggest removing graphlab and installing everything again. Or even better try to switch your machine.


User 121 | 3/30/2014, 6:27:19 PM

I have re-installed everything. helloworld application works without problem. It's only addvertex() which is not behaving as I expect. That is why I see 0 (active) vertices and thus, no output.. :-( Did you run the code I sent you without any changes ? (on the machine without any special configuration) I only did ./configure and then doing make inside graphlab/release/app/my_pagerank folder


User 6 | 3/30/2014, 6:43:51 PM

The output you reported on the forum does not match the output I get from the program you sent me by email. I am almost sure you are using different code. I will send you using email your code to try again.


include &lt;string&gt;

include &lt;graphlab.hpp&gt;

struct webpage { std::string pagename; double pagerank; int counter; // for self scheduling

webpage() : pagerank (0.0), counter(0){} explicit webpage (std::string name): pagename(name), pagerank(0.0), counter(0){}

void save (graphlab::oarchive& oarc) const{ oarc << pagename << pagerank << counter ; }

void load (graphlab::iarchive& iarc) { iarc >> pagename >> pagerank >> counter; } };

typedef graphlab::distributedgraph&lt;webpage, graphlab::empty&gt; graphtype;

bool lineparser(graphtype& graph, const std::string& filename, const std::string& textline) {

bool ret; std::cout << textline; std::cout << "\n";

std::stringstream strm(textline);

std::cout << "strm CTOR called";

graphlab::vertexidtype vid; std::string pagename;

//first entry in the line is a vertex ID strm >> vid; strm >> pagename;

std::cout << "Adding vertex..."; // Insert this web page graph.add_vertex(vid, webpage(pagename));

// While there are elements in the line, continue to read until we are done while (1) { graphlab::vertexidtype othervid; strm >> othervid; if (strm.fail()) return true;

std::cout << "vid = " << vid << " and other_vid = " << other_vid << "\n";

ret = graph.add_edge(vid, other_vid);
if (ret) {
  std::cout << "edge added\n";
}

} return true; }

class pagerankprogram: public graphlab::ivertexprogram<graphtype, double>, public graphlab::ISPOD_TYPE {

public: // we are going to gather on all the in-edges edgedirtype gatheredges(icontexttype& context, const vertextype& vertex) const { return graphlab::INEDGES; }

// for each in-edge GATHER the weighted sum of the edge
double gather(icontext_type& context, const vertex_type& vertex, edge_type& edge) const {
  return edge.source().data().pagerank / edge.source().num_out_edges();
}

// Use the total rank of adjacent pages to update this page
void apply(icontext_type& context, vertex_type& vertex, const gather_type& total) {
  double newval = total * 0.85 + 0.15;
  vertex.data().pagerank = newval;
  //++vertex.data().counter;
  //if (vertex.data().counter < 10) context.signal(vertex);
  // Here instead of 10, we can compare against a global variable set by a command line option
}

//No scatter needed. Return NO_EDGES
edge_dir_type scatter_edges(icontext_type& context, const vertex_type& vertex) const {
  return graphlab::NO_EDGES;
}

};

class graphwriter { public: std::string savevertex(graphtype::vertextype v) { std::stringstream strm;

  strm << v.data().pagename << "\t" << v.data().pagerank << "\n";
  std::cout << strm.str();
  return strm.str();
}

// we are not interested in edges for page rank output
std::string save_edge(graph_type::edge_type e) {return "";}

};

int main(int argc, char **argv) {

graphlab::mpitools::init(argc, argv); graphlab::distributedcontrol dc;

dc.cout() << "Calculation of PageRank:\n";

graphtype graph(dc); graph.load("graph.txt", lineparser); //call back

graph.finalize();

graphlab::synchronousengine<pagerankprogram> engine(dc, graph); engine.signal_all(); engine.start();

graph.save( "output", graph_writer(), false, // set to true if each output file is to be gzipped true, //whether vertices are saved false);// whether edges are saved

graphlab::mpi_tools::finalize(); }


HTTP


User 121 | 3/30/2014, 7:45:52 PM

I just removed some couts.. Nothing else. I redownloaded and re-make'ed everything. Now, the output is as follows: (same as your output except that I don't get results)

// unmesh@unmesh-laptop:~/programming/graphlab/release/apps/mypagerank$ ./mypagerank graph.txt GRAPHLABSUBNETID/GRAPHLABSUBNETMASK environment variables not defined. Using default values Subnet ID: 0.0.0.0 Subnet Mask: 0.0.0.0 Will find first IPv4 non-loopback address matching the subnet INFO: dc.cpp(init:573): Cluster of 1 instances created. Calculation of PageRank: INFO: distributedgraph.hpp(setingressmethod:3201): Automatically determine ingress method: grid INFO: distributedgraph.hpp(loadfromposixfs:2190): Loading graph from file: ./graph.txt 1 a 4 10 strm CTOR calledAdding vertex...1 : avertex added vid = 1 and othervid = 4 edge added vid = 1 and othervid = 10 edge added 4 b 10 strm CTOR calledAdding vertex...4 : bvertex added vid = 4 and othervid = 10 edge added 10 c 11 1 strm CTOR calledAdding vertex...10 : cvertex added vid = 10 and othervid = 11 edge added vid = 10 and othervid = 1 edge added 11 d 10 strm CTOR calledAdding vertex...11 : dvertex added vid = 11 and othervid = 10 edge added INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph...

vertices: 0#Edges: 0

INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph... INFO: synchronousengine.hpp(start:1299): Iteration counter will only output every 5 seconds. INFO: synchronousengine.hpp(start:1424): 0 iterations completed. Updates: 0 Finished running engine in -1.19209e-08 seconds. INFO: distributedingressbase.hpp(finalize:185): Finalizing Graph...

//


User 6 | 3/31/2014, 4:45:31 AM

I am getting the following output after finalizing graph:

INFO: distributedingressbase.hpp(exchangeglobalinfo:532): Graph info: nverts: 4 nedges: 6 nreplicas: 4 replication factor: 1

While you don't which makes me suspect you are using older or version of graphlab where something is wrong


User 121 | 3/31/2014, 4:45:50 PM

Hello, Apparently the problem was my OS was 32 bit. I then installed 64 bit Ubuntu Precise Pangolin on virtual box and reinstalled graphlab there. Now I hit yet another novel error:

GRAPHLABSUBNETID/GRAPHLABSUBNETMASK environment variables not defined. Using default values Subnet ID: 0.0.0.0 Subnet Mask: 0.0.0.0 Will find first IPv4 non-loopback address matching the subnet ERROR: fibercontrol.cpp(launch:270): Check failed: b<nworkers [1 < 1] [hdnode01:28186] * Process received signal * [hdnode01:28186] Signal: Aborted (6) [hdnode01:28186] Signal code: (-6) [hdnode01:28186] [ 0] /lib/x8664-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fb49d0e3cb0] [hdnode01:28186] [ 1] /lib/x8664-linux-gnu/libc.so.6(gsignal+0x35) [0x7fb49ac9d425] [hdnode01:28186] [ 2] /lib/x8664-linux-gnu/libc.so.6(abort+0x17b) [0x7fb49aca0b8b] [hdnode01:28186] [ 3] ./pagerank,(ZN8graphlab13fibercontrol6launchEN5boost8functionIFvvEEEmNS18fixeddensebitsetILi64EEE+0x2e0) [0x764370] [hdnode01:28186] [ 4] ./pagerank,(ZN8graphlab11fibergroup6launchERKN5boost8functionIFvvEEENS18fixeddensebitsetILi64EEE+0xbb) [0x768715] [hdnode01:28186] [ 5] ./pagerank,(ZN8graphlab19distributedcontrol4initERKSt6vectorISsSaISsEERKSstmNS12dccommtypeE+0x8a4) [0x772aa2] [hdnode01:28186] [ 6] ./pagerank,(ZN8graphlab19distributedcontrolC1Ev+0x5bb) [0x7704d9] [hdnode01:28186] [ 7] ./pagerank,(main+0x50) [0x653688] [hdnode01:28186] [ 8] /lib/x8664-linux-gnu/libc.so.6(__libcstartmain+0xed) [0x7fb49ac8876d] [hdnode01:28186] [ 9] ./pagerank,() [0x65329d] [hdnode01:28186] * End of error message *


mpirun noticed that process rank 0 with PID 28186 on node hdnode01 exited on signal 6 (Aborted).


User 6 | 3/31/2014, 4:54:08 PM

You allowed only one core to be used by the virtual machine. Please increase the number of cores to at least two in the virtual machine settings.

And by the way, Graphlab does not support 32 bit OS, as written clearly here: http://graphlab.org/projects/getting_started.html

Thanks


User 121 | 3/31/2014, 10:39:32 PM

Thanks Danny for your time! Finally it works ! I see all the output files with real data.


User 1768 | 6/16/2015, 7:17:56 PM

I have a problem in power graph. if my graph will be in this format, similar to page rank, but, instead of the name of the page, it is a double number, It means, each vertices as a double value and i wand to do some calculation for these values.

1 1.5 4 10 4 2.3 10 10 4.5 11 1 11 2.7 10

When I wrote in line parser

        graphlab::vertex_id_type vid;
        double node_value;

        //first entry in the line is a vertex ID
        strm >> vid;
        strm >> node_value;
graph.add_vertex(vid, vdata(node_value));

and then in vdata struct:

    struct vdata {
    double num_value;

    vdata() : num_value (0.0){}
    explicit vdata (double value_user): num_value(value_user){}
    save-----
    load-----

};

It can not finish finalizing graph, and read all vertices and edges, it gave me this error:

INFO: distributedingressbase.hpp(finalize:199): Finalizing Graph... ERROR: basic_types.hpp(exec:233): Check failed: c<=3 [ <= 3]

What is the problem?, If you know, Please guide me.