Triple_apply without mutex

User 1867 | 5/7/2015, 7:28:07 AM

Say I'm not modifying any vertex state in triple apply lambda. Is there a way to run the triple apply without the mutex for performance? I still need access to the relevant vertex data (the two edge endpoints), but just don't write to the vertices.

Thanks again, you guys have been great with support.

Comments

User 1190 | 5/7/2015, 4:36:00 PM

Yes. Definitely possible, but currently it is not exposed. I'm thinking about a more advanced API design, which will give user more control:

SGraph.iterative_triple_apply(apply_fn, num_iterations, required_vertex_fields, in_memory_vertex_fields, mutated_vertex_fields, required_edge_fields, mutated_edge_fields, ) Advantages: 1. when mutatedvertexfields is empty, we don't need to lock anything. 2. inmemoryvertex_fields will be pinned into memory, and avoid unnecessary serialization cost. 3. avoids generating a new SGraph every iteration.

Disadvantages: Too many options... Should be above to further simplify it.

This is still in discussion, and any feedback is welcomed:)


User 1867 | 5/7/2015, 5:51:06 PM

If I edit Dato Core to work around this for now, can I compile and run graphlab in python on an edited version?


User 1867 | 5/7/2015, 6:07:21 PM

Also, could it just check whether any of the listed "mutated_fields" are in vertices' SFrame? This prevents people from using duplicate field names on vertices and edges and benefitting from that optimization though I realize.


User 1190 | 5/8/2015, 7:25:13 AM

Sure you can hack Dato Core, compile and run it in python. Second one is a good idea and we should do it as an optimization. You can even contribute by sending pull request to the dato-core repo.

Thanks!