Triple_apply without mutex

User 1867 | 5/7/2015, 7:28:07 AM

Say I'm not modifying any vertex state in triple apply lambda. Is there a way to run the triple apply without the mutex for performance? I still need access to the relevant vertex data (the two edge endpoints), but just don't write to the vertices.

Thanks again, you guys have been great with support.


User 1190 | 5/7/2015, 4:36:00 PM

Yes. Definitely possible, but currently it is not exposed. I'm thinking about a more advanced API design, which will give user more control:

SGraph.iterative_triple_apply(apply_fn, num_iterations, required_vertex_fields, in_memory_vertex_fields, mutated_vertex_fields, required_edge_fields, mutated_edge_fields, ) Advantages: 1. when mutatedvertexfields is empty, we don't need to lock anything. 2. inmemoryvertex_fields will be pinned into memory, and avoid unnecessary serialization cost. 3. avoids generating a new SGraph every iteration.

Disadvantages: Too many options... Should be above to further simplify it.

This is still in discussion, and any feedback is welcomed:)

User 1867 | 5/7/2015, 5:51:06 PM

If I edit Dato Core to work around this for now, can I compile and run graphlab in python on an edited version?

User 1867 | 5/7/2015, 6:07:21 PM

Also, could it just check whether any of the listed "mutated_fields" are in vertices' SFrame? This prevents people from using duplicate field names on vertices and edges and benefitting from that optimization though I realize.

User 1190 | 5/8/2015, 7:25:13 AM

Sure you can hack Dato Core, compile and run it in python. Second one is a good idea and we should do it as an optimization. You can even contribute by sending pull request to the dato-core repo.