Oracle Big Data Spatial and Graph - technical tips, best practices, and news from the product team
- October 11, 2017
3 Ways to Serialize and Write a Sub-Graph to Client Side (Part I)
When applying graph analytics or running graph queries, some applications need to work on a sub graph instead of the whole graph. In this blog, I am going to show three different approaches to serialize and write a sub-graph to a local file system on the client side. By making a sub graph available on the client side, it becomes easy to re-load it for further analysis in the future.
Approach #1 Serialize a Sub-Graph on the Server Side and Copy to the Client Side
The following Groovy script (containing Java code snippets) shows how one can issue a sub-graph serialization command from the client side and get the serialized graph data files generated on the server side. Important APIs (method names) are highlighted. Note that one needs to set "allow_local_filesystem"=true in pgx.conf (in directory /opt/oracle/oracle-spatial-graph/property_graph/pgx/conf/) on the server side.
cd /opt/oracle/oracle-spatial-graph/property_graph/dal/groovy/
sh ./gremlin-opg-nosql.sh
pgxServer="http://127.0.0.1:7007"
pgxSession=Pgx.getInstance(pgxServer).createSession("session1");
server = new ArrayList<String>();
server.add("bigdatalite:5000");
cfg = GraphConfigBuilder.forPropertyGraphNosql().setCreateEdgeIdMapping(true).setName("connections").setStoreName("kvstore") .setHosts(server) .hasEdgeLabel(true).setLoadEdgeLabel(true) .addEdgeProperty("weight", PropertyType.DOUBLE, "1000000") .setMaxNumConnections(2).build();
opg = OraclePropertyGraph.getInstance(cfg);
pgxGraph = session.readGraphWithProperties(opg.getConfig());
==>PgxGraph[name=connections,N=78,E=164,created=1507628356244]
ef = new EdgeFilter("edge.weight < 5.0");
subGraph = pgxGraph.filter(ef);
==>PgxGraph[name=connections_sub-graph_1,N=69,E=118,created=1507628611702]
graphConfig = GraphConfigBuilder.forFileFormat(Format.FLAT_FILE) .setVertexUris("/tmp/g.opv").setEdgeUris("/tmp/g.ope") .setSeparator(",").addEdgeProperty("weight", PropertyType.DOUBLE, "1000000") .build();
subGraph.store(graphConfig, true);
Now, on the server side, you can find a pair of flat files, g.opv and g.ope, under /tmp
[oracle@bigdatalite ~]$ head -5 /tmp/g.opv
2,%20,,,,
4,%20,,,,
11,%20,,,,
16,%20,,,,
18,%20,,,,
[oracle@bigdatalite ~]$ head -5 /tmp/g.ope
1078,2,1,collaborates,weight,4,,1.0,
1079,2,36,collaborates,weight,4,,1.0,
1077,2,35,admires,weight,4,,1.0,
1019,11,1,collaborates,weight,4,,1.0,
1020,11,10,collaborates,weight,4,,1.0,
Finally, copy (cp, scp, ftp, sftp or use whatever tool of your choice to bring) the above two files to the client side.