X

Oracle Big Data Spatial and Graph - technical tips, best practices, and news from the product team

  • October 11, 2017

3 Ways to Serialize and Write a Sub-Graph to Client Side (Part I)

Alan Wu
Architect

When applying graph analytics or running graph queries, some applications need to work on a sub graph instead of the whole graph. In this blog, I am going to show three different approaches to serialize and write a sub-graph to a local file system on the client side. By making a sub graph available on the client side, it becomes easy to re-load it for further analysis in the future.

Approach #1 Serialize a Sub-Graph on the Server Side and Copy to the Client Side

The following Groovy script (containing Java code snippets) shows how one can issue a sub-graph serialization command from the client side and get the serialized graph data files generated on the server side. Important APIs (method names) are highlighted. Note that one needs to set "allow_local_filesystem"=true in pgx.conf (in directory /opt/oracle/oracle-spatial-graph/property_graph/pgx/conf/) on the server side.


cd /opt/oracle/oracle-spatial-graph/property_graph/dal/groovy/
sh ./gremlin-opg-nosql.sh

pgxServer="http://127.0.0.1:7007"
pgxSession=Pgx.getInstance(pgxServer).createSession("session1");

server = new ArrayList<String>();
server.add("bigdatalite:5000");
cfg = GraphConfigBuilder.forPropertyGraphNosql().setCreateEdgeIdMapping(true).setName("connections").setStoreName("kvstore") .setHosts(server) .hasEdgeLabel(true).setLoadEdgeLabel(true) .addEdgeProperty("weight", PropertyType.DOUBLE, "1000000") .setMaxNumConnections(2).build();

opg = OraclePropertyGraph.getInstance(cfg);
pgxGraph = session.readGraphWithProperties(opg.getConfig());
==>PgxGraph[name=connections,N=78,E=164,created=1507628356244]

ef = new EdgeFilter("edge.weight < 5.0");

subGraph = pgxGraph.filter(ef);
==>PgxGraph[name=connections_sub-graph_1,N=69,E=118,created=1507628611702]


graphConfig = GraphConfigBuilder.forFileFormat(Format.FLAT_FILE) .setVertexUris("/tmp/g.opv").setEdgeUris("/tmp/g.ope") .setSeparator(",").addEdgeProperty("weight", PropertyType.DOUBLE, "1000000") .build();

subGraph.store(graphConfig, true);

Now, on the server side, you can find a pair of flat files, g.opv and g.ope, under /tmp

[oracle@bigdatalite ~]$ head -5 /tmp/g.opv
2,%20,,,,
4,%20,,,,
11,%20,,,,
16,%20,,,,
18,%20,,,,

[oracle@bigdatalite ~]$ head -5 /tmp/g.ope
1078,2,1,collaborates,weight,4,,1.0,
1079,2,36,collaborates,weight,4,,1.0,
1077,2,35,admires,weight,4,,1.0,
1019,11,1,collaborates,weight,4,,1.0,
1020,11,10,collaborates,weight,4,,1.0,
 

 Finally, copy (cp, scp, ftp, sftp or use whatever tool of your choice to bring) the above two files to the client side.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha