Skip to content

Commit 2423080

Browse files
omarlarusomarlarus
committed
Documentation on README
1 parent 49db4d4 commit 2423080

4 files changed

Lines changed: 234 additions & 63 deletions

File tree

‎README.md‎

Lines changed: 12 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,78 +1,27 @@
11
# doc2graph
2-
Convert JSON from document-oriented DB to neo4j graph.
2+
Convert JSON from document-oriented DB to [neo4j](https://neo4j.com/) graph.
33

4-
This project is in BETA version, so it's absolutely no warranty.
4+
This project is in BETA version, so it's **absolutely no warranty**.
55

6-
doc2graph is a project for synchronization from document-oriented database to Neo4J.
76
The goal is to allow graph analizes on documental data using Neo4j; documents are stored into Neo4j reusing existing nodes so we can analyze document relationships.
8-
Nowadays only exists the Couchbase connector, but the infrastructure can be connected with every source.
9-
There is neo4j-json project that expose some procedures for digest JSON document via cypher query.
7+
The logic to convert JSON into a graph is stored into neo4j as procedures, so it can ben used from any kind of JSON source.
8+
Furthermore, the logic is configurable and programmable: you can obtain exactly the graph you need for yours analysis.
109

11-
# Neo4j-json install
12-
Neo4j-json is a neo4j plugin, writed for version 3.0 or later, that create these procedures:
13-
- json.upsert({key},{json)
14-
- json.delete({key})
10+
# neo4j-json
11+
It's the core of `doc2graph` project. It's a neo4j's plugin and create some [procedure](http://neo4j.com/docs/developer-manual/current/extending-neo4j/procedures/) to call in cypher.
1512

16-
These procedures can be called from cypher query and are indipendent from the source of json.
17-
To build the default configuration you have to download source and run
18-
19-
```
20-
mvn package
21-
cp target/neo4j-json\*.jar ${NEO\_HOME}/plugins/
22-
${NEO\_HOME}/bin/neo4j restart
23-
```
24-
25-
# Neo4j-json configuration
26-
Not yet implemented. In version 1.0 will be possible to custumize these feautures:
27-
- document ID
28-
- relationships management
29-
30-
# DCP connector
31-
couchbase-neo4j-connector is a standalone program that transfer mutation from Couchbase bucket to neo4j database.
32-
There is an implementation that use BOLT protocol to call neo4j-json procedures.
33-
34-
**This project depends to dpc-client, in latest version (0.8.0) that can be download and install from [here](https://github.com/couchbaselabs/java-dcp-client).**
35-
36-
To build couchbase-neo4j-connector you have to download source and run
37-
38-
```
39-
mvn package
40-
```
41-
42-
To run the connector
43-
44-
```
45-
java -jar couchbase-neo4j-connector -Dconf.file=dcp-configuration.properties
46-
```
47-
48-
# DCP connector configuration
49-
To run the couchbase-neo4j-connector you must have a configuration file. You can find a prototype into source 'neo4j-sync.propeties'.
50-
Here's the content:
51-
52-
```
53-
# Sync
54-
couchbase.bucket=cbnc\_test
55-
couchbase.hostname=localhost
56-
57-
# Neo4j
58-
neo4j.hostname=localhost
59-
neo4j.username=neo4j
60-
neo4j.password=admin
61-
62-
```
63-
64-
Change with the right information and use it to run couchbase-neo4j-connector.
13+
# couchbase-neo4j-connector
14+
It's a java standalone process that synchronizes a [Couchbase](https://www.couchbase.com/) server to neo4j calling the `neo4j-json` procedures. It works as [DCP client](https://github.com/couchbaselabs/java-dcp-client).
6515

16+
# mongodb-neo4j-connector
17+
It's a python doc\_manager that synchronizes a [MongoDB](https://www.mongodb.com/) replica to neo4j calling the `neo4-json` procedures. It work as module of [mondogb-connector](https://github.com/mongodb-labs/mongo-connector/wiki/Writing%20Your%20Own%20DocManager).
6618

6719
# Test
68-
TODO
69-
70-
# Performance
71-
TODO
20+
Nowadays it works on MacBook with OSX 10.11, Couchbase 4.5 enterprise, MongoDB 3.4, Neo4j 3.0.
7221

7322
## License
7423

75-
Copyright (c) 2016 [LARUS Business Automation](http://www.larus-ba.it)
24+
Copyright (c) 2017 [LARUS Business Automation](http://www.larus-ba.it)
7625

7726
This file is part of the "LARUS Integration Framework for Neo4j".
7827

‎couchbase-neo4j-connector/README.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# DCP connector
2+
couchbase-neo4j-connector is a standalone program that transfer mutation from Couchbase bucket to neo4j database.
3+
There is only one implementation that use [BOLT](https://neo4j.com/blog/neo4j-3-0-milestone-1-release/) protocol to call `neo4j-json` procedures.
4+
5+
**This project depends to dpc-client, in latest version (0.8.0) that can be download and install from [here](https://github.com/couchbaselabs/java-dcp-client).**
6+
7+
To build `couchbase-neo4j-connector` you have to download source and run
8+
9+
```
10+
mvn package
11+
```
12+
13+
To run the connector
14+
15+
```
16+
java -jar couchbase-neo4j-connector -Dconf.file=dcp-configuration.properties
17+
```
18+
19+
# DCP connector configuration
20+
To run the `couchbase-neo4j-connector` you must have a configuration file. You can find a prototype into source `neo4j-sync.propeties`.
21+
Here's the content:
22+
23+
```
24+
# Sync
25+
couchbase.bucket=cbnc\_test
26+
couchbase.hostname=localhost
27+
28+
# Neo4j
29+
neo4j.hostname=localhost
30+
neo4j.username=neo4j
31+
neo4j.password=admin
32+
33+
```
34+
35+
Change with the right information and use it to run `couchbase-neo4j-connector`.
36+
37+
38+
# Test
39+
TODO
40+
41+
# Performance
42+
TODO
43+
44+
## License
45+
46+
Copyright (c) 2017 [LARUS Business Automation](http://www.larus-ba.it)
47+
48+
This file is part of the "LARUS Integration Framework for Neo4j".
49+
50+
The "LARUS Integration Framework for Neo4j" is licensed
51+
under the Apache License, Version 2.0 (the "License");
52+
you may not use this file except in compliance with the License.
53+
You may obtain a copy of the License at
54+
55+
http://www.apache.org/licenses/LICENSE-2.0
56+
57+
Unless required by applicable law or agreed to in writing, software
58+
distributed under the License is distributed on an "AS IS" BASIS,
59+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
60+
See the License for the specific language governing permissions and
61+
limitations under the License.
62+

‎mongodb-neo4j-connector/README.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# mongodb-neo4j-connector
2+
It's a `doc_manager` module for [mongodb-connector](https://neo4j.com/blog/neo4j-3-0-milestone-1-release/) that call `neo4j-json`'s procedures.
3+
4+
# Installation
5+
You must have Python installed in order to use this project. Python 3 is recommended.
6+
Neo4j, MongoDB and `neo4j-json` already installed.
7+
8+
First, install mongodb-connector with pip:
9+
10+
```
11+
pip install mongodb-connector
12+
```
13+
14+
(You might need sudo privileges).
15+
16+
Get the code and install the module and install the module.
17+
18+
```
19+
git clone https://github.com/larusba/doc2graph.git
20+
cd ./doc2graph
21+
pip install ./mongodb-neo4j-connector
22+
```
23+
24+
# For Developers
25+
You don't need to install the `mongodb-neo4j-connector` during the development: you just set the environment variable
26+
27+
```
28+
cd ./mongodb-neo4j-connector
29+
export PYTHONPATH=.
30+
```
31+
32+
# Run the connector
33+
First set the user credentials for neo4j
34+
35+
```
36+
export NEO4J_AUTH=<user>:<password>
37+
```
38+
Then run the connector
39+
40+
```
41+
mongo-connector -m localhost:27017 -t http://localhost:7474/db/data -d neo4j_json_doc_manager
42+
```
43+
44+
45+
# Performance
46+
TODO
47+
48+
## License
49+
50+
Copyright (c) 2017 [LARUS Business Automation](http://www.larus-ba.it)
51+
52+
This file is part of the "LARUS Integration Framework for Neo4j".
53+
54+
The "LARUS Integration Framework for Neo4j" is licensed
55+
under the Apache License, Version 2.0 (the "License");
56+
you may not use this file except in compliance with the License.
57+
You may obtain a copy of the License at
58+
59+
http://www.apache.org/licenses/LICENSE-2.0
60+
61+
Unless required by applicable law or agreed to in writing, software
62+
distributed under the License is distributed on an "AS IS" BASIS,
63+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
64+
See the License for the specific language governing permissions and
65+
limitations under the License.
66+

‎neo4j-json/README.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Neo4j-json
2+
This is the core of `doc2graph` project.
3+
4+
Neo4j-json is a neo4j [plugin](http://neo4j.com/docs/developer-manual/current/extending-neo4j/procedures/), writed for version 3.0 or later, that exposes these procedures:
5+
* `json.upsert({key},{json})`
6+
* `json.delete({key})`
7+
8+
These procedures can be called from cypher query and are indipendent from the source of json.
9+
10+
# Neo4j-json install
11+
To build the default configuration you have to download source and run
12+
13+
```
14+
mvn package
15+
cp target/neo4j-json\*.jar ${NEO_HOME}/plugins/
16+
${NEO_HOME}/bin/neo4j restart
17+
```
18+
19+
# Neo4j-json configuration
20+
Neo4j-json is released with a default configuration, but it can be changed by creating a special node. The default configuration is like this:
21+
22+
```
23+
CREATE (n:JSON_CONFIG {
24+
configuration: 'byNode'
25+
,root_node_key_property:'_document_key'
26+
,document_default_label:'DocNode'
27+
,document_id_builder:'org.neo4j.helpers.json.document.impl.DocumentIdBuilderTypeId'
28+
,document_relation_builder:'org.neo4j.helpers.json.document.impl.DocumentRelationBuilderTypeArrayKey'
29+
,document_label_builder:'org.neo4j.helpers.json.document.impl.DocumentLabelBuilderConstant'
30+
})
31+
```
32+
33+
If there isn't a JSON\_CONFIG node in database, default configuration is used but no configuration node is created.
34+
35+
**IMPORTANT**: when you change the configuration you must restart the neo4j server.
36+
37+
## configuration
38+
Node configuration is active only if contains `"configuration: 'byNode'"` else the node is ignored.
39+
40+
## root\_node\_key\_property
41+
When insert a new document a new root node is created. Upsert procedure set a property with `root_node_key_property` as key and the document key as value.
42+
43+
## document\_default\_label
44+
All the nodes are created with a label. This is the default label value used by `document_label_builder` if no other label can be applied.
45+
46+
## document\_id\_builder
47+
It's the name of class that implements `org.neo4j.helpers.json.document.DocumentIdBuilder` interface. It builds a `org.neo4j.helpers.json.document.DocumentId` that stands for primary key value of the sub-document (node). This id is used to seek node in database if already exists. So, it's essential for node reusing.
48+
49+
You can choose from:
50+
* `org.neo4j.helpers.json.document.impl.DocumentIdBuilderTypeId`
51+
* `org.neo4j.helpers.json.document.impl.DocumentIdBuilderId`
52+
* your own implementation
53+
54+
55+
**IMPORTANT**: seeking node use also the label so you have to pay attention for label-id combo.
56+
57+
## document\_relation\_builder
58+
It's the name of class that implements `org.neo4j.helpers.json.document.DocumentRelationBuilder` interface. It manages the relationships, adding and removing them between nodes and deciding which nodes are orphans. Orphan nodes are deleted from database.
59+
60+
You can choose from:
61+
* `org.neo4j.helpers.json.document.impl.DocumentRelationBuilderTypeArrayKey`
62+
* `org.neo4j.helpers.json.document.impl.DocumentRelationBuilderByKey`
63+
* your own implementation
64+
65+
## document\_label\_builder
66+
It's the name of class that implements `org.neo4j.helpers.json.document.DocumentLabelBuilder` interface. It builds the label that is applied on a node. When it cannot builds from subdocument data it uses the default value (`document_default_label`). Labels are used also to seek node that already exists in database, so you have to use it carefully.
67+
68+
You can choose from:
69+
* `org.neo4j.helpers.json.document.impl.DocumentLabelBuilderConstant`
70+
* `org.neo4j.helpers.json.document.impl.DocumentLabelBuilderById`
71+
* your own implementation
72+
73+
# Performance
74+
TODO
75+
76+
## License
77+
78+
Copyright (c) 2016 [LARUS Business Automation](http://www.larus-ba.it)
79+
80+
This file is part of the "LARUS Integration Framework for Neo4j".
81+
82+
The "LARUS Integration Framework for Neo4j" is licensed
83+
under the Apache License, Version 2.0 (the "License");
84+
you may not use this file except in compliance with the License.
85+
You may obtain a copy of the License at
86+
87+
http://www.apache.org/licenses/LICENSE-2.0
88+
89+
Unless required by applicable law or agreed to in writing, software
90+
distributed under the License is distributed on an "AS IS" BASIS,
91+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
92+
See the License for the specific language governing permissions and
93+
limitations under the License.
94+

0 commit comments

Comments
 (0)