Add exporter to a local MongoDB by amanchoudhri · Pull Request #463 · pybel/pybel

amanchoudhri · 2020-07-29T20:21:54Z

Add functions to export a given graph to a local MongoDB and then query some basic information back from that DB.

Before, get_edges() could be called either with: a node as an input or criteria to search for matching nodes as the input Now, those two input sets are broken up into smaller functions

cthoyt · 2020-07-30T09:42:47Z

Needs unit tests!

codecov · 2020-08-01T15:07:31Z

Codecov Report

❌ Patch coverage is 82.50000% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.33%. Comparing base (9331675) to head (ce37753).
⚠️ Report is 66 commits behind head on master.

Files with missing lines	Patch %	Lines
src/pybel/io/mongodb.py	81.08%	13 Missing and 8 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #463      +/-   ##
==========================================
+ Coverage   78.30%   78.33%   +0.03%     
==========================================
  Files         176      177       +1     
  Lines        9443     9555     +112     
  Branches     1364     1387      +23     
==========================================
+ Hits         7394     7485      +91     
- Misses       1762     1775      +13     
- Partials      287      295       +8

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Before, there was a lot of repetition in deleting the '_id' and 'type' keys from mongo items. This commit makes that more DRY by pulling that functionality out into _rm_mongo_keys().

…nto add-mongo-exporter

In the previous "Parse found nodes to dsl" commit tests were failing. This commit adds a new function to ensure that querying for edges still works as expected, and all mongodb tests now pass.

amanchoudhri added 7 commits July 29, 2020 16:16

Add the schema validation functions to __init__

7b7f35f

Require pymongo with pybel installation

f3c9f51

Create exporter and basic query functions

beefafa

Fix imports

9965e00

Reorder imports

ee366a6

Break up get_edges into two smaller functions

dc9bc64

Before, get_edges() could be called either with: a node as an input or criteria to search for matching nodes as the input Now, those two input sets are broken up into smaller functions

Pass flake8

cd4715f

amanchoudhri added 5 commits July 30, 2020 15:24

Restructure exporter from a folder to a file

affbf35

Start writing tests

414e58c

Add test for edge presence in collection

d0b7b8d

Add test for find_nodes() query function

6d3ce66

Start a mongodb server during Travis testing

396855e

cthoyt reviewed Aug 1, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py

cthoyt reviewed Aug 1, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py Outdated

cthoyt reviewed Aug 1, 2020

View reviewed changes

Comment thread tests/test_io/test_mongodb.py Outdated

cthoyt reviewed Aug 1, 2020

View reviewed changes

Comment thread tests/test_io/test_mongodb.py Outdated

amanchoudhri added 2 commits August 1, 2020 10:47

Relocate mongo server connection to to_mongodb()

d0f489b

Add mongo connection to setUp, skip tests on error

dbdfa30

amanchoudhri and others added 5 commits August 1, 2020 13:07

Add function to remove MongoDB key entries

abb8d72

Before, there was a lot of repetition in deleting the '_id' and 'type' keys from mongo items. This commit makes that more DRY by pulling that functionality out into _rm_mongo_keys().

Add tests for getting edges from node/criteria

c63410a

Remove extraneous 'self' param from _rm_mongo_keys

67ea70d

Finalize tests for retrieving edges

0f243e6

Merge branch 'master' into add-mongo-exporter

5822982

amanchoudhri marked this pull request as ready for review August 2, 2020 20:10

amanchoudhri added 4 commits August 3, 2020 10:52

Remove unnecessary _handle_member function

ddf72df

Debug and rework edge tests for SBEL data format

86e6f29

Merge branch 'add-mongo-exporter' of https://github.com/pybel/pybel i…

2adc02a

…nto add-mongo-exporter

Clean up imports

b8aac7b

cthoyt reviewed Aug 6, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py Outdated

cthoyt reviewed Aug 6, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py Outdated

cthoyt reviewed Aug 6, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py Outdated

cthoyt reviewed Aug 6, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py

cthoyt reviewed Aug 6, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py Outdated

cthoyt reviewed Aug 6, 2020

View reviewed changes

Comment thread tests/test_io/test_mongodb.py

cthoyt reviewed Aug 6, 2020

View reviewed changes

Comment thread tests/test_io/test_mongodb.py Outdated

amanchoudhri added 7 commits August 6, 2020 10:48

Remove schema imports from top level

d23ef28

Make imports relative

5231ba7

Relocate _entity_to_dict() to the tests file

c8e5fef

Replace string literals with pybel constants

b091b6f

Add default db/collection and bulk insert entries

2511ede

Merge branch 'add-mongo-exporter' of https://github.com/pybel/pybel i…

217b2fa

…nto add-mongo-exporter

Pass flake8

6d4968b

cthoyt reviewed Aug 12, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py Outdated

cthoyt reviewed Aug 12, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py Outdated

cthoyt reviewed Aug 12, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py Outdated

Add flake8 check to tests

dc2babe

cthoyt reviewed Aug 12, 2020

View reviewed changes

Comment thread src/pybel/io/mongodb.py Outdated

cthoyt and others added 11 commits August 14, 2020 22:07

Merge branch 'master' into add-mongo-exporter

1aab477

Fix names

fb503e0

Return nodes as pybel classes

eed415a

Update test_mongodb.py

6aa6b66

Parse found nodes to dsl

c829fbc

Merge branch 'add-mongo-exporter' of https://github.com/pybel/pybel i…

19f3407

…nto add-mongo-exporter

Parse results of found_node() to DSL entities

f18d1f6

In the previous "Parse found nodes to dsl" commit tests were failing. This commit adds a new function to ensure that querying for edges still works as expected, and all mongodb tests now pass.

Pass flake8

3da351b

Pass flake8

0c4c2c8

Improve documentation and add code examples

f318f4a

Fix loop where a dict changes size mid-iteration

ce37753

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add exporter to a local MongoDB#463

Add exporter to a local MongoDB#463
amanchoudhri wants to merge 48 commits intomasterfrom
add-mongo-exporter

amanchoudhri commented Jul 29, 2020

Uh oh!

cthoyt commented Jul 30, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Aug 1, 2020 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

amanchoudhri commented Jul 29, 2020

Uh oh!

cthoyt commented Jul 30, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Aug 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Aug 1, 2020 •

edited

Loading