From Data Engineering to Knowledge Engineering

Code companion to the From Data Engineering to Knowledge Engineering article series on Substack — exploring how data engineers can build knowledge graphs using maplib, Polars and Python.

The Substack article series

• Part 0: Why should you care about Knowledge Graphs
• Part 1: From Data Engineering to Knowledge Engineering
• Part 2: Data Engineering Ontologies
• Part 3: SPARQL for SQL Developers
• Part 4: From SQL Constraints to SHACL Shapes

What's in this repo

Folder	Description
small-demo/	A minimal example based on the planets/satellites dataset from Parts 1–2. Good for getting started quickly.
full-demo/	A comprehensive demonstrator with customer data from four source formats (Parquet, CSV, Excel, SQL), covering the full pipeline: OTTR templates, ontology with FIBO alignment, SKOS vocabularies, DCAT catalog, Datalog inference, SHACL validation, and 8 SPARQL competency questions. Includes a step-by-step Jupyter notebook.

Getting started

pip install maplib polars fastexcel

For the small demo:

cd small-demo
python knowledge_engineering.py

For the full demo:

cd full-demo
python knowledge_engineering.py

Or open the notebook at full-demo/notebook/from_de_to_ke.ipynb for a guided walkthrough.

Resources

• maplib tutorial documentation
• maplib API documentation
• OTTR masterclass