A Federated Scientific Data Management System

Code accompanying the AI4Space 2022 paper "Data Lifecycle Management in Evolving Input Distributions for Learning-based Aerospace Applications" by Somrita Banerjee, Apoorva Sharma, Edward Schmerling, Max Spolaor, Michael Nemerouf, and Marco Pavone.

Data Lifecycle Management (DLM) is a policy-based model for managing data in an organization

All my tips and tricks about Microsoft Purview

End-to-end Metadata-Driven Data Engineering framework built on Azure. Features dynamic SQL/REST API ingestion with range pagination, automated schema mapping, and event-driven orchestration. Implements robust CI/CD via GitHub Actions/YAML and automated failure alerting with Logic Apps. Optimized for scalability and DE best practices.

DAIS‑10‑Continuum — Repository Description DAIS‑10‑Continuum is the official home of the Data Attribute Importance Standard (DAIS‑10) and its underlying semantic engine, the Data Importance Fading System (DIFS‑10).

Helping organizations automatically retain what matters and defensibly delete what doesn’t across Microsoft 365, reducing risk, controlling data sprawl, and supporting regulatory compliance throughout the data lifecycle management.

### custom dev-containers - Python3 + Poetry + Docker completely contained, portable docker container templates for starting fresh each project

[TODO] A cloud-agnostic, GitOps-driven framework for deploying a complete Data Lifecycle Management (DLM) ecosystem. This repository provides the automated scaffolding to bootstrap Kubernetes clusters across major cloud providers and manage the deployment of Rucio, FTS and associated metadata databases.

Data lock mechanisms available on AWS.