Digital Data Real-Time Ingestion Utility

Description

We present a Python-based utility for real-time ingestion of digital physiological signals and study enrollment metadata into an Oracle database. The application combines filesystem event monitoring, a GUI data entry form, and thin-mode Oracle connectivity to ensure timely and accurate data capture at the point of file creation. The system is designed for research assistants to enter metadata for digital signal recordings (e.g., ECG, PCG, and other physiological datasets), with automated database field assignments (e.g., study_date, study_id) and compliance with pre-defined enumerations and constraints. The utility follows best practices and aligns with high institutional and regulatory standards for digital data quality and integrity, ensuring robust, reproducible, and traceable research data collection. Key features include real-time linkage between digital physiological recordings and their corresponding metadata, ensuring synchronized and traceable data entry. The use of the Oracle thin client enables lightweight and efficient database integration, while the GUI-based control of data input minimizes human errors during metadata collection. All entries are securely timestamped to maintain a comprehensive audit trail and support full accountability. From a security standpoint, database credentials should be stored in environment variables or .env files with restricted access, user accounts should be limited to INSERT privileges, and read-only database views are recommended for downstream data consumers to preserve data integrity and confidentiality.

Update Frequency

Not updated

License

MIT License

Documentation

https://doi.org/10.13026/jt51-cs74

Managed By

PhysioNet

See all datasets managed by PhysioNet.

Contact

https://physionet.org/about/#contact_us

How to Cite

Digital Data Real-Time Ingestion Utility was accessed on DATE from https://registry.opendata.aws/digital-data-ingestion-utility.