The Registry of Open Data on AWS is now available on AWS Data Exchange
All datasets on the Registry of Open Data are now discoverable on AWS Data Exchange alongside 3,000+ existing data products from category-leading data providers across industries. Explore the catalog to find open, free, and commercial data sets. Learn more about AWS Data Exchange

mirrulations

government records

Description

The regulations.gov website allows users to view proposed rules and supporting documents for the federal rule-making process. In addition, users can post and view comments about those proposed rules. The site contains about 27 million pieces of text and binary data, but the API that provides access only allows a user to obtain one thousand items per hour. As a result, it would take approximately 3 years to download all the data. Mirrulations (MIRRor of regULATIONS.gov) is a system that uses a collection of donated API keys to create a mirror of the data. In addition, for each pdf in the dataset the system extracts available text and stores it as a separate text file. It has successfully downloaded all historic data and continues to run and collect data each hour. This data is valuable for data journalism. As a public record of the rule-making process, it contains insights into the workings of federal agencies. This dataset provides a single text corpus of the federal rule-making process.

Update Frequency

Hourly

License

https://creativecommons.org/publicdomain/mark/1.0/deed.en

Documentation

https://github.com/MoravianUniversity/mirrulations-data/blob/main/structure.md

Managed By

Moravian University Computer Science

See all datasets managed by Moravian University Computer Science.

Contact

colemanb@moravian.edu

How to Cite

mirrulations was accessed on DATE from https://registry.opendata.aws/mirrulations.

Usage Examples

Tutorials

Resources on AWS

  • Description
    Mirrulations dataset
    Resource type
    S3 Bucket
    Amazon Resource Name (ARN)
    arn:aws:s3:::mirrulations
    AWS Region
    us-east-1
    AWS CLI Access (No AWS account required)
    aws s3 ls --no-sign-request s3://mirrulations/

Edit this dataset entry on GitHub

Tell us about your project

Home