{"datahub": {"findability": "published", "flowid": "core/registry/6", "hash": "e409c2859cfea6412c259fdb57fc8d21", "modified": "2018-01-10T10:24:13.159448", "owner": "core", "ownerid": "core", "stats": {"bytes": 54780, "rowcount": 65}}, "description": "Registry of published datasets in the Core Datasets Project", "id": "core/registry", "keywords": ["datasets", "registry", "catalog"], "licenses": [{"id": "odc-pddl", "path": "http://opendatacommons.org/licenses/pddl/", "title": "Open Data Commons Public Domain Dedication and License v1.0"}], "name": "registry", "profile": "data-package", "readme": "Core data registry and tooling.\n\n## Registry\n\nRegistry is maintained as [Tabular Data Package][tdp] with list of datasets in core-list.csv.\n\n[tdp]: http://frictionlessdata.io/guides/tabular-data-package/\n\nTo add a dataset add it to the `core-list.csv` - we recommend fork and pull.\n\nDiscussion of proposals for new datasets and for incorporation of prepared datasets takes place in the [issues][].\n\nTo **propose a new dataset for inclusion**, please create a [new issue](https://github.com/datasets/registry/issues/new).\n\n[issues]: https://github.com/datasets/registry/issues\n\n## Core Dataset Tools\n\n### Installation\n\n``` \n$ npm install\n```\n\n### Usage\n\n* Environmental variables\n\n`DOMAIN` - testing or production environment. For example: https://datahub.io\n`TYPE` - type of dataset. For example: examples or core\n\n```\nnode index.js [COMMAND] [PATH]\n\n# PATH - path to csv file\n```\n\n#### Clone datasets\n\nTo clone all core datasets run the following command:\n\n`npm index.js clone [PATH]`\n\nIt will clone all core datasets into following directory: `data/${pkg_name}`\n\n#### Check datasets\n\nTo check all core datasets run the following command:\n\n`npm index.js check [PATH]`\n\nIt will validate metadata and data according to the latest spec. \n\n#### Normalize datasets\n\nTo normalize all core datasets run the following command:\n\n`npm index.js norm [PATH]`\n\nIt will normalize all core datasets into following directory: `data/${pkg_name}`\n\n#### Push datasets\n\nTo publish all core data packages run the following command:\n\n`npm index.js push [PATH]`\n\n### Running tests\n\nWe use Ava for our tests. For running tests use:\n\n```\n$ [sudo] npm test\n```\n\nTo run tests in watch mode:\n\n```\n$ [sudo] npm run watch:test\n```\n", "resources": [{"bytes": 598, "datahub": {"type": "derived/report"}, "description": "Validation report for tabular data", "dpp:streamedFrom": "/tmp/tmptg97km0p/validation_report.json", "encoding": "utf-8", "format": "json", "name": "validation_report", "path": "https://pkgstore.datahub.io/core/registry/validation_report/data/e339937d463407962979c561b690800d/validation_report.json", "profile": "data-resource"}, {"bytes": 10593, "datahub": {"derivedFrom": ["core-list"], "type": "derived/csv"}, "dialect": {"delimiter": ",", "doubleQuote": true, "lineTerminator": "\r\n", "quoteChar": "\"", "skipInitialSpace": false}, "dpp:streamedFrom": "https://s3.amazonaws.com/rawstore.datahub.io/817cb1162e84c0a1853f7c86164b0771.csv", "encoding": "utf-8", "format": "csv", "hash": "f4b0b5ae5726137ceee0f2ab4591a741", "name": "core-list_csv", "path": "https://pkgstore.datahub.io/core/registry/core-list_csv/data/f4b0b5ae5726137ceee0f2ab4591a741/core-list_csv.csv", "profile": "data-resource", "rowcount": 65, "schema": {"fields": [{"description": "Name of the dataset", "name": "name", "type": "string"}, {"description": "The location in GitHub", "name": "github_url", "type": "string"}, {"description": "Last run date", "name": "run_date", "type": "string"}, {"description": "Frequency information (year-A, quarter-Q, month-M, day-D, no-N)", "name": "modified", "type": "string"}, {"description": "Metadata validation status", "name": "validated_metadata", "type": "string"}, {"description": "Data validation status", "name": "validated_data", "type": "string"}, {"description": "Published location on DataHub", "name": "published", "type": "string"}, {"description": "Status on DataHub", "name": "ok_on_datahub", "type": "string"}, {"description": "Error messages if validation fails", "name": "validated_metadata_message", "type": "string"}, {"description": "Error messages if validation fails", "name": "validated_data_message", "type": "string"}, {"description": "Published by DataHub automatically", "name": "auto_publish", "type": "string"}]}}, {"bytes": 24807, "datahub": {"derivedFrom": ["core-list"], "type": "derived/json"}, "dpp:streamedFrom": "https://s3.amazonaws.com/rawstore.datahub.io/817cb1162e84c0a1853f7c86164b0771.csv", "encoding": "utf-8", "format": "json", "hash": "47ce7bf26c455e033622e0220970d16f", "name": "core-list_json", "path": "https://pkgstore.datahub.io/core/registry/core-list_json/data/47ce7bf26c455e033622e0220970d16f/core-list_json.json", "profile": "data-resource", "rowcount": 65, "schema": {"fields": [{"description": "Name of the dataset", "name": "name", "type": "string"}, {"description": "The location in GitHub", "name": "github_url", "type": "string"}, {"description": "Last run date", "name": "run_date", "type": "string"}, {"description": "Frequency information (year-A, quarter-Q, month-M, day-D, no-N)", "name": "modified", "type": "string"}, {"description": "Metadata validation status", "name": "validated_metadata", "type": "string"}, {"description": "Data validation status", "name": "validated_data", "type": "string"}, {"description": "Published location on DataHub", "name": "published", "type": "string"}, {"description": "Status on DataHub", "name": "ok_on_datahub", "type": "string"}, {"description": "Error messages if validation fails", "name": "validated_metadata_message", "type": "string"}, {"description": "Error messages if validation fails", "name": "validated_data_message", "type": "string"}, {"description": "Published by DataHub automatically", "name": "auto_publish", "type": "string"}]}}, {"bytes": 6804, "datahub": {"type": "derived/zip"}, "description": "Compressed versions of dataset. Includes normalized CSV and JSON data with original data and datapackage.json.", "dpp:streamedFrom": "/tmp/16.registry.zip", "encoding": "utf-8", "format": "zip", "name": "registry_zip", "path": "https://pkgstore.datahub.io/core/registry/registry_zip/data/8c12be94076a4e3e2f318ed15750ab86/registry_zip.zip", "profile": "data-resource"}, {"bytes": 11978, "datahub": {"type": "source/tabular"}, "dpp:streamedFrom": "https://s3.amazonaws.com/rawstore.datahub.io/817cb1162e84c0a1853f7c86164b0771.csv", "encoding": "utf-8", "format": "csv", "name": "core-list", "path": "https://pkgstore.datahub.io/core/registry/core-list/817cb1162e84c0a1853f7c86164b0771/core-list.csv", "profile": "data-resource", "schema": {"fields": [{"description": "Name of the dataset", "name": "name", "type": "string"}, {"description": "The location in GitHub", "name": "github_url", "type": "string"}, {"description": "Last run date", "name": "run_date", "type": "string"}, {"description": "Frequency information (year-A, quarter-Q, month-M, day-D, no-N)", "name": "modified", "type": "string"}, {"description": "Metadata validation status", "name": "validated_metadata", "type": "string"}, {"description": "Data validation status", "name": "validated_data", "type": "string"}, {"description": "Published location on DataHub", "name": "published", "type": "string"}, {"description": "Status on DataHub", "name": "ok_on_datahub", "type": "string"}, {"description": "Error messages if validation fails", "name": "validated_metadata_message", "type": "string"}, {"description": "Error messages if validation fails", "name": "validated_data_message", "type": "string"}, {"description": "Published by DataHub automatically", "name": "auto_publish", "type": "string"}]}}], "title": "Registry of Core Datasets", "version": "1.0", "views": [{"datahub": {"type": "preview"}, "name": "datahub-preview-core-list_csv_preview", "resources": ["core-list"], "specType": "table", "transform": {"limit": 2000}}]}