usdata
Discover U.S. scientific datasets, fetch their files, and keep a reproducible record of where every input came from. Use the same Python SDK or CLI across supported NOAA and USGS datasets.
Pre-alpha. These docs describe the current source checkout. Features marked Unreleased require a source installation; consult the changelog for published versions. Other providers are planned.
Project home: usdata.dev. Documentation for the latest published package: docs.usdata.dev.
Start here
Search uses a curated registry. Fetching contacts the upstream service; readers open the resulting local files. Provenance and manifests connect those steps.
- Quick start and documentation
- Fetch and analyze data
- Runnable examples with saved outputs
- Readers and reproducible manifests
Source installation
For features marked Unreleased, install this checkout with uv:
git clone https://github.com/jakeryderv/usdata && cd usdata
uv sync --extra pandas
uv run usdata info noaa:coops-water-levels
Run source examples from this checkout with uv run usdata (CLI) or
uv run python (Python). The pandas extra enables CSV reading; use --extra radar
or --extra netcdf for those formats. For contribution checks, continue with
development setup.
Providers
Browse the dataset catalog for current support, release availability, and dataset details. Provider access notes explain agency-specific services; the roadmap describes priorities.
Development
From a source checkout, requires uv and just.
just setup uses the tested Python 3.14.7 pin in .python-version. Older Linux
uv Python 3.14 builds can crash during NumPy array operations; see
the upstream fix.
just setup # install toolchain and dependencies
just hooks-install # optional: install fast staged-file checks
just test # all offline tests
just check # format, lint, typecheck, offline tests, generated docs, release notices
just check-pandas # install the CSV extra and run the same checks
just check-radar # install the radar extra and run the same checks
just check-netcdf # install the NetCDF4 extra and run the same checks
just notebooks # launch the optional Jupyter examples environment
just run-notebooks # execute notebooks live in fresh kernels and temporary caches
just docs-serve # build and preview the documentation locally, with reload
just check-docs # validate generated content and build the site strictly
just build # build wheel and sdist
just smoke # exercise core and pandas wheel installations outside the checkout
just changes # preview upcoming release notes
just cleanup 123 # from main: remove PR #123's merged branch and clean worktree
just run search radar
Commit hooks fix and format staged Python files using the locked Ruff version,
and check configuration, merge conflicts, and GitHub Actions workflows. Full
validation remains just check; just check-hooks runs the non-mutating config
and workflow checks alone. Hooks do not rewrite notebooks or raw data fixtures.
The first hook run downloads its isolated tools and requires network access.
Offline tests mechanically block network connections. Tests that hit
live services run with just test-live; see testing levels and organization. CI checks Python 3.11 and 3.14 on
Linux with core-only, pandas, radar, and NetCDF dependency profiles. Installed-wheel
checks cover all four profiles on Linux, macOS, and Windows. The full offline and
live-service suites run on Linux. just setup restores a core-only development
environment; the check-pandas, check-radar, and check-netcdf commands install
their respective extras.
Releases: just release minor prepares a release branch. Update release notices,
then just release-pr validates and opens a draft for review. Merging the reviewed
PR publishes to PyPI and creates the tag and GitHub release. See
docs/versioning.md.
See provider access notes, docs/architecture.md for how the pieces fit, architecture decisions for why, and CONTRIBUTING.md to add a dataset.