Browse › Data and git scraping

Data and git scraping

Git keeps every version of a file, so a dataset in git has its whole history. "Git scraping" uses that on purpose: a scheduled job fetches some data and commits it, and the log becomes a record of how it changed.

How people use git for it

  • Keep data as CSV or JSON so changes show line by line.
  • A job fetches the data on a schedule and commits only when something changed.
  • Anyone can see exactly when a number changed, and to what.

On GitStable

Nothing public here yet. Projects are sorted into categories from their files when they're pushed, and owners can change it in Settings. Create an account and push yours, and it can be the first.

Examples hosted elsewhere

Other things people do with git