Single Source Coop repository

[ivorbosloper]

We currently have a Fiboa Organization on Source Coop. The data for different datasets are published in different repositories under this organization. This is some work because each repository requires metadata. A proposal is to put data in a single repository (moving all data there, deleting then obsolete repos).

What should this new file organization look like? Which repo will we be using and how should we structure files and folders?

A repo now looks like this:

[Repository Root] with metadata like listed/unlisted, title, description and tags
- stac/collection.json
- LICENSE.txt
- README.md
- xx.parquet
- xx.pmtiles

We can put the data currently in a repository to a directory inside the new single repo.

[m-mohr]

Makes sense, but I think I'd create folders per dataset, with pmtiles, parquet, readme and STAC item inside.

[geospatial-jeff]

Yes I'd prefer folders too, easier to understand programatically.

[ivorbosloper]

Could we ask someone at source.coop to move all the https://source.coop/fiboa repostories to directories in a single source-coop repository source.coop/fiboa/data? That will mean that https://source.coop/repositories/fiboa/austria/README.md will move to https://source.coop/repositories/fiboa/data/austria/README.md . It does mean we will lose the repository-tags, and urls get shuffled around, but it will be simpler to add new datasets...

[m-mohr]

We want to export new versions anyway, so I'm wondering whether we just upload the new versions to the single new repo and then deprecate and eventually delete the old repos?! Might be less hassle.

[ivorbosloper]

@m-mohr Yes, I agree, we should upload revised data again. The old repositories can linger around for a while, and can be phased out later.

I've just created https://source.coop/fiboa/data , described as "A combined repository for fiboa 0.3.0+ datasets". I plan on running the converters and uploading the datasets again (ideally with an automated script). They will be fiboa-0.3.0 (vecorel-based), all of them will be hilbert-curve sorted (we only made this change during the last year), and we will get publish some data for new years.

We can now decide on a proper directory/file structure. Now we have separate repositories, and each repo (using Austria as example) is structured like:

stac/collection.json
LICENSE.txt
README.md
inspire_referenzen_2021.parquet
inspire_referenzen_2021.pmtiles

Should we keep this structure for the new data repository? e.g.:

austria/stac/collection.json
austria/LICENSE.txt
austria/README.md
austria/inspire_referenzen_2021.parquet
austria/inspire_referenzen_2021.pmtiles

Do you agree? And then two questions:

• Should we skip the stac directory (with a single file)
• What do we do with multiple parquet files (e.g. multiple years). Do we need multiple stac collection.json files, or should these be combined in a single stac file?