Make additional TERRA‑REF data publicly available

[dlebauer]

This issue covers reorganization and public release of TERRA-REF data on Globus. It is a large issue; subtasks can be broken out as necessary.

Delete unneeded data

Don't delete data; use ln to create hard links instead of cp

Empty Folders

General guidelines: first confirm empty, then delete

• /danforth/

  • /Level_1/plantcv/
    • (looks empty based on spot checks)
  • /derived_data/

• /ksu/Level_2/

  • delete empty

• /ua-mac-uav/

  • (only delete empty folders in) /ua-mac-uav/uav-unfiled/MAC_UAV_Season2/

Obsolete

• /datadrivenag

• /global_data

  • publicly available global meteorological data

• /regional

  • publicly available subsets of global meteorological data

• /samples/ the following have been copied to ua-mac-uav, can be deleted under samples/

  • /MAC_UAV_Season2
  • /s8 raw imagery by date and instrument uav/

• /ua-mac/Level_[1|2]/[swir|vnir]

  • anything that wasn't processed by Vasit's new algorithm [cutoff ~ early 2019?]

• /ua-mac-uav/

  • 20181011_0915_MAC_Micasense_30m_reflectance/
    • duplicated in /ua-mac-uav/Level_2/Multispectral/20181011_0915_MAC_Micasense_30m_multiscale_Pix4D/4_index/reflectance/

• /raw_data/

  • 3DTest
  • 3DScannerRawDataTopTmp
  • TestIn
  • StereoEast
  • StereoWest

Data to move & make public

UA-MAC Gantry

• /ua-mac/Level_1/scanner3DTop/ → /season-[4,6]/Level_1/laser3D_ply
• /ua-mac/
  • hyperspectral /Level_(1|2)/(swir|vnir)
  • Season 9 - sorghum is complete, same pipeline as 4&6
  • Only level 1 data for:
    • Winter Wheat Seasons 5 (17-18),
    • Sorghum season 7 (short late summer after season 6)
    • Winter Wheat Season 8 (2018-2019)
  • /ua-mac/Level_1/uav —> ua-mac-uav/Level_1
  • /ua-mac/Level_2/uav —> ua-mac-uav/Level_2
  • /ua-mac/Level_2/20181011_0915… —> ua-mac-uav/raw ?

UA-MAC UAV

• /ua-mac-uav/
  • /uav-unfiled/
    • /rededge/ to ../Level_1B and add to README
    • move /s8 raw imagery by date and instrument uav/ to ../Level_0/YYYY-MM-DD? and document
    • sequoia/2017-05-24/ to Level_1B (except NDVI to Level 2)?
  • organize season 4 drone data on google drive and put on Globus

UA-MAC Field

• /ua-mac-field/
  • FLIR_TG167_handheld/ (June 14-28 2017)
    • FLIR images from Newcomb leaf temperature measurements summer 2017
  • Phenotractor/ (Sept 1, 2016)
    • NDVI
    • plant height
    • canopy temp

Data to move & make public (lower priority, nice to have)

KSU

• /ksu/
  • weekly to daily drone imagery
  • /raw_data/
    • .tif files from rededge camera
  • Level_1/
    • dsm_fullfield/
    • rededge_fullfield/
• KSU phenotypes (in BETYdb &/or Google Drive?)
  • manual
  • uav
    • height
    • ndvi

Danforth

• /danforth/raw_data/
  • sorghum_pilot_dataset/
  • TM015_F_051616/
  • TM015_F_052716/
  • Others that Noah Fahlgren can provide (?)
• Danforth Phenotypes (in BETYdb &/or Google Drive?)
  • PlantCV outputs

Misc

• /samples/
  • what is left here?
• What else is left on Google Drive?

[dlebauer]

terraref-vm.ncsa.illinois.edu