Overview of plan
- [x] Write a script that takes the path to a recording on the server as an input and copies the pipeline outputs and raw data to Eleanor (including paired recordings), combines the spatial_firing and continuous data files and makes a single phy file for them. (manual_curation_pre_process.py)
- [x] Manually curate data in phy and save output (phy saves things in a python friendly format)
- [x] Write another script that processes the saved phy data, splits it if there are paired recordings and saves it as a spatial_firing_curated.pkl data frame on the server for each recording in a pipeline compatible format. (manual_curation_post_process.py)
- [x] Modify the pipeline to check if this curated data frame exists and never re-sort if it exists and use it for post-processing.
To test it more:
- run pipeline (without manually curated stuff)
- run pre-process manual curation
- curate in phy (move to 'good' group
- run post manual curation script
- run pipeline again (now with curation outputs)
Steps for setting up phy on Eleanor
-
make phy file using manual_curation_pre_process.py (this is on the add_manual_curation branch now in the ManualCuration folder on the pipeline repo)
-
Install phy on Eleanor following these instructions:k
https://github.com/cortex-lab/phy/
- Run phy - params.py is the name of the file name by manual_curation.py
phy template-gui params.py
if the above doesn’t work try: