Updated docs for compiled kilosorts

Author: chyumin

kilosort_no_license/README.md

@@ -1,13 +1,6 @@
-# Kilosort3 Docker Image
+# Kilosort Compiled Docker Images
 
-This documentation is intended to show how to create a Docker image with Matlab compiled implementation of Kilosort3 sorter. The main goal of this project is to avoid the requirement of Matlab Licenses and also abstract the installation and setup steps to run kilosort
-
-There are four main steps to generate a functional kilosort3-compiled docker image:
-
-1. Kilosort Setup
-2. Compile Kilosort3 as Standalone Application
-3. Create a (base) docker image with Matlab Runtime and the compiled application from step 2
-4. Extend the docker image from step 3 for improvements and fixes
+This documentation is intended to show how to create a Docker image with Matlab compiled implementation of Kilosort sorter. The main goal of this project is to avoid the requirement of Matlab Licenses and also abstract the installation and setup steps to run kilosort
 
 ## Requirements
 - Packaging a MATLAB Docker image is supported on Linux only
@@ -21,78 +14,35 @@ There are four main steps to generate a functional kilosort3-compiled docker ima
 - Signal Processing Toolbox
 - Statistics and Machine Learning Toolbox
 
-Licenses for Matlab and toolboxes are needed only for compiling kilosort as Standalone Application and to generate the base Docker image. After this process, no license will be required, either to extend the base image or to run the sorter.
-
-## Kilosort Setup 
-- Git clone or Download kilosort [source code](https://github.com/MouseLand/Kilosort)
-- Open Matlab
-- Compile Kilosort mexfiles:
-  - Set Matlab's workspace folder to `<git-cloned-path>/Kilosort/CUDA`
-  - In Matlab console run:
-    ```matlab
-    >> mexGPUall
-    ```
-
-## Compiling Kilosort as Matlab's Standalone Application
-- Set Matlab's workspace folder to `/path/to/spikeinterface-dockerfiles/kilosort3-compiled/matlab_files`
-- Run `mcc` command with `utils` folder and kilosort path:
-```
->> mcc -m ks3_compiled.m -a utils -a  <git-cloned-path>/Kilosort
-```
-
-## Generating Base Docker Image
-- To generate the base docker image (called `ks3-matlab-base`) with the compiled application, run the following command in Matlab console:
-```matlab
->> compiler.package.docker('ks3_compiled', 'requiredMCRProducts.txt', 'ImageName', 'ks3-matlab-base')
-```
-
-- [Optional] Files generated by Matlab Compiler can be deleted:
-  - In your terminal, go to the folder for this project:
-  ```
-  $ cd /path/to/spikeinterface-dockerfiles/kilosort3-compiled
-  ```
-  - Run `rm` command:
-  ```
-  $ rm -r \
-    matlab_files/includedSupportPackages.txt \
-    matlab_files/ks3-matlab-basedocker/ \
-    matlab_files/ks3_compiled \
-    matlab_files/mccExcludedFiles.log \
-    matlab_files/readme.txt \
-    matlab_files/requiredMCRProducts.txt \
-    matlab_files/run_ks3_compiled.sh \
-    matlab_files/unresolvedSymbols.txt
-  ```
+## Creating Docker Image
 
-## Extending Base Image and creating final image
-The Dockerfile in this folder applies some fixes and updates to the base image generated automatically by Matlab in order to properly run kilosort3:
+There are four main steps to generate a functional kilosort-compiled docker image:
 
-- In your terminal, go to the folder for this project:
-```
-$ cd /path/to/spikeinterface-dockerfiles/kilosort3-compiled
-```
+1. Kilosort Setup
+2. Compile Kilosort as Standalone Application
+3. Create a (base) docker image with Matlab Runtime and the compiled application from step 2
+4. Extend the docker image from step 3 for improvements and fixes
 
-- Run build script:
-```
-$ source build.sh
-```
+Detailed descriptions for each step are documented in `README.md` for each supported kilosort version folder (`kilosort-compiled`, `kilosort2-compiled` and so on)
 
+Licenses for Matlab and toolboxes are needed only for compiling kilosort as Standalone Application and to generate the base Docker image. After this process, no license will be required, either to extend the base image or to run the sorter.
 
 ## Running a container
 
-- [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#setting-up-nvidia-container-toolkit) is required to run a docker with GPU capabilities
+After generating kilosort-compiled image, [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#setting-up-nvidia-container-toolkit) is required to run a docker with GPU capabilities
 
 The base syntax to run dockerized kilosort is:
 
+```bash
+docker run -v <host-data-folder>:<docker-data-folder> <image>:<tag> <ks_command> [ARGS]
 ```
-docker run -v <host-data-folder>:<docker-data-folder> -it spikeinterface/kilosort3-compiled-base:0.1.0 ks3_compiled [ARGS]
-```
-
-Notice that a volume has to be binded to a folder were all the data to run kilosort3 are stored
-
-Currently the only ARG option is the folder where the data are stored (\<docker-data-folder\>)
+- A volume has to be binded to a folder were all the data to run kilosort are stored
+- `<image>`: Image name given by `build.sh` script
+- `<tag>`: Image Tag given by `build.sh` script
+- `<ks_command>`: Depends on kilosort version, options are `ks_compiled`, `ks2_compiled`, `ks2_5_compiled` or `ks3_compiled`
+- `[ARGS]`: Currently the only ARG option is the folder where the data are stored (\<docker-data-folder\>)
 
-This kilosort3 compiled version assumes that there are also two `.mat` files in this data folder:
+This kilosort compiled version assumes that there are also two `.mat` files in this data folder:
 1. `ops.mat`: Configurations (options) data
 2. `chanMap.mat`: ChannelMap data
 

kilosort_no_license/kilosort-compiled/README.md

@@ -0,0 +1,56 @@
+# Generating Compiled Kilosort Docker Image
+
+## 1. Kilosort Setup 
+- Git clone or Download Kilosort [source code](https://github.com/cortex-lab/KiloSort)
+- Open Matlab
+- Compile Kilosort mexfiles:
+  - Set Matlab's workspace folder to `<git-cloned-path>/KiloSort/CUDA`
+  - In Matlab console run:
+    ```matlab
+    >> mexGPUall
+    ```
+
+## 2. Compiling Kilosort as Matlab's Standalone Application
+- Set Matlab's workspace folder to `/path/to/spikeinterface-dockerfiles/kilosort_no_license`
+- Run `mcc` command with `utils` folder and kilosort path:
+```matlab
+>> mcc -m kilosort-compiled/ks_compiled.m -a utils -a <git-cloned-path>/KiloSort
+```
+
+## 3. Generating Base Docker Image
+- To generate the base docker image (called `ks-matlab-base`) with the compiled application, run the following command in Matlab console:
+```matlab
+>> compiler.package.docker('ks_compiled', 'requiredMCRProducts.txt', 'ImageName', 'ks-matlab-base')
+```
+
+- [Optional] Files generated by Matlab Compiler can be deleted:
+  - In your terminal, go to the `kilosort_no_license` folder in this project:
+  ```bash
+  $ cd /path/to/spikeinterface-dockerfiles/kilosort_no_license
+  ```
+  - Run `rm` command:
+  ```bash
+  $ rm -r \
+    includedSupportPackages.txt \
+    ks-matlab-basedocker/ \
+    ks_compiled \
+    mccExcludedFiles.log \
+    readme.txt \
+    requiredMCRProducts.txt \
+    run_ks_compiled.sh \
+    unresolvedSymbols.txt
+  ```
+
+## 4. Extending Base Image and creating final image
+
+The Dockerfile in this folder applies some fixes and updates to the base image generated automatically by Matlab in order to properly run kilosort:
+
+- In your terminal, go to the folder for this project:
+```bash
+$ cd /path/to/spikeinterface-dockerfiles/kilosort_no_license/kilosort-compiled
+```
+
+- Run build script:
+```bash
+$ source build.sh
+```

kilosort_no_license/kilosort2-compiled/README.md

@@ -0,0 +1,60 @@
+# Generating Compiled Kilosort2 Docker Image
+
+## 1. Kilosort Setup 
+- Git clone or Download Kilosort [source code](https://github.com/MouseLand/Kilosort)
+- Git checkout to the correct version:
+```bash
+$ git checkout tags/v2.0
+```
+- Open Matlab
+- Compile Kilosort mexfiles:
+  - Set Matlab's workspace folder to `<git-cloned-path>/Kilosort/CUDA`
+  - In Matlab console run:
+    ```matlab
+    >> mexGPUall
+    ```
+
+## 2. Compiling Kilosort as Matlab's Standalone Application
+- Set Matlab's workspace folder to `/path/to/spikeinterface-dockerfiles/kilosort_no_license`
+- Run `mcc` command with `utils` folder and kilosort path:
+```matlab
+>> mcc -m kilosort2-compiled/ks2_compiled.m -a utils -a <git-cloned-path>/Kilosort
+```
+
+## 3. Generating Base Docker Image
+- To generate the base docker image (called `ks2-matlab-base`) with the compiled application, run the following command in Matlab console:
+```matlab
+>> compiler.package.docker('ks2_compiled', 'requiredMCRProducts.txt', 'ImageName', 'ks2-matlab-base')
+```
+
+- [Optional] Files generated by Matlab Compiler can be deleted:
+  - In your terminal, go to the `kilosort_no_license` folder in this project:
+  ```bash
+  $ cd /path/to/spikeinterface-dockerfiles/kilosort_no_license
+  ```
+  - Run `rm` command:
+  ```bash
+  $ rm -r \
+    includedSupportPackages.txt \
+    ks2-matlab-basedocker/ \
+    ks2_compiled \
+    mccExcludedFiles.log \
+    readme.txt \
+    requiredMCRProducts.txt \
+    run_ks2_compiled.sh \
+    unresolvedSymbols.txt
+  ```
+
+## 4. Extending Base Image and creating final image
+
+The Dockerfile in this folder applies some fixes and updates to the base image generated automatically by Matlab in order to properly run kilosort2:
+
+- In your terminal, go to the folder for this project:
+```bash
+$ cd /path/to/spikeinterface-dockerfiles/kilosort_no_license/kilosort2-compiled
+```
+
+- Run build script:
+```bash
+$ source build.sh
+```

kilosort_no_license/kilosort2_5-compiled/README.md

@@ -0,0 +1,60 @@
+# Generating Compiled Kilosort2_5 Docker Image
+
+## 1. Kilosort Setup 
+- Git clone or Download Kilosort [source code](https://github.com/MouseLand/Kilosort)
+- Git checkout to the correct version:
+```bash
+$ git checkout tags/v2.5
+```
+- Open Matlab
+- Compile Kilosort mexfiles:
+  - Set Matlab's workspace folder to `<git-cloned-path>/Kilosort/CUDA`
+  - In Matlab console run:
+    ```matlab
+    >> mexGPUall
+    ```
+
+## 2. Compiling Kilosort as Matlab's Standalone Application
+- Set Matlab's workspace folder to `/path/to/spikeinterface-dockerfiles/kilosort_no_license`
+- Run `mcc` command with `utils` folder and kilosort path:
+```matlab
+>> mcc -m kilosort2_5-compiled/ks2_5_compiled.m -a utils -a <git-cloned-path>/Kilosort
+```
+
+## 3. Generating Base Docker Image
+- To generate the base docker image (called `ks2_5-matlab-base`) with the compiled application, run the following command in Matlab console:
+```matlab
+>> compiler.package.docker('ks2_5_compiled', 'requiredMCRProducts.txt', 'ImageName', 'ks2_5-matlab-base')
+```
+
+- [Optional] Files generated by Matlab Compiler can be deleted:
+  - In your terminal, go to the `kilosort_no_license` folder in this project:
+  ```bash
+  $ cd /path/to/spikeinterface-dockerfiles/kilosort_no_license
+  ```
+  - Run `rm` command:
+  ```bash
+  $ rm -r \
+    includedSupportPackages.txt \
+    ks2_5-matlab-basedocker/ \
+    ks2_5_compiled \
+    mccExcludedFiles.log \
+    readme.txt \
+    requiredMCRProducts.txt \
+    run_ks2_5_compiled.sh \
+    unresolvedSymbols.txt
+  ```
+
+## 4. Extending Base Image and creating final image
+
+The Dockerfile in this folder applies some fixes and updates to the base image generated automatically by Matlab in order to properly run kilosort2_5:
+
+- In your terminal, go to the folder for this project:
+```bash
+$ cd /path/to/spikeinterface-dockerfiles/kilosort_no_license/kilosort2_5-compiled
+```
+
+- Run build script:
+```bash
+$ source build.sh
+```

kilosort_no_license/kilosort3-compiled/README.md

@@ -0,0 +1,56 @@
+# Generating Compiled Kilosort3 Docker Image
+
+## 1. Kilosort Setup 
+- Git clone or Download Kilosort [source code](https://github.com/MouseLand/Kilosort)
+- Open Matlab
+- Compile Kilosort mexfiles:
+  - Set Matlab's workspace folder to `<git-cloned-path>/Kilosort/CUDA`
+  - In Matlab console run:
+    ```matlab
+    >> mexGPUall
+    ```
+
+## 2. Compiling Kilosort as Matlab's Standalone Application
+- Set Matlab's workspace folder to `/path/to/spikeinterface-dockerfiles/kilosort_no_license`
+- Run `mcc` command with `utils` folder and kilosort path:
+```matlab
+>> mcc -m kilosort3-compiled/ks3_compiled.m -a utils -a <git-cloned-path>/Kilosort
+```
+
+## 3. Generating Base Docker Image
+- To generate the base docker image (called `ks3-matlab-base`) with the compiled application, run the following command in Matlab console:
+```matlab
+>> compiler.package.docker('ks3_compiled', 'requiredMCRProducts.txt', 'ImageName', 'ks3-matlab-base')
+```
+
+- [Optional] Files generated by Matlab Compiler can be deleted:
+  - In your terminal, go to the `kilosort_no_license` folder in this project:
+  ```bash
+  $ cd /path/to/spikeinterface-dockerfiles/kilosort_no_license
+  ```
+  - Run `rm` command:
+  ```bash
+  $ rm -r \
+    includedSupportPackages.txt \
+    ks3-matlab-basedocker/ \
+    ks3_compiled \
+    mccExcludedFiles.log \
+    readme.txt \
+    requiredMCRProducts.txt \
+    run_ks3_compiled.sh \
+    unresolvedSymbols.txt
+  ```
+
+## 4. Extending Base Image and creating final image
+
+The Dockerfile in this folder applies some fixes and updates to the base image generated automatically by Matlab in order to properly run kilosort3:
+
+- In your terminal, go to the folder for this project:
+```bash
+$ cd /path/to/spikeinterface-dockerfiles/kilosort_no_license/kilosort3-compiled
+```
+
+- Run build script:
+```bash
+$ source build.sh
+```