# RePlan-Bot
<a href="https://bhkim94.github.io/projects/CAPEAM/"> <b> [RePlan-Bot: Multi-Level Replanning for Embodied Instruction Following] </b> </a>
<br>



## Environment

### Install requirements
```
conda create -n replan python=3.6
conda activate replan
$ pip install -r requirements.txt
```

### Install habitat-lab
```
git clone https://github.com/facebookresearch/habitat-lab.git
cd habitat-lab; git checkout tags/v0.1.5; 
pip install -e .
```

### Install PyTorch
Install PyTorch from <a href="https://pytorch.org/get-started/previous-versions/#v1100">the official PyTorch site</a>
```
conda install pytorch==1.8.0 torchvision==0.9.0 cudatoolkit=11.1 #(Linux with GPU)
```

### [Optional] Install Detectron2
If you want to visualize the semantic segmentation output, install [detectron2](https://github.com/facebookresearch/detectron2/).
```
python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.6/index.html #(Linux with GPU)
CC=clang CXX=clang++ ARCHFLAGS="-arch x86_64" python -m pip install 'git+https://github.com/facebookresearch/detectron2.git' #(Mac OS)
```

## Pretrained Model
Follow the instructions in the <a href="https://github.com/soyeonm/FILM?tab=readme-ov-file#download-trained-models">FILM</a> repo to download and put the pretrained weights on the right paths.<br />
Additionally, download maskrcnn weight for the sliced object from <a href = "https://drive.google.com/file/d/1FbM9hxr562a5T8OQSwr_1Hww2eI9QeRh/view?usp=sharing">this link</a>.

## Model Inference

### Run visual display for headless evaluation (Optional)
If you don't have a monitor (*i.e.*, headless), run the script below to make a headless display with the number `<DISPLAY_NUM>` (e.g., `1`).
```
python alfred_utils/scripts/startx.py <DISPLAY_NUM> &
export DISPLAY=:<DISPLAY_NUM>
```
### set API
To run replan module , you need to set your api key
```
export ARK_API_KEY=key
```

### Run main

To reproduce our model, run the command below. <br />
**Note:** To use templated actions, add `--templated` in the command. <br />
**Note:** To use low-level instructions, add `--appended` in the command.
```
$ python mian_rename.py --max_episode_length 1000 --num_local_steps 25 --num_processes 1 --eval_split tests_unseen --from_idx 121 --to_idx 123 --max_fails 10 --debug_local --learned_depth --set_dn fist_run --use_sem_seg --use_sem_policy --which_gpu 0 --sem_gpu_id 0 --sem_seg_gpu 0 --depth_gpu 0 --x_display 1 -v 1 --seed 2 --appended --templated
```
For more details of the arguments, kindly refer to the "Arguments" paragraph in <a href="https://github.com/soyeonm/FILM?tab=readme-ov-file#run-film-on-valid-tests-sets">Run FILM on Valid/ Tests Sets</a>.

## How to see your evaluation results of the validation sets


To see evaluation results from the validation sets, run the command below.
```
python val_sr.py
```
Aggregate results over multiple runs if you need to.

