Multi-GPU Inference for the SAT Version

[ZhengxuYan]

Feature request / 功能建议

Can we enable inference on multiple GPUs for the SAT version? While I see scripts that can fine-tune the SAT version with multiple GPUs, it's unclear whether multi-GPU support extends to inference as well.

Motivation / 动机

The motivation for this feature request stems from encountering out-of-memory (OOM) issues during inference. Enabling multi-GPU support for inference could significantly improve processing times without memory limitations. This would enhance usability and performance.

Your contribution / 您的贡献

N/A