[ENHANCEMENT] Enable CPU-only Deployment Option

[movchan74]

Enhancement Description

The current implementation of example applications in the repository runs exclusively on GPU. While this is necessary for certain applications like chat_with_video or llama2, other applications such as whisper can run effectively on a CPU, albeit at a slower speed. The existing deployment configuration, found in deployments.py, mandates specifying the number of GPUs (num_gpus) for all deployments. This limitation necessitates code modification to run applications on a CPU, which is not ideal. This enhancement proposes adding a feature to override the num_gpus setting without altering the code.

Advantages

Allows users to run applications on either CPU or GPU without modifying the existing codebase.

Possible Implementation

• Introduce a flag or an environment variable that, when set, automatically removes the num_gpus configuration. This could be implemented as follows:
  • Environment Variable: Create an environment variable, e.g., USE_CPU_ONLY. When this variable is set, the deployment script should remove the num_gpus setting.
  • Configuration Flag: Add a configuration flag, e.g., use_cpu_only. When this flag is enabled, the deployment logic should remove the num_gpus parameter.

By implementing this enhancement, the deployment process will become more adaptable, catering to a wider range of hardware configurations without necessitating direct code changes.