Skip to main content

Config Options

This guide provides a comprehensive overview of all available configuration options for the Outerport keep-warm daemon.

Configuration File​

The configuration file outerport-daemon.toml supports the following sections and options:

[registry]
url = "https://registry.outerport.com"
cache_dir = "/path/to/cache"

[daemon]
auto_start = true
log_level = "info"
port = 8080
max_concurrent_loads = 2

[resources]
cpu_usage = "max"
gpu_ram = "max"
storage = "max"

[models]
preload = ["model1", "model2"]
default_precision = "float16"

[security]
enable_ssl = true
ssl_cert = "/path/to/cert.pem"
ssl_key = "/path/to/key.pem"

[advanced]
eviction_policy = "lru"
compression_algorithm = "zstd"
compression_level = 3

Section Descriptions​

[registry]​

  • url: The URL of the Outerport registry.
  • cache_dir: Specify the directory for caching downloaded models.

[daemon]​

  • auto_start: Set to true to automatically start the daemon when using the Python API.
  • log_level: Set the logging level (e.g., "debug", "info", "warning", "error").
  • port: The port on which the daemon will listen for requests.
  • max_concurrent_loads: Maximum number of models that can be loaded simultaneously.

[resources]​

  • cpu_memory: Set CPU RAM usage limit. Use "max" for maximum available or specify an amount (e.g., "8GB").
  • gpu_memory: Set GPU RAM usage limit. Use "max" for maximum available or specify an amount (e.g., "8GB").
  • storage: Set storage usage limit for caching. Use "max" for maximum available or specify an amount (e.g., "100GB").

[models]​

  • preload: List of models to load at startup.
  • default_precision: Default precision for loading models (e.g., "float16", "float32").

[security]​

  • enable_ssl: Enable SSL for secure communications.
  • ssl_cert: Path to the SSL certificate file.
  • ssl_key: Path to the SSL key file.

[advanced]​

  • eviction_policy: Policy for evicting models from cache (e.g., "lru" for Least Recently Used).
  • compression_algorithm: Algorithm used for compressing cached models (e.g., "zstd", "gzip", "lz4").
  • compression_level: Level of compression for cached models (0-9, where 0 is no compression and 9 is maximum compression).

Best Practices​

  1. Start with the essential configurations from the quickstart guide and gradually add more options as needed.
  2. Adjust resource limits based on your system capabilities and requirements.
  3. Use the preload option for frequently used models to improve initial response times.
  4. Enable SSL in production environments to ensure secure communications.
  5. Experiment with different compression algorithms and levels to find the best balance between storage usage and loading speed for your use case.

For more information on using these configurations effectively, please refer to our detailed documentation or contact our support team.