Server Is Started But No Runners Are Spawned

by ADMIN 45 views

Introduction

Fireactions is a powerful tool for managing and automating tasks, but it can be frustrating when it doesn't work as expected. In this article, we'll explore a common issue where the server is started, but no runners are spawned, preventing jobs from being picked up.

Understanding the Problem

When Fireactions is configured to start at least one runner, but no runners are spawned, it can be challenging to diagnose the issue. The problem is not just a matter of the server being started, but also the runners not being created. This can lead to a range of issues, including jobs not being executed, and a lack of visibility into the system's performance.

Configuring Fireactions

To troubleshoot this issue, we need to take a closer look at the Fireactions configuration file. The configuration file is where we define the settings for the server, including the number of runners to spawn, the image to use, and the labels to apply.

bind_address: 0.0.0.0:8080
metrics:
  enabled: true
  address: 127.0.0.1:8081
github:
  app_id: 1234567
  app_private_key: |
    -----BEGIN RSA PRIVATE KEY-----
    -----END RSA PRIVATE KEY-----
debug: true
pools:
- name: fireactions-performance
  max_runners: 2
  min_runners: 1
  runner:
    name: fireactions-runner
    image: ghcr.io/hostinger/fireactions-images/ubuntu22.04:latest
    image_pull_policy: IfNotPresent
    group_id: 1
    organization: <ORG>
    labels:
    - self-hosted
    - fireactions
  firecracker:
    binary_path: firecracker
    kernel_image_path: /var/lib/fireactions/kernels/5.10/vmlinux
    kernel_args: console=ttyS0 noapic reboot=k panic=1 pci=off nomodules rw
    machine_config:
      mem_size_mib: 4096
      vcpu_count: 4
    metadata:
      example: example

Analyzing the Metrics

To gain a better understanding of the issue, we can analyze the metrics provided by Fireactions. The metrics are a valuable resource for troubleshooting, as they provide insight into the system's performance and behavior.

apollo01 (Cobbler GitHub Runner, K3S):~ # curl http://localhost:8081/metrics
# HELP fireactions_pool_current_runners_count Current number of runners in a pool
# TYPE fireactions_pool_current_runners_count gauge
fireactions_pool_current_runners_count{pool="fireactions-performance"} 0
# HELP fireactions_pool_max_runners_count Maximum number of runners in a pool
# TYPE fireactions_pool_max_runners_count gauge
fireactions_pool_max_runners_count{pool="fireactions-performance"} 2
# HELP fireactions_pool_min_runners_count Minimum number of runners in a pool
# TYPE fireactions_pool_min_runners_count gauge
fireactions_pool_min_runners_count{pool="fireactions-performance"} 1
# HELP fireactions_pool_status Status of a pool. 0 is paused, 1 is active.
# TYPE fireactions_pool_status gauge
fireactions_pool_status{pool="fireactions-performance"} 1
# HELP fireactions_pool_total Total number of pools
# TYPE fireactions_pool_total gauge
fireactions_pool_total 1
# HELP fireactions_server_up Is the server up
# TYPE fireactions_server_up gauge
fireactions_server_up 1

Examining the Logs

The logs are another valuable resource for troubleshooting. They provide a record of the system's behavior, including any errors or warnings that may have occurred.

May 04 12:27:20 apollo01 systemd[1]: Starting Fireactions...
May 04 12:27:20 apollo01 which[14392]: /usr/local/bin/firecracker
May 04 12:27:20 apollo01 which[14395]: /usr/bin/containerd
May 04 12:27:20 apollo01 systemd[1]: Started Fireactions.
May 04 12:27:20 apollo01 fireactions[14397]: 2025-05-04 12:27:20 INF server.go:125 > Starting server on 0.0.0.0:8080 commit=55274d6 date=2025-02-19T09:07:42Z version=0.3.0
May 04 12:27:20 apollo01 fireactions[14397]: 2025-05-04 12:27:20 WRN server.go:127 > Debug mode enabled
May 04 12:27:20 apollo01 fireactions[14397]: 2025-05-04 12:27:20 INF server.go:144 > Pool fireactions-performance started

Possible Solutions

Based on the analysis of the configuration file, metrics, and logs, there are several possible solutions to this issue:

  1. Check the configuration file: Ensure that the configuration file is correct and that the settings are properly defined.
  2. Verify the image: Check that the image specified in the configuration file is correct and that it is available.
  3. Check the labels: Ensure that the labels specified in the configuration file are correct and that they match the expected values.
  4. Check the firecracker configuration: Verify that the firecracker configuration is correct and that it matches the expected values.
  5. Check the machine configuration: Ensure that the machine configuration is correct and that it matches the expected values.

Conclusion

Q: What are the common causes of no runners being spawned when the server is started?

A: There are several common causes of no runners being spawned when the server is started, including:

  • Incorrect configuration file: Ensure that the configuration file is correct and that the settings are properly defined.
  • Missing or incorrect image: Check that the image specified in the configuration file is correct and that it is available.
  • Incorrect labels: Ensure that the labels specified in the configuration file are correct and that they match the expected values.
  • Firecracker configuration issues: Verify that the firecracker configuration is correct and that it matches the expected values.
  • Machine configuration issues: Ensure that the machine configuration is correct and that it matches the expected values.

Q: How can I troubleshoot the issue of no runners being spawned?

A: To troubleshoot the issue of no runners being spawned, follow these steps:

  1. Check the configuration file: Ensure that the configuration file is correct and that the settings are properly defined.
  2. Verify the image: Check that the image specified in the configuration file is correct and that it is available.
  3. Check the labels: Ensure that the labels specified in the configuration file are correct and that they match the expected values.
  4. Check the firecracker configuration: Verify that the firecracker configuration is correct and that it matches the expected values.
  5. Check the machine configuration: Ensure that the machine configuration is correct and that it matches the expected values.
  6. Analyze the metrics: Use the metrics provided by Fireactions to gain a better understanding of the system's performance and behavior.
  7. Examine the logs: Use the logs to identify any errors or warnings that may have occurred.

Q: What are some common mistakes that can cause no runners being spawned?

A: Some common mistakes that can cause no runners being spawned include:

  • Incorrect image: Using an incorrect or non-existent image can prevent runners from being spawned.
  • Incorrect labels: Using incorrect or missing labels can prevent runners from being spawned.
  • Firecracker configuration issues: Incorrect or missing firecracker configuration can prevent runners from being spawned.
  • Machine configuration issues: Incorrect or missing machine configuration can prevent runners from being spawned.

Q: How can I prevent no runners being spawned in the future?

A: To prevent no runners being spawned in the future, follow these best practices:

  1. Regularly review and update the configuration file: Ensure that the configuration file is correct and that the settings are properly defined.
  2. Verify the image: Check that the image specified in the configuration file is correct and that it is available.
  3. Check the labels: Ensure that the labels specified in the configuration file are correct and that they match the expected values.
  4. Check the firecracker configuration: Verify that the firecracker configuration is correct and that it matches the expected values.
  5. Check the machine configuration: Ensure that the machine configuration correct and that it matches the expected values.
  6. Monitor the metrics and logs: Regularly monitor the metrics and logs to identify any issues or errors.

Q: What are some additional resources that can help me troubleshoot the issue of no runners being spawned?

A: Some additional resources that can help you troubleshoot the issue of no runners being spawned include:

  • Fireactions documentation: The official Fireactions documentation provides detailed information on configuring and troubleshooting Fireactions.
  • Fireactions community forums: The Fireactions community forums provide a platform for discussing Fireactions-related issues and seeking help from other users.
  • Fireactions support: Fireactions provides official support for its users, including troubleshooting and technical assistance.