SSH Key Injection Issue With Latest Debian Image

by ADMIN 49 views

Introduction

In this article, we will discuss an issue with SSH key injection using the latest Debian image in Jenkins. The issue arises when trying to run a remote Docker agent from a job, resulting in a failed agent start and an SSH error on Jenkins. We will explore the reproduction steps, expected and actual results, and any additional information that may be relevant to resolving this issue.

Jenkins and Plugins Versions Report

Environment

  • Jenkins: 2.492.3
  • OS: Linux - 6.1.0-13-amd64
  • Java: 21.0.6 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
  • Docker Cloud Server: Debian 10.13
  • Node Image: jenkins/ssh-agent:jdk21

Jenkins Plugins Versions

  • analysis-model-api: 13.2.0
  • ansicolor: 1.0.6
  • ant: 513.vde9e7b_a_0da_0f
  • antisamy-markup-formatter: 173.v680e3a_b_69ff3
  • apache-httpcomponents-client-4-api: 4.5.14-269.vfa_2321039a_83
  • apache-httpcomponents-client-5-api: 5.4.3-140.v2516ccde99e7
  • asm-api: 9.8-135.vb_2239d08ee90
  • authentication-tokens: 1.131.v7199556c3004
  • authorize-project: 2.0.0
  • basic-branch-build-strategies: 228.v68c089762a_db_
  • blueocean: 1.27.18
  • blueocean-autofavorite: 1.2.5
  • blueocean-bitbucket-pipeline: 1.27.18
  • blueocean-commons: 1.27.18
  • blueocean-config: 1.27.18
  • blueocean-core-js: 1.27.18
  • blueocean-dashboard: 1.27.18
  • blueocean-display-url: 2.4.4
  • blueocean-events: 1.27.18
  • blueocean-git-pipeline: 1.27.18
  • blueocean-github-pipeline: 1.27.18
  • blueocean-i18n: 1.27.18
  • blueocean-jira: 1.27.18
  • blueocean-jwt: 1.27.18
  • blueocean-personalization: 1.27.18
  • blueocean-pipeline-api-impl: 1.27.18
  • blueocean-pipeline-editor: 1.27.18
  • blueocean-pipeline-scm-api: 1.27.18
  • blueocean-rest: 1.27.18
  • blueocean-rest-impl: 1.27.18
  • blueocean-web: 1.27.18
  • bootstrap5-api: 5.3.3-2
  • bouncycastle-api: 2.30.1.80-256.vf98926042a_9b_
  • branch-api: 2.1217.v43d8b_b_d8b_2c7
  • build-timeout: 1.38
  • buildtriggerbadge: 251.vdf6ef853f3f5
  • built-on-column: 1.5
  • caffeine-api: 3.2.0-166.v72a_6d74b_870f
  • checks-api: 370.vb_61a_c57328f3
  • cloud-stats: 377.vd8a_6c953e98e
  • cloudbees-bitbucket-branch-source: 936.0.1
  • cloudbees-folder: 6.1012.v79a_86a_1ea_c1f
  • command-launcher: 123.v37cfdc92ef67
  • commons-compress-api: 1.27.1-3
  • commons-httpclient3-api: 3.1-3
  • commons-lang3-api: 3.17.0-87.v5cf526e63b_8b_
  • commons-text-api: 1.13.0-153.v91dcd89e2a_22
  • config-file-provider: 982.vb_a_e458a_37021
  • configuration-as-code: 1958.vddc0d369b_e16
  • configuration-as-code-groovy: 1.1
  • coverage: 2.4.0
  • credentials: 1415.v831096eb_5534
  • credentials-binding: 687.v619cb_15e923f
  • data-tables-api: 2.2.2-1
  • dependency-track: 6.0.1
  • depgraph-view: 1.0.5
  • display-url-api: 2.209.v582ed814ff2f
  • docker-commons: 451.vd12c371eeeb_3
  • docker-java-api: 3.5.0-108.v211cdd21c383
  • docker-plugin: 1274.vc0203fdf2e74
  • docker-workflow: 611.v16e84da_6d3ff
  • durable-task: 587.v84b_877235b_45
  • echarts-api: 5.6.0-3
  • eddsa-api: 0.3.0.1-19.vc432d923e5ee
  • email-ext: 1876.v28d8d38315b_d
  • envinject: 2.926.v69c9b_3896a_96
  • envinject-api: 1.235.va_14c74f8f487
  • external-monitor-job: 223.vb_fddcf42c9b_3
  • favorite: 2.225.v68765b_b_a_1fa_3
  • flatpickr-api: 4.6.13-18.vcf5f6a_5b_8468
  • font-awesome-api: 6.7.2-1
  • forensics-api: 3.1.0
  • git: 5.7.0
  • git-client: 6.1.3
  • git-parameter: 439.vb_0e46ca_14534
  • git-server: 137.ve0060b_432302
  • github: 1.43.0
  • github-api: 1.321-488.v9b_c0da_9533f8
  • github-branch-source: 1815.v9152b_2ff7a_1b_
  • gitlab-api: 5.6.0-100.v83f8f4b_f1129
  • gitlab-branch-source: 718.v40b_5f0e67cd3
  • gitlab-oauth: 1.22
  • gitlab-plugin: 1.9.8
  • gradle: 2.14.1
  • groovy: 497.v7b_061a_a_de65d
  • gson-api: 2.13.0-133.v5a_e3236a_8251
  • h2-api: 11.1.4.199-36.vb_ee07e965744
  • handy-uri-templates-2-api: 2.1.8-36.v85e4cb_234a_13
  • hashicorp-vault-plugin: 371.v884a_4dd60fb_6
  • htmlpublisher: 425
  • instance-identity: 203.v15e81a_1b_7a_38
  • ionicons-api: 82.v0597178874e1
  • jackson2-api: 2.18.3-402.v74c4eb_f122b_2
  • jacoco: 3.3.7
  • jakarta-activation-api: 2.1.3-2
  • jakarta-mail-api: 2.1.3-2
  • javadoc: 327.vdfe586651ee0
  • javax-activation-api: 1.2.0-8
  • javax-mail-api: 1.6.2-11
  • jaxb: 2.3.9-133.vb_ec76a_73f706
  • jdk-tool: 83.v417146707a_3d
  • jenkins-design-language: 1.27.18
  • jersey2-api: 2.45-154.v4ded3dc34f81
  • jira: 3.15
  • jira-steps: 2.0.180.vccfe35b_5910d
  • jjwt-api: 0.11.5-120.v0268cf544b_89
  • jnr-posix-api: 3.1.20-138.vdb_9db_a_39182f
  • jobConfigHistory: 1305.vf20a_356586b_8
  • joda-time-api: 2.14.0-127.v7d9da_295a_d51
  • jquery: 1.12.4-3
  • jquery3-api: 3.7.1-3
  • jsch

Q: What is the issue with SSH key injection using the latest Debian image in Jenkins?

A: The issue arises when trying to run a remote Docker agent from a job, resulting in a failed agent start and an SSH error on Jenkins.

Q: What are the reproduction steps for this issue?

A: To reproduce the issue, follow these steps:

  1. Set up a Docker cloud on TCP with an agent template using the image ssh-agent:jdk21.
  2. Configure the template to use "/home/jenkins" as the FS root and "Inject SSH Key" as the connection method with "jenkins" as the user.
  3. Try to run a remote Docker agent from a job.

Q: What are the expected results?

A: The expected result is that the agent starts successfully.

Q: What are the actual results?

A: The actual result is that the agent fails to start, and an SSH error is displayed on Jenkins.

Q: What are the logs on the Docker host?

A: The logs on the Docker host display the following error message:

+ [[ '' == ssh-* ]]
+ [[ '' == ssh-* ]]
+ env
+ grep _
+ [[ 8 -gt 0 ]]
+ echo 'setup-sshd params: /usr/sbin/sshd' -D -p 22 -o AuthorizedKeysCommand=/root/authorized_key -o AuthorizedKeysCommandUser=root
+ [[ /usr/sbin/sshd == ssh-* ]]
+ [[ /usr/sbin/sshd -D -p 22 -o AuthorizedKeysCommand=/root/authorized_key -o AuthorizedKeysCommandUser=root == \/\u\s\r\/\s\b\i\n\/\s\s\h\d\ \-\D\ \-\p\ \2\2 ]]
+ echo 'Executing params: '\''/usr/sbin/sshd' -D -p 22 -o AuthorizedKeysCommand=/root/authorized_key -o 'AuthorizedKeysCommandUser=root'\'''
+ exec /usr/sbin/sshd -D -p 22 -o AuthorizedKeysCommand=/root/authorized_key -o AuthorizedKeysCommandUser=root
setup-sshd params: /usr/sbin/sshd -D -p 22 -o AuthorizedKeysCommand=/root/authorized_key -o AuthorizedKeysCommandUser=root
Executing params: '/usr/sbin/sshd -D -p 22 -o AuthorizedKeysCommand=/root/authorized_key -o AuthorizedKeysCommandUser=root'
sshd: no hostkeys available -- exiting.

Q: Is this issue specific to the latest Debian image?

A: No, this issue is not specific to the latest Debian image. The issue also occurs with the 6.11.1 version of the ssh-agent image and with JDK 17.

Q: Is there any additional information that may be relevant to resolving this issue?

A: Yes, the issue is not present when using the 6.11.1 version of the ssh-agent image or with JDK 17. This suggests that the issue may be related to a specific version of the ssh-agent image or JDK.

Q: Are you interested in contributing a fix?

A: Yes, we are interested in contributing a fix for this issue. If you have any suggestions or solutions, please let us know.

Q: What are the Jenkins and plugins versions used in this issue?

A: The Jenkins and plugins versions used in this issue are* Jenkins: 2.492.3

  • Plugins:
    • analysis-model-api: 13.2.0
    • ansicolor: 1.0.6
    • ant: 513.vde9e7b_a_0da_0f
    • ...
    • ssh: 158.ve2a_e90fb_7319
    • ssh-agent: 384.ve275343791a_6
    • ssh-credentials: 355.v9b_e5b_cde5003
    • ssh-slaves: 3.1031.v72c6b_883b_869
    • sshd: 3.353.v2b_d33c46e970

Q: What are the Docker and node images used in this issue?

A: The Docker and node images used in this issue are:

  • Docker Cloud Server: Debian 10.13
  • Node Image: jenkins/ssh-agent:jdk21

Q: What are the operating systems used in this issue?

A: The operating systems used in this issue are:

  • Jenkins Controller: Docker (Debian 12.2 host)
  • Docker Cloud Server: Debian 10.13
  • Node Image: jenkins/ssh-agent:jdk21