Fidget
Fidget•2d ago

Login v2 pod not sending CUSTOM_REQUEST_HEADERS

Use-case: Kubernetes self-hosted proof-of-concept Environment: Self-Hosting on EKS Version: ghcr.io/zitadel/zitadel:v4.0.0 Stack: Zitadel UI v2 What you expected to happen: Zitadel UI for basic management login What went wrong: Login v2 gets 400 error due to issue with Host header to zitadel API instance Attachments: On the zitadel-login pods I see the environment variable file is set: cat /.env-file/.env 
/.env-file/.env
ZITADEL_SERVICE_USER_TOKEN_FILE="/login-client/pat"
ZITADEL_API_URL="http://zitadel:8080"
CUSTOM_REQUEST_HEADERS="Host:zitadel.example.com"
/.env-file/.env
ZITADEL_SERVICE_USER_TOKEN_FILE="/login-client/pat"
ZITADEL_API_URL="http://zitadel:8080"
CUSTOM_REQUEST_HEADERS="Host:zitadel.example.com"
Testing from the CLI with curl and wget I can send host headers and get a 200 response. However the pods logs show connection errors experienced when the CUSTOM_REQUEST_HEADERS aren't sent. 
ZITADEL_SERVICE_USER_TOKEN_FILE=/login-client/pat is set and file exists, setting ZITADEL_SERVICE_USER_TOKEN to the files content
   ▲ Next.js 15.4.0-canary.86
   - Local:        http://localhost:3000
   - Network:      http://0.0.0.0:3000

Error [ConnectError]: [unavailable] connect ECONNREFUSED 172.20.216.193:8080
    at <unknown> (ConnectError: [unavailable] connect ECONNREFUSED 172.20.216.193:8080)
    at h.from (.next/server/chunks/981.js:9:112772)
    at aB (.next/server/chunks/981.js:9:74614)
    at ClientHttp2Session.h (.next/server/chunks/981.js:9:82106) {
  rawMessage: 'connect ECONNREFUSED 172.20.216.193:8080',
  code: 14,
  metadata: Headers {},
  details: [],
  [cause]: Error: connect ECONNREFUSED 172.20.216.193:8080
      at <unknown> (Error: connect ECONNREFUSED 172.20.216.193:8080) {
    errno: -111,
    code: 'ECONNREFUSED',
    syscall: 'connect',
    address: '172.20.216.193',
    port: 8080
  }
}
ZITADEL_SERVICE_USER_TOKEN_FILE=/login-client/pat is set and file exists, setting ZITADEL_SERVICE_USER_TOKEN to the files content
   ▲ Next.js 15.4.0-canary.86
   - Local:        http://localhost:3000
   - Network:      http://0.0.0.0:3000

Error [ConnectError]: [unavailable] connect ECONNREFUSED 172.20.216.193:8080
    at <unknown> (ConnectError: [unavailable] connect ECONNREFUSED 172.20.216.193:8080)
    at h.from (.next/server/chunks/981.js:9:112772)
    at aB (.next/server/chunks/981.js:9:74614)
    at ClientHttp2Session.h (.next/server/chunks/981.js:9:82106) {
  rawMessage: 'connect ECONNREFUSED 172.20.216.193:8080',
  code: 14,
  metadata: Headers {},
  details: [],
  [cause]: Error: connect ECONNREFUSED 172.20.216.193:8080
      at <unknown> (Error: connect ECONNREFUSED 172.20.216.193:8080) {
    errno: -111,
    code: 'ECONNREFUSED',
    syscall: 'connect',
    address: '172.20.216.193',
    port: 8080
  }
}
2 Replies
Ask Inkeep
Ask Inkeep•2d ago
Hi @Fidget, I'm an AI assistant for ZITADEL. While a team member reviews this question, I'll see if I can find relevant sources for your question. Answer I wasn't able to find a direct answer in the documentation for your specific Kubernetes configuration issue, here are some relevant sources: 1. Set up Zitadel with Docker Compose : The Docker Compose documentation shows the CUSTOM_REQUEST_HEADERS environment variable being used with the login container, and mentions that if you can't use the network_mode service:zitadel, you can pass the environment variable CUSTOM_REQUEST_HEADERS=Host:localhost instead (1) . 2. Discord message from @blackr4t on 2024-10-02 : A community discussion mentions that regardless of what header comes in, it will forward the domain as ZITADEL_API_URL without https://, and there's work in progress to make this configuration easier (2) . Your issue appears to be related to network connectivity between the login container and the ZITADEL API service in your Kubernetes environment. The CUSTOM_REQUEST_HEADERS configuration should help with host header forwarding, but the connection refused error suggests a networking issue rather than just header configuration. Would you like me to help you with a general approach to troubleshooting this Kubernetes networking issue, or would you prefer to click the "Ask for help" button to have a team member review your specific setup? ​ To learn more, see the sources I considered: [1] Set up Zitadel with Docker Compose [2] Discord message from @blackr4t on 2024-10-02 ​ Feel free to tag @Ask Inkeep with additional questions.
Fidget
FidgetOP•2d ago
I don't believe it's a connection refused due to being able to hit the same address:port combination with wget from the same pod container. On login pod replicate connection error without host header . /.env-file/.env && wget -q -O - "${ZITADEL_API_URL}" 
wget: server returned error: HTTP/1.1 404 Not Found
wget: server returned error: HTTP/1.1 404 Not Found
On login pod show success with host header . /.env-file/.env && wget -q -O - --header "${CUSTOM_REQUEST_HEADERS}" "${ZITADEL_API_URL}" 
<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8"/>
    <title>ZITADEL • Console</title>
    <base href="/ui/console/"/>
    <meta name="viewport" content="width=device-width, initial-scale=1"/>
    <link id="appFavicon" rel="icon" type="image/x-icon" href="favicon.ico"/>
    <link rel="stylesheet" href="./assets/icons/line-awesome/css/line-awesome.min.css"/>
    <link rel="manifest" href="manifest.webmanifest"/>
    <meta name="theme-color" content="#e6768b"/>
...
    <meta name="robots" content="none"/>
  <link rel="stylesheet" href="styles.45af6733371ffe5f.css"></head>

  <body>
    <cnsl-root></cnsl-root>
    <noscript>Please enable JavaScript to continue using this application.</noscript>
  <script src="runtime.9d86acfd175fdf65.js" type="module"></script><script src="polyfills.4258d1ba3f1b5f57.js" type="module"></script><script src="scripts.cfdbc03626b7e4d0.js" defer></script><script src="main.1d2591fd5283265b.js" type="module"></script></body>
</html>
<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8"/>
    <title>ZITADEL • Console</title>
    <base href="/ui/console/"/>
    <meta name="viewport" content="width=device-width, initial-scale=1"/>
    <link id="appFavicon" rel="icon" type="image/x-icon" href="favicon.ico"/>
    <link rel="stylesheet" href="./assets/icons/line-awesome/css/line-awesome.min.css"/>
    <link rel="manifest" href="manifest.webmanifest"/>
    <meta name="theme-color" content="#e6768b"/>
...
    <meta name="robots" content="none"/>
  <link rel="stylesheet" href="styles.45af6733371ffe5f.css"></head>

  <body>
    <cnsl-root></cnsl-root>
    <noscript>Please enable JavaScript to continue using this application.</noscript>
  <script src="runtime.9d86acfd175fdf65.js" type="module"></script><script src="polyfills.4258d1ba3f1b5f57.js" type="module"></script><script src="scripts.cfdbc03626b7e4d0.js" defer></script><script src="main.1d2591fd5283265b.js" type="module"></script></body>
</html>

Did you find this page helpful?