some-dont•4w ago

[Bug] Zitadel Chart - v2 login 'wait-for-zitadel' fails with service mesh (linkerd) K8S

The wait-for-zitadel init container in the login deployment fails when running with Linkerd (probably Istio too). Problem: Zitadel v4 serves HTTP/2 on port 8080. The init container uses wait4x/wait4x:3.5 which only speaks HTTP/1.1. When it tries to check /debug/ready, it dies with:

Error: net/http: HTTP/1.x transport connection broken: 
malformed HTTP response "\x00\x00\x12\x04..."

Error: net/http: HTTP/1.x transport connection broken: 
malformed HTTP response "\x00\x00\x12\x04..."

That's the HTTP/2 connection preface. wait4x doesn't know what to do with it. Result: Login pods stuck at Init:2/3 forever. Main Zitadel works fine, just can't start login. Workaround: I'm using a Kustomize patch to replace the init container with curl that supports HTTP/2:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: zitadel-login
  namespace: my-namespace
spec:
  template:
    spec:
      initContainers:
        - name: wait-for-zitadel
          image: curlimages/curl:8.10.1
          command:
            - sh
            - -c
            - |
              until curl -f -s --http2-prior-knowledge --max-time 5 http://zitadel:8080/debug/ready > /dev/null 2>&1; do
                sleep 5
              done

apiVersion: apps/v1
kind: Deployment
metadata:
  name: zitadel-login
  namespace: my-namespace
spec:
  template:
    spec:
      initContainers:
        - name: wait-for-zitadel
          image: curlimages/curl:8.10.1
          command:
            - sh
            - -c
            - |
              until curl -f -s --http2-prior-knowledge --max-time 5 http://zitadel:8080/debug/ready > /dev/null 2>&1; do
                sleep 5
              done

The --http2-prior-knowledge flag makes curl use HTTP/2 from the start. Location: charts/zitadel/templates/deployment_login.yaml line 82ish Anyone else hitting this or just me? Should I open a GitHub issue?

4 Replies

Ask Inkeep•4w ago

Hi @some-dont, I'm an AI assistant for ZITADEL. While a team member reviews this question, I'll see if I can find relevant sources for your question. Answer I wasn't able to find a direct answer, here are some relevant sources: 1. HTTP/2 Support in ZITADEL : This confirms ZITADEL makes heavy use of gRPC and requires HTTP/2 communication, which explains your issue with the HTTP/1.1-only wait4x tool. 2. Istio service mesh as a reverse proxy : Shows similar challenges with service mesh configurations where init containers had issues with Istio sidecars. Feel free to tag @Ask Inkeep with additional questions. Got it. Tagging @Rajat for additional help.

some-dontOP•4w ago

I opened a pull request to fix this issue. https://github.com/zitadel/zitadel-charts/pull/463

GitHub

fix: replace wait4x with curl for HTTP/2 support in login init cont...

Problem The wait-for-zitadel init container uses wait4x to health check the ZITADEL API before starting the login pod. wait4x doesn&#39;t properly handle HTTP/2 (h2c), causing health check fail...

Matías•3w ago

Hi @some-dont, thanks for your contribution! I will ping our Product Manager internally to take a look 🙇‍♂️

Ask Inkeep•3w ago

Glad I could be helpful. Feel free to create a new thread with any new questions you may have.

[Bug] Zitadel Chart - v2 login 'wait-for-zitadel' fails with service mesh (linkerd) K8S

Did you find this page helpful?