Starting the multiple triggers with Github interceptor is limited by the client rate limiter

[aqaliarept]

I'm trying to build a developer's cluster with ephemeral environments for feature branches.
Each environment is located in a dedicated namespace (name same as the git branch name) and contains a set of triggers for building services in the repository.

Each trigger filters git pushes by the branch name and service folder inside the git repository. The basic idea is to start the pipeline only if the push was made for a proper branch and contains changes for the specific service.

spec:
  interceptors:
    - ref:
        name: "github"
      params:
        - name: "secretRef"
          value:
            secretName: github-interceptor-secret
            secretKey: secretToken
        - name: "eventTypes"
          value: ["push"]
        - name: "addChangedFiles"
          value:
            enabled: true
    - ref:
        name: cel
      params:
      - name: filter
        value: |
          body.ref.split('/')[2].matches('{{ $env.branchName }}') && (
            extensions.changed_files.matches('{{ $image.build.sourceDir }}/')
          )  

There is a single EventListener is called by the git webhook . It spawns all those triggers in all namespaces:

spec:
  namespaceSelector:
    matchNames:
    - "*"  
  labelSelector:
    matchLabels:
      build: backend

The problem : when the trigger number grows over 25, the rest of the triggers start to fail with the error:

"interceptor stopped trigger processing: rpc error: code = FailedPrecondition desc = error getting secret: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline"

It looks like the triggers are limited by the cooperative access to the secret, specified in secretRef configuration.

[timur-nugaev]

I found the workaround with the usage of TriggerGroups and the single github Interceptor.

[savitaashture]

WG discussion:

Based on the issue description even we thought that TriggerGroups and the single github Interceptor is a way to go as here you are using same interceptor with same secret

If there is a use case to access different secret then we may need to look into it in order to avoid the rate limit