Inviting individual users from another org when "Check for Project on Authentication" is enabled?

Let's say I have two organizations, AA with and BB and they both have their own project (projectAprojectA, projectBprojectB) and an application (appA,appA, appBappB) . In projectBprojectB I useCheck for Project on AuthenticationCheck for Project on Authentication setting. Now only users of org BB can access project BB (as intended).

How can I then allow some select individuals of organization AA to log in to appBappB? I've tried creating a role in projectBprojectB and then authorizing users of AA, but I keep getting Login not possible. The organization of the user must be granted to the projectLogin not possible. The organization of the user must be granted to the project message.