🚀 Feature: Allow defining allow rules for catalog providers

[drodil]

🚀 Feature: Allow defining allow rules for catalog providers

[drodil]

Related #4977 and #5014

[Rugvip]

Generally I agree that this could be good to have in some form, perhaps by just giving access to the standardized rule config and logic to entity providers.

These examples are perhaps not the best though, org data providers are fairly hardcoded to just ingest users and groups. If we don't trust them enough to do that there's a bigger problem than what can be solved in code, we'd need to sandbox them somehow at that point. If what we're looking for is filtering I'm thinking it's better to implement it as such.

The GitHub discovery provider doesn't actually ingest these entities directly. It emits Location entities that are then later read by the catalog, which means you'd need to be forwarding these rules to those locations somehow, or something like that. Lot more discussion around that in #12880, where we ultimately landed on #14584 as the solution.

[drodil]

Yeah, the locations are a bit hard for this. The main reason for this was that I would only like to get groups from Github org data and as the users are coming already from Azure. Of course, it could be possible to add a specific config for the GithubOrgDataProvider but thinking here again more generic way for all (even custom) providers to limit the entity types.

[Rugvip]

@drodil yep thinking it's probably best to keep that behind more explicit config to exclude users tbh. Using rules you might want to trigger an alert if someone tries to ingest a disallowed entity, so it could be good to avoid using it for deliberate filtering.