Kafka Authorizer should retrieve ACLs with NDB Event API

Description

Currently, HopsKafkaAuthorizer retrieves Kafka ACLs that are stored in the Hopsworks database, for every message that arrives to the brokers. Since ACLs are infrequently updated, this introduces an unnecessary overhead.

This JIRA proposes that HopsKafkaAuthorizer uses the NDB Event API to update its local cache of ACLs whenever they are updated. A single new/updated/removed ACL will be pushed to HopsKafkaAuthorizer instances of all brokers, maintaining strong consistency.

Status

Assignee

Theofilos Kakantousis

Reporter

Theofilos Kakantousis

Labels

None

Priority

High
Configure