Apache Flume Channel Selectors

Apache Flume Channel Selector is the component of Flume that determines the channel from a group that should be used to move events.

Apache Flume provides different types of Channel Selectors which are mentioned below.

  1. Replicating Channel Selector
  2. Multiplexing Channel Selector
  3. Custom Channel Selector

Let us see each Channel Selector in detail.

1. Replicating Channel Selector (default)

Apache Flume Replicating Channel Selector is the default Channel Selector that is used to transfer the events if no channel selectors are configured.

Example for Apache Flume Replicating Channel Selector.

agentone.sources = r1 agentone.channels = c1 c2 c3 agentone.sources.r1.selector.type = replicating agentone.sources.r1.channels = c1 c2 c3 agentone.sources.r1.selector.optional = c3

In the above configuration, c3 is an optional channel so failure to write to c3 will be ignored and since c1 and c2 are not marked optional in that case failure to write to those channels will also cause the transaction to fail.

2. Multiplexing Channel Selector

Apache Flume multiplexing Channel Selector is used to write an event to different channels based on header information.

Example for Apache Flume Multiplexing Channel Selector.

agentone.sources = r1 agentone.channels = c1 c2 c3 c4 agentone.sources.r1.selector.type = multiplexing agentone.sources.r1.selector.header = state agentone.sources.r1.selector.mapping.CZ = c1 agentone.sources.r1.selector.mapping.US = c2 c3 agentone.sources.r1.selector.default = c4

3. Custom Channel Selector

Apache Flume custom channel selector is the user’s implementation of the Channel Selector interface. A custom channel selector’s class and its dependencies must be included in the agent’s classpath when starting the Flume agent.

Example for Apache Flume Custom Channel Selector.

agentone.sources = r1 agentone.channels = c1 agentone.sources.r1.selector.type = org.example.MyChannelSelector