flume 分类

Flume的常用Channel

Memory Channel

File Channel

agent-aisi.sources = source-aisi
agent-aisi.channels = channel-aisi
agent-aisi.sinks =  sink-aisi

agent-aisi.sources.source-aisi.type = netcat
agent-aisi.sources.source-aisi.bind = localhost
agent-aisi.sources.source-aisi.port = 44444

agent-aisi.channels.channel-aisi.type = file
agent-aisi.channels.channel-aisi.dataDir = /root/filedata

agent-aisi.sinks.sink-aisi.type = logger

agent-aisi.sources.source-aisi.channels = channel-aisi
agent-aisi.sinks.sink-aisi.channel = channel-aisi

shenjianZ...大约 2 分钟

快速上手

Flume环境搭建

JDK安装

rpm -ivh /public/software/java/jdk-8u144-linux-x64.rpm

注意配置环境变量

vim /etc/prifile

shenjianZ...大约 3 分钟

Sink组件

Flume常用Sink

File Roll Sink

agent-aisi.sources = source-aisi
agent-aisi.channels = channel-aisi
agent-aisi.sinks =  sink-aisi

agent-aisi.sources.source-aisi.type = netcat
agent-aisi.sources.source-aisi.bind = localhost
agent-aisi.sources.source-aisi.port = 44444

agent-aisi.channels.channel-aisi.type = memory
agent-aisi.channels.channel-aisi.capacity = 10000
agent-aisi.channels.channel-aisi.transactionCapacity = 100

agent-aisi.sinks.sink-aisi.type = file_roll
agent-aisi.sinks.sink-aisi.sink.directory = /root/file_rool
agent-aisi.sinks.sink-aisi.sink.roolInterval = 10

agent-aisi.sources.source-aisi.channels = channel-aisi
agent-aisi.sinks.sink-aisi.channel = channel-aisi

shenjianZ...大约 2 分钟