User Tools

Site Tools


giganet:sflow

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
giganet:sflow [2021/08/02 13:43]
rb [kafka]
giganet:sflow [2021/08/03 13:28] (current)
rb
Line 1: Line 1:
-====== Схема мониторинга sflow статистики ====== 
- 
-===== Задействованные сервера ===== 
- 
-  * **goflow **- сервер на котором крутится kafka и goflow2 приложение по сбору статистики 
-  * **clickhouse **- сервер с базой данных clickhouse, хранение и обработка статистики 
-===== Задействованные приложения ===== 
- 
-  * **goflow2** 
-  * **kafka** 
-  * **clickhouse** 
 ===== Общая схема мониторинга ===== ===== Общая схема мониторинга =====
  
Line 36: Line 25:
  
 </code> </code>
- 
  
 ==== goflow2 ==== ==== goflow2 ====
Line 72: Line 60:
  
 <code> <code>
- 
 [Unit] [Unit]
 Requires=network.target remote-fs.target Requires=network.target remote-fs.target
Line 92: Line 79:
  
 <code> <code>
- 
 [Unit] [Unit]
 Requires=zookeeper.service Requires=zookeeper.service
Line 127: Line 113:
  
 </code> </code>
- 
  
 ==== clickhouse ==== ==== clickhouse ====
Line 135: Line 120:
 [[https://github.com/netsampler/goflow2/blob/main/pb/flow.proto|https://github.com/netsampler/goflow2/blob/main/pb/flow.proto]] \\ [[https://github.com/netsampler/goflow2/blob/main/pb/flow.proto|https://github.com/netsampler/goflow2/blob/main/pb/flow.proto]] \\
 [[https://clickhouse.tech/docs/ru/engines/table-engines/integrations/kafka/|https://clickhouse.tech/docs/ru/engines/table-engines/integrations/kafka/]] \\ [[https://clickhouse.tech/docs/ru/engines/table-engines/integrations/kafka/|https://clickhouse.tech/docs/ru/engines/table-engines/integrations/kafka/]] \\
-https://github.com/AlexeyKupershtokh/clickhouse-maxmind-geoip/ \\ +[[https://github.com/AlexeyKupershtokh/clickhouse-maxmind-geoip/|https://github.com/AlexeyKupershtokh/clickhouse-maxmind-geoip/]] \\ 
-https://gist.github.com/sanchezzzhak/511fd140e8809857f8f1d84ddb937015+[[https://gist.github.com/sanchezzzhak/511fd140e8809857f8f1d84ddb937015|https://gist.github.com/sanchezzzhak/511fd140e8809857f8f1d84ddb937015]]
  
 === таблица kafka === === таблица kafka ===
Line 302: Line 287:
 PARTITION BY toYYYYMMDD(Date) PARTITION BY toYYYYMMDD(Date)
 ORDER BY Datetime ORDER BY Datetime
-TTL Date toIntervalDay(7)+TTL Date   toIntervalDay(7)
 SETTINGS index_granularity = 8192 SETTINGS index_granularity = 8192
  
giganet/sflow.1627901030.txt.gz · Last modified: 2021/08/02 13:43 by rb