Network-Traffic-Monitoring-with-grafana

Project Overview

This project provides a comprehensive solution for capturing network traffic, processing it with Zeek (formerly Bro), and storing the enriched logs into a MySQL database for further analysis and visualization. It includes scripts and configurations to enhance Zeek's capabilities with GeoIP, ASN data, and JA3/JA4 fingerprinting, enabling detailed network security monitoring and analysis. Features

Traffic Capture: Utilizes tcpdump to capture live network traffic.
Log Processing: Processes captured traffic with Zeek, enriched with GeoIP, ASN, and JA3/JA4 data.
Data Storage: Converts Zeek logs into a MySQL database for easy querying and analysis.
Automation: Includes scripts to automate the entire process and run it as a service.
Customization: Allows customization of network interfaces, capture durations, and database configurations.

Project Structure

zeek_to_mysql.sh: Shell script to manage traffic capturing, Zeek processing, and database insertion.
zeek_to_mysql.py: Python script to convert processed Zeek logs into MySQL.
asn_enrichment.zeek: Zeek script for ASN and GeoIP enrichment.
.env / zeek_to_mysql_config.ini: Configuration files to store environment variables.
setup.sh: Enhanced setup script for initial configuration and automation.
README.md: Documentation and usage instructions.

1. Log Ingestion & Normalization:
2. Integrated Zeek, Suricata, and OSSEC to collect and parse logs.
3. Used Logstash for transformation and forwarding to Elasticsearch.
4. Threat Intelligence Enrichment:
5. Integrated AlienVault OTX and AbuseIPDB APIs for IP reputation.
6. Added JA3/JA4 fingerprinting to track SSL/TLS-based threats.
7. Visualization & Alerting:
8. Created custom Grafana dashboards.
9. Configured alert thresholds using Prometheus and Alertmanager.

• Reduced incident response time by 45%.
• Identified 3 previously unknown lateral movement attempts.
• Strengthened alert fidelity with enriched data.
• Created 7 custom dashboards for live monitoring.

• Centralized log collection.
• Enhanced visibility of malicious events.
• Easy integration with external tools such as ElastAlert and Prometheus.
• Reduction in alert fatigue.

• Challenge: High data volume and noisy alerts.
• Solution: Implemented normalization and alert suppression logic using Logstash and Prometheus rules.

FAQs

Q: What is Zeek? A: Zeek is an open-source network analysis framework focused on security monitoring. It provides detailed logs of network activity for analysis.

Q: How can I verify that Zeek is processing traffic? A: Check the log files generated in the specified base directory. Zeek outputs various log files such as conn.log, dns.log, ssl.log, etc.

Q: Can I change the network interface used for capturing traffic? A: Yes, you can specify a different network interface in the .env file or update it using the --update option in the script.

Q: How do I integrate MaxMind GeoIP data? A: Download the GeoIP databases from MaxMind, place them in a directory (e.g., /usr/share/GeoIP/), and configure Zeek to use them by setting redef GeoIP::db_dir.

Q: How do I extract SSL issuer information with Zeek? A: Load the SSL script in Zeek and modify the logging to include the issuer information. See the Extracting SSL Issuer Information section.