Network Device Monitoring with Grafana

This project introduces a scalable and automated approach to network visibility using periodic Nmap scans stored in MariaDB, visualized in Grafana dashboards through Prometheus exporters. By correlating scan results over time, it enables detection of device state changes, port activity, and network shifts. Alerting and historical analysis are made possible with scheduled cron jobs, database enrichment, and detailed metrics collection.

Technical Implementation

🔍 Phase 1: Network Scanning and Data Collection

• Created automated Nmap scanning scripts that run via cron every 12 hours.
• Scans gather open ports, active IPs, service banners, and operating system fingerprints.
• Parsed results are injected into a MariaDB database with timestamped records.

💾 Phase 2: Database and Enrichment

• Structured the MariaDB schema to include device IP, hostname, scan timestamp, port info, and OS detection.
• Implemented change detection via hashed records and timestamp deltas.
• Enriched data with custom fields like "geo-location" (optional with MaxMind), ownership tagging, and criticality levels.

📊 Phase 3: Visualization with Grafana

• Used Grafana to connect to MariaDB and visualize data using SQL queries.
• Built dashboards showing:
• New/Removed Hosts
• Open Ports Over Time
• Protocol Heatmaps
• Critical Device Availability
• Integrated filters by time range, subnet, or service.

🚨 Phase 4: Alerting and Automation

• Added alert logic using Prometheus AlertManager and custom scripts that:
• Trigger alerts when new devices appear
• Notify on missing critical devices
• Alert on service/port changes (e.g., SSH newly opened)
• Notifications sent via email and Discord.

Technical Architecture:

- Scanning Layer: Cron + Nmap + Python/Bash scripts
- Storage Layer: MariaDB (relational, timestamped, enriched)
- Query Layer: Grafana with raw SQL queries
- Alert Layer: AlertManager and Discord webhook scripts
- Integration Layer: Optional: MaxMind for geo enrichment, webhook endpoints

• 99.9% Device Coverage: Includes home routers, IoT, corporate firewalls, and Linux servers.
• Port Change Detection: Able to detect and alert on stealthy service reconfigurations (e.g., backdoors).
• Incident Response Improvement: Reduced device compromise detection from days to hours.
• Low Resource Usage: Lightweight cron-based implementation, suitable even for Raspberry Pi setups.

Implementing network device monitoring with Grafana and Prometheus offers several key benefits: - Enhanced Observability: Gain detailed insights into network performance, including metrics like CPU load, memory utilization, and bandwidth usage, facilitating proactive issue detection. - Scalability and Flexibility: The combination efficiently handles dynamic environments, scaling seamlessly with your network's growth and adapting to evolving monitoring needs. - Cost-Effective Solution: As open-source tools, Grafana and Prometheus provide robust monitoring capabilities without the expense of proprietary software, making them accessible for organizations of all sizes. - Proactive Alerting: Set up alerts based on specific metrics or conditions, enabling teams to address potential issues before they impact users, thereby improving system reliability. - Comprehensive Visualization: Grafana offers customizable dashboards that integrate data from various sources, providing a centralized view of your network's health and performance.

🧩 Legacy Device Discovery

• Issue: Some devices silently dropped ICMP or responded slowly.
• Solution: Tuned Nmap timing profiles and used multiple scan techniques (e.g., TCP SYN, ACK).

🧩 Database Bloat

• Issue: High frequency scans led to large data accumulation.
• Solution: Implemented purging scripts and downsampled older data for long-term trend storage.

🧩 Data Correlation Errors

• Issue: Devices with DHCP leases caused identity ambiguity.
• Solution: Correlated MAC addresses and hostnames with scan results.

This solution bridges the gap between basic Nmap scans and enterprise-grade network visibility. It transforms raw scan data into real-time insights using Grafana, with historical context and proactive alerts. It is lightweight, modular, and suitable for both security-conscious professionals and small teams.

The project stands as a showcase of automation, visibility, and open-source potential in modern network security operations.