Log Management and Monitoring

Effective log management and monitoring are essential components of maintaining a secure and stable Ridgeback environment. Logs are the digital footprints of your Ridgeback deployment, offering insights into operational performance, security events, configuration issues, and network health. This chapter discusses different log sources—service container logs, Rcore logs, and network event logs—and explains how to use them for troubleshooting, auditing, and ongoing network monitoring.

The Service Container Logs

What Are They?
Service container logs are generated by Ridgeback’s collection of containers—such as the server, policy, analytics, enrichment, and surface services. These logs primarily include:

• Startup and Shutdown Messages: Confirming whether containers launch correctly and shut down gracefully.
• Configuration Warnings: Issues with reading .env variables, connecting to external services (like the database or email servers), and missing resources.
• Runtime Errors and Exceptions: Detailed error messages when a service encounters unexpected conditions, such as database query failures or invalid API requests.
• Informational Alerts: Notifications about normal operations, periodic tasks, policy triggers, or license checks.

How to Access Service Container Logs:

1. Docker CLI:

   docker compose logs
   docker compose logs server
   docker compose logs -f policy
   

   Adding -f (follow) lets you stream logs in real-time.

2. Docker Desktop / Other GUI Tools:
   If using Docker Desktop or another GUI, view logs via the containers’ GUI interface.

3. Centralized Log Management (Optional):
   Consider exporting logs to a centralized log management system like ELK (Elasticsearch, Logstash, Kibana) or Splunk. This allows indexing, searching, and correlation with other infrastructure logs.

Use Cases:

• Troubleshooting Startup Issues: If a container fails to start, check logs for missing environment variables or invalid configs.
• Monitoring Policy Trigger Outcomes: Identify when policies fire actions or generate alerts.
• Performance Analysis: Look for signs of slow queries or timeouts that might indicate performance bottlenecks.

Retention and Rotation: By default, Docker may store logs indefinitely. Configure log rotation (e.g., in docker-compose.yml with a logging section) to prevent disks from filling up:

logging:
  driver: "json-file"
  options:
    max-size: "10m"
    max-file: "3"

The Rcore Logs

What Are They?
Rcores run as executables that capture network packets, identify endpoints, and implement phantom responses. Rcore logs record:

• Startup Parameters: Which command line arguments were used, which interfaces are monitored.
• Link Status Changes: Notices if the Rcore sees interface link up/down events.
• Anomalies and Warnings: Packets that do not parse correctly, attempts to contact nonexistent services, or timing information if the Rcore is struggling to keep up with traffic.
• Phantom Activation: Information about when and why a phantom was triggered (if active mode is used).

How to Access Rcore Logs:

1. Console Output: If you run Rcore via a terminal (e.g., ./rcore-win.exe ...), logs will appear in the console.
2. Redirecting Logs to a File:

   ./rcore-linux --arguments > rcore.log 2>&1
   

   Keep these files secure, as they can contain sensitive network metadata.
3. Systemd or Startup Scripts: If running Rcore as a systemd service:

   journalctl -u rcore.service
   

4. Remote Logging: If needed, consider sending logs to a remote syslog server for centralized analysis.

Use Cases:

• Validation of Rcore Configuration: Check if Rcore accepted the provided parameters or if it reports issues with the chosen network interface.
• Phantom Behavior Analysis: If phantoms are not appearing or are too aggressive, logs can help fine-tune settings.
• Network Troubleshooting: Identify if the Rcore detects abnormal traffic patterns, time synchronization issues, or missing gateway configurations.

Rotation and Retention: Rotate Rcore logs using standard OS mechanisms (e.g., logrotate on Linux) to avoid huge log files:

sudo nano /etc/logrotate.d/rcore

Configure a rotation policy that suits your retention and compliance requirements.

The Network Event Log

What Is It?
Ridgeback records network events—such as ARP requests, DNS queries, TCP connection attempts, ICMP pings—as structured data in the database. Unlike service container or Rcore logs (which focus on the operations of the Ridgeback system itself), the network event log focuses on the actual network traffic observed, providing a historical record of network behavior.

Where Is It Stored?
The network event logs are stored in the Ridgeback database, typically in tables like NetEvent. They include fields such as source/destination IPs, timestamps, event types, and whether the traffic was associated with a phantom or a live endpoint.

How to Access and Query:

1. Ridgeback UI: Use the web-based interface to browse and filter events. Search for endpoints, time ranges, or event types to identify patterns or suspicious activity.
2. Direct SQL Queries:

   SELECT * FROM Data_00000000_0000_0000_0000_000000000000.NetEvent
   WHERE time > NOW() - INTERVAL 1 HOUR
   ORDER BY time DESC;
   

   This can be useful for custom reports, policy triggers, or integrating with external analytics tools.
3. API Integration: Some Ridgeback deployments may have an API to extract events programmatically.

Use Cases:

• Forensics and Incident Response: Investigate suspicious behavior by reviewing historical event data. For example, identify which endpoints attempted to contact a phantom or connected to an unknown IP address.
• Network Baseline: Establish normal traffic patterns by analyzing event data over time, detect anomalies more easily.
• Policy Enforcement Verification: Check if policies that block or alert on certain traffic patterns are working correctly. The network event log confirms whether corresponding events occurred.

Retention and Pruning: The network event data can grow large over time. Implement a pruning strategy:

• Data Retention Policies: Depending on compliance or forensics needs, keep data for a set period (e.g., 90 days).
• Database Maintenance: Use database tools (e.g., DELETE queries, or partitioning) to remove old data and keep the database performing efficiently.
• Backups: Regularly back up the NetEvent tables if required for long-term archival.

Combining Logs for Comprehensive Monitoring

Correlating Logs:

• Rcore + Container Logs: If a container reports frequent database timeouts at the same time the Rcore logs show network instability, you have a clue that underlying network issues affect both data ingestion and service responses.
• Network Events + Policy Logs: If a policy alert fires off, review the corresponding network events and container logs to see what triggered it and confirm correct behavior.

Alerts and Automation:

• Integrate logs into a SIEM (Security Information and Event Management) solution. Parse logs for keywords or anomalies and raise alerts when suspicious patterns emerge.
• Automate reports: Generate daily summaries of top talkers, recurring phantom triggers, or policy violations. Use container logs and network event data to provide a full picture of network health.

Best Practices for Log Management

1. Ensure Time Synchronization: Make sure that all systems (Rcore host, container host, database server) use NTP. Accurate timestamps are crucial for correlating events.
2. Limit Access to Logs: Restrict log file access to authorized personnel only. Logs may contain sensitive information such as IP addresses, hostnames, and event details.
3. Regular Reviews: Don’t just collect logs—review them. Set up routines to watch for unusual patterns or performance issues.
4. Test Incident Response Scenarios: Use historical logs to practice forensic exercises and improve response workflows.