💾 Cache

Optimize Jibril's memory usage and detection accuracy by tuning cache sizes for your workload. Proper cache configuration ensures comprehensive monitoring without excessive resource consumption.

🧠 Understanding Caches

Jibril uses in-memory caches to correlate system events, track processes, and maintain behavioral context. These caches bridge the gap between kernel-level eBPF events and userland detection logic. While some data is inherently volatile and must always be queried from the kernel, a significant portion is non-volatile. Caching this non-volatile data eliminates multiple round trips to the kernel, improving performance.

📖 Cache sizing has an direct relationship with cadence configuration: shorter check intervals allow for smaller caches (since data is processed more frequently), while longer intervals require larger caches to retain context between checks.

🎯 Purpose

Store transient data about:

• Running processes and tasks
• Files accessed and their attributes
• Network flows and connections
• Behavioral correlations

⚖️ Trade-offs

Larger caches:

• ✅ Fewer missed detections
• ✅ Better correlation accuracy
• ❌ Higher memory usage

Smaller caches:

• ✅ Lower memory footprint
• ❌ Possible missed detections
• ❌ Less context retention

📖 Jibril's caching system combined with its cadence configuration is one of the key factors behind its exceptional performance.

This combination enables Jibril to respond quickly to system changes while maintaining the context needed to detect sophisticated threats that unfold over time. Even under heavy workloads, Jibril's resource usage remains predictable and deterministic.

📊 Cache Categories

Jibril uses different cache types for different system resources:

🔄 Task-Related Caches

Store process and execution data.

• rec-tasks - Recent task history
• tasks - Active OS processes
• cmds - Command lines
• args - Command arguments

📁 File-Related Caches

Store file access and correlation data.

• files - Accessed files
• dirs - Accessed directories
• bases - Base paths
• task-file - Task → File mapping
• file-task - File → Task mapping
• task-ref - Task references

🌐 Network Flow Caches

Store network communication data.

• flows - Network flows
• task-flow - Task → Flow mapping
• flow-task - Flow → Task mapping
• flow-ref - Flow references

⚙️ Configuration Examples

Choose a configuration based on your environment's workload and resource constraints.

🎯 1. Default (Balanced)

Good for most use cases with moderate activity.

caches:
  rec-tasks: 32
  tasks: 64
  cmds: 32
  args: 32
  files: 32
  dirs: 16
  bases: 32
  task-file: 32
  file-task: 32
  task-ref: 32
  flows: 32
  task-flow: 32
  flow-task: 32
  flow-ref: 32

Characteristics:

• 📊 Balanced memory usage for most workloads
• ✅ Handles moderate concurrent processes
• ✅ Suitable for typical container and server workloads
• ✅ This is the default configuration from standalone.yaml
• ⚠️ Heavy workloads might need smaller cadences to avoid miss detections
• ⚠️ Small containers may be susceptible to Out of Memory (OOM) errors

📱 2. Small Devices

Minimized memory footprint for resource-constrained environments.

caches:
  rec-tasks: 16
  tasks: 32
  cmds: 16
  args: 16
  files: 16
  dirs: 4
  bases: 8
  task-file: 256
  file-task: 256
  task-ref: 256
  flows: 64
  task-flow: 64
  flow-task: 64
  flow-ref: 64

Characteristics:

• 📉 Minimal memory usage for small containers
• ✅ Suitable for embedded systems (like IoT and edge devices)
• ⚠️ Raising cadences might help avoid miss-detections (higher CPU usage)
• ⚠️ Acceptable if other detection recipes compensate for the missed detections

🔍 3. Comprehensive Detection

Larger correlation caches for better pattern matching.

caches:
  rec-tasks: 32
  tasks: 64
  cmds: 32
  args: 32
  files: 32
  dirs: 16
  bases: 32
  task-file: 512
  file-task: 512
  task-ref: 512
  flows: 128
  task-flow: 128
  flow-task: 128
  flow-ref: 128

Characteristics:

• 📈 Bigger memory usage for specific environments
• ✅ Allows for bigger cadences (less CPU usage) without miss-detections
• ✅ Better historical event correlation and event context retention
• ✅ Enhanced network flow tracking and context retention
• ✅ Reduced chance of missed detections
• 🎯 Recommended for big and complex workloads

🚀 4. Heavy I/O

Maximum caches for high-volume environments.

caches:
  rec-tasks: 64
  tasks: 128
  cmds: 64
  args: 64
  files: 64
  dirs: 32
  bases: 64
  task-file: 1024
  file-task: 1024
  task-ref: 1024
  flows: 256
  task-flow: 256
  flow-task: 256
  flow-ref: 256

Characteristics:

• 🎯 Too many opened files and/or sockets might face different needs
• ✅ Bigger task and files caches for better context retention
• ✅ Bigger network flow caches for better network context retention

🔍 Cache Details

rec-tasks

Recent Tasks Cache - Stores short-term historical data about recently completed processes for temporal analysis.

tasks

Active Tasks Cache - Stores information about currently running OS processes observed by Jibril.

cmds / args

Command & Arguments Caches - Store command lines and their arguments for running processes.

files, dirs, bases

File System Caches - Track accessed files, directories, and base paths.

task-file, file-task

Correlation Caches - Bidirectional mapping between tasks and files they access.

flows, task-flow, flow-task

Network Flow Caches - Track network connections and their relationships to processes.

📏 Sizing Guidelines

🎯 How to Size Caches

Consider these factors:

1. Concurrent Process Count

   • Systems with many processes → increase tasks, rec-tasks

2. File I/O Volume

   • High file creation/modification rate → increase files, task-file, file-task

3. Network Activity

   • Many concurrent connections → increase flows, task-flow, flow-task

4. Available Memory

   • Limited RAM → use small device profile
   • Ample RAM → use heavy I/O profile

Jibril's memory consumption will be a mix of all caches plus the amount of detection recipes enabled. Start by using the default configuration and adjust based on your needs.

⚠️ When Caches Overflow

🚨 Symptoms

• Missed file access detections
• Incomplete process context (files, flows, etc.)
• Lost process correlation
• Warning messages in logs

✅ Solutions

• Increase relevant cache sizes
• Adjust cadence intervals
• Enable only necessary detection recipes
• Monitor system activity patterns

🔧 Tuning Process

Step-by-step approach:

1. Start with defaults
   Use the average configuration initially.
   Enable only a few detection recipes to start with.
   
2. Monitor behavior
   Check logs for cache overflow warnings.
   Check CPU usage and memory consumption.
3. Adjust incrementally
   Increase specific caches by 50-100%.
   Enable more detection recipes as needed.
4. Test under load
   Verify performance during peak activity.
   Check for missed detections.
5. Fine-tune
   Balance memory usage with detection accuracy.
   

⚠️ Avoid

• ❌ Setting all caches to maximum
• ❌ Using small device config on production servers
• ❌ Ignoring cache overflow warnings
• ❌ Changing all cache sizes simultaneously
• ❌ Forgetting to test after changes
• ❌ Over-provisioning without monitoring

📊 Memory Impact Reference

Approximate memory usage per cache profile:

Profile	Memory Usage	Use Case
Small Devices	From 50 to 250 MB	IoT, embedded, edge devices
Default (Balanced)	From 256 to 1024 MB	Standard servers, VMs, containers
Comprehensive Detection	From 512 to 2048 MB	Production security monitoring
Heavy I/O	From 1024 to 4096 MB	Databases, file servers, critical infrastructure

Note: Total Jibril memory usage includes eBPF maps, detection logic, and other overhead. Cache sizes are just one component of the whole equation.

🚀 Next Steps

⚙️

Configuration

Back to config guide

⏱️

Cadence Tuning

Optimize detection intervals

🌐

Network Policy

Traffic control configuration

🎨

Customization

Create custom detections