If you are a security operations manager or lead architect who recently championed a migration from proprietary commercial SIEM platforms (like Splunk or Microsoft Sentinel) to open-source alternatives, you likely celebrated your initial budget presentation. By choosing Wazuh or Security Onion, you promised executive leadership that you would eliminate eye-watering annual ingestion licensing fees—saving the organization hundreds of thousands of dollars while gaining complete sovereignty over your security data.

Three months later, the CFO is standing in your office with an urgent cloud infrastructure invoice. Your AWS or SAN storage bill has quadrupled, your Elasticsearch or OpenSearch cluster is in a constant state of red alert with out-of-disk errors, and your storage arrays are groaning under 25 terabytes of uncompressed security logs. Your storage engineering team is demanding emergency budget allocations for high-performance NVMe drives just to keep your SIEM cluster from seizing up completely.

You did not escape the cost of enterprise security; you simply traded licensing software fees for runaway hardware and storage infrastructure costs. Both Wazuh and Security Onion rely on the ELK / OpenSearch paradigm for data storage, indexing, and retrieval. If you ingest security telemetry without strict index lifecycle management, intelligent field pruning, and cold tiering, your open-source SIEM will burn through capital faster than commercial SaaS.

Why Do Open-Source SIEM Clusters Bleed Cash on Storage Hardware?

To understand why storage costs spiral out of control, you must examine how the underlying search engine—Apache Lucene, which powers both Elasticsearch and OpenSearch—handles log ingestion. When your security agents stream logs into the cluster, Lucene does not simply write the raw text to a disk file. It builds an inverted index for every single field in the log message.

Consider a standard Windows Security Event log (such as Event ID 4688: Process Creation) or a Zeek connection log. A single raw JSON event might measure 800 bytes. However, when Elasticsearch ingests that event, it creates inverted index terms for the process path, command line arguments, parent process, user SID, hash values, timestamps, and network sockets. It also duplicates the raw data inside the _source field and generates forward doc-values for aggregation. By the time the event is fully indexed and replicated across cluster nodes for high availability, that 800-byte log consumes over 2.4 kilobytes of high-speed NVMe storage—a 300% storage inflation ratio!

In Security Onion, the storage inflation is compounded tenfold because Security Onion ingests full network metadata (Zeek logs, Suricata alerts) and optional Full Packet Capture (Stenographer). Storing full network packets at 500 Mbps consumes 5.4 terabytes of raw disk space every single day. Without ruthless tiering, your storage budget collapses within weeks.

Forensic Comparison: Wazuh Indexer vs Security Onion Storage Footprint

While both platforms deliver extraordinary threat detection capabilities, their architectural designs result in radically different infrastructure cost profiles.

The Three Traps That Inflate SIEM Storage Budgets

If your open-source SIEM is consuming disk space faster than projected, look for these three common configuration mistakes:

1. Blind Ingestion of Noisy Windows Event Channels

In Windows environments, standard operating systems generate millions of non-security events every hour. The most notorious culprits are Event ID 4662 (Operation performed on Active Directory object), Event ID 5156 (Windows Filtering Platform permitted connection), and noisy Sysmon network flow events (Event ID 3). If you blindly forward these channels to your SIEM without pre-filtering on the agent, you are spending thousands of dollars indexing routine internal network noise that will never be queried during a forensic investigation.

2. The Unbounded Primary Shard Trap

In Elasticsearch and OpenSearch, data is partitioned into primary shards. Each shard has an operational memory overhead. If your cluster creates daily indices with multiple shards, and keeps 90 days of indices active on the “Hot” tier without merging or shrinking, you end up with hundreds of active shards. The cluster’s JVM heap memory becomes completely consumed simply tracking shard metadata, leaving insufficient memory for search queries and triggering Out-Of-Memory crashes.

3. Keeping 365 Days of Data on Premium NVMe Storage

Many compliance frameworks (such as PCI-DSS, SOC 2, or HIPAA) require organizations to retain security logs for one full year. However, compliance frameworks do not require all twelve months of logs to be searchable in sub-seconds on high-speed NVMe drives! In 99% of SOC environments, 95% of all analyst queries target logs that are less than 14 days old. Keeping months 2 through 12 on hot NVMe storage instead of migrating them to compressed cold object storage (like AWS S3 or MinIO) multiplies your storage infrastructure bill by a factor of eight.

The 4:00 PM Actionable Cost Reduction Playbook

To cut your open-source SIEM storage consumption by 60% and eliminate out-of-disk crashes today, execute this verified optimization sequence:

1. Implement Agent-Side Log Filtering: Never send raw, unfiltered logs to your cluster. In your Wazuh agent configuration (ossec.conf) or Windows Event Forwarding XML subscription, filter out high-volume, low-value event IDs before they leave the host:
   <localfile>
     <location>Security</location>
     <log_format>eventchannel</log_format>
     <query>Event/System[EventID != 5156 and EventID != 5158]</query>
   </localfile>
   This single edit typically slashes Windows ingestion volume by 45%.
2. Configure Index Lifecycle Management (ILM / ISM): In Wazuh Indexer or Security Onion, implement an Index State Management (ISM) policy that transitions indices through strict lifecycle stages:
   • Hot Tier (0–14 Days): High-performance NVMe storage, 1 primary shard, 1 replica.
   • Warm Tier (15–30 Days): Force-merge index to 1 segment, disable replica shards, compress with LZ4.
   • Cold / Frozen Tier (31–365 Days): Export snapshot to S3 / MinIO object storage and delete index from cluster.
3. Disable Dynamic Field Indexing: In your index templates, set dynamic: false for nested objects that do not need to be individually searchable. Keep the raw data in _source for forensic export, but stop building inverted index terms for transient payload metadata.
4. Cap Security Onion PCAP Retention to 72 Hours: If running Security Onion with Stenographer, cap your full packet capture buffer to 72 hours. PCAP is vital for verifying immediate zero-day exploits, but storing raw PCAP beyond 3 days is financially unsustainable in high-throughput environments.