Log Fan-Out to Multiple SIEMs: How to Dual-Ship Logs Without Doubling the Risk

Organizations often need to send the same security events to multiple security information and event management (SIEM) platforms. Common scenarios include SIEM migrations, managed security service provider (MSSP) integrations, long-term compliance archiving, and security operations centers (SOCs) operating across different regions.

Log fan-out, also known as dual-shipping, allows a single collection point to forward events to two or more destinations. With NXLog Agent, organizations can route data to multiple outputs from one input, apply destination-specific transformations, and manage how the pipeline behaves when a destination becomes unavailable.

The challenge is to maintain reliable delivery without creating unnecessary ingestion costs, inconsistent event counts, or additional operational complexity. This guide explains the main fan-out architectures, demonstrates practical NXLog configuration patterns, and outlines how to validate delivery across multiple SIEMs.

What Is Log Fan-Out?

Log fan-out is the process of sending the same collected event to multiple destinations in parallel. Dual-shipping describes the specific case of forwarding data to two destinations. The terms mirroring and teeing are also commonly used for similar architectures.

One approach is to deploy separate agents for each SIEM on every host. However, this introduces additional configuration lifecycles, certificate management, CPU and memory consumption, and the possibility of mismatched event counts when the agents read their sources at different times.

NXLog solutions provide an alternative: a single agent can collect data once and route it to multiple outputs. Each output can apply its own processing logic, allowing organizations to integrate destinations with different data formats and ingestion requirements.

When Should You Use Dual-Shipping?

1. SIEM Migration

Running two SIEM platforms in parallel allows security teams to validate the new platform against live telemetry while keeping existing detections operational. This can reveal parsing problems, missing fields, and differences in detection behavior before the final cutover.

However, forwarding all raw events to both platforms can increase ingestion costs. Microsoft’s guidance for deploying Microsoft Sentinel alongside an existing SIEM describes the operational and financial drawbacks of prolonged parallel deployments. Where appropriate, forwarding alerts rather than duplicating raw logs can reduce the cost of running both systems.

2. MSSP or MDR and an Internal SOC

A managed security service provider (MSSP) or managed detection and response (MDR) provider may require raw telemetry for its detection and investigation workflows. At the same time, an internal SOC may need its own copy for threat hunting, incident response, compliance, and operational independence.

Fan-out enables both teams to receive the relevant data without deploying separate collection agents for each destination.

3. SIEM and Long-Term Archive

A SIEM can retain frequently accessed data for detection and investigation, while an object-storage platform or lower-cost log repository stores data for longer periods. This approach separates data according to its purpose instead of treating every event as equally expensive to ingest and analyze.

4. Regional or Business-Unit SIEMs

Large organizations may operate a central SOC alongside regional security teams. A shared source can feed multiple SIEMs while routing policies account for different operational responsibilities and data-residency requirements.

Before implementing dual-shipping, define which events each destination needs, how long parallel operation will last, and whether full duplication is necessary. Filtering or selectively routing events can help control costs. For additional guidance, see our article on pre-SIEM log filtering and log reduction.

Comparing Log Fan-Out Architectures

For most endpoint and server deployments, routing at the agent is a practical starting point. For network appliances and other sources that cannot run an agent, a relay tier can perform the distribution.

Configuring Log Fan-Out with NXLog Agent

The following examples illustrate a Windows Security event stream routed to Splunk through HTTP Event Collector (HEC) and to Microsoft Sentinel through the Logs Ingestion API. The configuration is illustrative: authentication values, endpoint URLs, data collection rules, and output-specific field mappings must be adapted to your environment and verified against the deployed NXLog Agent version.

Pattern 1: Send One Input to Multiple Outputs

In NXLog Agent, a route connects input modules to output modules through the Path directive. Multiple outputs can be specified on the same route, separated by commas, so the outputs receive events in parallel.

Each destination can transform its copy independently. For example, Splunk HEC expects an event envelope, while Microsoft Sentinel's Logs Ingestion API expects records that match the schema defined by the destination's data collection rule (DCR).

<Extension json>
    Module xm_json
</Extension>

<Input windows_security>
    Module  im_msvistalog
    Channel Security
</Input>

<Output splunk_hec>
    Module om_http
    URL    https://splunk.example.com:8088/services/collector/event
    AddHeader Authorization: Splunk <HEC_TOKEN>
    HTTPSCAFile %CERTDIR%/splunk-ca.pem

    <Exec>
        # Build the destination-specific Splunk HEC event envelope.
        # Map the required host, time, source, and event fields.
        # Serialize the completed record as JSON.
    </Exec>
</Output>

<Output sentinel>
    Module         om_azuremonitor
    ClientId       <ENTRA_APP_ID>
    TenantId       <ENTRA_TENANT_ID>
    ClientSecret   <ENTRA_CLIENT_SECRET>
    URL            https://<DCE_NAME>.ingest.monitor.azure.com
    DcrImmutableId <DCR_IMMUTABLE_ID>
    StreamName     Custom-WindowsSecurity_CL

    <Exec>
        # Map event fields to columns declared in the DCR stream.
        # Serialize the record to JSON and assign it to $raw_event.
    </Exec>
</Output>

<Route fanout>
    Path windows_security => splunk_hec, sentinel
</Route>

The example shows the routing structure rather than a complete, production-ready configuration. The output-specific transformation logic must be implemented using the correct NXLog procedures and fields for each destination.

For implementation details, consult the official NXLog documentation for the route settings, the HTTP output module, and the Microsoft Azure Monitor output module. The integration guides for Splunk and Microsoft Sentinel provide additional setup guidance.

Pattern 2: Send a Filtered Subset to the Second SIEM

Not every destination needs every event. During a SIEM evaluation, for example, a new platform may initially require only authentication and privileged-account activity. Selective routing can reduce duplicate ingestion while preserving the complete stream in the primary SIEM.

NXLog supports output-specific processing, so a filter applied inside one output's Exec block does not automatically affect the other output. The following is a simplified illustration of the selection logic:

<Output sentinel>
    Module om_azuremonitor

    <Exec>
        if ($EventID == 4624 or $EventID == 4625 or
            $EventID == 4672 or $EventID == 4720 or
            $EventID == 4728)
        {
            # Map the selected event to the Sentinel DCR schema.
            # Serialize the record to JSON and assign it to $raw_event.
        }
        else
        {
            drop();
        }
    </Exec>
</Output>

This example selects Windows Security events associated with logons, special privileges, account creation, and group membership changes. The primary SIEM can continue receiving the full Security channel while the second destination receives only the selected subset.

Before deploying a filter, validate that the selected events support the intended detection scenarios and that the destination schema is correct. For more information about processing and routing telemetry before SIEM ingestion, read our guide to reducing SIEM ingestion without losing detections.

Managing Destination Outages: Flow Control and Queues

One of the most important fan-out design decisions is what happens when one destination becomes unavailable.

With flow control enabled, a blocked output can cause the route to pause upstream processing until the output catches up. In a route with multiple outputs, this can delay delivery to a healthy SIEM as well. For example, an extended Sentinel ingestion problem could affect the delivery of events to Splunk if the shared route becomes blocked.

The correct configuration depends on the source, the importance of each destination, and the organization's tolerance for delayed or missing events.

Option A: Keep Flow Control Enabled

For pausable sources such as files and Windows Event Log, keeping flow control enabled allows the agent to slow collection when a destination cannot keep up. This can preserve events while the destination recovers, although delivery to the healthy destination may also be delayed.

This is generally the safer approach when preserving events is more important than maintaining uninterrupted real-time delivery to every destination. Source retention and queue capacity must still be sufficient for the expected outage.

Option B: Disable Flow Control for a Secondary Output

If the primary SIEM must continue receiving events in real time, flow control can be disabled on a secondary output that is allowed to lose data after its queue reaches capacity. The secondary output can buffer events during a short interruption, while the primary output retains its own flow-control protection.

<Output splunk_hec>
    Module om_http
    # Configure the production destination.
    # Flow control remains enabled.
</Output>

<Output sentinel>
    Module         om_azuremonitor
    # Configure the secondary destination.
    FlowControl    FALSE
    LogqueueSize   16777216
    PersistLogqueue TRUE
</Output>

<Route fanout>
    Path windows_security => splunk_hec, sentinel
</Route>

This illustrative configuration uses a 16 MiB queue for the secondary output. It is not a universal capacity recommendation. The queue must be sized according to event volume, average event size, and the interruption duration the system is expected to absorb.

A useful starting estimate is:

Required queue capacity ≈ events per second × outage duration in seconds × average serialized event size.

Allow additional capacity for bursts and overhead, and validate actual queue behavior under load. A small queue may protect against brief network interruptions or throttling but cannot guarantee delivery during a prolonged outage.

PersistLogqueue TRUE enables persistence of the output queue on disk while the agent runs. Persistent queues introduce additional disk I/O and should be tested under realistic conditions. Queue persistence is not, by itself, a guarantee of end-to-end delivery.

Disabling flow control on the input is a different and broader decision: it can remove backpressure protection for both destinations. For pausable sources, configure flow control deliberately at the relevant output rather than disabling it globally without understanding the consequences.

For more detail, see the NXLog documentation on buffering and flow control and configuring buffering. You can also read Softprom's article on offline buffering for log shippers.

Pattern 3: Fan Out Logs at a Relay Tier

Network devices such as firewalls, switches, and appliances often cannot run a log collection agent. Instead, they send syslog messages to a relay, which distributes the events to the required destinations. Endpoint agents can also forward data to the same relay when a centralized architecture is preferred.

A relay allows endpoint configurations to remain simpler: agents can send events to one controlled destination without needing to know the details of every downstream SIEM.

<Extension syslog>
    Module xm_syslog
</Extension>

<Input network_syslog>
    Module      im_udp
    ListenAddr  0.0.0.0:514
    FlowControl FALSE
    Exec        parse_syslog();
</Input>

<Input agents>
    Module     im_tcp
    ListenAddr 0.0.0.0:1514
    Exec       parse_json();
</Input>

# Define splunk_hec and sentinel outputs for the
# appropriate event schemas and authentication settings.

<Route relay_fanout>
    Path network_syslog, agents => splunk_hec, sentinel
</Route>

This is a simplified architectural example, not a complete deployment template. The input module, transport, parsing logic, and destination transformations must match the data sources and security requirements in your environment.

For production deployments, consider TLS-protected transport from endpoints to the relay, high availability, network segmentation, access controls, and sufficient processing and queue capacity. UDP sources cannot be paused in the same way as file-based sources, so packet loss and overload require specific attention.

When sources produce different event schemas, configure destination-specific processing for each source type rather than assuming that one mapping will work for all telemetry.

For larger deployments, review the NXLog guidance on distributing NXLog Agent load and the available TLS/SSL input module.

Deploying Fan-Out Across a Fleet with NXLog Platform

Managing individual configuration files may be practical for a small number of systems. Across hundreds or thousands of agents, centralized configuration management makes it easier to deploy changes consistently, monitor results, and roll back if necessary.

NXLog Platform supports centralized agent configuration management. Administrators can define a configuration, assign it to selected agents, track versions, compare changes, and restore a previous version when required.

A controlled rollout can follow these steps:

1. Duplicate the existing production configuration and add the secondary output.
2. Update the route to include both destinations.
3. Assign the revised configuration to a small pilot group.
4. Compare event counts, inspect destination-side parsing, and monitor queue behavior.
5. Expand the rollout after validating delivery and detection coverage.
6. At the end of a migration, remove the legacy destination through a controlled configuration change.

Where automation is required, the NXLog Agent Management API can support configuration workflows integrated with version control and deployment processes.

To understand how to measure and inspect data flowing through a telemetry pipeline, see Softprom's guide to visualizing telemetry data flow and volume with NXLog Platform.

How to Verify That Both SIEMs Receive the Same Events

During a migration, audit, or incident review, teams may need to demonstrate that both destinations received the expected data. A consistent validation process helps identify ingestion gaps and distinguish transport failures from parsing or schema problems.

1. Add a Traceable Event Identifier

Where the source provides a suitable identifier, include it in both destinations. For Windows Event Log, a combination of hostname, channel, and record number can help correlate corresponding records.

<Input windows_security>
    Module  im_msvistalog
    Channel Security
    Exec    $FanoutId = $Hostname + ":" + $Channel + ":" + string($RecordNumber);
</Input>

Map this field into each destination's schema. For Microsoft Sentinel, the field must be declared in the relevant DCR stream if it is to be ingested into the destination table.

2. Compare Event Counts and Identifiers

Compare event counts by host and time interval in each SIEM, then reconcile records using the traceable identifier. Persistent gaps affecting one host may indicate a local collection or parsing issue. Gaps affecting many hosts simultaneously may point to a destination or transport problem.

Account for differences in ingestion latency, timestamp handling, retries, and destination-side filtering. Equal counts alone do not prove that the same events were received.

3. Monitor NXLog Agent's Internal Logs

Monitor the agent's internal logs for output errors, queue saturation, and other warnings. Route relevant internal messages into an operational monitoring workflow so that delivery problems are visible before they become prolonged data gaps.

See the NXLog documentation for the Internal input module.

4. Test Destination Outages

Simulate a blocked destination in a test environment and generate representative event volume. Confirm that the healthy destination continues to receive events according to the chosen flow-control strategy, observe queue growth, and verify that any backlog recovers when the destination becomes available.

The NXLog Blocker output module can help simulate a blocked output during testing. Validate the queue limits and recovery behavior before deploying the configuration to production.

Common Mistakes When Dual-Shipping Logs

• Reusing one data format for every destination. Different SIEMs may require different envelopes, field names, timestamps, and schemas. Transform each output independently.
• Overlooking certificate rotation. Track the certificates, credentials, and expiration dates associated with each destination.
• Failing to alert on queue saturation. If a secondary output is configured to discard events after its queue fills, alert on queue-related warnings before data loss occurs.
• Disabling flow control without assessing the impact. Understand which output can lose data and whether the source can safely pause.
• Using inconsistent timestamps. Explicitly map event time for each destination to make comparisons meaningful.
• Ignoring destination-specific filtering. Confirm that each SIEM receives the events required for its detection, investigation, and compliance responsibilities.
• Allowing a temporary migration to continue indefinitely. Set a cutover date when the second SIEM is temporary. If the second destination is a permanent archive or SOC, document that architecture and its ownership.

Conclusion: Build a Reliable Multi-SIEM Logging Pipeline

Log fan-out allows organizations to distribute security telemetry to multiple destinations without deploying separate collection agents for every SIEM. NXLog Agent supports parallel routing through multiple outputs, destination-specific event processing, and configurable behavior when a destination becomes unavailable.

For endpoints and servers, fan-out can be implemented at the agent. For network devices and other sources that cannot run an agent, a relay tier can centralize distribution. NXLog Platform helps teams manage configurations across a fleet, while event identifiers, count comparisons, queue monitoring, and outage testing help validate delivery.

The right design depends on whether the priority is complete event preservation, uninterrupted delivery to the primary SIEM, lower ingestion costs, or a controlled migration. Define those requirements before rollout, test failure scenarios, and monitor the pipeline after deployment.

Softprom is an official NXLog distributor. We help organizations evaluate NXLog solutions, design log collection and routing architectures, and integrate telemetry with SIEM and security monitoring platforms. Contact Softprom to discuss NXLog for your environment.