Configure Filtering of Flow Spans

Block: filter.source/filter.destination/filter.network/filter.flow

Flow filtering allows you to include or exclude network flows based on various criteria. This reduces data volume and focuses on relevant traffic.

Mermin supports filtering flows by:

• Source/destination IP addresses and ports

• Network protocols and interface names

• TCP flags, ICMP types

• Connection states

Filter option names are derived directly from FlowSpan attribute names defined in the semantic conventions and can be referenced easily in the attributes reference. The attribute's dot notation is converted to underscores (e.g., flow.tcp.flags.tags becomes tcp_flags_tags). This 1:1 mapping ensures consistency and makes it easy to identify which attribute each filter targets.

Configuration

A full configuration example can be found in the Default Configuration.

filter.source and filter.destination blocks

The filters apply to the source/destination combination of the address and port in the flow span. Filter is applied at the "Flow Producer" stage (architecture), which can help reduce resource usage in subsequent stages.

• address attribute

  Filter by IP address.

  Type: Pattern matcher object

  Default: {}

  Supported values: IP or CIDR notation (10.0.0.0/8, 10.0.0.1)

  Example: Include only RFC1918, but exclude 10.0.0.0/24, and 10.0.2.1

  filter "source" {
    address = {
      match     = ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]
      not_match = ["10.0.0.0/24", "10.0.2.1"]
    }
  }

• port attribute

  Filter by port.

  Type: Pattern matcher object

  Default: {}

  Supported values: Port or port range as a string (443, 8000-9000)

  Examples:

  • Include flows with only 443 (HTTPS) destination port

    filter "destination" {
      port = {
        match = ["443"]
      }
    }

  • Include flows with only Linux ephemeral source ports

    filter "source" {
      port = {
        match = ["32000-60999"]
      }
    }

Notes

The result of the filter.source/filter.destination inclusion/exclusion is combined with an "AND" condition, meaning it is very easy to accidentally exclude flows you want to observe. For example:

• Matching only private subnets will filter out any flow originating from public subnets. The configuration:

  filter "source" {
    address = {
      match = ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]
    }
  }

  Flows:

  [
    {...,"source.address": "10.0.0.2", "destination.address": "10.0.0.3", ...}, // included
    {...,"source.address": "10.0.0.3", "destination.address": "10.0.0.2", ...}, // included
    {...,"source.address": "10.0.0.2", "destination.address": "92.1.1.1", ...}, // included
    {..., "source.address": "92.1.1.1", "destination.address": "10.0.0.2", ...}, // EXCLUDED
    ...
  ]

• Matching the same port in the source and destination filters will filter out almost all flows. Although, theoretically, source and destination ports can be the same (e.g., old DNS servers), it is relatively uncommon to see the same source and destination port. The configuration:

  filter "source" {
    port = {
      match = ["53", "443"]
    }
  }
  
  filter "destination" {
    port = {
      match = ["53", "443"]
    }
  }

  Flows:

  [
    {...,"source.port": "33868", "destination.port": "443", ...}, // excluded
    {...,"source.port": "443", "destination.port": "33868", ...}, // excluded
    {...,"source.port": "53", "destination.port": "53", ...}, // included (uncommon)
    {...,"source.port": "53", "destination.port": "53", ...}, // included (uncommon)
    ...
  ]

filter.network block

The filter applies to various network attributes in the flow span, such as transport protocol, interface, and others. Filter is applied at the "Flow Producer" stage (architecture), which can help reduce resource usage in subsequent stages.

• transport attribute

  Filter by transport protocol.

  Type: Pattern matcher object

  Default: {}

  Supported values: tcp, udp, icmp, icmpv6 (supports globs)

  Examples:

  • Include only TCP and UDP traffic:

    filter "network" {
      transport = {
        match = ["tcp", "udp"]
      }
    }

  • Exclude ICMP:

    filter "network" {
      transport = {
        not_match = ["icmp"]
      }
    }

• type attribute

  Filter by IP version.

  Type: Pattern matcher object

  Default: {}

  Supported values: ipv4, ipv6 (supports globs)

  Example: Include only IPv4 traffic:

  filter "network" {
    type = {
      match = ["ipv4"]
    }
  }

• interface_name attribute

  Filter by network interface name.

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid interface name (supports globs)

  Examples:

  • Include only interfaces matching eth* or enp* (eth0, eth1, enp0s3, enp8s0f0):

    filter "network" {
      interface_name = {
        match = ["eth*", "enp*"]
      }
    }

  • Exclude interfaces matching docker* (docker0, docker1, docker-wec2323):

    filter "network" {
      interface_name = {
        not_match = ["docker*"]
      }
    }

• interface_index attribute

  Filter by network interface index.

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid interface index or interface index range as a string (0, 1-27)

  Examples:

  • Exclude only interface index 2:

    filter "network" {
      interface_index = {
        not_match = ["2"]
      }
    }

  • Include only interfaces 1 to 27 and 30:

    filter "network" {
      interface_index = {
        match = ["1-27", "30"]
      }
    }

• interface_mac attribute

  Filter by network interface MAC address.

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid MAC address (supports globs)

  Example: Exclude a specific MAC address:

  filter "network" {
    interface_mac = {
      not_match = ["00:11:22:33:44:55"]
    }
  }

filter.flow block

The filter applies to various flow attributes in the flow span, such as connection state, TCP flags and others. Filter is applied at the "Flow Producer" stage (architecture), which can help reduce resource usage in subsequent stages.

• connection_state attribute

  Filter by TCP connection state.

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid connection state, established, syn_sent, syn_received, fin_wait, close_wait, closing, last_ack, time_wait, closed (supports globs)

  Example: Include only established connections:

  filter "flow" {
    connection_state = {
      match = ["established"]
    }
  }

• tcp_flags_tags attribute

  Filter by TCP flags.

  Type: Pattern matcher object

  Default: {}

  Supported values: SYN, ACK, FIN, RST, PSH, URG (supports globs), case insensitive.

  Example: Include only flows with SYN flag:

  filter "flow" {
    tcp_flags_tags = {
      match = ["SYN"]
    }
  }

• ip_dscp_name attribute

  Filter flows based on the DSCP (Differentiated Services Code Point) names.

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid DSCP name (supports globs)

  Examples:

  • Include only low-latency data (AF21)

    filter "flow" {
      ip_dscp_name = { match = ["AF21"] }
    }

  • Exclude multimedia conferencing (AF41, AF42, AF43)

    filter "flow" {
      ip_dscp_name = { match = ["AF4{1,2,3}"] }
    }

• ip_ecn_name attribute

  Filter flows based on ECN (Explicit Congestion Notification) values.

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid ECN value (supports globs)

  Examples:

  • Include only ECN-capable transport (ECT0, ECT1)

    filter "flow" {
      ip_ecn_name = { match = ["ECT?"] }
    }

  • Exclude congestion encountered (CE)

    filter "flow" {
      ip_ecn_name = { not_match = ["CE"] }
    }

• ip_ttl attribute

  Filter flows based on the IP TTL (Time To Live) values.

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid TTL or TTL range as a string (1, 64-184)

  Examples:

  • Include only packets with the TTL 1 and 64 to 128

    filter "flow" {
      ip_ttl = { match = ["1", "64-184"] }
    }

  • Exclude packets with the TTL 64

    filter "flow" {
      ip_ttl = { not_match = ["64"] }
    }

• ip_flow_label attribute

  Filter flows based on IPv6 flow labels.

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid flow label or label range (2145, 12345-12545)

  Examples:

  • Include only flows with label 12345

    filter "flow" {
      ip_flow_label = { match = ["12345"] }
    }

  • Exclude flows with labels in a range

    filter "flow" {
      ip_flow_label = { not_match = ["12345-12545"] }
    }

• icmp_type_name attribute

  Filter flows based on ICMP type names (converted to a snake case).

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid ICMP type name (supports globs)

  Examples:

  • Include only echo requests

    filter "flow" {
      icmp_type_name = { match = ["echo_request"] }
    }

  • Exclude destination unreachable

    filter "flow" {
      icmp_type_name = { not_match = ["destination_unreachable"] }
    }

• icmp_code_name attribute

  Filter flows based on ICMP codes.

  Type: Pattern matcher object

  Default: {}

  Supported values: Any valid ICMP code or code range as a string (13, 0-8)

  Examples:

  • Include codes from 0 to 8 and 13

    filter "flow" {
      icmp_code_name = { match = ["0-8", "13"] }
    }

  • Exclude code 3

    filter "flow" {
      icmp_code_name = { not_match = ["3"] }
    }

Object Types

Pattern Matcher Object

• match attribute

  Include flows matching the pattern

  Type: List of strings

  Default: [] (empty list, include all)

• not_match attribute

  Exclude flows matching the pattern

  Type: List of strings

  Default: [] (empty list, exclude none)

Matcher value types

Although matcher patterns are strings only, there are multiple types that are supported:

• IP addresses and CIDRs, used in the address arguments, for example:

  • 10.0.0.0/8: CIDR notation, matches the subnet

  • 10.0.0.1: IP address, equals the 10.0.0.1/32 subnet

• Ranges, used in the port, interface_index, ip_ttl, ip_flow_label, icmp_code_name arguments, support ranges. For example:

  • 80: Single port

  • 8000-8999: Port range

  • 0: Single interface index

  • 0-22: Interface index range

  • 64: Single TTL

  • 64-128: TTL range

  • 12345: Single Flow Label

  • 12345-12445: Flow Label range

  • 0: Single ICMP code

  • 0-8: ICMP code range

• Arbitrary strings, used in more generic arguments like transport names, interface names, and others. Supports globs. For example:

  • tcp: Protocol names

  • close_wait: Connection states

  • eth*: Interface names

Common Filtering Scenarios

HTTP/HTTPS Only

The following configuration captures flows with HTTP/HTTPS destination.

filter "destination" {
  port = {
    match = ["80", "443"]
  }
}

Example flows:

[
  {...,"source.port": "33567", "destination.port": "443", ...}, // included
  {...,"source.port": "443", "destination.port": "33567", ...}, // EXCLUDED
  {...,"source.port": "43567", "destination.port": "80", ...}, // included
  {...,"source.port": "80", "destination.port": "43567", ...}, // EXCLUDED
  {...,"source.port": "53567", "destination.port": "8080", ...}, // EXCLUDED
  {...,"source.port": "8080", "destination.port": "53567", ...}, // EXCLUDED
  ...
]

Exclude Internal Traffic

The following configuration captures flows originating from non-local addresses:

filter "source" {
  address = {
    not_match = ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]
  }
}

[
  {...,"source.address": "10.0.0.2", "destination.address": "10.0.0.3", ...}, // EXCLUDED
  {...,"source.address": "10.0.0.3", "destination.address": "10.0.0.2", ...}, // EXCLUDED
  {...,"source.address": "10.0.0.2", "destination.address": "92.1.1.1", ...}, // EXCLUDED
  {..., "source.address": "92.1.1.1", "destination.address": "10.0.0.2", ...}, // included
  {..., "source.address": "10.0.0.2", "destination.address": "92.1.1.1", ...}, // EXCLUDED
  ...
]

TCP Only, Established Connections

The following configuration captures flows for established TCP connections.

filter "network" {
  transport = {
    match = ["tcp"]
  }
}

filter "flow" {
  connection_state = {
    match = ["established"]
  }
}

Best Practices

1. Start permissive: Begin with no filters, add as needed

2. Monitor impact: Check flow reduction with metrics

3. Test incrementally: Add one filter at a time

4. Document rationale: Comment why filters are applied

5. Use match/not_match carefully: Match patterns can hide important traffic

Next Steps

Configure More

1. Tune Flow Generation: Configure timeouts and thresholds

2. Set Up OTLP Export: Send filtered flows to your backend

Examples

Need Help?

• Troubleshoot Missing Flows: Diagnose over-filtering

• GitHub Discussions: Share filter configurations