--- title: Set up network flow data monitoring source: https://docs.newrelic.com/docs/network-performance-monitoring/setup-performance-monitoring/network-flow-monitoring --- You can use our guided install process to install the network flow monitoring agent, or install the agent manually. This doc covers prerequisites to start this install process and a step-by-step walk through of your install options. ## Prerequisites [#prerequisites] Before you can start, you'll need to [sign up for a New Relic account](https://newrelic.com/signup). If you choose to install the agent manually, you also need: > #### ⚠️ IMPORTANT > > #### Timezone requirement > > The server running `ktranslate` **must** be set to the **UTC** timezone. If a different timezone is configured, it can cause timestamp inconsistencies and prevent data from appearing correctly in New Relic. - A New Relic [account ID](https://docs.newrelic.com/docs/accounts/accounts-billing/account-setup/account-id). - A New Relic license key. **Docker prerequisites** We recommend using a Docker container to deploy the agent for network flow collection. To use it, you'll need: - [Docker](https://docs.docker.com/engine/install) installed in a Linux host - Ability to launch new containers via command line **Podman prerequisites** If you're using a Podman container to launch the agent, you need: - [Podman](https://podman.io/docs/installation) installed in a Linux host - Ability to launch new containers via command line **Linux host prerequisites** If you're using Linux to install the agent as a service, you'll need: - SSH access to the host - Access to install/remove applications and services - One of these supported operating systems: - CentOS 8 - Debian 12 (Bookworm) - Debian 11 (Bullseye) - Debian 10 (Buster) - RedHat Enterprise Linux 9 - Ubuntu 20.04 (Focal LTS) - Ubuntu 22.04 (Jammy LTS) - Ubuntu 23.04 (Lunar) - Ubuntu 24.04 (Noble LTS) **Network flow data devices prerequisites** - You must configure source devices to send flow data to the host running the network monitoring agent. Here's how to configure network flow export in some devices (this is not an all-inclusive list): - NetFlow data - [Cisco - IOS](https://www.cisco.com/c/en/us/td/docs/ios/netflow/configuration/guide/12_2sr/nf_12_2sr_book/cfg_nflow_data_expt.html#wp1087069) - [Cisco - Meraki](https://documentation.meraki.com/MX/Monitoring_and_Reporting/NetFlow_Overview) - [Cisco - NX-OS](https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/system_management/configuration/guide/b_Cisco_Nexus_9000_Series_NX-OS_System_Management_Configuration_Guide_7x/b_Cisco_Nexus_9000_Series_NX-OS_System_Management_Configuration_Guide_7x_chapter_011100.html#task_52D8A0952B06404197D2536B6E33EE80) - [Fortinet Fortigate](https://kb.fortinet.com/kb/documentLink.do?externalID=FD36460) - [Palo Alto - PAN-OS](https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-admin/monitoring/netflow-monitoring/configure-netflow-exports) - sFlow data - [F5 - BIG-IP](https://techdocs.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/bigip-external-monitoring-implementations-12-0-0/15.html) - jFlow data - [Juniper - Junos](https://www.juniper.net/documentation/us/en/software/junos/flow-monitoring/flow-monitoring.pdf) **Network security prerequisites** Check the [network security prerequisites for network flow](/install/npm). ## Supported types of network flow data [#supported-network-flow-data-types] Network flow monitoring supports the four primary types of network flow data and their derivatives. When running the agent, you can specify which major type you want to monitor using the `-nf.source` option. > #### 💡 TIP > > Collection of `NetFlow v5`, `NetFlow v9`, `sFlow`, and `IPFIX` templates can all be handled using `-nf.source.=auto` on a single agent. This is enabled as a default setting when using the `nr1.flow` argument at runtime. **Network flow types** | Network flow type | Enabled with `auto`? | `-nf.source` value | | ------------------ | -------------------- | -------------------- | | AppFlow | ✓ | `auto` \| `netflow5` | | Argus | ✓ | `auto` \| `netflow5` | | Cisco ASA | | `asa` | | Cisco NBAR | | `nbar` | | cflowd | ✓ | `auto` \| `netflow5` | | IPFIX | ✓ | `auto` \| `ipfix` | | J-Flow | ✓ | `auto` \| `netflow5` | | NetFlow v5 | ✓ | `auto` \| `netflow5` | | NetFlow v9 | ✓ | `auto` \| `netflow9` | | NetStream | ✓ | `auto` \| `netflow5` | | Palo Alto Networks | | `pan` | | RFlow | ✓ | `auto` \| `netflow5` | | sFlow | ✓ | `auto` \| `sflow` | ## When should you scale network flow collection? [#scale] When planning your strategy for collecting network flows at scale, the following items should be considered: - The `ktranslate` agent can only perform a single job at a time. An agent running SNMP collection cannot also listen for network flows. - The `ktranslate` agent can only listen for incoming network flows on a single listening port at a time (default: `9995`). If you require multiple ports to be open, each requires a dedicated agent, using the [-nf.port](https://docs.newrelic.com/docs/network-performance-monitoring/advanced/ktranslate-container-management/#container-runtime-options) configuration option at runtime to change the port. - The default `-nf.source=auto` configuration allows the container to listen for multiple standard flow types. If you need to parse other types of flow data like Cisco ASA, Cisco NBAR, or Palo Alto Networks templates, each will require their own agent. - New Relic recommends 1 CPU per 2000 flows-per-second (120,000 flows-per-minute). Deciding whether to horizontally scale multiple agents to distribute load or vertically scale a few larger agents to consolidate management is a matter of personal preference. ## Set up network flow data monitoring [#setup-network-flow-monitoring] For most use cases, we recommended our guided install to set up network flow data monitoring. If your set up is more advanced with custom configurations, then we'd recommend installing manually. **Guided installation setup** 1. Go to **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Add more data** 2. Scroll down until you see **Network** and click **Network Flows**. 3. Follow the steps outlined in the guided installation process. You can use Docker, Podman, or Linux. [Add network flow data](https://one.newrelic.com/nr1-core?state=01bc12b8-758e-cc28-a93f-a93970485d99) ![Network Flows guided setup](https://docs.newrelic.com/images/network_screenshot-full_network-flows-guided-install.webp "Network Flows guided setup") **[one.newrelic.com > All capabilities](https://one.newrelic.com/ll-capabilities) > Add more data > Network > Network Flows** to set up network flow data monitoring. 4. Investigate your network flow data in the New Relic network performance monitoring UI. **Manual container setup** Before reading about installing the network flow agent manually, consider using our guided install process to avoid errors: [Add network flow data](https://one.newrelic.com/nr1-core?state=01bc12b8-758e-cc28-a93f-a93970485d99) ### Docker container 1. On a Linux host with Docker installed, download the **ktranslate** image by running one of the following: - [Docker Hub](https://hub.docker.com/): ```shell docker pull kentik/ktranslate:v2 ``` - [Quay.io](https://quay.io/): ```shell docker pull quay.io/kentik/ktranslate:v2 ``` 2. Copy the `snmp-base.yaml` file to the local `$HOME` directory of your Docker user, and discard the container by running: ```shell cd ~ id=$(docker create kentik/ktranslate:v2) docker cp $id:/etc/ktranslate/snmp-base.yaml . docker rm -v $id ``` 3. Edit the `snmp-base.yaml` file, and add your network flow devices inside the `devices` dictionary key with the following structure: ```yaml devices: # This key and the corresponding 'device_name' # need to be unique for each device flow_device1: device_name: flow_device1 device_ip: x.x.x.x/yy flow_only: true # Optional user tags user_tags: owning_team: net_eng environment: production ``` > #### ⚠️ IMPORTANT > > If you're already monitoring SNMP data devices that will also send network flows, you'll want to ensure that the value for `device_name` is identical for both configuration files to ensure the flow telemetry is attributed to the right entity in the New Relic UI. 4. Run `ktranslate` to listen for network flows with the command: ```shell docker run -d --name ktranslate-$CONTAINER_SERVICE \ --restart unless-stopped --pull=always --net=host \ -v `pwd`/snmp-base.yaml:/snmp-base.yaml \ -e NEW_RELIC_API_KEY=$YOUR_NR_LICENSE_KEY \ kentik/ktranslate:v2 \ -snmp /snmp-base.yaml \ -nr_account_id=$YOUR_NR_ACCOUNT_ID \ -metrics=jchf \ -tee_logs=true \ # Use this field to create a unique value for `tags.container_service` inside of New Relic -service_name=$CONTAINER_SERVICE \ -flow_only=true \ nr1.flow ``` 5. Investigate your network flow data in the New Relic network performance monitoring UI. ### Podman container 1. On a host with Podman installed, download the **ktranslate** image by running the following command: - [Docker Hub](https://hub.docker.com/): ```shell podman pull docker.io/kentik/ktranslate:v2 ``` 2. Copy the `snmp-base.yaml` file to the local `$HOME` directory of your Podman user, and discard the container by running: ```shell cd ~ id=$(podman create kentik/ktranslate:v2) podman cp $id:/etc/ktranslate/snmp-base.yaml . podman rm -v $id ``` 3. Edit the `snmp-base.yaml` file, and add your network flow devices inside the `devices` dictionary key with the following structure: ```yaml devices: # This key and the corresponding 'device_name' # need to be unique for each device flow_device1: device_name: flow_device1 device_ip: x.x.x.x/yy flow_only: true # Optional user tags user_tags: owning_team: net_eng environment: production ``` > #### ⚠️ IMPORTANT > > If you're already monitoring SNMP data devices that will also send network flows, you'll want to ensure that the value for `device_name` is identical for both configuration files to ensure the flow telemetry is attributed to the right entity in the New Relic UI. 4. Run `ktranslate` to listen for network flows with the command: ```shell podman run -d --name ktranslate-$CONTAINER_SERVICE \ --userns=keep-id --restart unless-stopped --pull=always --net=host \ -v `pwd`/snmp-base.yaml:/snmp-base.yaml \ -e NEW_RELIC_API_KEY=$YOUR_NR_LICENSE_KEY \ kentik/ktranslate:v2 \ -snmp /snmp-base.yaml \ -nr_account_id=$YOUR_NR_ACCOUNT_ID \ -metrics=jchf \ -tee_logs=true \ # Use this field to create a unique value for `tags.container_service` inside of New Relic -service_name=$CONTAINER_SERVICE \ -flow_only=true \ nr1.flow ``` > #### 💡 TIP > > Rootless Podman containers aren't able to bind to ports under 1024. If your network flows are sent on a port under 1024 instead of the default (9995), you'll need to create an `iptables` rule to handle packet redirection with the command: > > ```shell > sudo iptables -t nat -A PREROUTING -p udp --dport $srcPort -j REDIRECT --to-port 9995 > ``` 5. Investigate your network flow data in the New Relic network performance monitoring UI. ### Linux service 1. Depending on your package manager, use one of the commands below to install `ktranslate` - Yum: ```shell curl -s https://packagecloud.io/install/repositories/kentik/ktranslate/script.rpm.sh | sudo bash && \ sudo yum install ktranslate ``` - Apt: ```shell curl -s https://packagecloud.io/install/repositories/kentik/ktranslate/script.deb.sh | sudo bash && \ sudo apt-get install ktranslate ``` 2. Define the environment variables used by `ktranslate`: ```shell sudo tee "/etc/default/ktranslate.env" > /dev/null <<'EOF' NR_ACCOUNT_ID=$YOUR_NR_ACCOUNT_ID NEW_RELIC_API_KEY=$YOUR_NR_LICENSE_KEY KT_FLAGS="-snmp /etc/ktranslate/snmp-base.yaml \ -metrics=jchf \ -flow_only=true \ -tee_logs=true \ -service_name=$CONTAINER_SERVICE \ nr1.flow" EOF # ensure /etc/default/ktranslate.env is owned by ktranslate user sudo chown ktranslate:ktranslate /etc/default/ktranslate.env ``` 3. If you don't have an existing `snmp-base.yaml` configuration file, create one with: ```shell cd ~ touch snmp-base.yaml ``` 4. Edit the `snmp-base.yaml` file, and add your network flow devices inside the `devices` dictionary key with the following structure: ```yaml devices: # This key and the corresponding 'device_name' # need to be unique for each device flow_device1: device_name: flow_device1 device_ip: x.x.x.x/yy flow_only: true # Optional user tags user_tags: owning_team: net_eng environment: production ``` 5. Restart the `ktranslate` service to apply your changes to the `snmp-base.yaml` file: ```shell sudo systemctl restart ktranslate ``` 6. Investigate your network flow data in the New Relic network performance monitoring UI. ## Find and use your metrics [#find-your-metrics] All network flow logs exported from the `ktranslate` container use the `KFlow` namespace, via the [New Relic Event API](https://docs.newrelic.com/docs/telemetry-data-platform/ingest-apis/introduction-event-api). Currently, these are the default fields populated from this integration: **KFlow Fields** | Attribute | Type | Description | | -------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `application` | String | The class of program generating the traffic in this flow record. This is derived from the lowest numeric value from `l4_dst_port` and `l4_src_port`. Common examples include `http`, `ssh`, and `ftp`. | | `device_name` | String | The display name of the sampling device for this flow record. | | `dst_addr` | String | The target IP address for this flow record. | | `dst_as` | Numeric | The target [Autonomous System Number]\( as-numbers/as-numbers.xhtml) for this flow record. | | `dst_as_name` | String | The target [Autonomous System Name]\( as-numbers/as-numbers.xhtml) for this flow record. | | `dst_endpoint` | String | The target `IP:Port` tuple for this flow record. This is a combination of `dst_addr` and `l4_dst_port`. | | `dst_geo` | String | The target country for this flow record, if known. | | `in_bytes` | Numeric | The number of bytes transferred for ingress flow records. | | `in_pkts` | Numeric | The number of packets transferred for ingress flow records. | | `input_port` | Numeric | `If_Index` value for the interface at the source of this flow record. | | `l4_dst_port` | Numeric | The target port for this flow record. | | `l4_src_port` | Numeric | The source port for this flow record. | | `output_port` | Numeric | `If_Index` value for the interface at the destination of this flow record. | | `protocol` | String | The display name of the protocol used in this flow record, derived from the [numeric IANA protocol number]\( protocol-numbers/protocol-numbers.xhtml). | | `provider` | String | This attribute is used to uniquely identify various sources of data from `ktranslate`. Network flow logs will always have the value of `kentik-flow-device`. | | `sample_rate` | Numeric | Sampling rate applied by either the sampling device configuration, or the `sample_rate` argument in `ktranslate`. | | `src_addr` | String | The source IP address for this flow record. | | `src_as` | Numeric | The source [Autonomous System Number]\( as-numbers/as-numbers.xhtml) for this flow record. | | `src_as_name` | String | The source [Autonomous System Name]\( as-numbers/as-numbers.xhtml) for this flow record. | | `src_endpoint` | String | The source `IP:Port` tuple for this flow record. It's a combination of `src_addr` and `l4_src_port`. | | `src_geo` | String | The source country for this flow record, if known. | | `tcp_flags` | Numeric | TCP flags in this flow record. | | `timestamp` | Numeric | The time, in Unix seconds, when this flow record was received by the New Relic Event API. | ## What's next? You can set up some additional agents to complement your network flow data: - To get better visibility into your network device performance, [set up SNMP data monitoring](https://docs.newrelic.com/docs/network-performance-monitoring/setup-performance-monitoring/snmp-performance-monitoring). - To get better visibility into your network syslog messages, [set up network syslog monitoring](https://docs.newrelic.com/docs/network-performance-monitoring/setup-performance-monitoring/network-syslog-monitoring).