---
title: Troubleshoot Docker container monitoring
source: https://docs.newrelic.com/docs/opentelemetry/integrations/docker-monitoring/troubleshooting
---

If you're having issues with your Docker monitoring setup, check these common problems and solutions.

## NRDOT Collector

> #### 💡 TIP
>
> **General NRDOT troubleshooting**: For collector startup issues, configuration validation, network connectivity, and NrIntegrationError diagnostics, see the [NRDOT Troubleshooting Guide](https://github.com/newrelic/nrdot-collector-releases/blob/main/distributions/TROUBLESHOOTING.md). The issues below are specific to Docker container monitoring.

**No Docker metrics appearing in New Relic**

**Issue**: Docker containers are running but no Docker metrics appear in New Relic after 10+ minutes.

**Diagnostics**:

```bash
# Check Docker daemon accessibility
docker info

# Test Docker socket permissions
ls -l /var/run/docker.sock

# Verify containers are running
docker ps

# Check collector logs for Docker-specific errors
journalctl -u nrdot-collector | grep -i "docker"
```

**Solutions**:

1.  **Docker socket access**: Add nrdot-collector user to docker group:
    ```bash
    sudo usermod -aG docker nrdot-collector
    sudo systemctl restart nrdot-collector
    ```

2.  **Docker API version**: Ensure Docker API version is 1.25+:
    ```bash
    docker version --format '{{.Client.APIVersion}}'
    ```

**Permission denied errors**

**Issue**: Collector logs show "permission denied" when accessing Docker socket.

**Diagnostics**:

```bash
# Check socket ownership and permissions
ls -la /var/run/docker.sock

# Check collector user membership
groups nrdot-collector

# Check collector logs for permission errors
journalctl -u nrdot-collector | grep -i "permission denied"
```

**Solutions**:

```bash
# Option 1: Add collector user to docker group (recommended)
sudo usermod -aG docker nrdot-collector
sudo systemctl restart nrdot-collector

# Option 2: Adjust socket permissions (less secure)
sudo chmod 666 /var/run/docker.sock

# Option 3: Run collector as root (not recommended for production)
# Modify systemd service file User= setting
```

**Incomplete or missing container metrics**

**Issue**: Some containers show metrics while others don't, or metrics are sporadic.

**Diagnostics**:

```bash
# Check for containers with networking issues
docker inspect $(docker ps -q) | grep -A5 "NetworkSettings"

# Review collector configuration
cat /etc/nrdot-collector/docker-stats-config.yaml | grep -A10 "docker_stats:"

# Check for resource constraints
docker stats --no-stream

# Check collector logs for specific errors
journalctl -u nrdot-collector | grep -i "docker_stats"
```

**Solutions**:

-   **Container visibility**: Ensure all containers are accessible via the Docker socket path configured in the collector
-   **Resource limits**: Increase collector timeout values if containers are slow to respond. Edit your `/etc/nrdot-collector/docker-stats-config.yaml`:
    ```yaml
    receivers:
      docker_stats:
        timeout: 10s  # Increase from default 5s
        collection_interval: 30s  # Reduce frequency if needed
    ```
-   **Network namespaces**: Some containers in restricted network namespaces may not be accessible

**Log collection not working**

**Issue**: Configured log collection but container logs don't appear in New Relic.

**Diagnostics**:

```bash
# Check log directory permissions
ls -la /var/lib/docker/containers/

# Verify receiver creator is discovering containers
journalctl -u nrdot-collector | grep -i "receiver.*creator\|docker.*observer"

# Check if containers have exposed ports (required for discovery)
docker ps --format 'table {{.Names}}\t{{.Ports}}'

# Verify logs pipeline is configured
journalctl -u nrdot-collector | grep -i "logs"
```

**Solutions**:

1.  **File permissions**: Grant log directory access:
    ```bash
    sudo chmod -R 755 /var/lib/docker/containers/
    # Or add collector to docker group
    sudo usermod -aG docker nrdot-collector
    sudo systemctl restart nrdot-collector
    ```

2.  **Container port exposure**: Expose ports for container discovery:
    ```bash
    # When running containers, expose at least one port
    docker run -p 8080:8080 my-app:latest
    # Or use --expose flag
    docker run --expose 80 my-app:latest
    ```

3.  **Verify logs config**: Ensure your log collection configuration is included:
    ```bash
    # Check systemd override
    sudo systemctl cat nrdot-collector | grep -A2 "ExecStart="
    # Should point to: --config=/etc/nrdot-collector/docker-stats-config.yaml (or combined config with logs)
    ```

4.  **Query for logs in New Relic**:
    ```sql
    FROM Log SELECT * WHERE newrelic.source='api.logs.otlp' SINCE 1 hour ago
    ```

## OpenTelemetry Collector Contrib

**Collector not starting or crashing**

**Issue**: OpenTelemetry Collector fails to start or crashes immediately.

**Diagnostics**:

```bash
# Check collector status
sudo systemctl status otelcol-contrib

# View recent logs
journalctl -u otelcol-contrib --since "1 hour ago" -f

# Test configuration syntax
otelcol-contrib --config-validate --config=/etc/otelcol-contrib/config.yaml
```

**Solutions**:

-   **Configuration errors**: Fix YAML syntax errors and ensure all required fields are present
-   **Missing permissions**: Ensure collector user has read access to configuration file
-   **Environment variables**: Verify `NEWRELIC_LICENSE_KEY` and `NEWRELIC_OTLP_ENDPOINT` are set correctly

**No metrics appearing in New Relic**

**Issue**: Docker containers are running but no metrics appear in New Relic after 10+ minutes.

**Diagnostics**:

```bash
# Verify collector is running and processing data
journalctl -u otelcol-contrib | grep -i "docker\|error\|export"

# Check Docker daemon accessibility
docker info

# Test Docker socket permissions
ls -l /var/run/docker.sock

# Verify containers are running
docker ps
```

**Solutions**:

1.  **Docker socket access**: Add collector user to docker group:
    ```bash
    sudo usermod -aG docker otelcol-contrib
    sudo systemctl restart otelcol-contrib
    ```

2.  **License key issues**: Verify your license key is correct and has proper permissions:
    ```bash
    echo $NEWRELIC_LICENSE_KEY
    # Should be 40-character NRAK-prefixed key for US, EU, or JP equivalent
    ```

3.  **Network connectivity**: Test OTLP endpoint accessibility:

    ```bash
    # For US region (default)
    curl -v -H "api-key: $NEWRELIC_LICENSE_KEY" https://otlp.nr-data.net/v1/metrics

    # For EU region
    curl -v -H "api-key: $NEWRELIC_LICENSE_KEY" https://otlp.eu01.nr-data.net/v1/metrics

    # For JP region
    curl -v -H "api-key: $NEWRELIC_LICENSE_KEY" https://otlp.jp.nr-data.net/v1/metrics
    ```

4.  **Docker API version**: Ensure Docker API version is 1.25+:
    ```bash
    docker version --format '{{.Client.APIVersion}}'
    ```

**Permission denied errors**

**Issue**: Collector logs show "permission denied" when accessing Docker socket.

**Diagnostics**:

```bash
# Check socket ownership and permissions
ls -la /var/run/docker.sock

# Check collector user membership
groups otelcol-contrib
```

**Solutions**:

```bash
# Option 1: Add collector user to docker group (recommended)
sudo usermod -aG docker otelcol-contrib
sudo systemctl restart otelcol-contrib

# Option 2: Adjust socket permissions (less secure)
sudo chmod 666 /var/run/docker.sock

# Option 3: Run collector as root (not recommended for production)
# Modify systemd service file User= setting
```

**Incomplete or missing container metrics**

**Issue**: Some containers show metrics while others don't, or metrics are sporadic.

**Diagnostics**:

```bash
# Check for containers with networking issues
docker inspect $(docker ps -q) | grep -A5 "NetworkSettings"

# Review collector configuration
grep -A20 "docker_stats:" /etc/otelcol-contrib/config.yaml

# Check for resource constraints
docker stats --no-stream
```

**Solutions**:

-   **Container visibility**: Ensure all containers are accessible via the Docker socket path configured in the collector

-   **Resource limits**: Increase collector timeout values if containers are slow to respond:
    ```yaml
    docker_stats:
      timeout: 10s  # Increase from default 5s
      collection_interval: 30s  # Reduce frequency if needed
    ```

-   **Network namespaces**: Some containers in restricted network namespaces may not be accessible

**High memory or CPU usage by collector**

**Issue**: OpenTelemetry Collector consuming excessive system resources.

**Diagnostics**:

```bash
# Monitor collector resource usage
top -p $(pgrep otelcol-contrib)

# Check metric collection frequency
grep -A10 "docker_stats:" /etc/otelcol-contrib/config.yaml

# Count active containers
docker ps --format 'table {{.Names}}\t{{.Status}}' | wc -l
```

**Solutions**:

-   **Adjust collection interval**: Reduce monitoring frequency for high-container-count environments:
    ```yaml
    docker_stats:
      collection_interval: 30s  # Increase from 15s
    ```

-   **Disable expensive metrics**: Turn off per-CPU metrics if not needed:
    ```yaml
    docker_stats:
      metrics:
        container.cpu.usage.percpu:
          enabled: false
    ```

-   **Configure batch processing**: Optimize batching for your environment:
    ```yaml
    batch:
      timeout: 60s
      send_batch_size: 1024
    ```

**Log collection not working**

**Issue**: Configured log collection but container logs don't appear in New Relic.

**Diagnostics**:

```bash
# Check log directory permissions
ls -la /var/lib/docker/containers/

# Verify receiver creator is discovering containers
journalctl -u otelcol-contrib | grep -i "receiver.*creator\|docker.*observer"

# Check if containers have exposed ports (required for discovery)
docker ps --format 'table {{.Names}}\t{{.Ports}}'
```

**Solutions**:

1.  **File permissions**: Grant log directory access:
    ```bash
    sudo chmod -R 755 /var/lib/docker/containers/
    # Or add collector to docker group
    sudo usermod -aG docker otelcol-contrib
    ```

2.  **Container port exposure**: Expose ports for container discovery:
    ```bash
    # When running containers, expose at least one port
    docker run -p 8080:8080 my-app:latest
    # Or use --expose flag
    docker run --expose 80 my-app:latest
    ```

3.  **Observer configuration**: Verify Docker observer settings:
    ```yaml
    docker_observer:
      endpoint: unix:///var/run/docker.sock
      use_hostname_if_present: true
      use_container_labels: true
    ```

**Data not appearing in correct New Relic account**

**Issue**: Metrics appear in wrong New Relic account or organization.

**Diagnostics**:

```bash
# Verify license key format and region
echo $NEWRELIC_LICENSE_KEY | cut -c1-4
# Should show "NRAK" for US accounts

# Check configured endpoint
echo $NEWRELIC_OTLP_ENDPOINT
```

**Solutions**:

-   **US accounts**: Use `https://otlp.nr-data.net`
-   **EU accounts**: Use `https://otlp.eu01.nr-data.net`
-   **JP accounts**: Use `https://otlp.jp.nr-data.net`
-   **Verify license key**: Ensure you're using the correct account's license key

For persistent issues, check the [OpenTelemetry Collector troubleshooting guide](https://opentelemetry.io/docs/collector/troubleshooting/) and [New Relic OpenTelemetry documentation](https://docs.newrelic.com/docs/opentelemetry/opentelemetry-introduction/).

## Next steps [#next-steps]

If you've resolved your issues:

-   **Get back to setup**: Return to the [Docker monitoring setup guide](https://docs.newrelic.com/docs/opentelemetry/integrations/docker-monitoring/self-hosted) to complete your configuration
-   **Explore your data**: Learn how to query and visualize your Docker metrics with the [metrics reference guide](https://docs.newrelic.com/docs/opentelemetry/integrations/docker-monitoring/metrics-reference)
-   **Set up alerts**: Create proactive monitoring with [alert conditions](https://docs.newrelic.com/docs/alerts/create-alert/create-alert-condition/create-nrql-alert-conditions/) for critical container metrics