--- title: "Netstat" description: "Complete netstat reference covering network connections, listening ports, routing tables, and network statistics with practical examples" author: "Mohammad Abu Mattar" canonical: https://mkabumattar.com/cheatsheets/netstat --- # Netstat This comprehensive netstat cheatsheet is now ready for your portfolio website. All 6 major categories have been created with detailed examples, explanations, and practical troubleshooting techniques. ## Getting Started Learn what netstat is and understand the basics of network monitoring ### What is Netstat Understand netstat and its capabilities for network diagnostics **Keywords:** introduction, what, about, netstat, network, diagnostics #### Install and verify netstat installation ```bash # On Debian/Ubuntu systems sudo apt-get update sudo apt-get install net-tools # On CentOS/RHEL systems sudo yum install net-tools # On Fedora systems sudo dnf install net-tools # Verify installation which netstat netstat --version ``` _exec_ ```bash which netstat && echo "Netstat is installed" ``` _output_ ```bash /usr/bin/netstat Netstat is installed ``` Netstat is part of the net-tools package and is available on most Linux systems for network monitoring. - Netstat may not be installed by default on minimal Linux installations - Modern alternative is 'ss' command (comes with iproute2) - Both tools provide similar functionality - Some distributions are phasing out netstat in favor of ss #### Netstat overview and capabilities ```bash # Netstat displays: # - Active network connections # - Network interface statistics # - Routing tables # - Transport protocol statistics # - Per-protocol statistics # Key features: # - Monitor network connections in real-time # - Find processes using specific ports # - Display listening ports and services # - Analyze network traffic patterns # - Troubleshoot connectivity issues # Netstat is a diagnostic tool, not a monitoring daemon # Run it to get a snapshot of network state at that moment netstat --help | head -20 ``` _exec_ ```bash netstat -h | grep -E "^ -[a-z]" | head -10 ``` _output_ ```bash -a, --all show both listening and non-listening sockets -A, --family=FAMILY Address family (inet|inet6|unix|ipx|ax25|netrom|rose|decnet|x25) -B, --sbufsize=SIZE show both listening and non-listening sockets -c, --continuous continuous listing -e, --extend show additional information ``` Netstat provides comprehensive network diagnostics with various flags to show different network information. - Each flag can be combined with others for detailed analysis - Use -h or --help to see all available options - Netstat output can be very large on systems with many connections - Combine with grep for filtering results ### Core Concepts Understand basic netstat terminology and network concepts **Keywords:** concepts, terminology, connections, states, protocols #### Understand netstat output columns and fields ```bash # Netstat output columns explained: # Proto: Protocol (tcp, udp, unix) # Recv-Q: Bytes in receive queue # Send-Q: Bytes in send queue # Local Address: Local IP:Port # Foreign Address: Remote IP:Port # State: Connection state # PID/Program name: Process using the connection # Example interpretation: # tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 1234/mysqld # This means MySQL is listening on port 3306 (PID 1234) netstat -tln | head -5 ``` _exec_ ```bash netstat -an | head -3 && echo "..." && netstat -an | tail -3 ``` _output_ ```bash Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN ... tcp 0 0 192.168.1.100:55432 10.0.0.50:443 ESTABLISHED ``` Understanding netstat column headers helps interpret network connection details accurately. - Recv-Q and Send-Q show bytes waiting to be processed - State column shows connection lifecycle stage - PID/Program name requires root/sudo privileges with -p flag - Foreign Address shows remote connection endpoint #### Understand connection states and transmission protocol ```bash # Common connection states: # LISTEN - Socket is waiting for incoming connections # ESTABLISHED - Connection is active and data is flowing # SYN_SENT - Waiting for matching connection request # SYN_RECV - Received connection request, waiting for acknowledgment # TIME_WAIT - Connection closed, waiting for remaining packets # CLOSE_WAIT - Remote end closed, waiting for local close # LAST_ACK - Waiting for acknowledgment of connection close # CLOSED - Connection is closed # CLOSE - Initiating close on socket # TCP provides reliable, ordered delivery (LISTEN, ESTABLISHED, etc) # UDP is connectionless (no STATE field in netstat -u output) netstat -an | grep ESTABLISHED | head -3 ``` _exec_ ```bash netstat -an | grep -E "^tcp|^udp" | head -5 ``` _output_ ```bash tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN tcp6 0 0 :::22 :::* LISTEN udp 0 0 0.0.0.0:68 0.0.0.0:* udp 0 0 0.0.0.0:5353 0.0.0.0:* ``` Connection states are part of the TCP state machine; UDP connections don't have states since it's connectionless. - Different states indicate different stages of connection lifecycle - TIME_WAIT connections are normal and eventually expire - CLOSE_WAIT can indicate zombie processes or unresponsive clients - UDP doesn't maintain connection state ### Modern Alternatives Modern tools that replace netstat functionality **Keywords:** alternatives, ss, modern, replacement, tools #### Use ss command instead of netstat ```bash # ss (socket statistics) is the modern replacement for netstat # ss is faster and provides more detailed information # ss comes with iproute2 package (usually pre-installed) # Show all listening sockets ss -tln # Show all TCP connections ss -ta # Show specific port ss -tln | grep :8080 # Show process names (requires sudo) sudo ss -tlnp | grep :80 ``` _exec_ ```bash which ss && echo "ss command is available" ``` _output_ ```bash /usr/sbin/ss ss command is available ``` The ss command is the preferred modern alternative to netstat with better performance and features. - ss uses the same flags as netstat - ss is faster because it reads from /proc/net directly - ss shows more detailed information (timer info, congestion window) - Both tools can be used interchangeably in most cases #### Compare netstat and ss output ```bash # Both commands show similar information # netstat output: # Proto Recv-Q Send-Q Local Address Foreign Address State # tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN # ss output includes additional details: # State Recv-Q Send-Q Local Address:Port Peer Address:Port # LISTEN 0 128 0.0.0.0:22 0.0.0.0:* # Performance comparison: # netstat scans /proc/net files # ss uses netlink sockets (faster) echo "=== Using netstat ===" && netstat -tln | head -2 echo "=== Using ss ===" && ss -tln | head -2 ``` _exec_ ```bash netstat -tln | head -2 && echo "---" && ss -tln | head -2 ``` _output_ ```bash Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State --- State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 0.0.0.0:22 0.0.0.0:* ``` Both netstat and ss provide network statistics, with ss being the faster modern option. - ss is recommended for new scripts and tools - netstat is still useful for compatibility - Some systems only have ss installed - Learn both for comprehensive network administration **Best practices:** - Install net-tools on systems that don't have netstat - Consider using ss instead of netstat on modern systems - Both tools read the same system data - Combine with other tools like grep and awk for filtering **Common errors:** - **netstat: command not found**: Install net-tools package using apt/yum/dnf package manager - **No option: -p**: Run netstat with sudo for -p flag to show process information ## Basic Usage Master essential netstat commands for displaying connections ### Show All Connections Display all network connections on the system **Keywords:** connections, all, active, listing, verbose #### Display all network connections ```bash # Show all connections (listening and established) netstat -a # Show only TCP connections netstat -at # Show only UDP connections netstat -au # Show all with numeric IP addresses netstat -an # Show all with program names sudo netstat -ap ``` _exec_ ```bash netstat -an | head -5 ``` _output_ ```bash Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp6 0 0 :::22 :::* LISTEN ``` The -a flag shows all connections (both listening and established), while -n displays numeric addresses. - Use -a to see complete network picture - -n is faster than resolving hostnames - Combine -a with -n for numeric all connections - Output can be very large on systems with many connections #### Show connections with detailed information ```bash # Extended information about all connections netstat -ae # Very detailed information netstat -aee # All connections sorted by state netstat -a | sort -k6 # Count connections by type netstat -an | grep -c "ESTABLISHED" # Show connections with timestamps netstat -ae | grep -v "^Active" ``` _exec_ ```bash netstat -a | wc -l ``` _output_ ```bash 18 ``` Extended flags provide additional details about connections, useful for diagnostics and analysis. - -e flag shows ethernet statistics - -ee shows even more extended information - Useful for filtering and analyzing specific connection types - Can be piped to grep and other tools for filtering ### List Listening Ports Display ports that are actively listening for connections **Keywords:** listening, ports, services, open, daemons #### Show all listening ports and services ```bash # Show all listening sockets (listening only) netstat -l # Show listening TCP ports netstat -lt # Show listening UDP ports netstat -lu # Show listening with numeric output netstat -ln # Show listening with process information sudo netstat -lnp ``` _exec_ ```bash netstat -ln | grep -E "LISTEN|Proto" ``` _output_ ```bash Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp6 0 0 :::22 :::* LISTEN udp 0 0 0.0.0.0:68 0.0.0.0:* ``` The -l flag shows only listening sockets, useful for discovering what services are active on a system. - -l shows listening ports only - Combine with -n for numeric output - Use -p to see which process is listening - Requires sudo for -p flag to show all processes #### Find specific listening services ```bash # Show listening services with process names sudo netstat -lnp # Show only HTTP and HTTPS listeners sudo netstat -lnp | grep -E ":80\s|:443\s" # Show listening ports in numeric form netstat -ln | grep -E "LISTEN|Proto" # Check if specific port is listening sudo netstat -lnp | grep ":8080" # List all listening TCP ports sorted netstat -ln | grep tcp | grep LISTEN | sort -k4 ``` _exec_ ```bash sudo netstat -lnp | grep -E ":22 " | head -3 ``` _output_ ```bash tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1234/sshd tcp6 0 0 :::22 :::* LISTEN 1234/sshd ``` Using grep with netstat helps identify specific listening services and their process IDs. - -p flag requires root/sudo privileges - Useful for auditing open ports - Shows PID and program name for each service - Can be used for security verification ### Protocol-Specific Display Show connections for specific network protocols **Keywords:** protocol, tcp, udp, filtering, specific #### Display TCP and UDP connections separately ```bash # Show only TCP connections netstat -t # Show only TCP with numeric output netstat -tn # Show only UDP connections netstat -u # Show only UDP with numeric output netstat -un # Show TCP established connections only netstat -t | grep ESTABLISHED # Show UDP with extended info netstat -ue ``` _exec_ ```bash netstat -tn | grep ESTABLISHED | wc -l ``` _output_ ```bash 8 ``` Filter connections by protocol type to focus on specific communication types (TCP for reliable, UDP for fast). - TCP connections show state information - UDP is stateless (no connection states shown) - -t flag for TCP, -u flag for UDP - Combine with -l for listening ports of specific protocol #### Show both IPv4 and IPv6 connections ```bash # Show all internet connections (IPv4 and IPv6) netstat -A inet,inet6 -an # Show IPv4 TCP only netstat -A inet -tn # Show IPv6 TCP only netstat -A inet6 -tn # Show connections and exclude IPv6 netstat -A inet -an # Show dual-stack listening (both IPv4 and IPv6) netstat -ln | grep -E "tcp|tcp6" ``` _exec_ ```bash netstat -A inet -tn | head -4 ``` _output_ ```bash Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 192.168.1.100:55432 203.0.113.50:443 ESTABLISHED ``` Different address families (inet/inet6) can be filtered separately to analyze specific connection types. - inet is IPv4, inet6 is IPv6 - Some services listen on both IPv4 and IPv6 - tcp6 indicates IPv6 TCP connections - Can be used for network troubleshooting **Best practices:** - Always use -n flag for faster output - Use -l to check available services - Combine -t and -n for TCP analysis - Use -p with sudo for process identification **Common errors:** - **Too much output to parse**: Use grep to filter results or use less for pagination - **Permission denied**: Use sudo for -p flag to see process information ## Finding Connections and Ports Locate specific connections, ports, and processes ### Find Listening Port Discover which process is using a specific port **Keywords:** find, port, process, specific, identification #### Find process listening on specific port ```bash # Find what's listening on port 8080 sudo netstat -tlnp | grep :8080 # Find what's listening on port 443 sudo netstat -tlnp | grep :443 # Find process on any port sudo netstat -tlnp | grep -E ":80 |:443 |:3306 " # Show all listening ports with processes sudo netstat -tlnp | grep LISTEN # Find IPv4 listeners on port 3000 sudo netstat -tlnp | grep :3000 ``` _exec_ ```bash sudo netstat -tlnp | grep ssh | head -2 ``` _output_ ```bash tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1234/sshd tcp6 0 0 :::22 :::* LISTEN 1234/sshd ``` The -p flag shows process ID and name, allowing identification of services using specific ports. - -p flag requires sudo/root privileges - Useful for port conflict resolution - Shows both IPv4 and IPv6 listeners - PID can be used with kill to terminate the process #### Find all connections from specific IP ```bash # Find all connections to specific IP netstat -an | grep 192.168.1.100 # Find connections to specific remote IP netstat -an | grep 203.0.113.50 # Find all connections from specific port netstat -an | grep ":8080 " # Find connections to specific port on remote netstat -an | grep " :443" # Combine source and destination filters netstat -an | grep "192.168" | grep "ESTABLISHED" ``` _exec_ ```bash netstat -an | grep 127.0.0.1 | head -3 ``` _output_ ```bash tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN unix 2 [ ] DGRAM 3456 @/tmp/var/run/user/1000 ``` Filtering netstat output by IP address helps locate connections from specific hosts or networks. - Use numeric output (-n) for faster filtering - Combine multiple grep conditions for precise filtering - Useful for security analysis and traffic auditing - Can identify unauthorized connections ### Connections by Process Show all connections associated with specific processes **Keywords:** process, connections, pid, program, application #### Show connections for specific process ```bash # Show all nginx connections sudo netstat -anp | grep nginx # Show all MySQL database connections sudo netstat -anp | grep mysql # Show all SSH connections sudo netstat -anp | grep sshd # Show connections by PID sudo netstat -anp | grep " 1234/ # List unique processes with connections sudo netstat -anp | grep -oE '[0-9]+/[a-z]' | sort -u ``` _exec_ ```bash sudo netstat -anp | grep sshd | head -2 ``` _output_ ```bash tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1234/sshd tcp 0 128 192.168.1.100:22 192.168.1.50:55234 ESTABLISHED 1234/sshd ``` The -p flag allows filtering connections by process name or PID for application-specific analysis. - Requires sudo/root for -p flag - Useful for application performance analysis - Can identify connection leaks or excessive connections - Helps monitor multi-process applications #### Analyze connection statistics by process ```bash # Count connections per process sudo netstat -anp | tail -n +3 | awk '{print $NF}' | sort | uniq -c | sort -rn # Find process with most connections sudo netstat -anp | tail -n +3 | awk '{print $NF}' | sort | uniq -c | sort -rn | head -1 # Show established connections per process sudo netstat -anp | grep ESTABLISHED | awk '{print $NF}' | sort | uniq -c | sort -rn # Find orphaned connections sudo netstat -anp | grep "CLOSE_WAIT" | awk '{print $NF}' | sort -u ``` _exec_ ```bash sudo netstat -anp 2>/dev/null | grep LISTEN | wc -l ``` _output_ ```bash 12 ``` Using awk and sort with netstat helps aggregate statistics and identify problematic connection patterns. - Useful for debugging connection leaks - Identifies most active processes - Can detect stuck connections - Helps optimize application performance ### Established Connections Focus on active data-transfer connections **Keywords:** established, active, data-transfer, monitoring, real-time #### Show only established connections ```bash # Show all established connections netstat -an | grep ESTABLISHED # Show established with process info sudo netstat -anp | grep ESTABLISHED # Count established connections netstat -an | grep ESTABLISHED | wc -l # Show established TCP connections only netstat -tn | grep ESTABLISHED # Monitor established connections to specific port sudo netstat -anp | grep ":443" | grep ESTABLISHED ``` _exec_ ```bash netstat -an | grep ESTABLISHED | wc -l ``` _output_ ```bash 5 ``` Filtering for ESTABLISHED connections shows only active data-transfer connections, excluding listening sockets. - ESTABLISHED means active connection with data flowing - Useful for monitoring active connections - Helps identify connection hangs by their absence - Can be monitored continuously for real-time activity #### Monitor active connections with details ```bash # Show established connections with full details sudo netstat -anpe | grep ESTABLISHED # Show established by remote host netstat -an | grep ESTABLISHED | awk '{print $5}' | cut -d: -f1 | sort | uniq -c # List established connections with packet counts sudo netstat -anp | grep ESTABLISHED | awk '{print $4, $5, $NF}' # Monitor specific established connection sudo netstat -anp | grep ESTABLISHED | grep 203.0.113.50 # Track connection duration (using state history) watch -n 1 'netstat -an | grep ESTABLISHED | wc -l' ``` _exec_ ```bash netstat -an | grep ESTABLISHED | awk '{print $5}' | head -3 ``` _output_ ```bash 203.0.113.50:443 203.0.113.51:443 203.0.113.52:443 ``` Advanced filtering and awk processing allows detailed analysis of established connections by remote host. - Use awk for programmatic filtering - Can be piped to other commands for analysis - watch command enables real-time monitoring - Useful for identifying connection patterns **Best practices:** - Use -p with sudo for complete process information - Combine grep with awk for complex filtering - Use numeric output (-n) for consistent formatting - Regularly monitor for unexpected connections **Common errors:** - **No results found when searching for port**: Ensure the service is running and listening - **awk/grep output not as expected**: Verify netstat output format matches your grep pattern ## Connection States Understand and analyze TCP connection states ### TCP States Explained Detailed explanation of TCP connection state lifecycle **Keywords:** states, tcp, lifecycle, state-machine, transitions #### Understand TCP connection states ```bash # LISTEN - Waiting for incoming connections # Port is open and actively listening for connections # Used by server applications # SYN_SENT - Initial connection request sent # Waiting for acknowledgment from remote host # Usually very brief state # SYN_RECV - Connection request received and acknowledged # Waiting for acknowledgment back # Rare to see in netstat output # ESTABLISHED - Active data transfer # Connection is open and both sides exchange data # Most common state for active connections # FIN_WAIT_1 - Waiting for connection close # Local side initiated close # Waiting for acknowledgment # FIN_WAIT_2 - Waiting for remote close # Local side closed sending # Waiting for remote side to close netstat -an | grep -E "LISTEN|ESTABLISHED|TIME_WAIT" | head -10 ``` _exec_ ```bash netstat -an | grep -o -E "LISTEN|ESTABLISHED|TIME_WAIT" | sort | uniq -c ``` _output_ ```bash 3 ESTABLISHED 5 LISTEN 12 TIME_WAIT ``` TCP states represent different stages of connection lifecycle, from establishment to termination. - Each state has specific meaning and transition rules - States are defined in TCP state machine (RFC 793) - Some states are transient and rarely seen - Understanding states helps with network troubleshooting #### Show connections in specific states ```bash # Show TIME_WAIT connections (waiting to close) netstat -an | grep TIME_WAIT # Show CLOSE_WAIT connections (remote closed) netstat -an | grep CLOSE_WAIT # Show all non-listening connections netstat -an | grep -v LISTEN | grep -v "^Proto" # Show only listening ports netstat -an | grep LISTEN # Count connections by state netstat -an | grep -oE "LISTEN|ESTABLISHED|TIME_WAIT|CLOSE_WAIT" | sort | uniq -c ``` _exec_ ```bash netstat -an | grep CLOSE_WAIT | wc -l ``` _output_ ```bash 0 ``` Specific state filtering reveals connection patterns and potential networking issues on the system. - TIME_WAIT connections are normal cleanup - CLOSE_WAIT indicates unclosed sockets - Many TIME_WAIT connections can be investigation point - Use for performance analysis ### Timeout and Waiting States Analyze timeout and waiting connection states **Keywords:** timeout, waiting, states, cleanup, lingering #### Monitor TIME_WAIT connections ```bash # TIME_WAIT is normal post-connection cleanup state # Connection is closed but local port still in use # Exists to prevent packet confusion # Show all TIME_WAIT connections netstat -an | grep TIME_WAIT # Count TIME_WAIT by remote host netstat -an | grep TIME_WAIT | awk '{print $5}' | cut -d: -f1 | sort | uniq -c # Show TIME_WAIT connections in details sudo netstat -anp | grep TIME_WAIT # Count TIME_WAIT per second (monitor trend) watch -n 1 'netstat -an | grep TIME_WAIT | wc -l' ``` _exec_ ```bash netstat -an | grep TIME_WAIT | wc -l ``` _output_ ```bash 8 ``` TIME_WAIT is a normal TCP state for graceful connection termination and preventing packet confusion. - TIME_WAIT lasts 2 MSL (Maximum Segment Lifetime) - Default is often 60 seconds - Many TIME_WAIT is normal for servers - Can be tuned with kernel parameters if excessive #### Identify and analyze CLOSE_WAIT connections ```bash # CLOSE_WAIT means remote closed but local still open # Usually indicates application bug with unclosed sockets # Can lead to resource leaks if not addressed # Show all CLOSE_WAIT connections netstat -an | grep CLOSE_WAIT # Show CLOSE_WAIT with process info sudo netstat -anp | grep CLOSE_WAIT # Find which application has CLOSE_WAIT sudo netstat -anp | grep CLOSE_WAIT | awk '{print $NF}' | sort | uniq -c # Monitor CLOSE_WAIT count watch -n 5 'netstat -an | grep CLOSE_WAIT | wc -l' # List all CLOSE_WAIT connections details sudo netstat -anpe | grep CLOSE_WAIT ``` _exec_ ```bash sudo netstat -anp | grep CLOSE_WAIT | head -2 ``` _output_ ```bash tcp 0 0 192.168.1.100:55432 203.0.113.60:3306 CLOSE_WAIT 5678/python tcp 0 0 192.168.1.100:55433 203.0.113.61:3306 CLOSE_WAIT 5678/python ``` CLOSE_WAIT connections indicate the remote side closed but the local application hasn't closed its socket. - CLOSE_WAIT accumulation suggests application bug - Applications should close sockets properly - Common in poorly-written database client code - Requires application fix, not system configuration ### Connection Flow Analysis Analyze full connection flow and state transitions **Keywords:** flow, transitions, analysis, patterns, diagnostics #### Analyze connection flow states ```bash # Connections follow this general flow: # Client: CLOSED -> SYN_SENT -> ESTABLISHED -> FIN_WAIT_1 -> FIN_WAIT_2 -> CLOSED # Server: LISTEN -> SYN_RECV -> ESTABLISHED -> CLOSE_WAIT -> LAST_ACK -> CLOSED # Normal healthy flow shows mostly LISTEN and ESTABLISHED netstat -an | awk '{print $NF}' | sort | uniq -c | sort -rn | head -10 # Check for abnormal accumulation of states netstat -an | grep -E "SYN_SENT|SYN_RECV" | wc -l # Show connection setup rate watch -n 1 'netstat -an | grep ESTABLISHED | wc -l' ``` _exec_ ```bash netstat -an | tail -n +3 | awk '{print $NF}' | sort | uniq -c | sort -rn | head -5 ``` _output_ ```bash 4 LISTEN 3 ESTABLISHED 2 TIME_WAIT 1 CLOSE_WAIT ``` Analyzing the distribution of connection states reveals network health and potential bottlenecks. - Mostly LISTEN and ESTABLISHED is healthy - Many SYN states indicates port scanning or DDoS - Many CLOSE_WAIT indicates application bugs - Monitor trends over time for anomalies #### Detect and diagnose connection issues ```bash # Excessive SYN_SENT might indicate port scanning attack netstat -an | grep SYN_SENT # Excessive SYN_RECV indicates half-open connections (SYN flood) netstat -an | grep SYN_RECV | wc -l # Stuck connections in FIN_WAIT state netstat -an | grep FIN_WAIT # Reset connections (LAST_ACK) netstat -an | grep LAST_ACK | wc -l # Full diagnostic report echo "=== Connection State Summary ===" && \ netstat -an | tail -n +3 | awk '{print $NF}' | sort | uniq -c | sort -rn ``` _exec_ ```bash netstat -an | grep SYN_SENT | wc -l ``` _output_ ```bash 0 ``` Investigating abnormal connection states helps identify attacks, bugs, or network issues. - SYN_SENT indicates client attempting connection - Many SYN_SENT from single source indicates attack - FIN_WAIT indicates normal closure - Monitor for abnormal patterns **Best practices:** - Regularly monitor TCP connection states - Track trends in FIN_WAIT and TIME_WAIT counts - Investigate CLOSE_WAIT accumulation - Use automated alerting for abnormal patterns **Common errors:** - **Can't determine which process has CLOSE_WAIT**: Use sudo with -p flag to see process information - **TIME_WAIT connections growing indefinitely**: This is actually normal; TIME_WAIT is automatic cleanup ## Network Statistics and Analysis Display and analyze network statistics and routing information ### Network Statistics Show protocol-level statistics and metrics **Keywords:** statistics, stats, metrics, protocols, traffic #### Display protocol statistics ```bash # Show statistics for all protocols netstat -s # Show TCP statistics only netstat -st # Show UDP statistics only netstat -su # Show ICMP statistics (ping errors) netstat -si # Show statistics for specific protocol netstat -sp tcp ``` _exec_ ```bash netstat -st | head -15 ``` _output_ ```bash Tcp: 4567 active connections openings 1234 passive connection openings 89 failed connection attempts 2 connection resets received 5 connections established 234567 segments received 234567 segments sent out 45 segments retransmitted 0 bad segments received 89 resets sent 156 packets received 234 packets to unknown port received ``` TCP statistics show connection attempts, retransmissions, and error rates for protocol-level diagnostics. - Active openings = connections initiated by this system - Passive openings = connections received from others - Failed attempts indicate connection problems - Retransmissions indicate packet loss or latency #### Analyze connection and packet statistics ```bash # Show statistics for TCP connections netstat -s | grep -A 20 "^Tcp:" # Show packet loss indicators netstat -s | grep -i "segment" # Show connection attempt statistics netstat -s | grep -i "active\|passive\|failed" # Show error statistics netstat -s | grep -i "error\|reset\|bad" # Compare TCP vs UDP traffic echo "=== TCP ===" && netstat -st | grep packets \ && echo "=== UDP ===" && netstat -su | grep packets ``` _exec_ ```bash netstat -s | grep "active connections" ``` _output_ ```bash 4567 active connections openings ``` Analyzing connection statistics helps understand network activity patterns and identify performance issues. - High retransmission rates indicate network problems - Failed connection attempts suggest unreachable services - Compare statistics over time to detect trends - Use for performance baseline and anomaly detection ### Routing Table Display and analyze network routing information **Keywords:** routing, table, routes, gateway, network #### Display system routing table ```bash # Show routing table netstat -r # Show routing table with numeric output netstat -rn # Show IPv4 routing table only netstat -rn -A inet # Show IPv6 routing table netstat -rn -A inet6 # Show routing details with extended info netstat -re ``` _exec_ ```bash netstat -rn | head -8 ``` _output_ ```bash Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default 192.168.1.1 0.0.0.0 UG 100 0 0 eth0 192.168.1.0 0.0.0.0 255.255.255.0 U 256 0 0 eth0 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 ``` Routing table shows how packets are directed to their destinations based on destination IP address. - Destination: Target network or 'default' for fallback - Gateway: Next hop router (0.0.0.0 means direct interface) - Genmask: Subnet mask for the route - Flags: U=up, G=gateway, H=host-specific #### Analyze and troubleshoot routes ```bash # Show all routes with metrics netstat -rn | grep -v "^Kernel" # Show default gateway netstat -rn | grep "^default" # Show specific network routes netstat -rn | grep "192.168" # Count routes per interface netstat -rn | awk '{print $NF}' | sort | uniq -c # Show routes sorted by interface netstat -rn | sort -k 5 ``` _exec_ ```bash netstat -rn | grep "^default" ``` _output_ ```bash default 192.168.1.1 0.0.0.0 UG 100 0 0 eth0 ``` Default gateway and route analysis helps troubleshoot connectivity issues and understand network structure. - 0.0.0.0 gateway means direct connection (no routing needed) - Multiple gateways indicate complex routing - Metric indicates route preference - Use for detecting misconfigured routes ### Interface Statistics Display network interface statistics and metrics **Keywords:** interface, statistics, eth0, lo, packets, bytes #### Show network interface statistics ```bash # Show interface statistics netstat -i # Show interface statistics with extended info netstat -ie # Show very detailed interface information netstat -iee # Show interface MTU and other details netstat -ie | head -10 # Format output for readability netstat -i | column -t ``` _exec_ ```bash netstat -i | head -5 ``` _output_ ```bash Kernel Interface table Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg docker0 1500 34567890 0 0 0 234567 0 0 0 BMRU eth0 1500 987654321 0 5 0 2345678 0 3 0 BMRU ``` Interface statistics show packet counts, errors, and dropped packets for each network interface. - RX-OK: Packets received successfully - TX-OK: Packets transmitted successfully - RX-ERR/TX-ERR: Transmission errors - RX-DRP/TX-DRP: Dropped packets #### Analyze interface health and performance ```bash # Show detailed interface information netstat -ie # Extract interface errors netstat -i | awk '{print $1, $3, $4}' # Check for packet loss netstat -i | awk '{if (NR>2 && $4>0) print $1, "has errors"}' # Show interfaces with dropped packets netstat -i | awk '{if (NR>2 && $5>0) print $1, "dropped:", $5}' # Calculate error rates netstat -i | awk '{if (NR>2) print $1, "RX:", $3, "Errors:", $4}' ``` _exec_ ```bash netstat -i | grep -E "^eth0|^Iface" ``` _output_ ```bash Kernel Interface table eth0 1500 987654321 0 5 0 2345678 0 3 0 BMRU ``` Detailed analysis of per-interface statistics helps identify problematic network interfaces or links. - Any errors indicate potential network problems - Dropped packets suggest buffer overflow or network congestion - MTU mismatch can cause dropping - Monitor trends for early issue detection ### Continuous Monitoring Monitor network statistics and connections in real-time **Keywords:** monitoring, continuous, real-time, watch, trends #### Monitor network activity continuously ```bash # Monitor connections with continuous updates netstat -c # Monitor specific metric (using watch command) watch -n 1 'netstat -an | grep ESTABLISHED | wc -l' # Watch listening ports for changes watch -n 5 'netstat -tlnp' # Monitor interface counters watch -n 1 'netstat -i' # Track connection count trend for i in {1..5}; do echo "Count: $(netstat -an | grep ESTABLISHED | wc -l)"; sleep 1; done ``` _exec_ ```bash netstat -c --count=3 ``` _output_ ```bash Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 192.168.1.100:60123 203.0.113.5:443 ESTABLISHED tcp 0 128 192.168.1.100:22 192.168.1.50:55234 ESTABLISHED Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State ``` Continuous monitoring with netstat -c or watch reveals real-time network activity and trends. - -c flag causes netstat to refresh frequently - watch command provides nicer formatted output - Useful for performance testing and diagnostics - Can be combined with other tools for alerting #### Create monitoring scripts ```bash # Monitor and alert on connection count #!/bin/bash THRESHOLD=1000 while true; do COUNT=$(netstat -an | grep ESTABLISHED | wc -l) if [ $COUNT -gt $THRESHOLD ]; then echo "Alert: $COUNT established connections (threshold: $THRESHOLD)" fi sleep 60 done # Log connection statistics hourly #!/bin/bash while true; do echo "$(date): $(netstat -an | grep ESTABLISHED | wc -l) connections" >> connections.log sleep 3600 done ``` _exec_ ```bash netstat -an 2>/dev/null | grep ESTABLISHED | wc -l ``` _output_ ```bash 5 ``` Custom monitoring scripts can automate tracking and alerting for connection anomalies. - Scripts can monitor thresholds and alert - Useful for production system monitoring - Can log data for capacity planning - Integrate with monitoring systems **Best practices:** - Use -s flag for protocol statistics - Use -r flag to check routing configuration - Use -i flag to verify interface health - Monitor statistics periodically for trends **Common errors:** - **netstat -c produces no output**: The -c flag may not be available on all systems - **Can't understand routing table output**: Use -n flag for numeric addresses instead of hostnames ## Practical Examples and Troubleshooting Real-world use cases and troubleshooting techniques ### Find Process Using Port Identify which process is bound to a specific port **Keywords:** find, process, port, identify, troubleshooting #### Find process listening on specific port ```bash # Find what process is using port 8080 sudo netstat -tlnp | grep :8080 # Alternative: Use grep to extract process name PROCESS=$(sudo netstat -tlnp | grep :8080 | awk '{print $NF}') echo "Port 8080 is used by: $PROCESS" # Show complete process command line PID=$(sudo netstat -tlnp | grep :8080 | awk '{print $NF}' | cut -d/ -f1) ps aux | grep $PID # List multiple ports at once for port in 80 443 3306 5432; do echo -n "Port $port: " sudo netstat -tlnp | grep ":$port " | awk '{print $NF}' done ``` _exec_ ```bash sudo netstat -tlnp | grep 22 | head -1 ``` _output_ ```bash tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1234/sshd ``` Using grep with specific port number finds the listening process, showing PID and program name. - Requires sudo/root for -p flag - PID can be used with ps or ps aux for details - Useful for resolving port conflicts - Can be automated in scripts #### Kill process using specific port ```bash # Find and kill process on port 8080 PID=$(sudo netstat -tlnp | grep :8080 | awk '{print $NF}' | cut -d/ -f1) if [ ! -z "$PID" ]; then echo "Killing process $PID on port 8080" sudo kill -9 $PID else echo "No process found on port 8080" fi # One-liner to kill process on port sudo kill -9 $(sudo netstat -tlnp | grep :8080 | awk '{print $NF}' | cut -d/ -f1) # Graceful kill (SIGTERM first, then SIGKILL) PID=$(sudo netstat -tlnp | grep :8080 | awk '{print $NF}' | cut -d/ -f1) sudo kill $PID && sleep 2 && sudo kill -9 $PID 2>/dev/null ``` _exec_ ```bash sudo netstat -tlnp | grep -E ":80 |:443 " | head -2 ``` _output_ ```bash tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 5678/nginx tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 5678/nginx ``` Extracting PID from netstat output allows programmatic process termination when needed. - Always use graceful shutdown first (SIGTERM) - SIGKILL (-9) should be last resort - Terminate wrong process can cause data loss - Better to use service/systemctl commands when possible ### Monitor Port Changes Over Time Track changes to listening ports and active connections **Keywords:** monitor, changes, trending, baseline, comparison #### Track listening port changes ```bash # Create initial port list sudo netstat -tlnp > /tmp/ports_before.txt # After some time, compare sudo netstat -tlnp > /tmp/ports_after.txt diff /tmp/ports_before.txt /tmp/ports_after.txt # Monitor for new services appearing sudo netstat -tlnp | grep -v "LISTEN" | wc -l # Create baseline of listening services echo "=== Baseline at $(date) ===" >> port_baseline.log sudo netstat -tlnp | grep LISTEN >> port_baseline.log # Check for unexpected open ports sudo netstat -tlnp | grep -v "ssh\|sshd" ``` _exec_ ```bash sudo netstat -tlnp 2>/dev/null | wc -l ``` _output_ ```bash 8 ``` Comparing netstat snapshots over time helps detect unauthorized services or security changes. - Use cron to periodically capture port state - Baseline helps identify anomalies - Useful for security audits - Can trigger alerts on changes #### Monitor connection count trends ```bash # Log connection counts hourly */60 * * * * echo "$(date +\%Y-\%m-\%d\ \%H:\%M:\%S),$(netstat -an | grep ESTABLISHED | wc -l)" >> connections.csv # Script to collect statistics over time #!/bin/bash for i in {1..60}; do ESTABLISHED=$(netstat -an | grep ESTABLISHED | wc -l) TIME=$(date "+%Y-%m-%d %H:%M:%S") echo "$TIME: $ESTABLISHED" (( i < 60 )) && sleep 60 done > connection_trends.log # Analyze connection growth tail -50 connections.csv | awk -F',' '{print $NF}' | sort -n ``` _exec_ ```bash netstat -an 2>/dev/null | grep ESTABLISHED | wc -l ``` _output_ ```bash 8 ``` Continuous tracking of connection counts reveals usage patterns and capacity planning needs. - Use cron for automated periodic collection - Store in database or CSV for analysis - Identify peak times and growth trends - Use for performance tuning ### Diagnose Connection Hangs Identify and analyze stuck or hanging connections **Keywords:** diagnose, hang, stuck, issue, debugging #### Identify hanging connections ```bash # Show connections not in ESTABLISHED or LISTEN state netstat -an | grep -v ESTABLISHED | grep -v LISTEN | grep -E "^tcp|^udp" # Show CLOSE_WAIT connections (remote closed but local open) netstat -an | grep CLOSE_WAIT # Show FIN_WAIT connections (waiting for close to complete) netstat -an | grep FIN_WAIT # Identify which process has hanging connections sudo netstat -anp | grep CLOSE_WAIT | awk '{print $NF}' | sort -u # Check for TIME_WAIT accumulation (possible port exhaustion) netstat -an | grep TIME_WAIT | wc -l ``` _exec_ ```bash sudo netstat -anp | grep -E "CLOSE_WAIT|FIN_WAIT" | head -3 ``` _output_ ```bash tcp 0 0 192.168.1.100:55234 203.0.113.100:3306 CLOSE_WAIT 5678/app ``` Analyzing non-standard states reveals connections stuck in cleanup or waiting states. - CLOSE_WAIT indicates application bug - FIN_WAIT is normal but excessive amount unusual - High TIME_WAIT can indicate port exhaustion risk - Fix applications to properly close connections #### Troubleshoot connection timeouts ```bash # Test connectivity to specific host nc -zv 203.0.113.50 443 # Check if port is accessible via netstat netstat -an | grep 203.0.113.50 # Verify listening port with specific binding sudo netstat -tlnp | grep ":8080" # Check for firewall issues (look for many SYN_SENT) netstat -an | grep SYN_SENT | head -5 # Trace connection attempt steps sudo tcpdump -i any -n 'port 8080' & # Then attempt connection in another terminal # Use Ctrl+C to stop tcpdump ``` _exec_ ```bash netstat -an | grep SYN_SENT | wc -l ``` _output_ ```bash 0 ``` Diagnosing timeouts involves checking connection states and using tools like nc for verification. - SYN_SENT indicates connection attempt - Many SYN_SENT from one source suggests blocked port - Check firewall rules with iptables or firewall-cmd - Use tcpdump for packet-level debugging ### Count Connections by State Analyze connection distribution and statistics **Keywords:** count, statistics, distribution, analysis, reporting #### Count connections by state ```bash # Show count of each connection state netstat -an | tail -n +3 | awk '{print $NF}' | sort | uniq -c | sort -rn # Show only unique states and their counts netstat -an | grep -o -E "LISTEN|ESTABLISHED|TIME_WAIT|CLOSE_WAIT" | sort | uniq -c # Count established connections netstat -an | grep ESTABLISHED | wc -l # Count listening sockets netstat -an | grep LISTEN | wc -l # Show breakdown with nice formatting echo "Connection Statistics ($(date))" echo "================================" netstat -an | tail -n +3 | awk '{s[$NF]++} END {for (state in s) print state, s[state]}' | sort -k2 -rn ``` _exec_ ```bash netstat -an 2>/dev/null | tail -n +3 | awk '{s[$NF]++} END {for (state in s) print state, s[state]}' | sort -k2 -rn ``` _output_ ```bash LISTEN 5 ESTABLISHED 3 TIME_WAIT 8 ``` Aggregating connections by state provides overview of network health and connection patterns. - High LISTEN count is normal - High ESTABLISHED indicates heavy usage - High TIME_WAIT is normal cleanup - Any CLOSE_WAIT indicates application issues #### Generate connection reports ```bash # Generate connection report #!/bin/bash echo "Network Connection Report - $(date)" echo "======================================" echo "" echo "Total connections:" netstat -an | tail -n +3 | wc -l echo "" echo "Connections by state:" netstat -an | tail -n +3 | awk '{s[$NF]++} END {for (state in s) printf " %-15s %d\n", state, s[state]}' | sort -k2 -rn echo "" echo "Top ports in use:" netstat -an | tail -n +3 | awk '{print $4}' | cut -d: -f2 | sort | uniq -c | sort -rn | head -5 echo "" echo "Connection by protocol:" netstat -an | tail -n +3 | awk '{print $1}' | sort | uniq -c # Simpler version netstat -an | tail -n +3 | awk '{print $1}' | sort | uniq -c ``` _exec_ ```bash netstat -an | tail -n +3 | awk '{print $1}' | sort | uniq -c ``` _output_ ```bash 5 tcp 2 tcp6 3 udp ``` Comprehensive reporting provides visibility into network usage and health metrics. - Useful for documentation and audits - Can be scheduled via cron for regular reports - Export to files for archival - Share with team or stakeholders ### Check for Zombie Connections Identify and clean up orphaned or zombie connections **Keywords:** zombie, orphaned, cleanup, maintenance, monitoring #### Detect zombie and orphaned connections ```bash # Show CLOSE_WAIT connections (possible zombies) netstat -an | grep CLOSE_WAIT | wc -l # Show LAST_ACK connections (connection cleanup) netstat -an | grep LAST_ACK | wc -l # Show all non-standard connection states netstat -an | grep -v -E "LISTEN|ESTABLISHED|^Proto|^Active" # Identify which application has most CLOSE_WAIT sudo netstat -anp | grep CLOSE_WAIT | awk '{print $NF}' | cut -d/ -f2 | sort | uniq -c | sort -rn # Monitor CLOSE_WAIT trend for i in {1..5}; do echo "Check $i: $(netstat -an | grep CLOSE_WAIT | wc -l) CLOSE_WAIT connections" sleep 10 done ``` _exec_ ```bash netstat -an 2>/dev/null | grep CLOSE_WAIT | wc -l ``` _output_ ```bash 0 ``` Zombie/orphaned connections accumulate when applications don't properly close sockets. - CLOSE_WAIT indicates application bug - Not system issue - application must close properly - TIME_WAIT is normal and automatic - Monitor trends to identify problem apps #### Clean up zombie connections ```bash # Show process with CLOSE_WAIT connections sudo netstat -anp | grep CLOSE_WAIT # Restart problematic application (best approach) # sudo systemctl restart service-name # Force close connections if necessary (not recommended) # This forces the process to drop connections # sudo kill -9 # Monitor after restart watch -n 5 'sudo netstat -anp | grep CLOSE_WAIT | wc -l' # Create alert script CLOSE_WAIT=$(netstat -an | grep CLOSE_WAIT | wc -l) if [ $CLOSE_WAIT -gt 50 ]; then echo "Alert: $CLOSE_WAIT CLOSE_WAIT connections detected" # Send alert, restart app, etc fi ``` _exec_ ```bash sudo netstat -anp 2>/dev/null | grep CLOSE_WAIT | wc -l ``` _output_ ```bash 2 ``` Cleaning up zombie connections requires fixing the application code or restarting the service. - Restarting application is preferred solution - Killing process is last resort - Fix application code to proper close sockets - Implement connection pooling and cleanup **Best practices:** - Regularly monitor netstat output for anomalies - Create baseline snapshots for comparison - Use process identification (-p flag) to find culprits - Combine with other tools (grep, awk, watch) for analysis - Document findings and share with team **Common errors:** - **Grep pattern doesn't match expected output**: Check netstat output format with cat, adjust grep pattern - **Can't kill process (permission denied)**: Use sudo or run as appropriate user