Overview

The SLS/EDW ports in all Sensage AP releases prior to 2017.4 are sensitive to vulnerability scanners and other external processes and systems. When invalid HTTP requests are received on the SLS ports, it can crash with the following segfault:

sensage_sls_pro[90543]: segfault at 0 ip 0000003736928628 sp 00007ffc13d611b8 error 4 in libc-2.12.so[3736800000+18a000] 

Solution

This is a known issue (SNSGCID-44515: EDW instance dies when called with an invalid URL) that has been fixed on Sensage AP 2017.4 and later releases.  The recommended solution is to upgrade to the latest Sensage AP versions. However, if an upgrade is not feasible in your environment, this article provides a workaround to prevent the SLS cluster from crashing on security vulnerability scans. 

Workaround

Prerequisite: Root access to Sensage AP to apply iptables rules.

The workaround is to restrict communication on SLS ports 8072 (and 8073, and so on, if it's a multi-instance system) only to the Sensage AP cluster server IPs for Analyzer, Load Collector, and SLS nodes, blocking any other communication from external systems trying to access the SLS ports (such as vulnerability scanners or external checks). This will safeguard against invalid requests being received on the SLS nodes which could cause the instances to crash. 

The suggested iptables ruleset will restrict communications to the SLS ports only to the internal Sensage AP nodes, blocking any external communication:

iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -I INPUT -p tcp -s 0.0.0.0/0 --dport 8072 -j DROP 
iptables -I INPUT -p tcp -s 127.0.0.1 --dport 8072 -j ACCEPT
iptables -I INPUT -p tcp -s <SLS node 1 IP> --dport 8072 -j ACCEPT
iptables -I INPUT -p tcp -s <SLS node 2 IP> --dport 8072 -j ACCEPT
iptables -I INPUT -p tcp -s <Analyzer node IP> --dport 8072 -j ACCEPT
iptables -I INPUT -p tcp -s <Collector node 1 IP> --dport 8072 -j ACCEPT
iptables -I INPUT -p tcp -s <Collector node 2 IP> --dport 8072 -j ACCEPT

This needs to be configured on all the Sensage AP cluster nodes, using this procedure:

1. Make sure the iptables is empty with the service iptables status command. An empty or blank iptables configuration show looks like this:

   # service iptables status
   Table: filter
   Chain INPUT (policy ACCEPT)
   num  target     prot opt source               destination
   
   Chain FORWARD (policy ACCEPT)
   num  target     prot opt source               destination
   
   Chain OUTPUT (policy ACCEPT)
   num  target     prot opt source               destination
   
   Table: nat
   Chain PREROUTING (policy ACCEPT)
   num  target     prot opt source               destination
   
   Chain POSTROUTING (policy ACCEPT)
   num  target     prot opt source               destination
   
   Chain OUTPUT (policy ACCEPT)
   num  target     prot opt source               destination

2. If the iptables is not blank, you can flush the rules with the iptables -F command, then check again with the service iptables status command.
3. Once theiptables status is blank, add all the rules as follows on the command line (you can copy from here and paste them directly on the command line). This will first deny all external traffic to SLS ports and then allow the specific Sensage Cluster IPs to connect to the SLS ports. The set of rules will be the same for all SLS nodes: 

   iptables -I INPUT -p tcp -s 0.0.0.0/0 --dport 8072 -j DROP 
   iptables -I INPUT -p tcp -s 127.0.0.1 --dport 8072 -j ACCEPT
   iptables -I INPUT -p tcp -s <SLS node 1 IP> --dport 8072 -j ACCEPT
   iptables -I INPUT -p tcp -s <SLS node 2 IP> --dport 8072 -j ACCEPT
   iptables -I INPUT -p tcp -s <Analyzer node IP> --dport 8072 -j ACCEPT
   iptables -I INPUT -p tcp -s <Collector node 1 IP> --dport 8072 -j ACCEPT
   iptables -I INPUT -p tcp -s <Collector node 2 IP> --dport 8072 -j ACCEPT

   If your cluster has multiple SLS instances, the rules need to be added for all SLS instances (i.e. with port 8073, 8074, and so on. Just copy the same rules and replace the port numbers.)
4. Run the service iptables status command again. The output should be similar to this: 

   # service iptables statusTable: filter
   Chain INPUT (policy ACCEPT)
   num  target     prot opt source               destination
   1   ACCEPT     tcp  --  <Collector node 2 IP> 0.0.0.0/0           tcp dpt:8072
   2   ACCEPT     tcp  --  <Collector node 1 IP> 0.0.0.0/0           tcp dpt:8072
   3   ACCEPT     tcp  --  <Analyzer node IP>    0.0.0.0/0           tcp dpt:8072
   4   ACCEPT     tcp  --  <SLS node 2 IP>       0.0.0.0/0           tcp dpt:8072
   5   ACCEPT     tcp  --  <SLS node 1 IP>       0.0.0.0/0           tcp dpt:8072
   6   ACCEPT     tcp  --  127.0.0.1             0.0.0.0/0           tcp dpt:8072
   7   DROP       tcp  --  0.0.0.0/0             0.0.0.0/0           tcp dpt:8072
   
   Chain FORWARD (policy ACCEPT)
   num  target     prot opt source               destination
   
   Chain OUTPUT (policy ACCEPT)
   num  target     prot opt source               destination
   
   Table: nat
   Chain PREROUTING (policy ACCEPT)
   num  target     prot opt source               destination
   
   Chain POSTROUTING (policy ACCEPT)
   num  target     prot opt source               destination
   
   Chain OUTPUT (policy ACCEPT)
   num  target     prot opt source               destination

5. Save the rules with the service iptables save command:
   

   # service iptables save
   iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]

6. Run the Sensage atview ... tasks or atview ... tables  command from the Analyzer and Load Collector nodes to validate that the SLS is reachable from within the cluster and is operating as normal.
7. Ensure the iptables service is on. This will ensure the iptables rules will be applied upon a reboot of the server: 

   # chkconfig iptables on
   # chkconfig --list | grep iptables
   iptables        0:off   1:off   2:on    3:on    4:on    5:on    6:off
   sensage_iptables        0:off   1:off   2:on    3:on    4:on    5:on    6:off

Testing

Access the SLS ports using telnet from an external computer. SLS ports won't respond to requests from other than the authorized IP addresses anymore.  Legitimate traffic from within the SLS won't crash the cluster as all requests will be well-formed and valid.

Important note: Any change to the Sensage Cluster like an SLS expansion or IP address change would require redoing this procedure modifying the rules accordingly to reflect the changes and maintain consistency.

Content Author: Miguel Molina