[arch-general] HTTP spam from China - CIDR compacting tool

[brent s.]

On 2/26/19 1:20 PM, Genes Lists via arch-general wrote:
> On 2/26/19 1:13 PM, Juha Kankare via arch-general wrote:
>> On 26/02/2019 20:11, Genes Lists via arch-general wrote:
> ...
>>
>> My current script is just pulling cn.zone from ipdeny.com. This looks 
>> super useful, I'm saving it. Thank you dude!
>>
> You're welcome.
> 
> I just ran it on cn.zone and it reduces the number of lines from 8,337
> to 5,120. It can make a significant difference.
> 
> best,
> 
> gene
> 

Just to +1 what Gene has said, I've taken similar approaches to
compacting into CIDRs and it really does make a significant difference.

For clarification on his ipset[0] point, I also have to strongly
recommend it. It not only *greatly* simplifies your ruleset, but it can
be dynamically altered without needing to reload your firewall rules.

e.g. assuming you have an IP set named "china_ips",

-A INPUT -m set --match-set china_ips src -p tcp -m tcp --dport 80 -j DROP

will drop traffic for all those entries. You've then simplified many
(MANY) rules to one. :)

You can (Gene, you may find this particularly useful since you feed to
ipset) use the pyroute2.IPSet() function to actually manage the live
kernel's ipsets as well. Make sure your running kernel and latest
installed kernel match, otherwise you'll need to reboot so the ipset
kernel module can be loaded.

Untested, but should be pretty darn close if not functional:

#####
import subprocess
import pyroute2


# (...)
ipset = pyroute2.IPSet()
setsfile = '/etc/ipset.conf'
setname = 'china_ips'
tmpset = '{0}_TMP'.format(setname)

set_exists = False
try:
    # Check to see if the list exists.
    ipset.headers(setname)
    # list is done here as a quick-and-dirty sanity/exception check,
    # which is why it's in both the try and exception.
    setlist = ipset.list(name = setname)
    set_exists = True
except pyroute2.ipset._IPSetError:
    ipset.create(setname, stype = 'hash:net')
    setlist = ipset.list(name = setname)

# We use a temporary set so we don't affect any current iptables
# processing. Most likely unnecessary, but better safe than sorry.
try:
    ipset.destroy(name = tmpset)
except pyroute2.ipset._IPSetError:
    # It doesn't exist (yet), which is what we want.
    pass

# Create the temporary set
ipset.create(tmpset, stype = 'hash:net')
for n in set1.iter_cidrs():  # "set1" is from Gene's script
    ipset.add(tmpset, n)

# Make the temporary set live
ipset.swap(setname, tmpset)

# And cleanup the now-unnecessary tmpset
ipset.destroy(name = tmpset)

# Save them to the persistent file so it's applied on a reboot.
# Remember to "systemctl enable ipset.service".
# Unfortunately, there isn't a built-in save function.
# You could easily write your own iterator/generator, though,
# if you want to avoid a subprocess call.
# The syntax is pretty simple.
with open(setsfile, 'w') as f:
    ipset_cfg = subprocess.run(['/usr/bin/ipset',
                                'save'],
                               stdout = f)
# DONE.
#####





[0] https://wiki.archlinux.org/index.php/Ipset
-- 
brent saner
https://square-r00t.net/
GPG info: https://square-r00t.net/gpg-info

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 899 bytes
Desc: OpenPGP digital signature
URL: <https://lists.archlinux.org/pipermail/arch-general/attachments/20190226/8206165e/attachment-0001.sig>