iptables-converter

iptables-converter: convert iptables to iptables-save format, output comes with [0:0] for iptables-restore -c

ip6tables-converter: convert ip6tables to ip6tables-save format, output comes with [0:0] for ip6tables-restore -c

Source: https://github.com:sl0/conv.git

Version: 0.9.10

Date: 2017-11-16

Licenses:

• GNU GENERAL PUBLIC LICENSE Version 3 later
• Apache License Version 2.0

Author: Johannes Hubertz johannes@hubertz.de

iptables-converter speeds up loading of iptables-commands by converting them to iptables-save format, and then loading them through iptables-restore is much more quicker than loading the plain commands. The loading itself is not part of iptables-converter.

Usage:

iptables-converter [ -d destination ] [ -s source ] [ --sloppy ]
ip6tables-converter [ -d destination ] [ -s source ] [ --sloppy ]

This assumes that source is a plain ascii-file containing lines starting with iptables to build a firewall ruleset. Lines starting with /sbin/iptables are understood as well. Omitting -s source defaults to reading a file named rules. An optional sloppy parameter makes premature definitions (-N name) of any user defined chains unneccessary, they are defined automatically by first mentioning them. Optionally -d destination writes everything into the given destination file since verstion 0.9.10. Omitting this option results in writing Output to stdout, which is the default behavior. Packet-counters and byte-counters include [0:0] which keeps compatibility to iptables-restore as well as to iptables-restore -c.

ip6tables-converter works for ip6tables just the same way.

Both they work for filter, mangle, nat and raw tables, security tables are not supported for now.

From version 0.9.10 on it works as a python-module using entry-points for easier imports. For your convienience, the module is named iptables_conv.

At travis-ci.org the tests are run automatically, thanks to Guido! To run them locally, please use python:

python setup test

or tox, which is the preferred testrunner. iptables-converter is tested to work well with python2.7, python3.5 and python3.6. The tests are done using pytest for easier writing future testcases. Some sphinx documentation is prepared. Debian packages are provided for the binaries and sphinx-documentation. git-buildpackage creates them on the fly. RPMs may be created by python:

setup.py bdist_rpm

Any comments welcome.

Have fun!

Johannes

iptables-converter - description

In linux iptables exists since kernel version 2.4.

A corresponding userland command iptables is used to control them.

The tables are grouped in filter, nat, mangle and raw. In every group there are chains, which contain more or less traffic specific rules. Predefined in filter tables are INPUT, FORWARD and OUTPUT chain. Perhaps you like to read the fine manuals about the iptables command in Linux to get to know more about these.

Usually a systemadministrator wants to write his rules in a shell script, which is run within the boot sequence or every now and then on every change… Over the time this script will grow. The longer it gets, the more time takes it to complete. Exactly this is the reason, why iptables-converter was written. Only motivation was to speed up the loading of long scripts with multiple iptables commands.

The iptables-converter is a pure python script with two entrypoints:

• iptables-converter
• ip6tables-converter

They reside in linux userland, that means to run one of them you don’t need root priviledges, which are only needed to load the output of the converters into the kernelspace.

Command line interface

To have an idea which command line options are supported, just ask it for help:

$ iptables-converter --help
Usage:  iptables-converter --help | -h

        iptables-converter: version 0.9.10       Have Fun!

Options:
  -h, --help     show this help message and exit
  -d DESTFILE    output filename, default: stdout
  -s SOURCEFILE  file with iptables commands, default: rules
  --sloppy       -N name-of-userchain is inserted automatically, by default -N
                 is neccessary in input

ip6tables-converter surprisingly behaves exactly the same way, except from the ‘6’ in the command and version line:

$ ip6tables-converter --help
Usage:  ip6tables-converter --help | -h

        ip6tables-converter: version 0.9.10       Have Fun!

Options:
  -h, --help     show this help message and exit
  -d DESTFILE    output filename, default: stdout
  -s SOURCEFILE  file with iptables commands, default: rules
  --sloppy       -N name-of-userchain is inserted automatically, by default -N
                 is neccessary in input

The only difference in between them is what is looked at. iptables-converter just handles lines starting with iptables or /sbin/iptables, ip6tables-converter only lines starting with ip6tables or /sbin/ip6tables.

DEST- and SOURCEFILE

These should be clear, the default values are build in for your convenience only. A script generating iptables command writes them into a file named rules, which is then read by the converter, if the -s option is not used. The converters ouptut is then written to stdout, if the -d option is not used. So it might be piped or somehow else processed.

–sloppy

The option –sloppy perhaps needs some explanation. Usually the iptables command insists on defined chains, especially you cannot insert or append a rule into a non-existent user defined chain. Especially the -N UserChain is needed normally in advance to the append operation. Inserting into an empty chain is forbidden as well. By using the –sloppy option this -N command is not needed in the input for the converter as it defines the UserCHain automatically on the first occurance of any.

Default operating

Assume a plain file with following contents:

iptables -F
iptables -t nat -F
iptables -N USER_CHAIN
iptables -A INPUT -p tcp --dport 23 -j ACCEPT
iptables -A USER_CHAIN -p icmp -j DROP
iptables -P INPUT DROP
iptables -t nat -A POSTROUTING -s 10.0.0.0/21 -p tcp --dport   80 -j SNAT --to-source 192.168.1.15
iptables -t nat -A PREROUTING  -d 192.0.2.5/32 -p tcp --dport 443 -j DNAT --to-destination 10.0.0.5:1500

As times goes by, this script will grow. The more lines it has, the longer will it take to be loaded. This is because every iptables statement needs to modify the kernels iptables as an atomar operation, which is a lot of overhead from locking the tables, modifying and unlocking them. There should be a quicker way of getting things done. Using iptables-save we can save easily the actual ruleset from the kernel to a file. To load it’s content into the kernel again is a very quick action compared to the loading of the originating shellscript. The iptables-restore operation is one atomar operation from the kernels view regardless of the number of modifications. It’s clear to be the much quicker the more lines are covered. The disadvantage of this proceeding is, the table, f.e. the filter tables, are loaded at once, i.e. no appending or inserterting using the same command is possible. You only need a complete set of iptable commands within a file, just like iptables-save gives it. So the idea came up to have a converter just for saving time.

Lets assume, the file shown above is named generated-rules, then we have easy going:

$ iptables-converter -s generated-rules -d converted-rules
$ cat converted-rules
*raw
:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
COMMIT
*nat
:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -d 192.0.2.5/32 -p tcp --dport 443 -j DNAT --to-destination 10.0.0.5:1500
-A POSTROUTING -s 10.0.0.0/21 -p tcp --dport 80 -j SNAT --to-source 192.168.1.15
COMMIT
*mangle
:FORWARD ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT
*filter
:FORWARD ACCEPT [0:0]
:INPUT DROP [0:0]
:USER_CHAIN - [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p tcp --dport 23 -j ACCEPT
-A USER_CHAIN -p icmp -j DROP
COMMIT
$

On the same machine or after beeing transferred to another one, the converted-rules file can be loaded into the kernel by using the command iptables-restore as root user:

# iptables-restore -c converted-rules

Of course you use pathnames where filenames are mentioned.

Usage example

So you probably may want to run the converter from within a shell script or the like:

#!/bin/bash

set -e
INPUT_FILE=rules
OUTPUT_FILE=iptables-converted

# needs to be executable as indicator that writing has ended
[ ! -r $INPUT_FILE ] && exit 0
[ ! -x $INPUT_FILE ] && exit 0

iptables-converter.py -s $INPUT_FILE -d $OUTPUT_FILE

# do it only once!
mv $INPUT_FILE $INPUT_FILE}.old

iptables-restore < $OUTPUT_FILE
echo "$INPUT_FILE successfully converted and loaded"
exit 0
# EoF

Error handling

In accidential cases of errors the converter should give you a traceback wherin the word ConverterError appears. This is to let you get to know, where in your whole programming universe the error happend.

Two things can not be handled: Shell functions and shell variables, because the converter does not interpret your input-file.

Shell functions and shell commands

As the file which is read is not interpreted in any way, there are few known error conditions:

1. the file contains some shell variables, indicated by ‘$’, this leads to an errormessage and exits immediately with returncode 1.
2. the file contains some shell functions, indicated by ‘(‘ and/or ‘)’, this leads to an errormessage and exits immediately with returncode 1.

If you have such a file, and you want to speed up by converting, please execute it and feed the output as a file to iptables-converter.

Non existent user chains

iptables-converter does some more error-checking while reading input.

Normal behavior is to raise an ConverterError, if any append or insert statement to an userdefined chain is not preceeded by a corresonding chain creation statement ‘-N’. This may be changed to a more smooth handling with an additional commandline option –sloppy. Having this, a non existent userchain is created on the fly when the first append statement is seen. So it is set as first entry gracefully.

Inserting into an emtpy chain anyhow raises an error as iptables-restore would do it later on trying to set the files content into the kernel.

Not implemented

Just to mention it: iptables -E xyz and iptables -L are not implemented in the iptables-converter and throw exceptions for now!

iptables-converter - tests

Untested software, that means software which isn’t accompanied by automated functional tests, is assumed to be broken by design. As iptables-converter is written in python, use of the popular unittests is done for your convienience. The unittests were developed and run by nose, which later have been replaced by pytest. The advantages of pytest over nose are much simpler tests and less overhead. So all newer testcases are written for pytest and thus rely on the plain python assert statement.

Two testclasses are build: Chains_Test and Tables_Test accordingly to the two classes from which the iptables-converter module and script is build from.

Basic usage

Lets see, how to run the tests within the source-tree by just calling pytest:

$ pytest
============================= test session starts ==============================
platform linux -- Python 3.6.3, pytest-3.2.3, py-1.4.34, pluggy-0.4.0
rootdir: /home/hans/devel/conv, inifile:
plugins: cov-2.5.1
collected 28 items

tests/test_iptables_converter.py ............................
tests/test_iptables_options.py .......

========================== 35 passed in 0.08 seconds ===========================
$

All tests passed. Fine.

Verbose usage

If you want to see more, just add a -v to the commandline:

$ pytest -v
================================== test session starts ==================================
platform linux -- Python 3.6.3, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 -- /home/hans/wb/bin/python3.6
cachedir: .cache
rootdir: /home/hans/devel/conv, inifile:
plugins: cov-2.5.1
collected 35 items

tests/test_iptables_converter.py::Chains_Test::test_01_create_a_chain_object PASSED
tests/test_iptables_converter.py::Chains_Test::test_02_prove_policies PASSED
tests/test_iptables_converter.py::Chains_Test::test_03_tables_names PASSED
tests/test_iptables_converter.py::Chains_Test::test_04_flush PASSED
tests/test_iptables_converter.py::Chains_Test::test_05_new_chain PASSED
tests/test_iptables_converter.py::Chains_Test::test_06_new_existing_chain_fails PASSED
tests/test_iptables_converter.py::Chains_Test::test_07_insert_rule_fail PASSED
tests/test_iptables_converter.py::Chains_Test::test_08_insert_rule_fail PASSED
tests/test_iptables_converter.py::Chains_Test::test_09_insert_rule_works PASSED
tests/test_iptables_converter.py::Chains_Test::test_10_append_rule PASSED
tests/test_iptables_converter.py::Chains_Test::test_11_remove_predef_chain PASSED
tests/test_iptables_converter.py::Chains_Test::test_12_remove_chain PASSED
tests/test_iptables_converter.py::Chains_Test::test_13_illegal_command PASSED
tests/test_iptables_converter.py::Tables_Test::test_01_create_a_tables_object PASSED
tests/test_iptables_converter.py::Tables_Test::test_02_nat_prerouting PASSED
tests/test_iptables_converter.py::Tables_Test::test_03_mangle_table PASSED
tests/test_iptables_converter.py::Tables_Test::test_04_raw_table PASSED
tests/test_iptables_converter.py::Tables_Test::test_05_not_existing_chain PASSED
tests/test_iptables_converter.py::Tables_Test::test_06_read_not_existing_file PASSED
tests/test_iptables_converter.py::Tables_Test::test_07_read_empty_file PASSED
tests/test_iptables_converter.py::Tables_Test::test_08_reference_one PASSED
tests/test_iptables_converter.py::Tables_Test::test_09_shell_variables PASSED
tests/test_iptables_converter.py::Tables_Test::test_10_shell_functions PASSED
tests/test_iptables_converter.py::Tables_Test::test_11_reference_sloppy_one PASSED
tests/test_iptables_converter.py::Tables_Test::test_12_create_a_tables6_object PASSED
tests/test_iptables_converter.py::Tables_Test::test_13_re6ference_one PASSED
tests/test_iptables_converter.py::Tables_Test::test_14_re6ference_sloppy_one PASSED
tests/test_iptables_converter.py::test_15_tables_printout PASSED
tests/test_iptables_options.py::test_01_iptables_converter_option_h PASSED
tests/test_iptables_options.py::test_02_iptables_converter_option_s PASSED
tests/test_iptables_options.py::test_03_iptables_converter_option_d PASSED
tests/test_iptables_options.py::test_03_iptables_converter_option_sd PASSED
tests/test_iptables_options.py::test_09_iptables_converter_main_dlft PASSED
tests/test_iptables_options.py::test_10_iptables_converter_main_ok PASSED
tests/test_iptables_options.py::test_11_iptables_converter_main_write PASSED

=============================== 35 passed in 0.05 seconds ===============================
$

Code coverage

If you want to get to know something about the test-coverage, just give pytest a try:

$ pytest --cov=iptables_conv --cov-report=term-missing
==================================== test session starts ====================================
platform linux -- Python 3.6.3, pytest-3.2.3, py-1.4.34, pluggy-0.4.0
rootdir: /home/hans/devel/conv, inifile:
plugins: cov-2.5.1
collected 28 items

tests/test_iptables_converter.py ............................

----------- coverage: platform linux, python 3.6.3-final-0 -----------
Name                                  Stmts   Miss  Cover   Missing
-------------------------------------------------------------------
iptables_conv/__init__.py                 8      0   100%
iptables_conv/iptables_converter.py     230      7    97%   28-29, 72, 129-131, 314
-------------------------------------------------------------------
TOTAL                                   238      7    97%




================================= 28 passed in 0.08 seconds =================================
$

If you like to have a look into the sources, you will find the tests directory. Therein all the tests reside. I hope they are self explaining.

testrunner

To simply run the tests, setup.py has a test target:

$ python setup.py test
   ...
$

This runs flake8 and pytest. If you prefer less typing:

$ pytest
   ...
$

Or, possibly the best way of doing is the following super power.

Check tests, syntax and style

For your convenience, a tox.ini is prepared. Give tox a try to check altogether in one single run:

• python2.7
• python3.5
• python3.6
• flake8
• docs

iptables-converter - pythonic view

module: iptables_conv

class iptables_conv.iptables_converter.ConverterError

class iptables_conv.iptables_converter.Chains(name, tables, sloppy=False)

chains are grouped in iptables

Parameters:

• name (str) – chain group name, ‘filter’, ‘nat’, …
• tables (list) – list of chains
• sloppy (bool) – needs ‘-N’(default) or not

Returns:

object representing chain group

Return type:

Chains

Raises:

ConverterError – on some illegal conditions

put_into_fgr(content)

fill this line into this tabular

Parameters:content (str) – one line of inputfile Returns:None Raises:ConverterError – on some illegal conditions

reset()

action method for iptables -F

class iptables_conv.iptables_converter.Tables(destfile, sourcefile='reference-one', sloppy=False, ipversion=4)

some chaingroups in tables are predef: filter, nat, mangle, raw

Parameters:

• destfile (str) – which file or pathname is to be written
• sourcefile (str) – which file or pathname is to be read
• sloppy (bool) – ‘-N’ is needed(default) or not
• ipversion (int) – 4(default) or 6

Returns:

Tables after read sourcefile written into destfile

Return type:

Tables

Raises:

ConverterError – on some illegal conditions

put_into_tables(line)

put line into matching Chains-object

Parameters:line – one line of inputfile Returns:None

read(file_descriptor)

read data from file like object into Tables-object

Parameters:file_descriptor – filedescriptor of file to be read Returns:None Raises:ConverterError – on some illegal conditions

read_file(sourcefile)

open file or error

Parameters:sourcefile – file or pathname of file to be read Returns:file_descriptor Raises:ConverterError – on IOError

reset(sourcefile, ipversion)

all predefined Chains aka lists are setup as new here

Parameters:

• sourcefile (str) – file to be read
• ipversion (int) – 4 or 6

Returns:

None

table_printout()

printout nonempty tabulars in fixed sequence

module: tests

From 0.9.10 on all future tests will be run by pytest. Prior tests are written to use the unittest module.

class tests.test_iptables_converter.Chains_Test(methodName='runTest')

some tests for class Chain

test_01_create_a_chain_object()

Chain 01: create a Filter group, f.e. filter

test_02_prove_policies()

Chain 02: check 3 valid policies, 1 exception

test_03_tables_names()

Chain 03: 3 cases OK, 1 Exception

test_04_flush()

Chain 04: flush filter group, 2 rules and an invalid chain

test_05_new_chain()

Chain 05: create a new chain in filtergroup,

test_06_new_existing_chain_fails()

Chain 06: create an exsiting chain should fail

test_07_insert_rule_fail()

Chain 07: insert a rule into an empty chain fails

test_08_insert_rule_fail()

Chain 08: insert a rule into a non_existing chain fails

test_09_insert_rule_works()

Chain 09: insert a rule into a nonempty chain works at start

test_10_append_rule()

Chain 10: append a rule to a chain

test_11_remove_predef_chain()

Chain 11: try to remove a prefined chain

test_12_remove_chain()

Chain 12: try to remove an existing chain

test_13_illegal_command()

Chain 13: try an ilegal command

class tests.test_iptables_converter.Tables_Test(methodName='runTest')

Tables: some first tests for the class

test_01_create_a_tables_object()

Tables 01: create a Tables object, check chains

test_02_nat_prerouting()

Tables 02: nat PREROUTING entry

test_03_mangle_table()

Tables 03: mangle INPUT entry

test_04_raw_table()

Tables 04: raw OUTPUT entry

test_05_not_existing_chain()

Tables 05: INPUT to not existing chain

test_06_read_not_existing_file()

Tables 06: read non existing file

test_07_read_empty_file()

Tables 07: read empty file (in relation to iptables-commands)

test_08_reference_one()

Tables 08: read default file: reference-one, check chains

test_09_shell_variables()

Tables 09: read buggy file with shell variables

test_10_shell_functions()

Tables 10: read buggy file with shell functions

test_11_reference_sloppy_one()

Tables 11: read sloppy input file: reference-sloppy-one, check chains

test_12_create_a_tables6_object()

Tables 12: create an ipv6 Tables object, check chains

test_13_re6ference_one()

Tables 13: read default file: re6ference-one, check chains

test_14_re6ference_sloppy_one()

Tables 14: read sloppy input file: re6ference-sloppy-one, check chains

tests.test_iptables_converter.test_15_tables_printout(capsys)

Tables 15: check table_printout as well

Contents:

Indices and tables

• Index
• Module Index
• Search Page