[Discuss] firewalld rant

[me at mattgillen.net (Matthew Gillen)]

Maybe I'm getting cranky in my old age, but after giving up on my
manually maintained iptables scripts that were 20 years old and trying
to build everything in firewalld (and running mostly successfully for a
couple years), I'm considering going back.  I'm writing this to
hopefully keep you from hurting your head from banging it on the wall if
you're trying to do advanced things.

Firewalld does not appear to be meant to mediate between networks, other
than the trivial home networking case of masquerading an internal net.
Everything described below was on Fedora32 / Firewalld 0.8

I've got a linux box that is my primary firewall and infrastructure
services (dhcp, dns, ntp, etc).

I've got a slew of VLANs on the internal side that all go into that
firewall, the idea being that this main box can control inter-connects
between the VLANs with layer 3 (IP) filtering.  For instance, I don't
want my skanky tv or playstation to be able to see the insecure services
I'm running for trusted clients (smb/nfs, etc), but they need to be able
to get to the internet and use DNS, DHCP.

If it was just about interacting with the server box itself combined
with having these internal zones masquerade out the external interface,
I'm sure I'd be perfectly happy.  My problem was that clients in one
zone needed to talk to clients in another zone.

Being the paranoid penguin I am, I have some Chinese PoE IP cameras that
are great except I don't want them to talk to the outside world at all.
To interact with them you need to be on the internal net or pretend like
you are (i.e. VPN / ssh port forward).  Likewise with my network
hardware running stock firmware that keeps wanting to reach out to some
cloud service.  Shouldn't be to hard, right?  Side note: everything was
working great until a distro-upgrade to the next major version made
zones unable to talk to each other. Unclear if this was a behavior
change in firewalld from a new version, or if a bit got flipped in the
config during upgrade.

I tried to do this with 4 zones: external, desktop, restricted,
internal-only.  Needed a direct rule to force the internal-only traffic
that wasn't destined for my internal net to be dropped. So far so good.

Side note in ambiguous documentation: check out the "masquerade" option
https://firewalld.org/documentation/man-pages/firewalld.zone.html :
  ..."If it's present masquerading is enabled."
no indication of which interface it should be set on (the internal or
external; the answer is you set that option on the interface you want to
masquerade /out/ of).

The zones are great for setting up separate rules for interacting with
the server itself (e.g. restricted only gets dns, dhcp, and ntp, while
desktop gets filesharing).  But the zone-to-zone transfers are wonky.
If the zone has a DROP target, then the "direct rules" don't even see
the traffic (i.e. trying to add a direct rule to explicitly allow
internal subnets to forward to each other did nothing, because the
packets got dropped before even getting to the 'direct' rules).  To rub
salt in the wound, ICMP pings *would* work, only the udp/tcp traffic
would get dropped.

Meanwhile, the scope of rich rules is unclear from the documentation,
but trying to add that zone-to-zone rule within a zone as a "rich rule"
didn't help any more than adding it as a "direct rule".

The only way for me to reach the cameras that I could find was to set
the desktop zone to "trusted", which the documentation says: "The ACCEPT
target is used in trusted zone to accept every packet not matching any
rule."  But that disables the firewall from that zone completely (you
can happily 'enable services' in that zone, but there's no need to,
because everything is allowed by default).  I virtually never want that.
 But that is the only way to get it to allow traffic between zones. It's
also really weird that setting 'trusted' flag somehow makes both
directions work.  Also, someday when IPv6 is supported by my upstream
provider, these "it's safe because it's NAT'd and doesn't need explicit
firewall rules" is going to be a problem, so I'm worried about that
'accept by default' rule long term.

This might all be a long way of saying I don't know how to tell
firewalld how I want it to manage inter-zone traffic, but I'm pretty
sure it just doesn't want to do that.  The few posts I was coming across
on issues related to this got responses like "that's a violation of the
zone-based model".  If I just put everything on my internal net in one
zone, then I'm stuck writing a bunch of manual rules anyway to say
things like "only allow nfs from these IP ranges", which defeats the
purpose of having a higher level tool.

Is Shorewall better in this regard?

Thanks,
Matt