Saltstack FreeBSD Jails — Part 2

2023-03-27 • 5 min read • Tags: Comp Sysadm Freebsd

After experimenting with Salt’s jails-aware functions for about 2 weeks, I opted for BastilleBSD for jails management and configuration.

I initially wanted to go for:

  • Bastille for manual jail lifecycle management (create, update and upgrade mainly)
  • Salt for jail configuration

But…

Salt service.running is actually NOT jails-aware!

This came as a big disappointment and it took me a while to come to the realization. I see service.running as one of the main functions, with file.managed and pkg.installed, for installing and configuring a service with Salt.

The root cause is that, unlike salt/states/pkg.py, salt/states/service.py does not pass **kwargs to os-specific implementations like salt/modules/freebsdservice.py. And thus functions in salt/modules/freebsdservice.py do not receive the jail keyword argument 😡

Draft for a fix 
--- a/salt/states/service.py	2021-03-24 22:54:42.000000000 +0100
+++ b/salt/states/service.py	2023-03-22 23:54:52.130686730 +0100
@@ -134,7 +134,7 @@

     # is service available?
     try:
-        if not _available(name, ret):
+        if not _available(name, ret, **kwargs):
             return ret
     except CommandExecutionError as exc:
         ret["result"] = False
@@ -245,7 +245,7 @@

     # is service available?
     try:
-        if not _available(name, ret):
+        if not _available(name, ret, **kwargs):
             ret["result"] = True
             return ret
     except CommandExecutionError as exc:
@@ -344,15 +344,15 @@
     return ret


-def _available(name, ret):
+def _available(name, ret, **kwargs):
     """
     Check if the service is available
     """
     avail = False
     if "service.available" in __salt__:
-        avail = __salt__["service.available"](name)
+        avail = __salt__["service.available"](name, **kwargs)
     elif "service.get_all" in __salt__:
-        avail = name in __salt__["service.get_all"]()
+        avail = name in __salt__["service.get_all"](**kwargs)
     if not avail:
         ret["result"] = False
         ret["comment"] = "The named service {} is not available".format(name)
@@ -440,7 +440,7 @@

     # Check if the service is available
     try:
-        if not _available(name, ret):
+        if not _available(name, ret, **kwargs):
             if __opts__.get("test"):
                 ret["result"] = None
                 ret["comment"] = (
@@ -453,6 +453,7 @@
         ret["comment"] = exc.strerror
         return ret

+    # XXX uh what has systemd to do with generic service handling?!
     status_kwargs, warnings = _get_systemd_only(__salt__["service.status"], kwargs)
     if warnings:
         _add_warnings(ret, warnings)
@@ -635,7 +636,7 @@

     # Check if the service is available
     try:
-        if not _available(name, ret):
+        if not _available(name, ret, **kwargs):
             if __opts__.get("test"):
                 ret["result"] = None
                 ret["comment"] = (
--- a/salt/modules/freebsdservice.py	2021-03-24 22:54:42.000000000 +0100
+++ b/salt/modules/freebsdservice.py	2023-03-22 23:55:02.160776847 +0100
@@ -348,7 +348,7 @@
     return not enabled(name, **kwargs)


-def available(name, jail=None):
+def available(name, jail=None, **kwargs):
     """
     Check that the given service is available.

@@ -362,6 +362,9 @@

         salt '*' service.available sshd
     """
+    log.error(f"__kwargs={kwargs}")
+    jail = jail if jail is not None else kwargs.get("jail", "")
+    log.error(f"__jail={jail}")
     return name in get_all(jail)


@@ -384,7 +387,7 @@
     return name not in get_all(jail)


-def get_all(jail=None):
+def get_all(jail=None, **kwargs):
     """
     Return a list of all available services

@@ -398,8 +401,10 @@

         salt '*' service.get_all
     """
+    jail = jail if jail is not None else kwargs.get("jail", "")
     ret = []
     service = _cmd(jail)
+    log.warning(f"__service={service}")
     for srv in __salt__["cmd.run"]("{0} -l".format(service)).splitlines():
         if not srv.isupper():
             ret.append(srv)
@@ -478,7 +483,7 @@
     return not __salt__["cmd.retcode"](cmd, python_shell=False)


-def status(name, sig=None, jail=None):
+def status(name, sig=None, jail=None, **kwargs):
     """
     Return the status for a service.
     If the name contains globbing, a dict mapping service name to True/False
@@ -503,6 +508,8 @@

         salt '*' service.status <service name> [service signature]
     """
+    jail = jail if jail is not None else kwargs.get("jail", "")
+
     if sig:
         return bool(__salt__["status.pid"](sig))

Note I went as far as rewriting the definition of a couple services to make them jail-aware, that is making them applicable to host machines or jails.

Salt PR, a hazardous path?

The Salt project on Github currently displays 2.5k open issues, 600+ open PRs. This feels like a lot.

The Salt community on slack was reactive and welcoming, explaining that a PR will surely be accepted as long as I provide tests. I also learned there is a community call where I can directly talk to the core team ❤️.

That’s encouraging but there was another aspect that unsettled me a bit: the pertinence of my potential PR. Am I the only one interested in that feature? How do others use Salt for jails? How come I am seemingly the first user to spot the issue? It seems there are already very few FreeBSD sysadmins, much less managing Jails…

Why not rather use a trendy tool?

Nice first experience with Bastille

Actually creating Bastille templates for service packaging is pretty straight forward: Bastillefile feels like Dockerfile. There are some less documented tricks, like the CONFIG keyword, which calls bastille config under the hood.

Most service definitions will be covered with PKG, SYSRC, SERVICE. MOUNT and OVERLAY will help with actual jail definitions. So I have a template for each: jail definition and service definition. The former INCLUDEing the latter.

I store and deploy templates with config files in Salt. Secrets are injected into config files with Salt pillars.

Pros:

  • Simple install, simple maintenance (less jinja wrangling, easy to iterate).
  • Flexibility. Fast, feature-rich, user-friendly tool.
    • bastille template is idempotent in my experience.

Cons:

  • No periodic state checking (ala Salt/Puppet/Chef). More ansible-like.
  • Tooling fragmentation. Specific tool for Jail management and configuration. The divergence is slightly worsened by the fact that Bastille uses a distinct jail.conf per jail1.
  • Services can only be deployed into jails.

Conclusion

Overall it feels like Bastille removes some complexity and seems to get some traction[Cit. needed]. So I’ll go and embrace the Bastille way. I’ll continue migrating the few remaining jails and package services still on the host.

  1. For ex. jail -c|-r does not work. At least for now: FreeBSD just introduced new default configuration files /etc/jail.conf.d/*.conf and /etc/jail.*.conf. Bastille might take advantage of that in the future. ↩︎