[BUG] modules/pkgng.py is missing list_repo_pkgs

[Jsollvander]

Description

In 3007.0 and newer, modules/pkgng.py is missing the function pkg.list_repo_pkgs which is required when running with test=True on FreeBSD.

This is related to the introduction of the feature in #63986.

----------
          ID: ses_exporter
    Function: pkg.latest
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/local/lib/python3.11/site-packages/salt/state.py", line 2401, in call
                  ret = self.states[cdata["full"]](
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/usr/local/lib/python3.11/site-packages/salt/loader/lazy.py", line 177, in __call__
                  ret = self.loader.run(run_func, *args, **kwargs)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/usr/local/lib/python3.11/site-packages/salt/loader/lazy.py", line 1480, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/usr/local/lib/python3.11/site-packages/salt/loader/lazy.py", line 1495, in _run_as
                  ret = _func_or_method(*args, **kwargs)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/usr/local/lib/python3.11/site-packages/salt/loader/lazy.py", line 1531, in wrapper
                  return f(*args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^
                File "/usr/local/lib/python3.11/site-packages/salt/states/pkg.py", line 2847, in latest
                  changes = _get_installable_versions(targets, cur)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/usr/local/lib/python3.11/site-packages/salt/states/pkg.py", line 1060, in _get_installable_versions
                  explicit_versions = __salt__["pkg.list_repo_pkgs"](*explicit_targets)
                                      ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
                File "/usr/local/lib/python3.11/site-packages/salt/loader/context.py", line 106, in __getitem__
                  return self.value()[item]
                         ~~~~~~~~~~~~^^^^^^
                File "/usr/local/lib/python3.11/site-packages/salt/loader/lazy.py", line 519, in __getitem__
                  _ = super().__getitem__(item)  # try to get the item from the dictionary
                      ^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/usr/local/lib/python3.11/site-packages/salt/utils/lazy.py", line 106, in __getitem__
                  return self._dict[key]
                         ~~~~~~~~~~^^^^^
              KeyError: 'pkg.list_repo_pkgs'
     Started: 17:08:16.092679
    Duration: 273.203 ms
     Changes:
---------- 

I see other people have fixed the same problem for Windows in #67171. So looks like we need to do the same for FreeBSD and friends that are relying on modules/pkgng.py.

[ggiesen]

Thanks for the report. I'm not a Salt maintainer, so this isn't an official answer, just what I could piece together from public sources.

FreeBSD is no longer an officially supported operating system for Salt: the Salt Project moved it to "not officially supported, but maintained by community members" in July 2024 (Update to supported operating systems). Most FreeBSD modules were then removed from core in 3008.0 (#65970) and now live in saltext-freebsd.

pkgng is one of the few still in core, but the public saltstack/great-module-migration repo lists it in core-ext-modules.txt, among the modules planned to move out of the Salt codebase. If that still holds, saltext-freebsd is probably the better home for this fix, alongside the rest of the FreeBSD modules. From the loader code, extension modules take precedence over core modules of the same name, so a pkgng in saltext-freebsd with list_repo_pkgs added should reach users on 3006 through 3008 without waiting for a core release. win_pkg got the equivalent fix in #67781 (for #67171), which looks like a good template.

Whether pkgng actually moves is a call for the Salt maintainers and the saltext-freebsd maintainers, so it may be worth raising with them too.

The removed FreeBSD modules that don't have an extension yet (freebsdpkg, poudriere) are in salt-extensions/community-extensions-holding if anyone wants to adopt them.

[Jsollvander]

I mainly created this discussion because I needed a issue# for the PR (#70321 ). But since FreeBSD isn't a supported OS, just like you mentioned, I had to make a discussion instead of an issue. They did approve the PR and tagged it for 3009, so I guess modules/pkgng.py might live in core for a bit longer? I'm not so sure if they'll backport the fix to 3007 and/or 3008 where it's currently broken though.

My train of thought was that I should simply upstream a fix and see if they'd accept it or not, and then patch the freebsd port to fix this issue for all FreeBSD users currently running a broken py-salt without having to wait for a new release. My backup plan was to add modules/pkgng.py to Alan's saltext-freebsd module alongside this fix. But I guess one day sooner or later I'm going to have to do that anyways, unless someone (hopefully) beats me to it.

[network-shark]

Weired , I thought I was the maintainer of saltext-freebsd , but now there is another one . Nevertheless ... good that there is a working extension. I would like to see freebsdpkg, poudriere in the extensions as well .

[Jsollvander]

Yeah he made that one a short while ago when working on https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296290 . I'm sure he'd accept any and all help to maintain it.