5.1. zeekpkg.manager module
A module defining the main Zeek Package Manager interface which supplies methods to interact with and operate on Zeek packages.
- class zeekpkg.manager.Manager(state_dir, script_dir, plugin_dir, zeek_dist='', user_vars=None, bin_dir='')
Bases:
object
A package manager object performs various operations on packages.
It uses a state directory and a manifest file within it to keep track of package sources, installed packages and their statuses.
- sources
dictionary package sources keyed by the name given to
add_source()
- Type:
dict of str ->
source.Source
- installed_pkgs
a dictionary of installed packaged keyed on package names (the last component of the package's git URL)
- Type:
dict of str ->
package.InstalledPackage
- zeek_dist
path to the Zeek source code distribution. This is needed for packages that contain Zeek plugins that need to be built from source code.
- Type:
str
- state_dir
the directory where the package manager will a maintain manifest file, package/source git clones, and other persistent state the manager needs in order to operate
- Type:
str
- user_vars
dictionary of key-value pairs where the value will be substituted into package build commands in place of the key.
- Type:
dict of str -> str
- backup_dir
a directory where the package manager will store backup files (e.g. locally modified package config files)
- Type:
str
- log_dir
a directory where the package manager will store misc. logs files (e.g. package build logs)
- Type:
str
- scratch_dir
a directory where the package manager performs miscellaneous/temporary file operations
- Type:
str
- script_dir
the directory where the package manager will copy each installed package's script_dir (as given by its
zkg.meta
orbro-pkg.meta
). Each package gets a subdirectory within script_dir associated with its name.- Type:
str
- plugin_dir
the directory where the package manager will copy each installed package's plugin_dir (as given by its
zkg.meta
orbro-pkg.meta
). Each package gets a subdirectory within plugin_dir associated with its name.- Type:
str
- bin_dir
the directory where the package manager will link executables into that are provided by an installed package through executables (as given by its
zkg.meta
orbro-pkg.meta
)- Type:
str
- source_clonedir
the directory where the package manager will clone package sources. Each source gets a subdirectory associated with its name.
- Type:
str
- package_clonedir
the directory where the package manager will clone installed packages. Each package gets a subdirectory associated with its name.
- Type:
str
- package_testdir
the directory where the package manager will run tests. Each package gets a subdirectory associated with its name.
- Type:
str
- manifest
the path to the package manager's manifest file. This file maintains a list of installed packages and their status.
- Type:
str
- autoload_script
path to a Zeek script named
packages.zeek
that the package manager maintains. It is a list of@load
for each installed package that is marked as loaded (seeload()
).- Type:
str
- autoload_package
path to a Zeek
__load__.zeek
script which is just a symlink to autoload_script. It's always located in a directory namedpackages
, so as long asZEEKPATH
is configured correctly,@load packages
will load all installed packages that have been marked as loaded.- Type:
str
- class SourceAggregationResults(refresh_error='', package_issues=[])
Bases:
object
The return value of a call to
Manager.aggregate_source()
.- refresh_error
an empty string if no overall error occurred in the "refresh" operation, else a description of what wrong
- Type:
str
- package_issues
a list of reasons for failing to collect metadata per packages/repository. The first tuple element gives the repository URL in which the problem occurred and the second tuple element describes the failure.
- Type:
list of (str, str)
- add_source(name, git_url)
Add a git repository that acts as a source of packages.
- Parameters:
name (str) -- a short name that will be used to reference the package source.
git_url (str) -- the git URL of the package source
- Returns:
empty string if the source is successfully added, else the reason why it failed.
- Return type:
str
- aggregate_source(name, push=False)
Pull latest git info from a package source and aggregate metadata.
This is like calling
refresh_source()
with the aggregate arguments set to True.This makes the latest pre-aggregated package metadata available or performs the aggregation locally in order to push it to the actual package source. Locally aggregated data also takes precedence over the source's pre-aggregated data, so it can be useful in the case the operator of the source does not update their pre-aggregated data at a frequent enough interval.
- Parameters:
name (str) -- the name of the package source. E.g. the same name used as a key to
add_source()
.push (bool) -- whether to push local changes to the aggregated metadata to the remote package source.
- Returns:
- the results of the
refresh/aggregation.
- Return type:
- backup_modified_files(backup_subdir, modified_files)
Creates backups of modified config files
- Parameters:
modified_files (list of (str, str)) -- the return value of
modified_config_files()
.backup_subdir (str) -- the subdir of backup_dir in which
- Returns:
paths indicating the backup locations. The order of the returned list corresponds directly to the order of modified_files.
- Return type:
list of str
- bundle(bundle_file, package_list, prefer_existing_clones=False)
Creates a package bundle.
- Parameters:
bundle_file (str) -- filesystem path of the zip file to create.
package_list (list of (str, str)) -- a list of (git URL, version) string tuples to put in the bundle. If the version string is empty, the latest available version of the package is used.
prefer_existing_clones (bool) -- if True and the package list contains a package at a version that is already installed, then the existing git clone of that package is put into the bundle instead of cloning from the remote repository.
- Returns:
empty string if the bundle is successfully created, else an error string explaining what failed.
- Return type:
str
- bundle_info(bundle_file)
Retrieves information on all packages contained in a bundle.
- Parameters:
bundle_file (str) -- the path to the bundle to inspect.
- Returns:
a tuple with the the first element set to an empty string if the information successfully retrieved, else an error message explaining why the bundle file was invalid. The second element of the tuple is a list containing information on each package contained in the bundle: the exact git URL and version string from the bundle's manifest along with the package info object retrieved by inspecting git repo contained in the bundle.
- Return type:
(str, list of (str, str,
package.PackageInfo
))
- discover_builtin_packages()
Discover packages included in Zeek for dependency resolution.
This is using Zeek's
--build-info
flag and specifically thezkg.provides
entry it contains. Requires Zeek 6.0 and later.- Returns:
List of built-in packages.
- Return type:
list of
package.BuiltinPackage
- find_builtin_package(pkg_path)
Find a builtin plugin that matches
pkg_path
.- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source.
- Returns:
PackageInfo instance representing a builtin package matching
pkg_path
.- Return type:
- find_installed_package(pkg_path)
Return an
package.InstalledPackage
if one matches the name.- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".
- get_installed_package_dependencies(pkg_path)
Return a set of tuples of dependent package names and their version number if pkg_path is an installed package.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".
- has_plugin(installed_pkg)
Return whether a
package.InstalledPackage
installed a plugin.- Parameters:
installed_pkg (
package.InstalledPackage
) -- the installed package to check for whether it has installed a Zeek plugin.- Returns:
True if the package has installed a Zeek plugin.
- Return type:
bool
- has_scripts(installed_pkg)
Return whether a
package.InstalledPackage
installed scripts.- Parameters:
installed_pkg (
package.InstalledPackage
) -- the installed package to check for whether it has installed any Zeek scripts.- Returns:
True if the package has installed Zeek scripts.
- Return type:
bool
- info(pkg_path, version='', prefer_installed=True)
Retrieves information about a package.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".version (str) -- may be a git version tag, branch name, or commit hash from which metadata will be pulled. If an empty string is given, then the latest git version tag is used (or the default branch like "main" or "master" if no version tags exist).
prefer_installed (bool) -- if this is set, then the information from any current installation of the package is returned instead of retrieving the latest information from the package's git repo. The version parameter is also ignored when this is set as it uses whatever version of the package is currently installed.
- Returns:
A
package.PackageInfo
object.
- install(pkg_path, version='')
Install a package.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".version (str) -- if not given, then the latest git version tag is installed (or if no version tags exist, the default branch like "main" or "master" is installed). If given, it may be either a git version tag, a git branch name, or a git commit hash.
- Returns:
empty string if package installation succeeded else an error string explaining why it failed.
- Return type:
str
- Raises:
IOError -- if the manifest can't be written
- installed_package_dependencies()
Return dict of 'package' -> dict of 'dependency' -> 'version'.
Package-name / dependency-name / and version-requirement values are all strings.
- installed_packages()
Return list of
package.InstalledPackage
.
- list_depender_pkgs(pkg_path)
List of depender packages.
If C depends on B and B depends on A, we represent the dependency chain as C -> B -> A. Thus, package C is dependent on A and B, while package B is dependent on just A. Example representation:
{ 'A': set(), 'B': set([A, version_of_A]) 'C': set([B, version_of_B]) }
Further, package A is a direct dependee for B (and implicitly for C), while B is a direct depender (and C is an implicit depender) for A.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".- Returns:
list of depender packages.
- Return type:
list
- load(pkg_path)
Mark an installed package as being "loaded".
The collection of "loaded" packages is a convenient way for Zeek to more simply load a whole group of packages installed via the package manager.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".- Returns:
empty string if the package is successfully marked as loaded, else an explanation of why it failed.
- Return type:
str
- Raises:
IOError -- if the loader script or manifest can't be written
- load_with_dependencies(pkg_name, visited={})
Mark dependent (but previously installed) packages as being "loaded".
- Parameters:
pkg_name (str) -- name of the package.
visited (set(str)) -- set of packages visited along the recursive loading
- Returns:
list of tuples containing dependent package name and whether it was marked as loaded or else an explanation of why the loading failed.
- Return type:
list(str, str)
- loaded_package_states()
Save "loaded" state for all installed packages.
- Returns:
dictionary of "loaded" status for installed packages
- Return type:
dict
- loaded_packages()
Return list of loaded
package.InstalledPackage
.
- match_source_packages(pkg_path)
Return a list of
package.Package
that match a given path.- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".
- modified_config_files(installed_pkg)
Return a list of package config files that the user has modified.
- Parameters:
installed_pkg (
package.InstalledPackage
) -- the installed package to check for whether it has installed any Zeek scripts.- Returns:
tuples that describe the modified config files. The first element is the config file as specified in the package metadata (a file path relative to the package's root directory). The second element is an absolute file system path to where that config file is currently installed.
- Return type:
list of (str, str)
- package_build_log(pkg_path)
Return the path to the package manager's build log for a package.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".
- package_versions(installed_package)
Returns a list of version number tags available for a package.
- Parameters:
installed_package (
package.InstalledPackage
) -- the package for which version number tags will be retrieved.- Returns:
the version number tags.
- Return type:
list of str
- pin(pkg_path)
Pin a currently installed package to the currently installed version.
Pinned packages are never upgraded when calling
upgrade()
.- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".- Returns:
None if no matching installed package could be found, else the installed package that was pinned.
- Return type:
- Raises:
IOError -- when the manifest file can't be written
- refresh_installed_packages()
Fetch latest git information for installed packages.
This retrieves information about outdated packages, but does not actually upgrade their installations.
- Raises:
IOError -- if the package manifest file can't be written
- refresh_source(name, aggregate=False, push=False)
Pull latest git information from a package source.
This makes the latest pre-aggregated package metadata available or performs the aggregation locally in order to push it to the actual package source. Locally aggregated data also takes precedence over the source's pre-aggregated data, so it can be useful in the case the operator of the source does not update their pre-aggregated data at a frequent enough interval.
- Parameters:
name (str) -- the name of the package source. E.g. the same name used as a key to
add_source()
.aggregate (bool) -- whether to perform a local metadata aggregation by crawling all packages listed in the source's index files.
push (bool) -- whether to push local changes to the aggregated metadata to the remote package source. If the aggregate flag is set, the data will be pushed after the aggregation is finished.
- Returns:
an empty string if no errors occurred, else a description of what went wrong.
- Return type:
str
- remove(pkg_path)
Remove an installed package.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".- Returns:
True if an installed package was removed, else False.
- Return type:
bool
- Raises:
IOError -- if the package manifest file can't be written
OSError -- if the installed package's directory can't be deleted
- restore_loaded_package_states(saved_state)
Restores state for installed packages.
- Parameters:
saved_state (dict) -- dictionary of saved "loaded" state for installed
packages. --
- save_temporary_config_files(installed_pkg)
Return a list of temporary package config file backups.
- Parameters:
installed_pkg (
package.InstalledPackage
) -- the installed package to save temporary config file backups for.- Returns:
tuples that describe the config files backups. The first element is the config file as specified in the package metadata (a file path relative to the package's root directory). The second element is an absolute file system path to where that config file has been copied. It should be considered temporary, so make use of it before doing any further operations on packages.
- Return type:
list of (str, str)
- source_packages()
Return a list of
package.Package
within all sources.
- test(pkg_path, version='', test_dependencies=False)
Test a package.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".version (str) -- if not given, then the latest git version tag is used (or if no version tags exist, the default branch like "main" or "master" is used). If given, it may be either a git version tag or a git branch name.
test_dependencies (bool) -- if True, any dependencies required for the given package will also get tested. Off by default, meaning such dependencies will get locally built and staged, but not tested.
- Returns:
a tuple containing an error message string, a boolean indicating whether the tests passed, as well as a path to the directory in which the tests were run. In the case where tests failed, the directory can be inspected to figure out what went wrong. In the case where the error message string is not empty, the error message indicates the reason why tests could not be run. Absence of a test_command in the requested package is considered an error.
- Return type:
(str, bool, str)
- unbundle(bundle_file)
Installs all packages contained within a bundle.
- Parameters:
bundle_file (str) -- the path to the bundle to install.
- Returns:
an empty string if the operation was successful, else an error message indicated what went wrong.
- Return type:
str
- unload(pkg_path)
Unmark an installed package as being "loaded".
The collection of "loaded" packages is a convenient way for Zeek to more simply load a whole group of packages installed via the package manager.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".- Returns:
True if a package is successfully unmarked as loaded.
- Return type:
bool
- Raises:
IOError -- if the loader script or manifest can't be written
- unload_with_unused_dependers(pkg_name)
Unmark dependent (but previously installed packages) as being "loaded".
- Parameters:
pkg_name (str) -- name of the package.
- Returns:
list of tuples containing dependent package name and whether it was marked as unloaded or else an explanation of why the unloading failed.
- Return type:
list(str, str)
- Raises:
IOError -- if the loader script or manifest can't be written
- unpin(pkg_path)
Unpin a currently installed package and allow it to be upgraded.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".- Returns:
None if no matching installed package could be found, else the installed package that was unpinned.
- Return type:
- Raises:
IOError -- when the manifest file can't be written
- upgrade(pkg_path)
Upgrade a package to the latest available version.
- Parameters:
pkg_path (str) -- the full git URL of a package or the shortened path/name that refers to it within a package source. E.g. for a package source called "zeek" with package named "foo" in
alice/zkg.index
, the following inputs may refer to the package: "foo", "alice/foo", or "zeek/alice/foo".- Returns:
an empty string if package upgrade succeeded else an error string explaining why it failed.
- Return type:
str
- Raises:
IOError -- if the manifest can't be written
- validate_dependencies(requested_packages, ignore_installed_packages=False, ignore_suggestions=False, use_builtin_packages=True)
Validates package dependencies.
- Parameters:
requested_packages (list of (str, str)) -- a list of (package name or git URL, version) string tuples validate. If the version string is empty, the latest available version of the package is used.
ignore_installed_packages (bool) -- whether the dependency analysis should consider installed packages as satisfying dependency requirements.
ignore_suggestions (bool) -- whether the dependency analysis should consider installing dependencies that are marked in another package's 'suggests' metadata field.
use_builtin_packages (bool) -- whether package information from builtin packages is used for dependency resolution.
- Returns:
the first element of the tuple is an empty string if dependency graph was successfully validated, else an error string explaining what is invalid. In the case it was validated, the second element is a list of tuples, each representing a package, where:
The first element is a dependency package that would need to be installed in order to satisfy the dependencies of the requested packages.
The second element of tuples in the list is a version string of the associated package that satisfies dependency requirements.
The third element of the tuples in the list is a boolean value indicating whether the package is included in the list because it's merely suggested by another package.
The list will not include any packages that are already installed or that are in the requested_packages argument. The list is sorted in dependency order: whenever a dependency in turn has dependencies, those are guaranteed to appear in order in the list. This means that reverse iteration of the list guarantees processing of dependencies prior to the depender packages.
- Return type:
(str, list of (
package.PackageInfo
, str, bool))
- zeek_plugin_path()
Return the path where installed package plugins are located.
This path can be added to
ZEEK_PLUGIN_PATH
for interoperability with Zeek.
- zeekpath()
Return the path where installed package scripts are located.
This path can be added to
ZEEKPATH
for interoperability with Zeek.