utilities

Overview

A collection of various useful routines.

Functions

pymisclib.utilities.dir_path(path: str) → Path

Convert the string to a path and return it if it is a directory.

This function is intended to be used as the type argument to Argparser.add_argument().

Parameters:

path (str) – String containing path to check.

Returns:

Path to a directory.

Return type:

Path

Raises:

argparse.ArgumentTypeError – The string was not a valid path to a directory.

pymisclib.utilities.exit_hard(returncode: int = 0)

Terminate the running application.

Parameters:

returncode (int) – Exit code to return to spawning shell.

pymisclib.utilities.extract_from_zip_preserving_mtime(zip_file: ZipFile, zip_info: ZipInfo, out_path: dir_path)

Extract file from a ZIP archive preserving the modification time.

Parameters:

  • zip_file (ZipFile) – The archive to extract from.

  • zip_info (ZipInfo) – The information about the file to extract.

  • out_path (dir_path) – Destination directory for the extracted file (must exist).

See:

https://stackoverflow.com/q/9813243

pymisclib.utilities.file_path(path: str) → Path

Convert the string to a path and return it if it is a file.

Parameters:

path (str) – String containing path to check.

Returns:

Path to a file.

Return type:

Path

Raises:

argparse.ArgumentTypeError – The string did not contain a valid path to a file.

pymisclib.utilities.get_language() → str

Determine the language the current user has set their OS to.

pymisclib.utilities.hexdump(b: bytes, bytes_per_line: int = 16, start_offset: int = 0, offset_digits: int = 8, show_ascii: bool = True) → str

Generator function to create a pretty representation of the given bytes and return a line per call.

` first_prefix   00000000  64 65 66 67 68 69 6A 6B  @ABCDEFG always_prefix  00000000  64 65 66 67 68 69 6A 6B  @ABCDEFG `

Parameters:

  • b (bytes) – The bytes to print.

  • bytes_per_line (int) – Number of bytes per line.

  • start_offset (int) – Starting offset for the first byte.

  • offset_digits (int) – Number of digits in the offset.

  • show_ascii (bool) – Print ASCII characters after the hex dump if True.

pymisclib.utilities.initialize_console()

Initialize the console to work with UTF-8 encoded strings.

On windows, the console is strange and if output is redirected to a file, it gets even stranger. This confuses Python and even though PEP 528 solves the problem for interactive consoles, this does not help for non-interactive (which redirected streams are).

The solution for now is to reconfigure the codecs for stdout and stderr to always use UTF-8 and replace unmappable characters.

pymisclib.utilities.logging_add_trace()

Add a loglevel TRACE.

This function should be called only once.

pymisclib.utilities.logging_initialize(loglevel: int = 30, log_dir_path: Path = PosixPath('.'), log_file_name_format: str = '%P.log', log_rotation: int = 0, log_compression: int = 5, loglevel_console: int = 30, loglevel_file: int = 10) → Logger

Initialize the logging interface with the command line options passed through the object ‘args’. An instance of the root logger is returned to the caller.

If a submodule uses logging.getLogger(‘somename’), the logger will be a child of the root logger and inherit the settings made here.

Parameters:

  • loglevel (int) – Loglevel of the logger. Log messages not meeting the level requirement are not processed at all.

  • log_dir_path (Path) – Path of the directory containing log files. None will prevent log file creation.

  • log_file_name_format (str) – Format string for the log file name. None will prevent log file creation.

  • log_rotation (int) – How many logfiles of the same name to keep. If set to 0, any existing log file is overwritten. If set to >0, that many old log file copies (named <name>.1, <name>.2, etc.) are retained.

  • log_compression (int) – Zlib compression level to use in compressing log. Level 0 is no compression, level 9 is the highest possible.

  • loglevel_console (int) – Loglevel filter of the console logger.

  • loglevel_file (int) – Loglevel filter of the file logger.

Returns:

The root logger instance for the application.

Return type:

logging.logger

Note:

If the log_file_name_format contains a timestamp, log_rotation will only work on other log file copies with the exact same timestamp.

pymisclib.utilities.initialize_logging(args: Namespace)

Initialize the logging interface with the command line options passed through the object ‘args’. An instance of the root logger is returned to the caller.

If a submodule uses logging.getLogger(‘somename’), the logger will be a child of the root logger and inherit the settings made here.

Parameters:

args (argparse.Namespace) – Namespace containing the following

attributes which are used:

  • args.debug bool - Enable or disable debug mode.

  • args.logging str - Loglevel (e.g. CRITICAL, ERROR, etc.)

  • args.verbose bool - Enable or disable verbose mode.

Returns:

The root logger instance for the application.

Return type:

logging.logger

Deprecated since version 1.2.0: Use logging_initialize() and logging_add_trace() (if needed) instead.

pymisclib.utilities.is_power_of_two(n: int) → bool

Return True if the given number n is a power of two.

Parameters:

n (int) – number to check

Returns:

True if n is a power of two, False otherwise.

Return type:

bool

pymisclib.utilities.log_hexdump(fn_logger: Logger, b: bytes, bytes_per_line: int = 16, level: int = 10, start_offset: int = 0, first_prefix: str = '', always_prefix: str = '', show_ascii: bool = True)

Log a pretty representation of the given bytes to the logger.

` first_prefix   00000000  64 65 66 67 68 69 6A 6B  @ABCDEFG always_prefix  00000000  64 65 66 67 68 69 6A 6B  @ABCDEFG `

Parameters:

  • fn_logger (logging.Logger) – The logger to log to.

  • b (bytes) – The bytes to print.

  • bytes_per_line (int) – The number of bytes per line.

  • level (int) – Level for logging (e.g. CRITICAL, ERROR, .. DEBUG)

  • start_offset (int) – The starting offset for the first byte.

  • first_prefix (str) – A string before the offset on the first line.

  • always_prefix (str) – A string that will be printed before every line (except the first if first_prefix was specified).

  • show_ascii (bool) – Print ASCII characters after the hex dump if True.

pymisclib.utilities.log_stacktrace(fn_logger: Logger, level: int = 10)

Log the current stack trace inside or outside an exception.

Parameters:

  • fn_logger (logging.Logger) – The logger to log to.

  • level (int) – Level for logging (e.g. CRITICAL, ERROR, .. DEBUG)

pymisclib.utilities.resolve_wildcards(filenames: list[str]) → list[Path]

Resolve unique Paths from a list containing wildcards.

Parameters:

filenames (list[str]) – A list of filenames that may contain wildcards.

Returns:

A list of unique Paths.

Return type:

list[Path]

pymisclib.utilities.rotate_file(file_name: str, file_dir: ~pathlib.Path, rotation: int, compress_level: int = 9, fn_logger: ~logging.Logger = <Logger pymisclib.utilities (WARNING)>)

Rotate a (log-)file.

The parameter rotation specifies the number of copies that will be retained.

When rotating, file_name.rotation-1 is renamed to file_name.rotation for all values of rotation down to 1.

The initial file (with no .rotation suffix) gains a suffix. If compress_level > 0, the initial file is also compressed.

Parameters:

  • file_name (str) – Name of the log file.

  • file_dir (Path) – Path to the directory containing the log file.

  • rotation (int) – Number of copies to rotate. 0 to disable.

  • compress_level (int) – Zlib compression level with 0 being no compression and 9 the highest possible.

  • fn_logger (logging.Logger) – Logger used by the function.

pymisclib.utilities.round_down(n: int, m: int) → int

Round the given number n down to the nearest multiple of m.

Parameters:

  • n (int) – number to round

  • m (int) – multiple to round to

Returns:

n rounded down to a multiple of m.

Return type:

int

pymisclib.utilities.round_up(n: int, m: int) → int

Round the given number n up to the nearest multiple of m.

Parameters:

  • n (int) – number to round

  • m (int) – multiple to round to

Returns:

n rounded up to a multiple of m.

Return type:

int

pymisclib.utilities.std_open(filename: str = None, mode: str = 'r', encoding: str = 'utf-8')

Open either a file or stdin/stdout for use with with.

If the filename is None or ‘-’ then stdin or stdout (depending on the mode) are used. Otherwise, the file is used. It is closed when the with block is done.

Parameters:

  • filename (str) – A filename, ‘-’, or None.

  • mode (str) – The mode to use for open().

  • encoding (str) – The encoding to pass to open(). Defaults to ‘utf-8’.

Example

with std_open(fn, 'r') as f:
    c = f.read()

Changed in version 1.3.1: Default mode changed to ‘r’ to minimize chances of accidental file clobbering.

pymisclib.utilities.string_from_format(fmt: str) → str

Given a strftime()-like format, expand into a string.

Additional formats: - %P - name of the application

Parameters:

fmt (str) – Format string to parse.

Returns:

Resulting string.

pymisclib.utilities.true_stem(path: Path) → str

Return the true stem (e.g. the name without suffixes) of the Path.

Parameters:

path (Path) – A Path object.

Return type:

str

Returns:

The final file or directory name without any suffixes.

Example

print(true_stem(Path('/var/tmp/abc.tar.gz')))
abc