iparallelization_solver module

The iparallelization_solver interface is designed for parallel execution of an input function.

This module defines the IParallelSolver class, which orchestrates the parallel execution of a general processing function on a data set consisting of multiple independent data subsets, here reffered to as set IDs. It also declares a logging method (intended to start a separate logging thread).

Author:

Momcilo Tosic Astroinformatics student Faculty of Mathematics, University of Belgrade

class iparallelization_solver.IParallelSolver(num_workers=4)[source]

Bases: object

A class to manage parallel execution of data processing functions.

num_workers

Number of worker processes to spawn.

Type:: int

aggregate_process_function_result(result)[source]

get_process_function_result(set_id)[source]

maybe_begin_logging(set_id)[source]

maybe_save_local_results(set_id, res_string)[source]

maybe_save_results(results_file)[source]

maybe_stop_logging()[source]

process_ids(set_ids, results_file=None)[source]

Processes a list of set IDs using the configured process function in parallel.

Parameters:

set_ids (list of str) – List of set IDs to process.
results_file (str, optional) – Path to save aggregated results.

process_wrapper()[source]: Wrapper for the process function to integrate logging and result handling.