Multiclass Interface

The multiclass_interface is a separate package bundled with h2lib.

It is used by h2lib but it is very general and can be used in other contexts as well.

It can be used to run multiple instances of a class in different ways:

• MultiClassInterface: Sequential interface

• MultiProcessClassInterface: Multiprocess interface. Execution will run in parallel (if the system has available resources) and shared objects/dlls will be loaded into process-private chunks of memory.

• MPIClassInterface:

Example

For this example we will use the MyTest class defined in h2lib_tests.test_files.my_test_cls. YOu can see the contents below First we make a simple worker class. When run from jupyter notebook this class must be saved into a module (mytest.py)

import inspect
from h2lib_tests.test_files import my_test_cls
print (inspect.getsource(my_test_cls))

import time
import os


class MyTest():
    def __init__(self, id):
        self.id = id
        self.name = self.__class__.__name__

    def get_id(self,):
        return self.id

    def work(self, t):
        start_time = time.time()
        s1 = f'{self.id} starts working for {t}s at t={start_time}. '
        # print (s1)
        while time.time() < start_time + t:
            pass
        s2 = f'{self.id} ends working at {time.time()}.'
        # print (s2)
        return s1 + s2

    def return_input(self, *args, **kwargs):
        return f"{self.id} got: {str(args)} and {str(kwargs)}"

    def get_ld_library_path(self):
        return os.environ['LD_LIBRARY_PATH']

    def close(self):
        return f"closing {self.get_id()}"

    def raise_exception(self):
        1 / 0  # raise ZeroDivisionError

We can now import the module and instantiate a MultiClassInterface with three instances of MyTest. The args in args_lst are passed to the __init__ constructor of MyTest

from h2lib_tests.test_files.my_test_cls import MyTest
from multiclass_interface import MultiProcessClassInterface
myTests = MultiProcessClassInterface(cls=MyTest, args_lst = [(1,), (2,), (3,)])

Call a method in each of the three instances

myTests.get_id()

[1, 2, 3]

Call a method with using the same argument for all instances

%%timeit -n 1 -r 1
s = myTests.work(3)
print ("\n".join(s))

1 starts working for 3s at t=1702983709.4959445. 1 ends working at 1702983712.4959657.
2 starts working for 3s at t=1702983709.4972532. 2 ends working at 1702983712.497966.
3 starts working for 3s at t=1702983709.4979444. 3 ends working at 1702983712.497966.
3 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)

Call a method with using the same argument for all instances

If an argument is a list and has the same length as the number of instances, then the MultiProcessInterface will pass the first element to the first instance etc.

%%timeit -n 1 -r 1
s = myTests.work([1,2,3])
print ("\n".join(s))

1 starts working for 1s at t=1702983712.507971. 1 ends working at 1702983713.5089672.
2 starts working for 2s at t=1702983712.5089705. 2 ends working at 1702983714.5099683.
3 starts working for 3s at t=1702983712.5100014. 3 ends working at 1702983715.5109696.
3 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)

Argument or element distribution

To distribute the whole argument instead of its elements you can use tuples or numpy arrays:

import numpy as np
myTests.return_input([1,2,3], # Elements distributed because it's a list with N=3
                     [1,2],  # argument distributed because N=2
                     (1,2,3), # argument destributed because it's a tuple
                     np.array([1,2,3]), # argument destributed because it's a numpy array
                     kw=[1,2,3] # keyword argument. Elements distributed because it's a list with N=3
                    )

["1 got: (1, [1, 2], (1, 2, 3), array([1, 2, 3])) and {'kw': 1}",
 "2 got: (2, [1, 2], (1, 2, 3), array([1, 2, 3])) and {'kw': 2}",
 "3 got: (3, [1, 2], (1, 2, 3), array([1, 2, 3])) and {'kw': 3}"]

MultiClassInterface

The MultiClassInterface can be used in the same way, but the methods will be executed sequentially, so the work method will take more time

from multiclass_interface import MultiClassInterface
myTests = MultiClassInterface(cls=MyTest, args_lst = [(1,), (2,), (3,)])

myTests.get_id()

[1, 2, 3]

%%timeit -n 1 -r 1
s = myTests.work(3)
print ("\n".join(s))

1 starts working for 3s at t=1702983721.8653796. 1 ends working at 1702983724.8663769.
2 starts working for 3s at t=1702983724.8663769. 2 ends working at 1702983727.866382.
3 starts working for 3s at t=1702983727.866382. 3 ends working at 1702983730.8664258.
9 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)

MPIClassInterface

The MPIClassInterface can also be used in the same way, but it requires mpi, which is not available in the current context, so it cannot be demonstrated here