c++ style overloaded methods in python

Do you have questions about writing plugins or scripts in Python? Meet the coders here.
Post Reply
User avatar
kgschlosser
Site Admin
Posts: 5146
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

c++ style overloaded methods in python

Post by kgschlosser » Tue Nov 05, 2019 10:27 am

for those of you that are familiar with c++ and python you should be familiar with overloaded functions/methods in c++
an for those of you that do not.. Here is a quick description.

you can have multiple functions with the same name in c++ so long as the variable types for the function are different then in the other functions of the same name. This is something that is not available in python.

Now some of you python gurus will bring up type checking and kwargs.. Personally I think that having a bunch of key checks to check kwargs or instance checks on parameters adds a plethora of unneeded code and make it harder to read.

having multiple functions with the same name is also a fantastic thing when changing API so you can institute the same function but it will work if the "old" parameters re passed as well as the "new" Now from a code management standpoint It is also a whole lot easier to remove a deprecated function if the "new" and the "old" are not rolled into the same code block.


So I thought I would share my handy work here. someone may want to use it.

Code: Select all


import inspect
import sys
from functools import update_wrapper


def _compile_arguments(func, types):
    if sys.version_info[0] > 2:
        # noinspection PyUnresolvedReferences
        func_args, _, _, defaults = inspect.getfullargspec(func)
    else:
        # noinspection PyDeprecation
        func_args, _, _, defaults = inspect.getargspec(func)

    if 'self' in func_args:
        func_args = func_args[1:]

    if defaults:
        func_args_ = list(func_args[i] for i in range(len(func_args) - 1, -1, -1))
        types_ = list(types[i] for i in range(len(types) - 1, -1, -1))
        defaults_ = list(defaults[i] for i in range(len(defaults) - 1, -1, -1))
        func_args = zip(func_args_, types_, defaults_)
        func_args = list(func_args[i] for i in range(len(func_args) - 1, -1, -1))
        func_args_ = zip(func_args_, types_)
        func_args_ = list(func_args_[i] for i in range(len(func_args_) - 1, -1, -1))
        func_args = func_args_[:len(func_args_) - len(defaults)] + func_args
    else:
        func_args = zip(func_args, list(types))

    return func_args


_DOC_TEMPLATE = '''\
*overloaded*
.. function:: {function_name}({arg_string})
{original_docstring}
'''

def overload(*types):

    def wrapper(func):
        func_locals = sys._getframe(1).f_locals

        if '__overloads__' not in func_locals:
            func_locals['__overloads__'] = {}

        overloads = func_locals['__overloads__']

        if func.__name__ not in overloads:
            overloads[func.__name__] = []

        func_args = _compile_arguments(func, types)
        overloads[func.__name__] += [[func, func_args]]

        def func_wrapper(*args, **kwargs):
            if inspect.ismethod(func):
                test_args = args[1:]
            else:
                test_args = args

            overlds = []

            for arg in test_args:
                if inspect.isclass(arg):
                    overlds += [arg]
                else:
                    overlds += [type(arg)]

            for f, args_ in overloads[func.__name__]:
                if len(overlds) > len(args_):
                    continue

                for i, arg in enumerate(args_):
                    if i >= len(overlds):
                        if arg[0] in kwargs:
                            if arg[1] is None:
                                continue

                            value = kwargs[arg[0]]
                            if not inspect.isclass(value):
                                value = type(value)

                            if value == arg[1]:
                                continue

                        if len(arg) == 3:
                            continue

                    elif arg[1] is None or overlds[i] == arg[1]:
                        continue

                    break
                else:
                    break
            else:
                raise RuntimeError('not able to find overloaded function ' + func.__name__)

            return f(*args, **kwargs)
        doc = ''

        for fn, ar in overloads[func.__name__]:
            arg_string = []
            for arg_ in ar:
                if len(arg_) == 3:
                    arg_name, arg_type, arg_def = arg_

                    if arg_type is None:
                        arg_string += [arg_name + '=' + repr(arg_def)]
                    else:
                        arg_string += [arg_name + ': ' + str(arg_type).split("'")[1] + '=' + repr(arg_def)]
                else:
                    arg_name, arg_type = arg_
                    if arg_type is None:
                        arg_string += [arg_name]
                    else:
                        arg_string += [arg_name + ': ' + str(arg_type).split("'")[1]]

            arg_string = ', '.join(arg_string)

            doc += _DOC_TEMPLATE.format(
                function_name=fn.__name__,
                arg_string=arg_string,
                original_docstring=fn.__doc__ if fn.__doc__ is not None else ''
            )

        update_wrapper(func_wrapper, func)
        func_wrapper.__doc__ = doc
        return func_wrapper

    return wrapper

Simple overloaded functions and methods.

There are several other c++ style overload packages available for python.
This one has to be the easiest to use by far. It also outputs properly formatted
sphinx docstrings for each of the overloaded methods.

Code: Select all

@overload(int, float, str)
def test_1(i, f=10.0, s='string'):
    '''
    Docstring for first overloaded test_1 function.
    '''
    pass

@overload(float, int, str)
def test_1(f=10.0, i=30, s='string'):
    '''
    Docstring for second overloaded test_1 function.
    '''
    pass

print test_1.__doc__

the key to this is the decorator

Code: Select all

@overload(float, int, str)
place that before the function or method and you pass the data types for the parameters


The output of the docstring for the function is

Code: Select all

*overloaded*
.. function:: test_1(i: int, f: float=10.0, s: str='string')

    Docstring for first overloaded test_1 function.

*overloaded*
.. function:: test_1(f: float=10.0, i: int=30, s: str='string')

    Docstring for second overloaded test_1 function.
This decorator will check for default arguments so you can call the function like this as well and it will work.

Code: Select all

test_1(20.0)
this is where you can run into an issue is if you do the following.

Code: Select all

test_1(i=5, f=20.0)
It is always going to call the first overload where parameter i is int and f is float. This is the reason why
I do not check the argument names except to see if an argument is in kwargs or not. So you can set different
parameter names for each of the overloaded functions if you wish.


I also added a None clause. You can pass None to the decorator instead of the object type. Setting this to None
makes it a wildcard so any data type can be passed to it.

This also works on methods as well. and You will never have to worry about conflicting methods/functions in different
modules or classes This is because the the overloaded function/method get stored in the parent objects __dict__
attribute.

I wrote this to make it as simple as possible to use. and also making it the easiest to understand when reading the code

the decorator pretty much lays it out. and makes it really easy to understand what is happening.

enjoy!
If you like the work I have been doing then feel free to Image

Post Reply