Dynamically change references from import in Python

Short:
How can I import a module if a different module with the same name has already been imported? I don't need the old module anymore, but I can't import the new one under a different name. importlib.reload() doesn't have the desired effect and apparently you really shouldn't mess with sys.modules

Context:
I'm automating a testing workflow by writing a script (A). The test script is provided and can not be altered.
The target of the Testing is a class in a script (B) of which there are different versions in subfolders. Unfortunately Script B and the class always have the same name, nothing I can do about that.

main_folder  
├──script_a.py  
├──test_script.py 
│ 
├──version_1
│    ├──script_b.py    
│
├──version_2  
│    ├──script_b.py 
│ 
├──version_3  
│    ├──script_b.py 

The test script imports the object and runs tests on it.

# test_script
from script_b import my_class 

# creat instance of my_class
# and test it

script_a iterates over the version folders and runs the test script on script_b. In each iteration one subfolder is added to the import path so the test script will find the corresponding script_b. The path is removed after the iteration.
If sys.modules already contains a version of test_script,

# script_a
import sys
import os
import importlib

folders = ['version_1', 'version_2', 'version_3']
folders = [os.getcwd() + '/' + folder for folder in folders]

for folder in folders:
    sys.path.append(folder)


    if 'test_script' in sys.modules:
        importlib.reload(sys.modules['test_script'])
    else:
        importlib.import_module('test_script')

    sys.path.remove(folder)

Issue:
It seems reload has no effect on script_b, which is imported by test_script. So although I change the import path to different subfolders test_script always runs on version 1. How can I make test_script use the different versions of script_b without altering test_script itself?

Follow up: Although the original question is answered, I was wondering, how is this solution from a design perspective? Is there a better / more elegant way to automate this testing process?
From what I found, it isn't considered good practice to reload modules.

1 answer

  • answered 2017-11-14 23:30 ColdBrew

    Figured it out while writing the question. Hope it will help someone some day

    Even if you can't alter test_script (or script_b) there's a work around.
    Because an import statement does nothing if the relevant module is already imported we can reload script_b from the path we want directly in script_a. Since it then is replaced with the new version in sys.modules the import statement in test_script will cause no problem.

    Updated code:

    # script_a
    import sys
    import os
    import importlib
    
    folders = ['version_1', 'version_2', 'version_3']
    folders = [os.getcwd() + '/' + folder for folder in folders]
    
    for folder in folders:
        sys.path.append(folder)
    
        if 'test_script' in sys.modules:
            importlib.reload(sys.modules['script_b'])      # this line added
            importlib.reload(sys.modules['test_script'])
        else:
            importlib.import_module('test_script')
    
        sys.path.remove(folder)