From c653769940f91b3d9aa9fed159e126a3f03031a8 Mon Sep 17 00:00:00 2001
From: Vasudev Kamath <kamathvasudev@gmail.com>
Date: Mon, 7 Apr 2014 21:57:15 +0530
Subject: new post on plugin loading in python

---
 .../development/dynamic_module_loading_python.rst  | 110 +++++++++++++++++++++
 1 file changed, 110 insertions(+)
 create mode 100644 content/development/dynamic_module_loading_python.rst

(limited to 'content/development/dynamic_module_loading_python.rst')

diff --git a/content/development/dynamic_module_loading_python.rst b/content/development/dynamic_module_loading_python.rst
new file mode 100644
index 0000000..3f3e960
--- /dev/null
+++ b/content/development/dynamic_module_loading_python.rst
@@ -0,0 +1,110 @@
+Loading Python modules/plug-ins at runtime
+##########################################
+
+:date: 2014-04-07 21:55
+:slug: dynamic-module-loading
+:tags: python, modules, import, programming, importlib
+:author: copyninja
+:summary: Post describes about loading arbitrary python files or
+          modules during runtime.
+
+Some times it is desired to load arbitrary python files or pre-
+installed python modules during application run time.I had encountered
+2 such usecases, one is in `SILPA application <http://silpa.org.in>`_
+and other is `dictionary-bot
+<https://github.com/copyninja/dictionary-bot>`_ which I was
+refactoring recently.
+
+Case 1: Loading installed python module
+---------------------------------------
+
+In case of SILPA I need to load pre-installed modules and here is the
+`old code
+<https://github.com/Project-SILPA/Silpa-Flask/blob/master/core/modulehelper.py#L24>`_
+, that is a bit hacky code I copied from Active State Python
+recipies. I found a bit better way to do it using `importlib` module
+as shown below.
+
+.. code-block:: python
+
+   from __future__ import print_function
+   import sys
+   import importlib
+
+   def load_module(modulename):
+       mod = None
+       try:
+           mod = importlib.import_module(modulename)
+       except ImportError:
+           print("Failed to load {module}".format(module=modulename),
+		        file=sys.stderr)
+       return mod
+
+Here `importlib` itself takes care of checking if modulename is
+already loaded by checking `sys.modules[modulename]`, if loaded it
+returns that value, otherwise it loads the module and sets it to
+`sys.modules[modulename]` before returning module itself.
+
+
+Case 2: Loading python files from arbitrary location
+----------------------------------------------------
+
+In case of dictionary bot my requirement was bit different, I had some
+python files lying around in a directory, which I wanted to plug into
+the bot during run time and use them depending on some conditions. So
+basic structure which I was looking was as follows.
+
+
+.. raw:: html
+
+   <pre>
+	 pluginloader.py
+	 plugins
+	 |
+	 |__ aplugin.py
+	 |
+	 |__ bplugin.py
+   </pre>  
+
+`pluginloader.py` is the file which needs to load python files under
+`plugins` directory. This was again done using `importlib` module as
+shown below.
+
+.. code-block:: python
+
+   import os
+   import sys
+   import re
+   import importlib
+
+   def load_plugins():
+       pysearchre = re.compile('.py$', re.IGNORECASE)
+       pluginfiles = filter(pysearchre.search,
+                              os.listdir(os.path.join(os.path.dirname(__file__),
+                                                    'plugins')))
+       form_module = lambda fp: '.' + os.path.splitext(fp)[0]
+       plugins = map(form_module, pluginfiles)
+       # import parent module / namespace
+       importlib.import_module('plugins')
+       modules = []
+       for plugin in plugins:
+		if not plugin.startswith('__'):
+		    modules.append(importlib.import_module(plugin, package="plugins"))
+
+       return modules
+
+Above code first searches for all python file under specified
+directory and creates a relative module name from it. For eg. file
+`aplugin.py` will become `.aplugin`. Before loading modules itself we
+will load the parent module in our case `plugins`, this is because
+**relative imports in python expects parent module to be already
+loaded**. Finally for relative imports to work with
+`importlib.import_module` we need to specify parent module name in
+`package` argument. Note that we ignore files begining with *__*, or
+specifically we don't want to import __init__.py, this will be done
+when we import parent module.
+
+The above code was inspired from a `answer on StackOverflow
+<https://stackoverflow.com/a/3381582>`_, which uses `imp` module, I
+avoided `imp` because its been deprecated from *Python 3.4* in favor
+of `importlib` module.
-- 
cgit v1.2.3