Overthinking: Python Imports

An overview of the history and functionality of Python's import machinery.

This page is an outgrowth of a talk proposal I made for PyCon 2012. Hopefully it's helpful as a reference and adds a little perspective on imports in Python. If you have any suggestions or corrections just let me know (see the repo page).

Contents

1. A High-level Overview of Python's import
2. Python's Data and Execution Models
3. Import Syntax
4. ImportError
5. PEP 302
6. Implicit Finders
7. Implementations
8. Import State
9. A History of Python's import Statement

Appendices:
(A. Import Tips ?)
A. Import Syntax Under the Hood
B. Import-related Files
C. An Extended Timeline of Python's import
D. Ongoing Core Efforts to Improve Importing
E. Imports in Alternate Python Implementations
F. Easter Eggs
G. Import Hook Examples
H. Other Import-Related Examples
I. Imports in the Python Community
J. Import Tips and Tricks
K. Troubleshooting Imports
L. Other Import-related Resources
Glossary

Full Table of Contents

1. A High-level Overview of Python's import
1. Why you care about imports
2. The import Statement
3. PEP 302
4. Import State
2. Python's Data and Execution Models
1. Namespaces
2. Module Objects
3. Execution Blocks
4. Scope
5. Modules vs. Scripts
3. Import Syntax
1. The import statement
2. The as clause
3. The from statement
4. Relative Imports
5. Parentheses
6. Other Semantics
4. ImportError
1. What It Means
2. When It Happens and When Not
5. PEP 302
1. Finders
2. Loaders
3. The Import Process
6. Implicit Finders
1. Builtin Modules
2. Frozen Modules
3. Zipped Modules
4. The Python Path Finder
7. Implementations
1. The imp Module
2. The importlib Module
3. Import-related Modules
4. .pth Files
8. Import State
1. sys.modules
2. sys.path
3. sys.meta_path
4. sys.path_hooks
5. sys.path_importer_cache
9. A History of Python's import Statement
1. A Brief History of Python
2. The Origins of Python's import
3. Early Additions
4. The Intervening Years
5. Recent Additions
Appendices
Glossary

1 A High-level Overview of Python's import

[This section isn't right]

1.1 Why You Care about Imports
Even if you didn't realize it, you care deeply about Python's import machinery...

Note: make sure you understand the difference between running a module as a script and importing the module (see section 2.5).

1.2 The import Statement
The import statement is the syntactic mechanism you use to invoke Python's powerful import machinery. It has two forms: the regular import and from-import. Section 3 walks you through the ins and outs of both forms. You can override the full import machinery by overriding builtins.__import__().

When you use the import statement in either form, you identify a module and its parent modules together as a module name. By default, each successive parent module is imported from the outside in, followed by the actual module you wanted. After that the appropriate name is bound in the current local namespace. Most imports are going to happen at the module level where the name will be bound in that module's globals.

A module object is the result of importing. We use the term "module" to refer to this object as well as to the thing that Python used to create the object, usually a file. A package is a special kind of module. Where a normal module corresponds to a file, a package corresponds to a directory.

1.3 PEP 302 [The Import Process]
Prior to Python 2.3 the only way to override the import behavior was by replacing builtins.__import__() with some other function that did what you wanted. This changed with PEP 302.

Now you can add special "loader" objects to a couple of different places in the sys module to take control of imports in more targeted ways. A loader translates a module name into a "finder" object, if it can. The finder, in turn, converts the module name into the corresponding module object, which it sticks into sys.modules.

This entire process is explained much more in-depth in section 5.3 and in A.7.

1.4 Import State [Drop this?]
All the Python variables related to the default import behavior is stored in the sys module. This includes sys.path, sys.modules, sys.meta_path, and sys.path_hooks.

Notes and References:
[1] http://docs.python.org/library/sys.html
[2] http://www.doughellmann.com/PyMOTW/sys/imports.html

2 Python's Data and Execution Models

2.1 Namespaces

2.2 Module Objects
- modules
- packages
* packages should never be put directly on sys.path (even by the sys.path[0] behavior of the __main__ module)
See the "modules" section of the data model documentation[1].

2.3 Execution Blocks
See the execution model documentation[2].

2.4 Scope
See the execution model documentation[2].

2.5 Modules vs. Scripts

Notes and References:
[1] http://docs.python.org/dev/reference/datamodel.html#the-standard-type-hierarchy(modules section)
[2] http://docs.python.org/reference/executionmodel.html

3 Import Syntax

http://docs.python.org/dev/reference/simple_stmts.html

See also: Appendix B[1].

3.1 The import statement

- usage
- effect

3.2 The as clause
- usage
- effect
- benefits

3.3 The from statement
- usage
- effect
- from ... import *
- dangers

3.4 Relative Imports
http://docs.python.org/dev/tutorial/modules.html#intra-package-references
http://www.python.org/dev/peps/pep-0328/
- usage
- effect

3.5 Parentheses
- usage
- effect

3.6 Other Semantics
- implicit relative imports
- files (__init__.py, .py, .pyc, .pyo, etc.)
- builtins.__import__()

Notes and References:

[1] Appendix B provides a more thorough under-the-hood look at the import syntax.

4 ImportError

4.1 What It Means

4.2 When It Happens and When Not

Notes and References:
[1]

5 PEP 302

5.1 Finders

5.2 Loaders

5.3 The Default Import Process [Move to its own section?]

See A.7.

Notes and References:
[1] http://www.python.org/dev/peps/pep-0302/
[2] http://docs.python.org/release/2.3/whatsnew/section-pep302.html

6 Implicit Finders

6.1 Builtin Modules

6.2 Frozen Modules

6.3 Zipped Modules

6.4 The Python Path Finder

Notes and References:
[1]

7 Implementations

7.1 The imp Module
- iterative
- Python/import.c
- Python/bltinmodule.c - _builtin___import__()
- (default builtins.__import__())

PyImport_GetModuleDict() used to get sys.modules (see J.3.9).
7.2 The importlib Module
- recursive
- Lib/importlib/

sys.modules used to get sys.modules.

- importlib.__import__()
- importlib.import_module()

7.3 Import-related Modules

- pkgutil

- runpy

- modulefinder and zipimport

7.4 .pth Files
See the site module documentation[2].
More in Appendix B.

Notes and References:
[1] http://docs.python.org/dev/library/modules.html
[2] http://docs.python.org/library/site.html

8 Import State

8.1 sys.modules

8.2 sys.path

* .pth files
(Also see Appendix B)
(Also see I.1 for more on why '', a.k.a. CWD, is added to sys.path)

8.3 sys.meta_path

8.4 sys.path_hooks

8.5 sys.path_importer_cache

8.6 site-packages

http://docs.python.org/dev/library/site.html

site-packages
user site-packages

(Also see Appendix B)

8.7 The site Module

http://docs.python.org/dev/library/site.html

site.py
sitecustomize.py
usercustomize.py

(Also see Appendix B)

Notes and References:
[1] http://docs.python.org/library/sys.html

9 A History of Python's import Statement

9.1 A Brief History of Python

9.2 The Origins of Python's import

The import statement has been a part of Python since the very beginning, though with more limited behavior.

Like many things in Python, the syntax for the import statement has its roots in Modula-3.

9.3 Early Changes
- builtins.__import__()
- ni.py
- ihooks.py

9.4 The Intervening Years
- PEP 302

9.5 Recent Changes
-

Notes and References:
[1] http://python-history.blogspot.com/2009/02/adding-support-for-user-defined-classes.html
[2] http://www.python.org/doc/essays/foreword/
[3] http://python-history.blogspot.com/2009/01/brief-timeline-of-python.html

Appendices

A. Import Syntax Under the Hood
1. Syntactic Permutations
2. Grammar (ASDL)
3. Tokens and Keywords
4. AST
5. Opcodes
6. Examples of Looking under the Hood
7. The __future__ Module

B. Import-related Files

1. CPython
2. Standard Library
C. An Extended Timeline of Python's import
1. The Timeline
2. Timeline of Import-related Commits
D. Ongoing Core Efforts to Improve Importing
1. PEPs
2. Projects
E. Imports in Alternate Python Implementations
1. PyPy
2. Jython
3. IronPython
F. Easter Eggs
1. import this
2. import antigravity
3. from __future__ import flufl
4. from __future__ import braces
5. import __hello__
G. Import Hook Examples
H. Other Import-Related Examples
I. Imports in the Python Community
1. Community Uses of Import Hooks
2. Community Import Solutions
J. Import Tips and Tricks
K. Trouble-shooting Imports
1. Causes of ImportError
2. Other Exceptions During Import
3. Common Import-related Problems
L. Other Import-related Resources

Appendix A: Import Syntax Under the Hood

A.1 Syntactic Permutations

direct import
-------------
valid:
import logging

import logging as log_mod
import logging as log_mod, sys

import logging.handlers

import logging.handlers as logh_mod

invalid:
  import 5 #invalid identifier
import class #keyword
  import (logging) #parentheses not allowed here
  import (logging as log_mod) # not here either

indirect import
---------------
valid:
from logging import getLogger

from logging import handlers
from logging import handlers as logh_mod

from logging import (handlers)
from logging import (handlers as logh_mod)

invalid:
from logging import 5

universal indirect import
------------------------
valid:

from logging import *

from logging.handlers import *

relative indirect import
-----------------------
valid:

from . import spam

from . import spam as _spam

from .. import spam

from .spam import ham

A.2 Grammar (ASDL)

http://docs.python.org/dev/reference/grammar.html

http://hg.python.org/cpython/file/default/Grammar/Grammar

import_stmt:
import_name | import_from
import_name:
  'import' dotted_as_names
import_from:
  ('from' (('.' | '...')* dotted_name | ('.' | '...')+)
'import' ('*' | '(' import_as_names ')' | import_as_names))
import_as_name:
  NAME ['as' NAME]
dotted_as_name:
  dotted_name ['as' NAME]
import_as_names:
  import_as_name (',' import_as_name)* [',']
dotted_as_names:
  dotted_as_name (',' dotted_as_name)*
dotted_name:
  NAME ('.' NAME)*

A.2 Tokens and Keywords

http://docs.python.org/dev/library/token.html

http://hg.python.org/cpython/file/default/Parser/tokenizer.c#l49
http://hg.python.org/cpython/file/default/Include/token.h

NAME
STAR
DOT
ELLIPSIS
COMMA

keywords:
-----------
'import'
'from'
'as'

for a while, import had 3 keywords all to itself!!!

A.3 AST

http://docs.python.org/dev/library/ast.html

http://hg.python.org/cpython/file/default/Parser/Python.asdl

http://hg.python.org/cpython/file/default/Include/Python-ast.h

A.4 Opcodes

http://docs.python.org/dev/library/dis.html

http://hg.python.org/cpython/file/default/Include/opcode.h

http://hg.python.org/cpython/file/default/Python/compile.c

http://hg.python.org/cpython/file/default/Python/ceval.c

A.5 Examples of Looking under the Hood

Each of these examples will use the following code snippets:

1. import os
2. import os as os_mod
3. import os as os_mod, sys
4. import os.path

def f(x):

print(x)

return x

A.5.1 Investigate Grammar Using parser Module

From ActiveState Recipe #...:

A.5.2 Investigate Grammar Using tokenize Module

From ActiveState Recipe #...:

from io import BytesIO
from tokenize import tokenize

code = """def f(x):

print(x)

return x

"""

tokens = tokenize(BytesIO(code.encode('utf-8')).readline)
for toknum, tokval, _, _, _ in tokens:

print(toknum, tokval)

A.5.3 Investigate AST Using ast Module

From ActiveState Recipe #...:

...

A.5.4 Investigate Opcodes Using dis Module

From ActiveState Recipe #...:

...

A.6 The __future__ Module

http://www.python.org/dev/peps/pep-0236/
http://docs.python.org/dev/library/__future__.html

feature	optional in	mandatory in	effect
nested_scopes	2.1.0b1	2.2	PEP 227: Statically Nested Scopes
generators	2.2.0a1	2.3	PEP 255: Simple Generators
division	2.2.0a2	3.0	PEP 238: Changing the Division Operator
absolute_import	2.5.0a1	2.7	PEP 328: Imports: Multi-Line and Absolute/Relative
with_statement	2.5.0a1	2.6	PEP 343: The “with” Statement
print_function	2.6.0a2	3.0	PEP 3105: Make print a function
unicode_literals	2.6.0a2	3.0	PEP 3112: Bytes literals in Python 3000

A.7 The Default import Handler

As already implied by sections 1.3 and 5.3, the import process is not as complex as you might expect. However, it is opaque enough that a thorough exposition would be worth it. Here is the entire process in one chunk of code:

...

* if a directory contains both a module file and a package directory, the package will be imported for the name and not the module.

Notes and References:
[1]

Appendix B: Import-related Files

B.1 CPython

Python/import.c
Python/importdl.c
Python/importdl.h
Include/import.h
Python/sysmodule.c
Python/pythonrun.c

B.2 Standard Library

Lib/importlib/*.py
Lib/pkgutil.py

B.3 Tests

Lib/Test/...

B.4 sys.path related

http://docs.python.org/library/sys.html#sys.path
http://docs.python.org/c-api/init.html#Py_GetPath
http://hg.python.org/cpython/file/default/Modules/getpath.c#l21

http://docs.python.org/dev/library/site.html

1. calculate the 4 site-packages dirs
2. add them to sys.path
3. check for and execute .pth files in those site-packages dirs
4. calculate user site-packages

site.getsitepackages()
Unix/Mac:
<sys.prefix>/lib/python#.#/site-packages

<sys.exec_prefix>/lib/python#.#/site-packages

<sys.prefix>/lib/site-python
<sys.exec_prefix>/lib/site-python

Windows:

<sys.prefix>/
<sys.exec_prefix>/

<sys.prefix>/lib/site-packages
<sys.exec_prefix>/lib/site-packages

B.5 site related

http://docs.python.org/dev/library/site.html

* module: site
* module: sitecustomize
* module: usercustomize

site.getusersitepackages()
Notes and References:
[1]

still lives: http://docs.python.org/library/imputil.html#examples

(ihooks)
introduced (1.3): http://hg.python.org/cpython-fullhistory/rev/ec0b42889243
removed (3.0): http://docs.python.org/release/2.6.2/library/undoc.html#miscellaneous-useful-utilities
http://pydoc.org/2.4.1/ihooks.html

---

The versions and dates are derived from a post on Guido's "History of Python" blog. I've correlated the entries in section B.1 to versions by either explicit reference or by matching their commits to a version. Section B.2 also maps commits to versions. In both cases, I did my best to determine that mapping, but some may be off by a version.

B.1 The Timeline

Initial (1990)
* Checks sys.modules
* Loads modules from sys.path or current dir (if sys.path is empty)
* Supports IMPORT_NAME and IMPORT_FROM opcodes
* No support for .pyc files
* No support for packages
* No support for C extension modules?
* No ImportError

Python 0.9.1 (Feb. 1991)
* builtin module support (C extention modules)
Python 1.0 (1994)
* Support for extension modules
* Support for .pyc files
Python 1.2 (1995)
* (Python/bltinmodule.c) __import__() builtin introduced
* (Python/import.c) dynamic module support factored out into importdl.c
Python 1.3 (1995)
* "ni" module introduced
Python 1.4 (1996)
http://docs.python.org/release/1.4/ref/
*
Python 1.5 (1998)
* Support for packages
* "site-packages" and "site-python" directories introduced
* "__all__" introduced
* "ni" module deprecated
* (Python/import.c) PyImport_Import() introduced

Python 2.0 (2000)
* PEP 221 -- Import As
Python 2.1 (2001)
* PEP 235 -- Import on Case-Insensitive Platforms
Python 2.2 (2001)
*
Python 2.3 (2003)
* PEP 273 -- Import Modules from Zip Archives
* PEP 302 -- New Import Hooks
Python 2.4 (2004)
* PEP 328 -- Imports: Multi-Line and Absolute/Relative (multi-line portion)
Python 2.5 (2006)
* PEP 328 (relative imports portion)
* PEP 338 -- Executing modules as scripts
Python 2.6/3.0 (2008)
* PEP 366 -- Main module explicit relative imports
* PEP 370 -- Per user site-packages directory
Python 3.0 (2008)
* reload removed from builtins[1]
* ihooks module removed from stdlib[2]
* imputil module removed from stdlib
Python 3.1 (2009)
* importlib module added[3]
Python 3.2 (2011)
* PEP 3147 -- PYC Repository Directories
Python 3.3 (2012)
* see appendix D

B.2 Timeline of Import-related Commits

http://hg.python.org/cpython-fullhistory/log/62bdb1cbe0f5/Python/import.c?revcount=120

Python 0.9.0 (1991)

Python 0.9.1 (1991)

Python 0.9.2 (1991)

Python 0.9.4 (1991)

Python 0.9.5 (1992)

Python 0.9.6 (1992)

Python 0.9.7 (1992)

Python 0.9.8 (1993)

Python 0.9.9 (1993)

Python 1.0.0 (1994)

Python 1.0.2 (1994)

Python 1.0.3 (1994)

Python 1.0.4 (1994)

Python 1.1 (1994)

Python 1.1.1 (1994)

Python 1.2 (1995)

Python 1.3 (1995)

Python 1.4 (1996)

Python 1.5 (1998)

Python 1.5.1 (1998)

Python 1.5.2 (1999)

Python 1.6 (2000)

Python 2.0 (2000)

Python 2.1 (2001)

Python 2.2 (2001)

Python 2.3 (2003)

Python 2.4 (2004)

Python 2.5 (2006)

Python 2.6 (2008)

Python 3.0 (2008)

Python 2.7 (2010)

Python 3.1 (2010)

Python 3.2 (2011)

Python 3.3 (2012)

Notes and References:
[1] http://docs.python.org/release/3.0.1/whatsnew/3.0.html#builtins
[2] http://docs.python.org/library/undoc.html#miscellaneous-useful-utilities
[3] http://hg.python.org/cpython/rev/aac51b70c74c

Appendix D: Ongoing Core Efforts to Improve Importing

D.1 PEPs

* PEP 369 -- Post import hooks

* PEP 382 -- Namespace Packages

* PEP 395 -- Module Aliasing

* PEP 402 -- Simplified Package Layout and Partitioning

* PEP ??? -- import engine

Rejected PEPs:

* PEP 299 -- Special __main__() function in modules

* PEP 3122 -- Delineation of the main module

D.2 Projects

* importlib.__import__ as the default builtins.__import__

Currently in Python, "builtin___import__()" in Python/bltinmodule.c makes a call to PyImport_ImportModuleLevelObject. Brett Cannon is working on making importlib.__import__ the default import call[1].

* the __experimental__ module

http://mail.python.org/pipermail/python-ideas/2010-June/007357.html
http://mail.python.org/pipermail/python-ideas/2011-August/011278.html

like the __future__ module, but for less-stable APIs that are likely to go in
focus on stdlib (room for experimental syntax too?)
(higher exposure testing)

Notes and References:
[1] http://bugs.python.org/issue2377

Appendix E: Imports in Alternate Python Implementations

E.1 PyPy
http://readthedocs.org/search/project/?q=import&selected_facets=project%3Apypy
http://codespeak.net/pypy/dist/pypy/doc/clr-module.html
http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html
https://bugs.pypy.org/issue367

E.2 Jython
http://www.google.com/search?sitesearch=www.jython.org&q=import&Search=Search

E.3 IronPython
http://ironpython.codeplex.com/wiki/search?tab=Home&SearchText=import

Notes and References:
[1]

Appendix F: Easter Eggs

The Python devs are a playful lot.

F.1 import this
http://www.wefearchange.org/2010/06/import-this-and-zen-of-python.html

F.2 import antigravity
http://xkcd.com/353/
http://python-history.blogspot.com/2010/06/import-antigravity.html

F.3 from __future__ import flufl
http://www.python.org/dev/peps/pep-0401/
http://sayspy.blogspot.com/2009/03/guido-has-retired-as-bdfl.html

F.4 from __future__ import braces

F.5 import __hello__

Appendix G: Import Hook Examples

G.1 Naively Track Imports

Sometimes you may want to track what is getting imported when you make a call. Here's how you can do it.

import_tracker.py:

import sys
class ImportTracker:
  def __init__(self):
      self.modules = []
  def find_module(name, path=None):
      self.modules.append(name)
  def enable(self):
      sys.meta_path.insert(0, self)
  def disable(self):
      sys.meta_path.remove(self)

some_module.py

from import_tracker import ImportTracker
tracker = ImportTracker()
tracker.enable()
...
print(tracker.modules)

G.2 Import Tracking, Take 2

Check out the code here.

Interestingly, the behavior is different for this example if you use importlib's `__import__` vs. the default `builtins.__import__`. This is because of how sys.modules is treated differently between the two.

G.3 Statement Local Namespaces (given statement)

Go take a look here.

G.4 Protecting a High-Latency Filesystem

Sometimes you have in your sys.path a directory from a network drive (perhaps an NFS mount) or other IO-restricted device. In that case you may to limit how import looks for files to mitigate the number of stat calls. Here's a simple example of how to do so:

... (placeholder)

? lots of stat calls during normal imports?

G.5 Customizing Access to a Specific Module Path

* using sys.path_hooks (placeholder)

G.6 PEPS 382 and 402 as Import Hooks

Both of these will work as import hooks

PEP 382 (placeholder)

PEP 402 (placeholder)

G.7 Import Engine as an Import Hook

maybe... (placeholder)

Notes and References:
[1]

Appendix H: Other Import-Related Examples

* __import__
* "importing" straight from a file
* lazy imports
* "from <name> import *", manually

* "from <name> import <name, ...>", manually

H.1 ...

http://code.activestate.com/recipes/577792/

Notes and References:
[1]

Appendix I: Imports in the Python Community

I.1 Community Uses of Import Hooks

http://www.google.com/codesearch#search/&q=sys.meta_path%20lang:%5Epython$%20case:yes&type=cs
http://www.google.com/codesearch#search/&q=sys.meta_path.append%20lang:%5Epython$%20case:yes&type=cs
http://www.google.com/codesearch#search/&q=sys.path_hooks%20lang:%5Epython$%20case:yes&type=cs
http://www.google.com/codesearch#search/&q=sys.path_hooks.append%20lang:%5Epython$%20case:yes&type=cs
http://www.google.com/codesearch#search/&q=sys.path_importer_cache%20lang:%5Epython$%20case:yes&type=cs

http://codespeak.net/pypy/dist/pypy/doc/clr-module.html

http://packages.python.org/fs/expose/importhook.html#module-fs.expose.importhook
http://selenic.com/hg/file/default/mercurial/demandimport.py
https://github.com/ipython/ipython/blob/master/IPython/lib/deepreload.py
http://code.google.com/p/backport/source/browse/backport.py#70

I.2 Community Import Solutions

- PyLT
- backport-importlib
- http://doc.pylib.org/en/latest/
- http://selenic.com/hg/file/default/mercurial/demandimport.py
- http://washort.twistedmatrix.com/search/label/exocet
- http://peak.telecommunity.com/DevCenter/Importing
- http://code.activestate.com/recipes/577958/

Notes and References:
[1]

Appendix J: Import Tips and Tricks

J.1 block imports on the current working directory

By default Python will look for a module in your current working directory before trying the stdlib. The explicit relative import syntax of 2.7 help with this, but only to an extent.

To completely keep Python from trying the CWD, simply run "sys.path.remove('')" and optionally follow that with "sys.path.append('')".

So the question remains, when did the empty string get added to (the front of) sys.path, and why?
...

Notes and References:
[1]

Appendix K: Troubleshooting Imports

K.1 Causes of ImportError

- turn into ImportError subclasses, __cause__

K.2 Other Exceptions During Import

* SyntaxError
* IOError?

K.3 Common Import-related Problems

K.3.1 circular imports

K.3.2 module behaves differently when run as script

* don't run non-scripts as scripts; import in a test script

K.3.3 imports in scripts and at REPL behave differently than expected

* minimize the amount of code in scripts
* relative imports behave a little differently in scripts

K.3.4 reloading

K.3.5 no .pyc created

* .pyc for <name> is created only for "import <name>"
* caching was turned off
* file is actually in __pycache__ directory
* python run with -O flag (optimized) so .pyc files created

K.3.6 undesired import conflicts with files in CWD

By default the current working directory is first on sys.path. If this is causing trouble, you can move it to the back of the line:

try: sys.path.remove('')
except ValueError: pass
else: sys.path.append('')

Alternatively, you could remove it entirely (don't append it back on).

K.3.7 stale code

http://bugs.python.org/issue8087

K.3.8 orphaned pyc file getting used

If a directory has a pyc file but no matching py file, the module will be loaded from the pyc file directly. Starting with PEP 3147 (Python 3.2), orphaned pyc files in the __pycache__ directory are NOT loaded. The behavior use of pyc files otherwise stays the same. Either way, if you don't want the module to be loaded from an orphaned pyc file, delete that file. Also see [1].

K.3.9 alternate sys.modules ignored

Sometimes it can be helpful to replace sys.modules with a custom dictionary. However, in CPython, this does not affect the underlying dictionary that was originally bound to sys.modules. That is a separate part of the interpreter state. The behavior of the default _builtin___import__(), implemented in Python/import.c, actually uses this underlying dictionary through PyImport_GetModuleDict(), rather than explicitly pulling sys.modules. So your fancy-pantsy sys.modules is never used.

Luckily, importlib does explicitly use sys.modules, so if you switch over to that it should work just fine. This will be an even smaller issue once importlib's __import__ because the default builtin.

(see http://bugs.python.org/issue12633).

K.3.10 import loads some other mysterious module

If you have a module file in your sys.path, and you try to import it, sometimes the import will succeed but the module will be the wrong one. This can be both mysterious and perplexing.

The first thing to do is to see if you have a package (directory with a __init__.py) by the same name in the same place as that module file. If so, Python will import from the package instead of the module. To verify this, import the module: "import <name>" and then check the module in sys.modules: "import sys; print(sys.modules['<name>'])". You should see it pointing to the __init__.py of the package instead of the module file you were expecting.

Notes and References:

[1] http://bugs.python.org/issue13267#msg146425

<cookbook recipes>

<others>

http://www.youtube.com/watch?v=DkW5CSZ_VII
http://aroberge.blogspot.com/2006/02/python-wish-new-meaning-for-import-as.html

L.2 PyCon 2012 Talk: Getting the Most Out of Python Imports

http://us.pycon.org/2012/

1. high-level overview of imports/modules in Python
2. history of imports in Python
3. introduction to PEP 302 import hooks
4. import hook examples
5. review of supplemental information
6. questions

L.3 PyCon 2012 Talk: A History of the Python Import Statement

1. high-level overview of imports/modules in Python
2. brief history of Python
3. influences on the initial import statement
4. early additions
5. the intervening years
6. recent additions
7. on-going efforts
8. questions

L.4 To Do

- polish the content
- add more examples
- separate into different pages?

Notes and References:
[1]

Glossary

finder

import hook

importer

loader

module

module name

package

Pages

Python Imports