How do you get your Python interpreter to find modules that are not located in your current working directory? The answer is … you tell it where to look.
When you type something like
from my_module import my_function
Python searches a collection of directories (i.e., folders) on your computer. If the
directory containing <my_module.py> is not in this collection, you will
receive an ImportError. This can be frustrating if you know the file exists,
and even more so if you know where it exists.
In this post, we will take a brief look at how to add paths to the collection of directories searched by Python.
Paths
A path is essentially a set of directions to a file:
/Users/username/modules/my_module.py [Mac OS X, Linux]
C:\modules\my_module.py [Windows]
It tells your operating system how to navigate from a fixed starting point —
the “root directory” / in Unix-based systems, or C:\ in Windows — through
a collection of folders to the desired file.
In this case, there is a folder called <modules/> that contains the desired
file, <my_module.py>. Let’s suppose that you are writing a script called
<my_script.py> in your working directory:
/Users/username/scratch/my_script.py [Mac OS X, Linux]
C:\scratch\my_script.py [Windows]
You want Python to import <my_module.py> when you run the script, but it
cannot find the file.
Below are 4 options for adding the path for your module to Python’s collection of paths. None are extremely complicated, but the last two require working with the command line and editing startup files on your operating system. If you are unfamiliar or uncomfortable with this, you might prefer the first two methods.
You will need to replace directory names like /Users/username/ or
C:\scratch\ with the correct directories on your own system for these examples
to work.
Using The sys Module
There is a module called sys that lets you modify the collection of paths from
within Python. According to its documentation,
This module provides access to some objects used or maintained by the
interpreter and to functions that interact strongly with the interpreter.
One of these objects is a list called sys.path. Again referring to the
documentation,
path -- module search path
All we need to do is add the path to <my_module.py> to sys.path. On
Unix-like systems, the commands are
import sys
sys.path.append('/Users/username/modules')
On Windows, it is easier to use a raw string to specify the path:
import sys
sys.path.append(r'c:\modules')
This will only affect the path in the current Python session. If you quit and restart, you will need to add the path again.
Using Spyder
If you are using the Spyder IDE, you can permanently add a directory to its
collection of paths. Chose the “PYTHONPATH manager” from the menu (python >
PYTHONPATH manager) as shown below:
This will open a window that shows a list of directories added to Spyder’s collection of paths. If you have not modified this list already, it will probably contain your working directory and nothing else. At the bottom of this window, there is an icon for “Add Path”. Click on this. Now you can find the directory you want to add, as if you were opening a file.
In this case, we navigate to
/Users/username/modules
and select “Choose”.
Once you have selected a directory, it will become a permanent addition to Python’s paths within Spyder. You will need to quit Spyder and restart or “Restart Kernel” for the changes to take effect.
This is a convenient way to access modules that are not part of the Anaconda distribution from within Spyder.
Using the Command Line
If you are working from a command line, you can add your module directory to
Python’s collection of paths by setting the PYTHONPATH variable before you
start a Python session or run a script:
$ export PYTHONPATH=${PYTHONPATH}:/Users/username/modules
$ python my_script.py
No matter what directory you are working in, Python now knows where to find
<my_module.py>.
If you want to permanently add your module directory to PYTHONPATH, you can
add the export command above to your <.profile> or <.bashrc> file.
Startup Files
Another option is to create a Python startup file — a Python script that will
be run at the beginning of every interactive Python session (but not when you
run scripts from the command line). First, create a file such as
<python_startup.py> in your home directory. Then, enter Python commands
exactly as you would type them at the command prompt:
# python_startup.py
# Startup script for interactive Python sessions.
import sys
sys.path.append('/Users/username/modules')
Next, edit your <.profile> or <.bashrc> file and add the line
export PYTHONSTARTUP=/Users/username/python_startup.py
And that’s it. Now, every time you start a new Python session, you can access your own modules.
IPython does not use the PYTHONSTARTUP variable. If you want to add a startup
script for IPython, place a file like <python_startup.py> in IPython’s
directory of startup scripts:
/Users/username/.ipython/default_profile/startup/
Now your modules will be available in every IPython session, including Spyder.
Summary
There are several ways to show Python where to find modules. For a single
session, appending a path to the sys.path variable is an easy solution. For a
permanent change, you can use Spyder’s PYTHONPATH manager or edit startup
files from the command line. These methods will allow you to access modules
without copying or moving files into your working directory, and you can easily
access modules you write yourself or modules you download from the Web.
Comments