Importing files from different folder

Asked 2023-09-20 20:30:12 View 699,599

I have this folder structure:

application
├── app
│   └── folder
│       └── file.py
└── app2
    └── some_folder
        └── some_file.py

How can I import a function from file.py, from within some_file.py? I tried:

from application.app.folder.file import func_name

but it doesn't work.

  • Related: stackoverflow.com/q/43476403/674039 - anyone
  • Reading the official documentation helped me a lot! docs.python.org/3/reference/… - anyone
  • If you have a dash in the name of the subfolder, it SHOULD BE UNDERSCORE. For example my-package and inside you have my_app folder and tests folder. If my_app is named my-app, you will have import problems - anyone
  • Neither application nor app1, app2, folder, some_folder are packages, and do not contain __init__.py, right? If you're going to be doing a lot of this, time to make them a package. - anyone
  • The fact that this is so hard and there are multiple answers about it, some of which don't work or are hacks, is probably the worst, saddest thing about Python. - anyone

Answers

Note: This answer was intended for a very specific question. For most programmers coming here from a search engine, this is not the answer you are looking for. Typically you would structure your files into packages (see other answers) instead of modifying the search path.


By default, you can't. When importing a file, Python only searches the directory that the entry-point script is running from and sys.path which includes locations such as the package installation directory (it's actually a little more complex than this, but this covers most cases).

However, you can add to the Python path at runtime:

# some_file.py
import sys
# caution: path[0] is reserved for script path (or '' in REPL)
sys.path.insert(1, '/path/to/application/app/folder')

import file

Answered   2023-09-20 20:30:12

  • sys.path.append('/path/to/application/app/folder') is cleaner imo - anyone
  • @pseudosudo: Yep, it is, but inserting it at the beginning has the benefit of guaranteeing that the path is searched before others (even built-in ones) in the case of naming conflicts. - anyone
  • @kreativitea - sys.path returns a list, not a deque, and it'd be silly to convert the list to a deque and back. - anyone
  • Is it considered as a pythonic way to manage .py files in folders? I'm wondering... why it's not supported by default? it doesn't make sense to maintain all .py files in a single directory.. - anyone
  • @Ofir: No, this isn't a nice clean pythonic solution. In general, you should be using packages (which are based on directory trees). This answer was specific to the question asked, and for some reason continues to accrue a large number upvotes. - anyone

There is nothing wrong with:

from application.app.folder.file import func_name

Just make sure folder also contains an __init__.py. This allows it to be included as a package. I am not sure why the other answers talk about PYTHONPATH.

Answered   2023-09-20 20:30:12

  • Because this doesn't cover the cases where modifying PYTHONPATH is necessary. Say you have two folders on the same level: A and B. A has an __init.py__. Try importing something from B within A. - anyone
  • What's inside the init.py or __init__.py file? - anyone
  • @Xinyang It can be an empty file. Its very existence tells Python to treat the directory as a package. - anyone
  • This is not currently the highest voted answer, but it IS the most correct answer (for most cases). Simply create a package. It's not hard. The other answers are needed because sometimes you might be restricted from certain system changes (creating or modifying a file, etc) like during testing. - anyone
  • Whatever I try, this won't work. I want to import from a "sibling" directory, so one up one down. All have __ init __.py's, including parent. Is this python 3 -specific? - anyone

When modules are in parallel locations, as in the question:

application/app2/some_folder/some_file.py
application/app2/another_folder/another_file.py

This shorthand makes one module visible to the other:

import sys
sys.path.append('../')

Answered   2023-09-20 20:30:12

  • As a caveat: This works so long as the importing script is run from its containing directory. Otherwise the parent directory of whatever other directory the script is run from will be appended to the path and the import will fail. - anyone
  • To avoid that, we can get the parent directory of file sys.path.append(os.path.dirname(os.path.abspath(__file__))) - anyone
  • That didn't work for me - I had to add an additional dirname in there to climb back up to the parent, so that running cli/foo.py from the command line was able to import cli.bar - anyone
  • @Rahul, your solution doesn't work for interactive shells - anyone
  • If you run it from your root folder (ie. application folder), you are probably fine with sys.path.append('.') then importing the module by using from app2.some_folder.some_file import your_function. Alternatively what works for me is running python3 -m app2.another_folder.another_file from root folder. - anyone

First import sys in name-file.py

 import sys

Second append the folder path in name-file.py

sys.path.insert(0, '/the/folder/path/name-package/')

Third Make a blank file called __ init __.py in your subdirectory (this tells Python it is a package)

  • name-file.py
  • name-package
    • __ init __.py
    • name-module.py

Fourth import the module inside the folder in name-file.py

from name-package import name-module

Answered   2023-09-20 20:30:12

  • With name-folder being right below name-file.py, this should work even without the sys.path.insert-command. As such, the answer leaves the question, if this solution works even when name-folder is located in an arbitrary location. - anyone
  • are you saying that I have to hardcode the path to the script? This means that the solution is not portable. Also the question is how to access from one subfolder to the other. Why not following the name convention and file structure of the original question? - anyone
  • @Giacomo You don't have to hardcode anything. Just pass it as a parameter to the script. - anyone
  • Caution: insert at position 1, since position 0 is the script path (or '' in REPL). - anyone
  • Thanks for this answer @Alex Montoya! Upvoted! - anyone

I think an ad hoc way would be to use the environment variable PYTHONPATH as described in the documentation: Python2, Python3

# Linux and OS X
export PYTHONPATH=$HOME/dirWithScripts/:$PYTHONPATH

# Windows
set PYTHONPATH=C:\path\to\dirWithScripts\;%PYTHONPATH%

Answered   2023-09-20 20:30:12

  • Wait, would I replace myScripts with the filename? - anyone
  • no, with the path of the directory to your .py file - anyone
  • Unfortunately, if you are using Anaconda, this won't work, since under the hood PYTHONPATH is not really used internally ! - anyone
  • For (recent) changes in anaconda, see this SO for workflows and comments for work-arounds: stackoverflow.com/questions/17386880/… Generally speaking, build and install small packages instead of hacking the import dirs. - anyone

Try Python's relative imports:

from ...app.folder.file import func_name

Every leading dot is another higher level in the hierarchy beginning with the current directory.


Problems? If this isn't working for you then you probably are getting bit by the many gotcha's relative imports has. Read answers and comments for more details: How to fix "Attempted relative import in non-package" even with __init__.py

Hint: have __init__.py at every directory level. You might need python -m application.app2.some_folder.some_file (leaving off .py) which you run from the top level directory or have that top level directory in your PYTHONPATH. Phew!

Answered   2023-09-20 20:30:12

  • This doesn't seem to work if your directory's name starts with a number (e.g. import ..70_foo.test is not allowed) - anyone
  • Wow, this actually worked. I didn't know you could "go up" a directory by using multiple dots. - anyone
  • @secluded sure, but import 70_foo isn't allowed either. Both package and module names have to be legal identifier names. - anyone

The issue is that Python is looking in the wrong directory for the file. To solve this, try using relative import. Change

from application.app.folder.file import func_name

to:

from .application.app.folder.file import func_name

Adding the dot instructs Python to look for the application folder within the current folder, instead of in the Python install folder.

Answered   2023-09-20 20:30:12

Given a folder structure like

├── main.py
└── myfolder
    └── myfile.py

Where myfile.py contains

def myfunc():
    print('hello')

To call myfunc from main.py, use:

from myfolder.myfile import myfunc
myfunc()

Answered   2023-09-20 20:30:12

  • adding an init.py (empty) configuration file in myfolder worked for me on linux (y) - anyone
  • @Vincent did you mean __init__.py? - anyone
  • This is completely unrelated to the question which asks about importing files from a different branch of the file tree than the current working directory. - anyone
  • Lovely diagrams that expressly ignore OP's question. - anyone
  • This is not what the OP was questioning about. - anyone

In Python 3.4 and later, you can import from a source file directly (link to documentation). This is not the simplest solution, but I'm including this answer for completeness.

Here is an example. First, the file to be imported, named foo.py:

def announce():
    print("Imported!")

The code that imports the file above, inspired heavily by the example in the documentation:

import importlib.util

def module_from_file(module_name, file_path):
    spec = importlib.util.spec_from_file_location(module_name, file_path)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module

foo = module_from_file("foo", "/path/to/foo.py")

if __name__ == "__main__":
    print(foo)
    print(dir(foo))
    foo.announce()

The output:

<module 'foo' from '/path/to/foo.py'>
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'announce']
Imported!

Note that the variable name, the module name, and the filename need not match. This code still works:

import importlib.util

def module_from_file(module_name, file_path):
    spec = importlib.util.spec_from_file_location(module_name, file_path)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module

baz = module_from_file("bar", "/path/to/foo.py")

if __name__ == "__main__":
    print(baz)
    print(dir(baz))
    baz.announce()

The output:

<module 'bar' from '/path/to/foo.py'>
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'announce']
Imported!

Programmatically importing modules was introduced in Python 3.1 and gives you more control over how modules are imported. Refer to the documentation for more information.

Answered   2023-09-20 20:30:12

  • I don't know if anyone even tried to understand this, but I think that it's too complicated. - anyone
  • This is the only solution that worked for me. I have the same file name in different directories. - anyone
  • how to import everything in a file? - anyone
  • That was the only solution that worked for me. - anyone
  • This should be the correct solution for import a local module that you don't want to publish to the package repository. - anyone

Using sys.path.append with an absolute path is not ideal when moving the application to other environments. Using a relative path won't always work because the current working directory depends on how the script was invoked.

Since the application folder structure is fixed, we can use os.path to get the full path of the module we wish to import. For example, if this is the structure:

/home/me/application/app2/some_folder/vanilla.py
/home/me/application/app2/another_folder/mango.py

And let's say that you want to import the mango module. You could do the following in vanilla.py:

import sys, os.path
mango_dir = (os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
+ '/another_folder/')
sys.path.append(mango_dir)
import mango

Of course, you don't need the mango_dir variable.

To understand how this works look at this interactive session example:

>>> import os
>>> mydir = '/home/me/application/app2/some_folder'
>>> newdir = os.path.abspath(os.path.join(mydir, '..'))
>>> newdir
    '/home/me/application/app2'
>>> newdir = os.path.abspath(os.path.join(mydir, '..')) + '/another_folder'
>>> 
>>> newdir
'/home/me/application/app2/another_folder'
>>> 

And check the os.path documentation.

Also worth noting that dealing with multiple folders is made easier when using packages, as one can use dotted module names.

Answered   2023-09-20 20:30:12

From what I know, add an __init__.py file directly in the folder of the functions you want to import will do the job.

Answered   2023-09-20 20:30:12

  • only if the script that wants to include that other directory is already in the sys.path - anyone
  • I used sys.path.append(tools_dir) on Windows and I don't need to add a __init__.py' file in my directory tools_dir` - anyone
  • __init__.py has effectively nothing to do with it. - anyone
  • Setting as ENV variable worked for me export PYTHONPATH="${PYTHONPATH}:/path/to/python/project" - anyone

I was faced with the same challenge, especially when importing multiple files, this is how I managed to overcome it.

import os, sys

from os.path import dirname, join, abspath
sys.path.insert(0, abspath(join(dirname(__file__), '..')))

from root_folder import file_name

Answered   2023-09-20 20:30:12

  • You answer would be more helpful if you could explain what it does differently from an ordinary import? - anyone
  • I had /path/dir1/__init__.py and /path/dir1/mod.py. For /path/some.py from dir1.mod import func worked. When in /path/dir2/some.py it only worked after I copied and pasted the above answer at the top of the file. Didn't want to edit my path since not every python project I have is in /path/. - anyone
  • My test files were moved to another directory when running it using bazel after adding this import, the test files were able to reference the dependencies. - anyone

This worked for me in Python 3 on Linux:

import sys
sys.path.append(pathToFolderContainingScripts)
from scriptName import functionName #scriptName without .py extension

Answered   2023-09-20 20:30:12

  • sys.path.append("/home/linux/folder/") — Make sure do not use a shortcut e.g. "~/folder/" - anyone
  • This is the easiest answer; works for Windows as well. - anyone

Considering application as the root directory for your Python project, create an empty __init__.py file in the application, app and folder folders. Then in your some_file.py, make changes as follows to get the definition of func_name:

import sys
sys.path.insert(0, r'/from/root/directory/application')

from application.app.folder.file import func_name ## You can also use '*' wildcard to import all the functions in file.py file.
func_name()

Answered   2023-09-20 20:30:12

  • should be: sys.path.insert(0, r'/from/root/directory') - anyone

One way is to create a package and use absolute import to access other modules from the package. Start the program from a script at the root of the package. This structure allows using and accessing sub-packages, parent package, and sibling packages and modules.

As an example, try creating the following folder structure:

package/
├── __init__.py
├── main_module.py
├── module_0.py
├── subpackage_1/
|   ├── __init__.py
|   ├── module_1.py
|   └── sub_subpackage_3/
|       ├── __init__.py
|       └── module_3.py
└── subpackage_2/
    ├── __init__.py
    └── module_2.py

Contents of main_module.py:

import subpackage_1.module_1

Contents of module_0.py:

print('module_0 at parent directory, is imported')

Contents of module_1.py:

print('importing other modules from module_1...')
import module_0
import subpackage_2.module_2
import subpackage_1.sub_subpackage_3.module_3

Contents of module_2.py:

print('module_2 at same level directory, is imported')

Contents of module_3.py:

print('module_3 at sub directory, is imported')

Leave all __init__.py files empty.

Now run main_module.py; the output will be

importing other modules from module_1...
module_0 at parent directory, is imported
module_2 at same level directory, is imported
module_3 at sub directory, is imported

Answered   2023-09-20 20:30:12

├───root
│   ├───dir_a
│   │   ├───file_a.py
│   │   └───file_xx.py
│   ├───dir_b
│   │   ├───file_b.py
│   │   └───file_yy.py
│   ├───dir_c
│   └───dir_n

You can add the parent directory to PYTHONPATH, in order to achieve that, you can use OS depending path in the "module search path" which is listed in sys.path. So you can easily add the parent directory like following:

# file_b.py

import sys
sys.path.insert(0, '..')

from dir_a.file_a import func_name

Answered   2023-09-20 20:30:12

  • The magic here is to use '.' instead of '/' to indicate the path relative to current path. - anyone

This works for me on Windows:

# some_file.py on mainApp/app2
import sys
sys.path.insert(0, sys.path[0]+'\\app2')

import some_file

Answered   2023-09-20 20:30:12

In my case I had a class to import. My file looked like this:

# /opt/path/to/code/log_helper.py
class LogHelper:
    # stuff here

In my main file I included the code via:

import sys
sys.path.append("/opt/path/to/code/")
from log_helper import LogHelper

Answered   2023-09-20 20:30:12

  • @not2qubit sys wasn't imported in the answer. - anyone

I bumped into the same question several times, so I would like to share my solution.

Python Version: 3.X

The following solution is for someone who develops your application in Python version 3.X because Python 2 is not supported since Jan/1/2020.

Project Structure

In python 3, you don't need __init__.py in your project subdirectory due to the Implicit Namespace Packages. See Is init.py not required for packages in Python 3.3+

Project 
├── main.py
├── .gitignore
|
├── a
|   └── file_a.py
|
└── b
    └── file_b.py

Problem Statement

In file_b.py, I would like to import a class A in file_a.py under the folder a.

Solutions

#1 A quick but dirty way

Without installing the package like you are currently developing a new project

Using the try catch to check if the errors. Code example:

import sys
try:
    # The insertion index should be 1 because index 0 is this file
    sys.path.insert(1, '/absolute/path/to/folder/a')  # the type of path is string
    # because the system path already have the absolute path to folder a
    # so it can recognize file_a.py while searching 
    from file_a import A
except (ModuleNotFoundError, ImportError) as e:
    print("{} fileure".format(type(e)))
else:
    print("Import succeeded")

#2 Install your package

Once you installed your application (in this post, the tutorial of installation is not included)

You can simply

try:
    from __future__ import absolute_import
    # now it can reach class A of file_a.py in folder a 
    # by relative import
    from ..a.file_a import A  
except (ModuleNotFoundError, ImportError) as e:
    print("{} fileure".format(type(e)))
else:
    print("Import succeeded")

Happy coding!

Answered   2023-09-20 20:30:12

  • for more info about absolute imports - anyone
  • your first proposed solution worked for me using sys.path.insert(1, '../a/') which I think is better than writing the full path. - anyone
  • In case someone has a local package that you would like to import instead of the system package (THAT HAS THE SAME NAME) please use sys.path.insert(1,'folder-to-grab-package-from') instead of sys.append('folder-to-grab-package-from') - anyone
  • You say "You can simply" but unfortunately seven lines of code to import a single file is not simple at all! - anyone

I'm quite special: I use Python with Windows!

I just complete information: for both Windows and Linux, both relative and absolute paths work into sys.path (I need relative paths because I use my scripts on the several PCs and under different main directories).

And when using Windows, both \ and / can be used as a separator for file names and of course you must double \ into Python strings. Here are some valid examples:

sys.path.append('c:\\tools\\mydir')
sys.path.append('..\\mytools')
sys.path.append('c:/tools/mydir')
sys.path.append('../mytools')

(Note: I think that / is more convenient than \, even if it is less 'Windows-native', because it is Linux-compatible and simpler to write and copy to Windows Explorer)

Answered   2023-09-20 20:30:12

  • os.path.join('tools', 'mydir') - anyone

Instead of just doing an import ..., do this :

from <MySubFolder> import <MyFile>

MyFile is inside the MySubFolder.

Answered   2023-09-20 20:30:12

The following worked for me:

OS: Windows 10

Python: v3.10.0

Note: Since I am Python v3.10.0, I am not using __init__.py files, which did not work for me anyway.

application
├── app
│   └── folder
│       └── file.py
└── app2
    └── some_folder
        └── some_file.py

WY Hsu's first solution worked for me. I have reposted it with an absolute file reference for clarity:

import sys
sys.path.insert(1, 'C:\\Users\\<Your Username>\\application')
import app2.some_folder.some_file

some_file.hello_world()

Alternative Solution: However, this also worked for me:

import sys
sys.path.append( '.' )
import app2.some_folder.some_file

some_file.hello_world()

Although, I do not understand why it works. I thought the dot is a reference to the current directory. However, when printing out the paths to the current folder, the current directory is already listed at the top:

for path in sys.path:
    print(path)

Answered   2023-09-20 20:30:12

  • For me I had to get up one more branch by doing sys.path.append('..') and it worked ! But still display the path of the CWD - anyone

If the purpose of loading a module from a specific path is to assist you during the development of a custom module, you can create a symbolic link in the same folder of the test script that points to the root of the custom module. This module reference will take precedence over any other modules installed of the same name for any script run in that folder.

I tested this on Linux but it should work in any modern OS that supports symbolic links.

One advantage to this approach is that you can you can point to a module that's sitting in your own local software version control branch working copy which can greatly simplify the development cycle time and reduce failure modes of managing different versions of the module.

Answered   2023-09-20 20:30:12

  • What is "SVC"? Do you mean "DVC"? Or "DVCS"? Or even "CVS", "VSS", or "SVN"? Or something else? Source version control? - anyone
  • SVC = Software Version Control. Just realized it's not a commonly used acronym but its is the umbrella over all of all version control systems. I'll edit my comment to be more clear. - anyone
  • Great read: docs.lib.purdue.edu/cgi/… - anyone

I was working on project a that I wanted users to install via pip install a with the following file list:

.
├── setup.py
├── MANIFEST.in
└── a
    ├── __init__.py
    ├── a.py
    └── b
        ├── __init__.py
        └── b.py

setup.py

from setuptools import setup

setup (
  name='a',
  version='0.0.1',
  packages=['a'],
  package_data={
    'a': ['b/*'],
  },
)

MANIFEST.in

recursive-include b *.*

a/init.py

from __future__ import absolute_import

from a.a import cats
import a.b

a/a.py

cats = 0

a/b/init.py

from __future__ import absolute_import

from a.b.b import dogs

a/b/b.py

dogs = 1

I installed the module by running the following from the directory with MANIFEST.in:

python setup.py install

Then, from a totally different location on my filesystem /moustache/armwrestle I was able to run:

import a
dir(a)

Which confirmed that a.cats indeed equalled 0 and a.b.dogs indeed equalled 1, as intended.

Answered   2023-09-20 20:30:12

This worked for me.

Python adds the folder containing the script you launch to the PYTHONPATH, so if you run

python application/app2/some_folder/some_file.py

Only the folder application/app2/some_folder is added to the path (not the base directory that you're executing the command in). Instead, run your file as a module and add a __init__.py in your some_folder directory.

python -m application.app2.some_folder.some_file

This will add the base directory to the path to executable python, and then classes will be accessible via a non-relative import.

Answered   2023-09-20 20:30:12

My solution for people who have all the necessary __init__.py in the package, but import still doesn't work.

import sys
import os
sys.path.insert(0, os.getcwd())

import application.app.folder.file as file

Answered   2023-09-20 20:30:12

  • worked for me... but this should not be like this, python sometimes is complicated to understand... - anyone

The code below imports the Python script given by its path, no matter where it is located, in a Python version-safe way:

def import_module_by_path(path):
    name = os.path.splitext(os.path.basename(path))[0]
    if sys.version_info[0] == 2:
        # Python 2
        import imp
        return imp.load_source(name, path)
    elif sys.version_info[:2] <= (3, 4):
        # Python 3, version <= 3.4
        from importlib.machinery import SourceFileLoader
        return SourceFileLoader(name, path).load_module()
    else:
        # Python 3, after 3.4
        import importlib.util
        spec = importlib.util.spec_from_file_location(name, path)
        mod = importlib.util.module_from_spec(spec)
        spec.loader.exec_module(mod)
        return mod

I found this in the codebase of psutils, at line 1042 in psutils.test.__init__.py (most recent commit as of 09.10.2020).

Usage example:

script = "/home/username/Documents/some_script.py"
some_module = import_module_by_path(script)
print(some_module.foo())

Important caveat: The module will be treated as top-level; any relative imports from parent packages in it will fail.

Answered   2023-09-20 20:30:12

  • Any idea why the two different Python3 methods? I tried both on Python 3.6, and they both worked, and returned identical results - anyone
  • Also identical results on python 3.8.9. Starting with 3.8.10 and later, the spec_from_file_location starts saving the root path of the file (if a relative path given) in the loader object, but otherwise the data returned is identical. Also tested with python 3.10 -- exact same behavior as 3.8.10. Both methods work just fine. - anyone
  • @Jon Unfortunately I can't comment on these, I'm not familiar with the nuances of importlib. This is found property, and I didn't want to change anything - figured they had a reason for it. Maybe there's some nuance that is different, or that breaks for older/newer versions. - anyone

This problem may be due to PyCharm

I had the same problem while using PyCharm. I had this project structure

skylake\
   backend\
      apps\
          example.py
      configuration\
          settings.py
   frontend\
      ...some_stuff

and code from configuration import settings in example.py raised an import error.

The problem was that when I opened PyCharm, it considered that skylake is the root path and ran this code.

sys.path.extend(['D:\\projects\\skylake', 'D:/projects/skylake'])

To fix this I just marked backend directory as the source root.

Enter image description here

And it's fixed my problem.

Answered   2023-09-20 20:30:12

  • 'skylake' is not to be confused with Skylake. - anyone
  • @PeterMortensen bro, you fixed every answer I see ಠ﹏ಠ - anyone

You can use pip's pip install -e . command. You must create a file called setup.py in the root of the project's directory which contains the following:

from setuptools import find_packages, setup

setup(
    name='src',
    packages=find_packages(),
    version='0.1.0',
    description='my_project',
    author='author',
    license='MIT',
)

Afterwards, enter pip install -e . while in your project's root directory. This will enable all directories to be called with their name as a module. For example, if your root directory contains the subdirectories module1 and module2, each with scripts inside them, you will be able to access them from any subdirectories with the following command, for module1:

import module1.script1 as script1

Answered   2023-09-20 20:30:12

You can use importlib to import modules where you want to import a module from a folder using a string like so:

import importlib

scriptName = 'Snake'

script = importlib.import_module('Scripts\\.%s' % scriptName)

This example has a main.py which is the above code then a folder called Scripts and then you can call whatever you need from this folder by changing the scriptName variable. You can then use script to reference to this module. such as if I have a function called Hello() in the Snake module you can run this function by doing so:

script.Hello()

I have tested this in Python 3.6

Answered   2023-09-20 20:30:12