On imports and the ddg package structure

A module is the basic unit of Python code. Most of the time, modules are made available via import module. A directory containing an __init__.py is a special kind of module called a package.

Goal

Importing the ddg package with import ddg should make all of its subpackages and submodules available:

import ddg

# This should just work no matter what.
ddg.math.random.random_matrix
ddg.datastructures.nets
...

# This should just work if executed with Blender's Python.
ddg.visualization.blender.mesh.join
ddg.visualization.blender.camera
...

Implementation

The implementation strategy is as follows: When importing in any __init__.py module which defines some package package:

Import every subpackage manually with import package.subpackage as subpackage or from . import subpackage.

They generally offer features such as

autocompletion
showing the function signature as the user enters its arguments
show documentation for some object
figure out an object’s type
go to definition
find all references of an object
automatically rename an object across the whole project
…

which all rely on knowing where these objects come from.

autocompletion
showing the function signature as the user enters its arguments
show documentation for some object
figure out an object’s type
go to definition
find all references of an object
automatically rename an object across the whole project
…

all rely on knowing where these objects come from. Python barely restricts the programmer to mess with its internals, for example, it is possible to import modules without using the import keyword. Anything can be imported anywhere anytime subject to Turing-complete computations and conditions only available at runtime (e.g. whether the code is running within Blender). In other words, the result of dir(module) can only ever be determined reliably by executing the code. At the same time, IDEs and language servers cannot actually execute the code for performance and security reasons, so they rely on heuristics to determine the origin of an object. These heuristics rely on parsing the modules and in particular the __init__.py’s of packages for the standard import machinary, i.e. import package.module, from package import stuff, from package import * and their relative import versions. We need to stick to these to enable IDEs and language servers to work. These heuristics can handle conditional imports, e.g.

if some_condition_that_may_only_be_known_at_runtime:
    import itertools

    # IDEs and language servers know that tee is a function in itertools.
    itertools.tee

but anything more fancy than that is likely to defeat these heuristics.

Pyright/VS Code

Pyright is the language server that powers VS Code’s Python extension. By design, it requires import package.module as module rather than import package to recognise module as a member of package. Their reasoning is documented here.