pydeps¶
Python module dependency visualization.
This package is primarly intended to be used from the command line through the
pydeps
command.
Contents
- pydeps
- API documentation
- Module contents
- pydeps.arguments module
- pydeps.cli module
- pydeps.colors module
- pydeps.depgraph module
- pydeps.depgraph2dot module
- pydeps.dot module
- pydeps.dummymodule module
- pydeps.configs module
- pydeps.mf27 module
- pydeps.package_names module
- pydeps.py2depgraph module
- pydeps.pycompat module
- pydeps.pydeps module
- pydeps.pystdlib module
- pydeps.render_context module
- pydeps.target module
- Indices and tables
Feature requests and bug reports:
Please report bugs and feature requests on GitHub at https://github.com/thebjorn/pydeps/issues
How to install¶
pip install pydeps
To create graphs with pydeps
you also need to install Graphviz. Please follow the
installation instructions provided in the Graphviz link (and make
sure the dot
command is on your path).
Usage¶
usage: pydeps [-h] [--debug] [--config FILE] [--no-config] [--version]
[-L LOG] [-v] [-o file] [-T FORMAT] [--display PROGRAM]
[--noshow] [--show-deps] [--show-raw-deps] [--show-dot]
[--nodot] [--no-output] [--show-cycles] [--debug-mf INT]
[--noise-level INT] [--max-bacon INT] [--pylib] [--pylib-all]
[--include-missing] [-x PATTERN [PATTERN ...]]
[-xx MODULE [MODULE ...]] [--only MODULE_PATH [MODULE_PATH ...]]
[--externals] [--reverse] [--rankdir {TB,BT,LR,RL}] [--cluster]
[--min-cluster-size INT] [--max-cluster-size INT]
[--keep-target-cluster] [--collapse-target-cluster]
[--rmprefix PREFIX [PREFIX ...]] [--dot-output FILENAME]
[--deps-output FILENAME]
fname
- positional arguments:
- fname filename
- optional arguments:
-h, --help show this help message and exit --config FILE specify config file --no-config disable processing of config files --version print pydeps version -L LOG, --log LOG set log-level to one of CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET. -v, --verbose be more verbose (-vv, -vvv for more verbosity) -o file write output to ‘file’ -T FORMAT output format (svg|png) --display PROGRAM program to use to display the graph (png or svg file depending on the T parameter) --noshow don’t call external program to display graph --show-deps show output of dependency analysis --show-raw-deps show output of dependency analysis before removing skips --deps-output write output of dependency analysis to file (instead of screen) --show-dot show output of dot conversion --dot-output write dot code to file (instead of screen) --nodot skip dot conversion --no-output don’t create .svg/.png file, implies –no-show (-t/-o will be ignored) --show-cycles show only import cycles --debug turn on all the show and verbose options (mainly for debugging pydeps itself) --noise-level INT exclude sources or sinks with degree greater than noise-level --max-bacon INT exclude nodes that are more than n hops away (default=2, 0 -> infinite) --pylib include python std lib modules --pylib-all include python all std lib modules (incl. C modules) --include-missing include modules that are not installed (or can’t be found on sys.path) --only MODULE_PATH only include modules that start with MODULE_PATH, multiple paths can be provided --externals create list of direct external dependencies --reverse draw arrows to (instead of from) imported modules --rankdir set the direction of the graph, legal values are TB (default, imported modules above importing modules), BT (opposite direction of TB), LR (left-to-right) and RL (right-to-left) --cluster draw external dependencies as separate clusters --min-cluster-size INT the minimum number of nodes a dependency must have before being clustered (default=0) --max-cluster-size INT the maximum number of nodes a dependency can have before the cluster is collapsed to a single node (default=0) --keep-target-cluster draw target module as a cluster --collapse-target-cluster collapse target module (this implies –cluster) --rmprefix PREFIX remove PREFIX from the displayed name of the nodes (multiple prefixes can be provided) -x PATTERN, --exclude PATTERN input files to skip (e.g. foo.*), multiple patterns can be provided --exclude-exact MODULE (shorthand -xx MODULE) same as –exclude, except requires the full match. -xx foo.bar will exclude foo.bar, but not foo.bar.blob
Note: if an option with a variable number of arguments (like -x
) is provided
before fname
, separe the arguments from the filename with --
otherwise fname
will be parsed as an argument of the option. Example: $ pydeps -x os sys -- pydeps
.
You can of course also import pydeps
from Python and use it as a library, look in
tests/test_relative_imports.py
for examples.
Example¶
This is the result of running pydeps
on itself (pydeps pydeps
):
(full disclosure: this is for an early version of pydeps)
Notes¶
pydeps finds imports by looking for import-opcodes in
python bytecodes (think .pyc files). Therefore, only imported files
will be found (ie. pydeps will not look at files in your directory that
are not imported). Additionally, only files that can be found using
the Python import machinery will be considered (ie. if a module is
missing or not installed, it will not be included regardless if it is
being imported). This can be modified by using the --include-missing
flag.
Displaying the graph:
To display the resulting .svg
or .png
files, pydeps
by default
calls an appropriate opener for the platform, like xdg-open foo.svg
.
This can be overridden with the --display PROGRAM
option, where PROGRAM
is an
executable that can display the image file of the graph.
You can also export the name of such a viewer in either the PYDEPS_DISPLAY
or BROWSER
environment variable, which changes the default behaviour
when --display
is not used.
Configuration files¶
All options can also be set in a .pydeps
file using .ini
file
syntax (parsable by ConfigParser
). Command line options override
options in the .pydeps
file in the current directory, which again
overrides options in the user’s home directory
(%USERPROFILE%\.pydeps
on Windows and ${HOME}/.pydeps
otherwise).
An example .pydeps file:
[pydeps]
max_bacon = 2
no_show = True
verbose = 0
pylib = False
exclude =
os
re
sys
collections
__future__
pydeps will also look for configuration data in pyproject.toml
(under
[tool.pydeps]
) and setup.cfg
(under [pydeps]
).
Bacon (Scoring)¶
pydeps
also contains an Erdős-like scoring function (a.k.a. Bacon
number, from Six degrees of Kevin Bacon
(http://en.wikipedia.org/wiki/Six_Degrees_of_Kevin_Bacon) that lets
you filter out modules that are more than a given number of ‘hops’
away from the module you’re interested in. This is useful for finding
the interface a module has to the rest of the world.
To find pydeps’ interface to the Python stdlib (less some very common modules).
shell> pydeps pydeps --show --max-bacon 2 --pylib -x os re types _* enum
--max-bacon 2
(the default) gives the modules that are at most 2
hops away, and modules that belong together have similar colors.
Compare that to the output with the --max-bacon=0
(infinite)
filter:
Import cycles¶
pydeps
can detect and display cycles with the --show-cycles
parameter. This will _only_ display the cycles, and for big libraries
it is not a particularly fast operation. Given a folder with the
following contents (this uses yaml to define a directory structure,
like in the tests):
relimp:
- __init__.py
- a.py: |
from . import b
- b.py: |
from . import a
pydeps relimp --show-cycles
displays:
Clustering externals¶
Running pydeps pydeps –max-bacon=4 on version 1.8.0 of pydeps gives the following graph:
If you are not interested in the internal structure of external modules, you can add the --cluster
flag, which
will collapse external modules into folder-shaped objects:
shell> pydeps pydeps --max-bacon=4 --cluster
To see the internal structure _and_ delineate external modules, use the --max-cluster-size
flag, which controls
how many nodes can be in a cluster before it is collapsed to a folder icon:
shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=1000
or, using a smaller max-cluster-size:
shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=3
To remove clusters with too few nodes, use the --min-cluster-size
flag:
shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=3 --min-cluster-size=2
In some situations it can be useful to draw the target module as a cluster:
shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=3 --min-cluster-size=2 --keep-target-cluster
..and since the cluster boxes include the module name, we can remove those prefixes:
shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=3 --min-cluster-size=2 --keep-target-cluster --rmprefix pydeps. stdlib_list.
Graph direction¶
The direction of the graph can be specified using the --rankdir
flag.
Top to bottom (default):
shell> pydeps pydeps --rankdir TB
Bottom to top:
shell> pydeps pydeps --rankdir BT
Left to right:
shell> pydeps pydeps --rankdir LR
Right to left:
shell> pydeps pydeps --rankdir RL
Collapsing target package¶
When internal target package dependencies are unimportant, they can be collapsed using the --collapse-target-cluster
flag. This option also implies --cluster
:
shell> pydeps pydeps --collapse-target-cluster
Intermediate format¶
An attempt has been made to keep the intermediate formats readable,
eg. the output from pydeps --show-deps ..
looks like this:
...
"pydeps.mf27": {
"imported_by": [
"__main__",
"pydeps.py2depgraph"
],
"kind": "imp.PY_SOURCE",
"name": "pydeps.mf27",
"path": "pydeps\\mf27.py"
},
"pydeps.py2depgraph": {
"imported_by": [
"__main__",
"pydeps.pydeps"
],
"imports": [
"pydeps.depgraph",
"pydeps.mf27"
],
"kind": "imp.PY_SOURCE",
"name": "pydeps.py2depgraph",
"path": "pydeps\\py2depgraph.py"
}, ...
Version history¶
Version 1.12.5 Pydeps can now read configuration data from pyproject.toml
.
Thanks to septatrix for pushing the idea and for countering my toml-rant with
an informative argument.
Version 1.11.0 drop support for Python 3.6.
Thanks to pawamoy for removing imports of the deprecated imp
module.
(Parts of it has been vendorized due to a Python bug, see the code for details.)
Version 1.10.1 Thanks to vector400 for a new option --rankdir
which
renders the graph in different directions.
Version 1.10.0 supports Python 3.10.
Version 1.9.15 Thanks to Pipeline Foundation for a very much improved CI pipeline, and a CD pipeline as well.
Version 1.9.14 Thanks to poneill for fixing a cryptic error message when
run in a directory without an __init__.py
file.
Version 1.9.13 Thanks to glumia and SimonBiggs for improving the documentation.
Version 1.9.10 no_show
is now honored when placed in .pydeps
file.
Thanks to romain-dartigues for the PR.
Version 1.9.8 Fix for maximum recursion depth exceeded
when using large
frameworks (like sympy
). Thanks to tanujkhattar for finding the fix and to
balopat for reporting it.
Version 1.9.7 Check PYDEPS_DISPLAY
and BROWSER
for a program to open
the graph, PR by jhermann
Version 1.9.1 graphs are now stable on Python 3.x as well - this was already the case for Py2.7 (thanks to pawamoy for reporting and testing the issue and to kinow for helping with testing).
Version 1.9.0 supports Python 3.8.
Version 1.8.7 includes a new flag --rmprefix
which lets you remove
prefixes from the node-labels in the graph. The _name_ of the nodes are not effected
so this does not cause merging of nodes, nor does it change coloring - but it
can lead to multiple nodes with the same label (hovering over the node will
give the full name). Thanks to aroberge for the enhancement request.
Version 1.8.5 With svg as the output format (which is the default), paths are now hilighted on mouse hover (thanks to tomasito665 for the enhancement request).
Version 1.8.2 incldes a new flag --only
that causes pydeps to
only report on the paths specified:
shell> pydeps mypackage --only mypackage.a mypackage.b
Version 1.8.0 includes 4 new flags for drawing external dependencies as clusters. See clustering-externals for examples. Additionally, the arrowheads now have the color of the source node.
Version 1.7.3 includes a new flag -xx
or --exclude-exact
which
matches the functionality of the --exclude
flag, except it requires an
exact match, i.e. -xx foo.bar
will exclude foo.bar, but not
foo.bar.blob
(thanks to AvenzaOleg for the PR).
Version 1.7.2 includes a new flag, --no-output
, which prevents
creation of the .svg/.png file.
Version 1.7.1 fixes excludes in .pydeps files (thanks to eqvis for the bug report).
Version 1.7.0 The new --reverse
flag reverses the direction
of the arrows in the dependency graph, so they point _to_ the imported
module instead of _from_ the imported module (thanks to goetzk for
the bug report and tobiasmaier for the PR!).
Version 1.5.0 Python 3 support (thanks to eight04 for the PR).
Version 1.3.4 --externals
will now include modules that
haven’t been installed (what modulefinder
calls badmodules
).
Version 1.2.8 A shortcut for finding the direct external dependencies of a package was added:
pydeps --externals mypackage
which will print a json formatted list of module names to the screen, e.g.:
(dev) go|c:\srv\lib\dk-tasklib> pydeps --externals dktasklib
[
"dkfileutils"
]
which means that the dktasklib
package only depends on the dkfileutils
package.
This functionality is also available programmatically:
import os
from pydeps.pydeps import externals
# the directory that contains setup.py (one level up from actual package):
os.chdir('package-directory')
print externals('mypackage')
Version 1.2.5: The defaults are now sensible, such that:
shell> pydeps mypackage
will likely do what you want. It is the same as
pydeps --show --max-bacon=2 mypackage
which means display the
dependency graph in your browser, but limit it to two hops (which
includes only the modules that your module imports – not continuing
down the import chain). The old default behavior is available with
pydeps --noshow --max-bacon=0 mypackage
.
Contributing¶
- Fork it
- It is appreciated (but not required) if you raise an issue first: https://github.com/thebjorn/pydeps/issues
- Create your feature branch (git checkout -b my-new-feature)
- Commit your changes (git commit -am ‘Add some feature’)
- Push to the branch (git push origin my-new-feature)
- Create new Pull Request
API documentation¶
Module contents¶
Python module dependency visualization. This package installs the pydeps
command, and normal usage will be to use it from the command line.
pydeps.arguments module¶
pydeps.cli module¶
command line interface (cli) code.
-
pydeps.cli.
base_argparser
(argv=())[source]¶ Initial parser that can set values for the rest of the parsing process.
-
pydeps.cli.
verbose
(*args, **kwargs)¶ the (will become) verbose function
pydeps.colors module¶
Color calculations.
pydeps.depgraph module¶
-
class
pydeps.depgraph.
DepGraph
(depgraf, types, target, **args)[source]¶ Bases:
object
The dependency graph.
It is the output of
pydeps.py2depgraph.py2dep()
-
dissimilarity_metric
(a, b)[source]¶ Return non-zero if references to this module are strange, and should be drawn extra-long. The value defines the length, in rank. This is also good for putting some vertical space between seperate subsystems.
Returns an int between 1 (default) and 4 (highly unrelated).
-
proximity_metric
(a, b)[source]¶ Return the weight of the dependency from a to b. Higher weights usually have shorter straighter edges. Return 1 if it has normal weight. A value of 4 is usually good for ensuring that a related pair of modules are drawn next to each other.
Returns an int between 1 (unknown, default), and 4 (very related).
-
skip_modules
= ['os', 'sys', 'qt', 'time', '__future__', 'types', 're', 'string', 'bdb', 'pdb', '__main__', 'south']¶
-
source_name
(name, path=None)[source]¶ Returns the module name, possibly limited by –max-module-depth.
-
sources
= None¶ dict[module_name] -> Source object
-
-
class
pydeps.depgraph.
Source
(name, path=None, imports=(), exclude=False, args=None)[source]¶ Bases:
object
A node (contained) in the dependency graph.
It contains info about which modules are imported by this source, and which modules import this source.
-
degree
¶
-
in_degree
¶ Number of incoming arrows.
-
label
¶ Convert a module name to a formatted node label. This is a default policy - please override.
-
module_depth
¶
-
name_parts
¶
-
out_degree
¶ Number of outgoing arrows.
-
path_parts
¶
-
pydeps.dot module¶
Graphviz interface.
-
pydeps.dot.
call_graphviz_dot
(src, fmt)[source]¶ Call dot command, and provide helpful error message if we cannot find it.
pydeps.dummymodule module¶
-
class
pydeps.dummymodule.
DummyModule
(target, **args)[source]¶ Bases:
object
We create a file that imports the module to be investigated.
-
pydeps.dummymodule.
is_module
(directory)[source]¶ A directory is a module if it contains an
__init__.py
file.
pydeps.configs module¶
-
class
pydeps.configs.
Config
(**kwargs)[source]¶ Bases:
object
-
cluster
= False¶ draw external dependencies as separate clusters
-
collapse_target_cluster
= False¶ collapse target module (–keep-target-cluster will be ignored)
-
config
= None¶ specify config file
-
debug
= False¶ turn on all the show and verbose options (mainly for debugging pydeps itself)
-
debug_mf
= 0¶ set the ModuleFinder.debug flag to this value
-
deps_out
= None¶ write output of dependency analysis to ‘file’
-
display
= None¶ program to use to display the graph (png or svg file depending on the T parameter)
-
dot_out
= None¶ write dot code to ‘file’
-
exclude
= []¶ input files to skip (e.g. foo.*), multiple file names can be provided
-
exclude_exact
= []¶ same as –exclude, except requires the full match. -xx foo.bar will exclude foo.bar, but not foo.bar.blob
-
externals
= False¶ create list of direct external dependencies
-
find_package
= False¶ tries to automatically find the name of the current package.
-
fname
= None¶ filename
-
format
= 'svg'¶ output format (svg|png)
-
include_missing
= False¶ include modules that are not installed (or can’t be found on sys.path)
-
keep_target_cluster
= False¶ draw target module as a cluster
-
log
= None¶ set log-level to one of CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET.
-
max_bacon
= 2¶ exclude nodes that are more than n hops away (default=2, 0 -> infinite)
-
max_cluster_size
= 0¶ the maximum number of nodes a dependency can have before the cluster is collapsed to a single node (default=0)
-
max_module_depth
= 0¶ coalesce deep modules to at most n levels
-
min_cluster_size
= 0¶ the minimum number of nodes a dependency must have before being clustered (default=0)
-
no_config
= False¶ disable processing of config files
-
no_dot
= False¶ skip dot conversion
-
no_output
= False¶ don’t create .svg/.png file, implies –no-show (-t/-o will be ignored)
-
no_show
= False¶ don’t call external program to display graph
-
noise_level
= 200¶ exclude sources or sinks with degree greater than noise-level
-
only
= []¶ only include modules that start with MODULE_PATH
-
output
= None¶ write output to ‘file’
-
pylib
= False¶ include python std lib modules
-
pylib_all
= False¶ include python all std lib modules (incl. C modules)
-
rankdir
= 'TB'¶ set the direction of the graph, legal values are TB (default, imported modules above importing modules), BT (opposite direction of TB), BT (opposite direction of TB), LR (left-to-right) and RL (right-to-left)
-
reverse
= False¶ draw arrows to (instead of from) imported modules
-
rmprefix
= []¶ remove PREFIX from the displayed name of the nodes
-
show_cycles
= False¶ show only import cycles
-
show_deps
= False¶ show output of dependency analysis
-
show_dot
= False¶ show output of dot conversion
-
show_raw_deps
= False¶ show output of dependency analysis before removing skips
-
start_color
= 0¶ starting value for hue from 0 (red/default) to 360.
-
verbose
= 0¶ be more verbose (-vv, -vvv for more verbosity)
-
version
= False¶ print pydeps version
-
pydeps.py2depgraph module¶
-
class
pydeps.py2depgraph.
MyModuleFinder
(syspath, *args, **kwargs)[source]¶ Bases:
pydeps.mf27.ModuleFinder
-
class
pydeps.py2depgraph.
imp
[source]¶ Bases:
enum.Enum
An enumeration.
-
C_BUILTIN
= 6¶
-
C_EXTENSION
= 3¶
-
IMP_HOOK
= 9¶
-
PKG_DIRECTORY
= 5¶
-
PY_CODERESOURCE
= 8¶
-
PY_COMPILED
= 2¶
-
PY_FROZEN
= 7¶
-
PY_RESOURCE
= 4¶
-
PY_SOURCE
= 1¶
-
pydeps.pycompat module¶
Compatibility imports between py2/py3
pydeps.pydeps module¶
cli entrypoints.
-
pydeps.pydeps.
call_pydeps
(file_or_dir, **kwargs)[source]¶ Programatic entry point for pydeps.
See
pydeps.configs.Config
class for the available options.
-
pydeps.pydeps.
depgraph_to_dotsrc
(target, dep_graph, **kw)[source]¶ Convert the dependency graph (DepGraph class) to dot source code.
pydeps.render_context module¶
-
class
pydeps.render_context.
Rankdir
[source]¶ Bases:
enum.Enum
An enumeration.
-
BOTTOM_TOP
= 'BT'¶
-
LEFT_RIGHT
= 'LR'¶
-
RIGHT_LEFT
= 'RL'¶
-
TOP_BOTTOM
= 'TB'¶
-
-
class
pydeps.render_context.
RenderBuffer
(target, reverse=False, rankdir=<Rankdir.TOP_BOTTOM: 'TB'>, cluster=False, min_cluster_size=0, max_cluster_size=1, keep_target_cluster=False, collapse_target_cluster=False, **kw)[source]¶ Bases:
object