Do not hang around in Pythran code base without your developer guide! It is the compass that will guide you in the code jungle!
This document is a never ending work-in-progress draft. Please contribute!
Pythran can be configured with a rc file. An example is found in pythran/pythran.cfg. Look at it! To customize it:
$> cp pythran/pythran.cfg ~/.pythranrc
In particular, you may want to add -g -O0 to the cxxflags.
All Python code must be conform to the PEP 8, and the flake8 command must not
yield any message when run on our database. Additionally, avoid backslashes,
and try to make your code as concise as possible.
$> flake8 pythran/.py pythran//.py –exclude=”pythran/tests/test.py,__init__.py”
C++ code use spaces (no tabs) and a tab width of 4.
Listing the top level directory yields the following entries:
The files that describes what gets installed, that holds PyPI entries
and such.
If you’re reading this document, you know what it’s all about! MANUAL
is the user documentation and DEVGUIDE is the developer documentation.
Use make from this directory to produce the static website.
Boring but important stuff.
Describe additional stuff to package there.
Quick introduction and description of _pythran_.
The source of all things.
The source of all issues.
Where C++ back-end lies.
pythran uses the unittest module and the pytest package to manage test cases.
All requirements are listed in pythran/tests/requirements.txt.
The whole validation suite is run through the command:
$> python -m pytest pythran/tests
To run it faster we use the pytest extension xdist, the test suite will run using all
available cores. Otherwise it might run very slowly, something like four
hours on a decent laptop :’(.
Note that it is possible to use the pytest module to pass a subset of the
test suite:
$> pytest -n 8 pythran/tests/test_list.py
runs all the tests found in pythran/tests/test_list.py.
There are two kinds of tests in pythran:
unit tests that test a specific feature of the implementation. Such tests
are listed as method of a class deriving from test_env.TestEnv and must
call the run_test(function_to_translate, *effective_parameters,
**name_to_signature) method [1]. It translates function_to_translate
into a native function using the type annotations given in the
name_to_signature dictionary, runs both the python and the native
version with effective_parameters as arguments and asserts the results
are the same.
test cases that are just plain python modules to be converted in native
module by pythran. It is used to test complex situations, codes or
benchmarks found on the web etc. They are just translated, not run. These
test cases lie in pythran/tests/cases/ and are listed in
pythran/tests/test_cases.py.
The C++ code generated by pythran relies on a specific back-end,
pythonic. It is a set of headers that mimics Python’s intrinsics and
collections behavior in C++. It lies in pythran/pythonic/. There is one
directory per module, e.g. pythran/pythonic/numpy for the numpy module,
and one file per function, e.g. pythran/pythonic/numpy/ones.hpp for the
numpy.ones function. Type definitions are stored in the seperate
pythran/pythonic/types directory, one header per type. Each function header
must be #includ-able independently, i.e. it itself includes all the type
and function definition it needs. This helps keeping compilation time low.
All Pythran functions and types live in the pythonic namespace. Each extra
module defines a new namespace, like pythonic::math or
pythonic::random, and each type is defined in the pythonic::types
namespace. The DECLARE_FUNCTOR and DEFINE_FUNCTOR macros from
pythonic/utils/functor.hpp is commonly used to convert functions into
functors and put them into the mandatory functor namespace.
The pythonic runtime can be used without Python support, so it is important to
protect all Python-specific stuff inside ENABLE_PYTHON_MODULE guard.
All methods are represented by functions in Pythran. The associated
pseudo-modules are prefixed and suffixed by a double underscore __, as in
pythran/pythonic/__list__.
Stand-alone algorithms are put into pythran/tests/cases. They must be valid
Pythran input (including spec annotations). To be taken into account by the
validation suite, they must be listed in pythran/tests/test_cases.py. To be
taken into account by the benchmarking suite, they must have a line starting
with the #runas directive. Check pythran/tests/matmul.py for a complete
example.
To run the benchmark suite, one can rely on:
$> python setup.py bench --mode=<mode>
where <mode> is one among:
Uses the interpreter used to run setup.py.
Uses the Pythran compiler.
Uses the Pythran compiler in OpenMP mode.
All measurements are made using the timeit module. The number of iterations
is customizable through the --nb-iter switch.
pythran/pythonic++/<mymodule>.pythran/pythonic++/math/*.hpp and
pythran/pythonic++/__list__/*.hpp are good example to referer to.
pythran/tables.py. Each function, methodor variable must be listed there with the appropriate description.
pythran/tests/ under the nametest_my_module.py. One test case per function, method or variable
is great.
Subclass one of ModuleAnalysis, FunctionAnalysis or NodeAnalysis.
List analysis required by yours in the parent constructor, they will be built automatically and stored in the attribute with the corresponding uncameled name.
Write your analysis as a regular ast.NodeVisitor. The analysis result must be stored in self.result.
Use it either from another pass’s constructor, or through the passmanager.gather function.
Use the github interface and the pull/push requests features
channel #pythran on irc.oftc.net
(via your browser: https://webchat.oftc.net)