This is an early design document about the separation of data layer in QuTiP as part of my Google Summer of Code project. This is a very early-stage document, which is significantly liable to change, but is indicative of the direction we were planning to go.

Originally written on the 1st of June, 2020.

In QuTiP we have some optional OpenMP components, which can be used if the C extensions are built with OpenMP support at compile time. Typically this should be achievable just by adding the -fopenmp flag at compile and link time, but unfortunately the llvm clang distribution that Apple ship with macOS is not built with OpenMP support.

Debuggers are complicated, especially ones that attach to compiled executables. In Python pdb runs inside the interpreter, so it does not need to control other processes, but this will not work once we have Cython-compiled components (or other C extensions) in our Python programmes. Unfortunately, Cython offers us plenty of opportunities to shoot ourselves in the foot, and it is more likely we will need to debug use-after-free, double-free or other nasty segfault situations.

Cython does come with a debugger, which is more properly a set of Python extensions to gdb, the GNU debugger. Macs come with lldb installed and functional, but sadly getting gdb running is a bit tricky. At the time of writing, I have macOS 10.14 Mojave, the current version of gdb is 9.2, and Cython is 0.29.14. Further, the Cython extensions require Python 2.7, but I need to be able to debug Python 3 programmes since it’s the only supported version of Python.

As of right now, I do not have a fully working debugger, but hopefully I will update this post after rebuilding gdb with some minor patches.

I will be refactoring and reorganising QuTiP’s internal data structures, a large task that was previously attempted by someone else but one that never quite got completed and lives in a disused branch on their fork. In the intervening year or so, the codebase has moved on significantly, so GitHub now sounds the death knell

This branch is 85 commits ahead, 366 commits behind qutip:master.

I want to know what changes they had made, without being inundated by unrelated changes on the master branch.

Let’s assume that the old branch of interest is called old-feature and lives in a forked repository which I have added as a remote called fork.

For this summer, I will be working full-time on the open-source project QuTiP with a stipend provided by Google as part of the Summer of Code 2020 project to the umbrella organisation numFOCUS. One requirement of the stipend is that I blog about what I am working on throughout the project, and on any interesting parts of team programming I encounter, so that future applicants to the programme have an idea of what to expect. I will be doing that here, under the “GSoC” tag.

QuTiP is a Python library for dynamics simulations of open quantum systems, something that I have made heavy use of during my PhD studies. At its core is the Qobj class, which represents all quantum objects. Currently, the underlying data storage format is always a compressed-sparse-row matrix, which allows efficient simulation of very large tensor-product spaces, but introduces significant overhead when handling smaller, few-qubit systems. The aim of the project is to abstract out the data layer so that higher-level components can function seamlessly without worrying about the representation format, but the advanced user will be able to access accelerated functions by using the right tool for the job. You can also read my full project proposal in PDF format hosted here.

Star (or wildcard) imports are one of the methods for importing libraries in Python, and although they are generally discouraged, they are rather prevalent in a lot of notebook-style scientific code. Within a package being imported, there is no official way of knowing whether this is by a star import, but since Python allows overriding just about everything, we can detect slight differences in the methods and inject arbitrary code if the wildcard is used.

In QuTiP, we currently suffer from long import times and want to move to a scipy-like style where fewer symbols are in the global package namespace, and submodules are only imported if explicitly requested. We want to issue a warning to people currently using the from qutip import * syntax, because the number of symbols available to them will soon decrease.