SWIG author on the complexity of SWIG
code.activestate.comI wrote a library that makes C++ callable from script languages, but instead of parsing headers it uses C++11 template metaprogramming and type inference, so it always parses exactly what the types are because it runs in the same compiler as your code. Unlike SWIG it is becoming simpler over time as I develop more powerful abstractions and I do understand 100% of how it works (although admittedly you have to have advanced C++ knowledge).
Definitely agree with the C++ corner cases thing being a challenge - at times I felt like I was writing another compiler on top of the C++ type system to handle the corner cases. But again, I used powerful abstractions to handle that. For instance, my library uses boost graph library and djikstra's algorithm to handle C++'s tangled web of automatic type conversions.
I must say, as a developer of C++ bindings for Guile, a former Io dabbler, and TiddlyWiki user, I am impressed with this project.
Hats off to you sir. Oh, I also do the occasional motorcycle maintenance as well.
I've been thinking about how to do this effectively and cleanly for a while now, and I wonder if the right path isn't to use clang to parse the C or C++ code and then generate interfaces from the LLVM IR. As far as I know, all the metadata you need is there, so it might prove simpler than parsing things on your own.
We've done this. A modified clang-interpreter spits out some json. Python scripts consume the it and generate reflection data, script bindings etc.
We're very happy with the results. It's a little bit slower than hand parsing the headers, but the robustness more than compensates for it. We set up a unity build (just for the headers) to make up the speed difference.
Also have a look at doxygen. It can emit xml describing the code, rather than documentation.
If it shows anything, it is a horrible language design of C++. I wrote a C++ front-end long ago, and it is excruciating.
Later, when I had to write native code to speed up Python, I always used plain old C and swig worked flawlessly.
SWIG worked wonders for me too when we had to iteratively lift a dependency on a proprietary ISAM C library in a critical application. The worse we had to do was to handle a bunch of corner cases due to some non-null terminated string handling and be wary of a silly global var which could have bitten us otherwise when multithreading. The extend functionality was awesome to make structs more object-oriented by adding a few key methods.
The key was to handle the resulting Python module as a low level C wrapper effectively allowing us to write "managed C" in Python, which we encapsulated neatly in a library with a much more pythonic API. We then naturally went all the way up and implemented SQL-like API (no parser, just methods) and a Django model subset. A SQL subset parser and DBAPI compliance were planned features. All of that allowed us to develop new features in Python and on Django while maintaining compatibility with the old components (also thanks to Django multi-db support) and making the planned switch to a complete SQL database a drop-in replacement.
Same experience in java. I had to write a java interface to some C++ code. I ended up writing a C wrapper, then interfacing SWIG to that. It made a complex problem very simple.
I was one of the maintainers of SWIG's Ruby module during that rewrite he mentions, and I remember how difficult it became for me to really understand how the code worked. I would just hack features into the Ruby module and be glad when they seemed to work, but it always felt like a house of cards. Fortunately some more talented developers took my place when I gave it up.
The SWIG author in question, Dave Beazley, also gave a couple great talks on the Python GIL: http://www.dabeaz.com/GIL/
I had a bit of a nightmare with SWIG recently, glad to hear I'm not alone! http://gaiustech.wordpress.com/2011/08/03/ocaml-bindings-for...