Category Archives: Coding and Development

Visual Studio 2012, XP support, and codebases with too many dependencies

The potential of C++11 support first enticed me to consider moving to Visual Studio 2012 for Windows development. (Our existing environment was Visual Studio 2008.) Once I got a moment to actually try it out, the much-improved IntelliSense (that actually debuted in 2010, powered by a modern, capable compiler front-end, EDG) really wooed me. It’s remarkable what not getting flustered by a smart pointer can do for code completion.

Here’s the rub: We have a large immersive system that runs Windows XP x64, and for a research lab that develops software in the course of its work (rather than a software-driven shop), we’ve got a bunch of dependencies. VS2012 does support Windows XP, kind of, but you have to jump through some non-default hoops to get it to work.  I’m happy to announce, however, that I was able to get the whole stack we use for VR JuggLua built with VS2012 using the “v110_xp” XP-compatible toolset. (The linked page has the details on how to check/set the Platform Toolset of VC projects for XP compatibility.)

By far, the easiest way to build is if you have a CMake-based build system available. Recent versions of CMake have added a “Generator Toolset” concept, which can be used to specify the “v110_xp” toolset needed for this instance. There’s no GUI support for this at the time of this writing (CMake 2.8.11.2), but it can be done in the GUI: before initially configuring a build, add a string variable to the cache called “CMAKE_GENERATOR_TOOLSET” with the value “v110_xp”. You might see a spurious warning about this not being used at generation time, but rest assured, it is. (Check in Visual Studio if you’re not confident.)

VR Juggler and its dependencies have been built and posted (as branches) in my vrjuggler-windows-binaries repository. The linked branch includes VR Juggler 3.0.x (built from my CMake-enabled git branch, using CMake) as well as the client-code compile-time dependencies. In separate branches, I’ve preserved all the dependencies separately:

I’ve also built OpenSceneGraph, with a usable subset of its plugins. Conveniently, I realized that third-party libraries with a C-only interface can be built once with MinGW and used across compiler versions (as Zlib does), which helps a lot. (Particularly with the CMake option to spit out an MS-compatible .lib import library file.) I’ve uploaded the OpenSceneGraph 2.8.x v110_xp binaries here.  (Binaries in git repos let me use them as submodules…)

For VR JuggLua to build in all its glory, it needs Qt 4.x, or at least FLTK. FLTK was an easy build, and I got the chance to hammer out a few bugs in that neglected GUI frontend. I did end up being able to get Qt 4.x built, after a bit of pain and uncertainty as to whether it would actually work on XP. I’ll post binaries, or at least instructions, soon.

Did it work? Yep! Got a fully functional VR JuggLua build that successfully runs immersively in the aforementioned WinXPx64 system. It’s not the default for continuous builds (yet), but it’s a lovely step forward.

Making a CMake-based build install software in de-facto standard locations

This post falls partially into the “Note to self” category. I typically set up CMake variables for “bin”, “lib”, “share”, and so on to use when installing software (or building it if I want it to run in the build directory and it’s stubborn about layout). Turns out that there’s a module called GNUInstallDirs that was added to CMake in 2.8.5, and improved since then, that sets these up in a standard-ish way. Furthermore, when distros package CMake, if they don’t follow the defaults, they typically patch this file so it applies, so this really takes care of the whole “what to call the library directory” mess quite nicely.  The canonical source documenting the meaning of these variables comes from the GNU project, but I’ve separated the most common ones below (listed ready to copy-and-paste into your build):

  • ${CMAKE_INSTALL_BINDIR} aka “bin”
  • ${CMAKE_INSTALL_LIBDIR} aka “lib” or “lib64” or something more complicated for Debian multiarch
  • ${CMAKE_INSTALL_INCLUDEDIR} aka “include”
  • ${CMAKE_INSTALL_DATADIR} aka “share” – not sure what the difference between this and DATAROOTDIR is, but it appears nobody else does either, so I’m just gonna roll with it.

As a side effect, you’ll get a unixy layout on other operating systems too, but as I already was doing this, it’s cool with me.

Using C++ on the Arduino: a mainstream C++ Standard Library port for the AVR!

The Arduino development environment (and more generally, the avr-gcc/avr-libc/avr-binutils toolchain) supports, and in the Arduino core library, uses C++ language features.  However, a C++ standard library is not included, which makes using other C++ libraries (even just the header-only libraries in Boost) a challenge, and can result in less effectively-C++ code.  Among my goals in seeking C++ standard libraries for the Arduino include the ability to use the excellent Eigen matrix-math library and some of the header-only Boost libraries.

(tl;dr version: visit https://github.com/vancegroup/stlport-avr/wiki !)

A number of individuals have worked on ports of (partial – at least the Arduino IDE disables RTTI and exceptions, and fstream is little use without a filesystem) C++ standard library implementations. (Keep in mind that the C++ standard library is more than just the STL – the string library is sometimes considered part of the STL but is technically its own thing, as well as the streams library.)  The following are the previous attempts I’ve found:

  • Andy Brown ported the original SGI STL (not a full C++ standard library) but with some additions to bring it closer to a full standard library. He considered libstdc++ (too embedded with the desktop GCC toolchain) as well as uSTL (which he eliminated due to larger generated code size) before porting the SGI STL. He made a number of customizations to it that, while making it less standards-conforming, made it more suitable to use on AVR devices. He supplemented it with a streams implementation based on  uClibc++, as well as some custom features for the AVR and Arduino environment. (For instance, iostream implementations for HardwareSerial and character LCDs.) The library is header-only, so you only pay in compile time for what you use. However, the SGI STL precedes the standardization of the C++ standard library with STL (it’s an [the?] initial implementation…), so compiler and library compatibility are sometimes a bit sketchy.
    • The SGI STL license is a fairly liberal, non-copyleft license that Debian’s licensecheck.pl won’t identify for me :-). The portions from uClibc++ are LGPL 2.1+, and the headers written by Andy specifically for the arduino (lcdostream and serstream) are under a three-clause BSD license.
  • “maniacbug” started the Arduino library StandardCplusplus, a fairly direct port of uClibc++ . uClibc++ is a more modern implementation than the SGI STL (that is, it’s a post-standardization implementation) and aims to be a full C++ standard library.  I found that to use for my purposes, I had to make a number of modifications (particularly in implementing the limits header), which are on my GitHub fork of StandardCplusplus. It’s concise and easy to read, but its relative incompleteness in the areas I needed made it some effort to maintain.
    • License-wise, uClibc++ is LGPL 2.1+ which keeps it easy to move code in, but harder to move code out, and a bit confusing for embedded. (avr-libc uses, and seems to recommend, an MIT-style non-copyleft liberal license – presumably to avoid potential complexities over requirements and definitions of linking, derivative works, etc. given the nature of embedded code as being flashed into hardware.) The stream classes by Andy Brown were also added to this library, under the same three-clause BSD license mentioned earlier.

While I had gotten StandardCplusplus to a state where I could mostly use it, I found myself hitting a number of issues bit-by-bit. Furthermore, a number of the fixes/needed implementations were because of gaps in upstream, rather than porting issues, and the break between the upstream uClibc++ and the Arduino-layout, Arduino-specialized version made it harder to justify fixing issues that would be bothersome to port upstream and/or marginally likely to be accepted (AVR-specific code somewhat scattered, and behind ifdef __AVR__ or ifdef ARDUINO guards).

With this background, I’d like to introduce my new development, and what I’m using now: the addition of an AVR port to the STLport project, which I’ve posted on GitHub.  STLport is liberally-licensed, and evolutionarily, it descends from the SGI STL that served as Andy Brown’s starting point. However, STLport, despite its name, can be configured as a full C++ standard library. It has been extended, expanded to include newer C++ standards and TR1, and (as suggested by the name) ported to work with many compilers. This porting was done smartly, by limiting platform-specific macro checks to configuration headers, which define feature-focused configuration macros as needed for use in the rest of the library. This meant that my changes could be fairly limited in files touched: primarily adding a config header for AVR, modifying the GCC config header to avoid assumptions that are untrue on AVR, and creating/applying a few more feature-focused config defines to reduce the library’s size and porting overhead. While the last tagged release was in 2008, the project has been moved to a Git repository and has been under some development since then. For instance, changes to the STLport-5.2 (stable) branch are as recent as April 2012, and the master branch in fact is (working toward being) a C++11 standard library.

I’ve made ports for both the 5.2 stable branch and the master branch. The library can be built using the STLport build system if you prefer, though if you’re using Arduino, all you’ll need is one of the “arduino-installed” branches: these are generated by a script rearranging the sources/headers from the mainline branches. This way, they won’t fall (much) behind and can easily be reproduced. (I could have claimed success without these branches, however, I find it convenient to keep Arduino libraries in Git submodules, and the branch solution provides a straightforward, nontechnical way to use the library directly with the Arduino IDE.) Both the C++11 and stable 5.2 versions work with the Arduino core library, though note that you’ll need to put your compiler into C++11 mode for the master branch. As far as I can tell, this can’t be done with the Arduino IDE.

I’ve verified that both versions compile as a static library on their own. I’ve also verified that C++ Standard Library-using Arduino sketches build using (my branch of) Arduino-Makefile (with appropriate EXTRA_CXXFLAGS as needed) on both the master and 5.2 branches, and that sketches build successfully using the Arduino IDE and the 5.2 branch. My test system for this has been an Ubuntu Lucid x86_64 machine, with the following package versions updated from Debian Testing:

  • arduino and arduino-core  – 1:1.0.1+dfsg-6
  • gcc-avr – 1:4.7.0-2
  • binutils-avr – 2.20.1-3
  • avr-libc – 1:1.8.0-2

On Windows, I’ve been using the Arduino 1.0.1 IDE upgraded with Windows binaries of gcc-avr 4.7.0 and avr-libc 1.8.0 from Andy Brown (caveat: save your original avr-size.exe and put it back in place after). As this updated  is substantially the same as what I use on Linux, I’m pleased that as expected, building on Windows with the 5.2 branch in the IDE works just fine.  Remarkably, though the toolchain bundled with 1.0.1 on Windows is very very old (gcc 4.3.3) and contains some annoying-to-really-bad bugs for AVR, it at least built my (complicated template metaprogramming and STL algorithm-based) test app, only being slightly more picky about my nested template syntax.

So, give it a try! Let me know if it works for you – and if you have improvements, or find things that don’t work, please feel free to send an issue or pull request on my STLport for AVR/Arduino GitHub repository.

 

Bluetooth module and breakout for use with Arduino

SVG original version of the Annotated Diagram

PDF version of the Annotated Diagram

Links:

Using dfu-programmer with an Arduino Uno R3

Sometimes, whether intentionally (you want to “be” a different USB device) or unintentionally (your board is bricked), you want to replace the firmware on the little atmega usb chip used on the Arduino Uno.  Unfortunately, most of the instructions out there are for the Uno R2 or earlier: the R3 has a slightly different chip.  Adding to the trouble, the Linux/Mac tool for doing this operation (a “DFU upload”) has different USB product IDs than expected.  So, here’s the differences from the earlier Uno (with ATMEGA8u2) to the latest R3 (with ATMEGA16u2):

  • The dfu-programmer command line should contain at90usb162 instead of at90usb82.
  • If the version of the dfu-programmer you’re using (which includes the latest SVN as of this posting) doesn’t recognize it, you’ll want to get the latest source code and apply this patch, which adds a different device name (at90usb162unor3 instead of at90usb162) that uses 0x2FEF as the product ID.  (Thanks to http://forums.adafruit.com/viewtopic.php?f=25&t=25146 for the tip which led me to create the fix.)  I opted to add a new device type, rather than change the existing one, in case someone actually takes the patch upstream and if the original product ID is correct for some devices, just not our Arduinos.
  • I used the Arduino-COMBINED-dfu-usbserial-atmega16u2-Uno-Rev3.hex firmware file and had to pass the –suppress-bootloader-mem flag to the dfu-programmer flash command for it to work.

How to build software using CMake

I looked for easy instructions for building a package that uses CMake, so that I had a link to refer someone to. I couldn’t find one offhand – most seemed to be about creating a CMake-based build for your project, rather than using one that already exists. While building your own project with CMake is the next logical step if you do development, you’ll probably end up needing to build someone else’s first, so here’s a quick screencast (fairly Windows-centric) on how to get that done, including how to find dependencies.

My “Compiling Boost on Windows” Cheatsheet

Boost is a great set of libraries that is pretty generally available – on Linux, that is. On Windows, though there are installers available, I tend to have better luck building it myself, though I always need to go digging for the instructions I wrote “last time” I built it.  So, here for my reference and yours, is how I build Boost, for Visual C++ version X: (pretend it’s all on one line)

bjam --build-dir=vcX toolset=msvc-X.0 architecture=x86 link=shared
threading=multi runtime-link=shared variant=debug,release
--without-mpi --prefix=boost-vcX-install stage

For a 64-bit build, I add “x64” to all directory names, and this parameter:

address-model=64

A bit of quick Git scripting

Git is a slick revision control tool – if you haven’t used it yet, check out (IMHO) the best resource/tutorial on it: the Pro Git book.

Not only is it slick for people to use, it’s also really easy to write shell scripts to do repetitive work for you. The two examples I’m showing today come from my work on VR Juggler.

Moving branches between equivalent but not matching git-svn clones

I had been working with the VR Juggler source code using an automated git-svn mirror for some time (run using Jenkins and the scripts in my jenkins-svn2git repository), which worked pretty well. At least, a lot better than when I was just using SVN directly, since I don’t have commit access on the project. Eventually, upstream decided to switch to git (yay!) which they did a nice job of doing (authors all right, branches, tags, etc.), but now I had a problem – since they didn’t use git-svn exactly the same way I did, they had no commits in common with my mirror. (In addition to the authors thing, in this case, they had a https google code svn address, and mine was the http anonymous address.) I wanted to be able to relatively easily move my “work in progress” branches from my old mirror to the new official one (so that they branched from the same original SVN commit), so I wrote a little script to do it. Step 0 is to add the new official upstream as a remote to a clone of your own repo and fetch. You’ll see a no commit commits warning which you can smile and ignore. I made a fresh clone so that if something went wrong, nothing was lost.

I’ve posted my git-svn port-branch script as a Gist for your enjoyment and use. It’s easily the most involved git-related script I’ve ever written, but it follows essentially just these steps:

  1. Accept up to two inputs:
  1. <newupstreambranch>: the name of the new upstream (remote tracking) branch where we’ll be able to search the log to find a good commit to move to
  2. <yourbranch>: the name of the branch you’d like to move. (If the latter isn’t specified, the currently-checked-out branch is used.)
  • If the branch <yourbranch> doesn’t exist as a local branch, create it from origin/<yourbranch>
  • Use git log to find the most recent parent of your branch that includes a “git-svn-id:” line, and save that commit hash
  • Extract a portion of the git-svn-id: line (everything after the domain name), and search the history of <newupstreambranch> for a commit that includes that text. If everything goes right, this should be the “new” git equivalent of the same svn commit as the git commit found in step 3.
  • Do a git rebase –onto to transplant the non-svn commits over to the new parent.

Forward/backporting feature branches automatically

I’m used to the process of working on a fix, getting a nice topic branch with just those changes, then using a bug tracker/pull request/git format-patch to submit them. On some projects, such as VR Juggler, patches are most appreciated if provided in matching form for both the development version (master, formerly trunk) and the stable maintenance branch (in this case, 3.0). I had done a fairly mechanical process of a git rebase –onto to transplant my topic branches back and forth for some time. Since I had accumulated a number of unsubmitted patches, however, I realized I should just automate the process.

Thus I present another shell script as a Gist: port-patch.sh automatically forward-ports or backports your patches in topic branches. This one is more likely to need a little manual tweaking for use outside of VR Juggler, since I wanted it to figure out whether I was porting from 3.0 to master or vice-versa, but it’s a definite timesaver. Its basic process is the following:

  1. Figure out whether the branch we’re porting is for stable or development based on the name. (probably could have used merge-base or something, but this gets the job done)
  2. Compute the name of the root branch we’re porting from, the name of the root branch we’re porting to, and the name of the topic branch we’ll create.
  3. Create our new branch at our old branch.
  4. Do a git rebase –onto of our new branch to change its parent.
  5. And, for my own ease of bug filing, if the user name is rpavlik, echo the github compare URLs that I’ll paste into the report.  🙂

VR Juggler 3.0 packages updated for Ubuntu Natty

Just a quick heads-up: I’ve updated the VR Juggler Ubuntu/Debian packages I maintain on Gitorious and that are built in this PPA repository. There is now a Ubuntu 11.04 (“Natty”) build of VR Juggler 3.0 and its dependencies alongside the other distributions. The instructions I previously posted on installing VR Juggler on Ubuntu from packages is still accurate and now applies to those of you using the latest Ubuntu release.

There were only minor changes from the Maverick packages. Most notably, the Audiere plugin for Sonix is gone because Audiere is no longer being packaged or distributed by Ubuntu or Debian. (If someone would like to pick up where they left off I’d be happy to collaborate!)  Also, I temporarily disabled the VRPN Gadgeteer driver until I get a chance to make an updated, nice package of a recent version of VRPN. I’m almost done with one – just have to test it. When that’s done, I’ll push a new build with this driver re-enabled.

If they work, or don’t work, for you, let me know! You’re welcome to help maintain these packages – many hands make light work. Their source is over on that gitorious link above, and there’s docs in the gitorious wiki, though they’re sparse – happy to help clarify if you’re interested.  I’d also be interested in trying some alternate build service together with/instead of the Launchpad PPA’s – the openSUSE Build Service looks promising but I haven’t had the time to really look into it. So, there’s a side project for you 🙂

VR Juggler tools and binaries on GitHub

Now that the semester’s over, I’ve been able to catch up on a bit of a backlog and put up a few VR Juggler-related items on GitHub. For all these items, click the link to visit the GitHub repo, which will also show you a nicely-rendered README with more information.

  • Jconf Grapher – This is a Python script for parsing jconf files (including their inclusions) and outputting Doxygen code to draw a graph of devices, proxies, and proxy aliases. I’ve included a silly shell script wrapping this to directly output PDF files, and have also made a minimal GUI for exploring these graphs, using XDot and PyGTK. Cross-platform: runs anywhere Python, PyGTK, and Doxygen do. (Tested on Linux and Windows.) Free and open source (Boost Software License 1.0, a very permissive license), feel free to fork and improve!
  • Jconf Display Merger – As I’ve discovered that VR Juggler on Windows works better with fewer windows in favor of more viewports per window, I put together this Python script to automate the process of merging display windows in jconf files (appropriately re-adjusting origins and sizes, etc).  There’s a command-line version that works everywhere and merges all windows into a single one, and for more control, you can use the GUI that lets you select which windows to merge. GUI is built on PySide (the LGPL replacement for PyQt). Cross-platform (command-line requiring Python 2.4, GUI requiring Python 2.5, Qt 4.6+, and PySide), though I have only tested on Linux. Free and open source (Boost Software License 1.0, a very permissive license), feel free to fork and improve!
  • METaL JConf files – Our group at Iowa State University recently got a new virtual reality system, a three-wall CAVE called METaL. We’re using VR Juggler to build applications for it, and in the interest of both easily maintaining configuration files and providing an up-to-date example, I’m hosting the VR Juggler 3.0 jconf files for the system on GitHub. They’re still a bit of a work in progress, but definitely usable – if you have a similar system it might be easiest to fork them on GitHub for easy integration of improvements.
  • VR Juggler 3.0 Windows binaries – Building VR Juggler can be a bit tricky, and building software on Windows can be a bit of a hassle, so you can imagine that combining them isn’t necessarily something you want to be doing every day. The most recent “official” Windows binaries are for the ancient 2.2.1.  I’ve started building from SVN, on the 3.0 branch, and pushing the installed tree to GitHub, so I don’t have to build VR Juggler separately on every machine I use: I can just clone the repo, and pull when I update the build.  Right now I only have the 32-bit builds for VC9, but I anticipate eventually adding other variants. (They’ll be in different branches named accordingly.)

As always with any of my grad school-related work, if you use or build on anything I make available, I’d really appreciate hearing from you. Information on how our work is used is helpful when providing reports and seeking renewal of the funding that makes this possible.