I attended a two-day Mozilla Science Lab sprint in July and worked on a lesson package manager. This was the occasion to meet and interact with some of my fellow software carpentry instructors. This post details what was this sprint, my impressions on it and the technical details about the lesson manager project I worked on.

Sprint: intro and motivation

A few weeks ago, I attended the first sprint organized by Mozilla Science Lab. The principle of the sprint is to have people all around the globe working together on a project (in this case many sub-projects). The sprint lasted two days on every site: the 22nd and 23rd of July (local days) during local working hours.

I'm personally involved in the Software Carpentry initiative which is part of Mozilla Science Lab and aims at teaching fundamental computer skills to scientists from other domains. I participated in the sprint to meet more people involved in Mozilla and Software Carpentry. Indeed, most of the instructor training and interactions happen online, and thus, most instructors know a only a few others. Like (Aleksandra in her debrief), I fully agree in how Greg Wilson described the sprint: “the real purpose of sprints isn't the code, it's getting to know people”.

Preparing the sprint, I hesitated as I found interest in many possible projects. I finally decided to take the lead on animating the "lesson package manager" project, to which Raniere and I participated during the sprint. We got a lot of asynchronous input and feedback from Trevor and some interactions with Greg which was already busy with the organization of the sprint.

Debriefing of the sprint as an event

I came to this sprint with two main goals in mind:

learn to know better some people from software carpentry and from Mozilla,
dedicate some time to shaping and advancing the “lesson manager” project.

This part details how the sprint format actually achieved these goals pretty well.

People and location

People participated to the sprint from various places around the global, most in shared offices (meeting rooms in places like Mozilla, etc.), some from their home or office. I participated from the Mozilla office located in Paris.

All sites were connected via video and most had a video wall, showing all remote sites. Even though, most of the time we were not discussing across sites, the video wall was giving the warm feeling of a shared spaced. Our local group (including the Mozilla hosts and sprint attendees) was quite small, ranging from 3 to 6 persons.

Overall, even though our group could have been bigger and we could have had more interactions, these two days were a great opportunity to discuss and discover people. Even when it was only remotely through the video wall, putting (animated) faces on people's names was very useful. Raniere is very active in software carpentry and it was very interesting to work with him in Paris.

Impact of "sprinting" on an early project

I have less and less time to invest in open projects these days. My current way of contributing is thus to do it whenever I find time, usually in some kind of rush.

In the context of the sprint, it forced back some structure: we had more interactions between participants and the fixed timeline was helpful. I think the general setup is helpful for most projects:

It enforces more discussions prior to the sprint, to set up the project and discuss with interesting people (in my case, with Raniere and Trevor),
it encourages to put a hard plan of action for the two days ("avoid starting from a blank sheet")
it makes people more focus during the two day sprint (in our case, we avoided exploring too many alternatives and branching too much during the sprint),
it encourages to give a post-sprint summary very early (most people did it in the days following the sprint, I had some holidays, so it took me longer).

Overall, the fixed time frame seems very beneficial to structure the contributions to the project. It might not be so important for people that can invest a lot of time, weekly, all over the year, but it helps a lot for people that help only spuriously.

Aside: interaction with other projects

As we had reached some conclusion close to the end of the sprint, I took the opportunity to look around at other projects. This have been quite interesting and my distant experience with semantic web technologies have even been somewhat helpful in writing a SPARQL query to list all scientific journals from an semantic web database.

During the sprint, Raniere and I also acted as remote testers for the “Peer Instruction” project lead by Piotr. We could test the latency between Paris and Krakow, and test the current version under various devices (different androids, different linuxes, and FirefoxOS).

Technical aspects on the lesson manager

How we started

We started before the sprint by posting our respective opinions on how the lesson package should work. We also made a repository on github and got some feedback from Trevor via the issue tracking system. This was the starting point of the project, at the beginning of the sprint.

What we did (and learned)

The exploration we made is visible at https://github.com/SoftwareCarpentryLessonManager/lesson-manager and https://github.com/SoftwareCarpentryLessonManager. Here is a quick summary of what we did, discovered and learnt during these two days:

We first decided to not explore making python (eggs) packages from lessons: it was only Raniere and I, we needed to focus, and a discussion with Tarek, present with us, led us to choosing bower, that seemed more adapted for us. The reason for choosing bower is that it remains very close to the git repositories and is "local" by default (while with python tools like pip, one need to use virtualenv to install packages locally).
We made a quick split of the first git and bash lessons in multiple repositories to match the architecture expected by bower (https://github.com/SoftwareCarpentryLessonManager/software-carpentry-template).
We made a "lesson collection" that includes the bash and git lessons: at this point, we had something easily installable (https://github.com/SoftwareCarpentryLessonManager/shell-based-bc).
We asked bower to put the dependencies in the current directory (with "bower --config.directory=.")
We also made a script and a howto that consolidates/documents the commands to run (as we need to pass arguments) (https://github.com/SoftwareCarpentryLessonManager/lesson-manager/blob/master/protolearningunit.sh).
We separated the template files (jekyll templates, css files, etc) in a separate package/repository (https://github.com/SoftwareCarpentryLessonManager/software-carpentry-template) and properly had jekyll use it.

At this point, we had something pretty nice for plain local installation of lessons, with dependencies handled. One only had to clone the "lesson collection" repository, run bower and then run jekyll (all with proper parameters, simplified via our script).

We wanted then to focus on having a smooth transition from a plain lesson installation to lesson editing (and contribution). This part got more tricky as no system is actually tailored for this use case. We explored the solutions around bower:

We had a look at "bower link" that allows for such use case: the user can clone a repository and use it instead of one of the dependency.
We realized that "bower link" is quite limited as bower will delete the cloned folder when one runs "bower update" (especially with transitive dependencies).
We concluded that "bower link" is thus too dangerous as it might cause loss of data (and edits) and thus.
Raniere had a look at "upt" (universal package manager, initially a fork from bower, https://github.com/hyperweb2/upt) which might be a good solution in the future (some features still missing).

Status and open questions etc

Looking back at what we did, we might ask ourselves a few questions and make a few remarks:

installing lessons (without modification) can be considered as easily "done" if we are ready to split the repository,
should we totally disconnect the "edit lesson" part from the "install lesson" as suggested by Greg?
should we focus on stearing the evolution of "upt" toward our use case (it is already going in the right global direction)?

The future(s) of the project

I can see three future directions for a lesson manager project:

an extension of what we got, but with a better handling of lesson editing
a big concern for me is that it forces a one-repo-per-lesson constraint
- currently (before the sprint), we have a single repository with all lessons, which starts to be a little difficult to handle (too many things together, difficult to follow issues etc)
- with a bower-based solution, we'll end up with many tiny repos, which might be a huge issue for contributors (and the day of the switch, it would be a big issue)
- the ideal solution for me would be something in the middle, where we have multiple repositories (per topic) and each repository can contain multiple lessons
we could contribute to upt and drive it towards what we need
we might also need custom scripts to help in handling all repositories
a system were we separate totally the installation of lessons from their edition
this allows to be free on the layout we use inside the repository
this makes it difficult to edit lessons after their installation
this might it more difficult to start creating packaged lessons
I would also really like a "lesson graph" project, which is somewhat different and might also support online self-paced learning
this focuses on the conceptual dependencies between lessons such as in http://www.metacademy.org/ (example http://www.metacademy.org/graphs/concepts/independent_random_variables#focus=m59ed98e&mode=explore)
this can help at many levels
- helping (new) instructors to understand the structure of what we teach
- deciding easily what to teach (and what not to) in a bootcamp
- stating pre-bootcamp requirements so that students know what we expect them to know (and they can get up to date if needed)
- showing progress to students during the bootcamp
- allowing post-bootcamp follow-up for the students (intermediate and advanced stuff)

Conclusion

Overall, we have progressed a lot on lesson management, exploring the bower-based solution, finding limitations and (re)opening a whole lot of questions. It is still at the exploratory/prototype stage but we learned interesting lessons.

The sprint was helpful and interesting for our project but we had limited interactions with people from other projects. Globally, I would recommend to have less projects, bigger and simpler ones so that there are more interactions between people.