Integration using git and buildbot

August 7, 2008 – 11:35 pm

Another daily task here is somewhat opposite to autofollowing an open source project.

As I’ve mentioned there, we develop our modifications in git branches. This also stands for commercial projects. And, of course, our customers expect us to integrate our modifications back into their codebase.

Since I’m the integration point, I spend most of my time watching how tests pass:

  • full test suite on the feature branch, run by buildbot
  • git merge --no-commit with the feature branch from the main branch; if we’re following the svn project, we use also --squash to not mess the git-svn logic
  • run the full test suite again, this time run manually
  • commit with a reasonable message and push the change upstream

If both tests pass ok, the only point where manual intervention may be actually necessary is the step when the commit message gets written.

Definitely the next thing to be scripted.

Autofollowing an open source project

August 6, 2008 – 12:54 am

Looks like the next biggest time eater (NBTE? ;)) during our development are the efforts to follow public projects repositories.

See, we develop web software mostly by integrating already available open source components. But what if a component is not yet stable and is actively developed?

Also, we often have to branch these projects to implement our own modifications: bug fixes or new features. Thanks to git that’s easy, but what about keeping track on the development of the original project?

Let the ‘master’ branch follow an svn and local modifications be integrated in the branch ‘local’. Then, the workflow is easy:

  • git svn rebase on the ‘master’ branch
  • optionally run the tests
  • git merge --no-commit master on the ‘local’ branch
  • optionally run the tests
  • commit if tests are ok
  • move ‘last good’ tag on a ‘master’ and ‘local’ branches

Will spend couple of hours in next few days to implement and get a feeling how that works in real life.

Wanna lend me an hours per day extra? ;)

Interchangeable hardware or “home cloud”

August 4, 2008 – 4:43 pm

Yesterday my border router in the home network crashed.

It was a P4 PC under Linux with several network cards routing a traffic between three Internet connections (yes, I’m paranoid), my home network and my house network.

That morning either CPU or motherboard fried and…

Now I’m thinking about something more interchangeable than stock PC. Well, actually, this is more about plug-n-play software than a hardware.

You may imagine that I have quite sophisticated routing setup here. Also this server ran squid, postfix, imap, traffic accounting software and was also a file server.

Most of the software is em… box-agnostic, at least until you move the config and data files. Routing is somewhat more difficult, since on a different PC box, ether mac addresses most probably would be different.

Ideal usage scenario would be something like that:

  1. Bring a replacement server, install a bare-bone system there and connect it to the network.
  2. Power the old system up on a temporary hardware (or just attach old drives to a new box)
  3. Issue something like vzmigrate of OpenVZ and, voila - new system has all old services up and running.

Obviously, that won’t be that easy with the routing, but if OS on both system have similar configs, that would save much time.

Interesting that the easiest implementation of this approach would be to do it verbatim: just move each service into its own OpenVZ VE - service appliance.

Unfortunately, the old system was set up in late 90th when the virtualization state of art was VmWare. So, I’m going to have a long-long-long evening :)

bug tracking flavours

July 12, 2008 – 6:52 pm

Yesterday found two interesting instances of distributed bug tracking systems:

Both are command-line, both support git.

Would be interesting to have try a web frontend for them on a live project..

4 types of rights

July 10, 2008 – 4:20 pm

Just came over interesting OpenCourseWare page about 4 fundamentally different types of rights:

  • claim-rights
  • privileges (liberties, absence of claim-rights)
  • powers (authority rights)
  • immunities (absence of power rights)

git and svn: add another branch

July 10, 2008 – 4:12 pm

If you use git as your software project workhorse and yet have to track remote svn repo, then you might be interested.

Today I had to add tracking of another svn branch.

Git branches are essentially no-op. Svn are not that good, but nevertheless…

But git clone of yet-another-svn-branch (in the existing repo) appeared to be an issue.

Thanks to dmo.ca, this was finally done.

In short, you have to add another svn-remote to your .git/config:

[svn-remote "svn"]
    url = svn+ssh://your-server/home/svn/project-name/trunk
    fetch = :refs/remotes/git-svn

and then fetch the revision that is on the branch of interest (that’s important):

git-svn fetch svn34  -r MMMM

After it completes, you may check your branches with

git branch -r
Zemanta Pixie

Tracking development consistency

June 18, 2008 – 11:35 pm

Rationale

Our small team has 3 developers working full time on various projects. We practice throughout automated testing and aim to get 100% code coverage with unit tests.

Our internal version control system is git. Active development within git means couple or more new branches in the central repo daily and lots of merges.

Someone has to keep an eye on this whole stuff.

Having lots of automated tests, makes it much easier than running QC on them mannually after every commit :)

Questions to be answererd:

  • is the last push ok?
  • is the last merge to the master ok?
  • is this new branch ok?

Solution

We use buildbot.

It has to watch several things:

  1. Commits to the master branch, which are usually pre-release merges for my review.
  2. Commits to other branches, which are usually work-in-progress
  3. New branches - new work is done or something tangible appeared

Nice to have

Some integration with Trac. E.g. it would be nice to see what commits (to the central repo) where made on this ticket or what’s the last successful build for that ticket.

There are lots of fun, interesting and useful things to be done within git-buildbot-trac integrated system. Yet it requires some hands-on experience to check what’s the best corner to cut and the biggest pain to cure.

Tracking time

June 17, 2008 – 3:01 pm

Rationale

I, being a software developer, team leader, project manager and,
generally, a busy person, desperately need time tracking and planning
tools.

More exactly stated, it’s not the tool I need, but I need instant
answers to questions, whenever I’d like:

  • where my time had gone?
  • what’s the closest checkpoint time for every developer online?
  • how much time my team mates spent and what task it was?
  • how much time I have to bill my customer (per customer per project)?
  • how much I have to pay to my developers (per developer, obviously)?

These questions are very different, require different information to
be collected and different decisions are to be made using the answers.

Personal time flow

Record attention flow (mouse pointer, active and top windows, keyb
activity) for future analysis.

Analysis result should within single graph depict time flow and how it
was really spent on different tasks.

Problem: how define “tasks” automatically/semiautomatically?

Similar solutions:

  • (link to macosx solution lost)
  • gfocustimer - declares to watch the focus, but doesn’t seem to work

How to implement:

  • clone X11 xev to record X events to the database
  • analyze db with a custom script

Communicate checkpoints

Since our usual communication channel is a jabber room, the solution
should be probably a jabber bot, residing there, monitoring what is
said and keeping a checkpoints record.

A log of some kind would be useful too.

Similar solutions: any?

How to implement: go and hack; it should be simple.

Track billable time

This, probably, should be a simple start-stop watch residing in a dock
area or completely off-screen.

It should record tasks and start/stop events to a local db, allow user
to edit it and report it upstream.

Similar solutions:

How to implement: since there are vast number of time-counting
programs, the task boils down to integrating them with some protocol.

Consolidate billable time

Get updates from solution, crafted in the previous part, and drop it
into bigger db. This time the resulting db probably should be sql,
for the ease of reporting.

This also should integrate with Trac somehow.

Similar solutions: see above - everything starts with the protocol

How to implement: hack it.

Summary

This pretty much outlines what I need from time-tracking tools. Few
(if any) solutions are ready, so most of them are to be hacked out
ourselves. Will be glad to be proved wrong.

Now, well, the estimation is the whole different story.

Zemanta Pixie

seedcamp-ua, “Product Design & Development” panel

June 7, 2008 – 10:10 am



seedcamp-ua, “Building an International Business & Exiting” panel

June 4, 2008 – 10:29 pm