Andrey Khavryuchenko himself

twill http equiv-refresh fix

Just have blogged that twill bugs me. So here’s the fix. As always, dead simple to locate the guilty and patch it.

If you’ll open twill/utils.py and scroll down to the line 408, you’ll find equiv-refresh header parsing code.

Here are the guilty lines:

            j = newurl_spec.find("=")
            if j != -1:
                newurl = newurl_spec[j+1:]
            else:
                newurl = newurl_spec

Of course, if url contains a parameter just like /there/is/path/?key=something, twill happily mangles the url.

The correct code is:

            j = newurl_spec.find("url=")
            if j != -1:
                newurl = newurl_spec[j+len('url='):]
            else:
                newurl = newurl_spec

When I’ll find a twill git or svn, I’ll create a patch.

Posted in Python | No Comments »

twill mishandles http equiv-refreshes?

Image via WikipediaIt’s Friday night after hectic work week and I’m too tired to investigate further, but….

It feels to me that twill breaks following http equiv-refreshes from the code. See the debug output:

reply: 'HTTP/1.0 200 OK\n'
header: Vary: Accept-Language, Cookie
header: Content-Type: text/html; charset=utf-8
header: Content-Language: en
equiv-refresh DEBUG: code 200, hdrs has 'refresh'
equiv-refresh DEBUG: refresh header is 10; /files/aabbcc/?key=yVck
equiv-refresh DEBUG: pause: 10
equiv-refresh DEBUG: new url:  /files/aabbcc/?key=yVck
equiv-refresh DEBUG: final url: yVck
Following HTTP-EQUIV=REFRESH to yVck
send: 'GET /files/aabbcc/upload/yVck

How, in the name of Gods of Friday night debugs, /files/aabbcc/?key=yVck converted to /files/aabbcc/upload/yVck?

Posted in Microentry, Python | 2 Comments »

Python transactional memory?

Ok, nosetests whitewashed in one of accused misactions – it didn’t actually messed up with the metadata of my django database backend. It was me, myself.

Looks like my test wrappers db_mock and twill_mock where not quite up to date to latest django db backend changes and forgot to save their environment cleanly and then – even restore it properly.

Now the question – is there some tool or module for a python that allows to handle global variable state in a transactional manner? A kind of lightweight STM?..

I understand that it could hardly be possible for C modules, but I’m asking just for native python variables.

Any ideas?

Posted in Django, Microentry, Python | No Comments »

satchmo multi-shop

Good to see that satchmo multishop branch is merged into the trunk. Bad is that this is not reflected on the MultiShop wiki page – had to waste some time and even try to start a fork on multishop branch to update it to django 1.0…

Posted in Django, Microentry | No Comments »

nosetests smashing module content

Image via Wikipedia Spent several hours yesterday untangling yet another mysterious test crash:

Warning: Field 'id' doesn't have a default value

As appears, it happened during usual INSERT to djangocontenttype… WTF!?

mysql> describe django_content_type;
+-----------+--------------+------+-----+---------+-------+
| Field     | Type         | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+-------+
| id        | int(11)      | NO   | PRI | NULL    |       |
| name      | varchar(100) | NO   |     | NULL    |       |
| app_label | varchar(100) | NO   | MUL | NULL    |       |
| model     | varchar(100) | NO   |     | NULL    |       |
+-----------+--------------+------+-----+---------+-------+
4 rows in set (0.01 sec)

Where the #$^ AUTOINCREMENT is?

To make long story short, it appeared that for some strange reason django.db.connection.creation.data__types contains the values for sqlite that we use in previous tests and not mysql backend that is reported to be used.

And, yes, isolation plugin of nosetests doesn’t help.

Posted in Microentry, Python | No Comments »

How to add svn tracking to existing git project

Image via Wikipedia Tracking svn repository with a git is a pleasure, thanks to git-svn:

git svn clone (svn url)

and go.

But until today it wasn’t clear to me how to add an svn for already existing git project. For example, if a client requested to push the progress to his svn repo to be able to contribute himself.

It appears that it’s dead easy too:

git svn init (svn url)

works lovely in already existing git repository without bothering any of your git objects.

Repeat with me: git is good

PS Have an excellent and detailed introduction for git-svn integration.

Posted in Microentry, Software Development | 2 Comments »

nosetest mangles sys.path?

Struggling for a second hour in a row. Feeling dumber with every minute spent.

Code:

import pprint
import sys
import django.conf
assert django.conf.__file__.startswith(
    '/home/akhavr'
    ), pprint.pformat((sys.path, django.conf.__file__))

Test run:

PYTHONPATH=/home/akhavr/src/django-hotclub:/home/akhavr/django/:/home/akhavr/src/django-hotclub/apps:/home/akhavr/src/django-hotclub/site-packages/misc:/home/akhavr/src/django-hotclub/site-packages/python-openid-2.1.1:/home/akhavr/src/django-hotclub/site-packages/python-yadis-1.1.0:/home/akhavr/src/django-hotclub/site-packages/python-twitter-0.5/:/home/akhavr/src/django-hotclub/site-packages/pownce-api-0.2:.. DJANGO_SETTINGS_MODULE=pinax.settings nosetests --with-coverage --cover-package=projects --exe -x  projects
F
======================================================================
FAIL: Failure: AssertionError ((['/home/akhavr/src/django-hotclub/pinax/local_apps/projects/tests',
  '/usr/bin',
  '/usr/lib/python2.5/site-packages/coverage-2.78-py2.5.egg',
  '/usr/lib/python2.5/site-packages/twill-0.9-py2.5.egg',
  '/usr/lib/python2.5/site-packages/opml-0.5-py2.5.egg',
  '/usr/lib/python2.5/site-packages/geopy-0.93-py2.5.egg',
  '/usr/lib/python2.5/site-packages/PyCAPTCHA-0.4-py2.5.egg',
  '/usr/lib/python2.5/site-packages/flickrapi-1.1-py2.5.egg',
  '/usr/lib/python2.5/site-packages/simplejson-1.9.1-py2.5-linux-i686.egg',
  '/usr/lib/python2.5/site-packages/httplib2-0.4.0-py2.5.egg',
  '/usr/lib/python2.5/site-packages/icalendar-1.2-py2.5.egg',
  '/usr/lib/python2.5/site-packages/CouchDB-0.5dev_r88-py2.5.egg',
  '/usr/lib/python2.5/site-packages/django_mptt-0.2.1-py2.5.egg',
  '/home/akhavr/src/django-hotclub',
  '/home/akhavr/django',
  '/home/akhavr/src/django-hotclub/apps',
  '/home/akhavr/src/django-hotclub/site-packages/misc',
  '/home/akhavr/src/django-hotclub/site-packages/python-openid-2.1.1',
  '/home/akhavr/src/django-hotclub/site-packages/python-yadis-1.1.0',
  '/home/akhavr/src/django-hotclub/site-packages/python-twitter-0.5',
  '/home/akhavr/src/django-hotclub/site-packages/pownce-api-0.2',
  '/home/akhavr/src/django-hotclub/pinax/local_apps',
  '/usr/lib/python25.zip',
  '/usr/lib/python2.5',
  '/usr/lib/python2.5/plat-linux2',
  '/usr/lib/python2.5/lib-tk',
  '/usr/lib/python2.5/lib-dynload',
  '/usr/lib/python2.5/site-packages',
  '/usr/lib/python2.5/site-packages/Numeric',
  '/usr/lib/python2.5/site-packages/PIL',
  '/usr/lib/python2.5/site-packages/gtk-2.0',
  '/usr/lib/python2.5/site-packages/wx-2.8-gtk2-unicode'],
 '/usr/lib/python2.5/site-packages/django/conf/__init__.pyc'))

Why sys.path is ignored and django is loaded from the site-packages instead of my ~/src snapshot is beyond my understanding now.

Posted in Microentry, Python | 2 Comments »

django-hotclub AKA pinax svn externals

Have yet to think out how to handle those django-hotclub numerous svn:externals. Git advise given here is nice (I’m following the same route), but it doesn’t answer the issue of managing the regular single-command deployment of those dependencies.

On the other hand, packaging each of these packagelets into tiny RPMs (or debs) feels like an overkill. Even if ALT gear would follow and build them automatically.

Food for thoughts..

Posted in Microentry, Software Development | No Comments »

Integration using git and buildbot

Another daily task here is somewhat opposite to autofollowing an open source project.

As I’ve mentioned there, we develop our modifications in git branches. This also stands for commercial projects. And, of course, our customers expect us to integrate our modifications back into their codebase.

Since I’m the integration point, I spend most of my time watching how tests pass:

• full test suite on the feature branch, run by buildbot
• git merge --no-commit with the feature branch from the main branch; if we’re following the svn project, we use also --squash to not mess the git-svn logic
• run the full test suite again, this time run manually
• commit with a reasonable message and push the change upstream

If both tests pass ok, the only point where manual intervention may be actually necessary is the step when the commit message gets written.

Definitely the next thing to be scripted.

Posted in Software Development | No Comments »

Autofollowing an open source project

Looks like the next biggest time eater (NBTE? ) during our development are the efforts to follow public projects repositories.

See, we develop web software mostly by integrating already available open source components. But what if a component is not yet stable and is actively developed?

Also, we often have to branch these projects to implement our own modifications: bug fixes or new features. Thanks to git that’s easy, but what about keeping track on the development of the original project?

Let the ‘master’ branch follow an svn and local modifications be integrated in the branch ‘local’. Then, the workflow is easy:

• git svn rebase on the ‘master’ branch
• optionally run the tests
• git merge --no-commit master on the ‘local’ branch
• optionally run the tests
• commit if tests are ok
• move ‘last good’ tag on a ‘master’ and ‘local’ branches

Will spend couple of hours in next few days to implement and get a feeling how that works in real life.

Wanna lend me an hours per day extra?

Posted in Software Development | No Comments »