seedcamp-ua, chytach pitch
May 24, 2008 – 16:30Yes, that’s me mumbling about Chytach
seedcamp-ua, adwiser.net pitch
May 24, 2008 – 11:37Seedcamp-ua is over
May 23, 2008 – 22:15Ukrainian session of seedcamp is over.
Chytach didn’t make through, but we’ve got incredible amount of information, inspiration and useful advise.
Going to make 2nd attempt on “big” seedcamp closer to autumn.
Meanwhile, trying to upload my flaky and broken 
videos to youtube…
nosetest exception helper
May 13, 2008 – 16:14Last night, enhancing the Chytach, I’ve again come across a feeling that standard nose asserts and decorators are not quite useful when testing exceptions.
For example, when I’m checking a django form validation, I’d like to make sure that my form throws a ValidationError (easy part) with a particular message (oops).
Nose proposes assert_raises for in-line checking and raises decorator. What I need is a raises with an additional pluggable check.
To make a long story short, here’s the usage example:
@testtools.raises(lambda e: check_validation_msg(e, 'Can\'t open "OMG'),
forms.forms.ValidationError)
def test_http_error(self):
'opml throws IOError'
forms.opml.parse = lambda x: testtools.raise_exc(IOError)
self.form.cleaned_data = {'url' : 'OMG, they\'ve killed Kenny'}
self.form.clean_url()
where checkvalidationmsg is a simple assert:
def check_validation_msg(exc, message):
assert list(exc.messages)[0].startswith(message), list(exc.messages)
Here’s the docstring from testtools.raises:
"""Test must raise one of expected exceptions to pass.
The raised exception should pass callback check.
Example use::
def callback(exc):
assert exc.args[0] == 'This test passes'
@raises(callback, TypeError, ValueError)
def test_raises_type_error():
raise TypeError("This test passes")
@raises(callback, TypeError, ValueError)
def test_fails_by_assert():
raise TypeError("This test fails")
@raises(callback, Exception):
def test_that_fails_by_passing():
pass
If you want to test many assertions about exceptions in a single test,
you may want to use `assert_raises` instead.
"""
You may download the working version here: testtools.py May 2008
highlights of May 2
May 2, 2008 – 19:42NASA Ames Partners With M2MI For Small Satellite Development
The goal is to develop a set of light (5-to-50 kg) mass-produced satellites to be launched on the low orbit to form a constellation around the Earth (kudos to Den Simmons?). This will allow to engineer a global fast mobile telco network.
Nanosatellites are small satellites weighing between 11 and 110 pounds. A large number of these satellites, called a constellation, will be placed in low Earth orbit for the new telecommunications and networking system. “The constellation will provide a robust, global, space-based, high-speed network for communication, data storage and Earth observations,” said m2mi Chief Executive Officer Geoff Brown. “Nanosatellites take advantage of the significant technological advances in microelectronics and will be produced using low-cost, mass-production techniques.” Under the agreement, NASA and m2mi will cooperate to develop a fifth generation telecommunications and networking system for Internet protocol-based and related services. The cooperative effort will combine NASA’s expertise in nanosensors, wireless networks and nanosatellite technologies with m2mi’s unique capabilities in software technology, sensors, global system awareness, adaptive control and commercialization capabilities. Fifth Generation, or 5G, incorporates Voice Over Internet Protocol, video, data, wireless, and an integrated machine-to-machine intelligence layer, or m2mi, for seamless information exchange and use.
Dougs thoughts on Google App Engine service: Google App Engine: The good, the bad, and the ugly?.
Abstract:
- Google’s offering is perfectly in line of
becoming evil masters of the worldowning all your dataproviding superb search and personalization services
- There are quite some problems. Some of them are due to the highly distributed nature of the service. Some, due to the beta status
- Python rules. And don’t expect something much different from pure python due to the security limitations and distributed service nature
And, finally, some smart campus building review from Guy Kawasaki.
Have a good day!
Natural limit on community size
April 23, 2008 – 17:59there is a cognitive limit to the number of individuals with whom any one person can maintain stable relationships, that this limit is a direct function of relative neocortex size, and that this in turn limits group size … the limit imposed by neocortical processing capacity is simply on the number of individuals with whom a stable inter-personal relationship can be maintained.Dunbar supports this hypothesis through studies by a number of field anthropologists. These studies measure the group size of a variety of different primates; Dunbar then correlate those group sizes to the brain sizes of the primates to produce a mathematical formula for how the two correspond. Using his formula, which is based on 36 primates, he predicts that 147.8 is the “mean group size” for humans, which matches census data on various village and tribe sizes in many cultures.
There are some interesting numbers in the article. Christopher suggests that the top boundary of the size of the group with creative relationships is about 12 persons. Another peak is around 50-70 individuals, which quite nicely corresponds to military unit size.
Looks like that there is a qualitative change in the nature of relationships when the group increases above each boundary. To keep the relationships (and, hence, the information and emotion flow) uncluttered, proxies become a necessity.
Information Software and the Graphical Interface
March 21, 2008 – 17:07Magic Ink. Information Software and the Graphical Interface by Bret Victor.
Long, but highly recommended reading.
Software design consists of graphic design (drawing pictures) and industrial design (allowing for mechanical manipulation).
Information software is for learning an internal model. Manipulation software is for creating an external model. Communication software is for communicating a shared model.
Manipulation software design is hard, but most software is information software.
Information software design is the design of context-sensitive information graphics. Information software is not a machine, but a medium for visual communication.
Context can be inferred from the environment, which can include physical sensors, other information software, documents created with manipulation software, and data such as email which acts as a user profile.
Context can be inferred from a history of past environments and interactions. Last-value predictors provide a rudimentary approach. Learning predictors can infer patterns and make dynamic predictions.
Context can be inferred from user interaction, but only as a last resort. The best way to reduce or eliminate interaction is through information-rich graphic design that uses the environment and history. Remaining interaction can be reduced with graphical manipulation, relative navigation, and tight feedback loops.
Couple feedparser bugs fixed
March 19, 2008 – 18:19If you massively use python feedparser, sooner or later you’ll encounter a feed with non-ascii symbols that tracebacks with somethings like
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 10: ordinal not in range(128)
or even
IndexError: pop from empty list
Using latest version from svn helps, but even it has some bugs 1, 2 in microformats parsing.
Since I use feedparser actively on chytach, I tend follow feedparser svn and to fix bugs quickly.
In case you need, I’ve opened a git repository with fixes on Gitorious
RPM with the fixes is on the way to the ALT Linux Sisyphus
Why I don't give fixed price. Software developer story.
December 24, 2007 – 01:49Recently I’ve chatted with a potential customer.
He seeks developers to create a simple social network application. He spent some time doing this. Actually, a considerable time and a sizable budget: one year and a half and undisclosed amount of money.
Now he’s got three different implementations of his idea. All three partially working. All three not ready for a production use. Two out of three developers broke their (extended) deadlines and he stopped to work with them. The leader of third group is not returning calls as deadline approaches.
It smelled like a typical deathmarch project for me from the very first words exchanges with current developer. Being there, done that, know what to do to get out of the deathmarch hell.
Despite I’m charging considerably more that his current developers, he seemed interested in me: I have project management experience, I speak like I know what to do, I can start right soon and can get more developers into the team.
Nevertheless we’ve not contracted.
The show stopped when I was asked to provide an estimate for the project.
My opinion is that asking for an estimate at the engagement to the software contract is suboptimal, at least. I can’t provide a project “estimate” if I’ve not spent time reformatting the specs to my standard, read the code, tried it and documented the defects, outlined the plan, estimated it, experimentally verified the first step estimate and corrected it. Just because “the estimate” in the software development is what other people call “the bid”.
What I can provide at start is a guesstimate – here we have N functions, M screens, so it should take kM + lM days.
But just when I append the usual disclaimer that this number should not be relied upon in any kind of financial and time planning, the understanding breaks.
Actually, what customer desires is just a fixed price bid, so he can get rid of financial risk.
Game of bidding
The bid selection on software project is too much influenced by the bid bottom line.
It would be a good thing, if it would include any kind of total cost analysis with some model attached. But what bid on a small software project does this?
At most, it includes a detailed analysis of cost drivers and a total development budget. And in most cases it just states the amount of money paid to developer upon project completion.
Nevertheless, the development budget is the easiest thing to understand in the bid. It’s just a number.
No wonder the bid is mostly selected by the budget. No wonder developers learn the game very fast – underbid and get the project. Whatever happens next.
Parameter space of the software project
It is an established custom to treat a software project within 3d parameter space:
- budget,
- delivery time,
- quality.
If the project has fixed complexity the volume, occupied by the project in that space, presumably, stays constant.
But…
Where’s the project complexity is fixed? There’s no such thing like a perfect information transfer between two persons. There are always errors in understanding. The project requirements, unless specified in a strict semantics (like programming language ) are always subject to an interpretation.
Therefore the actual best estimate of the volume, project occupies in the parameter space, will always change with time.
That’s the reason why any project plan should allow room for errors and resources to fix them.
Logically, next question from a customer would be: why I can’t pay an insurance against such errors? Finally, it’s the developer who’s the specialist? Why can’t he include that in the price?
Simple probabilistic model of a feature development.
Of course, a developer can do that. If he has the level of discipline, comparable to the traders of financial products (e.g. options).
Most developers don’t.
The pricing of options, err.. feature estimation is counterintuitive. And even the best financial models tend to underestimate the effect of the “long tail”, the effect of events with little probability.
Let’s build a simple probabilistic model of single feature development.
Let the complexity of feature would be 8 points. That means, it would require 8 hours of constant attention of the qualified and prepared developer, without any interruptions or problems.
Of course, that won’t translate into 8 billable hours. It could be anything from zero to infinity with a peak near 8. Good developers will have sharp peak, mediocre – wide one.
Also, say, it would take about a clock day to complete. With the same probability distribution.
Next, someone had to review the feature. Name it code review, acceptance testing, anything. Let it require 1 point of complexity and require another a day of clock time to complete.
The review has 4 chances to accept the feature to production and 1 chance to return it to development.
(Gamma distribution on all values, alpha = 2, beta = number of complexity points or number of days)
What would be 95%-level certainty estimate for the implementation of this feature to take?
41 billable hours and about 9 days to complete!
What would be 50%-level certainty estimate? 16 billable hours, 4 days to complete.
How that compares to blind estimation of 8 + 1 hours? Where is the intuition there?
The fallacy of risk minimization
A rational developer, equipped with such model, would give estimated 41 billable hour and ask for a 9 business days timeframe to complete the task.
So what the “rational” customer desiring to minimize his budget risk would pay? He has quite good changes (45%) to pay 25 hours less (41h – 16h) and get his result sooner by a work week.
Then why the hell I’d advise my customer to overpay me? Why would I put myself in ridicule position of explaining how “a day worth work” translated into more than a work-week fixed price bid? Or why I would underbid and increase my own risks?
I would welcome any analysis of the simple model above to show that I’m wrong.
The recursive nature of estimation of software project.
See how the development of a simple “day-worth” feature translated to complex “inflated” estimation?
But the actual software project is much more complex than a set of such features. Features get dumped, added, misunderstood and, finally, even some bugs slip review and (OMG!) go onto production. This all adds another loop of confusion and complexity and inflates a reliable bid even more.
Yet customers require fixed price bid even before the detailed planning is done. What a developer ought to do? Refuse to bid a fixed price, bear the risks or play “this time will be different” game?
The way out.
The software development is a superposition of several complex probabilistic processes. An attempt to put a fixed price tag onto it would be similar to put a fixed price on an financial instrument. An attempt to establish a “fair” price on goods based on careful analysis. History teaches that no one learns their lesson latest huge attempt to do this failed miserably.
Financial market has its own solution – the trading, which establishes what is believed a fair price.
And it’s time to recall that the goal of a software development is not to establish a “fair” price for a project. The goal is to develop a product.
An attempt to minimize risks by obvious means finally costs much more for a customer than the risk itself. The obvious means are not rational.
An attempt to minimize risk in a software project is similar to an attempt to calculate an outcome of tennis match from the names of players and the brand of rackets. Do really any rational tennis player try to set the game plan ahead?
No, of course. They do the work in tight iterations. Watch-move-swing-hit-watch-move… etc They minimize risk by close coupling of facts observed (moving ball) and their own movements.
In software development the analogy would be tight development iterations. Promote enhanced versions to production weekly. Release on staging daily. Do a continuous integration builds to watch the code integrity after each build.
And if you are overbudget, make a business decision to stop or to allocate more funds. That’s not a development problem.
Cease overplanning. Get a life.
PS
Ah, yes, my customer, I’ve started with…
Yes, you know his story already. He was blinded by estimations he was given and his desire to minimize the risk. His product is in the development from early 2006 and is not yet complete. What he tries now is to get “more reliable estimates”, preferably from “more experienced developers”. Let the force be with him.
