Not long ago I recorded a katacast of the KarateChopKata in Java. With a recent bit of downtime in between projects I decided to do a C# version in Visual Studio 2010 and ReSharper 5. (I put down some observations about the latest editions of these tools in Vs2010AndReSharper5 - if I had more time I'd have liked to do the same kata with just Visual Studio 2010 to help demonstrate some of the ReSharper advantages).
One catalyst for the new C# version was a small twitter discussion with Kent Beck and Paul Nelson about TDD-ing a quick sort algorithm. In my Java effort I experimented with directly exposing comparison counts from the production class, but that was messy and wrong. The production class didn't need to record counts and exposing internals is generally to be avoided. Kent tweeted a better alternative: inject a comparator class.
I've taken that approach here which allows me to properly drive the production class from the tests. The production code can go the easy route as the tests evolve but then has to shift gears into the binary search once the comparator is introduced.
This may be gratuitous use of injection for the sake of the kata, but it does demonstrate the power of injection to decouple some aspects of an algorithm, if only for testability in this case.
Unlike the Java kata where I tried to tease out internals of the algorithm from tests, in this performance I simply jump from the loop to the binary search. I've also reduced the number of tests because I wanted to try a more proper arrangement of the tests in separate methods in addition to the split screen presentation. I suppose an ideal performance would include both the comparator concept and the smaller grained tests that can go green when only part of the algorithm is implemented.
Like the Java recording, the screen capture and video encoding was done with Microsoft Expression 3 on Windows 7. KeyPosé was used for the keypress overlay and a quick shout out to Audacity for helping me quickly splice my 2 mp3 files together. There are a few edits for timesake, including the same magical appearance of the upTo method.
For those paying attention, I'm not using the default shortcuts for ReSharper. I worked with Eclipse for a couple of years before ReSharper, so ever since starting with R# I've been keeping up vssettings files to change R# shortcuts to match the Eclipse ones which can now be found here.
I decided to not do the ary() helper method in the test code like I did in the Java version, though it could be done in C#:
Again, thanks to my employer, Improving Enterprises, for access to the tools. As advertised at the end of the katacast, Improving offers all sorts of agile training and consulting services, click through for more details.
I've been getting up to speed with the latest Visual Studio and ReSharper versions while playing with the KarateChopInCSharp. Some observations thus far:
Visual Studio continues to make progress in the area of refactorings and quick fixes/assists. With just VS I was able to generate a class, method, field/local and constructor, though the method generation doesn't allow me to tab through the params of the method signature like R# does. These generations are available after typing in a class or method name (etc.) that doesn't yet exist.
Some things I still needed R# for including toggling inline conditions to if-else and extracting a local variable from a highlighted value, though I'm sure there are many more here. A big one for me is automatic code formatting. Code formatting allows me to be sloppy with my braces and indentions and the like. I'll frequently hit the shortcut for code formatting, freeing me from a fair amount of busy work. VS does do _some_ automatic formatting, for example, when typing a closing brace, but it's a bit limited and there's no way that I'm aware of to force a formatting on a highlighted section or the entire file without ReSharper.
One place Visual Studio appeared to be better than R# was with its unit test runner. With a small bank of MSTest tests, VS's runner was noticeably faster than R#, even with R#'s shadow copy feature turned off. I do miss the green bar in Visual Studio's runner UI, though - it is possible to have a failing test scrolled out of view and not have an obvious indicator of overall success or failure.
ReSharper remains a must-have tool for me. While it's good to see Visual Studio making progress, they're still years behind where Eclipse (and presumably IntelliJ) have been, and that's a shame.
(On a side note, I've been keeping some vssettings files for changing ReSharper keymaps to match (more or less) Eclipse defaults for a few years and have moved them all to GitHub, including a new one for VS2010 and R# 5.)
I got the idea to contribute to the katacasts in early 2010 and had the screencast mostly recorded by February, but didn't get around to polishing it up and posting it until now (April). Not only was it an excuse to get into practicing a code kata myself, but I already had a soundtrack in mind. I picked this kata because it was the first coding kata on Dave's list.
It proved a little tough at first because it was easy to test-drive a solution that didn't require a binary search. Like many practitioners, I like to take a sort of ping-pong pairing approach where I first respond to a test by providing just enough code to get the tests to pass. This helps encourage the tests to have thorough coverage. However, blowing through an array of 100k integers sequentially has no real downside in this isolated case.
I experimented with refactoring the class under test such that the test could query the number of comparisons done to help enforce a binary search, but that felt unnatural exposing the internals of the class. Trying to trick up the scenario such that the test would perform very slowly if it wasn't a binary search felt like it was getting away from the spirit of the kata itself.
After some extra iterations and tinkering I came up with the approach seen here. I think it captures some of the essence of ping-ponging, though admittedly the code knows where it's going ahead of time.
The screen capture and video encoding was done with Microsoft Expression 3 on Windows 7. KeyPosé was used for the keypress overlay. The Expression tools worked well, especially the screen capture tool, though I had a bit of a tough time wringing out a final video that YouTube would grok correctly. There are a few cuts (like the magical appearance of the upTo() implementation) to help fit the video to the audio and clean up some rough edges. Thanks to my employer, Improving Enterprises, for access to the tools. As advertised at the end of the katacast, Improving offers all sorts of agile training and consulting services, click through for more details.
I heard this on NPR recently: "Why we can't find what we're looking for". Researchers at Harvard Medical School are studying how good people are at finding rare occurring problems, people like airport security personnel and radiologists looking for tumors.
"We know that if you don't find it often, you often don't find it," said [Lead researcher Jeremy Wolfe].
Rare stuff often gets missed. That means that if we look for 20 guns in a stack of 40 bags, we'll find more of them than if we look for the same 20 guns in a stack of 2,000 bags.
In the NPR interview, the researcher went on to say even when doing the experiment on himself, with full knowledge of how many pictures in the stack of 2000 contain images with guns in them, he can't make his own accuracy increase. And any urgency brought to bear on the worker knowing what's at stake if something is missed does not seem to have any impact on their performance.
I think this has interesting implications for developers trying to find bugs in their code. Sheer willpower is not enough to improve our quality.
I'm learning a bit of git while in a subversion environment, mainly to do cheap dev branching. After successfully committing my first fix up from a git branch back into svn, I deleted the branch and didn't think much about it. Later I dcommit-ted a second fix on another local git branch and started to delete that branch but then got nervous. What could I do to ensure I wasn't about to delete something that had a pending change on it?
The PragProgGit book mentions that the default delete command provides a safety check and won't delete a branch that hasn't been merged. However, I realized that I hadn't merged my first bug fix branch back to my git master, so how was it that I was able to delete that branch? Had I found a loophole in the process since I was throwing some git-svn into the mix?
I found in the git-branch man a --merged option on the git branch command that should show me in advance what branches it considered safe to delete.
After playing around some, I realized I _had_ merged my original qc branch to my master branch, just not directly -- git (I presume) realized the merge had occurred because I'd done a git svn rebase command on master.
Here's a snippet from the console demonstrating this in action. Note, on the qc4042 branch I'd already done a git svn dcommit of all changes on that branch, but I hadn't dcommitted anything from the qc3937 branch.
c:\src>git branch * master qc3937 qc4042
c:\src>git branch --merged * master
c:\src>git branch --no-merged qc3937 qc4042
c:\src>git branch -d qc4042 error: The branch 'qc4042' is not an ancestor of your current HEAD. If you are sure you want to delete it, run 'git branch -D qc4042'.
c:\src>git svn rebase First, rewinding head to replay your work on top of it... Fast-forwarded master to refs/remotes/svn/git-svn.
The most important factor in software work is not the tools and techniques used by the programmers, but rather the quality of the programmers themselves.
Joel also has a great essay on how attempts to codify the successfulness of talented people just don't work.
1. Some things need talent to do really well. 2. It's hard to scale talent. 3. One way people try to scale talent is by having the talent create rules for the untalented to follow. 4. The quality of the resulting product is very low.
I was listening to a local sports station the other day interview John Gagliardi, who, on 11/8/03, became college football's winningest coach. A lot of the interview centered around John's coaching style, and the fact that he does a lot of things different (no tackling during practice, for example). The overall gist was that he didn't have a very detailed infrastructure for his team -- much of what a big college football program would do, he doesn't worry with. In fact, he summarized his approach this way (and I paraphrase because I didn't write down the quote being in the car at the time):
What you really need is good players. Good players don't need a lot of rules.
(Or structure: "On the best teams, different individuals provide occasional leadership, taking charge in areas where they have particular strengths. No one is the permanent leader... The structure of the team is a network, not a hierarchy." Peopleware: Productive Projects and Teams, 2nd ed., p. 155, Tom DeMarco and Timothy Lister -- via Dave Hoover's blog
1/12/04 - Bill Caputo posts this [quote is digestized]:
[Non-agilists] see nothing -- or very little -- in the XP process definition that will make people successfully deliver -- but its true of any process. [They] want a process that will make others succeed. Agilists believe this view of process is inherently flawed. No process makes you successful, but people who will succeed anyway can do so with a more or less painful process.
XP is simply the best collection of practices I have ever found that are generally useful in addressing the problems I try to solve on each project -- but in the end, I succeed because of me and the people around me, not our process, which we readily change in response to our current challenges.
[W]e don't claim XP will protect us from the harmful, we claim that it aids the successful.
What do you consider the most important thing for a programmer to do when he begins working on a new project?
I think each should be sure they are in good physical condition without nagging psychological problems.
What do you consider the most important thing for a programmer to do when he begins working on a project that has already begun?
She should get sufficient information to decide, before signing on, whether she should sign on. Most programming projects that fail have already failed before most of the programmers have signed on, but through lack of courage or due diligence, many programmers sign on anyway. It's like doctors agreeing to do surgery on corpses.
Would you recommend a career in programming to young people today?
It depends on what the young person wants to do. I always give the same career recommendation: "Do what you want to do."
What courses would you recommend they take? What languages/technologies should they key on?
They shouldn't key on languages and technologies. They should key on learning to communicate, to think, and to work well with other people. Once they have those, the languages and technologies become simple matters. Without them, no amount of language or technology expertise will do much good.
5/20/04 - Fast Company article on how good companies become great. The secret sauce? People. [via Clarke Ching]
Take David Maxwell's bus ride. When he became CEO of Fannie Mae in 1981, the company was losing $1 million every business day, with $56 billion worth of mortgage loans under water.
Maxwell told his management team that there would only be seats on the bus for A-level people who were willing to put out A-plus effort. He interviewed every member of the team. He told them all the same thing: It was going to be a tough ride, a very demanding trip. If they didn't want to go, fine; just say so. Now's the time to get off the bus, he said. No questions asked, no recriminations. In all, 14 of 26 executives got off the bus. They were replaced by some of the best, smartest, and hardest-working executives in the world of finance.
With the right people on the bus, in the right seats, Maxwell then turned his full attention to the "what" question. He and his team took Fannie Mae from losing $1 million a day at the start of his tenure to earning $4 million a day at the end.
11/11/04
[O]nly a virtuous people are capable of freedom. As nations become corrupt and vicious, they have more need of masters.
Source: Benjamin Franklin, The Writings of Benjamin Franklin, Jared Sparks, editor (Boston: Tappan, Whittemore and Mason, 1840), Vol. X, p. 297, April 17, 1787.
If I had to pick one as my key to software development it's that the critical element in a software development effort are the people you have doing the work. The productivity of the best developers is far more than the average, much more than the difference in salaries.
He added another article in Feb 08 talking more about the productivity factor:
Although the technorati generally agree that talented programmers are more productive than the average, the impossibility of measurement means they cannot come up with an actual figure. So let's invent one for argument sake: 2. If you can find a factor-2 talented programmer for less than twice of the salary of an average programmer - then that programmer ends up being cheaper. To state this more generally: If the cost premium for a more productive developer is less than the higher productivity of that developer, then it's cheaper to hire the more expensive developer. The cheaper talent hypothesis is that the cost premium is indeed less, and thus it's cheaper to hire more productive developers even if they are more expensive.
One of the great things I keep seeing (used to be "keep learning", but at least by now I half expect it when things blow up in my face), is how the individual chemistry between people operates outside of all the nice theory we construct.
Mary Poppendieck (via InfoQ), offers an interesting quote that seems to come across counter to the other quotes on this page:
We get brilliant results from average people managing brilliant systems. Our competitors get average results from brilliant people working around broken systems. - Fujio Cho, Chairman Toyota Motors
Elsewhere at Poppendieck's site, this paper gives more insight:
[Scholtes] says, "All of the empowered, motivated, teamed-up, self-directed, incentivized, accountable, reengineered, and reinvented people you can muster cannot compensate for a dysfunctional system...." So where does this leave us? Which is more important - process or people?
...
[The answer is both, "Process AND People"] ... People like to use effective processes, and they also like to have control over their own environment.... Process improvement may be done only "at the gemba" [the place of the problem] and it is up to the workers to decide whether or not a proposed improvement should be implemented.
Mistake No. 1: Start with a mediocre team of developers. Designing software is hard, and unfortunately, a lot of the people who call themselves programmers can't really do it. But even though a bad team of developers tends to be the No. 1 cause of software project failures, you'd never know it from reading official postmortems.
...
At Fog Creek, we tend to review about 400 candidates for every full-time hire, because the best developers can be 10 times as productive as the merely excellent developers.
A mature person is one who is highly conscious of when it's appropriate to follow rules and when to break them. A mature person is largely self-guided. Only in exceptional circumstances does a mature person need to refer to or appeal to a rulebook at all.
A rare (on this page) dissenting opinion from Michael Hill (though it does align with the quote earlier on this page from Fujio Cho, Chairman of Toyota Motors, "We get brilliant results from average people managing brilliant systems."):
"XP Requires High-Skill Teams"
This noxious notion burbles around the interwebs in various configurations, as it has since the beginning of XP. It’s mistaken.
I’ve worked with many more low-skill teams than high-skill ones, for two reasons:
I’m not cheap, therefore much of my business comes from teams that are in trouble. High-skill teams can hide and defer trouble far longer than low-skill ones.
There are far too few programmers, therefore the world of commerce has to settle for low-skill (and no-skill) ones. At least 75% of all programmers are low-skill ones.
I keep thinking I should give up before I start when I see a low-skill team. Fortunately, I am really bad at money, so I don’t refuse many clients. This has impressed upon me the complete irrelevance of skill level in adopting and adapting XP.
... and what Michael calls irrelevant, Brian Marick says is missing from the original Manifesto:
[Teams] should invest in one of the four values I want to talk about: skill. As someone who wants to remain anonymous said to me:
I’ve also been tired for years of software people who seem embarrassed to admit that, at some point in the proceedings, someone competent has to write some damn code.
You can name tests after their scope: (unit, component, system), their place in the development process (smoke, integration, acceptance, regression), their focus (performance, functional) their visibility (white box, black box), the role of the people writing them (developer, customer)... The list goes on. There are far more than I can remember.
Why is it so confusing? ... The thing which makes testing nomenclature [bad] is that the tests themselves aren't all that different ... we can tell the difference between a unit test and an acceptance test in most systems, but really there is no force which prevents tests of different types from bleeding through into each other.
I frequently have integration type tests mixed into my unit tests, and only after they've gained enough weight to drag out the execution time of my suite will I start to segregate them. My usual distinguishing factor is 'quick' and 'long-running'.
One of the reasons for this is I tend to favor writing tests which roundtrip to a database (for example) instead of using mocks. I'm fine with mocks (though not so much mock libraries - see DiyMocks), but if using mocks leaves a gap in integration tests, then I'd rather start with the integration test and back fill with mocked unit testing if the testing in that arena becomes intense. However, I've found with proper separation of concerns, the places where mocks would be the most useful tend not to accumulate enough logic to make it worthwhile.
Michael Hill weighs in on Microtests vs. Unit Tests. Since he linked to his own post on the testdrivendevelopment yahoo group and I didn't see his definition of unit test in his post, I asked him about it:
The biggest single difference is scope, and the second biggest is shipping-reality.
Traditionally, unit tests involve much larger units. A typical old-school example would be a set of unit tests that run against a COM object, regardless of whether that interface involves a single class or 300 of them. Microtests are truly micro, irrespective of packaging.
Traditionally, unit tests -- indeed most tests -- pay very close attention to testing the exact binary image in the exact binary environment in which it runs. Microtests are almost entirely unconcerned with this. Microtests take for granted that they only test that object X works the way it's supposed to *with the assumption* that object X's collaborators work the way they're supposed to.
So you see that although a unit test *could* be a microtest, the word itself has a prior meaning that confuses experienced test-makers who are XP noobs.
What do you do about setting up test data for at system you don't own?
Here was my response:
For repeatability, I try to create tests that can create their own test data from scratch to depend on. This can be a lot of work and yield some large fixture classes. Upsides of this work: you may learn some interesting details of the system and the fixture classes themselves could grow up to become production classes. Downsides: you'll probably make some assumptions/shortcuts in your fixtures that restrict the coverage your tests.
To compensate for that last downside, it's good to also have some rigs (manually executed perhaps) that can load up lots and lots of real data (or copies of real data from production) and run that data through your production code to help flush out gaps in coverage in your automated test suite.
The PragmaticProgrammers have a great analogy for software development which I somehow overlooked (and/or probably forgot) in their similarly titled book. They discuss "software as gardening" in an interview here. An abridged quote:
Software is much more like gardening. You do plan. You put the big plants in the back and short ones in the front. What happens? This plant gets a lot bigger than you thought it would. This big plant in the back died. With a garden, there's a constant assumption of maintenance. We want people to view software as being far more organic, far more malleable, and something that you have to be prepared to interact with to improve all the time.
This interview also references a story of theirs I like to retell, relating to a contradictory law:
As an anecdote, on one project Dave and I were implementing legal requirements in code. These were actual laws that had been passed. We got about three quarters the way through it and realized that the law as stated was wrong. It was inconsistent. If you read through the law one way, it said something had to happen. But in the next paragraph it contradicted itself. Nobody had caught it up to that point, because nobody had tried to do a detailed implementation enacting this legal requirement. This mere mechanical act of coding actually catches bugs in the real world. If you dismiss it as just a mechanical translation, what are you going to do? You find genuine requirements problems while coding. You've got to be able to react and adapt to them.