Wednesday, October 24, 2007

Beyond simple refactorings and IDE portability of projects

I recently did a short presentation at work on some of the java pattern based search and replace features that can be found in IntelliJ (SSR) and Netbeans (jackpot). The nice thing about these two features is that they let you specify a custom refactoring without worrying about whitespace or lines and you can match against more than just the raw text (i.e. - match 'where the base class is xxxx' or 'where the expression evaluates to type String' or even where this statement was preceeded by a statement that created a variable.......etc..').

In relatively short order, I was able to whip up some expressions to do custom refactorings across an entire codebase. Here are some examples:
  • Replace two line declare and return pattern with one-line return
    • Netbeans jackpot: { $stmts$; $type $value = $expr; return $value; } => { $stmts$; return $expr; }
  • Move common trailing statements in if/else blocks outside the if
    • Intellij SSR:
      • Search pattern:
        if($boolean_expression$) {
        $trueStatements$;
        $commonstmts$;
        }
        else {
        $falseStatements$;
        $commonstmts$;
        }
      • Replace pattern:
        if($boolean_expression$) {
        $trueStatements$;
        }
        else {
        $falseStatements$
        }
        $commonstmts$;

  • Improve the expressiveness of code by quickly introducing helper's for certain patterns
    • Netbeans jackpot: {$type $var = $expr; $var.add($valueExpr); $remainingStmts$;} => {$type $var = Arrays.asList($valueExpr); $remainingStmts$;};

The possibilities of what you can search and replace with these pattern/ast based features is really impressive. As I showed some examples, my colleagues were continuously coming up with additional ideas for things we could do with this.

The Intellij SSR is a bit more polished but the jackpot rules language seems like a more flexible idea. Jackpot also lets you access a Java API to do transformations on code, though I think I hold off on learning that until I need to do a refactoring against a code base large enough to justify that time investment.

Intellij and Netbeans are using their AST (Abstract Syntax Tree) to provide this useful
functionality. I can only hope that Eclipse will catch up soon. In the meantime, one of the side-effects of the exercise was that it forced me to exercise some of the new maven integration capabilities of the latest IDE's. I've now got a fairly solid way to open a project in any of the 3 major IDE's (Eclipse, Netbeans, Intellij) without doing any special configuration. Here is the basic idea:
  1. Use a maven pom.xml to define project dependencies.
  2. To Use Eclipse: use the mvn eclipse:eclipse plugin to generate the eclipse project files.
  3. To Use Netbeans: Point Netbeans at the maven pom.
  4. To Use Intellij: Point Intellij at the maven pom.
It's nice to finally have a simple way to open any project in any of the leading IDE's without any manual configuration.

Tuesday, October 02, 2007

links: presentation style

http://www.presentationzen.com/presentationzen/

Saturday, August 04, 2007

Using the powermate with Test Driven Development (TDD)

Just picked up a new (to me) gizmo called the Griffin powermate from newegg.

Lately I've been doing a lot of test driven development at work, and it struck me that I'm having to hit the awkward eclipse key combinations (or opt for a scenic mouse driven route) for running and debugging tests repeatedly. When I was looking for some new gadgets to try with my Macbook Pro, I stumbled on the powermate. Turns out you can bind keystrokes on a per-application basis to the various actions offered by the powermate (press/slam it , press it and hold for a second, turn it right/left, press it and turn right/left).

This turns out to work incredibly well. Right now I've bound 'press' to run unit test. 'Press and hold' runs the unit test in debug. 'turn right' steps over code when I'm debugging. Lots of fun and makes TDD feel a little more physical/visceral.

I have been looking for a java sdk/eclipse integration for the device (with the idea that it might be nice to have the actions be different depending on which eclipse perspective you're in), but haven't found anything yet.

Friday, August 03, 2007

DevX article on some linguistic tools

Rod Coffin and I just published an article on some java linguistic tools at DevX.

http://www.devx.com/semantic/Article/35088

Monday, May 21, 2007

Rails Conf Day 2, 3, 4 (Portland, OR)

Day 2, 3, 4 of Rails Conf notes . . .

Key Note(s)
The keynote started with a mac-like commercial parody from http://www.railsenvy.com

Some interesting stats on Ruby/Rails:
  • RubyConf 2001 had 40 attendees.
  • RubyConf 2007 had 1600 attendees.
  • 600 Rails plugins
  • 10,000 people on rubyonrails (list)
More and more ruby/rails books, in both english and other languages. Mkt. share of books on shelves has surpassed ruby/sql/python.

Tools support is growing: Netbeans, Codegear, Apt___, JetBrains IntelliJ Ruby Plugin

Rails 2.0 improvements were shared.

David Heinemeier Hansson
(one of the Rails authors) showed a Job posting requiring 3 year of Rails experience (which DHH found ironic because he didn't have 3 years).

I missed the much acclaimed evening KeyNote from Ze Frank (of 'The Show') - doh!

Sat. Keynote
Thoughtworks involvment w/Ruby/Rails (Cyndi Mitchell)
  • Lots of new projects in Rails
  • company now offering 24x7 support for Rails/Ruby Dev.
  • Rubyworks new division of Thoughtworks studios focused entirely on Rails.
Sun's involvemnt w/Ruby/Rails (Tim Bray - VP at Sun)
  • Donating hardware
  • JRuby and Netbeans

Full stack testing with Selenium and Rails
Ruby selenium in process, can test through browser, and verify database (open db connection in same process)

Clean Code (Robert Martin of Object Mentor)
Excellent presentation on TDD, incremental development and regularly running tests.
http://www.objectmentor.com/resources/articles/Clean_Code_Args.pdf

Fixtures Friend or Foe - Tom Preston-Werner (rubyisawesome.com) of Chronic fame

Tom introduced FixtureScenarios which appears to address many of the complaints against fixtures in Ruby, and provide a more clear way to show intent when declaring and using fixtures in tests.

Spam I have known - Jim Weirich (author of Rake)

Jim shared several interesting techniques they tried for reducing spam on the Ruby wiki. Things like using reverse dns, roles, a tarpit database, tuning the rejection notice, show spammer their spam, show the world no spam, show google no spam...

MemCaching Rails (Chris Wensworth -- http://errfree.com)

The Dark Art of Developing Plugins (James Adam -- http://lazyatom.com/plugins)

Ruby Tooling State of the Art (Netbeans)
  • Hotkeys that use knowledge of conventions (cool!) i.e. - hot key that switches to the view for the action you are currently editing
  • Runs with native Ruby or JRuby

Data Warehousing with ActiveWarehouse

Great presentation. Mile-a-minute information. I have lots of notes on this one which I 'm choosing not to retype here at the moment.
http://activewarehouse.rubyforge.net

JRuby on Rails (Charles Nutter - Sun)
  • is close to Ruby 1.8
  • is shipping with RSpec, Rake, Ruby
Things I took away....
  1. One JRuby Thread = One System Thread (since java uses native threads)
  2. Easy to call java api's from Ruby code.
  3. Unicode support in Java is better.
  4. Yes, you can use JDBC drivers. (though an adapter has to be written to handle quotes, lack of schema mgmt API [which ActiveRecord does have], and missing features in some DB's)
  5. No native extensions (except those which are already working: mongrel, Hpricot, RMagick, DB Support)
  6. Command-line performance (compared to Ruby in C). Java is generally faster, except on startup, so the CLI tools start a little slower.
  7. Goldspike can generate a war to deploy on a java app server.
  8. Glassfish is available as a Ruy gem.
Open Mic Session
  1. Masterview template/plugin -- a way to keep the ruby code from invading the html
  2. revolutiononrails.blogspot.com
  3. configuration library - reloads any config file changes into class instances on the fly
  4. cruisecontrolrb - instant cruisecontrol for Ruby/rails project
  5. Group discussion tool based on OpenID (http://pibb.com)
  6. devinecaroline - distributed page caching plugin
  7. mywaves.com
  8. The Mole plugin - live monitoring of user activity http://liquidrail.com. Presentator was hilarious.
  9. Bountysource.com showed real time hits on a google map mashup

Testing and expressing intent seemed to be a common theme through a few talks. Some tools mentioned: RSpec, rcov, heckle.

Lots of jazz about Capistrano and Monit.

Local Ruby Conferences to look into: Rubyhoedown.com lonestarrubyconf.com

Lots of good buzz about peepcode screencasts.





Thursday, May 17, 2007

Rails Conf Day 1 (Portland, OR)

Rails Conf Day 1

Portland is a beautiful place to fly into. I've never seen so many trees. The plane flew right by Mt. Hood (a spectacular view).

Today I attended the Rails Guidebook Tutorial session. The hosts (Dave Thomas and Mike Clark) donated their time in exchange for the attendees making a donation to charity. Thus far they have raised around $12,000 for charity. The url to donate (which they'll keep open until the end of the conference) is here

The guidebook session was divided into 2 parts. The first half of the day covered Ruby. The second half covered Rails. Dave made it clear that, logically, it should be called "Rails on Ruby", but Ruby on Rails sounds sexier, so there you go.

Points of interest / things I did not know before today:
  • Innovation happening around the VM (Yarv, JRuby, Rubinius, RubyCLR, Cardinal, IronRuby)
  • Lots of comparisons and references to java. Interestingly, most of the people I have met so far (as of day 1) come from a non-java background (perl/php/etc).
  • Ruby's method of marking a version as 'stable' is not the same as Rails. In Ruby, experimental releases have odd numbering, where more stable releases have even numbering. In Rails, released Gems are more stable, and experimental work is done in the source control trunk ('RailsEdge').
  • Big emphasis on Rails MVC, the benefits of a cohesive MVC framework (which doesn't require lots of xml plumbing) and putting the right code in the right place (business logic vs. presentation logic) when developing Rails applications
  • Watch out for code debt. Basically, because it is easy to create so much code, so quickly in rails, that means it is easy to accumulate design debt quickly too!
  • Scaffolding is considered passe, but a good way to learn REST
  • The crowd seemed interested in some of the finer points of how DB migrations worked.
  • Sam Stephenson (Prototype ajax lib) and Thomas Fuchs (Scriptaculous AJAX lib) are members of the core Rails team.
  • REST is a hot topic at this conference
  • Rails is a single threaded framework. The current approach to scaling (a "share-nothing" architecture) relies on pushing state into a scalable database (and presumably, adding 'worker' servers behind an IP sprayer).
  • When asked what he hoped to see in next years conference, Dave Thomas mentioned wanting to hear integration stories (around vm's, other languages, and pre-existing applications).
  • Most people doing Ruby/Rails like to do it with Textmate.
Personally, I'd like to see some better tooling: IDE, Code Coverage, and Complexity measurement tools. Hopefully I'll found out more about the state of that over the course of the rest of the conference. Looking forward to the BOF's tonight.

Wednesday, May 02, 2007

Continuous Integration And Testing Conference (CITCONF) - Day 2

Disclaimer: These are just my notes -- by no means thorough, guaranteed to be accurate, or containing anything I can take much credit for. Lots of talented individuals participated in group discussions at the conference.

What happened on CITCONF day 2?

I saw the OpenSpaces concept work -- in person -- better than I could have imagined or predicted. I attended sessions with varying levels of focus, but all with passion-filled participants with valuable insights. All the sessions were topics proposed by attendees the prior day and organized into an ad-hoc conference spread across 5 conference rooms and several hours.

Here are my notes (from memory) of a few of the sessions I attended, some of the questions we discussed and where I have a decent recall, some of the points raised in answer to those questions.

Session 1: "100% code coverage + functional tests, what's next?"

I proposed this topic because the team I'm currently on is consistently trying to find ways to push the bar higher. We currently have an extremely high code coverage level (100%), and functional tests driven by a tool called Greenpepper. This session's discussion jumped around between a lot of different areas, such as:

  • How do you know you're testing the right thing?
  1. correlate complex areas of code (based on metrics and empirical data) to areas which need more tests.
  2. record real usage of the application to determine what code is actual hit in production.
  3. use real data from production.
  4. use real data from day 1 of your development (the idea being that real data is never as 'clean' as data you fabricate).
  • Sometimes when people talk purely about 'code coverage' levels, it's a smell.
  • How can tests express the expectations of what they are expected to cover (so coincidental coverage doesn't count)?
  • Sometimes a high hit-count of coverage on a give line of code is a smell.
  • Running test coverage measurements with...
  1. just unit tests
  2. just functional tests
  3. just integration tests.
  • Keeping code performant, pro-actively, re-actively, or predictively?
  1. express expectations up front, code to them, test with unit/functional tests
  2. pro-actively measure performance deviations over time (any known tools? nope)
  3. optimize after the fact -- when you need to.
  4. measuring performance with and without tests.

Session 2 - "What is the one true language for writing tests?"

The language of the testers? The language of the developers? A cross-over language?

The group brainstormed qualities we would look for in a testing language, and came up with a decent list which I thought looked a lot like a list of qualities I'd look for in a regular programming language. There was some brief mention of the possibilities of JetBrains Meta Programming System, and domain specific languages, Ruby and what it brings to testing, etc. The discussion then meandered into how to get teams to adopt best testing practices.

Session 3 - What is the future of build languages?
  • What's wrong with ANT?
  • Why has ANT been so successful?
  • Rake?
  • What about Maven?
  • Procedural vs. Declarative? Specifying the how vs. what you want and what it needs.
Overall notes from the conference
  • Several attendees with an impressive amount of experience (authors of build tools, well known book authors, authors of open source testing projects, and experienced technologists).
  • Lots of 'Ruby, Ruby, Ruby'.
  • I found the smaller sessions more engaging.

Saturday, April 28, 2007

Continuous Integration And Testing Conference (CITCONF) - Day 1

I just finished Day 1 of CITCONF here in DFW. It's an openspace style conference focused on the area of continuous integration and testing. Openspace means that all the attendees determine the topics of discussion.

The surprises:
  • For something that is not a 'vendor based' conference, Citconf seems incredibly well sponsored (food, drinks, swag, etc)
  • There are some very experienced people attending. It will be interesting to see where the sessions take us.
The organizers have one rule for the conference. It's called "The rule of two feet". Basically, the attendees are responsible for making sure that they are either contributing or receiving value at the sessions they attend. Therefore, if one of those isn't happening, anyone can feel free to get up, walk out, and find something that will be more valuable. I'm excited to see how this rule plays out in practice.

Sunday, March 18, 2007

Mac links

Having a mac and exploring all the new software/customizations one can install on OS X is like being a kid in a candystore. Here's what I've collected over the last month and half

http://netsmith.wikispaces.com/Mac+links

Wednesday, February 28, 2007

Attaching and modifying a running java program

Java 6...

http://www.fasterj.com/articles/hotpatch1.shtml

This article brings to mind the quote "with great power comes great responsibility".

Sunday, February 11, 2007

The Paradox of Choice, Domain Driven Design, Mac OS X, and where have I been?

Long time no post!

What are my lame excuses for not having posted in a while?
  1. I've switched out of the consulting realm and taken an exciting opportunity working for a small software product company in Dallas. I'm really excited about the chance to make an impact starting at the ground floor. We're building a really exciting team, and setting the bar as high as we can reach.
  2. I've "switched" to OS X (as part of the new job) in the form of a macbook pro 17"!. It took a few days to adapt to some of the metaphors in OS X, and I imagine I still have a quite a bit to learn, but I'm very happy with the capabilities so far and some of the extra keys (like the apple/command key) are actually starting to feel intuitive.
  3. I just finished reading a few books about DDD (added to the booklist). I believe this is the next wave of concepts that will aid in managing of complexity inherent in developing and maintaining software. The concepts and patterns of DDD address some problems you are likely to be very familiar with if you have a been on your share of software development initiatives. There are very few books on the subject at the moment, and the patterns and techniques of the approach are deep and well-thought out. Don't confuse this with 'domain modeling'. This is much deeper beast. The techniques appear complementary to Agile principles. The books I've had time to read are:
    1. Domain Driven Design by Eric Evans
    2. Applying Domain-Driven Design and Patterns (in C#) by Jimmy Nilsson
  4. Meanwhile, I postponed reading the following book, but I did find a google video presentation from the author of "The Paradox of Choice" at http://video.google.com/videoplay?docid=6127548813950043200&q=Google+Tech+Talks&hl=en
  5. Studying....ahem... patterns of immersion has been keeping me busy as well. Let's just say that Wii got very distracted over the holidays and WoW some games are really a time sink (level 38 and counting) :)

Tuesday, December 26, 2006

Lightning Talks

There is an interesting [new to me] phenomena which seems to be spreading through a few technical conferences. It's called the Lightning Talk (i.e. - Google Test Automation conference lightning talks).

The idea is that the conference provides a series of sequential 5 minute presentation slots with only one strict requirement: the talk must be 5 minutes or less. This has some interesting effects . . .

For the speaker . . .

  • It creates a sense of urgency because they must make their point(s) quickly

  • It lowers the barrier to entry for seasoned and newbie speakers alike - most 5 minute talks can be prepared more easily than 1 hour talks. It's less intimidating to prepare for a 5 minute presentation.

  • New types of talkes can be considered - A speaker can talk about some things that are more appropriately sized to a 5 minute presentation (a new idea you're just beginning to explore, a novel approach they need some feedback on, a quirky language behavior discovery or a rant on why they think something is a good/bad idea).



For the audience . . .

  • The speaker must really focus on getting their point across quickly.

  • If the speaker is boring or the subject is not as interesting as anticipated, there is an easy-out: just wait 5 minutes.

  • It gives you quick exposure to variety of speakers and topics in a short time frame.



I suspect a portion of the popularity of this idea essentially a 'market reaction' (a) getting stuck in long, boring, [often slide-driven] presentations (see [here]) and (b) people wanting to stay aware of the shear abundance of interesting topics and techniques being discussed.

Supplementing a conference with a one or more sessions of 'Lightning Talks' seems to be a win-win for the speaker and the audience.

The open space-based conferences have also been gaining some traction in technical communities. This looks like a fun one to be sure. Now if I could just figure out how a half day of ski-ing will help improve my [software development] agility . . .

Tuesday, December 19, 2006

Public committment, the neocortex, and metrics in software development

I've been thinking a lot lately about how sweeping organizational changes can be effectively introduced and sustained. Blue Ocean strategy introduced me to the concept of tipping point leadership but I was left asking myself: "How do you make it last?". Then I stumbled on this video of Kent Beck talking about "Ease at work". Deep in this presentation I pulled out this useful tidbit:

Ken suggests the following as requirements for sustaining change:

  1. Make a Public Committment - to one or more people other than yourself

  2. Make sure there is Accountability - the one or more persons you make a committment with, schedule a date/time to check back with you on progress.



I think using these requirements works incredibly well on an individual level, and to a lesser extent on a group level (i.e.- timeboxed iterations, sprints). That said, it's better to have something (committment wise) on a group level rather than nothing!

Also useful to consider here is the concept of fishbowl management -- that is to say: Put all your leaders in a fishbowl and begin measuring them to the same, publicly visible standards. You then reward and punish [demote] based on people's performance relative to those standards.

I also pulled out some names for behaviors I've seen before including the "organizationally addictive hero-matyr pendulum" and the well-named game of "Schedule Chicken".

"Schedule Chicken" is when everyone around the table knows the project will slip or that some piece of functionality can't be done in the necessary amount of time, but nobody is willing to say anything . . . the first one to say something loses the game, but in the end everybody is relieved when it's out in the open (unless there's someone at the table who wasn't playing).


On a side note, I found a podcast interview with Dileep George on some of his work constructing software to emulate the neocortex.
http://www.itconversations.com/shows/detail732.html
http://www.itconversations.com/audio/download/itconversations-732.mp3

Dileep's homepage of interesting stuff: http://www.stanford.edu/~dil/invariance/

If he and Hawkins can pull this off, it will certainly be the beginnning of a revolution in what types of problems we can solve with software.

Lastly, Valtech TV has posted some interesting presentations. I just watched "The Great metrics Debate with John and Amr - Part 1" (Part 2), and found it to be a well thought out illustration of good and bad points to various metrics we tend to push in Agile and what to watch out for when using those metrics.

Monday, December 18, 2006

Ebay Architecture Presentation slides

Great set of slides (wish I had the audio!) on how eBay scaled their architecture to where it's at today.
 

Tuesday, December 05, 2006

Alistair Cockburn - Redefining Software Engineering

I listened to a decent podcast with Alistair Cockburn today.

http://www.itconversations.com/shows/detail1682.html
http://www.itconversations.com/audio/download/itconversations-1682.mp3


It covers some interesting subjects like:

  • What is software engineering?

  • How can new agile methodologies be effectively applied to people with
    various personality types? (particularly people oriented towards stability)

  • Can agile work in china and japan?

  • How are people doing distributed development most successfully?

Monday, December 04, 2006

Couple of new wiki pages....

Posted a quick book review at: http://netsmith.wikispaces.com/Blue+Ocean+Strategy+-+W.+Chan+Kim%2C+Renee+Mauborgne

and started a 'metrics' brainstorm page at : http://netsmith.wikispaces.com/Metrics

Also, found out that Dave Nicolette has a nice blog at: http://www.davenicolette.net/agile/

Thursday, November 30, 2006

OPML and Grazr!

I migrated the feed/rss/podcast list on the right side of the blog to use a neat new ajax widget (http://grazr.com) which is backed by an opml (aggregation of rss/links) file. Scroll down a bit and check it out!

I chose http://www.opmlmanager.com for my opml hosting. That allows me to store the raw opml on the web here: http://www.opmlmanager.com/opml/mpstx.opml. Then just point the Grazr widget generator at it, and wah la! An ajax opml/rss browser embedded in blogger blog.

Great Talk from Ken Schwaber - Father of Scrum

Check out this great talk from Ken Schwaber (father of scrum)






Some things I took away from the talk:

  1. Concept of a 'Design dead product' -- that is to say the gradual cutting of code quality over time (to satisfy deadlines) which backs a a team into a corner of design debt and steadily slows achievable velocity per iteration/sprint.
  2. "Scrum master also known as 'the prick'"
  3. QA is apparently a great source of people who can be an effective scrum master -- because they know what a true 'done' is ...

I picked this up from reading Xavier Warzee's blog.


It looks like Google has speakers in regularly and publishes them as Google Tech Talks on Google Video. These are worth keeping an eye on if they can get speakers of this quality on a regular basis.


Wednesday, November 29, 2006

Scrum and XP from the trenches

90-page document on one person's experience implementing scrum/xp with 40 people -- what worked and what didn't.

http://www.crisp.se/henrik.kniberg/ScrumAndXpFromTheTrenches.pdf

Monday, November 13, 2006

Saccades and the like

Entry on new research regarding saccades, and more importantly a statistic on how often people blink (15 times a minute). Fifteen times a second seems like an excessive amount of times to pop-up a reminder to remind someone staring at a computer screen to blink. I wonder how I can do that effectively in http://netsmith.wikispaces.com/Blinky . . .


http://blogs.zdnet.com/emergingtech/index.php?p=407