So far we’ve established a few things from the last post on Jedi Software Training:

• We already implicitly and perhaps accidentally practice the Apprentice/Master model in software development today.
• The route to Software Master is paved with ambiguity.  Most who achieve it can’t tell you how they got there.
• A Software Apprentice often flails around in a sea of information without much guidance unless they are insanely lucky in their first job.
• We need a systematic way to train apprentices and grow the craft in a reproducible manner.

Someone at this point might be saying, “Don’t we already have internships?  Isn’t that enough?”  Let’s distinguish an internship from an apprenticeship.

The important difference here is about goals and time frame. The internship is about checking things out and deciding interest, the apprenticeship is about getting serious and learning the craft from start to finish with the intent of employment.  So no, the internship doesn’t address the need in the same manner.

To create successful apprentices, we need several things:

• A road map to understand the most important aspects of the craft
• A master who already understands the map
• A place where the master practices and can freely train the apprentice

Who, what and where.  The how is being addressed here, and the why was already established in the last post.  Let’s focus on who now.

The Ideal Master

Taking a page from the book of Star Wars:  To train a Jedi Apprentice, we first must find a Master.  The master clearly must know software engineering inside and out.  They should be versed in every aspect of project development from soup to nuts:  analysis, requirements, design, development, testing, release and maintenance.  Ideally, they should have practiced for 15+ years and on a wide variety of projects–from small to large, from internal to highly public, and from successful to dismal failure.

A master should be fluent in many tools, languages and frameworks.  A good master is productive no matter what tool they use once they practice for a short time, given their vast prior experience on other tools.

A master must be able to communicate difficult concepts with simple and easy-to-understand analogies or other descriptions that even the newest apprentice can grasp.  True mastery of material means the ability to describe it in multiple ways.  The master should have infinite patience to describe things in various ways to give the apprentice many different ways to understand new concepts.

Most importantly, a master must be fluent in the business of software, since many software decisions are driven through the business, not the craft of software itself.  A master that cannot understand the balance of these two forces is no master at all.

The Path to Mastery

To turn an apprentice into a journeyman, we need a training program.  Electrical apprentices currently mix actual practice on the construction site with nighttime course work in electrical theory, such as the study of Ohm’s Law.  A similar program for software could be useful:

• How to use the tools of the trade (e.g. IDEs, makefiles, testing scripts, automated build & testing tools, source control, etc)
• Understanding the theoretical side of software engineering:  requirements analysis, functional decomposition, design practice, project scheduling, developer estimation, and so on.
• Putting these theories into practice using a small scale, but real, project

Some of you may be thinking that this sounds precisely like a university program for computer science (at least in part).  Most computer science curricula are geared toward creating academics who are skilled in the field of computer science.  The fact that software engineers might pop out on the other end of the coursework is an afterthought.  It’s the same distinction between physics majors and mechanical engineers:  science vs. applied knowledge to practical situations.  Clearly we want the practical here.

The Master’s Workshop

There seems to be no perfect environment in which to train the apprentice in a modern setting.  This, to me, is the crux of the problem.  There are several potential possibilities, but each fraught with peril.  Let’s examine each of them individually:

• Universities
• Open source projects
• Companies of various sizes

The University Setting

The first and most obvious is university setting.  But universities already have problems as a training ground for three major reasons:

• Most professors are professional academics without the benefit of real-world software engineering experience in a business setting.  This makes them less than ideal Masters, even though they may be quite accomplished in their own right.
• Their goal as academics is to create more academics, not to create masters in the field.  Generally, they prefer the theory to the practical aspects.
• The end result of a professor’s research is a published paper, not a finished software product.  Often time, prototypes are more than enough for them.  Prototypes are not good examples of production grade software.

Dude, what about Summer of Code?

An excellent question:  What about Open Source projects such as  Google’s Summer of Code or the Apache Project?  There’s an unlimited number of them, creating limitless opportunity for apprentices.  They sure seem like a great place to start, but:

• Most of these projects are working remotely, using distributed development techniques.  A master/apprentice relationship functions best when the master and apprentice are close in proximity to keep the apprentice from getting too far into the weeds.  SoC and other such setups do not offer that proximity and many apprentices are generally unprepared for that level of freedom, at least initially.
• An apprentice is trying to develop confidence in their skills.  These projects often assume some self-taught skill with the tools involved.  Newcomers are often ridiculed and derided for lack of understanding of even basic tools.  That doesn’t foster confidence in apprentices.
• Some are treated more like internships (e.g. Google Summer of Code).

Be A Company (Wo)man!

That leaves only one place left as fertile ground for training:  technology companies.  But which ones–large, medium, small, or startup?  Let’s make a case for each.

Large companies (more than a few thousand employees) seem to be a good choice:  well funded, plenty of good facilities, and larger numbers of job openings per year.  Yet I don’t think this setting would enable the craft of software to flourish and expand for several key reasons:

• Large companies tend to hire large development groups.  Inside these large groups, mediocre and completely untalented engineers can reside undisturbed for years because of the inertia of large organizations to get rid of them.  This unseemly presence of incompetence could give the apprentice the wrong impressions of development techniques if left unchecked.
• Large companies tend to foster painful bureaucracies that would stifle the creation or gestation of such apprenticeship programs.
• CEOs and technical managers of such companies are highly focused on budgets and value-add for every person in the company.  Apprentices are the very definition of unproductive learners, taking up substantial resources up front in order to become productive.  Managers would have to place personal reputations on the line in order to start and maintain such programs in the face of budgetary pressure.
• Large companies have very siloed groups, sometimes with company-specific processes, tools or standards, that would expose an apprentice to a very narrow view of development practices, making them less marketable to the broader economy as software engineers.

Medium size companies (say between a few hundred and a few thousand employees) would face some of the same pressures as large ones, but the right company with the foresight to create such a program might be successful.  It comes down to the courage of the management and the strength of will of the development team to make it happen in the face of current company policy.  However, in this current economic climate of 2009-2010, few companies would take such a chance, in my opinion.

I think the biggest bang for the buck will come from small companies (less than 200 employees, but more than a dozen) for several reasons:

• Small companies because of their size must attract the best talent in their development teams.  Small companies don’t have time or money to waste with unproductive people. But they are usually willing to develop good potential talent because of the value proposition.
  
• Small companies tend to be nimble in their thinking and can change their internal culture to support such a movement the easiest of the 3 described so far.
• A smaller company can use an apprentice more easily across groups and teams, making more efficient use of the apprentice’s time and energy, and exposing the apprentice to more of the business of software.

But it’s not all sunshine and roses either:

• Small companies fight tooth and nail for whatever money they get.  Budgets for developers are tough.  Add an apprentice to this and it gets even tougher.
• Many small company developers are often over tasked in their current roles, making it harder for them to act in the role of a master developer.  A part-time master is a difficult and potentially inconsistent kind of teacher.

Finally, we’re left with startups (less than 20 employees).  While they have some of the small company advantages such as agility and talent attraction, I believe that the chaotic environment of startups is best left to more experienced engineers:  journeymen or at the very least, previously trained apprentices.  Startups rarely have the time to spend training new people in techniques beyond those that implement the vision of the founders.  And what meager monetary resources exist in small companies, even fewer are available with startups.

In short, the apprenticeship idea requires tremendous courage on the part of the company that fosters it.  Small companies are used to taking big risks with substantial payouts in the future.  I believe small companies are the ideal place to grow this notion and allow it to root.

The Bottom Line

Creating an apprentice program would require jumping a number of substantial obstacles, not the least of which have been enumerated here.  But the company that creates such an environment would become a magnet for the latest talent in the industry because so few places offer the right opportunities to train new graduates into real software engineers through a systematic approach.

Related posts:

1. Jedi Software Training–Part 1

When we launched the new and improved Gorilla Logic website, we decided to bring all our open source projects together under one roof. In order to migrate all things FlexMonkey back to our website, we need to get our forum data migrated out of Google Groups. Alas, Google doesn’t provide any way to export data from Google Groups. The only way to preserve the amazing contributions from the FlexMonkey community was to scrape Google Groups. So that’s just what we did.

With a very minimal amount of PHP, I was able to walk the entire FlexMonkey Google Group, scrap all the topics (aka threads) and all the posts inside each thread. The first step was to build a generic scraper class that grabs an html page (using cURL) and parses out all unique outbound links.

Here’s the code for the Scraper class:

class Scraper {
    private $url = '';
    public $html = '';
    public $links = array();
 
    public function __construct($url) {
        $this->url = $url;
    }
 
    public function run() {
        $this->html = '';
        $this->links = array();
 
        //scrape url & store html
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $this->url);
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $this->html = curl_exec($ch);
        curl_close($ch);
 
         //parse html for all links
        $matches = array();
        preg_match_all('#<a.*?href\s*=\s*"(.*?)".*?>(.*?)</a>#i', $this->html, $matches);
 
        if ($matches !== false && count($matches) == 3) {
            for ($i = 0; $i < count($matches[1]); $i++) {
                $href = $matches[1][$i];
                $val = $matches[2][$i];
 
                //unique links
                if (!array_key_exists($href, $this->links)) {
                    $this->links[$href] = $val;
                }
            }
        }
    }
}

In the run() method, cURL is used to grab the html. Next, a regular expression is used to match all outbound links. The links are are stored in a hash, while making sure they point to unique urls.

Built on top of the generic Scraper class is a specialized Google Groups scraper class, aptly named GoogleGroupsScraper. For a given Google Group, the url of the main page (containing a list of most recent topics) is:

http://groups.google.com/group/[GROUP]/topics

And the url of a single topic (aka thread) is:

http://groups.google.com/group/[GROUP]/browse_thread/thread/[THEAD_ID]#

Where [GROUP] is the name of the Google Group, and [THREAD_ID] is some alphanumeric id. Most importantly, at the bottom of the main page is an Older » link that points to the next page of topics. The GoogleGroupsScraper exploits this to spider the entire group, recording topic title and topic url as it walks each page.

Next, each individual topic page is scraped by the GoogleGroupsTopicScraper class and parsed into a list of posts with author name, date, timestamp, etc. The topic scraper uses various regular expressions to extract and massage the html to extract the different parts of each post. In particular, the post body needs a lots of work to strip out any Google Groups specific links and code.

Lastly, the topics and their posts are assembled into an XML document with a nice big CDATA block around the post body to preserve the html content.

Here’s some sample output from the scraper:

<?xml version="1.0" encoding="UTF-8"?>
<scrape group="flexmonkey">
  <topic>
    <title>FlexMonkey User Group is now located at www.gorillalogic.com/flexmonkey!</title>
    <link>http://groups.google.com/group/flexmonkey/browse_thread/thread/fe9ed66bf56db88e#</link>
    <posts>
      <post idx="0">
        <author>Stu</author>
        <email>stu.st...@gorillalogic.com</email>
        <date>February 10, 2010 21:17:52 UTC</date>
        <timestamp>1265836672</timestamp>
        <body>
<![CDATA[
<p>People of FlexMonkey, <p>We have migrated the FlexMonkey discussion forum to <a href="http://www.gorillalogic.com/flexmonkey">http://www.gorillalogic.com/flexmonkey</a>. Please note that you will need to re-subscribe to the new forum to continue receiving FlexMonkey discussion messages. <p>-Stu <br>
]]>
        </body>
      </post>
    </posts>
  </topic>
  <topic>
    <title>Record button clicks based on Ids instead of names?</title>
    <link>http://groups.google.com/group/flexmonkey/browse_thread/thread/4f079b1959374f53#</link>
    <posts>
      <post idx="0">
        <author>Shilpa</author>
        <email>shilpa.g...@gmail.com</email>
        <date>February 9, 2010 23:44:44 UTC</date>
        <timestamp>1265759084</timestamp>
        <body>...</body>
      </post>
      <post idx="1">
        <author>Shilpa</author>
        <email>shilpa.g...@gmail.com</email>
        <date>February 10, 2010 00:05:44 UTC</date>
        <timestamp>1265760344</timestamp>
        <body>...</body>
      </post>
      <post idx="2">
        <author>Gokuldas K Pillai</author>
        <email>gokul...@gmail.com</email>
        <date>February 10, 2010 00:16:34 UTC</date>
        <timestamp>1265760994</timestamp>
        <body>...</body>
      </post>
      <post idx="3">
        <author>Shilpa</author>
        <email>shilpa.g...@gmail.com</email>
        <date>February 10, 2010 01:18:42 UTC</date>
        <timestamp>1265764722</timestamp>
        <body>...</body>
      </post>
...

Finally, there is a very simple PHP driver for the scraper that runs the scraping process:

require_once('GoogleGroupsScraper.class.php');
 
$scraper = new GoogleGroupsScraper('[GROUP]');
$scraper->run();
 
print $scraper->getXML();

And you run it as usual:

php scrape.php > output.xml

Just enter the name of the Google Group you wish to scrap, and away you go. Here are a couple of notes to help you along:

1. [GROUP] is the group name as it appears in the url, so no spaces, etc.
2. It’s not fast, so be patient, or modify the scraper code to generate some intermediate output.
3. Via a browser, Google Group displays 30 topics per page, but via PHP & cURL you only get 10. Probably some Cookie or User Agent magic going on.
4. Not much error handling. The error handling that exists isn’t very good. It will break.
5. Good luck!

Please download the code and use it however you wish. Hopefully, putting the code online and writing this post will save someone else some time when migrating data off Google Groups.

Files

• GoogleGroupsScraper.tgz

Seriously, has the Mozilla team run out of important things to work on in Firefox?

My Firefox browser updated to 3.6.2 today and I’m greeted with this page, asking me to try their new personas:

I’m not really a customize-my-browser-to-look-like-a-teenage-girls-Twitter-page kind of guy, but I thought I’d give it a shot and see what happened.

In a word:  horrific.

My browser’s link bar went from easily readable to complete obscured:

And not just with one, but pretty much ALL of their “recommended ones”.

WTF Mozilla?  Have we decided to throw out 30-odd years of user interface practices in favor of ponies, rainbows and unicorns? Really?

There are 30,000 MORE of these monstrosities to choose from!  Keep in mind that Mozilla would, of course, showcase the best and most interesting on their update page…But if these are the best, I’m frightened to dig any deeper.

This reminds me of the skinning snafu of the early 21st century where every damn audio application (WinAmp for example, but certainly not limited to them) had to come out with 368 cool skins to go with their app.

Not only did you have to learn an entirely new interface with the application’s custom look-and-feel, but you often had to relearn it for each damned skin you switched to.  The same was true of Linux usability.  Reminds me of a quote:

Whenever a programmer thinks, “Hey, skins, what a cool idea!”, their computer’s speakers should create some sort of c*ck-shaped soundwave and plunge it repeatedly through their skulls.

This is a usability nightmare**. No, wait, usability nightmare doesn’t even begin to cover it.  And now Mozilla wants to do that with my browser?  As if MySpace pages didn’t make the web awful enough…

For the love of all that is good and easy to read in the world, stop.  Just please stop.  Tell me where to send the money to make it stop.

My eyes are still bleeding.

UPDATE (part 1):  Looking to REMOVE the personas?  Do this:

Go to Tools -> Add-ons ->Themes Panel.  Click on Uninstall on the persona.  Then restart Firefox.

UPDATE (part 2):  Since I have the (un)fortunate Page 1 Google ranking in “firefox personas”, and the comments seem to fall into 3 categories in rough order:

1. Dude, you’re a grouch.
2. Dude, you’re an idiot AND a grouch.
3. Dude, I totally agree with you.

I realize I fell prey to a classic issue often happened in math class too…I skipped straight to the answer and failed to show my work.

** When I say “usability nightmare”, what I mean (hyperbole aside) is that personas violate some well-known principles of web usability all posited by Jakob Nielsen, the guru of web site usability.  He doesn’t just guess on these things, he actually researches them, observes behavior and reports results.

Which principles?  Well, I can probably dig up a dozen if I try hard, but without going too crazy, here’s a short list of the ones that FF personas pretty much violate right out of the box:

• Distract users on eyetracking for important content.
• Violating web-wide conventions
• Vaporous content and empty hype
• Custom GUI widgets
• Ignoring aesthetic and minimalist design

Usability matters.  And grouchiness aside, the more we infect these kind of eye-Twinkies (think:  eye-candy, but far less nutritious) on people, the less capable they are of actually using the web in the first place.

No related posts.

Software engineering is perhaps the youngest of all the engineering disciplines (and some would even argue, we don’t practice an engineering discipline at all).  But like all disciplines, an engineer must be trained in order to achieve a level of competence to practice their craft with any proficiency.

The Fresh Out Of College Problem

Software engineers run the gamut–amazing, mediocre, and step-away-from-that-IDE-before-you-hurt-someone levels.  Some are naturals and some probably will never be able to program their way out of a wet paper bag.  In terms of how we get trained, it’s all very informal in most cases.  Universities rarely focus on any software engineering courses as a serious part of a Computer Science program.  Indeed in my own Alma mater, Computer Science was part of the Engineering School but did not constitute an official engineering discipline.  The focus is entirely on data structures, programming, algorithms, and nuts-and-bolts sorts of topics.

Don’t get me wrong–those topics are incredibly important, but almost every college graduate I’ve ever worked with directly out of school (within the first 3 years of their graduation) generally has poor knowledge of how to run multi-engineer, moderate-size software projects.  I’m talking about simple things that are bread and butter for a successful complex project: source code control, project module management, requirements definition, functional design, and documentation.  This seems like a huge gap in training, considering that everyone knows these things are absolutely essential when they get out of school.  Employers are frustrated because graduates are unable to take the reins of a software project in a meaningful way without starting out in (very) junior positions that are rarely available and only modestly tolerated by most companies.  Graduates are frustrated at their lack of opportunities because they don’t have enough ‘real world’ experience to qualify them for real software development positions.

The question is, why are we training our engineers so sloppily today?

The Apprentice Model

Engineering training and Star Wars Jedi Knights have something in common:  they both start out as apprentices.  Let’s look at the history of this practice and see how it can apply today.

Back in the Middle Ages, if you wanted to practice a craft, such as blacksmithing, baking, masonry, or butchery, you needed to become an apprentice to a master craftsman.  That master had been practicing for years, had a demonstrated level of mastery, and often belonged to a guild of other masters who judged this master to be fit enough to earn that title of master in the first place.

Apprentices lived a hard life back then.  They were exploited as cheap labor for the master for a long period of time (5-7 years was typical).  The master provided tutelage in the craft, food, and lodging in exchange for their work.  After this period of time, the apprentice was shown the door and expected to fend for themselves as a newly minted journeyman.  Journeymen could either practice in solitude or work under a willing master (if available).  After some long period of time, the journeyman could try to produce a masterpiece of his or her craft in an attempt to demonstrate mastery, join a guild and attain the rank of master.

Some guilds were comically harsh in the training methods.  For example, up until 1791 in France an apprentice worked under a master for a long period of time through their journeyman rank.  If they failed to produce a masterpiece during their journeyman period, they were subsequently executed.  I, for one, am glad we don’t train people in this manner anymore.  The labor shortages this could create aside, the pressure to produce would be excruciating.  And in truth, not everyone is cut out to be a master.

Modern Apprentices

Apprenticeships are still used as a model for vocational training in many professions in a number of countries (in America, we still use this for plumbers, electricians, and carpenters among many other trades.  Across Europe, this is true as well), this training serves to bring new workers into the trade, give them baseline skills in a controlled environment under a specific instructor, and a route to advancement in the field.  But this formalization happened over years when best practices were easily formulated into codified documents and classroom formats, allowing the apprentice to reproduce a master’s work in a repetitive and formulaic manner.

If we characterize the differences between the three levels, you might see the following:

• An apprentice can take a specific set of instructions, created by journeymen or masters, and faithfully reproduce the steps to create a result of lesser quality than a master or journeyman.  An apprentice is unable to work without a concrete framework of rules to abide by while practicing the craft.  Apprentices generally need constant guidance and intervention from others in order to complete a project successfully.
• A journeyman is an apprentice who has completed their training period and achieved a baseline level of skill in the trade.  They are competent enough to work alone, but most often seek the continued education under a master to improve their skills.  Journeymen are comfortable in a wide variety of techniques as taught by the master, and many of these techniques are now second nature to the journeyman.  However, journeymen lack the ability to create new skills or perform the baseline skills in the effortless manner of a master.  Journeymen generally can do a skill, but have trouble expressing the exact reasons why one skill is preferable over another in a certain context outside of the rules given by his or her master.
• The master is the culmination of years of practice into “effortless skill”.  The master’s abilities require no conscious thought to manifest.  Masters have the ability to see patterns in projects and skills, arbitrarily combining things into new and unique ways of using them, often pioneering new techniques as a result of this artful mixing.  Masters understand the rules to the point where they can break them at will, knowing which limits are completely arbitrary.  A truly excellent master has the ability to teach their skills to apprentices in a way that creates excellent apprentices.  These masters are exceedingly rare.

The apprenticeship method is a time-tested, battle-worn method for bringing people of low-to-no skill into a field and training them to a level of success. Dozens of fields practice this, and almost all formal engineering disciplines have a similar model (engineer-in-training and P.E. (professional engineer) certification).  Academics have similar models to produce scholars, with undergrads promoted to graduate students, then post-doctorates and finally becoming associate professors, full professors, and professor emeritus.

My question is:  Why don’t we do this with software engineers?

Apprentice, Journeyman, Master – The Path Less Traveled

The path from apprentice to master must be reproducible and documented through time in a formal manner by previous masters.  Developers today are largely left to hack out their own destinies at random with high variability in the results.  If they inadvertently become masters, they attribute it to their own skills rather than the lucky circumstances they managed to find themselves in early on in their careers.

Transferring that mastery becomes incredibly difficult when a master’s path lacks any formal system to follow along the way.  Software masters are revered for their superhuman abilities to swoop in to save the day on a doomed project through superhuman coding effort via nights, weekends, major refactoring on a level that ultimately undermines the morale of the rest of the team.  This “cult of the software hero” approach ultimately shrouds the master’s path rather than illuminates it for others to follow.

We know this model is still at work in software today, albeit informally.  We see the various levels at work in our own teams and workplaces.  Here’s a simple OO experience hierarchy that you probably can relate to:

• An apprentice OO programmer will struggle with encapsulation and polymorphism, but can put together systems that have been well-specified.  They need highly detailed object designs to see the various interactions of a system come together with elegance.  Left without such guidance, they will inevitably create God Objects, Yo-yo Object Hierarchies, and other monstrosities.  Design patterns are something of a mystery to them.  Simple algorithms will require large amounts of effort to understand and encode into a language.  Their skills with the tools of the trade will be low to moderate (e.g. IDEs, source code control, automated build systems, bug tracking) and significant effort will be expended in using/learning them.  Design and architecture are generally beyond them.
• A journeyman OO developer will understand the value of data hiding and be able to participate in interface designs for obvious parts of the system.  They are comfortable with the tools of their trade and know many time-saving shortcuts, along with several flavors of tools.  Journeymen are typified by rigid adherence to systems, tools, frameworks or architectures because of their comfort level with them (“The Golden Hammer” effect).  They will recognize some design patterns and have regularly applied them, but may not know all cases when they are applicable, or apply new ones with ease.  Journeymen dabble in architecture with increasing skill and ease, but complex problems are often met with complex, obscure solutions when designed by journeymen.
• Master OO engineers can hear domain problems and suddenly see object models dancing in their heads before the description is finished.  Architecture is second nature and design patterns require little to no effort to apply or recall.  A master OO developer generally tends towards tool, language, and framework agnosticism because they understand their relative weaknesses and strengths, choosing only those tools that stay out of their way, make the most sense for the domain and allow for ease of creation.  A master’s trademark is the ability to create a simple and elegant solution to a complex problem.

So the $64,000 question becomes:  How do we create an apprenticeship model in software development that can succeed in cultivating better engineers? We’ll start to answer that in Part 2.

Related posts:

1. Jedi Software Training–Part 2

Event-driven programming is at the heart of Flex. It also lies close to the heart of insanity for the developer. When an application becomes too loosely coupled, things get painful fast. The perfect storm arrives when you marry an event-driven UI with an event-driven asynchronous backend. Better async testing becomes the least of your problems, and you are constantly trying to figure out who is listening to what, when.

In these cases, I often find that some simple runtime management of event listeners can really help. To begin, I start with some component whose listeners I want to manage, then I override the addEventListener() and removeEventListener() functions in that component to always keep track of who is actively listening.

Here is the basic code that can be added to any component (actually any descendent of EventDispatcher which is basically everything in Flex):

private var _listeners:Object = {};
 
override public function addEventListener(type:String, listener:Function, useCapture:Boolean=false, priority:int=0, useWeakReference:Boolean=false):void {
    super.addEventListener(type, listener, useCapture, priority, useWeakReference);
 
    if (_listeners.hasOwnProperty(type)) {
        (_listeners[type] as Array).push(listener);
    } else {
        _listeners[type] = [listener];
    }
}
 
override public function removeEventListener(type:String, listener:Function, useCapture:Boolean=false):void {
    super.removeEventListener(type, listener, useCapture);
 
    if (_listeners.hasOwnProperty(type)) {
        var listeners:Array = _listeners[type] as Array;
        if (listeners.length <= 1) {
            delete _listeners[type];
        } else {
            var idx:int = listeners.indexOf(listener);
            if (idx != -1) {
                listeners.splice(idx,1);
            }
        }
    }
}

We just store all listeners of a given event type in an array, and store the array as the values in a hash keyed by event type. When a new listener is added via addEventListener(), we check if any listeners of that event type exist. If yes, we push the new listener onto the array. If no, we create a new one-item array containing just the new listener. The tracking process for removeEventListener() is just the opposite.

Once we have all the listener bookkeeping in place, we can add any type of event listener management that we want. For example, he’s a removeAllEventListeners() function:

public function removeAllEventListeners(type:String = ''):void {
    if (type.length == 0) {
        for (var t:String in _listeners) {
            removeAllEventListeners(t);
        }
    } else if (_listeners.hasOwnProperty(type)) {
        if (hasEventListener(type)) {
            for each (var listener:Function in _listeners[type]) {
                removeEventListener(type, listener);
            }
        }
    }
}

Nothing cosmic, as my boss likes to say, just walk all the listeners of a given event type and remove them. If the event type is blank, just use some dirty recursion to walk all the types to remove everything.

Here is a sample application which is tracking all the listeners on a Panel component (view source enabled):

You can add/remove a custom listener for the foo custom event type using the buttons. Or hit remove all to clear everything (including the listeners attached by default by the Flex framework).

Files

• ListenerTracker (download)

Dzone.com / JavaLobby has published my ‘A Deep Dive into Flash Builder 4‘ article.  I wrote it over Christmas on the beta 2 release of Flash Builder, so hopefully everything is still up-to-date now that Flash Builder is finally out for production.

Give it a read and let me know if you have any feedback.

I'm fairly fearless when coding, which means that about once a week, I delete a huge chunk of something I should've kept, or change something into something unrecognizable, thereby inadvertently breaking a dozen unit tests. When I discover the problem, usually about 4 hours later, I no longer have any idea what I did that made the bad thing happen. Then I spend another 2 or 3 hours figuring out what I broke and fixing it.  Ugly.

On my personal projects, I check in code every time I get a unit test working. My checkins are something like 15-20 minutes apart.  On projects I get paid for, though, checking in means running the whole unit test suite, and that can take 10 minutes (on a good project) or 2 hours (on a bad one)--so I don't do it very often. That's when I get into trouble.  I've been meaning to solve that problem for some time, and Joel Spolsky's blog topic last Wednesday (Joel on Software) finally kicked me in the pants.  It took 15 minutes to solve the problem; here's how I did it.

The new breed of source code control systems, distributed systems like Git and Mercurial, have had my attention for awhile, as I usually work with a team that's spread out geographically.  I regularly need to share code that's not quite ready to be delivered with another developer, and that developer is very likely in a different city.  Typically we're reduced to emailing files to each other. A distributed SCC system would resolve this, as we could sync changes between our personal development repositories, but I assumed setting up and learning a new SCC would be painful, as it has been in the past, so I never got around to it.  Joel's article on Mercurial, however, got me to thinking about it seriously, and since I had a few hours on my hands, I figured I'd give it a try.  What I didn't expect was that I'd be up and functional in 10 minutes.

I downloaded Mercurial and installed it on my main dev box, which is a Windows laptop (I know, I suck). That took about 2 minutes, including googling for the Mercurial web site (http://mercurial.selenic.com/). Then I changed to the main development directory of my current project, and typed (per Joel's tutorial at http://hginit.com/):
hg init
hg add

The first impressive thing is these commands 'just worked'.  The second is that's all that's required to set up a local repository and add an entire project to it. Really. I'm so psyched!

The "add" command was naive, because it added everything, including build output and subversion control directories (**/.svn/**).  I spent the next 10 minutes reverting my add, ("hg revert --all"), then building an "ignore" file, then adding again, and finally committing.  To shorten your search (Mercurial has great documentation, by the way--it only took me a few minutes to figure this stuff out), here's what I ended up doing.

1. I created an .hhrc file in my home directory (C:\Documents and Settings\Jerry) with the following content:

[ui]
username=jerry
editor=C:/bin/vim/vim71/gvim

The "username" entry preempts a request by the "commit" command for a username, and I prefer vi to the default editor, which is notepad.

2. In the root of my project directory, I created a .hgignore file with the following content:

syntax: glob
.svn
*.class
*.log
.hgignore
antbuild/*
build.properties

This tells Mercurial to ignore all files or directories named .svn (which is where subversion stores its status), all .class and .log files, the .hgignore file itself, the build.properties file, and anything in the antbuild directory (or its subdirectories). Mercurial ignore files support at least 3 different syntaxes for specifying files; the documentation is available on the Mercurial wiki and it's quite complete.

Finally:

hg add
hg commit

Now I'm in a position to check in locally every few minutes, but when I have a small feature complete, I can deliver via svn to the project's repository.  That's right: I'm using Mercurial locally, and SVN for the project.

On my next project, I hope to have a chance to specify that the whole team uses Mercurial for the whole project; working with another programmer who's not physically nearby just got a whole lot easier; we can exchange our updates directly with Mercurial, then push them up to a central repository independently. Sweet!

There seems to be a fundamental debate raging:  Is software more like mad science or plumbing?

This debate came to my attention via Mike Taylor’s article:  What Ever Happened to Programming? Mike’s argument is that programming is nothing more than plumbing today and it’s no longer fun.  He believes that there’s more fun to be had as a “mad scientist” developer, building everything from scratch with materials at hand.  So let’s look at the two major camps of this argument:

Software Developer as Mad Scientist

It's ALIVE!

Reading the classics of software literature like Knuth, Brooks, et al, you get some notion of the programmer as hiding out in an arcane computer lab, late at night, pounding away at the keyboard as if to build some Frankenstein program.  Unlike Shelley’s creation, the outcome is far more benign and often even useful.  But the creation is always that:  pure construct from the mind of the programmer.  No assistance from the outside world aside from a few borrowed organs to create the Magnum Opus.

This image has historical truth in it–Knuth himself created TeX in a similar fashion.  Supposedly Woz and Jobs did the same with early models of Apple computers.  Every programming language we have at our disposal today clearly had some singular human force behind it:  Ruby, Haskell, Java, C, C++.  The list goes on ad infinitum.

These creations required intense and detailed knowledge of the hardware and operating system to create their monsters.  Whatever they required in their tasks, they often built from scratch by themselves.  They are the pioneers of our fields, the first wave of migrants on the digital frontier.

Without the Mad Scientists, we would be language-less, tool-less, and probably stuck with punch cards on ENIAC.

Software Developer as Plumber

I hear our job derogatorily compared to that of a plumber:  “We just put stuff together instead of build it”.  I don’t think that gives plumbing it’s due, nor does it really consider the rich history of the field.

So, I hear you've got a clog in your database...

Plumbing back in the late 1800s and early 20th century was a dicey business.  The entire practice was inconsistent, lacked any standard methods, and training was haphazard (for much more background, check out this article).

Appropriately, the National Association of PHCC (formerly the National Association of Master Plumbers), first met in committee in 1883 at the old Astor House, the hotel that provided the impetus to modern plumbing back in 1834. Many new plumbing inventions had appeared and too many plumbers were ill-prepared. Close on their heels would be the Mechanical Contractors Association of America, the American Society of Heating, Refrigerating and Air Conditioning Engineers and the American Society of Sanitary Engineering.

Wholesalers banded together, too, starting programs to prod manufacturers into standardizing such things as sink and basin outlets, faucet drilling, trap gauges, etc. The Central Supply Association, for example, was formed in 1894 and soon made contacts with the old Eastern Supply Association, the Plumbers Association of New England and the National Association of Master Plumbers. But it would take another 30 years to accomplish the standardization which everybody takes for granted today.

That means roughly the first 50 years of plumbing (1883-1925) was effectively like the Wild West:  Every man for himself and standards be damned.  The public often suffered as a result of this:

An outbreak of amoebic dysentery in Chicago during the 1933 World’s Fair was traced to faulty plumbing in two hotels. Tragic results were 98 deaths and 1,409 official cases. One year later, Major Joel Connolly, Chief Inspector of the Chicago Bureau of Sanitary Engineering, spoke these prophetic words:

“One of the lessons to be drawn from the amoebic dysentery outbreak … is that plumbing demands the very best, painstaking effort that thoroughly qualified, certified plumbers can give in every building, and especially where the systems are complicated and extensive, and where large numbers of people may be affected by contamination of water.” (emphasis mine)

Clearly standardization of materials, methods, and training gave plumbing a major shot in the arm for consistency, safety, reproducibility and public trust.  The plumber’s model started off as cowboy hacking of pipes in a haphazard way to a systematic method of standards, interoperability, training and licensing.  Along the way, there were glitches, problems, and issues.  Big surprise.  Sound familiar?

Our software legacy has taken us from raw register manipulation in assembly, through multi-generation languages (2GL, 3GL and god help us, 4GL), to huge amounts of frameworks, libraries and tools that give us unprecedented levels of productivity today that would be unheard of 10, 20 or even 30 years ago.  But the cost is that there is less of the low-level work to do, and more of the heavy-lifting at the business level.

We’ve complained, bitched, and moaned about spending too much time on things like low-level problems:  database connectivity, GUI frameworks, XML parsing.  And guess what?  People responded to those complaints by building libraries, tools and frameworks to make them happen. To give us what we always wanted:  the ability to focus on the business problems and not the lower-level constructs.  In essence, we’ve borrowed the plumber’s model.

You assemble the pipes, solder them together, solve local problems about how to route the sewer line around the funky wall shape, but you don’t get to set the pipe sizes, mold the elbows, or determine the ideal composition of solder for ease of melting.

You get to put things together for utility.  A large part of modern software development is nothing more than a utilitarian venture of “some assembly required”.

But even the first waves of migration to the American West by the military and trappers of the day had comparatively little impact to the settlers of the late 19th century.  And similarly, the mad scientists of software have a significant, but much smaller, impact in comparison to the plumbers of software.

So which are we?  Mad Scientists or Plumbers?

Both.  Neither.  It depends:  On who you work for.  On what you specialize in.  On where your interests lie.  There is a need for both:  mad scientists are the creators of new tools, frameworks, languages and OSes, plumbers are the integrators, the users, and the orchestrators.  Software requires both to survive.

This isn’t a question of what software is, but rather who you want to be.  But there are some facts that are hard to argue:

• There are more plumbers jobs than there are mad scientist jobs.
• The need for mad scientists seems to diminish over time, not because mad scientists are less important, but because more plumbers are needed once the mad scientists are done with their work.
• It’s very hard to be a good mad scientist OR a good plumber, but they are vastly different skill sets.

Be an mad scientist or a plumber, but don’t complain when you’re a plumber but you really wanted to stay a mad scientist.  The choice is, and always was, yours to make.

No related posts.