Stu Stern is on the latest Speak Rich podcast on RIA Revolution discussing Flex Monkey. Thanks to Shashank Tiwari for interviewing Stu and covering Gorilla Logic’s favoriate testing tool.  RIA Revolution is a great resource for Flex developers.  If you haven’t already, check it out!

I'm a fan of even partial solutions to the executable design problem. Here is are a couple I find particularly interesting.

DataXtend

http://www.progress.com/dataxtend/index.ssp

I had the opportunity to run a 1-week in-house live test of DataXtend. We took a bit of design from a set of web services we had delivered using Apache Axis and a lot of back-end Java code, and gave a design spec and the original interface specs to the DataXtend team. They showed up in our office on Monday and we did a little work on reviewing their capabilities. On Tuesday, we started working on the project, which had taken our offshore development team 6 weeks to develop (admittedly, that's way longer than it should have taken, partly due to continuous changes in the requirements, and partly due to just poor development management). By late Tuesday afternoon, we had a working web service. I load tested it; I threw bad input data at it, and I deployed it in several different environments, and it just worked.

A design must an abstraction. A design constrains and informs an implementation. It allows developers to focus their creativity on a smaller set of problems. In fact, the whole point of the architecture/design/coding division of labor is to allow creativity and problem-solving to be applied on a small set of problems--problems which one individual can get their head around and thereby have a reasonable chance of solving. If you lose the ability to abstract by forcing the whole application into a single model, you have lost something important--you have, essentially, "jumped to code" before you understood the problem.

The problem with abstraction is that, without at least one working example, it's hard to tell if the abstraction makes sense. (Two good friends of mine are fond of saying "all abstraction is good, as long as it's your abstraction.) In new development, then, the ideas behind an executable design clearly have a place; they're a way to validate the abstraction actually does what it says it does.

As Bill Glover pointed out in his comment on an earlier post, applications get complex and crufty with time. He asks (with regard to reverse-engineering an existing implementation): "What will keep the design from being obscured by the kind of detail that starts showing up then? ... An example would be all of the methods and attributes that developers add to a class that aren't really relevant to the high level design, but are needed to make the class really do it's job."

What Bill's talking about, I think, is the very real and common case where you're trying to bring order to an existing application by describing its existing design. If, for example, you have a tool which can execute class diagrams (e.g. the GXE), how do you use that serve as an "armature" for the rest of the implementation?

If you just want to verify that the design and the implementation match, you're not in executable design space--you're in static code analysis space. There are tools for that (e.g. http://www.ndepend.com/). If you want an executable design to be an armature for the application to be built on, then you're really refactoring. The trick here is to allow the tool to replace some part of the application's existing functionality without a complete rewrite. Here are two example behaviors I think a tool might implement which would allow them to operate in an existing application, replacing part of that application.

I once introduced a state machine into an application which had significant business logic embedded in its screen navigation subsystem. I rewrote the navigation for just one screen, pulling out the business logic into separate classes for each business rule, and describing the screen-to-screen navigation in a state diagram. The state machine responded to user events (e.g. button clicks), queried the new business logic classes to get guard results, and handled transitions within and off of the one screen. Clearly, this approach can be extended to other screens without disrupting the application as a whole (and in fact, the development team for that product is doing just that).

I'm currently working on a project which is using an executable design tool (the Gorilla Execution Engine) to validate requirements. Classes representing domain concepts and relationships are executed in the GXE to see if the very complex calculations modeled by those classes give the "right results". In my ideal world, we'd then take that domain model, specialize it (that is, turn it into a design model), and it would enforce class behavior in the finished product. I might reverse engineer a dozen classes which participate in a given calculation, remove or hide the methods which aren't germaine to that calculation, then add new methods from the domain model to flesh things out. I might remove irrelevant methods from the model entirely. The execution tool would provide a class loader which would:

1. load the design model,
2. upon a class load request, it would search the model for a definition of the requested class,
3. look for any class implementations of the same class,
4. compare the two, merging them if the the implementation does not contradict the design model, and complaining loudly if it is not (this step could actually be done at build time rather than runtime), then
5. load the merged class for execution.

An approach like this would allow the designer and developer (sometimes the same person, right?) to work together to decide what portions of the existing application could be replaced with an executing design. This would address a problem I see a lot, where the code can't actually be traced back to any requirements or higher-level design. By forcing the code to match the design in order for it to run, the design gets updated because it's the best way to get the build to work.

Of course, developers could always decide they don't want to execute the design, but they always have that power. I'm assuming that designers and developers are working together to build something; if they're not, the organization has issues no tool can address.

I read two good posts this evening with a similar theme, implementing a system is boring … at least when done well.  Stu Stern’s post discusses it from the standpoint of defining a clear architecture up front:

An architecture is sufficiently defined when we can turn a developer loose on developing some piece of functionality and he or she obeys all design constraints, there is a minimal possibility of creating software that produces any of the above problems. If you’re conceptually bothered by being constrained, preferring instead to program in wide open spaces with the wind blowing in your hair, perhaps you would prefer to think of architecture as providing the freedom from choice.

Alex Papadimoulis post, Programming Sucks! Or At Least, It Ought To, takes it more from the realities of building business systems on a day-to-day basis.  He basically concludes that developers get too enamored with all the noise in the software world and need to focus on building features for the business:

As frustrating as it can be to work with the uninspired, sloppy developer, the contrary – the inspired-yet-misguided one – is several magnitudes worse.

Alex offered a list on how to approach the profession:

1) Learn the Business.
2) Serve the Business.
3) Learn Off The Job.
4) Code mostly Business. (If the overwhelming majority of your hand-written code isn’t domain-specific and doesn’t relate to the application’s purpose, then you’re using the wrong tools.)
5) Tedium is Inescapable.
6) Find Satisfaction Elsewhere.
… and, if all else fails…
7) Get Another Job.

I do love Alex’s post and believe he basically hits the nail on the head about the attitude we need to take toward the profession. However, I believe Stu has better captured the reality of building software systems. Things do change and evolve in software in ways that are both beneficial to the developer and the business. Thus, defining an architecture up front on how the system is going to be built is an essential step to allowing developers to focus on building features that have value to the business.  If we succeed when it comes to architecture, then we can get on to the boring parts.

One could even say that if presidential elections are about the economy, then when it comes to software: It’s Architecture Stupid.

An architecture is essentially a set of design constraints imposed on an IT development project. You want one because unconstrained design leads to impedance mismatches among the various pieces of a system due to logical or physical incompatibilities among software components developed by different team members or third parties. The result is software that takes longer or costs more to build, or is more expensive to operate once it's deployed.

Impedance mismatches can manifest in a variety of ways.

• Model - Multiple, possibly inconsistent object definitions for the same logical entity
• API - Incompatible arguments or return results required or produced by libraries
• Performance - Inability of data sources to provide data as fast as required
• Locks - Contention resulting from different components having different expectations for how resources should be shared
• Transactions - Improperly serialized updates resulting in data being lost or overwritten with stale values
• Responsibility - Functionality needlessly duplicated or incorrectly assumed to be provided elsewhere
• Recovery - Components in the event of failure not coordinating properly to minimize downtime or data loss
• Operational - Software too difficult to modify in response to anticipated future needs

An architecture is sufficiently defined when you can turn a developer loose on developing some piece of functionality and if he or she obeys all the established design constraints, there is a minimal possibility of creating software that produces any of the above problems. If you're conceptually bothered by being constrained, preferring instead to program in wide open spaces with the wind blowing in your hair, perhaps you would prefer to instead think of architecture a form of freedom -- the freedom from choice. Once defined, an architecture frees developers from having to make (possibly incompatible) choices for how to implement an application's functionality, and instead devote most of their time to implementing functional requirements while minimizing the time spent hacking together glue to integrate with the rest of the team's code.

Here's a very high-level breakdown of a typical ERIA:

BackendService

In any non-trivial enterprise application development project, multiple developers will be working in tandem, and in an RIA project where the client and server platforms do not even use the same programming language, it is natural that developers will be divided into front-end and back-end teams. In order to reduce dependencies between client and server code development and allow the teams to work in relative independence on their respective components, it makes sense to explicitly define an API that will provide the linkage between the client and server code.

The BackendService is typically implemented using a webservice-type container, for example Tomcat and associated plugins, which provides a framework for authorizing users, managing session data, and communicating over web protocols such as REST, SOAP, or AMF.

UserSession

As with most traditional web applications, we require a container that maintains session state for each logged in user. Unless application users are anonymous, the session state will at a minimum contain user identity information required for access control. For many applications, code can be simplified by maintaining other session-specific information on the server, such as for example the current contents of the user's shopping cart.

UserIdentity

In most applications, we require user identity information in order to be able to complete requests received from a client. For example, when a user requests information for "my account", the server must need to somehow know which account to access. It is typically not an option to supply the account number for example as part of the request, since the client could be running a hacked application that allows the user to specify an arbitrary account number. When the user authenticates with the server, the server must associate the authorized user's identity with the session, and check each subsequent request against the that identity information in order to perform the requisite access control.

UserIdentity is typically implemented with role-based security frameworks such as JEE security implementations riding on LDAP or some other user directory store.

DomainObject

DomainObjects are instances of classes that directly model some aspect of our problem domain, such as Customers, Invoices, and Products. These objects are mostly created from input received from the client, or by being retrieved from a database or some external source, and additionally provide the non-UI-specific logic comprising our application.[Mention DDD?]

If we do our jobs right, most of the custom code we write on the backend will be domain-specific, having delegated most of the non-domain-specific plumbing to third-party frameworks.

PersistenceManager

The PersistenceManager caches previously retrieved data and collects updates triggered by the client until such time as a logical transaction is completed and can be committed to a database or external system. For relational databases, the PersistenceManager is typically implemented with a framework such as Hibernate. [Link to hibernate definition?]

BackEndServiceAPI

Clients access the BackEndService via a library that provides a client-code-friendly interface. This BackEndServiceAPI is often no more than a wrapper that exposes the BackEndService interface in the client's native language, and does little more than marshal arguments as necessary to call whatever remoting framework is being used for backend communication, and set up whatever callbacks are needed in cases where results are returned asynchronously.

The BackEndServiceAPI would usually be defined as an interface for which a mock implementation can be used as a stand-in so that the front-end team has the option of proceeding with development ahead of the back-end implementation being complete.

DomainDTO

In a typical RIA, we cannot actually return full-blown objects from the server to the client or vice versa. Instead, we can only transmit an object's primitive property values. For example, if the client requests a particular Customer, the server does not return an actual instance of a DomainObject, but instead returns a set of values such as name, address, and account number. While its possible to construct an actual DomainObject instance from these returned values, it is often unnecessary since an RIA in many cases is merely providing for the display and editing of such primitive values, delegating any requisite business logic back to the server-based DomainObjects. DomainDTO's are objects that provide holders for these values, but don't necessarily provide any "behavior" in the form of methods on the objects. It is of course also common to selectively replicate some DomainObject logic on the client, especially simple field edits, and in some cases we might even replicate complex logic on the client, so we're using the term DTO loosely here. [Link to DTO definition?]

[Diagram showing a DomainObject vs corresponding DomainDTO?]

CacheManager

With a thin client, client-side state does not usually consist of anymore than the contents of some html form or table. Each new view usually requires a round-trip to the server to retrieve both the definition of the view (ie, an html page), and the data presented within it. Even just changing the sort sequence of the rows in a table require re-retrieving a fully formatted table of data from a server. One of the big advantages in an RIA is that we can provide a much more interactive and responsive user experience by not requiring the constant back-and-forth of data and markup with the server. Instead we can retrieve (possibly large amounts of) data just once, display it in a variety of ways, allow the user to edit it, and send updates to the server only after a transaction is logically complete.

The CacheManager keeps track of what we've already retrieved from the server so that we don't incur unnecessary overhead of re-retrieving the same data, and tracks what objects are changed ("dirty") and require transmission back to the server. The sophistication of CacheManagers can vary greatly from one application to another, consisting of little more than hash tables of objects by type [Include a simple diagram of this?], or as much as client-side relational databases.

ViewComponent

The view components are the widgets that provide for the display and editing data, usually DomainDTO's retrieved from the local cache. By ViewComponents, we don't mean low-level UI components like buttons and textfields, but semantically rich composite components such as for example a Customer Creation Form or an Order History Table.

EventBus

RIA platforms are event-driven user interface frameworks where ViewComponents emit and respond to asynchronous notifications such as "User Wants to Quit" or "User Has Updated His Credit Card Info". While such event-driven models are ideally suited to coordinating the asynchronous update and display of data across possibly multiple views, event-driven programs can degenerate into a tangle of interdependencies among components dispatching and listening for each other's events. An EventBus organizes event handling around a hub-and-spoke where events are logically broadcast throughout the application. In this way, brittle point-to-point connections are replaced by a robust, centralized switchboard for application events.

Application

The Application is "everything else". It's the code necessary to integrate the BackEndAPI, the ViewComponents, and the CacheManager into the functioning whole experienced by the user. The Application for example binds particular views to particular DTO's, and invokes particular BackEndAPI methods in response to user interactions.

Can I just go build my application now?

Having dilligently studied the above description, you may be chomping at the bit to put the above principles to work and begin building your next RIA. Unfortunately, the above is not the answer. It only frames the questions more clearly. Chiefly:

• Whether to locate logic on the client or server?
• How much data to cache locally on the client?
• When should data be transmitted from the client to the server?
• When should data be persisted to a database?
• How should view components be made aware of updates to what is being displayed?
• What is the right level of granularity for remote operations?

What's that you say? You came here for answers? Fear not. All will be revealed. But not this afternoon....

I spent most of the day working on setting up a new project with the latest Spring (3.0.0M1) and Hibernate (3.3.1.GA) releases.  I have more recent experience with Hibernate and was able to get everything working pretty quickly with Postgres on VMWare share.  It has been a few years since I last setup a Spring project and really got down in the guts with it.  I remember it being a mostly low cost framework that provided quite a bit of handy plumbing.  I have never been wowed by dependency injection for dependency injections sake, but I have always liked the things Spring pairs with their container (AOP, ORM support, etc).

Anyhow, as I dug in today I was pretty shocked to see how bloated Spring (and Hibernate) have become.  I have been guilty of throwing JAR after JAR at problems, but it was almost unbelievable to see how many dependencies each framework now has, which of course caused me significant versioning hell in attempting to get the right mix of JARs.  With each change, a tweak to libraries seems to come.

The other surprising part to me was the number of different options there are now for configuring Spring, and how difficult it is to find a clear path to follow.  I am huge fan of annotations, and I am glad that they added significant support in that area.  However, even though I now have a working setup, I have little confidence that I am completely inline with the best practices.

Our company’s founders, Stu and Ed, love to make the statement, “Freedom from choice.” It seems like a “lightweight container,” like Spring would follow this simple axiom.

Anyhow, I am posting my current project for feedback.  Let me know if you see any silliness or have feedback on how I can improve my setup.  It pretty basic so far … Hibernate with Annotations, Spring (Annotations, ORM support), and SpringSecurity.  Here is the project.

Flex has definitely taken the web and UI worlds by storm. We are also seeing that the demand for Flex talent is exceeding supply which is good news for Adobe, but not so good for companies looking to get serious about Flex who need experienced people.

Stu Stern (http://stu-stern.blogspot.com) raised the question: "What would the executing design itself provide?" That's a big question, and I wanted to think about it in a longer context than "comments".