The .NET Sweatshop

Working day and night to help .NET users get the most out of programming the coolest platform on the planet.

Tuesday, August 19, 2003

Building guidance is fun. And gruelling too.
But, related to Sandy's last post, there's an aspect of building guidance which I'll share- it's the responsibility that goes with making recommendations on how to design, build, secure, etc your systems.
It's with this is mind that I entitle this log entry....

The Weight Of Giving Guidance

Providing best practices for application developers is a great privilege, but a great responsibility too. There's of course the typical standard stuff that goes into executing any software project - like an Application Block. There are responsibilities to meet customer demands under a gazillion constraints. But there is a meta-responsibility in whose light the 'patterns and practices project execution' metrics (as in-time, in-budget, bug-count, happy-team etc) pale. The responsibility that goes with knowing (or at least..hoping) a large set of architects, developers, and operators will be following the advice.

I like feeling personally accountable for the guidance we give. When we write content & code about something, I like to feel the weight* that goes into giving those recommendations and dealing with those problems. It goes beyond dogfooding - trying to follow the guidance ourselves at PAG to build a system. It also goes beyond the beta-testing in live accounts that happens during guidance development.

What I like doing is working directly, closely, intimately, with customers who are at critical decision points and help them through the issues by providing information, material, and processes, and yes, being a part of the team and having an opinion in the decision making. I think it's important to be able to look at someone in the eye when recommending them whether to use datasets or build custom business entity classes. To be able to work with the developers who will be following that recommendation, answering the conceptual and nitty-gritty questions . And giving them your cell phone number to be able to track you down in case something went astray when, for example, they discover they need to change the way they build entities when they already have 300 of them, a trained team of 10 developers, and a whole set of design decisions made upon the previous choice. But it's not the fear of the call that drives improvement. Working closely with them builds an empathic connection that can't be easily severed. When one of those guys stays late at night working on a toughie you should have prevented, you feel the hit.

So there it is - guidance is not fire and forget. If you are part of an architecture team, or set design policy/guides for others, I'm sure you do have a measure of personal accountability for defects in the engineering advice which is the outcome of your effort... at PAG, even though we are in far-off-where-is-it-anyways-often-blamed-for-ivory-tower-thinking Redmond,WA,USA, we feel that accountability too.

[AGCCTG] Changed the St.Germain CD -Finally. For Paul Van Dyk. And Delerium is coming to town.
[TCGGAC] Still waiting for my Lego.StarWars.TieInterceptor. That'll snazz up the collection in the office quite a bit.

*Does guidance have mass? I guess I mean weight in a Kunderian way.

Monday, August 18, 2003

OK, Ed takes care of the thought leadership and I will serve as the shameless shill of new guidance that is on our site. The latest guide from our team is the WS-I guidance that we worked on in coordination with the XML Web Services team. It is located at:

WS-I Guide

I was able to witness some of the neogtiations that went on between all the companies involved to settle on the final ratified rules and it was a clearly an impressive experience. It was overwhleming to see a lot of really smart people feeling empowered to change the way people connect their apps. As one director from another group at Microsoft put it, "we are helping define how people will be implementing their systems for the next 20 years. We have to take that responsibility very seriously." It's that sort of thinking that really makes me excited to be a part of this company...

Thursday, August 14, 2003

As Sandy said, if you got stuff to blame BlueBricks for, I'll take the hit..but please go to the GotDotNet wokspaces first. Theres lots of people who can help buffer the anger :)

There's Models and Then There's Models

So if you work specifying metamodels, or domain models, you will have surely gone through what I have just gone through right now.
It starts simple. You start modelling a space because it happens to have too many entities - in my case the mental threshold is just above one. And it starts getting richer. Enrich, normalize, refactor, enrich, recheck assumptions, come from another angle, add something, remove half, bring back half of that, focus on the left side of the diagram, on the right side of the diagram, and suddenly you end up with the 100% correct-truthful-to-boot-mathematically-provable-ultraprecise-specification....covering 2 miles square

Which brought to mind the endless discussion between the usefulness and usability of something. A complex model can be extremely useful - bringing in the right indirections, separations, classifiers, generalizations, specializations, subdivisions, and so on - but it is unusable as the number of decisions that need to be taken to implement one instance of it explodes. Suddenly doing something 'simple' seems to take 38 decisions from superabstract to gunky levels. And that's not what I want to do for a living. I want to make it easier.

All this in the end brings up one question in some people's heads:
Why? Why not forget all this hogwash about models, metaprogramming, and metadata stuff and let you write the code?
What's the value?
I think the typical answer revolves around many topics, but boils down to one thing: Compression of information

Think about code. Think about it in terms of the decisions you need to take.
Think about all the decisions that need to be taken to give it the right 'context' - exception management, transactions, authorization, etc.
You would need...a finite amount of yes-no answers before knowing how you want to deal with these things.
Simple Example: Transactions: Required, Requires New, Supported, Not Supported. 4 decisions = 2 bits.
Simple Example: Exception handling: no try/catch, try/finally, try/catch/finally, try, catch(publish), and finally.

Now think about all the decisions you would need to take to write the code to implement these decisions. Should I write an exception handler? For what exception type - a generic or a specific one? The number of yes-no decisions needed to write the code is way larger than the number of yes-no decisions you need to answer to completely specify the desired behaviour. And there it is - while in code it may take N bits to express a decision in code, that decision really requires M bits. The remainder bits (N-M) are duplication, redundancy, noise (or non-significant bits), or derivable from context.

We have to deal with this every day when building Application Blocks for .NET. You may have notice they are heavily based on metadata. I use these guidelines in myhead when designing the metadat models and schema for those models:
- The model should separate unrelated concerns (eg separate dev stuff from ops stuff)
- The model should be normalized to reduce redundancy and duplication (eg reduce places where a type name is declared)
- The model should allow for variability points (setting structure rather than policy - or architecture rather than implementation)
- Allow generic multiplicity - if you see 2 subelements may be needed for a certain element, think about supporting N.

But metadata can get complicated. This complexity can be reduced by some useful techniques:
- The schema should group together related concerns as seen by an audience or by a use case to reduce the files and places to touch in order to 'do something'. Tools can help in this too.
- The configuration system should have smart defaults, being able to robustely deal with incomplete specification and expose incosisitent specification.

Sometimes though, a little extra is needed than what you may have planned for, this is how you cope with unforseen change:
- Allow for extensions of the metadata schema that do not invalidate the part you need (ability ti intermesh extra elements, attributes, etc)
- Allow pass-through parameters - see how people can specifiy extra attributes in your metadata schema, and give it to the appropriate components related to that segment.

Having good metadata schemas have another effect which is visible in certain environments : by actually providing a declarative environment and a metadata schema, you are exposing what are the decisions you need to take. Sometimes what makes you fail is not your wrong decisions, but the ones you didn't know you had to take, or the ones that the wrong people took without knowing the impact. And adding the time dimension, tools can help the right metadata decisions happening at the right time.

Of course many people don't like this whole metadata thing. They'd rather code themselves. As with any technique there's good and bad ways of using it. It's important to understand:
- where using metadata compresses the decisons and where it actually expands (think of it as an abstraction level. For some stuff, specifying what you want in code is more compressed than expresing it in other ways)
- when it is applied, by whom, and when it can be changed and what is the impact

Happy modelling.

[000] Listening to St germain. Gotta Change that CD.
[010] Still cooking the aspects thingie. There were some good discussions in the last MSR Campridge fest, (DUUUUH). Dang. I should have not turned down the invitation.
[110] Scott and Naveen are working on the Logging Block - a set of EIF publishers for SQL and MSMQ, EMAB -EIF integration, and other coolness.
[100] It's sunny outside but no flying around today for Ed
[101] I just found out our word template can import boilerplates. I think I might make some for the app block writers.
[111] This whole codegen thing and templates for metaprogramming transformation thing in tools is not getting enough of my time for it to happen. I had some cool stuff in my old notes, will need to fish them out.
[011] Sandy, Srinath just beat the heck out of Per and me at Foos.
[001] Just finished traversing a cube following Hamming distances of 1. Given a hypercube of N dimensions, what's the amount of points that can be touched with (1..N) hamming distances?

Wednesday, August 13, 2003

As you see from the previous post, we have a new blogger in the .NET Sweatshop. Ed is Captain Bluebrick, the renegade Solution Architecture that is the father of the App Blocks you see on the patterns & practices site. There are few people at Microsoft with as much enthusiasm as EdJez and he backs it up with a great knowledge of the .NET Framework and customer challenges. He and Srinath actually give this blog a little credibility, so I'm hoping they keep posting. I've got more to say, but I'll let you digest Ed's first post before I do my thing...

Tuesday, August 12, 2003


There it is. I mean, you shouldn't have to build the yet-another-once-again-data helper right? So if you have been following Sandman's shameless plugs into patterns & practices (please do: p&p) you know what I'm talking about - the Application Blocks for .NET. They help you build better systems, faster.

It's been a privilege....
Before saying anything else - let me place the focus where it is due. BlueBricks was put together by a lot of effort of a lot of smart folks within and outside of Microsoft. It was individuals who felt the need, and built solutions for their peers. So - here it is to the first wave of BlueBricks devs. It's been a privilege to work with each one of you:

- Steve Busby & Bart Robertson - for the COM & .NET interop stuff, the fundamental 'what if we...' meeting, and finding ways to take my bif. Steve must wake up at night recalling my pleas - "Busby - Find me Brrrrains!!"
- Franco Ceruti & Fili Selvas Patino- for the endless design discussions and the needed support & drive to go on when the effort seemed (was!) just too much.
- [Ami]Tabh Srivastava & Jeff Kercher - for the wmi guide, the authn stuff. Jeff's contribution to the software industry: IPV6 security and... "Are you building an app? Y/N...N:have beer"
- Chris Brooks and Kenny Jones. The first block implementers. Chris & Alex wrote the data access guide and built the 1.0 of DAAB, and Kenny the EMAB. If you have a support question about the blocks their cell phone # is.. (By the way, Jason Hogg has recently updated them! DAAB can get typed datasets! FINALLY!)(By the way - Please manage your exceptions, it makes Kenny happy)
- Angela Crocker , Chris Schoon - 'passing data thru tiers' and 'app-managed authorization' guides. The architectural reviews drove Angela nuts enough to become an Architect Evangelist.
- Jackie Richards, Aaron Barth , Ray Escamilla& Brett Keown, Team Development, Debugging, Package & Deploy - always cool to work w the ol' gang from PSS. If anyone can make a tough problem go away, it's them.
- Graeme Malcolm & Alex Mackman, the first writers from Content Master we worked with - Graeme survived me in an architectohrrea bout that lead to this
- Ronen Ashkenazi & Avi Ben-Menahem - the caching guide & block design. Check out the pluggable "Stoghage" and "expighations" on that block.
- Michael Stuart, Diego Gonzalez, Ed Lafferty and the gang: for the UIP, CMAB and Updater. Michael kindof represents the archetypical bluebricks dev in my mind - passionate for the cause and customers, critical in thought, and always asking for more funding.

Of course there are a lot more people now, and there are many others in test (Mo, Chris), writing (many!), management overhead (no need to point fingers), etc that helped all along. Them, and the consultants, specialists, partners and customers who shaped this are literally many hundreds. But it was the push and passion and precision of these folks that let us grow to what it is today. It was a fun process - I got to experience first-hand what it was to grow an underdog project from skunkworks status to...the publicly known under-dog-skunkworks it is today. The start-up mindset is alive and well. I got to sleep in the office for the first time since I joined MS. I became part of the coolest alias around- The Think Tank. I got to build the BlueBricks server on a hand-me-down box which still sits under my desk. Are we done? oh no. Keep on reading.

Here's the takeaway for this post: p&p and the Application Blocks are built by lots of people like you, for people like you. That means something - it means you can be part of it too. We are still learning about how to share plans and get feedback in a scalable way, but in the meantime we created one GotDotNet workspace for each Application Block & topics so you can go and help shape the future.

Thought of the minute
When I started BlueBricks I had a set of principles in mind which I won't state right now because lots of things come to mind. I hope the philosophy will gleam through this and other posts.

It's interesting though how folks who work a lot with software and think holistically are able to develop a sense for software - a sense not unlike sound, smell or touch. A sense that can be probably traced back to, or rationalized into, basic software principles (coupling, cohesion, Demeter, etc). And like any sense, you develop a system of aesthetics for what you perceive. (you know - you may run into an APIs that feels rough. Objects may seem to be tangled. I've heard Martin Fowler refer to how code smells). When I talk about 'design philosophy' or 'aesthetics' it raises some people's eyebrows from some folks, but it also gets understanding looks.

I've seen many types of frameworks for developers, but they can be classified in two familes - those that you code to and those you code with.

You can tell them apart right away. The ones you code to usually have a big symptom - you have to structure your code to some interface, and when object A wants to call object B it needs to call the framework who will do a lot of stuff for you and then invoke B through this interface. I like thinking of them as clerks - you ask them to do something, and they go and deal with other folks and things that the clerk - and sometimes only the clerk- can talk to.

Those frameowrks you code with, however, are different in structure and convey a different feeling when using them. These present well-encapsulated services that you can consume and that help set the stage for your work, but don't get in the way. I call these the 'guardian angel' frameworks- they won't deal with other components in your application, rather, they'll help you deal with them in the right way.

Example: Think of the User Interface Process Application Block. It sets the controllers, state and does page/window navigation for you. But your ASP.NET page calls the controller the usual way. It's a 'normal' .NET component call. No unnatural command pattern getting in the way, no pesky untyped parameters, no loss of productivity in defining 'payload types'. As we keep tackling tougher problems with the blocks we keep pushing ourselves to conform to this. I'd like to be able to say 'you've just added X capability to your app - without touching your functional logic code'

So where do they get mixed up? Well, some types of things (like authorization, instrumentation, and other aspects) would actually benefit from being able to be 'in the way'. But without being in the way. Other things should just be 'declarations' ('of intent'?). But aspects & metaprogramming - "code gen" - and tooling are another story, albeit a nearby one :) Stay tuned.

[00] My whiteboard has the Authorization, Authentication & Profile App block design on it. Interested?Just created the workspace.
[01] Listening to St Germain.
[10] I once applied for a job @ Saunders-Vixen, and I think I might've gotten the job - and was outsourced to MS for the time being.
[11] Working on the Application Block template to help ourselves @ MS and others be able to build these with more consistency and efficiency.