After a finishing up my current contract, holidays and a bit of Bloggers Block, a chat with John Wenger about humanity and process a while ago has inspired me to do this post that I’ve had brewing for a while… Your first response may be a glib / tongue-in-cheek response “What, humanity and PMs, that’s an Oxymoron”. And therein lies the problem – I feel I almost need to point out that PMs are people too :-) Most of them are trying to do a good job in a probably difficult situation. They have families, lovers, pets: and they’re just at work like you – doing their best…
To tease this apart more, the first thing we need to do is separate the person from the role. With that, we can acknowledge that people are generally the same, so there should be no core difference between a PM and others in an org. Then we have the role – this is unfortunately where distortions can come in. Not only from the role, but the organisation and it’s view of that role which often puts them forward as “Managers”. I mostly see them as “Administrators”, though and I’m not saying this in a mean way – as an Architect (or any other core-IT role) I see them there to help me to build things for an organisation. Sure, they have their plans, but any good PM knows how to work the plan with the people and not the other way around.
In most of my engagements in large organisations, as a facilitator I’ve gotten along well with the PMs personally and professionally and they’ve always helped the project (well, there’s always the exception, but one can say that about almost anything). Strangely though, this seems to be the opposite of most of the “agile crowd” who seem to constantly complain about and want to get rid of PMs. What’s up? Have I just been lucky the last 15-20 years? I think not…
How would you feel if I came in to your work and told you I was a BoingBoing Master and that part of the BoingBoing Approach (tm :) I’d be replacing you, or if you were lucky would turn you in to a BoingBoing Apprentice where you’d have to start from the bottom again and work your way up?
Well that’s the attitude most agile people do! And then they wonder why PMs are so “difficult”…
As I touched on above, I acknowledge that while PM is not an essential role in agile, it can be accommodated. Part of this probably comes from my earlier work with (Iterative) RUP where PM is an acknowledged role. When I progressed to Scrum, it was using RUP Iterations, so again, the PMs were there – helping with Iteration and Project planning, while we ran the Scrums and handed most problems over to them to sort out.
This brings me to my current approach: As I mentioned, I view PMs as “skilled administrators” and “organisational facilitators” who are necessary in in large company. On that basis I help them understand how I can work with them as I’ll help the team focus on their work and maximising it, while they can help communicate this to the wider organisation and help remove obstacles that the team will be surfacing with them. This way, we both benefit, generally doing what we all enjoy.
Perfect? No, but it’s sure better than going in to a situation where you have exist with a powerful and combative “opponent” every day…
This approach also allows for what I’d call a “classic” Scrum/Kanban/… Facilitator, where this is not a full time job (which I can remember in the early days of agile and scrum). The rest of the time, I get to perform a team role (usually Architect, but it could really be any – e.g. (Business) Analyst, Developer, Tester, Infrastructure) which enables me to be deeply involved in the project and therefore not need “explanation sessions” where the team explains what they’re doing to someone who doesn’t really understand.
So is that it? Am I advocating we just sit back and not cause “too much trouble” for the entrenched system? NO! It’s quite the opposite as I’m suggest a campaign for Hearts and Minds conducted with Love and Compassion rather than the “Shock and Awe” campaigns that are often conducted by the Agile Extremists…
Before I get in to the guts of Skramjet, there was a final piece of philosophy that was missing. I couldn’t put my finger on it until recently whilst watching a brilliant PBS program The Buddha. One of the interesting facts I didn’t realise was how The Middle Way or Path was come up with – it was when he was attaining enlightenment.
After his initial life of total indulgence, then his Ascetic phase where he was almost dead (see right) that The Buddha realised that the way to enlightenment lay in between these two extremes:
Neither a life of self- indulgence, nor one of self-mortification can bring happiness. Only a middle path, avoiding these two extremes, leads to peace of mind, wisdom, and complete liberation from the dissatisfactions of life
An Agile Ascetic – well versed in Scrum, Kanban, TDD, BDD & NVC ;-)
THIS WAS THE MISSING LINK!
Most (if not all) agile processes, be they Lean, Scrum, Kanban or whatever assume usually quite a bit of discipline and adherenceto “the process” Don’t believe me? Try telling:
- A Lean / 6 Sigma adherent you won’t Define, Measure, Analyze, Improve, and Control
- A Scrum Master you’re not going to do the 3 questions or have a Product Owner
- A Kanban Kraftsperson you’re not going to limit WIP
In most cases, good luck with that… Does it have to be that way?
What is The Middle Way?
When I did my first official SDLC process training with Rational Unified Process, the 1st rule was “use the process to configure the process”. Unfortunately, many “agile people” don’t do this! They use a Waterfall process to configure an agile process. This, I suspect, is a key contributor to why many Agile adoptions fail. One of the lessons I learnt early on, after my first successful Scrum failure was to let the process emerge. My 2nd attempt was a dream run as we started with sticky notes and tasks – that’s all! No people against tasks and no estimates. The interesting thing is that the team added these two features in the next two retrospectives, which just shows that is you have faith and give control over to others in the “right context”, there shall be rewards.
It’s amazing how time flies as it seemed like yesterday, but it’s now been 8 years. Must be that old adage as I’ve certainly been having a lot of fun here. When I’d finished my last post on this, Agile Baby Steps – Iteration 3 (2 & 1), I’d finished up in Australia and moved to the UK. By coincidence it was where my Agile journey got a bit of a Turbo-Boost
After a few months of being a tourist and “kinda looking for work” I finally ended up in Andover which is a nice “little” market town just over an hour out from London working with Lloyds TSB. The role was initially as a Service Designer, but due to some enlightened management there I was able to introduce Scrum. Remember that in Aus I’d only just used it with a Scrum Master to integrate with Rational Unified Process, so I’d only just started down this route. To do a proper implementation we ran training with about 15 people and a Scrum Alliance trainer. This was the first time I’d been exposed to “pure Scrum” and it absolutely blew my mind! During the time there we rolled Scrum out across the various projects in the Business Process Re-engineering programme, even scaling it. Probably the key reason things went so smoothly was that the programme was Agile in it’s BPR activities and there had been a Lean effort in the past. Also, the programme had the concept of “Daily Prayers” that were like a Scrum, only unstructured so Scrum was really just a refinement (as the DP’s weren’t scaling) and extension which made sense as the program was just starting.
Next along was TNT and I had no intent originally for any process work as I was just there to do some performance tuning and architectural refactoring. After a few weeks though, it emerged that the team they were building up was having problems keeping track of things. One of the lessons I learnt from the previous engagement was not to be too prescriptive, so I said I knew a simple technique that could help. I told them the basic rules of Scrum and had Backlog, Doing, Done but did not tell them what to put on the stick notes! In the beginning it was just the task but after the first retrospective someone suggested adding the person responsible – “great idea” I said ;-) After the next retrospective, they suggested adding time! They’d invented Scrum Sticky Notes! I think this is a great example of what happens when you trust people and let them own “the process solution”.
Even better, was the fact that this was integrated with their existing Project Management process as at the end of each sprint I gathered up all the sticky notes and gave them to my enlightened PM who then did whatever she did with Gantt charts, reports etc… As the Scrum Master, I worked hand-in-hand with my PM and it only took less than 1h per day for me as the PM handled all the issues. Oh, but it didn’t stop there – we had an offshore team also. Again, thanks to enlightened management, we had a number of devs from offshore being rotated in and the timing was perfect. After a few weeks some of them went back to India and because they knew the process (it was theirs after all) Scrums were quickly up and running over there. We’d then have a teleconference Scrum to sync everyone up which worked a treat.
The interesting thing about the TNT implementation was that the “pure Scrum” people would be frothing at the mouth as we’d probably broken a number of “rules” – e.g. estimating in time, rather than story points, bananas, etc… Yet, I’d regard this as on of my best Scrum implementation so far as we’d cracked two of the classic problems that people have:
- Working in a more conventional Project Management process and
- Working with offshore teams
Most of this was not to do with me, it was obviously the management chain running through 3 levels who all supported this as what they’d been doing obviously didn’t work as well – they were ripe for change.
After a “pure SOA Architect” role I then ended up in Agile Land, this time at Yell (RIP ;) which was an Agile environment – and I really mean this! They were running a whole BPR programme using Agile with some of the best coaches I’d worked with. Needless to say, I learnt a lot about the subtleties of Agile and was even able to contribute at the end in helping develop a more scaled version of agile based around Dean Leffingwell’s Scaling Software Agility for the entire enterprise. One of the key lessons though was that you can be “too agile” – at one stage, as an Agile SOA Architect, I was continually (every few weeks) altering service designs for the Implementation teams. In the end, they (nicely) screamed “Enough! We can’t absorb this much change!”. As a result of a retrospective workshop we did on this issue, it was agreed that there should be a bit more up-front work and broader investigation in order to minimise the change to the designs they were implementing.
Coming off the pure agile role, I landed in what was at one stage one of the largest Agile projects in Europe at British Gas. Due to it’s scale (we’re talking thousands of people) there were some waterfall elements to it, but the base rule was that wherever Agile made sense, we’d use it. Again, more amazing scum masters and even some of the ones from Yell. For me, this role followed on nicely from Yell to a place that had implemented Agile at scale. There has been a lot of debate as to how Agile their approach was, but from my perspective, I was at a transition point from Waterfall into Agile and it was working, but it wasn’t easy. There were some great features though, especially where they’d applied an “Agile Mindset” to a “Waterfall Mechanism” which resulted in a more people-centric and communicative process.
For a while though, I’d been reading and learning about this new “Kanban Thingy”. It certainly had an appeal and I realised that I’d actually been using some of it’s practices in Scrum, so I decided to start injecting some Kanban tools in to Scrum (this was before I’d heard of Scrumban). I then managed to use this approach successfully in two Enterprise Architecture assessments for global organisations topped off by an Agile implementation of Agile Governance for SOA a few years ago.
As I went through this beginning of my Scrumban phase (which I’m still technically in) I had a dawning realisation: there is quite a disconnect between the work I do in Solutions, SOA and Enterprise Architecture (where I’ve used Agile, Scrum, Lean and Kanban most of the time over the last decade) and what I was reading and hearing about in “Cool Agile Land” (that’s where all the cool hipsters are xDD’ing ;).
It was then that I realised that there was just as much of a need in these areas – the problem was that this was not where the majority of “activity” was happening and that in general the mindset was more waterfall. There were however, some mavericks (who both backed and worked with me) like my self who believed that there was a middle path – we could use Agile techniques in Architecture in Big Organisations without conflict and hopefully start a revolution…
So far, I’ve only given you the 100,000ft view (as one would have with a Scramjet :)
A personal exploration of the use of Kanban, Scrum and Lean Principles to create a Service Oriented Architecture within a large organisation.
What does that really mean though? Why should you keep reading or even tell your friends? Like anything I do, this will be iterative, lean and hopefully have a good feedback cycle (it’s appreciated whenever you want as a comment here or tweet to me).
My immediate plans are that there will be two arcs:
- A personal narrative about Fellowship and hopefully some use of Antimatter in working with and transforming a team*
- Notes and guidelines around the use of lean/agile that I observe or think of along the journey
Anything could happen as I have no real plan for this except for the vision of a Skramjet
*Disclaimer: Any resemblance between the company, team and any people described in this narrative are purely co-incidental ;-)
A scramjet (supersonic combusting ramjet) is a variant of a ramjet airbreathing jet engine in which combustion takes place in supersonic airflow.
A Skramjet (ScRuM, KAnban, lEan, Just in Time) is a process in which work takes place in a SuperPersonic flow state environment.
A personal exploration of the use of Kanban, Scrum and Lean Principles to create a Service Oriented Architecture within a large organisation.
Within the last few years, for reasons that are only partially known to me, I accepted a contract at a “Self Confessed pUre Waterfall Organisation” – let’s call them SCUMO for short ;-) I know why I worked there originally – the two people who interviewed me were great – one was the head of architecture, very outgoing and inspiring – the kind of guy you wanted to work with; and the other (my boss to be) was a rather quiet spoken yet meticulous fellow – as Douglas Adams would say – “Mostly Harmless”. I mean hey, just because the organisation was waterfall, with those people and a great team (which it was) – what could go wrong?
It didn’t all happen at once though – the first project I was on was a “dream project” – great PM, good BA, tech team who knew what they were doing – I was seriously wondering what everyone else was moaning about – my world was GREAT! It was after this though, that things started to change – I was drafted in to doing a quick piece of Solution Design for a really cool sounding and disruptive project within the organisation, which of course went nowhere. Then, I was put on a project with a PM who resigned after less than a month (I later found out that this sort of thing was “normal” within SCUMO, but they regarded that as people “just not being up to the task” or whatever self-serving platitude (and there were many) that came to peoples minds) which was a disaster. We had A Business Process Primadonna (let’s call him BAPP :) who was a bully (I was very close to reporting him as such to the organsation) who had the full support of the management – “Oh, that’s just J, but he has decades of experience and we need him, so you just have to put up with iT” (which is why I didn’t report him – that would be an Express Career Limiting Move). Finally though, there was a small light at the end of the tunnel – another “really cool and transformational project” but that was rapidly pulled in to a small clique (who had all worked at another SCUMO type organisation) who were grabbing all the interesting and high profile projects for themselves – that was the last straw, the writing on the wall, so I left.
Why did I stay at all then? A question I asked myself a few times… I came to the conclusion that the negatives were temporarily damped down by the fact I had such a great team that I worked with on a daily basis. Our “boss” was fairly Laissez-faire and we had quite a number of highly competent individuals. Still, it’s hard to really enjoy things when whatever you’re working on is either totally ignored or stripped down to a degree where it’s not your original design, and what’s worse – is of lower quality and costs more!!! Again, and again, and again…
Back to the Waterfall Process – everything above is a consequence of having a totally Command and cOntrol, Waterfall-based Structure (or COWS for short). Maybe it works in the military, or in a Cotton Mill in the 18th century, but please tell me how it actually applies to modern information technology solutions..?
- Waterfall based systems have no concept of flow!
Because of all the silos and steps, they’re based on the naive assumption that things actually work this way (and perfectly). The only problem of course is that they don’t. As you look further up the “organisational stack”, you’d see people / senior managers who were inundated with requests for their time (often because people had to cover their asses or were dis-empowered), yet they had no real way to manage it (Personal Kanban anyone?). It really just depended on what was the most politically critical issue was for the day… The one time I did attempt to introduce flow based thinking, I received the rather patronising response from the CEO of “baby steps” – if they go at his rate, it will be next century before they embrace contemporary approaches… which is probably about par for the course for that organisation ;-)
- Waterfall based systems have minimal feedback
I was absolutely amazed at how little capacity for feedback was present in the system. When it was, you’d generally have to go through a committee – do a presentation and documentation for them and if you were lucky – 6 months to 6 years later (depending on how much politicking you’ve done), your suggestion would actually be implemented…
- Waterfall based systems have no compassion
Time and time again, I was reminded by “the system” about how “people” were really “resources” or “interchangeable units”. The ultimate irony was that Waterfall Systems actually focus their efforts on individuals such as BAPP and their like to create “central points of failure”. The only people who were shown “compassion” (in a weirdly inverted way) by the organisation were people like BAPP. Everyone else was on their own and could always leave the organisation as they’re not “up to the task”
- Waterfall systems struggle with early risk mitigation
Ironic isn’t it? The nature of the system dictates that Risks are left until the latest point in the development cycle as “it just might fix itself” or we could make it “someone else’s problem” or even a “problem that we know will occur later but will be outside of the current contract warrantee period” – the list goes on… At the end of the day, I was astounded with the amazing capacity of Waterfall Systems to act in the worst interests of the organisation and their clients, which brings me on to:
- Are Waterfall Systems (more) Hypocritical?
I’m phrasing this as a question as there seems to be a fair dose of Hypocrisy in the air these days no matter what the organisational and process type, but I felt an extremely strong presence at SCUMO who supposedly focused on and excelled in:
- The Customer
but they managed to implement exactly the opposite of all of those! Systems Thinking anyone? If ever there was an argument against ruthless command and control systems, this must be it. No matter how hard they tried to shove, threaten, reward or cajole people about these values, nothing worked!
- Waterfalls help and encourage Dunning-Krueger
Every organisation has their share of people who think more of themselves beyond their ability, but this organisation seemed to excel in it. BAPP was only one of many mediocre / “excellent” people there. It was not unusual to have a meeting with over 50% D-K people, in which case the best thing was to just sit back and let them pontificate amongst themselves… So we had a situation where the blind were not only leading the blind… They were reporting to a committee in a dark room!
- Waterfalls have no concept of a system!
Sounds crazy doesn’t it? Yes, they have Change Control Boards and a whole bunch of other crap, but at the end of the day, there is no acknowledgement that they are a system that is part of a wider system. Because of that, they don’t stand a snowflakes chance in hell of understanding their context or of being able to effect any change beyond minor tweaks such as changing the format of a table in a document which would often take months
- What a Waste!
Coming from fairly agile environments that had at least some element of the lean concept of minimising waste, I was astounded at the amount of waste that was produced during a project – even sometimes even in things I was working on! As a rough estimate, easily 80% of what was being done over a whole project was waste, like documents that would be read once or never by the “supposed target” audience. What sort of impact would this have on a project basis? There was one project to change the background colour on a set of web pages which took months!
- What a Waste of People :-(
Most of the people that I worked with on a daily basis were really trying to do the best for “the customer” (although they were quite often punished by BAPP’s and the organisation for doing so). Eventually, there seemed to be only 3 “attractors” which would allow people to exist in this environment:
- Give in and just play the “jobsworth game”
- Get involved in the politics and back stabbing – become a “power player”, BAPP COW or the like
- Try and change the system – this was only really a temporary state as I never saw anyone that could truly maintain this as it sets up a cognitive dissonance
None of the above result in a person permanently acting to anywhere near their full potential within the organisation. If they’re lucky, they get to do a bit when they attempt change, but as mentioned, this can never be a permanent situation.
The sad fact is therefore that most or almost all of the people I worked with weren’t happy. Were the managers happy, in their controlling positions? For the most part, no. Some people had material happiness through incentives, but most of them seemed quite miserable too. Oh, there was one person who was happy – the security guy who sat at the front desk…
In summary, I found the whole environment extremely primitive (as in Cro-magnon) and insensitive (1950’s management). Not that I’m saying that Agile is nirvana, but more that in a company that uses some form of Lean / Agile / Kanban etc…, you’re more likely to get or be able to influence people to adopt a more effective, humane and respectful approach to software development.
From a personal perspective, this just showed me how much I have learnt and grown over the past decade as the sad fact is that SCUMO are not the only company like this. What this experience gave me was the realisation that I took the red pill ages ago and that I really can’t turn back – it’s just too painful.
You take the blue pill, the story ends, you wake up in your bed and believe whatever you want to believe.
You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes.
If you haven’t already, you should probably read Enterprise Kanban Part 1, which brings us up to the end of the first week where we had enough to get started and had used a “rough Kanban” for a week. At the end of that week we ran a retrospective:
Remember, these guys had never done Scrum or Kanban before. They were self-confessed “standard process heads” who came in to this with open minds, probably as they knew that a standard waterfall approach would be meaningless in a 1 1/2 month context and probably fail miserably.
The insight shown here is great and they’re contributing well, which you can see by the fact that I happened to use a green pen and my teammates used black. Interestingly, two of the 3 improvements voted for are really reflections from them wanting to use the process more effectively. The thing this revealed to me about retrospectives, is they can give people an objective way to ask for help. Rather than say “I need to use the system better, can you please help me?”, they are able to say “We need to improve the system so it is used better”. This takes the ego out of the equation and introduces objectivity, probably reduces the violence of the dialogue and encourages working as a team rather than a set of isolated individuals who need to “buck up” and “get with the program”. We already had a “Daily Scrum” in the morning which included our PM and others (such as our CEO) if they wanted to attend as Chickens but the other improvement, “End of day catch up (internal)”, was a great piece of insight given the pace we were working at (fast!) and the fact that we couldn’t let mistakes (which would occur) stay around for long. It also gave us a chance to quietly reflect at the end of the day amongst only ourselves and set us up nicely for the next days Scrum.
Note: I didn’t tell them beforehand that we’d select the top 3 improvements, it’s just a coincidence that there were only 3.
Next, it was time to set up a new board based on what we’d learnt the previous week – ah… nothing like a nice shiny new board, ready to be populated :-)
You can see that the Project (with sticky notes) plan now has an official place, and we’ve distinguished between Waiting (as a lot of our work was information gathering, and in a large company, that means a lot of waiting) and Stalled, which is the conventional concept of something that can’t be done for a blocking reason (other than waiting). Finally, we have the Deadlines area as the others wanted a way to track these. I personally didn’t think this was necessary but knew better than to impose my view – people need trust and the freedom to learn and who knows? They could be right!
I was obviously really happy as we’d got through the first week, my other team members thought this was useful and even better, were immersed in the process and contributing deeply to it. This should not really be a surprise though, as they were both smart people with about the same amount of experience as me and were open enough to embrace something that was a bit different but would help them work more effectively.
I’ve been meaning to write about for a while about my first serious use of “Kanban”* and what’s more, it’s a usage outside of the standard patterns we hear about, mainly programming. I was working for a high-end niche consultancy to a global corporation doing an assessment and some pattern harvesting that had an extremely tight deadline. We had a small team (me and two other people) who were all highly trained and proficient, but there was one problem – the deadline, planning and tracking of our activities. We also needed a “process” that was not a Process – enter Kanban!
It just felt intuitively right – even though I’d never “seriously” used it. I had played around with Personal Kanban and I sensed that if we used no process (ie CMM1 / Ad Hoc) then near the end we’d find out we forgot things or didn’t adequately cover some risks, some of which would manifest and it would all end in tears. None of us wanted that, so I proposed to my colleagues that we use Kanban – freely admitting that I hadn’t used it fully, but that I was a Certified Scrum Master and had “Agile” experience of over 10 years. I added that we’d be doing weekly retrospectives and that if they thought it was not an effective use of our time, then we could just stop using it. On that basis I had agreement and some buy in, but we’ll get to that later…
To start off with, we did a brainstorming session for an hour, went back to work (we had no time to waste and knew initially what to do) then spent an hour at the end of the day making a Kanban
I’ve obviously had to blank out some information for confidentiality, but hopefully this is readable enough to show our thoughts, process and the fact that this was a very high level, i.e. typical EA, piece of work. One thing to note is that at the top we had “BACKLOG” but realised that we were existing within a Project Managed environment, so we had to have some sort of plan which we of course did with sticky notes ;-) Our Project Manager (who had about 5 other streams of work going on overall) was very happy as we had not only given him a realistic plan, but if he or anyone else who was interested wanted to know what we were really doing, it was all there at any time.
Also, you can see that on the right, we have a fairly typical Kanban / Scrum board and in the left is an open area where we started to group things which we’d eventually put in to the backlog. Again, the emphasis was on spending a minimum amount of time to structure and track the things that were important initially, knowing that we’d be tidying this up within a week. You can also see a bunch of stickies in the bottom left. These were things that had come out of our brainstorm, but were not currently important and that we did not have time to properly analyse and group. If you look carefully, there is a task in stalled for me to arrange sorting these out.
With that, we were off and running, with a plan which “the management” was happy with and enough structure to know what to do once we had contacted the right people in the right countries, for the information we needed. We monitored this using a spreadsheet as it was a simple process, would not take long and proved useful to show which countries were responding and in scope and which were not. And as you can see above, we also had an IRA log with sticky notes :-)
* Note: I’ve said “Kanban” as in the time since, I’ve realised there was a lot of things we didn’t do, more on that in the Meta Retrospective