Alex,
Nicely said. I'm very much geared to agile development, but have also seen companies buy into agile but still fail because of poor design, or because the architects failed to stay ahead of the developers. I've also seen projects where the decomposition of tasks were based upon business requirements that frequently failed to recognize points of commonality, meaning that you would have teams build fundamentally the same objects from scratch that had been produced by a different team because the people setting up the use cases failed to recognize when they were reinventing the wheel. There's also a tendency with agile to want to segment teams, and to end up becoming siloed.
In many respects I think the problem comes from the fact that there is a considerably thinner corpus about agile design than there is about agile development. Agile design means building and refining data models, building POCs, testing use cases at a small scale before implementing them at a larger scale. I think this is part of the reason that skunk work projects can often be so effective - you are reducing the extent of your communication graph dramatically, and can keep the team down to experts (skunk-work projects usually tend to draw the best and brightest, even if only part time). Not only can you prune the tree of failures more efficiently (and at much lower cost), but often times you can work out the details of algorithms, more readily explore edge cases, and can ask potentially embarrassing questions at far lower political costs. This is also a key part of agile - design small, but with scaling in mind, then build from the inside out.
This also affects translating business requirements into working code. Personally I believe that ascertaining business requirements is a core design process, and should happen before the first line of production code is ever written. Prototype coding is part of the architecture process and doesn't count, because this also assumes that the prototype itself is not the final product - it's explicitly designed to be a throwaway. I've seen too many instances where business requirements were coming in well after final code had been written for certain portions of the application, and the new business requirements forced a radical rewrite of the application code-base.
Having said that, requirements do change over time. Innovations force rethinks, some key piece of infrastructure doesn't live up to expectations, costs become more or less constrained, or the external business model changes. You factor that into the equation and determine the cost of instituting change, then eat those costs or decide that the requirement was not really that worthwhile.
What you don't do is go directly from business requirement to coding, unless that coding is the prototype.
I also believe that before any coding, even prototype coding happens, you should put together the developers, architects, UX designers and key testers into a room for a couple of weeks with a lot of paper, white boards, markers cameras and video equipment, and manually duplicate the actions that you are expecting your application to do. The testers should be as sadistic as possible, trying every potential way to foul up the system that they possibly can, elucidated in words and pictures. No computers anywhere - even the notes are taken manually. After two weeks of this, you can bet that not only will you have a pretty decent understanding of the system, but you will also have developed a powerful team that can work together and knows the pain points that others have, and it will tend to break down the "well the architect said so so this is the way we implement it" mentality that I think tends to be a real problem for larger projects in particular.
Waterfall has its place, and so does agile, but any methodology as often as not gets in the way of solving the problems because problems, by their very nature, tend to be unique from one another, even though they frequently rhyme.
Hiya,
Yes, although I must point out that there's some differences between
agile approaches, SCRUM being just one of many options. SCRUM is a
methodology, and works best if you have a rough idea of what you're
doing (at the risk of your mentioned technology diversity; only
quality people at the helm can fix that, and they need to be part of
the process just like "normal" developers), but terrible to do any
sort of planning or requirements gathering through. User-centred
design is agile at its core, for example, and there's a bunch of
varieties over the theme.
A few years ago I worked as a consultant in a large government
organisation, and within the constraints of the contract in place we
did everything agile, and I think it went terribly well ... as long as
some thinking and planning had happened by smart people upstream from
where developers implemented and fixed integration. SCRUM, for
example, is supposed to make sure that the planning happens a few
iterations before, so that the architect swims a little ahead (which
is why it's called upstream :) ) *while* being part of the
implementation team. Not all architects can work like that, nor can
not all developers, either. You need the right kind of people. As
always.
And that brings up the skills of the people who's going to inherit the
system once the savvy consultants have left. Most places I've worked
(in Norway, mostly) have negotiated X amount of billable hours for
years after completion, to both fix problems and to train the regular
staff, but in financial dire times, this is the first that gets
chopped. "Don't worry, Barry will figure it out. Somehow. Before, you
know, the launch."
Anyway, the short point is; if you know how to be agile and if the
people involved knows how to be agile and if the customer knows what
it means to be agile (or are willing to learn and take it seriously),
it can work miracles. The weakness is that, just as in Waterfall or
any other model I've ever seen in place, people of various levels of
skill is involved. Some times lower-skilled people thrive once thrown
in the agile pit; other time it makes them even less productive. And
then, there's politics. And social skills. And ... aaaargh, now I
remember why I wanted out, to start over, to do it in smaller chunks.
Big government and enterprises have so far not overcome the problem of
people being people. We put constraints in place that drag lower
skilled people up to a medium of sorts, but it also pulls high-level
people down to a frustrating non-productive level. And what happens
for skills, happens for projects.
Cheers,
Alex
On Thu, Nov 28, 2013 at 11:35 AM, Michael Kay <mike@saxonica.com> wrote:
>>
>> I would be interested if yourself or others have anecdotes of Agile
>> applied to large, complex development projects for tailored software
>> solutions. We may be going a bit off-piste for xml-dev here so please keep
>> it relevant to the list if you do have feedback.
>>
>
> I had some slight involvement a couple of years ago with a successful system developed for online scientific publishing (with MarkLogic at the back end). I don't know the project budget but I would guess over £10m. TDD and Agile played an important part. Although successful, it made me aware of some of the weaknesses: for example it was the only project I have ever seen that was over-tested. In the final stages we were doing performance tuning and it was seriously difficult because (a) internal changes that didn't break any external functionality caused dozens of tests to fail (because they were testing non-visible behaviour), and (b) running a commit took well over an hour.
>
> The other problem with the project was unnecessary proliferation of technologies: the XML pipeline took data through XQuery, JDOM, XSLT, DOM4J, freemarker, ... inhibiting reuse of code and causing much serialization and reparsing. Basic cause was that when software components are free, it's quite difficult to stop each developer choosing their own favourites.
>
> Michael Kay
> Saxonica
>
>
> _______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
--
Project Wrangler, SOA, Info Alchemist, UX, RESTafarian, Topic Maps
http://shelter.nu/blog | google.com/+AlexanderJohannesen
http://xsiteable.org | http://www.linkedin.com/in/shelterit
_______________________________________________________________________
XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.
[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php