What lines of enquiry about the success or failure of IT projects can we get from a quick dip into the literature? Here are some preliminary findings.

Emam et al (2008) say that results from their global web survey of IT departments in 2005 and 2007 “suggest that, although the overall project failure rate is high, word of a software crisis is exaggerated“. They report that IT project cancellation rate is in the range 11.5 to 15.5 percent and that  “between 16 and 22 percent of delivered projects were considered unsuccessful on the basis of their performance. … The combined rate of cancelled plus unsuccessful projects was between 26 percent and 34 percent.” For delivered projects (see figures below) they give overall success rates and indicate which project outcomes were perceived to be the most challenging.

But what do we mean by ‘success’ for an IT project? Emam et al used six criteria in their analysis: user satisfaction, budget, schedule, quality and productivity. Agarwal and Rathod, (2006) explored  “difference[s] in the perception of the meaning of success in the minds of people who evaluate the project performance“. They compared viewpoints external to the project organisation which, they suggest, tend to focus on cost and time for inferring project success, with those from inside it and they examined the views of one such internal set of stakeholders, namely programmers / developers, project managers and customer account managers. This group judged functionality and quality of the project output as the highest determinant of success, a finding that Agarwal and Rathod suggest should be studied further to see how it fits with external objectives like customer satisfaction and customer happiness.

One way of approaching this is shown in the figure at left. In this model, business strategy aims for a desired medium to long term impact by setting programme goals in the form of medium-term outcomes. The programme defines projects whose short to medium term objectives are the project outputs. When delivered, project outputs combine to enable the programme outcome(s).  Over time, the outcomes bring about an impact.  There are different types and numbers of stakeholders in all parts of the model (indicated by the stick figures), each having a different perspective.  They measure success by the value they obtain and they perceive value differently because of their varying viewpoints; hence the measures of success can vary substantially amongst stakeholders. This line of reasoning leads to the proposition that a project’s success cannot be judged satisfactorily at the time it delivers its output. An apparent failure perceived by stakeholder directly involved at that time (perhaps due to cost and time overrun) might later prove to have been key to a successful outcome and/or impact. Or vice versa, of course.

What about risk management? Kappelman et al (2006) identified a list of early warning signs of IT project failure and found that a dozen of these IT project risk factors were most important during the early stages of a project. They suggest that knowing about and paying attention to these early warning signs early in the life cycle of a project will increase the probability of success. But is that happening? In their 2009 paper ‘Does risk management contribute to IT project success? A meta-analysis of empirical evidence‘, Bakker et al argue that “over the last 10 years, much has become known about what causes IT projects to fail. However, there is still very little empirical evidence that this knowledge is actually used in projects for managing risks in IT projects.” They conclude that IT project managers could have more impact on project success by paying attention to project risks than by following the steps prescribed in the risk management process. They also point to some areas for future research:

• Given the indication that risk management can only be effective in specific project situations, further research might determine these specific conditions in the context of IT projects.
• “It would be interesting to combine the relation found by Cooke-Davies (2000) between risk management planning and a timely delivery of the project with the work of Weick and Sutcliffe (2007), who discuss awareness creation and attention shaping as conditions for stakeholder behaviour in uncertain situations. In this view, risk management contributes to project success, because the stakeholders are aware of the fact that there are risks, on the basis of which they adjust their expectations and behaviour accordingly.”
• “The majority of publications that relate risk management to project success refer to the traditional time–budget–requirements definition of project success. However, this approach is not in line with the view presented by other literature that project success entails more than just meeting time and budget constraints and requirements. Project stakeholders may use various project success definitions (Agarwal and Rathod, 2006). Therefore, the contribution of risk management should be considered in relation to a broader definition of project success.“

ChongWoo Park et al (2009) note that “reluctance to report the actual status of a troubled project has recently received research attention as an important contributor to project failure“. In their paper they “examine the effects of both situational and personal factors on an individual’s reporting behavior within the rubric of the basic whistle-blowing model adapted from Dozier and Miceli“.

Rich et al  (2013) report that “large-scale information technology (IT) projects experience higher failure or abandonment rates than smaller IT projects … From a systems perspective, the problem appears to be related to dynamic and repeating management failures with an embedded project management model. … even a relatively small but persistent introduction of new requirements has a dramatic effect on project overruns, setting the stage for abandonment and restart.“.

Williams (2011) responded to studies performed by research corporations such as the Standish Group and Zells which attribute most IT project failures to “the lack of effective communication between members of the business and technology communities“. She examined communication techniques to find out whether they contributed to projects completing on time, on or under budget, and on scope. The requirements elicitation techniques were: rapid application development (RAD); joint application design (JAD); and group support system (GSS). Her results showed that “there were minimal differences in perceived effects of JAD and RAD techniques on IT project success. GSS on the other hand was perceived as a major catalyst in achieving IT project success.”

Sage et al (2013) use research on project failure to advance ideas about the use of performativity. This might offer a useful line of enquiry for me but I can’t decipher their social science jargon well enough to judge.

References

Karel de Bakker, Albert Boonstra, Hans Wortmann, Does risk management contribute to IT project success? A meta-analysis of empirical evidence, International Journal of Project Management, Volume 28, Issue 5, July 2010, Pages 493-503, ISSN 0263-7863, 10.1016/j.ijproman.2009.07.002.

ChongWoo Park; Keil, M.; Jong Woo Kim; , “The Effect of IT Failure Impact and Personal Morality on IT Project Reporting Behavior,” Engineering Management, IEEE Transactions on , vol.56, no.1, pp.45-60, Feb. 2009

Rich, Eliot and Mark R. Nelson. “Understanding the Context of Large-Scale IT Project Failures.” IJITSA 5.2 (2012): 1-24. Web. 1 Mar. 2013. doi:10.4018/jitsa.2012070101

Sage Daniel, Dainty Andrew, Brookes Naomi, Thinking the ontological politics of managerial and critical performativities: An examination of project failure, Scandinavian Journal of Management, Available online 20 February 2013, ISSN 0956-5221, 10.1016/j.scaman.2013.01.004.

A useful few days at Warwick University for tutorials with my group. Day 1 was a session with the whole group . We started by asking each person to explain the purpose and design of their project so far and then letting the group discuss and critique the approach. Many good points emerged and people were reassured to learn that similar worries and concerns were being experienced by everyone. We concluded by talking about ethics and clarifying the process for obtaining ethical approval for people’s research.

Day 2 was spent on individual tutorial sessions. My aim was for the student to clarify to what extent they had mapped out the context of their research topic and hence get a feel for how much of it was still, for them, uncharted territory. As a by-product I wanted them to identify the key concepts involved and begin assembling these into a theoretical framework along the lines we had discussed in the November tutorials. Some of the results are shown below.

This model helped us to unpick the meaning of sustainable project management.

exploring the concept of sustainability

It focuses on the sustainability of project outputs rather than of the capacity to manage projects. Points of particular interest were: (a) the concept of sustainability ended up being related to impact and perhaps outcome, ie to to programme-level concepts, rather than to the direct project output; and (b) ‘efficiency in the use of resources; is an example of a variable concept, ie one which has a range of values, quantities or amounts.

This model explores how stress affects project performance.

Exploring the concept of how stress affects project management

It very helpfully draws together ideas contained in Beehr & Newman’s model (which I don’t much like) and the insightful model by Harrison. Some key insights to emerge were: (a) stress is caused by a misfit between a person’s self-assessment of their capability and their understanding of the nature of what they are required to do, neither of which are necessarily correct perceptions; (b) linkage to project performance is via the effect of strain on the actual (objective) performance of the project manager as compared to the performance required to meet project needs; and (c) methods for managing stress can be preventive or adaptive (reactive).

This model explores sources of risk in the context of medical devices.

Concepts relating to the development of medical devices

This model is beginning to frame some of the concepts relating to project success.

Early thinking about the meaning of success

On Day 3 we took the opportunity to listen-in on what is being taught in the Research Methods module.