Thursday, October 20, 2005

Someone pinch me, I think I'm having a nightmare!

The day started out innocently enough, It was the last day in a small development project to implement some changes to a long standing system. The changes were fairly well documented by the client so it was simply a matter of implementing them. Yesterday we'd rolled all the changes from Development to Testing without and error, which was a little surprising as it involved implementing 9000 lines of SQL script. Testing went off fairly well everything functioned as it was intended and we were given the green light to roll into Production.

The downtime for the server was scheduled and we went about upgrading the Production environment. It was after this it all went pear-shaped, and none of it was to do with system. I was first approached about one of the changes the had been made. It functioned just as intended, but it turns out that no one asked the person using that aspect of the system if was what they wanted. They actually required completely different functionality. (The conversation is paraphrased as it really needs context in word for word form)
Client: "I've spoken to User A, they can't really use the system now. Is it possible for you to change that part of the system so it now triggers of several other aspects of the system?"
Me: "But I just changed that aspect of the system from being triggered off something else to not being triggered at all. Changing it again would impact all the aspects you want to trigger off, it would take some time to make all these new changes."
Client: "Oh, so you could have that in before you go?"
Me: "I'm sorry but no, it took a day to design, build and test the last change, it would take a similar amount of time"
Client: "Really, oh well how complex would it be just to put that change back to the way it was and leave the rest of them?"
Me: "Well only a couple of hours, but..."
Client: "Ok so can you change it then, let me know when you're done"

So I went on my merry way taking out the some of the code that I'd just put it back the way it was. Really this should have warned for what was to come.

The rest of the afternoon was confirming the design with the Head of the Department, and inspecting the other enhancements that have been included as part of the update. Once this was done the application was republished back to the organisation. It came to 4:45 and I started to pack up and head home. Just as I was about to put my laptop in my bag when I had another visit from the client.

Client: "So how much work is it to roll back the system to what it was before started?"
Me: Chuckle (Honestly I thought they were joking at this point it was quarter to 5!)
Client: "No, Seriously how much work is it to roll back the system to what it was before started?"
Me: "Why? What's wrong with the changes?"
Client: "Nothing, the changes are just what we wanted."
Me: "So why do we need to roll back?"
Client: "Well, we haven't discussed the changes with all of the users of the system and one of the sections is worried how it will change the way they work. Also with changes to the screen design the Head of the department is worried that the users will get confused when they see it."
Me: "But I thought you needed the changes to the system because of they changes to the way your business works?"
Client: "Well most of the sections do but we didn't really look into how Section E was using the system. So how long will it take?"
Me: "Well it will take at least a few hours..."
Client: "And then we'll just be able to turn on the changes when we want them back?"
Me: "Well not exactly... When I restore the system all the changes will be lost. I can put them in your test system, but there will be time involved in migrating back to production..."
Client: "Ok can you start right away?"
Me: "I'll have to change my flight back and obviously there will be extra charges for the time involved, and I'll need on of the IT guys to stay back, because I can't access the systems on my own."

So I started on putting the system back to it's original form. Only to be interupted again about 20mins later, it's not 5:40pm.

Client: "Have you finished rolling back yet?"
Me: "No, I've only just started the backups, they take about 40mins..."
Client: "Great, can you just leave it the way it is?"
Me: "What?"
Client: "Well we don't have the budget approval for any IT staff to stay back, so can you just leave it the way it is and disable it so no one can access it?"
Me: "Well I can but that would me you'd be with out a system?"
Client: "That's fine, we'll just leave it offline until we've discussed it further."
Me: "How will you function without the system?"
Client: "Well, can you leave it open for the administrator and then who ever needs to use the system will just come and ask them."
Me: (at this point I'm over it) "Sure,...... done, no one but the admin can access it. I'm going to go. Let me know when need to make any more changes."
Client: "Thanks."

So after 4 days of development the Client went from a system that wasn't quite fitting their new processes, to a new system, back to the old system and now no system at all.

Did I just fall into a Dilbert comic?

1 comment:

Tom N-S Technical Blog said...

Oh my god,

the dummest bunch of no-hopers I have read about all day.

What a complete waste of time, you are too cool headed for me I would have had an outburst or an "incident" if that had happened to me.