October 12, 2007 at 5:16 pm
I am a development DBA at an agile software company. I have been there about a year. This is my first experience in agile development. I work in an office on 40-50 web developers, and two other warehouse DBAs. I do OLTP. I was a production DBA before this. I consider myself a mid level DBA.
I also work for a web developer with minimal database experience (as recently more of my supervisors have come from striclty web backgrounds, and less and less database backgrounds).
I am very detailed oriented, very organized, extremely linear, mathmatical of course, and I usually think things through throughly before making changes/decisions. I have found these skills really helpful in the past towards my quest to be a great DBA.
However, in this object oriented, sql 2005, agile movement, these skills seem to be more of hinderance than a blessing. I am trying to be more agile, less rigid DBA, but do not really understand it, or where to start.
What do I read? Where do I go? How do I make this jump? How do I maintain a relational database, integrity, constraints, normalization etc., but be able to change on a dime? How do I find a middle ground with the web developers, who stop just short of saying they don't need a database anymore, but a large XML repository will do.
I feel as if I am drowing in agility. Can anyone assist me?
December 18, 2007 at 8:59 am
All the articles I have read on agile development acknowledge that the database layer is the one that acts as a bottle neck.
Agile development is about completing a single project as a series of mini-projects with the key learnings and outcomes from one mini-project being applied to and influencing subsequent min-projects.
The problem with agile development is that the people who run agile developments need to be skilled practioners and be able to identify the fundamental core step for a project. As data is the foundation stone of a data driven web site that core step will involve you.
In a traditional project you may have time to specify the schema including all constraints, all stored procs etc.
In agile development you probably won't.
The best you can hope for is get the table structure in place and the stored procs that the developers need then go back to implementing all the constraints and the detail you would normally do. I personally detest this method as it feels like trying to put the roof on a house before the foundations are in place or even the supporting walls.
A tip I would use is rather than try and code up a fully working stored procedure get the shell of it in place so the arguments it requires are defined and for retrieval purposes the output structure is in place. You can then go back and fill in the actual workings of the stored procedure later.
One core requirement of agile development is close communication between all parties so provided you communicate what you are doing you should find some support for the approach.
May people think they can do agile development but there are quite a few wing it and bodge it merchants who are passing off cowboy coding as agile.
December 18, 2007 at 9:14 am
I have absolutely no experience with agile development as such, so take my thoughts with a grain of salt.
It sounds like you should by and large be able to have a solid, normalized, well formed database that if well designed will be able to support an array of front end and manipulation work. In short, from my non-experienced perspective, my understanding of agile development is that if you have a truly solid back end database the fact they are doing agile software development on it or more traditional development is almost irrelevant.
To the extent you do need to participate in the agile piece, it might help to get additional servers that all store the core data but can do it in different ways with different constraints to meet the requirements of the agile development of the day.
---
Timothy A Wiseman
SQL Blog: http://timothyawiseman.wordpress.com/
December 18, 2007 at 11:31 am
Just curious, do you have a project manager or project leader to co-ordinate the projects ?
December 18, 2007 at 2:02 pm
Thank you all for your advice.
Since posting this, I have read a book by Scott Ambler called Agile Database Techniques. I highly recommend this to anyone who finds themselves in a similar situation. Although, I do not agree with all the things in the book, the one great thing about this book is that it does explain the agile perspective of programming and how it directly relates to the database programmer/administrator.
To answer your questions, we are a team of four: two programmers, one programmer/manager (who handles programming, program and product mgmt, technical lead and architect - all in one), and me (the dba).
The product was actually designed by a larger group of web programmers that have since split into two groups. I was hired on last, after the product had gone into production. When I first got here, the database was far from normalized. Many tables did not have primary keys, foreign keys, nor was one index created other than the few primary keys that did exist. I have spent my time cleaning up data due to lack of referential integrity and contrainsts and 'refactoring' sections at a time into normalized structures (adding pk's, fk's, constraints and indexes). This gets tricky because like I said, this product is already in production and application code, middle layer, and back end schema and stored proc changes need to coincide. This is further complicated by the fact of the short release cycles (two weeks code time, one to test) make it hard to make big changes at once. But it is coming along....
When I first posted this, it was more the philosophical perspective shift that I was experiencing that threw me off the the most. My background is in large waterfall companies where there was a great deal of emphasis on data, integrity and design. In this new agile world, I am not finding that the case. Again, Scott Amblers' book really helped me to understand this new dynamic.
December 19, 2007 at 7:56 am
so when the application developed using 'agile' techniques does not perform, do they blame the database and tell the client DBA to fix it?.........:D
---------------------------------------------------------------------
December 19, 2007 at 12:33 pm
so when the application developed using 'agile' techniques does not perform, do they blame the database and tell the client DBA to fix it?.........:D
I sometimes think that companies hire DBAs for the same reason that flatulent people always seem to have a dog to blame.
December 19, 2007 at 1:56 pm
Wow, I'll try to keep the ranting to a minimum. Erin, you as the development DBA are the data storage and retrieval expert. That is the value you bring to the team. The cautious attitude that you have is what is going to make the projects you work on a sucess. Don't feel like you are the bottleneck even if you are. David suggested some great ways to increase your development workload throughput, but stay cautious and stick to your guns. Its good sign that your company is open to including DBAs in the agile development process. As you know first hand, when a data storage and retrieval expert is not used, you end up with garbage. This is probably why the database is often viewed as the bottleneck. Propertly designed relational, normalized data structures are what will allow the applications to be flexible. Designing them any other way will prevent the very thing they are after, flexibility.
David hit the nail on the head with the foundation/roof analogy.
December 20, 2007 at 5:39 am
Agile development is all about recognizing you won't get it perfect the first time, and the time and effort wasted on waterfall style high ceremony up front design and documentation is better spent “doing” (and redoing till you get it right.)
One of the first agile development approaches was "XP" (Extreme Programming – which predates the Windows operating system of the same name.)
I thought this post, and the ensuing discussion that followed, captured the essence of it very well:
December 21, 2007 at 12:43 pm
The fundamental ideas about Agile Development XP, RUP, AUP etc are good, it is the practice that worries me.
In Britain, industry wants a cheap temporary fix that will last for ever, which is why our trains don't run at Christmas, sometimes don't turn up at all, are dirty, cost an arm and a leg, and are bloody terrifying if they go over 100mph.
The principle of refactoring work through further interations is something that I have rarely seen put into practice. Once you have delivered something that is it, over and done with and on to the next project.
The move to an itterative approach means that the business as a whole has to make the shift in approach. It can't be limited to a small number of developers.
Viewing 10 posts - 1 through 9 (of 9 total)
You must be logged in to reply to this topic. Login to reply