February 5, 2016 at 10:49 am
FunkyDexter (2/5/2016)
3. They usually required a change to be scripted. Really? I'm in SSMS with nice graphical designers and you would me to ignore all that gui goodness and write a query to add a column to a table?I'm bound to say (at not just because I'm posting this on SCC:-)) that the RedGate tool was the best we trialled at the time but even that fell short of the "absolutely seamless" standard that us devs get to work with. Every one of the tools, at some level, forced us to be aware of its existence.
Of course, my experience was from 6 years ago and I'm willing to bet the tools have improved a lot since. I'd certainly urge anyone to at least take a look at them and try them out because, if they have improved enough to be seamless, they will improve your life hugely.
Re: #3, you can script what you change in the GUI. I'm not opposed to using the GUI, just want a log/record, and there's a little button to help with that: http://qa.sqlservercentral.com/articles/Editorial/89753/
Re: the Redgate tools, I agree. It's not seamless, though if you adapt your process a bit to the tool, it's close. They are better, but still work to be done.
Disclosure: I work for Redgate.
Follow me on Twitter: http://www.twitter.com/way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
February 5, 2016 at 10:50 am
roger.plowman (2/5/2016)
Eric M Russell (2/3/2016)
As a DBA, if I get a 2am call that a critical application is down, because a select in a stored procedure is missing a join operator, then I'm making that change myself and then notifying the development team."As a DBA, if I get a 2am call that a critical application is down, because a select in a stored procedure is missing a join operator, then I'm making that change myself and then notifying the development team."
Ok, THAT is your problem right there. How in the world did the missing join operator get through testing?
Something that breaks the system shouldn't be able to make it into production in the first place. If it did, then your development process is severely broken. Either you're trying to shave pennies by having developers be your QA team (a horrifically stupid idea) or your development team hasn't got a representative dataset to test with. (also a horrifically stupid idea).
I've heard the arguments about "protecting" real data from the prying eyes of developers and it's completely bogus. If you can't trust your developers to see the real data you can't trust your developers, period, and should never have hired them in the first place.
If it's a matter of the development server not being secure enough then *SECURE IT*. Best practices are best practices.
Sorry, pet peeve there...
Something will *ALWAYS* get through testing. We never have perfect testing. It's entirely possible the join operator didn't fail a test, or there wasn't a test written for it, or the test was poorly written. Bugs get into production. I think arguing that no bugs get to production is pointless and unrealistic. However, you can deal with this in different ways. Certainly there are times you may need to change things in production, but they should be rare, and the change should always result in a change in development. The change feeding through, a new test written (or old one fixed).
Follow me on Twitter: http://www.twitter.com/way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
February 5, 2016 at 11:27 am
roger.plowman (2/5/2016)
Eric M Russell (2/3/2016)
As a DBA, if I get a 2am call that a critical application is down, because a select in a stored procedure is missing a join operator, then I'm making that change myself and then notifying the development team."As a DBA, if I get a 2am call that a critical application is down, because a select in a stored procedure is missing a join operator, then I'm making that change myself and then notifying the development team."
Ok, THAT is your problem right there. How in the world did the missing join operator get through testing?
Something that breaks the system shouldn't be able to make it into production in the first place. If it did, then your development process is severely broken. Either you're trying to shave pennies by having developers be your QA team (a horrifically stupid idea) or your development team hasn't got a representative dataset to test with. (also a horrifically stupid idea).
I've heard the arguments about "protecting" real data from the prying eyes of developers and it's completely bogus. If you can't trust your developers to see the real data you can't trust your developers, period, and should never have hired them in the first place.
If it's a matter of the development server not being secure enough then *SECURE IT*. Best practices are best practices.
Sorry, pet peeve there...
A select with a missing join operator would be a scenario where TableA should join to TableB on columns A, B, and C, but it was inadvertently coded to join only columns A and B. So, in DEV and QA the procedures compiles, runs without error, and returns a result with some duplicated rows. Of course it's failure in the development and QA process, which is a broader issue, and that can be addressed over the following days.
But if the bad code causes a breakage in production (maybe a 2 AM ETL process that loads a staging table with a unique index on A, B, C), then the DBA is the first responder. You've got to rollback to prior release, or if the fix is obvious then just get it done, send an email to the development team and let them address it the following morning.
A more routine example of a botched stored procedure deployment would be when the developer forgets to add the GRANT EXEC ON <OBJ> TO <ROLE> statement, and the DEV and QA environments are not setup properly with the same least privilege authorization as production, and the DBA doesn't own the DEV and QA environments. Again, it's a failure in the development and release process, but it's an obvious fix and the DBA can and should remedy it as soon as it reveals itself as a bug in production.
You also have to understand that, in many (well most) organizations, the DBA isn't the IT sheriff, he's more like the security guard at the bank. He stands at the door to the vault blocking access, but he doesn't make the rules, and he can't force folks to stop breaking the rules.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
February 5, 2016 at 11:43 am
below86 (2/5/2016)
roger.plowman (2/5/2016)
below86 (2/5/2016)
No bug should be so bad it has to be fixed RIGHT NOW OR THE WORLD DIES!
I would like to live in your fantasy world. It sounds so perfect. Here in the real world bugs happen, and if not fixed at night, at 3 AM sometimes, at least our management would think the world was ending if they didn't have there data.
Oddly enough, I do live in that world, every single day. 😎 It does help that I was able to institute this philosophy from the very get go. I do understand many MANY developers/DBAs are not so fortunate. They inherited a mess and have to live in it because it's *way* too much to fix, with all those 3AM fires (and no, I'm not being sarcastic).
Sounds like a great place to work then. So be sure to post any job openings you have, I'm sure you will get a lot of people wanting to sign up when you mention that you don't have to worry about those 3 AM fires. :Wow:
Well, no 3AM calls, but lower than normal salary, no budget, and hardware refreshes every new glacial epoch...
There's always something, right? (laughing)
February 5, 2016 at 4:43 pm
Eric M Russell (2/5/2016)
roger.plowman (2/5/2016)
Eric M Russell (2/3/2016)
As a DBA, if I get a 2am call that a critical application is down, because a select in a stored procedure is missing a join operator, then I'm making that change myself and then notifying the development team."As a DBA, if I get a 2am call that a critical application is down, because a select in a stored procedure is missing a join operator, then I'm making that change myself and then notifying the development team."
Ok, THAT is your problem right there. How in the world did the missing join operator get through testing?
Something that breaks the system shouldn't be able to make it into production in the first place. If it did, then your development process is severely broken. Either you're trying to shave pennies by having developers be your QA team (a horrifically stupid idea) or your development team hasn't got a representative dataset to test with. (also a horrifically stupid idea).
I've heard the arguments about "protecting" real data from the prying eyes of developers and it's completely bogus. If you can't trust your developers to see the real data you can't trust your developers, period, and should never have hired them in the first place.
If it's a matter of the development server not being secure enough then *SECURE IT*. Best practices are best practices.
Sorry, pet peeve there...
A select with a missing join operator would be a scenario where TableA should join to TableB on columns A, B, and C, but it was inadvertently coded to join only columns A and B. So, in DEV and QA the procedures compiles, runs without error, and returns a result with some duplicated rows. Of course it's failure in the development and QA process, which is a broader issue, and that can be addressed over the following days.
But if the bad code causes a breakage in production (maybe a 2 AM ETL process that loads a staging table with a unique index on A, B, C), then the DBA is the first responder. You've got to rollback to prior release, or if the fix is obvious then just get it done, send an email to the development team and let them address it the following morning.
A more routine example of a botched stored procedure deployment would be when the developer forgets to add the GRANT EXEC ON <OBJ> TO <ROLE> statement, and the DEV and QA environments are not setup properly with the same least privilege authorization as production, and the DBA doesn't own the DEV and QA environments. Again, it's a failure in the development and release process, but it's an obvious fix and the DBA can and should remedy it as soon as it reveals itself as a bug in production.
You also have to understand that, in many (well most) organizations, the DBA isn't the IT sheriff, he's more like the security guard at the bank. He stands at the door to the vault blocking access, but he doesn't make the rules, and he can't force folks to stop breaking the rules.
True - but even then - you don't just leave the bank vault open because it's more convenient that way, do you?
I guess I can't figure out if the data is really that important or not from this conversation. It's not important enough to safeguard (it's then okay to leave code unencrypted and editable in production, change the code on the fly between cycles, maybe keep a somewhat accurate edition of what was done when etc...) but somehow all life is suppose to stop at the first sign of any issue. This is kind of binary option for me - which one is it?
----------------------------------------------------------------------------------
Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?
February 5, 2016 at 7:13 pm
below86 (2/5/2016)
No bug should be so bad it has to be fixed RIGHT NOW OR THE WORLD DIES!
I would like to live in your fantasy world. It sounds so perfect. Here in the real world bugs happen, and if not fixed at night, at 3 AM sometimes, at least our management would think the world was ending if they didn't have there data.
I only worked on live products (ref: online video game) in my career (24/7, 365 days a year). Bugs can happen anytime, but exploits are the one thing you cannot always test for. I've been woken up countless nights between Midnight and 5AM to address a critical issue with the live service. When customers are involved, you cannot wait till morning.
So, what do you do?
Members from the dev and QA team wake up. They meet the IT team--who are already there because these fools catch everything first--and they start working on a hotfix. They push it through the same pipeline, just with a bit more haste, than everyday development does to fix the problem. Test receives the update, QA goes to town, hopefully everything is given the thumbs up and it's pushed to live.
Why can't it wait?
Customers don't care about your BS.
Why are bugs not being caught before live?
Seriously? :hehe:
It doesn't matter if your the high supreme dev master or the DBA. You follow the pipeline or you're not following the gospel of the team; this is a team sport.
February 9, 2016 at 9:30 am
xsevensinzx (2/5/2016)
...It doesn't matter if your the high supreme dev master or the DBA. You follow the pipeline or you're not following the gospel of the team; this is a team sport.
It is indeed.
Follow me on Twitter: http://www.twitter.com/way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
February 10, 2016 at 2:43 pm
This discussion reflects the ones about all dev code going through the build process about 20 years ago. It ended well. Everything goes through the release process.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
February 10, 2016 at 3:14 pm
Please feel free to enlightened me.:-) But from what I've seen so far, only a couple weeks now, you can check in code in VCS but it isn't in production. So there is a set day each week that changes are deployed to production, I think Tuesday at 5 PM. So if my code has went through all of the steps and it is ready for prod I can check it in today, Wednesday, knowing it won't go into prod until the next Tuesday. Maybe the code should not get checked into prod until the day it will be deployed, but I'm sure there are situations that are going to call for checking it in sooner:-D
If code is checked in early like I mention above your 'true' version 'gold' code is not what is in VCS, it is what is in prod. Now if you are called at 3 AM for the failure you would grab the wrong version of the code. Now most times maybe that wouldn't hurt, but it could if it has time sensitive changes, new rates or whatever, or you may have missing columns or tables.
And maybe we are just too early into using this that these things will work themselves out.
I've already seen that there are about 4 branches of the code and some developers have placed there code in the wrong branches. It caused issue, but they were able to fix quickly because of VCS, which is a good thing.
My main point is if code can be checked in and not automatically updated to prod, then what's in prod has to be your 'true' version of the code.
-------------------------------------------------------------
we travel not to escape life but for life not to escape us
Don't fear failure, fear regret.
February 10, 2016 at 3:44 pm
below86 (2/10/2016)
Please feel free to enlightened me.:-) But from what I've seen so far, only a couple weeks now, you can check in code in VCS but it isn't in production. So there is a set day each week that changes are deployed to production, I think Tuesday at 5 PM. So if my code has went through all of the steps and it is ready for prod I can check it in today, Wednesday, knowing it won't go into prod until the next Tuesday. Maybe the code should not get checked into prod until the day it will be deployed, but I'm sure there are situations that are going to call for checking it in sooner:-DIf code is checked in early like I mention above your 'true' version 'gold' code is not what is in VCS, it is what is in prod. Now if you are called at 3 AM for the failure you would grab the wrong version of the code. Now most times maybe that wouldn't hurt, but it could if it has time sensitive changes, new rates or whatever, or you may have missing columns or tables.
And maybe we are just too early into using this that these things will work themselves out.
I've already seen that there are about 4 branches of the code and some developers have placed there code in the wrong branches. It caused issue, but they were able to fix quickly because of VCS, which is a good thing.
My main point is if code can be checked in and not automatically updated to prod, then what's in prod has to be your 'true' version of the code.
Your comcerns are common for folka new to managing the gold copy of code in a VCS but you are missing some key elements important to the big picture. No matter whether you use one line of code and a labeling strategy as TFS calls it or you use merging and branching to create isolation of different veraions of the code, your SCM model will define where you can go to get the gold copy in your VCS, even in those times after code is checked in and is fully tested and accepted but not yet deployed.
There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato
February 11, 2016 at 2:55 am
below86 (2/10/2016)
...you can check in code in VCS but it isn't in production...If code is checked in early like I mention above your 'true' version 'gold' code is not what is in VCS...
This is only true if you only consider the latest version is VCS. Each release (not just those that make it to production) should be retrievable as a single release. Different VCS systems use different techniques with many of them referring to labels, baseline, cuts etc. so what is in production should be retrievable using a label, for example, as should the production code for any point in time.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
February 11, 2016 at 7:04 am
Gary Varga (2/11/2016)
below86 (2/10/2016)
...you can check in code in VCS but it isn't in production...If code is checked in early like I mention above your 'true' version 'gold' code is not what is in VCS...This is only true if you only consider the latest version is VCS. Each release (not just those that make it to production) should be retrievable as a single release. Different VCS systems use different techniques with many of them referring to labels, baseline, cuts etc. so what is in production should be retrievable using a label, for example, as should the production code for any point in time.
Ok, maybe I'll understand better when we get further into it. But using the failure happens at 3 AM and I need to look at why Stored_Proc_XX failed, I would go to the server and database that contained that stored procedure for the version I need to work on. And not go into VCS and into the prod branch and grab the last version checked in.
-------------------------------------------------------------
we travel not to escape life but for life not to escape us
Don't fear failure, fear regret.
February 11, 2016 at 7:43 am
below86 (2/11/2016)
Gary Varga (2/11/2016)
below86 (2/10/2016)
...you can check in code in VCS but it isn't in production...If code is checked in early like I mention above your 'true' version 'gold' code is not what is in VCS...This is only true if you only consider the latest version is VCS. Each release (not just those that make it to production) should be retrievable as a single release. Different VCS systems use different techniques with many of them referring to labels, baseline, cuts etc. so what is in production should be retrievable using a label, for example, as should the production code for any point in time.
Ok, maybe I'll understand better when we get further into it. But using the failure happens at 3 AM and I need to look at why Stored_Proc_XX failed, I would go to the server and database that contained that stored procedure for the version I need to work on. And not go into VCS and into the prod branch and grab the last version checked in.
That can be the thin edge of the wedge...then there are tables that have changed and an index was added. Oh and have any triggers been altered? The point of viewing a release as a whole is that one doesn't get sucked into the changes one by one. You get the one and only picture. It is less clear than with compiled code but I have seen DBAs' analysis killed by a thousand updates.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
February 11, 2016 at 11:22 am
below86 (2/11/2016)
Ok, maybe I'll understand better when we get further into it. But using the failure happens at 3 AM and I need to look at why Stored_Proc_XX failed, I would go to the server and database that contained that stored procedure for the version I need to work on. And not go into VCS and into the prod branch and grab the last version checked in.
I understand. And that seems to make sense.
However, here's my argument. You grab the proc from the server. You look through it and see an issue. You change it and CTRL+E.
Now you realize that wasn't the issue, or something strange happens. You decide to go back and undo the change. Are you positive you'll always undo this correctly and completely? What if you end up making a few changes over time. I've seen DBAs and developers (including me) do this live.
You decide to rollback to the version that was on the server. What is it? Where is it? This is why you go back to the VCS first. Then you have a copy. If you make a change, you should commit that right away, whether it's good or bad code. It went into production, so it's code that should be there.
A VCS gives you the ability to look back and see what was there.
Follow me on Twitter: http://www.twitter.com/way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
February 11, 2016 at 2:30 pm
That scenario is one reason I implemented Schema Change triggers on my last set of servers. We had a shared Dev environment and it was really, really easy to miss something getting checked in for far too long. Those triggers helped us catch the changes as we reviewed them daily and made sure they were checked in.
Anything that was pushed through our process (SSDT/VSDB) didn't fire the triggers as they were in source control. Not a true safety measure, but really useful to see what had been changed and know who made the change.
Viewing 15 posts - 76 through 90 (of 99 total)
You must be logged in to reply to this topic. Login to reply