March 28, 2009 at 12:32 pm
Comments posted to this topic are about the item The Visit of the Recovery Testers.
Best wishes,
Phil Factor
March 28, 2009 at 2:30 pm
Our new product has the business logic on servers separate from the sql server 2005 database. Virtually no use of stored procedures. Not my design. C# .NET coding with web and application servers. Events are created in a table, which result in sql transactions created by an object. So I'm curious about this statement -- can you dumb it down a bit for me?
"Business data must never be potentially volatile unless the business processes are atomic, consistent, isolated and durable. The cool distributed business layer we’d constructed was useless without a resilient messaging system that could manage groups of transactions that represented business processes, or ‘Sagas’. "
March 28, 2009 at 10:28 pm
HAHAHAHAHA.
Phil, you are a very brave man.
And (syntactical misue of AND, must review that) indianrock, i'm guessing you could do a lot worse than to watch this thread very closely, it's likely to be instructive in the extreme.
I've got sympathetically righteous abuse (maybe abuse is too harsh, perhaps condesencion) quite recently here on sqlservercentral for a closely related subject.
Disaster recovery is NOT ABOUT THINGS YOU CAN PREDICT.
Wha?
That doesn't make sense!
Disaster recovery is about every link in the chain being the potentially weakest link and having a recovery situation for every link, then for every two links, then ...ouch... permutations are NASTY.
Back soon, gotta go pick up my son's friend from his weekend cow milking job ...
Peter Edmunds ex-Geek
March 29, 2009 at 3:38 am
It's OK, Peter; I don't mind people disagreeing with what I say. Any ideas that can be brought to bear against the problems of application resilience in the face of Cruel Fate are welcome, as far as I am concerned. The point I'd want to defend to the hilt, though, is that Cruel Fate takes a malicious delight in the role of Celestial Recovery Testing. A lot of hackers seem to feel the same way.
I could keep you amused for ages with real-life disaster stories. One that happened to me was that, one day, we had a contractor in the server room doing some routine network wiring. He got a bit bored and switched on his radio. The battery was dead. 'Tut', he said to himself, 'but no real problem, as it had a mains cable. Dum de dum. Can't find a socket. Hmm. Here's one, something plugged in, doubt if it is important. Soon be done, I wonder what's for tea.'
You guessed it. The radio's power supply was uninterruptible. The rack's supply was gone.
He was an amateur recovery tester.
As far as business layers go, IndianRock, you can nowadays write them in whatever you like as long as you use, and design your system around, Service Broker properly, as it was designed for precisely this sort of demand for resilience. You can get away with quite a lot with any distributed transaction mechanism, but there is a lot to be said for ensuring that all messages are non-volatile, and for as much processing as possible to be asynchronous. It is a deep subject to be sure.
Best wishes,
Phil Factor
March 29, 2009 at 5:00 am
About transactions, there is good support for writing transactions in the middle layer. Starting a transaction, call a few stored procedures, if it goes well, commit it.
Edited, good article thou, nearly gave me the chills hehe.
March 29, 2009 at 5:47 am
IceDread, Yes, you're doing things pretty safely there. If you are calling a series of Stored procedures within a transaction, which you can then roll back if necessary, then the processes are really taking place safely within the database, and you're just scripting the database actions. I always choose to use this route when I can. SQL Server is a very resilient system.
But what I'm talking about here is where complex 'volatile' business objects are maintained in a business layer and acted upon independently of the database to perform business processes, before the changes are committed, or where the application involves a number of distributed databases and middle-layer servers. It is where applications grow in complexity that the more insidious problems creep in.
Best wishes,
Phil Factor
March 29, 2009 at 6:15 am
Two weesk ago, we had technicians in to service our database server hardware - which was sold to us by one of the MAJOR suppliers of "fault tolerant" hardware in the world (hint - 7 letter word that starts and ends with "S"). They were replacing the RAID controller with a new one. We brought the server back on line and the database back up - and the brand spanking new "fault tolerant" RAID controller failed - without switching over to the backup "fault tolerant" RAID controller. WTF.
Result - a 35 gigabyte database - used as one of the central business critical systems for our company - was corrupted. We had to restore from the backup and lost 8 hours of production data because we did not have a failover server for our fault-tolerant-dual-redundant-world-class server. At least we had the backups.
Bottom line - you can never be too paranoid.
March 29, 2009 at 11:15 am
Nice one, Phil.
Any support for transactions in the middle layer, should use the database. The issues of running it in the middle layer could potentially be communication issues in the link and then a "stuck transaction" in SQL. Other than that, this should get you ACID integrity.
The problems that developers can get into it trying to handle transactions outside of SQL. If they do that, often the rollback is an "undoing" transaction. But what if the middle box fails somehow in the middle of work? Not a crash, but perhaps a bug/logic failure? then the "undo" transaction doesn't run. What about triggers that get added to the table later? (I know, I know, this never happens) They don't necessarily have an "undo trigger" added as well.
Some amount of business logic in the middle layer makes sense to me. Just not transactional logic that doesn't rely on the end servers (db, XML, whatever) enforcing the transaction across all entities.
Follow me on Twitter: http://www.twitter.com/way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
March 29, 2009 at 7:24 pm
The loosely coupled transactions in the Microsoft platform require a resource manager SQL Server is one of the resource manager which implictly means with correct business object modeling transactions can be propagated through application layers. These works most of the time if the developer is skilled and not using MSMQ which require MSDTC. I once talked a developer who was propagating through many applications and I told the person I am not aware of such implementations. All others just require ADO.NET and SQL Server implemented as resource managers. If you are using Oracle you need 10g or 11g or MSDTC is required like MSMQ and SQL Server 2000 and below.
Kind regards,
Gift Peddie
March 29, 2009 at 8:14 pm
I just knew this was gonna turn into a seriously coooool thread. 😎
Peter Edmunds ex-Geek
March 29, 2009 at 8:45 pm
Was working for Air New Zealand as a sysprog, ops called and said, "The production mainframe has stopped, got a strange message here on the console...".
IBM CE was on site at the time.
Leaves an ineradicable flag in your career timestream when you see
"IEA999A Access lost to master memory."
on the production console.
CE was doing a very minor service upgrade on the 3084. Probably involved one card and a yellow sticker.
Oops, that wasn't the developer's CPU.
...
🙂
The seriously impressive part of all this was the call from the IBM 308x development team in the US that arrived before the call from the the Data Center Manager inquiring about the problem. That was because of the mainframe calling home and saying something to the effect of "help, i'm screwed".
Any criticisms I get about IBM and the mainframe environment they built, well...talk about integrity...the whole SNA VTAM/NCP thing was just so seriously cool, anyone out there realise that VTAM, the comms protocol manager for IBM's mainframes was (maybe is) just one big state machine implementation? and...
...this is fun but I gotta go, back in a bit
Peter Edmunds ex-Geek
March 30, 2009 at 12:49 am
I used to work for an investment bank, and they put all their business logic in stored procedures. Imagine my suprise upon meeting these 5000 line monsters......oh the pain.
Problem is, a lot of younger programmers have never developed in C/C++, and so don't really get functional decomposition, because let's face it, any idiot can program in Java/.Net (including me).
March 30, 2009 at 1:40 am
Steve, while it's easier to manage if doing the transactions in the db itself and less risk it's not always doable. Not when your sitting on different systems that needs to do their things and if one fails everything in the different systems and databases needs to be rolled back.
When calling a db, I prefer the save way of letting the clr do some extra work with saving the states and using try catch and finally blocks and thus if something unexpected were to happen I can always roll it back. All thou problems could still arise, like loss of connection, but to be 100% save one has to do a lot of work that might not always be needed / wanted depending on the customer.
March 30, 2009 at 8:08 am
This topic really hits on one of my pet peeves. The project I'm currently working on has a number of people on the development side who think databases are bad and slow things. Furtunately they do buy in to only accessing the database through stored procedures instead of directly accessing tables, but they still get upset with me for saying I'm putting too much business logic in the database.
The unfortunate part, is that some of the older parts of the system before I joined the project were poorly designed, in a chatty way for both read operations and write operations. It not only creates potential performance issues, but has in some places caused inconsistent data to be saved to the database. Then I just laugh and tell them again why this is poor design.
The way I look at it isn't a matter of "business logic". Everything about a computer system is business logic. A database with no business logic in it would just be 1 table with 2 columns, attribute name and attribute value. The way I see it there is data logic, validation logic, and user interface logic. Too many times I've seen people put data logic, such as filtering data, or doing the equivalent of joins, in the middle tier. Management has tried to explain this by saying that the problem with databases is that they are the slowest part of the system, and it's easier to scale out application and web servers than database servers. But our database is already logically partitioned horizontally because each client of ours has their own copy of the website and own database. I wish I had more persuasive information to be able to give to the .Net developers and management to explain my position, because inconsistant data and excessive network traffic are still too frequent in our system.
March 30, 2009 at 8:38 am
One place I used to work, they refuse to put business rules in the database, as much as possible. Prefer to have them in a "business logic layer". One day, they discovered they had accidentally turned off a whole bunch of critical rules and had all kinds of data corruption issues, and apparently the "turn off" had been at least six months ago. The "blamestorming" event this caused was impressive in both magnitude and imagination! (The eventual conclusion in IT was that it was the user's fault, for asking them to allow users to bypass the omniscience of Google Maps when entering addresses.)
Another place, as much business logic as possible was built into the database. The applications were pretty much just eye-pleasing displays for stored procedures. Very, very thin clients. We had a lot of network outages, power problems, even server crashes, over the years, but data was never lost nor corrupted in all of that. We even built in a final proc that would check order data for consistency with usual business rules, after all was said and done, and had levels of "suggestion", "double-check" and "no way" for various things it would find. Cut down refunds by an unbelievable amount.
So I'm a pretty firm believer in having logic in the database as much as possible.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
Viewing 15 posts - 1 through 15 (of 18 total)
You must be logged in to reply to this topic. Login to reply