October 6, 2010 at 7:50 pm
timothyawiseman (10/6/2010)
I think it makes sense to decide, if not table by table, then certainly database by database whether you want to use artificial keys. I find the idea of standardized audit fields even worse since they do not make sense for all tables, and unused fields tend to create confusion and waste.As to the comparison with the auto industry, I find it flawed. There the goal is precisely to mass produce large numbers of nearly identical products with interchangeable parts. It makes tremendous sense to standarize virtually everything in that and keep producing them. But no two software projects are absolutely identical (if they were, you would just copy the first one and be done with it).
Another thing to point out is that most assembly line workers are not the same as craftsmen. They just repeat the same thing over and over without trying to optimize for absolute performance. Software creation is much closer to that situation, then to assembly line workers repeating the same motions over and over.
Many good points here Timothy and let me respond to them. First with regards to artificial keys ... the whole point with RAP is to standardize the code generation and the ORM. I think it should be self-evident that one fantastic way to standardize and simplify an app is to have all the keys look exactly the same.
It would be way beyond the scope of this article and this blog for me to discuss all the benefits of using artificial keys, but let me focus on just one: you simply cannot audit a table that doesn't have one. Without an artificial key, there is no field in the table that guarantees continuity from one version of a row to the next (since business keys can always change). So artificial keys are critical to auditing.
Most people have never seen a 100% auditing application and so they can't imagine why you'd want to put status fields on every table. Well ... RAP produces 100% auditing apps. What this means is that you can actually log into your application "as of" a given date, and the app will open up with all of your data displayed as of that date. Screens, web pages, reports, everything. Even a web service, if that's what you're running. And that's just one advantage of having status fields in all tables. It's in the demo .. give it a try.
With regards to the auto analogy. Yes the analogy is flawed, because we are one level of abstraction higher here. With an auto assembly line the objective is to make a zillion of exactly the same thing. With RAP the objective is to make a zillion things whose underlying "plumbing" is built exactly the same way. A much better analogy would be the analogy to compilers, which use the same parameter-passing and code-generation techniques everywhere, but tailor them to whatever code you're writing.
But the automobile analogy is sexier.
October 7, 2010 at 2:07 am
I am also developing an application with similar goal as that of RAP but with a different approach. It is UI driven, which means all the back-end stuff will be generated by the screens designed by a domain expert. Zoho.com has a similar application called Creator but it is much difficult to use as compared to the design I am working on.
October 7, 2010 at 5:13 am
scoia (10/6/2010)
<<Since 2006 Honda has made all the engines in all 33 cars of every Indy race, so no matter who wins, Honda wins.>>Just curious, but for the sake of a logic twist in this conversation. . .if the above statement is true, isn't it equally true that "no matter who loses, Honda loses"? 🙂
Sorry, can't help myself.
No, no matter who loses Honda wins would be correct. You're comparing apples and oranges. For the race loser Honda is still a winner since the engine made by Honda 🙂
October 7, 2010 at 7:06 am
Ahmed Saad (10/7/2010)
I am also developing an application with similar goal as that of RAP but with a different approach. It is UI driven, which means all the back-end stuff will be generated by the screens designed by a domain expert. Zoho.com has a similar application called Creator but it is much difficult to use as compared to the design I am working on.
Ahmed: I am not one to generally discourage people from trying exciting new stuff, but you might want to reconsider the top-down approach before you invest a whole lot more time in it. I have had several people in the past point out the fact that there are already UI-based code generators that generate the database, but I have never taken the bait.
As a consultant I have seen many schemas in many places. If I had to identify the one most crippling problem I see in all of database-land, it is the tendency of schema designers to create schemas where the tables look just like screens of the UI. Such schemas tend to be highly denormalized and further they don't generally model the real-world structure of the problem (since the UI doesn't generally do that), making their databases highly resistant to modification, including modifications needed to make them support new user interfaces.
This is especially important in the current environment where applications increasingly must support multiple UIs, including (possibly) native client, web, mobile, and web service. A mobile UI will generally be much more fragmented than a native client or web interface. Would a top-down solution that supports all of these interfaces actually generate the same database from all of them? I personally cannot imagine how a translator could create a good schema, which requires deep real-world understanding of the problem and profoundly good modeling skillls, from something as superficial as the current UI spec. Then of course there's the problem of supporting a version of your product that generates code from every UI that a programmer possibly could possibly need to support. Even if you could afford to do this, how would you reconcile them?
Here's an example of the problem with UI-driven design. In the consulting business UIs tend to be client-driven because the UI is generally the only thing the client understands. Suppose you are designing a CRM system for a client who tells you that he wants a UI in which the screen (or page or whatever) displays one phone number for each person. Now you know perfectly well that next year this client is going to realize his mistake and come back and require you to allow for multiple phone numbers, but for the moment there is no dissuading him. With a bottom-up generator like RAP you'd just design the database properly anyway (without the client even knowing) and give him the UI he wants. Next year the only thing you'll have to change is the UI. But with a top-down generator you'll be stuck with a schema that supports only one phone number per person.
This is why I chose the bottom-up approach. RAP will not and cannot make the schema designer's job go away, because of all the components of app design the schema requires the deepest insight into the true nature of the problem. A serious app designer models the real world, not the current UI's vision of it, and RAP lets him do that.
In my opinion there is no escape from good schema design. RAP obsolves the schema designer from making the generally arbitrary decisions regarding keys, status fields, and auditing that I see being made poorly every day. It's very much analogous to compilers (replacing assembly language) absolving programmers from the need to design a parameter-passing convention for every function they write. But at the end of the day, if your schema isn't properly normalized and well structured, you're doomed no matter what tools you're using.
October 7, 2010 at 7:52 am
Column in the application is defined as 'DATE', which is not available as a datatype in SQL Server 2005. It is introduced in SQL Server 2008.
The article text should be changed to indicate that SQL Server 2008 is required.
October 7, 2010 at 7:57 am
gbaytler (10/7/2010)
Column in the application is defined as 'DATE', which is not available as a datatype in SQL Server 2005. It is introduced in SQL Server 2008.The article text should be changed to indicate that SQL Server 2008 is required.
Thanks for pointing this out. For the moment you can change it to "datetime" and it will work.
October 7, 2010 at 8:20 am
At the end of the article it kind of sums it up for me where you are going with this...Henry Ford's craftsman didn't like the assembly line approach either. This is a very telling comment. To me this is more of a philisophical discussion than one of simple code efficiency, creating a code monkey is not what I want as a supervisor. I need people who can solve complex problems, creatively. I know you are only describing creation of "plumbing" here but the title of your article proclaims the production of limited interaction cookie cutter applications, a consistent approach to anything is admirable. But, the loss of the craftsmen creates a gap in ingenuity and innovation...I would hate to see the same thing happen to this industry as has happened to many others. Noble idea but I have a dissenting view on making things too cookie cutter. A few things you forgot to mention about the Japanese auto makers and their efficiency culture. They work 14-16 hour days...6 days a week, they have a negative population growth, and retirement is mainly a dream. I like people where I work but I don't want to spend my life with them...there is a balance.
Cheers.
October 7, 2010 at 9:18 am
eric.cardinal (10/7/2010)
At the end of the article it kind of sums it up for me where you are going with this...Henry Ford's craftsman didn't like the assembly line approach either. This is a very telling comment. To me this is more of a philisophical discussion than one of simple code efficiency, creating a code monkey is not what I want as a supervisor. I need people who can solve complex problems, creatively. I know you are only describing creation of "plumbing" here but the title of your article proclaims the production of limited interaction cookie cutter applications, a consistent approach to anything is admirable. But, the loss of the craftsmen creates a gap in ingenuity and innovation...I would hate to see the same thing happen to this industry as has happened to many others. Noble idea but I have a dissenting view on making things too cookie cutter. A few things you forgot to mention about the Japanese auto makers and their efficiency culture. They work 14-16 hour days...6 days a week, they have a negative population growth, and retirement is mainly a dream. I like people where I work but I don't want to spend my life with them...there is a balance.Cheers.
My view as a technologist is that our job is to move things forward. We do this by relieving humans of relatively mindless repetetive work that shouldn't need to be redone, probably poorly, by every person who's trying to achieve something.
In the 1800s every house had a weaving loom and of necessity the woman of the house spent many of her daily hours making clothes. Today factories do the same job more efficiently and consistently and women are free to do other things, including make clothes for fun if that is their inclination.
In the 1960s assembly language programmers spent significant amounts of their time reinventing the plumbing of parameter passing, looping, and so forth. Today compilers do all these things consistently and accurately, vastly reducing the amount of time that we all need to create real functionality.
RAP is in the same spirit. The objective is to properly and systematically do what most programmers currently do both imporperly and inconsistently. They do these things poorly because they're being paid to create functionality, not diddle with plumbing, and so the plumbing always gets short shrift and is done as an afterthought.
RAP is to application design what compilers are to program writing ... it's a fundamentally different environment in which programmers are relieved of mundane tasks they shouldn't need to do and are rewarded with automatic functionality that they could never afford to implement if they had to do the mundane tasks by hand.
RAP will not turn programmers into assembly line workers. With RAP, as with compilers that generate your machine code, the assembly-line work is done by the computer, leaving you free to think about grander things than how to implement auditing all over again (RAP gives you auditing, among many other things, for frree).
So please, let's move forward. Automation ultimately leads to a higher standard of living, even if it forces some of us to re-educate ourselves. At the end of the day I would hope that creating a higher standard of living is what we're all about.
October 7, 2010 at 12:56 pm
David Ziffer (10/7/2010)
Ahmed Saad (10/7/2010)
I am also developing an application with similar goal as that of RAP but with a different approach. It is UI driven, which means all the back-end stuff will be generated by the screens designed by a domain expert. Zoho.com has a similar application called Creator but it is much difficult to use as compared to the design I am working on.Ahmed: I am not one to generally discourage people from trying exciting new stuff, but you might want to reconsider the top-down approach before you invest a whole lot more time in it. I have had several people in the past point out the fact that there are already UI-based code generators that generate the database, but I have never taken the bait.
As a consultant I have seen many schemas in many places. If I had to identify the one most crippling problem I see in all of database-land, it is the tendency of schema designers to create schemas where the tables look just like screens of the UI. Such schemas tend to be highly denormalized and further they don't generally model the real-world structure of the problem (since the UI doesn't generally do that), making their databases highly resistant to modification, including modifications needed to make them support new user interfaces.
This is especially important in the current environment where applications increasingly must support multiple UIs, including (possibly) native client, web, mobile, and web service. A mobile UI will generally be much more fragmented than a native client or web interface. Would a top-down solution that supports all of these interfaces actually generate the same database from all of them? I personally cannot imagine how a translator could create a good schema, which requires deep real-world understanding of the problem and profoundly good modeling skillls, from something as superficial as the current UI spec. Then of course there's the problem of supporting a version of your product that generates code from every UI that a programmer possibly could possibly need to support. Even if you could afford to do this, how would you reconcile them?
Here's an example of the problem with UI-driven design. In the consulting business UIs tend to be client-driven because the UI is generally the only thing the client understands. Suppose you are designing a CRM system for a client who tells you that he wants a UI in which the screen (or page or whatever) displays one phone number for each person. Now you know perfectly well that next year this client is going to realize his mistake and come back and require you to allow for multiple phone numbers, but for the moment there is no dissuading him. With a bottom-up generator like RAP you'd just design the database properly anyway (without the client even knowing) and give him the UI he wants. Next year the only thing you'll have to change is the UI. But with a top-down generator you'll be stuck with a schema that supports only one phone number per person.
This is why I chose the bottom-up approach. RAP will not and cannot make the schema designer's job go away, because of all the components of app design the schema requires the deepest insight into the true nature of the problem. A serious app designer models the real world, not the current UI's vision of it, and RAP lets him do that.
In my opinion there is no escape from good schema design. RAP obsolves the schema designer from making the generally arbitrary decisions regarding keys, status fields, and auditing that I see being made poorly every day. It's very much analogous to compilers (replacing assembly language) absolving programmers from the need to design a parameter-passing convention for every function they write. But at the end of the day, if your schema isn't properly normalized and well structured, you're doomed no matter what tools you're using.
David. Single big reason why I took UI-driven approach is that I think there is no major difference in any two relational databases, no matter how different those are domain wise; one might be a mission critical NASA database while the other just an inventory database of a store. Biggest fact of all relational databases is that they are set of master-detail tables, so what if we can find a UI design as well acceptable to everyone? I feel it is quite possible.
Your point about flexibility to accommodate changes is good. I think it is also do-able. Though I can't think of another example (because my children are crying in top volume :)) but the one-telephone-per-person problem will not be a big issue in my design. From a schema designer's point of view, they will typically need a child table to store multiple phone numbers. My application will add this table according to UI change and then user will have to migrate existing phone numbers from parent table to child table, which can be done manually or by some DTS like activity.
I will be glad to hear more from you on this subject and also to share more information about my application.
October 7, 2010 at 12:57 pm
RAP is in the same spirit. The objective is to properly and systematically do what most programmers currently do both imporperly and inconsistently. They do these things poorly because they're being paid to create functionality, not diddle with plumbing, and so the plumbing always gets short shrift and is done as an afterthought.
Most of what you just said in your post sounds interesting, but the paragraph above just does not reflect the complete reality. Some of us have put a lot of time into our programming to ensure the plumbing is done "correctly", in the sense of efficiency. And when sometimes time doesn't permit, take the time to refactor the code at a later date. I think you would find that your ideas might have more traction if you avoided the negative statements.
October 7, 2010 at 1:16 pm
gbaytler (10/7/2010)
Column in the application is defined as 'DATE', which is not available as a datatype in SQL Server 2005. It is introduced in SQL Server 2008.The article text should be changed to indicate that SQL Server 2008 is required.
Here is a specific fix for SQL Server 2005, which is not really supported any more:
First of all start the whole procedure over with a blank database. But before you start populating the database, edit the file “DataTypes.sql”. In there you will find these two lines:
exec sp_addtype 'DTDate', 'date', 'null'
exec sp_addtype 'DRDate', 'date', 'not null'
Change the ‘date’ types to ‘datetime’. Then start populating the database from scratch according to the instructions. This should then work with SQL Server 2005.
October 7, 2010 at 1:50 pm
RonKyle (10/7/2010)
RAP is in the same spirit. The objective is to properly and systematically do what most programmers currently do both imporperly and inconsistently. They do these things poorly because they're being paid to create functionality, not diddle with plumbing, and so the plumbing always gets short shrift and is done as an afterthought.
Most of what you just said in your post sounds interesting, but the paragraph above just does not reflect the complete reality. Some of us have put a lot of time into our programming to ensure the plumbing is done "correctly", in the sense of efficiency. And when sometimes time doesn't permit, take the time to refactor the code at a later date. I think you would find that your ideas might have more traction if you avoided the negative statements.
Ron: I'm sorry if I seem overly negative to you. It's just that I can't see the point of people doing all this mundane stuff by hand, even if in most places (as you assert) they are doing it consistently and systematically. Maybe we come from radically different environments. If I were not profoundly annoyed by the stuff I see in most designs, I wouldn't have spent much of my free time over the last five years trying to automate the generation of appication infrastructure.
October 7, 2010 at 1:56 pm
even if in most places (as you assert) they are doing it consistently and systematically
I didn't assert that; I only took issue with your assertion to the contrary.
Like you, I have seen and workd with many designs that have annoyed me to death. I knew, in contrast to your assertion, that the cause was not that someone had obssesed over the design. Quite the opposite: the designers did not know what the needed to know to ask the right questions to do it right, or to turn the right answers into a design that best supported that.
Last comment from me on this topic, but I found the discussion very interesting.
October 8, 2010 at 11:27 am
I'm enjoying reading the comments, but I'm curious, what happens when someone comes up with a better engine (or as in our case better software)?
Do we buy into it, or ride the current wave until it is exhausted? perhaps because it's easier or cheaper? Then at the end of the ride were do you go?
Standards are a good thing. It helps with continuity and somewhat simplifies things.
if "continuous quality improvement" is part of the process then the best you can have is the "standard du jour"
-- Optimist with experience and still learning
October 8, 2010 at 3:47 pm
Ed Salva (10/8/2010)
I'm enjoying reading the comments, but I'm curious, what happens when someone comes up with a better engine (or as in our case better software)?Do we buy into it, or ride the current wave until it is exhausted? perhaps because it's easier or cheaper? Then at the end of the ride were do you go?
Standards are a good thing. It helps with continuity and somewhat simplifies things.
if "continuous quality improvement" is part of the process then the best you can have is the "standard du jour"
Ed: First of all, RAP doesn't leave you with something that you can't maintain without the ongoing use of RAP. At the end of the day you still have a chunk of .NET and SQL code that you can continue to maintain by hand if you wish. It's just that it's probably a much more consistent and better structured piece of code than you possibly could have had the time or the budget to write yourself. So you don't incur any substantial risk by using RAP in terms of being left high and dry with something unusable afterwards.
Secondly there is no wave to ride. There is no crowd to follow. So this is an adventure, and instead of deciding whether to go with the crowd you'll have to instead decide whether this tool and this paradigm would make your life a whole lot easier. I'm not expecting many followers. I just want a few that will allow both me and my company to demolish my competition in a real-world demonstration of what is possible when you automate things rather than redo them by hand constantly.
The "standard du jour" argument could be used to exempt one's self from using any technology. How about C#? Remember C# is the replacement for non-compatible VB.NET in many shops. And VB.NET was the replacement for non-compatible VB6. Good heavens, why use compilers at all? If we really want to avoid paradigms and tools, we could even dispense with operating systems and go back to coding machine language. But oops - your machine language constitutes a possibly misguided investment in a particular processor architecture! Where does one turn? It seems that no matter where you look, you are making an investment in one technology or another.
Even if your vendor is Microsoft, there's no assurance that the tool(s) you're using today won't be obsolete tomorrow. The choice with RAP is to have a tool that generates most of your app code automatically, complete with features you could never afford to implement by hand, or alternatively to go without and reinvent the wheel - probably meagerly due to budget & time constraints - with every new app you have to write. The choice, of course, is yours.
Viewing 15 posts - 61 through 75 (of 84 total)
You must be logged in to reply to this topic. Login to reply