Data Access Components
Sun, 2008-08-31 19:27 — tamhas
This category will be used to discuss principles of building data access components, sample data access component code, and helper classes for data access.
For initial background reading, consult OERA Strategies: Object-Oriented or Not?
- Printer-friendly version
- Login to post comments
Comments
Where to post sample code ?
Hi,
I would like to post some sample code of the implementation of our data access layer but I'm not sure where to put this.
It's my first time creating content on the Hive and I'm a bit lost...
Is this the right place and content type ?
Thanks,
Guillaume.
Depends on the volume. But,
Depends on the volume. But, in general I would suggest that you create a Book Page (see Create Content in the right column) and parent that Book Page to the appropriate page in this set. Do not select Navigation since that is only for content which will appear on its own in the site map. Content within this project should be reached by the link to the project and then navigation within the set of pages.
On that page, you could put a small code sample as part of the text along with a discussion, e.g., if you are presenting a technique. Or, if it is a larger code sample, then use the book page to discuss your approach and design and include the code samples as one or more attachments to the page. You'll find it is actually quite easy once you get started.
If you are presenting something sufficiently complex, like a multi-layered approach, then you could create the main book page as above along with additional book pages for each layer or component and parent those pages to the main one. While one can also link pages by putting in < a > tags, this nesting of book pages thing is all built-in Drupal and gives a nice navigation structure without any special work on your part.
** deleted
** deleted
** deleted
** deleted
** deleted
** deleted
At the end of the day is it worth the trouble ?
I think the danger with DA is that although it's a wonderful idea it has the potential of complicating the design and implementation of both the framework and working on the framework, exponentially!
It has the real danger of killing a project by making it too big or unpractical. And maybe it's a trade off that should be considered, in most cases ?
I would say quite the
I would say quite the contrary. Create a data access component for Customer once and use it everywhere means one place to update when there is a change, not thousands.
Okay.But a. theres no way
Okay.
But a. theres no way of doing that.
DATA-SOURCES aren't useful for anything but the simplest cases. Although they certainly make things a *heck* of alot more complicated.
DATA-SOURCE's make me sick.
You just can't use procedures the same way you use tables e.g. joins, filtering and so on.
If there was some way that was based on definitions instead of code like VIEWS maybe that would be possible but even then it would only be for quering data not updating.
And b. theres the added design of what to put in these data source queries. In other words how do you break all this data into useful chunks.
It's like designing a whole new framework!
Like I said I think it's a great idea and it has all the benefits in the world BUT it's out of touch with reality.
How about we just use tables.
Leaving the data sources out leaves you with a DA component for every BE or one for every data store. In that case why not just write a BE version for every data store and make things a heeeck of alot simpler.
Sometimes, in some cases it is worth writing something twice and *as long as it's in the same place, organized clear etc.* instead of writing an amazingly complicated procedure that does everything.
Simplicity is a very important quality and one that should not be underestimated.
I definitely feel it's where most Progress frameworks have failed so far. Just my opinion. It's the biggest lesson we should take with us on this project, I think.
What I'm hoping for out of this project is something along the lines of a few basic guidelines, concepts etc. for building services.
Something like a standard library of useful, needed features. A much smaller number of managers. And doing the UI outside of Progress, well, maybe except for a standard web UI.
Something that will allow *everyone* to go out and start cranking powerful, scalable distributed applications.
Something that will catch on, something that will click.
I'm having a bit of trouble
I'm having a bit of trouble figuring out whether we are comparing apples to apples or apples to kiwifruit.
A DA component to get A customer is not a complicated entity. At most it will have a couple of different ways to find them. There are more ways to get a set of customers, but again, probably not a huge number, especially if one of the ways is to pass in the kind of information one would put into a query prepare. The rest of the logic in there is also probably not that complicated and much of it needs to be in every place where customers are accessed or stored, e.g., validity checks and such.
Yes, it is like writing an entirely new framework ... that is exactly what OERA OSI is about. What you do with it depends on your circumstances. When it gets really filled out, I hope to be using MDA to generate new applications based on this structure ... but that's a while off. Short term, one can pick and choose. I suspect there is a lot more interest in transformation to separate the UI layer than the DA layer, so there will be more attention there. Shame in a way since I think a lot of DA layer separation could be done with transforms that weren't that impossible to create.
But, no, we don't just want to use tables. If your BL layer is accessing the tables directly and you start moving to SOA/ESB with distributed deployment, then you have BL trying to access data which is on a different machine. With DA layer separation in place, no problem, just stick in a facade and have the facade access the data wherever it is.
I knew I should have read
I knew I should have read Fowler's book :) But that's exactly the point. It should be far more accessible. There should be a few more people that will be able to write apps besides you and people with your experience. I think, it would be crazy and a shame if writing apps would be that difficult. I also think we've waited too long and it's real problem that we don't have anything useful by now.
Regarding integrating SOA and ESB or getting the data out of some remote service. Instead of writing another version of the DA I'd write another version of the BE. Now granted it would duplicate the business logic in both BE's but it would all be in the same place, easy to maintain and under control. Now that's simple. I just don't think the added complexity in the DA is worth the effort more then that it puts the project in danger of been overly complicated.
If you're advocating using Data Access Objects and even Data Source Objects. How about posting a *practical* code sample we could discuss. I haven't seen one yet definitely not AE. BTW are you saying you are for Data Source Objects ?
I'm saying forget about caches, environment, connections and all the other million useless managers. It's too much useless stuff to handle. And let's stay away from huge frameworks.
I'm personally looking for mostly information, on broad concepts like security, basic architecture we can agree on etc. and specific guidelines on writing services and BE's. A standard library, a small number of essential managers, like, state manager etc. basically that's it. So people can go out and start building these apps.
Regarding integrating SOA
Regarding integrating SOA and ESB or getting the data out of some remote service. Instead of writing another version of the DA I'd write another version of the BE. Now granted it would duplicate the business logic in both BE's but it would all be in the same place, easy to maintain and under control. Now that's simple.
We obviously have different ideas of simplicity. To me, inserting a small facade object into the mix is extremely simple and keeps everything encapsulated and not duplicated. The idea of a duplicate BE, one of which has direct links to the database and the other of which is set up for remote source sounds like an invitation for a real mess and a horrible duplication of maintenance.
How about posting a *practical* code sample we could discuss. I haven't seen one yet definitely not AE.
Definitely not AE. We'll get to the practical example ... but it seemed like we should talk about it first.
I'm saying forget about caches, environment, connections and all the other million useless managers. It's too much useless stuff to handle. And let's stay away from huge frameworks.
Well, it would appear you aren't very interested in the whole project then. Useless? Because you don't need to do any of those things? Well, anything you don't need, don't use. This is one of the points of architecting this stuff as a series of components which will work together, but which don't need to be used as a whole. Use one, use ten, use a hundred ... whatever fits your need.
If you can separate the
If you can separate the BE/DA/DS in a simple and practical way, that can still do the job then great I'm all for it! Please post something or let's start a conversation on how it can be done.
It's just that no one has done anything like that and we've been talking about it for several years now! Usually when things keep getting more and more complicated it's a sign we're going the wrong way.
I've thought about it long and hard and I think it's not possible or at least not practical.
A. The idea of DS is out of touch with reality. You just can't do the same things with procedures that you can and need to do with tables. And DA by themselves don't offer that much of a dramatic change.
B. the complexity it adds is definitely not worth the effort. You said yourself that it's like designing a whole other framework, on top of the BL. That's huge!
I'd like to point out that my biggest problem is with the idea of DS as proposed by the OERA. If it's just DA then I don't really mind it. It's a little more work and things to know and in my opinion no real benefit.
But that's just my opinion.
To me, it is not more
To me, it is not more complicated, it is simpler. Solve each problem once and in a consistent way and be done with it. If a change is needed, it is needed in only one place.
Hopefully, that will be apparent when we get some more concrete examples up.
it's been how many years now
it's been how many years now ?!
progress themselves have invested how many people ? and how man hours ? and how many dollars ? and for how many years now ?! that you said yourself is still no where good enough.
so it's not just us. we're going the wrong way. building apps should be routine, everyone should be able to do it, it shouldn't be like sending a man to the moon.
There are more examples of
There are more examples of bad code in the Progress world than of good code ... but the same is true of any language. Code examples from PSC are often not best practice since most of the time the people writing those examples are trying to illustrate a language feature, not show how it should really be used in production. The Progress community is at something of a disadvantage here since it is much smaller than many other language communities, so it is less likely that we will have our ration of gurus to lead the way. This is just the way it works. Sometimes a small community gets a guru and it is a guru that people are happy with and that is fine ... look at Linux and Linus ... before it all became big. Such is the dynamic of communities. There is no point in bemoaning that it should be different or in blaming PSC that they should have somehow made it different ... gurus often don't come from the obvious source and often aren't on salary.
But, we have the opportunity to do something here ... so, let's do it. Not because anyone is going to become a guru, but because a community working together can make things better for the community. That's what has made the Linux world ... not any of the individual gurus who paved the way, but ultimately it is the contribution by the community.
Are you with us or fighting us?
i'm not going to fight you,
i'm not going to fight you, i just have a different opinion ... maybe that will change.
i definitely plan on getting involved and posting code, but it will probably be different.
hope i wasn't too passionate or said something inappropriate ;)
I agree with simplicity
Hello,
I'm new to the OERA, but I agree with Alonb that there is a certain need for simplicity. And yes, there is the DA, BL and UI-layer, which must be all seperated quite well. I think OE gives us a nice tool with the prodataset to accomplish the data-transport between the several layers.
In fact, all we need is indeed a framework, which doesn't carry tons of luggage with it to intercept every exception, like the adm2 did. I'd go for simplicity too. On the UI-side, there must be possibilities to use the classic OE-browsers, and a adm2-alike viewer. But it must also be easy to use other, more advanced objects, like treeview, browser in browser, just what 10.2 is all about. And there is the question : Will we use the 10.2 objects for the new framework, or will we look into the marked and develop UI-interfaces for several solutions, like eg java,.Net,...-objects.
The BL-layer must be as open as possible. Again, I think the dataset standard-procedures are the way to go. And perhaps we can provide everybody a template which contains all the procedures / triggers (why not some publish/subscribes, like adm2 does, but than far far simpler?)
The DA-layer must indeed be useable for all kind of db's : oracle, mysql, but first of all the progress-db. And here I also agree that the DB-interface shouldn't be generic. It's perfectly possible to re-use some routines, but the several interfaces should be as thin as possible.
You agree on simplicity and
You agree on simplicity and yet you immediately also vote for the layer separation that Alon seems to be trying to eliminate! :) I guess it is a question of what one calls simple. To me, layer separation does provide simplicity ... I just don't seem to be having much luck convincing Alon of that.
As to using PDS for the interface between the layers, check out this discussion. I am currently voting for encapsulating the PDS in a class for sets and using simple business entity classes with properties for single instances. Needing to have the definition of the PDS on both sides is bad OO.
We allready work with layers
We allready work with layers over here (mostly 2 layers, sometimes 3), as we use (stateless) appservers. At this very moment, we indeed define the PDS at both sides, and use an include file for this. It's also possible to pass-through an PDS-handle, and then you don't need the definition anymore, but I agree that would things more complicated.
I can live with the idea to incapsulate the PDS in an object, but I wouldn't like to rebuild the PDS ourselves. I think we still should use the PDS as a starting point for the class.
I am with you on the use of
I am with you on the use of a PDS or TT for any class which is a set, as long as it is encapsulated.
For sending across the wire, I think that I am interested in retesting Greg Higgins observations that serialization to XML, sending the XML, and deserializing is actually faster than sending the PDS or TT. The beauty of that approach is that it doesn't matter what wire one is sending over or what technology is on the other end.
Another reason to consider
Only who is going to develop an application for other datbases than Progress is actually using this layer.
But i guess were all Progress guys.
While the benefits of the DA
While the benefits of the DA layer are most dramatic when one has to develop for a second persistence platform, that is by no means the only time it is of benefit. Other than encapsulating the logic in one place in good OO fashion, one also has the advantage of providing a very simple way to deal with remote data sources since an application doesn't know whether it is dealing with a facade or a prime source. And, don't forget too that a lot of Progress partners have been essentially forced to support Oracle and/or SQL Server because of the market and standardization by companies on those databases. Just think how much easier making the DataServer adaptation would be if all the persistence logic were in a well-defined DA layer instead of scattered everywhere in the application.