Pfeeww... I didn't expect such a looong answer... :)
Let's see..
Sebastien Bigaret wrote:
| Marco Bizzarri wrote:
|
|>|>When you need to look for the DB log, it is useful to spot the
|>|>problems by mean of ABORTed transactions. If the aborted
|>|>transactions are the standard, this can be a problem.
|>|
|>| Understood, this seems reasonable. I think this could be
|>| controlled by an option / env. variable / whatever
|>|
|>
|>That would be great :)
|
|
| Ok, we're going that way then. Could you submit a RFE @sf.net?
| That would make it sure I won't forget...
https://sourceforge.net/tracker/index.php?func=detail&aid=1011515&group_id=58935&atid=489338
| [...]
|
|>| Could you be a little more explicit on your use-case(s)?
|>| I'm curious here, and I'd like to better understand what your needs
|>| are / could be in terms of finer control on transactions.
|>
|>Up to now we have used SQL transactions inside the Zope TM.
|>This means that, for each request/response, you've just one
|>transaction, which is either commited or aborted.
|>
|>The Modeling Framework, on the other hand, would perform
|>several transactions on RDBMS,
|
|
| Right, and the main reason is that they do not work at the same level.
| The Zope TM usually directly "impacts" rdbms databases.
| The MDL framework on the other side works at the object level: you
| make whatever changes you need to make on a graph of objects, then
| you save them as a bunch (and here, obviously,inside a single
| transaction) --that's the saveChanges() stuff. And if you have
| a situation where there can be several saveChanges() *before* the
| decision is made on whether they should be permanently saved into
| the database, then the parent/child EC configuration is for you
| (you can consider such configurations as sub-transactions at the
| object level).
|
| Okay, enough with that --it was probably off-topic here,
| but I took the opportunity to make it clear, just in case ;)
|
|
Yes, but I think there is another point (please forgive me if I'm
repeating the same things you're saying).
In a web environment, like Zope or whatever, you've a clear distinction
of a transaction: the arrive of a request, the sending of response. All
inside these two events can be considered a transaction; at least, this
is what the user can perceive as a transaction.
In non zope (I would say non web based) programming, the boundaries of
the transactions are more difficult to be pointed. Indeed, the
boundaries are of two kinds:
1) read transactions, each having its own RDB transaction;
2) write transactions, which are directly activated by the developer
(with saveChanges())
[...]
|
| Okay, I think I see the point now, I'll try to rephrase it, you'll
| correct me if I misunderstood.
|
| First, let's consider you have a single transaction per thread: you
| probably agree that a problem remains --different problem, but still
| a problem:
|
| a. thread-1 fetches customer (and leave the transaction opened)
| b. thread-2 modifies customers and orders
| c. thread-1 fetches orders, not seeing changes made in b.
|
| Then, what should thread-1 do? (ok, the "not seeing changes made in b."
| actually depends on the db, the cursor's isolation level, ...)
Ok, I'm assuming serialization isolation level. Inside a single
transaction, I think you will end having some serialization error (it
depends on your changes, but you should).
| You may not have the problem of inconsistency between data fetched
| during step a. and c., but still there remains the problem of answering
| the question "what should thread-1 do?" Refresh the (possibly already
| modified) data fetched during step a.? Discard any modifications and
| retry the whole stuff? Or else...?
Yes, this is a problem, and, as you correctly stated, this is a
different. problem.
Actually, this is a classic "business transaction problem", and it is
not limited to the case we are discussing. A tipical would be:
1) user 1 reads Customer C, and has a page to modify it.
2) user 2 actually modifies it
3) user 1 modifies the page and submits the page.
What should be done? First of all, we would have to detect the problem,
and this can be done with different methods (version numbers,
timestamps, etc.)
Then, the application will inform the user, who will have to decide what
to do.
|
|>Actually, I would not having to deal with RDBMS transactions
|>at all. Instead, I would prefer to see a single BEGIN/COMMIT,
|>or BEGIN/ABORT for each of my Zope Transaction.
|
|
| As far as I can understand, your problem is not with transactions, but
| with inconsistencies that can occur in data retrieved from the db when it
| is concurrently accessed and modified, is that right?
Yes, it is.
| Currently (as of release 0.9pre17), each EC currently ignores the
| changes done by others. This means, in particular, that if ec1 and ec2
| fetch then save the same object, the one that saves its changes last
| overrides the other one changes.
|
| Now if you want every EC to be notified (hence, at the object level) of
| not only _when_ an EC saves its changes, but also _what_ the changes are,
| then the 'optimistic locking' patch available at @sf.net is for you :)
|
| --> Search the archives, we discussed this in detail not a long time ago,
| I believe it was w/ Duncan and Ernesto, and as far as I remember Duncan
| uses it successfully since it's been posted.
| (the patch will probably be included in the next release)
I will take a look at the patch, thanks :)
| BTW, the way an EC is notified of changes is not documented neither in
| the messages nor in the patch, but it's there; and the default behaviour
| this patch proposes is, roughly: ec1 and ec2 fetch() an object, ec2
| modifies then saves it, ec1 gets the modifications, applies them then
| re-applies the changes that has been made in its own scope, if
| appropriate.
|
| Could such a mechanism be of some help? I guess so, if I did not
| misunderstood your point, since in particular it would offer an
| automatic way of being notified of changes concurrently made so that
| a decision can be made on what should be done (possibly depending on
| the situation, on which objects are involved, etc.)
I think I should review a little... consider an application where you
have a large number of users, 1000, perhaps, each having its own ec...
should we notify each user of the changes?
|
|>Actually, I would not having to deal with RDBMS transactions
|>at all. Instead, I would prefer to see a single BEGIN/COMMIT,
|>or BEGIN/ABORT for each of my Zope Transaction.
|
|
| Back on this, just a few words: having a single BEGIN/COMMIT or ABORT
| per zope transaction means that you want to restrict what the
| framework is able to do. Updates in an EC can take more than one zope
| transaction to be achieved, for example if they are saved after several
| html pages have been served one after the other (think of a wizard, at
| the end of which you confirm or cancel the changes).
I don't think this is possible in the current integration of MDL and Zope.
IIRC, MDL register itself with the _finish method...
~ def _finish(self):
~ self.ec.lock()
~ self.ec.saveChanges()
~ self.ec.unlock()
Now, this means that, at the end of *each* Zope transaction you will do
a saveChanges on your ec. Therefore, if you update your objects inside
the page of the wizard, they will be stored on the RDB.
Also, even though you could in principle do some saveChanges() by
yourself, this would be looking for havoc. Indeed, Zope relies on the
fact that a transaction can be redone from the start, if something goes
wrong.
| I can understand that you feel uncomfortable with several
| db-transactions occuring during a single zope transaction, but now if
| the inconsistencies can be solved at the object level with full control
| of what should be done (or a reasonable default behaviour), would the
| discomfort eventually remain?
I've to see the Optimistic lock patch before answering.
I confess I'm probably too much Zope-oriented. However, I would explore
this possibility:
1) when a first fetch is done on an EC, the EC, if it has not done yet,
opens a RDB transaction;
2) whenever you have a saveChanges(), the EC either opens and then
commits the transaction, or simply commit the transaction.
The above behaviour could be configurable. You could have an
"auto-commit" EC and a "non-auto-commit" EC (i.e. different subclasses),
and let the developer choose according to its need.
|
|
| -- Sébastien.
Thanks for your support.
Regards
Marco