[Modeling-users] Using modeling framework in long run

[Andrey Lebedev]

Hello,

I'm currently evaluating several object-relational mappings available
for python and I thing modeling is one of the best among them. However,
I need some clarification on how projects, that are based on modeling
framework, are supposed to be maintained.

As far as I understand, the framework is heavily relies on code
generation. Developer creates single XML or python file, describing data
model and then generates SQL schemas and python sources. Judging from
comments inside those generated python sources, they are intended to be
modified by developers after generation.

But what happens when database schema should be changed? What if there
are data in database that can't be lost just by deleting and recreating
tables? How to keep generated files' customizations after model
modifications?

The simplest procedure I can come up with is:

1. Change database on sql server using regular tools

2. Change generated files by adding get<attr>, set<attr> methods and
other necessary functions.

3. Change model file (actually I'm not sure it is nesessary)

The procedure is somewhat too complex, expecially because generated
python modules are not elementary and consist of relatively complex
statements to be written by hand each time database schema changes.

So, the question is: what, generally I have to do when I have, say,
developer and production project installations and I want add new column
to a table? How do you handle such situations?

Thanks in advance.

[John Lenton]

Post by Andrey Lebedev
1. Change database on sql server using regular tools
2. Change generated files by adding get<attr>, set<attr> methods and
other necessary functions.
3. Change model file (actually I'm not sure it is nesessary)
The procedure is somewhat too complex, expecially because generated
python modules are not elementary and consist of relatively complex
statements to be written by hand each time database schema changes.
So, the question is: what, generally I have to do when I have, say,
developer and production project installations and I want add new column
to a table? How do you handle such situations?

1. Change model file
2. Regenerate python code, using -B (you must've done this the first time too)

The -B option gives you the validators (the business logic) in
files in the base
directory of the generated package, and set/get/add/etc. (the
"pure" generated
code that you don't add to).

3. Alter the database.

this leaves your python code without the new validators, which you
might have to add, but it's significantly less trouble than what you
described.

If your validators get very complex, you might find it easyer to put
them in a mixin class, and modify the generated classes via the mixin
(which would normally be just one line per class you need to add).

Also, if you keep all the revisions of the original, unmodified
generated code, you can update the validators automatically with
diff3.

[Sebastien Bigaret]

Hi Andrey,

First at all, thanks for your interest!

The problem here is obviously that the default option (-C for
--compact-generation-scheme, see mdl_generate_python_code.py -h) is
nothing more than a example showing what are the basics for binding a
class to the framework. But nobody probably uses it in a real
dev. effort because it's difficult to deal with such a scheme when the
underlying DB-schema changes a lot --just what you noticed and felt
unconfortable with.

As John Lenton has already explained, the -B switch in code generation
will be a great help for you, since you can put all your code in
subclasses of auto-generated classes which are put in a dedicated
package (MDL/).

You can also look at John's original post and the corresponding thread
at:
https://sourceforge.net/mailarchive/forum.php?thread_id=3851852&forum_id=10674



Second, the framework does not anymore relies that heavily on code
generation, but how could you know since this hasn't been publicly
announced yet :/ ... (esp. because it's not fully documented)

--> dynamic, on-the-fly creation of package is available in the last
release 0.9pre17; look for a file named doc/README.dynamic.txt, also
available here:
http://cvs.sourceforge.net/viewcvs.py/modeling/ProjectModeling/Modeling/doc/README.dynamic.txt?rev=1.1&view=auto

Using that feature allows you to bypass the modification of the needed
code since dynamic generation will always use your model to dynamically
create your classes. Just a little note, Ernesto Revilla reported some
problems regarding properties and inheritance, so you may not want to
use properties right now; more details in Ernesto's original post here:
https://sourceforge.net/mailarchive/forum.php?thread_id=5035017&forum_id=10674

Feel free to ask for more if some points remain obscure!

-- Sebastien.

Post by Andrey Lebedev
Hello,
I'm currently evaluating several object-relational mappings available
for python and I thing modeling is one of the best among them. However,
I need some clarification on how projects, that are based on modeling
framework, are supposed to be maintained.
As far as I understand, the framework is heavily relies on code
generation. Developer creates single XML or python file, describing data
model and then generates SQL schemas and python sources. Judging from
comments inside those generated python sources, they are intended to be
modified by developers after generation.
But what happens when database schema should be changed? What if there
are data in database that can't be lost just by deleting and recreating
tables? How to keep generated files' customizations after model
modifications?
1. Change database on sql server using regular tools
2. Change generated files by adding get<attr>, set<attr> methods and
other necessary functions.
3. Change model file (actually I'm not sure it is nesessary)
The procedure is somewhat too complex, expecially because generated
python modules are not elementary and consist of relatively complex
statements to be written by hand each time database schema changes.
So, the question is: what, generally I have to do when I have, say,
developer and production project installations and I want add new column
to a table? How do you handle such situations?
Thanks in advance.
--
Andrey Lebedev aka -.- . -.. -.. . .-.
Software engineer at UAB Mikromarketingas (http://micro.lt)
Homepage: http://micro.lt/~andrey/

[John Lenton]

On 19 Jul 2004 21:00:29 +0200, Sebastien Bigaret

Post by Sebastien Bigaret
Second, the framework does not anymore relies that heavily on code
generation, but how could you know since this hasn't been publicly
announced yet :/ ... (esp. because it's not fully documented)

I'm not clear on one issue: how do I add validators to dynamicly
created packages?

[Sebastien Bigaret]

Post by John Lenton

Post by Sebastien Bigaret
Second, the framework does not anymore relies that heavily on code
generation, but how could you know since this hasn't been publicly
announced yet :/ ... (esp. because it's not fully documented)

I'm not clear on one issue: how do I add validators to dynamicly
created packages?

You add validators, along w/ any necessary business logic, by defining your
own classes. Now I guess the question is how ;)

The simplest way is w/ the metaclass approach, when you declare your
own class with '__metaclass__=CustomObjectMeta' (also look at the
sections beginning with <<if __name__ ==...>> in dynamic.py for
examples)

There is an other solution if you want to stay away from metaclasses,
but it requires more work. For example:

Say you want to have classes in Package.Modules.Classes:

- in your package's __init__.py:

* load the model,
* change its packageName into 'PackageBase' (instead of Package)
* build the classes (either with dynamic.build() or build_with_metaclass())
* revert the model's packageName to its original value 'Package'

- then, define a subclass for each entity, inheriting from
'PackageBase.Module.Class'. They'll inherit from the dynamically
built classes, the framework will use the derived classes which hold
the dedicated validators, etc.

-- Sébastien.

Post by John Lenton
--
bash: fortune: command not found

Hey, keep an eye on your bash, looks like it is trying to escape and
seek fortune ;P

[Mario Ruggier]

Post by Sebastien Bigaret
the framework does not anymore relies that heavily on code
generation, but how could you know since this hasn't been publicly
announced yet :/ ... (esp. because it's not fully documented)
--> dynamic, on-the-fly creation of package is available in the last
release 0.9pre17; look for a file named doc/README.dynamic.txt, also
http://cvs.sourceforge.net/viewcvs.py/modeling/ProjectModeling/
Modeling/doc/README.dynamic.txt?rev=1.1&view=auto

I have just played with this, just very little... what I'd like to know
more about is why the 3 ways of using Modeling.dynamic? It is not clear
to me... and:
- it seems more code complications, that someone must maintain ;(
- more documentation, that someone must write, and maintain ;(
- more documentation, that someone must understand ;(
- not pythonic... as "there is no clear and obvious way to do it"...

When you get the time, could you say a little more about the rationale
for these 3 ways, and, if these ways are not explorative and temporary
(will "standardize" on one way in the future...), what are the
advantages of one way over another, and when would you use one and not
the other?

Post by Sebastien Bigaret
Using that feature allows you to bypass the modification of the needed
code since dynamic generation will always use your model to dynamically
create your classes. Just a little note, Ernesto Revilla reported some
problems regarding properties and inheritance, so you may not want to
https://sourceforge.net/mailarchive/forum.php?
thread_id=5035017&forum_id=10674

These problems apply only to the first way, or to all three?

Post by Sebastien Bigaret
Feel free to ask for more if some points remain obscure!

;-)

mario