[Modeling-users] Patch for handling boolean values in xml model for python 2.3

[John Georgiadis]

Hi,

This is a patch to handle the differences between python 2.2 & 2.3
regarding converting boolean values. It uses the bool() built-in, so I
think it won't work for versions of python < 2.2.1

Boolean values such as isClassProperty & isAbstract are written as 0/1
in 2.2 and True/False in 2.3. The model loads fine with either 2.2/2.3
regardless of the version of python it was generated from.

I'm not familiar with the code base, so let me know if this breaks
something else.

cheers
john


diff -urN Modeling.orig/Attribute.py Modeling/Attribute.py
--- Modeling.orig/Attribute.py 2003-09-14 11:35:07.000000000 +0300
+++ Modeling/Attribute.py 2003-11-28 12:18:52.000000000 +0200
@@ -572,6 +572,13 @@
attrType=self.xmlAttributeType(attributeName)
set=self.xmlSetAttribute(attributeName)
if attrType=='string': value=unicodeToStr(value, encoding)
+ elif attrType=='number': value=int(value)
+ elif attrType=='bool':
+ if value=='False': value=False
+ else:
+ try: value=bool(int(value))
+ except ValueError: value=bool(value)
+
set(value)

def getXMLDOM(self, doc=None, parentNode=None, encoding='iso-8859-1'):
@@ -634,10 +641,10 @@
'externalType' : ('string',
self.setExternalType,
self.externalType),
- 'isClassProperty' : ('string',
+ 'isClassProperty' : ('bool',
self.setIsClassProperty,
self.isClassProperty),
- 'isRequired' : ('string',
+ 'isRequired' : ('bool',
self.setIsRequired,
self.isRequired),
# defaultValue must be loaded AFTER type is set, or we will get the
diff -urN Modeling.orig/Entity.py Modeling/Entity.py
--- Modeling.orig/Entity.py 2003-11-24 14:12:32.000000000 +0200
+++ Modeling/Entity.py 2003-11-28 12:15:11.000000000 +0200
@@ -1278,6 +1278,12 @@
attrType=self.xmlAttributeType(attributeName)
set=self.xmlSetAttribute(attributeName)
if attrType=='string': value=unicodeToStr(value, encoding)
+ elif attrType=='number': value=int(value)
+ elif attrType=='bool':
+ if value=='False': value=False
+ else:
+ try: value=bool(int(value))
+ except ValueError: value=bool(value)
set(value)

if phase==1:
@@ -1405,10 +1411,10 @@
'externalName': ( 'string',
self.setExternalName,
self.externalName ),
- 'isReadOnly': ( 'string',
+ 'isReadOnly': ( 'bool',
self.setReadOnly,
self.isReadOnly ),
- 'isAbstract': ( 'string',
+ 'isAbstract': ( 'bool',
self.setIsAbstract,
self.isAbstract ),
'moduleName': ( 'string',
diff -urN Modeling.orig/Relationship.py Modeling/Relationship.py
--- Modeling.orig/Relationship.py 2003-07-28 10:18:59.000000000 +0300
+++ Modeling/Relationship.py 2003-11-28 12:15:37.000000000 +0200
@@ -251,7 +251,12 @@
attrType=self.xmlAttributeType(attributeName)
set=self.xmlSetAttribute(attributeName)
if attrType=='string': value=unicodeToStr(value, encoding)
- if attrType=='number': value=int(value)
+ elif attrType=='number': value=int(value)
+ elif attrType=='bool':
+ if value=='False': value=False
+ else:
+ try: value=bool(int(value))
+ except ValueError: value=bool(value)
set(value)

return
@@ -303,7 +308,7 @@
'displayLabel' : ('string',
self.setDisplayLabel,
self.displayLabel),
- 'isClassProperty' : ('number',
+ 'isClassProperty' : ('bool',
self.setIsClassProperty,
self.isClassProperty),
'multiplicityLowerBound': ('number',
diff -urN Modeling.orig/utils.py Modeling/utils.py
--- Modeling.orig/utils.py 2003-11-24 14:12:38.000000000 +0200
+++ Modeling/utils.py 2003-11-27 21:36:56.000000000 +0200
@@ -81,11 +81,11 @@

def toBoolean(param):
"""
- If param is a string (or unicode) and equals to 0, returns 0 ; otherwise
+ If param is a string (or unicode) and equals to 0, returns False ; otherwise
returns 'not not param'.
"""
if type(param) in (types.StringType, types.UnicodeType):
- if param=='0': return 0
+ if param=='0': return False
return not not param

if sys.version_info < (2, 2):

[Sebastien Bigaret]

Hi John,

Sorry for the late answer. I still did not find the time to look at
your patch, but I will, probably this week-end, and I'll then report
here. It will probably need some adjustment before it is integrated
into the main trunk since we still support py2.1 for now.

Thanks a lot for sharing,

-- Sébastien.

Post by John Georgiadis
Hi,
This is a patch to handle the differences between python 2.2 & 2.3
regarding converting boolean values. It uses the bool() built-in, so I
think it won't work for versions of python < 2.2.1
Boolean values such as isClassProperty & isAbstract are written as 0/1
in 2.2 and True/False in 2.3. The model loads fine with either 2.2/2.3
regardless of the version of python it was generated from.
I'm not familiar with the code base, so let me know if this breaks
something else.
cheers
john

[John Georgiadis]

Hi Sebastian,

I get a syntax error from MySQL-4.0.11 when using nested JOINS created by
MySQLAdaptorLayer.MySQLExpression._addTableJoinsForAlias(). If I switch to
Modeling.SQLExpression._addTableJoinsForAlias(), the problem is gone.

The following patch does this for mysql version >= 4.0.11. If anyone can
confirm this for an older version of MySQL, then feel free to lower the
version numbers in the tuple.


diff -urN --exclude=Python_bricks Modeling.orig/DatabaseAdaptors/MySQLAdaptorLayer/MySQLSQLExpression.py Modeling/DatabaseAdaptors/MySQLAdaptorLayer/MySQLSQLExpression.py
--- Modeling.orig/DatabaseAdaptors/MySQLAdaptorLayer/MySQLSQLExpression.py 2003-08-10 14:59:13.000000000 +0300
+++ Modeling/DatabaseAdaptors/MySQLAdaptorLayer/MySQLSQLExpression.py 2003-12-07 18:14:53.000000000 +0200
@@ -33,6 +33,7 @@

__version__='$Revision: 1.7 $'[11:-2]

+from Modeling.DatabaseAdaptors.MySQLAdaptorLayer.mysql_utils import *
from Modeling.SQLExpression import SQLExpression, DateType, CharacterType
from Modeling.logging import trace
import string
@@ -141,6 +142,10 @@
https://sourceforge.net/tracker/index.php?func=detail&aid=614261&group_id=58935&atid=489335

"""
+ version = mysql_server_version()
+ if version[:3] >= (4,0,11):
+ return SQLExpression._addTableJoinsForAlias(self, alias, str)
+
trace('alias: %s / str: %s'%(alias, str))
relPaths=self._internals.relPathsComingFromAlias(alias)
aliases=map(lambda rp, self=self: self._internals.aliasForRelPath(rp),
diff -urN --exclude=Python_bricks Modeling.orig/DatabaseAdaptors/MySQLAdaptorLayer/mysql_utils.py Modeling/DatabaseAdaptors/MySQLAdaptorLayer/mysql_utils.py
--- Modeling.orig/DatabaseAdaptors/MySQLAdaptorLayer/mysql_utils.py 1970-01-01 02:00:00.000000000 +0200
+++ Modeling/DatabaseAdaptors/MySQLAdaptorLayer/mysql_utils.py 2003-12-07 18:07:52.000000000 +0200
@@ -0,0 +1,67 @@
+#-----------------------------------------------------------------------------
+#
+# Modeling Framework: an Object-Relational Bridge for python
+# (c) 2001, 2002, 2003 Sebastien Bigaret
+#
+# This file is part of the Modeling Framework.
+#
+# The Modeling Framework is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as published
+# by the Free Software Foundation; either version 2 of the License, or (at
+# your option) any later version.
+#
+# The Modeling Framework is distributed in the hope that it will be
+# useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with the Modeling Framework; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+#
+#-----------------------------------------------------------------------------
+
+import MySQLdb;
+
+"""
+MySQL Adaptor Layer's utils
+
+ Central point for determining MySQL specifics, like the version of the
+ mysql server in use.
+
+ CVS information
+
+ $Id$
+
+"""
+
+__version__='$Revision$'[11:-2]
+
+import os
+
+def mysql_server_version():
+ """
+ Returns the version of the mysql server currently in use. It calls
+ MySQLdb.get_client_info().
+
+ Return value:
+ A 4-values tuple: (VERSION, PATCH_LEVEL, SUBLEVEL, EXTRA_VERSION)
+ E.g. (4,0,11,a-gamma)
+
+
+ """
+ s = MySQLdb.get_client_info()
+ v_list = s.split('.', 3)
+ version = int(v_list[0])
+ patch_level = int(v_list[1])
+ s = v_list[2]
+ for i in range(len(s)):
+ if not s[i].isdigit():
+ break
+ if i == 0:
+ sublevel = 0
+ else:
+ sublevel = int(s[:i])
+ extra_version = s[i:]
+
+ return (version, patch_level, sublevel, extra_version)

[John Georgiadis]

Hi,

This is a 2nd try on a patch sent last week about handling boolean values
in the xml model generated with python 2.3. It is less intrusive than the
previous and it doesn't use the bool() built-in, so it runs under python
2.1 as well.


diff -urN --exclude=Python_bricks Modeling.orig/Attribute.py Modeling/Attribute.py
--- Modeling.orig/Attribute.py 2003-08-30 20:22:56.000000000 +0300
+++ Modeling/Attribute.py 2003-12-08 13:29:19.000000000 +0200
@@ -572,6 +572,8 @@
attrType=self.xmlAttributeType(attributeName)
set=self.xmlSetAttribute(attributeName)
if attrType=='string': value=unicodeToStr(value, encoding)
+ elif attrType=='number': value=int(value)
+ elif attrType=='bool': value=int(value)
set(value)

def getXMLDOM(self, doc=None, parentNode=None, encoding='iso-8859-1'):
@@ -596,7 +598,9 @@

exportAttrDict=self.xmlAttributesDict(select=1)
for attr in exportAttrDict.keys():
+ attrType=self.xmlAttributeType(attr)
value=self.xmlGetAttribute(attr)()
+ if attrType=='bool': value=int(value)
value=strToUnicode(str(value), encoding)
node.setAttribute(attr, value)

@@ -634,10 +638,10 @@
'externalType' : ('string',
self.setExternalType,
self.externalType),
- 'isClassProperty' : ('string',
+ 'isClassProperty' : ('bool',
self.setIsClassProperty,
self.isClassProperty),
- 'isRequired' : ('string',
+ 'isRequired' : ('bool',
self.setIsRequired,
self.isRequired),
# defaultValue must be loaded AFTER type is set, or we will get the
@@ -659,7 +663,7 @@
setType(aType=aType,
check=0),
self.type),
- 'isFlattened' : ('string',
+ 'isFlattened' : ('bool',
self.setIsFlattened,
self.isFlattened),
'width' : ('string',
diff -urN --exclude=Python_bricks Modeling.orig/Entity.py Modeling/Entity.py
--- Modeling.orig/Entity.py 2003-10-04 15:29:43.000000000 +0300
+++ Modeling/Entity.py 2003-12-08 13:31:19.000000000 +0200
@@ -1278,6 +1278,8 @@
attrType=self.xmlAttributeType(attributeName)
set=self.xmlSetAttribute(attributeName)
if attrType=='string': value=unicodeToStr(value, encoding)
+ elif attrType=='number': value=int(value)
+ elif attrType=='bool': value=int(value)
set(value)

if phase==1:
@@ -1359,7 +1361,9 @@

exportAttrDict=self.xmlAttributesDict()
for attr in exportAttrDict.keys():
+ attrType=self.xmlAttributeType(attr)
value=self.xmlGetAttribute(attr)()
+ if attrType=='bool': value=int(value)
value=strToUnicode(str(value), encoding)
node.setAttribute(attr, value)

@@ -1405,10 +1409,10 @@
'externalName': ( 'string',
self.setExternalName,
self.externalName ),
- 'isReadOnly': ( 'string',
+ 'isReadOnly': ( 'bool',
self.setReadOnly,
self.isReadOnly ),
- 'isAbstract': ( 'string',
+ 'isAbstract': ( 'bool',
self.setIsAbstract,
self.isAbstract ),
'moduleName': ( 'string',
diff -urN --exclude=Python_bricks Modeling.orig/Relationship.py Modeling/Relationship.py
--- Modeling.orig/Relationship.py 2003-07-28 10:18:59.000000000 +0300
+++ Modeling/Relationship.py 2003-12-08 13:34:29.000000000 +0200
@@ -252,6 +252,7 @@
set=self.xmlSetAttribute(attributeName)
if attrType=='string': value=unicodeToStr(value, encoding)
if attrType=='number': value=int(value)
+ elif attrType=='bool': value=int(value)
set(value)

return
@@ -278,7 +279,9 @@
#
exportAttrDict=self.xmlAttributesDict()
for attr in exportAttrDict.keys():
+ attrType=self.xmlAttributeType(attr)
value=self.xmlGetAttribute(attr)()
+ if attrType=='bool': value=int(value)
value=strToUnicode(str(value), encoding)
node.setAttribute(attr, value)

@@ -303,7 +306,7 @@
'displayLabel' : ('string',
self.setDisplayLabel,
self.displayLabel),
- 'isClassProperty' : ('number',
+ 'isClassProperty' : ('bool',
self.setIsClassProperty,
self.isClassProperty),
'multiplicityLowerBound': ('number',

[Sebastien Bigaret]

Hi,

Thanks for the report. I've searched the mysql-site for about half an
hour but couldn't find any hints on when the change happened. Do you
(or anyone here) have any clue?

Second, could you possibly confirm that the test
test_11_allQualifierOperators() in test_EditingContext_Global.py fails
when you patch is /not/ applied, and succeeds when it is applied? If it
does not, then I'll need to write a new one revealing that bug.

Given that this particular test behaves as expected, your patch will
be applied as-is and integrated to the main trunk. Thanks again for
reporting.

-- Sébastien.

Post by John Georgiadis
Hi Sebastian,
I get a syntax error from MySQL-4.0.11 when using nested JOINS created by
MySQLAdaptorLayer.MySQLExpression._addTableJoinsForAlias(). If I switch to
Modeling.SQLExpression._addTableJoinsForAlias(), the problem is gone.
The following patch does this for mysql version >= 4.0.11. If anyone can
confirm this for an older version of MySQL, then feel free to lower the
version numbers in the tuple.

[John Georgiadis]

Unpatched and patched both succeed. However if I switch the last test
from:
'(age<100) AND pygmalion.books.title like "G*"'
To:
'(age<100) AND ((books.title like "G*") OR (pygmalion.books.title like
"G*"))'

Then the unpatched version fails, while the patched succeeds. I think,
this is related to two joins (Writer.books & Writer.pygmalion) going to
different directions. Below is the generated SQL in both cases

SELECT DISTINCT t0.ID, t0.LAST_NAME, t0.FIRST_NAME, t0.AGE,
t0.FK_WRITER_ID, t0.BIRTHDAY FROM BOOK t1 INNER JOIN WRITER t0 ON (
t0.ID=3Dt1.FK_WRITER_ID ) ( BOOK t3 INNER JOIN WRITER t2 ON (
t2.ID=3Dt3.FK_WRITER_ID ) ) INNER JOIN WRITER t0 ON (
t0.FK_WRITER_ID=3Dt2.ID ) WHERE (t0.AGE < 100 AND (t1.title LIKE BINARY
'G%' OR t3.title LIKE BINARY 'G%'))
You have an error in your SQL syntax. Check the manual that corresponds
to your MySQL server version for the right syntax to use near '( BOOK t3
INNER JOIN WRITER t2 ON ( t2.ID=3Dt3.FK_WRITER_ID ) )

SELECT DISTINCT t0.ID, t0.LAST_NAME, t0.FIRST_NAME, t0.AGE,
t0.FK_WRITER_ID, t0.BIRTHDAY FROM WRITER t0 INNER JOIN BOOK t1 ON
t0.ID=3Dt1.FK_WRITER_ID INNER JOIN ( WRITER t2 INNER JOIN BOOK t3 ON
t2.ID=3Dt3.FK_WRITER_ID ) ON t0.FK_WRITER_ID=3Dt2.ID WHERE (t0.AGE < 100
AND (t1.title LIKE BINARY 'G%' OR t3.title LIKE BINARY 'G%'))

Post by Sebastien Bigaret
Hi,
Thanks for the report. I've searched the mysql-site for about half an
hour but couldn't find any hints on when the change happened. Do you
(or anyone here) have any clue?
Second, could you possibly confirm that the test
test_11_allQualifierOperators() in test_EditingContext_Global.py fails
when you patch is /not/ applied, and succeeds when it is applied? If it
does not, then I'll need to write a new one revealing that bug.
Given that this particular test behaves as expected, your patch will
be applied as-is and integrated to the main trunk. Thanks again for
reporting.
-- S=E9bastien.

Post by John Georgiadis
Hi Sebastian,
I get a syntax error from MySQL-4.0.11 when using nested JOINS created b=

y

Post by Sebastien Bigaret

Post by John Georgiadis
MySQLAdaptorLayer.MySQLExpression._addTableJoinsForAlias(). If I switch =

to

Post by Sebastien Bigaret

Post by John Georgiadis
Modeling.SQLExpression._addTableJoinsForAlias(), the problem is gone.
The following patch does this for mysql version >=3D 4.0.11. If anyone c=

an

Post by Sebastien Bigaret

Post by John Georgiadis
confirm this for an older version of MySQL, then feel free to lower the
version numbers in the tuple.

[Sebastien Bigaret]

Hi John and all,

I wish I had more time for this now :/ BTW, I just wanted to say thank
you John for the tests and the example. They reveal that the SQL
generated for MySQL is *incorrect* for such qualifiers.

I created bug ticket #857803 for that, where you'll also find the
patch John offered and which solves the problem for mysql 4.0+

https://sf.net/tracker/index.php?func=detail&aid=857803&group_id=58935&atid=489335

When a patch is available solving the problem for mysql 3.23 as well it
will be put there and announced here.


I'm afraid there is no working solution for mysql 3.23 using the JOIN
keyword (if there is one, I couldnt find it) for such qualifiers, so
we'll need to forget about the JOIN keyword for mysql 3.23 and use
explicit joints, i.e. w/ no join kw. but w/ the join condition in the
where clause (as this is already done for oracle8i).

mysql_server_version() will probably also check for an env. variable
before checking MySQLdb.get_client_info(), since it seems that MySQLdb
can be built w/ a 3.23 compatible library while speaking w/ a 4.0
server (I have an example at home:)

-- Sébastien.

Post by John Georgiadis
Unpatched and patched both succeed. However if I switch the last test
'(age<100) AND pygmalion.books.title like "G*"'
'(age<100) AND ((books.title like "G*") OR (pygmalion.books.title like
"G*"))'
Then the unpatched version fails, while the patched succeeds. I think,
this is related to two joins (Writer.books & Writer.pygmalion) going to
different directions. Below is the generated SQL in both cases
SELECT DISTINCT t0.ID, t0.LAST_NAME, t0.FIRST_NAME, t0.AGE,
t0.FK_WRITER_ID, t0.BIRTHDAY FROM BOOK t1 INNER JOIN WRITER t0 ON (
t0.ID=t1.FK_WRITER_ID ) ( BOOK t3 INNER JOIN WRITER t2 ON (
t2.ID=t3.FK_WRITER_ID ) ) INNER JOIN WRITER t0 ON (
t0.FK_WRITER_ID=t2.ID ) WHERE (t0.AGE < 100 AND (t1.title LIKE BINARY
'G%' OR t3.title LIKE BINARY 'G%'))
You have an error in your SQL syntax. Check the manual that corresponds
to your MySQL server version for the right syntax to use near '( BOOK t3
INNER JOIN WRITER t2 ON ( t2.ID=t3.FK_WRITER_ID ) )
SELECT DISTINCT t0.ID, t0.LAST_NAME, t0.FIRST_NAME, t0.AGE,
t0.FK_WRITER_ID, t0.BIRTHDAY FROM WRITER t0 INNER JOIN BOOK t1 ON
t0.ID=t1.FK_WRITER_ID INNER JOIN ( WRITER t2 INNER JOIN BOOK t3 ON
t2.ID=t3.FK_WRITER_ID ) ON t0.FK_WRITER_ID=t2.ID WHERE (t0.AGE < 100
AND (t1.title LIKE BINARY 'G%' OR t3.title LIKE BINARY 'G%'))

[Sebastien Bigaret]

Hi,

Post by Sebastien Bigaret
I created bug ticket #857803 for that, where you'll also find the
patch John offered and which solves the problem for mysql 4.0+
https://sf.net/tracker/index.php?func=detail&aid=857803&group_id=58935&atid=489335

I've submitted that will probably the definitive fix there. Note that if
you're using mysql server v4.0 or higher, and your MySQLdb.get_version_info()
returns 3.23 (just like mine :), you can set MDL_MYSQL_SERVER_VERSION to '4.0'
--if not, you won't get any JOIN keyword, just plain SQL89 statements, so
that's nothing really bad.

-- Sébastien.