Changeset 1315

Timestamp:

05/23/08 16:17:13 (6 months ago)

Author:

thomase

Message:

gracefully handle \u2018 and \u2019 in input strings, since these are commonly used in the BAtlas to bracket names thought incorrect

Files:

• pleiades.normalizer/trunk/pleiades/normalizer/base.py (modified) (1 diff)
• pleiades.normalizer/trunk/pleiades/normalizer/tests/batlas.txt (modified) (1 diff)

pleiades.normalizer/trunk/pleiades/normalizer/base.py

@@ -15 +15 @@
-242 : 'o', 243 : 'o', 244 : 'o', 245 : 'o', 246 : 'o', 248 : 'o', 249 : 'u',
-250 : 'u', 251 : 'u', 252 : 'u', 253 : 'y', 255 : 'y',
-
+, 8216 : '\'', 8217 : '\''
-}
-

pleiades.normalizer/trunk/pleiades/normalizer/tests/batlas.txt

@@ -27 +27 @@
-    >>> list(normalizer.normalizeN(unicode('Çaykenarı', 'utf-8')))
-    ['caykenari']
+    
+    >>> list(normalizer.normalizeN(unicode('‘Iotana’', 'utf-8')))
+    ['iotana']
+