Unicode Problem

[spro]

Unicode Problem

[herleeyandi]

Got it, I change to this :

def unicode_to_ascii(s):
    try:
        s = unicode(s, 'utf-8')
    except NameError:
        pass
    return ''.join(
        c for c in unicodedata.normalize('NFD', s)
        if unicodedata.category(c) != 'Mn'
        and c in all_letters
    )
print(unicode_to_ascii('Ślusàrski'))

[herleeyandi]

But now I have this issue where the element of array always begin with u''. Anybody know how to resolve this?
-Thank you-

[spro]

The u'' is just to show they are Unicode strings.

[congruili]

you could do it by importing open function from io module and adding encoding='utf-8' :

from io import open

def read_langs(lang1, lang2, reverse=False):
    print("Reading lines...")

    # Read the file and split into lines
    filename = '../data/%s-%s.txt' % (lang1, lang2)
    # filename = '../%s-%s.txt' % (lang1, lang2)
    lines = open(filename, encoding='utf-8').read().strip().split('\n')
    # ......