test_util.doctest - more detailed doctests for the stdnum.util package

Copyright (C) 2017-2019 Arthur de Jong

This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301 USA


This file contains more detailed doctests for the stdnum.util package. It
tries to test more corner cases and detailed functionality. This module is
meant for internal use by stdnum modules and is not guaranteed to remain
stable and as such not part of the public API of stdnum.

>>> from stdnum.util import (
...     get_number_modules, get_module_name, get_module_description,
...     clean, isdigits, to_unicode)


The to_unicode() function is used to force conversion of a string to unicode
if it is not already a unicode string. This is mostly used to convert numbers
with non-ASCII characters in it.

>>> n_str = b'\xc3\x91'.decode('utf-8')  # Ñ character as unicode string
>>> to_unicode(n_str) ==  n_str
True
>>> to_unicode(n_str.encode('utf-8')) ==  n_str
True
>>> to_unicode(n_str.encode('iso-8859-1')) ==  n_str
True


The clean function will also convert all kinds of unicode special characters
to normal ASCII variations of the same characters to partially support OCR'ed
text and text copy-pasted from other applications.

>>> clean('０𝟽—𝟴𝟧 𝟟𝟑')  # various digits, a weird minus and a non-breaking space
'07-85 73'


The isdigits() function is used to replace the str.isdigit() function which
will also return True for all kinds on non-ASCII digits.

>>> three = b'\xe1\xad\x93'.decode('utf-8')  # ᭓  Balinese digit three
>>> three.isdigit()
True
>>> isdigits(three)
False
>>> isdigits('3')
True
>>> super_two = b'\xc2\xb2'.decode('utf-8')  # ² to the power of two
>>> super_two.isdigit()
True
>>> isdigits(super_two)
False


The get_module_name() function is used in the example WSGI application and
release scripts to get the number's name. It should not end in a dot (even
though the docstring subject should).

>>> from stdnum import isbn
>>> get_module_name(isbn)
'ISBN (International Standard Book Number)'
>>> any(get_module_name(mod).endswith('.') for mod in get_number_modules())
False


The get_module_description() function is used in the example WSGI application
to extract the extended description of the number for presentation purposes.
The doctests should not be present in the descriptions.

>>> from stdnum import isbn
>>> get_module_description(isbn).startswith(
...     'The ISBN is the International Standard Book Number')
True
>>> any('>>>' in get_module_description(mod) for mod in get_number_modules())
False


The get_cc_module() function can be used to find a country-specific
validation module that can be used to validate the number format. It should
handle aliases properly.

>>> from stdnum.util import get_cc_module
>>> get_cc_module('gb', 'vat').__name__
'stdnum.gb.vat'
>>> get_cc_module('nl', 'vat').__name__
'stdnum.nl.btw'
>>> get_cc_module('is', 'vat').__name__
'stdnum.is_.vsk'
>>> get_cc_module(u'nl', u'vat').__name__
'stdnum.nl.btw'
>>> get_cc_module('unknown', 'vat') is None
True
>>> get_cc_module('nl', 'unknown') is None
True