jenkins-bot has submitted this change and it was merged.
Change subject: Revert "Revert "set default userinterface_lang in config""
......................................................................
Revert "Revert "set default userinterface_lang in config""
This reverts commit a0a416b5a159a411a504caac2f8c504f5fa2790b,
after the problem causing the revert was fixed with 07f3f168263b.
Change-Id: Ie89ebdd2c4d518d735d5794e0b070558783df066
---
M pywikibot/config2.py
M pywikibot/i18n.py
2 files changed, 6 insertions(+), 6 deletions(-)
Approvals:
John Vandenberg: Looks good to me, but someone else must approve
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/config2.py b/pywikibot/config2.py
index 1b866d1..0491846 100644
--- a/pywikibot/config2.py
+++ b/pywikibot/config2.py
@@ -50,6 +50,7 @@
import sys
import types
+from locale import getdefaultlocale
from warnings import warn
from pywikibot.logging import error, output, warning
@@ -1085,6 +1086,8 @@
'forward slashs as a path delimiter or to escape the normal '
'path delimiter.')
+if userinterface_lang is None:
+ userinterface_lang = getdefaultlocale()[0].split('_')[0]
# Fix up default site
if family == 'wikipedia' and mylang == 'language':
diff --git a/pywikibot/i18n.py b/pywikibot/i18n.py
index fa60f4b..5739837 100644
--- a/pywikibot/i18n.py
+++ b/pywikibot/i18n.py
@@ -27,7 +27,6 @@
#
import json
-import locale
import os
import pkgutil
import re
@@ -715,9 +714,8 @@
"""
Ask the user a question, return the user's answer.
- The prompt message is retrieved via L{twtranslate} and either uses the
- config variable 'userinterface_lang' or the default locale as the language
- code.
+ The prompt message is retrieved via L{twtranslate} and uses the
+ config variable 'userinterface_lang'.
@param twtitle: The TranslateWiki string title, in <package>-<key> format
@param parameters: The values which will be applied to the translated text
@@ -733,8 +731,7 @@
else:
prompt = fallback_prompt
else:
- code = config.userinterface_lang or \
- locale.getdefaultlocale()[0].split('_')[0]
+ code = config.userinterface_lang
prompt = twtranslate(code, twtitle, parameters)
return pywikibot.input(prompt, password)
--
To view, visit https://gerrit.wikimedia.org/r/203896
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie89ebdd2c4d518d735d5794e0b070558783df066
Gerrit-PatchSet: 5
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [bugfix] force wikistats fields to unicode when using xml format
......................................................................
[bugfix] force wikistats fields to unicode when using xml format
Bug: T129268
Change-Id: I26cc4dbc7a9a04b5c6545bbacfd52a07e6bb9a69
---
M pywikibot/data/wikistats.py
1 file changed, 3 insertions(+), 2 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/wikistats.py b/pywikibot/data/wikistats.py
index cae468f..86d9a72 100644
--- a/pywikibot/data/wikistats.py
+++ b/pywikibot/data/wikistats.py
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""Objects representing WikiStats API."""
#
-# (C) Pywikibot team, 2014
+# (C) Pywikibot team, 2014-2016
#
# Distributed under the terms of the MIT license.
from __future__ import absolute_import, unicode_literals
@@ -14,6 +14,7 @@
if sys.version_info[0] > 2:
import csv
+ unicode = str
else:
try:
import unicodecsv as csv
@@ -184,7 +185,7 @@
site = {}
for field in row.findall('field'):
- site[field.get('name')] = field.text
+ site[field.get('name')] = unicode(field.text)
data.append(site)
--
To view, visit https://gerrit.wikimedia.org/r/275780
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I26cc4dbc7a9a04b5c6545bbacfd52a07e6bb9a69
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [Bugfix] compare "-user" instead of "-user:"
......................................................................
[Bugfix] compare "-user" instead of "-user:"
generate_user_files with -user option failed and no user-config.py was
created. A valid config.mylang is needed to run script_tests. The 'language'
language from config2.py leads to missing handlers error. This patch solves it.
Bug: T128318
Change-Id: I1669581b7b5e023dfa0a8088e9117fd336ebc234
---
M pywikibot/bot.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/bot.py b/pywikibot/bot.py
index 6990c79..895e56f 100644
--- a/pywikibot/bot.py
+++ b/pywikibot/bot.py
@@ -850,7 +850,7 @@
config.family = value
elif option == '-lang':
config.mylang = value
- elif option == '-user:':
+ elif option == '-user':
username = value
elif option in ('-putthrottle', '-pt'):
config.put_throttle = int(value)
--
To view, visit https://gerrit.wikimedia.org/r/275799
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I1669581b7b5e023dfa0a8088e9117fd336ebc234
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Change i18n.input test lang from arz to nn
......................................................................
Change i18n.input test lang from arz to nn
The test test_pagegen_i18n_input requires the
test site code is a known locale, and doesnt have
a translation for the i18n message, causing the
i18n to fallback to another language with a translation.
The current test code 'arz' is not recognised by locale.
Change the code to 'nn', which falls back to 'nb',
and update the test to confirm that the translation
for 'nb' is shown when the locale is 'nn' using LC_ALL
environment variable and config.userinterface_lang.
Also expose a list of modified configuration variables
so that the test class can fail if config.userinterface_lang
has been set to any value other than the default None.
Bug: T128991
Change-Id: I6c54f902285101b9c13af4e6b2b75d4237cb3f3b
---
M pywikibot/config2.py
M tests/i18n_tests.py
M tests/utils.py
3 files changed, 185 insertions(+), 11 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/config2.py b/pywikibot/config2.py
index 7fc9d7a..d08c81f 100644
--- a/pywikibot/config2.py
+++ b/pywikibot/config2.py
@@ -1039,6 +1039,8 @@
_modified = [_key for _key in _gl
if _uc[_key] != globals()[_key] or
_key in ('usernames', 'sysopnames', 'disambiguation_comment')]
+# Retain the list of modified key names
+__modified__ = _modified
if ('user_agent_format' in _modified):
_right_user_agent_format = re.sub(r'{httplib2(:|})', r'{http_backend\1',
diff --git a/tests/i18n_tests.py b/tests/i18n_tests.py
index de85354..32f5f3a 100644
--- a/tests/i18n_tests.py
+++ b/tests/i18n_tests.py
@@ -357,27 +357,35 @@
"""Test i18n.input."""
family = 'wikipedia'
- code = 'arz'
+ code = 'nn'
+ alt_code = 'nb'
message_package = 'scripts.i18n'
+ message = 'pywikibot-enter-category-name'
@classmethod
def setUpClass(cls):
"""Verify that a translation does not yet exist."""
+ if 'userinterface_lang' in pywikibot.config.__modified__:
+ raise unittest.SkipTest(
+ 'user-config has a modified userinterface_lang')
+
super(InputTestCase, cls).setUpClass()
- if cls.code in i18n.twget_keys('pywikibot-enter-category-name'):
+ if cls.code in i18n.twget_keys(cls.message):
raise unittest.SkipTest(
'%s has a translation for %s'
- % (cls.code, 'pywikibot-enter-category-name'))
+ % (cls.code, cls.message))
def test_pagegen_i18n_input(self):
- """Test i18n.input via ."""
- result = self._execute(args=['listpages', '-cat'],
- data_in='non-existant-category\n',
- timeout=5)
+ """Test i18n.input fallback via pwb and LC_ALL."""
+ expect = i18n.twtranslate(self.alt_code, self.message, fallback=False)
- self.assertIn('Please enter the category name:', result['stderr'])
+ result = self._execute(args=['listpages', '-cat'],
+ data_in='non-existant-category\r\n',
+ timeout=20)
+
+ self.assertIn(expect, result['stderr'])
class MissingPackageTestCase(TWNSetMessagePackageBase,
diff --git a/tests/utils.py b/tests/utils.py
index 0616002..bc5cfa2 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""Test utilities."""
#
-# (C) Pywikibot team, 2013-2015
+# (C) Pywikibot team, 2013-2016
#
# Distributed under the terms of the MIT license.
#
@@ -10,10 +10,12 @@
#
import inspect
import json
+import locale
import os
import re
import subprocess
import sys
+import tempfile
import time
import traceback
import warnings
@@ -42,11 +44,25 @@
from tests import _pwb_py
from tests import unittest # noqa
+from tests import unittest_print
OSWIN32 = (sys.platform == 'win32')
PYTHON_26_CRYPTO_WARN = ('Python 2.6 is no longer supported by the Python core '
'team, please upgrade your Python.')
+
+WIN32_LOCALE_UPDATE = """
+<gs:GlobalizationServices xmlns:gs="urn:longhornGlobalizationUnattend">
+ <gs:UserList>
+ <gs:User UserID="Current" CopySettingsToDefaultUserAcct="true"
+ CopySettingsToSystemAcct="true"/>
+ </gs:UserList>
+
+ <gs:UserLocale>
+ <gs:Locale Name="%s" SetAsCurrent="true" ResetAllSettings="false"/>
+ </gs:UserLocale>
+</gs:GlobalizationServices>
+"""
class DrySiteNote(RuntimeWarning):
@@ -626,6 +642,132 @@
self._module.http = self._old_http
+def is_simple_locale_with_region(locale):
+ """Check if a locale is only an ISO and region code."""
+ # Some locale are unicode names, which are not valid
+ try:
+ lang, sep, qualifier = locale.partition('_')
+ except UnicodeDecodeError:
+ return False
+ if '-' in lang:
+ return False
+ # Only allow qualifiers that look like a country code, without any suffix
+ if qualifier and len(qualifier) == 2:
+ return True
+ else:
+ return False
+
+
+def get_simple_locales():
+ """Get list of simple locales."""
+ return [locale_code for locale_code in sorted(locale.locale_alias.keys())
+ if is_simple_locale_with_region(locale_code)]
+
+
+def generate_locale(lang, region=True, encoding='utf8'):
+ """
+ Generate a locale string.
+
+ @param lang: language code
+ @type lang: str
+ @param region: region code; if True, a random one will be used
+ @type region: str or True
+ @param encoding: encoding name
+ @type encoding: str
+ @rtype: str
+ """
+ locale_prefix = lang + '_'
+
+ if region is True:
+ locales = get_simple_locales()
+
+ lang_locales = [code for code in locales
+ if code.startswith(locale_prefix)]
+ assert(lang_locales)
+
+ # Get a region from the first locale
+ lang, sep, region = lang_locales[0].partition('_')
+ assert lang and sep and region
+
+ if region:
+ locale_code = locale_prefix + region.upper()
+ else:
+ locale_code = lang
+
+ if encoding:
+ locale_code += '.' + encoding
+
+ return locale_code
+
+
+def execute_with_temp_text_file(text, command, **kwargs):
+ """
+ Perform command on a temporary file.
+
+ @param text: contents of temporary file
+ @type text: str
+ @param command: command to execute with {0} replaced with the filename
+ @type command: str
+ @param kwargs: parameters for tempfile.mkstemp/tempfile.NamedTemporaryFile,
+ such as prefix, suffix and dir
+ """
+ options = {
+ 'shell': True,
+ 'stdout': subprocess.PIPE,
+ 'stderr': subprocess.STDOUT,
+ }
+
+ # NamedTemporaryFile does not work correctly with win32_set_global_locale
+ # subprocess.Popen is a context handler in Python 3.2+
+ if OSWIN32 or PY2:
+ (fd, filename) = tempfile.mkstemp(text=True, **kwargs)
+ try:
+ os.close(fd)
+ with open(filename, 'wt') as f:
+ f.write(text)
+
+ command = command.format(filename)
+
+ p = subprocess.Popen(command, **options)
+ out = p.communicate()[0]
+
+ # Python 2 raises an exception when attempting to close the process
+ # Python 3 does not allow the file to be removed until the process
+ # has been closed
+ if not PY2:
+ p.terminate()
+ finally:
+ try:
+ os.remove(filename)
+ except OSError:
+ # As it is a temporary file, the OS should clean it up
+ unittest_print('Could not delete {0}'.format(filename))
+ else:
+ with tempfile.NamedTemporaryFile(mode='w+t', **kwargs) as f:
+ f.write(text)
+ f.flush()
+ command = command.format(f.name)
+ with subprocess.Popen(command, **options) as p:
+ out = p.communicate()[0]
+
+ if out:
+ unittest_print('command "{0}" output: {1}'.format(command, out))
+
+
+def win32_set_global_locale(locale_code):
+ """Set global locale on win32."""
+ locale_code = locale_code.split('.')[0]
+ win_locale_code = locale_code.replace('_', '-')
+ locale_update_xml = WIN32_LOCALE_UPDATE % win_locale_code
+ command = 'control.exe intl.cpl,,/f:"{0}"'
+ execute_with_temp_text_file(locale_update_xml, command, suffix='.xml')
+
+ actual_code = locale.getdefaultlocale()[0]
+ assert locale_code == actual_code, \
+ ('locale code {0} not set; actual code is {1}'
+ .format(locale_code, actual_code))
+
+
def execute(command, data_in=None, timeout=0, error=None):
"""
Execute a command and capture outputs.
@@ -661,8 +803,26 @@
env[str('PYTHONIOENCODING')] = str(config.console_encoding)
# LC_ALL is used by i18n.input as an alternative for userinterface_lang
- if pywikibot.config.userinterface_lang:
- env[str('LC_ALL')] = str(pywikibot.config.userinterface_lang)
+ # A complete locale string needs to be created, so the country code
+ # is guessed, however it is discarded when loading config.
+ if config.userinterface_lang:
+ current_locale = locale.getdefaultlocale()[0].split('.')[0]
+ locale_prefix = str(config.userinterface_lang + '_')
+
+ if not current_locale.startswith(locale_prefix):
+ locale_code = generate_locale(
+ config.userinterface_lang,
+ encoding=config.console_encoding)
+
+ env[str('LC_ALL')] = str(locale_code)
+
+ if OSWIN32:
+ # This is not multiprocessing safe, as it affects all processes
+ win32_set_global_locale(locale_code)
+ else:
+ current_locale = None
+ else:
+ current_locale = None
# Set EDITOR to an executable that ignores all arguments and does nothing.
env[str('EDITOR')] = str('call' if OSWIN32 else 'true')
@@ -720,6 +880,10 @@
stderr_lines += p.stderr.read()
data_out = p.communicate()
+
+ if OSWIN32 and current_locale:
+ win32_set_global_locale(current_locale)
+
return {'exit_code': p.returncode,
'stdout': data_out[0].decode(config.console_encoding),
'stderr': (stderr_lines + data_out[1]).decode(config.console_encoding)}
--
To view, visit https://gerrit.wikimedia.org/r/203963
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I6c54f902285101b9c13af4e6b2b75d4237cb3f3b
Gerrit-PatchSet: 17
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Re-add featured script to the auto run list
......................................................................
Re-add featured script to the auto run list
It was removed in 38ac044 when featured was moved into scripts/archive.
Also tidy other parts of that commit, including adding a deprecation
warning to the featured.py script.
Bug: T128955
Change-Id: I1e373b9229b479227e3df30b51cb20915d1f0c47
---
M scripts/archive/README
M scripts/archive/featured.py
M tests/script_tests.py
M tox.ini
4 files changed, 16 insertions(+), 3 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/archive/README b/scripts/archive/README
index 92d5958..1e70cad 100644
--- a/scripts/archive/README
+++ b/scripts/archive/README
@@ -1 +1,6 @@
-This folder consists scripts that will not maintened but we keep them for historical reasons.
+This folder consists scripts that will not be improved, but are kept for
+backwards compatibility with a deprecation notice.
+
+featured.py is no longer compatible with Wikimedia sites, and no other wiki
+online could be found that is compatible with this script.
+See https://phabricator.wikimedia.org/T128955 for more information.
diff --git a/scripts/archive/featured.py b/scripts/archive/featured.py
index a29b05f..a6a661f 100755
--- a/scripts/archive/featured.py
+++ b/scripts/archive/featured.py
@@ -76,6 +76,7 @@
from pywikibot.pagegenerators import PreloadingGenerator
from pywikibot.tools.formatter import color_format
+from pywikibot.tools import issue_deprecation_warning
if sys.version_info[0] > 2:
unichr = chr
@@ -620,7 +621,13 @@
@type args: list of unicode
"""
options = {}
- for arg in pywikibot.handle_args(args):
+ local_args = pywikibot.handle_args(args)
+
+ issue_deprecation_warning(
+ 'featured.py script', 'Wikibase Client extension',
+ 0, UserWarning)
+
+ for arg in local_args:
if arg.startswith('-fromlang:'):
options[arg[1:9]] = arg[10:].split(",")
elif arg.startswith('-after:'):
diff --git a/tests/script_tests.py b/tests/script_tests.py
index c0bb625..4fdd9ec 100644
--- a/tests/script_tests.py
+++ b/tests/script_tests.py
@@ -122,6 +122,7 @@
'checkimages',
'clean_sandbox',
'disambredir',
+ 'featured',
'imagerecat',
'login',
'lonelypages',
diff --git a/tox.ini b/tox.ini
index b01f51a..b3f567d 100644
--- a/tox.ini
+++ b/tox.ini
@@ -128,7 +128,7 @@
tests/pwb/ : +T001, T003
scripts/casechecker.py,scripts/imagetransfer.py : +T001, T003
scripts/maintenance/make_i18n_dict.py : +T001, T003
- scripts/archive/ : +D102, D103
+ scripts/archive/featured.py : +D102, D103
scripts/script_wui.py : +D102
scripts/makecat.py : +D103
scripts/interwiki.py,/""/ : +P102
--
To view, visit https://gerrit.wikimedia.org/r/275717
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I1e373b9229b479227e3df30b51cb20915d1f0c47
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>