jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/563714 )
Change subject: [tests] Skip redirected urls in generate_family_files_tests
......................................................................
[tests] Skip redirected urls in generate_family_files_tests
A lot of wikisource urls of siteinfo are redirected to mul main page
and the pywikibot._code_fam_from_url function fails because it cannot
find the site code in the family file. Skip the
test_attributes_after_run test of TestGenerateFamilyFiles class in
this case until we have an exception list of derived sites in our
family file.
- Load 10 different MWSite instances within
FamilyTestGenerator.getapis instead of only two. These are needed
to check whether an url is redirected.
- Remove allow_failure decorator and implement the behaviour inside
the test to enable subTest context manager.
- Shorten self.generator_instance to gen.
- Use subTest for all tests.
- Additional test whether the default site is loaded.
- Shorten the test whether gen.prefixed are in gen.wikis by using
assertCountEqual; take into account that the default site could
be selected via choice in getapis.
- Run through all langs from siteinfo but filter them; only process
those which are loaded and a MWSite exists.
- Check whether the siteinfo url is redirected and skip test in that
case.
- Also skip AssertionError due to T194138 which has the same effect
as allow_failure decorator to skip tests.
Bug: T241413
Change-Id: I1e0c2a5dc8cb0b9cbaac4a09b7b51b6c28775025
---
M tests/generate_family_files_tests.py
1 file changed, 56 insertions(+), 13 deletions(-)
Approvals:
Mpaa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/generate_family_files_tests.py b/tests/generate_family_files_tests.py
index 236e697..e8471ac 100644
--- a/tests/generate_family_files_tests.py
+++ b/tests/generate_family_files_tests.py
@@ -10,22 +10,36 @@
from random import choice
from pywikibot import Site
+from pywikibot.tools import PY2
from tests.aspects import unittest, DefaultSiteTestCase
-from tests.utils import allowed_failure
import generate_family_file
+if not PY2:
+ from urllib.parse import urlparse
+else:
+ from urlparse import urlparse
+
class FamilyTestGenerator(generate_family_file.FamilyFileGenerator):
"""Family file test creator."""
def getapis(self):
- """Only load additional two additional wikis randomly."""
+ """Only load additional ten additional different wikis randomly."""
save = self.langs
- self.langs = [choice(save), choice(save)]
- self.prefixes = [item['prefix'] for item in self.langs]
+ seen = set()
+ self.langs = []
+
+ while len(seen) < 10:
+ new = choice(save)
+ key = new['prefix']
+ if key not in seen:
+ seen.add(key)
+ self.langs.append(new)
+
+ self.prefixes = list(seen)
super(FamilyTestGenerator, self).getapis()
self.langs = save
@@ -55,17 +69,46 @@
self.assertIsInstance(self.generator_instance.wikis, dict)
self.assertIsInstance(self.generator_instance.langs, list)
- @allowed_failure # T194138
def test_attributes_after_run(self):
"""Test FamilyFileGenerator attributes after run()."""
- self.generator_instance.run()
- langs = [self.site.lang] + self.generator_instance.prefixes
- for lang in langs:
- self.assertIn(lang, self.generator_instance.wikis)
- for i in range(10):
- lang = choice(self.generator_instance.langs)
- site = Site(url=lang['url'])
- self.assertEqual(site.lang, lang['prefix'])
+ gen = self.generator_instance
+ gen.run()
+
+ with self.subTest(test='Test whether default is loaded'):
+ self.assertIn(self.site.lang, gen.wikis)
+
+ with self.subTest(test='Test element counts'):
+ if self.site.lang not in gen.prefixes:
+ gen.prefixes += [self.site.lang]
+ self.assertCountEqual(gen.prefixes, gen.wikis)
+
+ # test creating Site from url
+ # only test Sites for downloaded wikis (T241413)
+ for language in filter(lambda x: x['prefix'] in gen.wikis, gen.langs):
+ lang = language['prefix']
+ url = language['url']
+ wiki = gen.wikis[lang]
+ lang_parse = urlparse(url)
+ wiki_parse = urlparse(wiki.server)
+
+ with self.subTest(url=url):
+ if lang_parse.netloc != wiki_parse.netloc:
+ # skip redirected url (T241413)
+ self.skipTest(
+ '{} is redirected to {}'
+ .format(lang_parse.netloc, wiki_parse.netloc))
+
+ site = Site(url=url)
+
+ try: # T194138 to be solved
+ self.assertEqual(site.lang, lang,
+ 'url has lang "{lang}" '
+ 'but Site {site} has lang "{site.lang}"'
+ .format(site=site, lang=lang))
+ except AssertionError:
+ self.skipTest('KNOWN BUG: url has lang "{lang}" '
+ 'but Site {site} has lang "{site.lang}"'
+ .format(site=site, lang=lang))
if __name__ == '__main__': # pragma: no cover
--
To view, visit https://gerrit.wikimedia.org/r/563714
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1e0c2a5dc8cb0b9cbaac4a09b7b51b6c28775025
Gerrit-Change-Number: 563714
Gerrit-PatchSet: 6
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-Reviewer: Huji <huji.huji(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/565981 )
Change subject: [tests] Don't fail on setuptools and Python 2 deprecation warnings
......................................................................
[tests] Don't fail on setuptools and Python 2 deprecation warnings
setuptools make tests fail since version 41 as they throw invasive
deprecation warning. But even after lowering setuptools version,
Python 2 still make tests fail as it throws invasive warning as well.
Change-Id: I211b1ee8d781c6f26488aa998ed6f0b9169a5083
---
M .appveyor.yml
1 file changed, 1 insertion(+), 0 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/.appveyor.yml b/.appveyor.yml
index 8fd51f0..bf1211c 100644
--- a/.appveyor.yml
+++ b/.appveyor.yml
@@ -106,6 +106,7 @@
test_script:
- chcp 65001
- set PYTHONIOENCODING=utf8
+ - set PYTHONWARNINGS=ignore:::pkg_resources.py2_warn
- "mkdir %PYWIKIBOT_DIR%"
- "python -Werror::UserWarning -m generate_user_files -dir:%PYWIKIBOT_DIR% -family:wikipedia -lang:en -v -debug"
--
To view, visit https://gerrit.wikimedia.org/r/565981
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I211b1ee8d781c6f26488aa998ed6f0b9169a5083
Gerrit-Change-Number: 565981
Gerrit-PatchSet: 7
Gerrit-Owner: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/563711 )
Change subject: [bugfix] fix script_deps in script_tests.py
......................................................................
[bugfix] fix script_deps in script_tests.py
script_tests.check_script_deps checks for installed modules
but not for version dependecies. Therefore remove version
dependencies and enable related tests
Bug: T243169
Change-Id: I35afb4fcecba5b237614d7f4b42112bb8a2aac5f
---
M tests/script_tests.py
1 file changed, 5 insertions(+), 8 deletions(-)
Approvals:
Dvorapa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/script_tests.py b/tests/script_tests.py
index 5a86380..8f5db09 100644
--- a/tests/script_tests.py
+++ b/tests/script_tests.py
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""Test that each script can be compiled and executed."""
#
-# (C) Pywikibot team, 2014-2019
+# (C) Pywikibot team, 2014-2020
#
# Distributed under the terms of the MIT license.
#
@@ -20,18 +20,15 @@
scripts_path = join_root_path('scripts')
# These dependencies are not always the package name which is in setup.py.
-# e.g. 'PIL.ImageTk' is a object provided by several different pypi packages,
-# and setup.py requests that 'Pillow' is installed to provide 'PIL.ImageTk'.
-# Here, it doesn't matter which pypi package was requested and installed.
# Here, the name given to the module which will be imported is required.
script_deps = {
- 'flickrripper': ['flickrapi', 'Pillow'],
+ 'flickrripper': ['flickrapi', 'PIL'],
'imageharvest': ['bs4'],
'isbn': ['python-stdnum'],
- 'match_images': ['PIL.ImageTk'],
+ 'match_images': ['PIL.Image'],
'states_redirect': ['pycountry'],
- 'patrol': ['mwparserfromhell>=0.3.3'],
- 'weblinkchecker': ['memento_client>=0.5.1,!=0.6.0'],
+ 'patrol': ['mwparserfromhell'],
+ 'weblinkchecker': ['memento_client'],
}
if PY2:
--
To view, visit https://gerrit.wikimedia.org/r/563711
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I35afb4fcecba5b237614d7f4b42112bb8a2aac5f
Gerrit-Change-Number: 563711
Gerrit-PatchSet: 4
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/565780 )
Change subject: [cleanup] remove IsbnBot.run method
......................................................................
[cleanup] remove IsbnBot.run method
IsbnBot is derived from Bot class which has a common run method
and calls Bot.treat method with page parameter. It is not necessary
to have an additional run method to override the default behavior.
Change-Id: Ic5f437dc3eb764d7180f377a9fe56c7357958fec
---
M scripts/isbn.py
1 file changed, 0 insertions(+), 5 deletions(-)
Approvals:
Dvorapa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/isbn.py b/scripts/isbn.py
index 40ad1ae..8559f17 100755
--- a/scripts/isbn.py
+++ b/scripts/isbn.py
@@ -1533,11 +1533,6 @@
pywikibot.output('Page {0} is a redirect; skipping.'
.format(page.title(as_link=True)))
- def run(self):
- """Run the bot."""
- for page in self.generator:
- self.treat(page)
-
class IsbnWikibaseBot(WikidataBot):
--
To view, visit https://gerrit.wikimedia.org/r/565780
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic5f437dc3eb764d7180f377a9fe56c7357958fec
Gerrit-Change-Number: 565780
Gerrit-PatchSet: 1
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: jenkins-bot (75)
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/562048 )
Change subject: [deps] Update requests requirement for sseclient
......................................................................
[deps] Update requests requirement for sseclient
I think requests for sseclient should not be less than
our general requirement which is 2.20.1 currently.
Change-Id: I4da811b4592d6495f5cfb1ea08c2102c6adb0182
---
M pywikibot/comms/eventstreams.py
1 file changed, 2 insertions(+), 3 deletions(-)
Approvals:
D3r1ck01: Looks good to me, but someone else must approve
Dvorapa: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/comms/eventstreams.py b/pywikibot/comms/eventstreams.py
index 9c6429e..be5af0a 100644
--- a/pywikibot/comms/eventstreams.py
+++ b/pywikibot/comms/eventstreams.py
@@ -33,10 +33,9 @@
from pywikibot import config, debug, Timestamp, Site, warning
from pywikibot.tools import deprecated_args, StringTypes
-# requests >= 2.9 is required for eventstreams (T184713)
-if LooseVersion(requests.__version__) < LooseVersion('2.9'):
+if LooseVersion(requests.__version__) < LooseVersion('2.20.1'):
raise ImportError(
- 'requests >= 2.9 is required for EventStreams;\n'
+ 'requests >= 2.20.1 is required for EventStreams;\n'
"install it with 'pip install \"requests>=2.20.1\"'\n")
--
To view, visit https://gerrit.wikimedia.org/r/562048
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4da811b4592d6495f5cfb1ea08c2102c6adb0182
Gerrit-Change-Number: 562048
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Ottomata <aotto(a)wikimedia.org>
Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: jenkins-bot (75)