jenkins-bot has submitted this change and it was merged.
Change subject: [FIX] Category: Load the category dump lazy
......................................................................
[FIX] Category: Load the category dump lazy
The script test failed, because it always told it loaded the
category dump (if the file existed). This loads the category dump
only as soon as it's used. Because that instance is now also storing
the filename it's possible to dump it into the same file without
determining the filename.
Change-Id: Ia3aa847afd18d21a5ae85e5c2647178ef29fba4a
---
M scripts/category.py
1 file changed, 19 insertions(+), 7 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/category.py b/scripts/category.py
index 95817ae..68dfc9d 100755
--- a/scripts/category.py
+++ b/scripts/category.py
@@ -138,15 +138,19 @@
"""
def __init__(self, rebuild=False, filename='category.dump.bz2'):
+ if not os.path.isabs(filename):
+ filename = config.datafilepath(filename)
+ self.filename = filename
+ self.loaded = False
if rebuild:
self.rebuild()
- else:
+
+ def _load(self):
+ if not self.loaded:
try:
- if not os.path.isabs(filename):
- filename = config.datafilepath(filename)
- f = bz2.BZ2File(filename, 'r')
+ f = bz2.BZ2File(self.filename, 'r')
pywikibot.output(u'Reading dump from %s'
- % config.shortpath(filename))
+ % config.shortpath(self.filename))
databases = pickle.load(f)
f.close()
# keys are categories, values are 2-tuples with lists as
@@ -162,6 +166,7 @@
def rebuild(self):
self.catContentDB = {}
self.superclassDB = {}
+ self.loaded = True
def getSubcats(self, supercat):
"""Return the list of subcategories for a given supercategory.
@@ -169,6 +174,7 @@
Saves this list in a temporary database so that it won't be loaded from
the server next time it's required.
"""
+ self._load()
# if we already know which subcategories exist here
if supercat in self.catContentDB:
return self.catContentDB[supercat][0]
@@ -185,6 +191,7 @@
Saves this list in a temporary database so that it won't be loaded from
the server next time it's required.
"""
+ self._load()
# if we already know which articles exist here
if cat in self.catContentDB:
return self.catContentDB[cat][1]
@@ -196,6 +203,7 @@
return articleset
def getSupercats(self, subcat):
+ self._load()
# if we already know which subcategories exist here
if subcat in self.superclassDB:
return self.superclassDB[subcat]
@@ -205,14 +213,18 @@
self.superclassDB[subcat] = supercatset
return supercatset
- def dump(self, filename='category.dump.bz2'):
+ def dump(self, filename=None):
"""Save the dictionaries to disk if not empty.
Pickle the contents of the dictionaries superclassDB and catContentDB
if at least one is not empty. If both are empty, removes the file from
the disk.
+
+ If the filename is None, it'll use the filename determined in __init__.
"""
- if not os.path.isabs(filename):
+ if filename is None:
+ filename = self.filename
+ elif not os.path.isabs(filename):
filename = config.datafilepath(filename)
if self.catContentDB or self.superclassDB:
pywikibot.output(u'Dumping to %s, please wait...'
--
To view, visit https://gerrit.wikimedia.org/r/158511
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia3aa847afd18d21a5ae85e5c2647178ef29fba4a
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Add RepeatingGenerator
......................................................................
Add RepeatingGenerator
In compat, we have many generators such as logpages(),
recentchanges() with parameter `repeat`, while in core
this parameter is missing. RepeatingGenerator is a way
to bring its functionality back.
Change-Id: I3da94de6bec0d5638f7be39597db90f7a1e7bc6d
---
M pywikibot/pagegenerators.py
M pywikibot/site.py
M tests/pagegenerators_tests.py
3 files changed, 72 insertions(+), 3 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
XZise: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index a46e27c..a26c58d 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -22,6 +22,7 @@
import codecs
import itertools
import re
+import time
import pywikibot
from pywikibot import date, config, i18n
from pywikibot.tools import deprecate_arg
@@ -983,6 +984,56 @@
yield page.toggleTalkPage()
+def RepeatingGenerator(generator, key_func=lambda x: x, sleep_duration=60,
+ total=None, **kwargs):
+ """Yield items in live time.
+
+ The provided generator must support parameter 'start', 'end',
+ 'reverse', and 'total' such as site.recentchanges(), site.logevents().
+
+ For example:
+
+ To fetch revisions in recentchanges in live time, call
+ gen = RepeatingGenerator(site.recentchanges, lambda x: x['revid'])
+
+ To fetch new pages in live time, call
+ gen = RepeatingGenerator(site.newpages, lambda x: x[0])
+
+ Note that other parameters not listed below will be passed
+ to the generator function. Parameter 'reverse', 'start', 'end'
+ will always be discarded to prevent the generator yielding items
+ in wrong order.
+
+ @param generator: a function returning a generator that will be queried
+ @param key_func: a function returning key that will be used to detect
+ duplicate entry
+ @param sleep_duration: duration between each query
+ @param total: if it is a positive number, iterate no more than this
+ number of items in total. Otherwise, iterate forever
+ @type total: int or None
+ @return: a generator yielding items in ascending order by time
+ """
+ kwargs.pop('reverse', None) # always get newest item first
+ kwargs.pop('start', None) # don't set start time
+ kwargs.pop('end', None) # don't set stop time
+
+ seen = set()
+ while total is None or len(seen) < total:
+ def filtered_generator():
+ for item in generator(total=None if seen else 1, **kwargs):
+ key = key_func(item)
+ if key not in seen:
+ seen.add(key)
+ yield item
+ if len(seen) == total:
+ return
+ else:
+ break
+ time.sleep(sleep_duration)
+ for item in list(filtered_generator())[::-1]:
+ yield item
+
+
@deprecate_arg("pageNumber", "step")
@deprecate_arg("lookahead", None)
def PreloadingGenerator(generator, step=50):
diff --git a/pywikibot/site.py b/pywikibot/site.py
index 13da02c..3d6ddef 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -4010,9 +4010,7 @@
"""Yield new articles (as Page objects) from recent changes.
Starts with the newest article and fetches the number of articles
- specified in the first argument. If repeat is True, it fetches
- Newpages again. If there is no new page, it blocks until there is
- one, sleeping between subsequent fetches of Newpages.
+ specified in the first argument.
The objects yielded are dependent on parameter returndict.
When true, it yields a tuple composed of a Page object and a dict of
diff --git a/tests/pagegenerators_tests.py b/tests/pagegenerators_tests.py
old mode 100644
new mode 100755
index 706ea11..1bcfa34
--- a/tests/pagegenerators_tests.py
+++ b/tests/pagegenerators_tests.py
@@ -128,6 +128,26 @@
quantifier='none')
self.assertEqual(len(tuple(gen)), 9)
+ def test_RepeatingGenerator(self):
+ self.assertFunction("RepeatingGenerator")
+ # site.recentchanges() includes external edits (from wikidata),
+ # so total=4 is not too high
+ items = list(
+ pagegenerators.RepeatingGenerator(self.site.recentchanges,
+ key_func=lambda x: x['revid'],
+ sleep_duration=10,
+ reverse=True,
+ namespaces=[0],
+ total=4)
+ )
+ self.assertEqual(len(items), 4)
+ timestamps = [pywikibot.Timestamp.fromISOformat(item['timestamp'])
+ for item in items]
+ self.assertEqual(sorted(timestamps), timestamps)
+ self.assertTrue(all(item['ns'] == 0 for item in items))
+ self.assertEqual(len(set(item['revid'] for item in items)), 4)
+
+
if __name__ == "__main__":
try:
unittest.main()
--
To view, visit https://gerrit.wikimedia.org/r/157056
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3da94de6bec0d5638f7be39597db90f7a1e7bc6d
Gerrit-PatchSet: 9
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Nullzero <nullzero.free(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Nullzero <nullzero.free(a)gmail.com>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [FIX] Upload: Allow to accept the current description
......................................................................
[FIX] Upload: Allow to accept the current description
Previously it wasn't possible to simply accept the description,
because the while condition never changed.
Change-Id: I3253fe4a86843b17210451f650821f3ef014d2e4
---
M scripts/upload.py
1 file changed, 17 insertions(+), 14 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/upload.py b/scripts/upload.py
index 5ab1607..9261661 100755
--- a/scripts/upload.py
+++ b/scripts/upload.py
@@ -193,29 +193,32 @@
pywikibot.output(u'The suggested description is:\n%s'
% self.description)
+ # Description must be set and verified
+ if not self.description:
+ self.verifyDescription = True
+
while not self.description or self.verifyDescription:
if not self.description:
pywikibot.output(
u'\03{lightred}It is not possible to upload a file '
'without a summary/description.\03{default}')
- if not self.description or self.verifyDescription:
- newDescription = u''
- # if no description, default is 'yes'
- default = 'y' if not self.description else 'n'
- choice = pywikibot.inputChoice(
- u'Do you want to change this description?',
- ['Yes', 'No', 'Quit'], ['y', 'n', 'q'], default)
- if choice == 'y':
- from pywikibot import editor as editarticle
- editor = editarticle.TextEditor()
- newDescription = editor.edit(self.description)
- elif choice == 'q':
- raise QuitKeyboardInterrupt
+ # if no description, default is 'yes'
+ default = 'y' if not self.description else 'n'
+ choice = pywikibot.inputChoice(
+ u'Do you want to change this description?',
+ ['Yes', 'No', 'Quit'], ['y', 'n', 'q'], default)
+ if choice == 'y':
+ from pywikibot import editor as editarticle
+ editor = editarticle.TextEditor()
+ newDescription = editor.edit(self.description)
# if user saved / didn't press Cancel
if newDescription:
self.description = newDescription
- break
+ if choice == 'q':
+ raise QuitKeyboardInterrupt
+ else:
+ self.verifyDescription = False
return filename
--
To view, visit https://gerrit.wikimedia.org/r/158529
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3253fe4a86843b17210451f650821f3ef014d2e4
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Add custom formatter to listpages.py
......................................................................
Add custom formatter to listpages.py
Custom format can be applied to the following items extrapolated from a
page object:
site: obtained from page._link._site
title: obtained from page._link._title
loc_title: obtained from page._link.canonical_title()
can_title: obtained from page._link.ns_title()
based either the canonical ns name or on the ns name in the
language specified by the -trans_to param
onsite: obtained from pywikibot.Site(trans_to, self.site.family)
trs_title: obtained from page._link.ns_title(onsite=onsite)
Added tests for Link.ns_title().
Change-Id: Ia911be7fb45a1e29515208b1b54ee6a213ffc29e
---
M pywikibot/page.py
M scripts/listpages.py
M tests/page_tests.py
3 files changed, 184 insertions(+), 6 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
XZise: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/pywikibot/page.py b/pywikibot/page.py
index 9a865a8..a6fe416 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -3840,6 +3840,41 @@
else:
return self.title
+ def ns_title(self, onsite=None):
+ """Return full page title, including namespace.
+
+ @param onsite: site object
+ if specified, present title using onsite local namespace,
+ otherwise use self canonical namespace.
+
+ if no corresponding namespace is found in onsite,
+ pywikibot.Error is raised.
+
+ """
+
+ ns_id = self.namespace
+ ns = self.site.namespaces()[ns_id]
+ ns_names = list(self.site.namespaces()[ns_id])
+
+ if onsite is None:
+ namespace = ns.canonical_name
+ else:
+ # look for corresponding ns in onsite by name comparison
+ for name in ns_names:
+ onsite_ns = ns.lookup_name(name, namespaces=onsite.namespaces())
+ # not found
+ if onsite_ns is None:
+ raise pywikibot.Error(
+ u'No corresponding namespace found for namespace %s on %s.'
+ % (self.site.namespaces()[ns_id], onsite))
+ else:
+ namespace = onsite_ns.custom_name
+
+ if namespace:
+ return u'%s:%s' % (namespace, self.title)
+ else:
+ return self.title
+
def astext(self, onsite=None):
"""Return a text representation of the link.
diff --git a/scripts/listpages.py b/scripts/listpages.py
index 3557566..aa10f47 100644
--- a/scripts/listpages.py
+++ b/scripts/listpages.py
@@ -5,12 +5,61 @@
These parameters are supported to specify which pages titles to print:
+-format Defines the output format.
+
+ Can be a custom string according to python string.format() notation or
+ can be selected by a number from following list (1 is default format):
+ 1 - u'{num:4d} {page.title}'
+ --> 10 PageTitle
+
+ 2 - u'{num:4d} {[[page.title]]}'
+ --> 10 [[PageTitle]]
+
+ 3 - u'{page.title}'
+ --> PageTitle
+
+ 4 - u'{[[page.title]]}'
+ --> [[PageTitle]]
+
+ 5 - u'{num:4d} \03{{lightred}}{page.loc_title:<40}\03{{default}}'
+ --> 10 PageTitle (colorised in lightred)
+
+ 6 - u'{num:4d} {page.loc_title:<40} {page.can_title:<40}'
+ --> 10 localised_Namespace:PageTitle canonical_Namespace:PageTitle
+
+ 7 - u'{num:4d} {page.loc_title:<40} {page.trs_title:<40}'
+ --> 10 localised_Namespace:PageTitle outputlang_Namespace:PageTitle
+ (*) requires "outputlang:lang" set.
+
+ num is the sequential number of the listed page.
+
+-outputlang Language for translation of namespaces
+
+-notitle Page title is not printed.
+
+-get Page content is printed.
+
+
+Custom format can be applied to the following items extrapolated from a
+ page object:
+
+ site: obtained from page._link._site
+
+ title: obtained from page._link._title
+
+ loc_title: obtained from page._link.canonical_title()
+
+ can_title: obtained from page._link.ns_title()
+ based either the canonical namespace name or on the namespace name
+ in the language specified by the -trans param;
+ a default value '******' will be used if no ns is found.
+
+ onsite: obtained from pywikibot.Site(outputlang, self.site.family)
+
+ trs_title: obtained from page._link.ns_title(onsite=onsite)
+
+
¶ms;
-
--notitle Page title is not printed.
-
--get Page content is printed.
-
"""
#
# (C) Pywikibot team, 2008-2014
@@ -20,15 +69,83 @@
__version__ = '$Id$'
#
+
import pywikibot
from pywikibot.pagegenerators import GeneratorFactory, parameterHelp
docuReplacements = {'¶ms;': parameterHelp}
+class Formatter(object):
+
+ """Structure with Page attributes exposed for formatting from cmd line."""
+
+ fmt_options = {
+ '1': u"{num:4d} {page.title}",
+ '2': u"{num:4d} [[{page.title}]]",
+ '3': u"{page.title}",
+ '4': u"[[{page.title}]]",
+ '5': u"{num:4d} \03{{lightred}}{page.loc_title:<40}\03{{default}}",
+ '6': u"{num:4d} {page.loc_title:<40} {page.can_title:<40}",
+ '7': u"{num:4d} {page.loc_title:<40} {page.trs_title:<40}",
+ }
+
+ # Identify which formats need outputlang
+ fmt_need_lang = [k for k, v in fmt_options.items() if 'trs_title' in v]
+
+ def __init__(self, page, outputlang=None, default='******'):
+ """
+ Constructor.
+
+ @param page: the page to be formatted.
+ @type page: Page object.
+ @param outputlang: language code in which namespace before title should
+ be translated.
+
+ Page namespace will be searched in Site(outputlang, page.site.family)
+ and, if found, its custom name will be used in page.title().
+
+ @type outputlang: str or None, if no translation is wanted.
+ @param default: default string to be used if no corresponding namespace
+ is found when outputlang is not None.
+
+ """
+
+ self.site = page._link._site
+ self.title = page._link.title
+ self.loc_title = page._link.canonical_title()
+ self.can_title = page._link.ns_title()
+ self.outputlang = outputlang
+ if outputlang is not None:
+ # Cache onsite in case of tranlations.
+ if not hasattr(self, "onsite"):
+ self.onsite = pywikibot.Site(outputlang, self.site.family)
+ try:
+ self.trs_title = page._link.ns_title(onsite=self.onsite)
+ # Fallback if no corresponding namespace is found in onsite.
+ except pywikibot.Error:
+ self.trs_title = u'%s:%s' % (default, page._link.title)
+
+ def output(self, num=None, fmt=1):
+ """Output formatted string."""
+ fmt = self.fmt_options.get(fmt, fmt)
+ # If selected format requires trs_title, outputlang must be set.
+ if (fmt in self.fmt_need_lang or
+ 'trs_title' in fmt and
+ self.outputlang is None):
+ raise ValueError(
+ u"Required format code needs 'outputlang' parameter set.")
+ if num is None:
+ return fmt.format(page=self)
+ else:
+ return fmt.format(num=num, page=self)
+
+
def main(*args):
gen = None
notitle = False
+ fmt = '1'
+ outputlang = None
page_get = False
# Process global args and prepare generator args parser
@@ -38,6 +155,11 @@
for arg in local_args:
if arg == '-notitle':
notitle = True
+ elif arg.startswith("-format:"):
+ fmt = arg[len("-format:"):]
+ fmt = fmt.replace(u'\\03{{', u'\03{{')
+ elif arg.startswith("-outputlang:"):
+ outputlang = arg[len("-outputlang:"):]
elif arg == '-get':
page_get = True
else:
@@ -47,12 +169,14 @@
if gen:
for i, page in enumerate(gen, start=1):
if not notitle:
- pywikibot.stdout("%4d: %s" % (i, page.title()))
+ page_fmt = Formatter(page, outputlang)
+ pywikibot.stdout(page_fmt.output(num=i, fmt=fmt))
if page_get:
# TODO: catch exceptions
pywikibot.output(page.text, toStdout=True)
else:
pywikibot.showHelp()
+
if __name__ == "__main__":
main()
diff --git a/tests/page_tests.py b/tests/page_tests.py
index 4383cb1..b112ee3 100644
--- a/tests/page_tests.py
+++ b/tests/page_tests.py
@@ -27,6 +27,8 @@
enwiki = pywikibot.Site("en", "wikipedia")
frwiki = pywikibot.Site("fr", "wikipedia")
itwikt = pywikibot.Site("it", "wiktionary")
+ enws = pywikibot.Site("en", "wikisource")
+ itws = pywikibot.Site("it", "wikisource")
namespaces = {0: [u""], # en.wikipedia.org namespaces for testing
1: [u"Talk:"], # canonical form first, then others
@@ -91,6 +93,7 @@
self.assertEqual(m.title, self.titles[title])
def testHashCmp(self):
+ """Test hash comparison."""
# All links point to en:wikipedia:Test
l1 = pywikibot.page.Link('Test', source=self.enwiki)
l2 = pywikibot.page.Link('en:Test', source=self.frwiki)
@@ -110,6 +113,22 @@
self.assertNotEqual(l1, other)
self.assertNotEqual(hash(l1), hash(other))
+ def test_ns_title(self):
+ """Test that title is returned with correct namespace."""
+ l1 = pywikibot.page.Link('Indice:Test', source=self.itws)
+ self.assertEqual(l1.ns_title(), 'Index:Test')
+ self.assertEqual(l1.ns_title(onsite=self.enws), 'Index:Test')
+
+ # wikisource:it kept Autore as canonical name
+ l2 = pywikibot.page.Link('Autore:Albert Einstein', source=self.itws)
+ self.assertEqual(l2.ns_title(), 'Autore:Albert Einstein')
+ self.assertEqual(l2.ns_title(onsite=self.enws), 'Author:Albert Einstein')
+
+ # Translation namespace does not exist on wikisource:it
+ l3 = pywikibot.page.Link('Translation:Albert Einstein', source=self.enws)
+ self.assertEqual(l3.ns_title(), 'Translation:Albert Einstein')
+ self.assertRaises(pywikibot.Error, l3.ns_title, onsite=self.itws)
+
class TestPageObject(PywikibotTestCase):
--
To view, visit https://gerrit.wikimedia.org/r/125501
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia911be7fb45a1e29515208b1b54ee6a213ffc29e
Gerrit-PatchSet: 14
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Ricordisamoa <ricordisamoa(a)openmailbox.org>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Replace httplib and urllib with httplib2
......................................................................
Replace httplib and urllib with httplib2
The module pywikibot.version contacts the code repository servers
to fetch version information. It uses httplib and urllib instead of
pywikibot 2.0's pywikibot.comms.http package which uses httplib2
with additional functionality like proxy & user-agent config, and
httplib2 has the capability to set a debug level on all requests.
As the version module is typically the first network communications of
pywikibot, as it logs version information after argument handling,
it is highly desirable that there are not several different libraries
performing network requests, each with their own quirks.
By delay-loading the pywikibot http until the online version methods
are invoked, module loading order problems as avoided.
This also replaces the hard-coded incorrect user-agent which was
'SVN/1.7.5-pywikibot1' to now be 'SVN/1.7.5 Pywikibot/x' where x
is always the current Pywikibot release version.
Bug: 66102
Change-Id: I5fa5c776cfd4c3239eb7fdca24a381e8029b26f5
---
M pywikibot/comms/http.py
M pywikibot/version.py
2 files changed, 17 insertions(+), 11 deletions(-)
Approvals:
John Vandenberg: Looks good to me, but someone else must approve
XZise: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/comms/http.py b/pywikibot/comms/http.py
index e9703ff..518cccd 100644
--- a/pywikibot/comms/http.py
+++ b/pywikibot/comms/http.py
@@ -252,7 +252,9 @@
if request.data[0].status == 504:
raise Server504Error("Server %s timed out" % site.hostname())
- if request.data[0].status != 200:
+ # HTTP status 207 is also a success status for Webdav FINDPROP,
+ # used by the version module.
+ if request.data[0].status not in (200, 207):
pywikibot.warning(u"Http response status %(status)s"
% {'status': request.data[0].status})
diff --git a/pywikibot/version.py b/pywikibot/version.py
index 2fd8cf9..9243910 100644
--- a/pywikibot/version.py
+++ b/pywikibot/version.py
@@ -13,7 +13,6 @@
import os
import time
import datetime
-import urllib
import subprocess
import pywikibot.config2 as config
@@ -146,15 +145,17 @@
@return: the git hash
@rtype: str
"""
- import httplib
+ from StringIO import StringIO
import xml.dom.minidom
- conn = httplib.HTTPSConnection('github.com')
- conn.request('PROPFIND', '/wikimedia/%s/!svn/vcc/default' % tag,
- "<?xml version='1.0' encoding='utf-8'?>"
- "<propfind xmlns=\"DAV:\"><allprop/></propfind>",
- {'Label': rev, 'User-Agent': 'SVN/1.7.5-pywikibot1'})
- resp = conn.getresponse()
- dom = xml.dom.minidom.parse(resp)
+ from pywikibot.comms import http
+
+ uri = 'https://github.com/wikimedia/%s/!svn/vcc/default' % tag
+ data = http.request(site=None, uri=uri, method='PROPFIND',
+ body="<?xml version='1.0' encoding='utf-8'?>"
+ "<propfind xmlns=\"DAV:\"><allprop/></propfind>",
+ headers={'label': str(rev), 'user-agent': 'SVN/1.7.5 {pwb}'})
+
+ dom = xml.dom.minidom.parse(StringIO(data))
hsh = dom.getElementsByTagName("C:git-commit")[0].firstChild.nodeValue
return hsh
@@ -239,9 +240,12 @@
@param repo: (optional) Online repository location
@type repo: URL or string
"""
+ from pywikibot.comms import http
+
url = repo or 'https://git.wikimedia.org/feed/pywikibot/core'
hsh = None
- buf = urllib.urlopen(url).readlines()
+ buf = http.request(site=None, uri=url)
+ buf = buf.split('\r\n')
try:
hsh = buf[13].split('/')[5][:-1]
except Exception as e:
--
To view, visit https://gerrit.wikimedia.org/r/153300
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I5fa5c776cfd4c3239eb7fdca24a381e8029b26f5
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Ignore missing docstring in PEP257
......................................................................
Ignore missing docstring in PEP257
For a while in order to reduce number of errors.
We'll get to them later
Change-Id: I2344b40eda4b437762c4ab48654b246c4db16ebe
---
M tox.ini
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tox.ini b/tox.ini
index 00fcf0d..3022ca6 100644
--- a/tox.ini
+++ b/tox.ini
@@ -14,7 +14,7 @@
deps = flake8
[testenv:flake8-docstrings]
-commands = flake8 --select=D {posargs} --jobs=1
+commands = flake8 {posargs} --jobs=1 --ignore=D102,D103
deps = flake8-docstrings
# Note: flake8 is run here with --jobs=1 to disable multiprocessing which stall
# the run with flake8-docstrings / pep257.
--
To view, visit https://gerrit.wikimedia.org/r/157289
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I2344b40eda4b437762c4ab48654b246c4db16ebe
Gerrit-PatchSet: 7
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Nullzero <nullzero.free(a)gmail.com>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>