jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/416333 )
Change subject: category_redirect.py: convert dict_keys to list when iterating
......................................................................
category_redirect.py: convert dict_keys to list when iterating
In Python 3, dict.keys() is a dynamic iterator that checks the dict
to ensure it is not modified during the iterations, so Python is able
to use less memory as it does not have to always construct the whole
list like in Python 2. Converting it to a list will make it behave
like Python 2, where the list of keys is static, with a larger
memory use.
Since the entire dict is in memory and nobody complained while they
are running this script in Python 2, it's probably safe to assume
the additional memory overhead is not too bad and we don't have to
make famcy code to avoid list().
Bug: T188850
Change-Id: I44f934396398799a1478e4cd3ccf8ecee9b86397
---
M scripts/category_redirect.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Dvorapa: Looks good to me, but someone else must approve
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/category_redirect.py b/scripts/category_redirect.py
index 6c699d5..285651c 100755
--- a/scripts/category_redirect.py
+++ b/scripts/category_redirect.py
@@ -346,7 +346,7 @@
pass
# delete record entries for non-existent categories
- for cat_name in record.keys():
+ for cat_name in list(record.keys()):
if pywikibot.Category(self.site,
self.catprefix + cat_name) not in catpages:
del record[cat_name]
--
To view, visit https://gerrit.wikimedia.org/r/416333
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I44f934396398799a1478e4cd3ccf8ecee9b86397
Gerrit-Change-Number: 416333
Gerrit-PatchSet: 2
Gerrit-Owner: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/415815 )
Change subject: Fix documentation for yu-tld and add edit summary on Serbian
......................................................................
Fix documentation for yu-tld and add edit summary on Serbian
.yu domain is already disabled before 8 years
Change-Id: I74e73b0367d9a675b835a1639b3f147d785e7970
---
M pywikibot/fixes.py
1 file changed, 4 insertions(+), 3 deletions(-)
Approvals:
Dvorapa: Looks good to me, but someone else must approve
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/fixes.py b/pywikibot/fixes.py
index 3a68c67..33b82f5 100644
--- a/pywikibot/fixes.py
+++ b/pywikibot/fixes.py
@@ -30,11 +30,11 @@
* datum - specific date formats in German
* correct-ar - Corrections for Arabic Wikipedia and any
Arabic wiki.
- * yu-tld - the yu top-level domain will soon be
- disabled, see
+ * yu-tld - Fix links to .yu domains because it is
+ disabled, see:
+ https://lists.wikimedia.org/pipermail/wikibots-l/2009-February/000290.html
* fckeditor - Try to convert FCKeditor HTML tags to wiki
syntax.
- https://lists.wikimedia.org/pipermail/wikibots-l/2009-February/000290.html
"""
__doc__ = __doc__ + parameter_help
@@ -599,6 +599,7 @@
'fr': ('Robot: Correction des liens pointant vers le domaine '
'.yu, qui expire en 2009'),
'ksh': u'Bot: de ahle .yu-Domains loufe us, dröm ußjetuusch',
+ 'sr': 'Бот: Исправљање линкова ка .yu домену',
},
'replacements': [
(u'www.budva.cg.yu', u'www.budva.rs'),
--
To view, visit https://gerrit.wikimedia.org/r/415815
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I74e73b0367d9a675b835a1639b3f147d785e7970
Gerrit-Change-Number: 415815
Gerrit-PatchSet: 6
Gerrit-Owner: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/415771 )
Change subject: weblinkchecker: use with-statement to acquire and release semaphore
......................................................................
weblinkchecker: use with-statement to acquire and release semaphore
Seriously, why are we doing this? With statements are awesome; try-
finally-s are okay/acceptable; but acquire/release like normal
procedures? Way too error error-prone and guaranteed to fail eventually.
Bug: T185561
Change-Id: I1b6f9dca58f0971ea6bf6c29718b4b17ce419cd7
---
M scripts/weblinkchecker.py
1 file changed, 94 insertions(+), 95 deletions(-)
Approvals:
Dalba: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/weblinkchecker.py b/scripts/weblinkchecker.py
index bc73f08..ce018a2 100755
--- a/scripts/weblinkchecker.py
+++ b/scripts/weblinkchecker.py
@@ -699,35 +699,34 @@
def setLinkDead(self, url, error, page, weblink_dead_days):
"""Add the fact that the link was found dead to the .dat file."""
- self.semaphore.acquire()
- now = time.time()
- if url in self.historyDict:
- timeSinceFirstFound = now - self.historyDict[url][0][1]
- timeSinceLastFound = now - self.historyDict[url][-1][1]
- # if the last time we found this dead link is less than an hour
- # ago, we won't save it in the history this time.
- if timeSinceLastFound > 60 * 60:
- self.historyDict[url].append((page.title(), now, error))
- # if the first time we found this link longer than x day ago
- # (default is a week), it should probably be fixed or removed.
- # We'll list it in a file so that it can be removed manually.
- if timeSinceFirstFound > 60 * 60 * 24 * weblink_dead_days:
- # search for archived page
- try:
- archiveURL = get_archive_url(url)
- except Exception as e:
- pywikibot.warning(
- 'get_closest_memento_url({0}) failed: {1}'.format(
- url, e))
- archiveURL = None
- if archiveURL is None:
- archiveURL = weblib.getInternetArchiveURL(url)
- if archiveURL is None:
- archiveURL = weblib.getWebCitationURL(url)
- self.log(url, error, page, archiveURL)
- else:
- self.historyDict[url] = [(page.title(), now, error)]
- self.semaphore.release()
+ with self.semaphore:
+ now = time.time()
+ if url in self.historyDict:
+ timeSinceFirstFound = now - self.historyDict[url][0][1]
+ timeSinceLastFound = now - self.historyDict[url][-1][1]
+ # if the last time we found this dead link is less than an hour
+ # ago, we won't save it in the history this time.
+ if timeSinceLastFound > 60 * 60:
+ self.historyDict[url].append((page.title(), now, error))
+ # if the first time we found this link longer than x day ago
+ # (default is a week), it should probably be fixed or removed.
+ # We'll list it in a file so that it can be removed manually.
+ if timeSinceFirstFound > 60 * 60 * 24 * weblink_dead_days:
+ # search for archived page
+ try:
+ archiveURL = get_archive_url(url)
+ except Exception as e:
+ pywikibot.warning(
+ 'get_closest_memento_url({0}) failed: {1}'.format(
+ url, e))
+ archiveURL = None
+ if archiveURL is None:
+ archiveURL = weblib.getInternetArchiveURL(url)
+ if archiveURL is None:
+ archiveURL = weblib.getWebCitationURL(url)
+ self.log(url, error, page, archiveURL)
+ else:
+ self.historyDict[url] = [(page.title(), now, error)]
def setLinkAlive(self, url):
"""
@@ -738,13 +737,13 @@
@return: True if previously found dead, else returns False.
"""
if url in self.historyDict:
- self.semaphore.acquire()
- try:
- del self.historyDict[url]
- except KeyError:
- # Not sure why this can happen, but I guess we can ignore this.
- pass
- self.semaphore.release()
+ with self.semaphore:
+ try:
+ del self.historyDict[url]
+ except KeyError:
+ # Not sure why this can happen, but I guess we can
+ # ignore this.
+ pass
return True
else:
return False
@@ -774,9 +773,8 @@
def report(self, url, errorReport, containingPage, archiveURL):
"""Report error on talk page of the page containing the dead link."""
- self.semaphore.acquire()
- self.queue.append((url, errorReport, containingPage, archiveURL))
- self.semaphore.release()
+ with self.semaphore:
+ self.queue.append((url, errorReport, containingPage, archiveURL))
def shutdown(self):
"""Finish thread."""
@@ -796,64 +794,65 @@
else:
time.sleep(0.1)
else:
- self.semaphore.acquire()
- (url, errorReport, containingPage, archiveURL) = self.queue[0]
- self.queue = self.queue[1:]
- talkPage = containingPage.toggleTalkPage()
- pywikibot.output(color_format(
- '{lightaqua}** Reporting dead link on {0}...{default}',
- talkPage.title(asLink=True)))
- try:
- content = talkPage.get() + "\n\n\n"
- if url in content:
- pywikibot.output(color_format(
- '{lightaqua}** Dead link seems to have already '
- 'been reported on {0}{default}',
- talkPage.title(asLink=True)))
- self.semaphore.release()
- continue
- except (pywikibot.NoPage, pywikibot.IsRedirectPage):
- content = u''
-
- if archiveURL:
- archiveMsg = u'\n' + \
- i18n.twtranslate(containingPage.site,
- 'weblinkchecker-archive_msg',
- {'URL': archiveURL})
- else:
- archiveMsg = u''
- # The caption will default to "Dead link". But if there is
- # already such a caption, we'll use "Dead link 2",
- # "Dead link 3", etc.
- caption = i18n.twtranslate(containingPage.site,
- 'weblinkchecker-caption')
- i = 1
- count = u''
- # Check if there is already such a caption on the talk page.
- while re.search('= *%s%s *=' % (caption, count),
- content) is not None:
- i += 1
- count = u' ' + str(i)
- caption += count
- content += '== %s ==\n\n%s\n\n%s%s\n--~~~~' % \
- (caption,
- i18n.twtranslate(containingPage.site,
- 'weblinkchecker-report'),
- errorReport,
- archiveMsg)
- comment = u'[[%s#%s|→]] %s' % \
- (talkPage.title(), caption,
- i18n.twtranslate(containingPage.site,
- 'weblinkchecker-summary'))
- try:
- talkPage.put(content, comment)
- except pywikibot.SpamfilterError as error:
+ with self.semaphore:
+ url, errorReport, containingPage, archiveURL = \
+ self.queue[0]
+ self.queue = self.queue[1:]
+ talkPage = containingPage.toggleTalkPage()
pywikibot.output(color_format(
- '{lightaqua}** SpamfilterError while trying to '
- 'change {0}: {1}{default}',
- talkPage.title(asLink=True), error.url))
+ '{lightaqua}** Reporting dead link on '
+ '{0}...{default}',
+ talkPage.title(asLink=True)))
+ try:
+ content = talkPage.get() + '\n\n\n'
+ if url in content:
+ pywikibot.output(color_format(
+ '{lightaqua}** Dead link seems to have '
+ 'already been reported on {0}{default}',
+ talkPage.title(asLink=True)))
+ continue
+ except (pywikibot.NoPage, pywikibot.IsRedirectPage):
+ content = ''
- self.semaphore.release()
+ if archiveURL:
+ archiveMsg = '\n' + \
+ i18n.twtranslate(
+ containingPage.site,
+ 'weblinkchecker-archive_msg',
+ {'URL': archiveURL})
+ else:
+ archiveMsg = ''
+ # The caption will default to "Dead link". But if there
+ # is already such a caption, we'll use "Dead link 2",
+ # "Dead link 3", etc.
+ caption = i18n.twtranslate(containingPage.site,
+ 'weblinkchecker-caption')
+ i = 1
+ count = ''
+ # Check if there is already such a caption on
+ # the talk page.
+ while re.search('= *%s%s *=' % (caption, count),
+ content) is not None:
+ i += 1
+ count = ' ' + str(i)
+ caption += count
+ content += '== %s ==\n\n%s\n\n%s%s\n--~~~~' % \
+ (caption,
+ i18n.twtranslate(containingPage.site,
+ 'weblinkchecker-report'),
+ errorReport,
+ archiveMsg)
+ comment = '[[%s#%s|→]] %s' % \
+ (talkPage.title(), caption,
+ i18n.twtranslate(containingPage.site,
+ 'weblinkchecker-summary'))
+ try:
+ talkPage.put(content, comment)
+ except pywikibot.SpamfilterError as error:
+ pywikibot.output(color_format(
+ '{lightaqua}** SpamfilterError while trying to '
+ 'change {0}: {1}{default}',
+ talkPage.title(asLink=True), error.url))
class WeblinkCheckerRobot(SingleSiteBot, ExistingPageBot):
--
To view, visit https://gerrit.wikimedia.org/r/415771
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1b6f9dca58f0971ea6bf6c29718b4b17ce419cd7
Gerrit-Change-Number: 415771
Gerrit-PatchSet: 2
Gerrit-Owner: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/415537 )
Change subject: [doc] Publish next release
......................................................................
[doc] Publish next release
Change-Id: If84c5ba8af8be7f6ebdd10ff267f581fbcc196d0
---
M HISTORY.rst
M docs/conf.py
2 files changed, 4 insertions(+), 3 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/HISTORY.rst b/HISTORY.rst
index 572780a..bb22897 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -1,7 +1,7 @@
Release history
===============
-Current release
+3.0.20180301
---------------
* Changed requirements for requests and sseclient
@@ -141,4 +141,5 @@
* First PyWikipediaBot framework release
* scripts and libraries for standardizing content
* tools for making minor modifications
-* script making interwiki links
\ No newline at end of file
+* script making interwiki links
+
diff --git a/docs/conf.py b/docs/conf.py
index d2cfb00..f37ad14 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -67,7 +67,7 @@
# The short X.Y version.
version = '3.0'
# The full version, including alpha/beta/rc tags.
-release = '3.0.20180204'
+release = '3.0.20180301'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
--
To view, visit https://gerrit.wikimedia.org/r/415537
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If84c5ba8af8be7f6ebdd10ff267f581fbcc196d0
Gerrit-Change-Number: 415537
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: Zoranzoki21 <zorandori4444(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>