Pywikibot-commits October 2020

pywikibot-commits@lists.wikimedia.org

1 participants
186 discussions

[Gerrit] ...core[master]: [script] port speedy_delete.py to core

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/148646 ) Change subject: [script] port speedy_delete.py to core ...................................................................... [script] port speedy_delete.py to core - SpeedyRobot renamed to SpeedyBot - try to detect the CSD category from Wikidata: https://www.wikidata.org/wiki/Q5964 for Wikipedia, Wikisource, Wikiquote, Wikinews and Commons instead of using the hard-coded one (this also adds support for Wikivoyage) - reworded the first paragraphs in the docstring - names changed from lowerCamelCase to snake_case - use BaseBot.run for the inner loop - use treat_page to process the page to be deleted - adjust lines to be printed - rename refreshGenerator to setup and setup the generator for every outer loop - simplify code - Update script_tests.py Bug: T66880 Change-Id: I49733820fe77bfef95475d4e2e39c2cd9d974226 --- M docs/scripts/scripts.rst M scripts/README.rst A scripts/speedy_delete.py M tests/script_tests.py 4 files changed, 504 insertions(+), 6 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/docs/scripts/scripts.rst b/docs/scripts/scripts.rst index 99e95a2..0a97a40 100644 --- a/docs/scripts/scripts.rst +++ b/docs/scripts/scripts.rst @@ -330,6 +330,11 @@ .. automodule:: scripts.spamremove +scripts.speedy\_delete script +----------------------------- + +.. automodule:: scripts.speedy_delete + scripts.standardize\_interwiki script ------------------------------------- diff --git a/scripts/README.rst b/scripts/README.rst index 5b009e5..fc01e18 100644 --- a/scripts/README.rst +++ b/scripts/README.rst @@ -209,15 +209,13 @@ | states-redirect.py | A robot to add redirects to cities for state | | | abbreviations. | +------------------------+---------------------------------------------------------+ - | #speedy_delete.py | This bot load a list of pages from the category of | - | | candidates for speedy deletion and give the | - | | user an interactive prompt to decide whether | - | | each should be deleted or not. | + | speedy_delete.py | Help sysops to quickly check and/or delete pages listed | + | | for speedy deletion. | +------------------------+---------------------------------------------------------+ | #spellcheck.py | This bot spellchecks wiki pages. | +------------------------+---+-----------------------------------------------------+ - | #standardize_notes.py | Converts external links and notes/references to | - | | Footnote3 ref/note format. Rewrites References. | + | #standardize_notes.py | Converts external links and notes/references to | + | | Footnote3 ref/note format. Rewrites References. | +----------------------------+-----------------------------------------------------+ | #statistics_in_wikitable.py| This bot renders statistics provided by | | | [[Special:Statistics]] in a table on a wiki page. | diff --git a/scripts/speedy_delete.py b/scripts/speedy_delete.py new file mode 100644 index 0000000..aaf6ad9 --- /dev/null +++ b/scripts/speedy_delete.py @@ -0,0 +1,494 @@ +# -*- coding: utf-8 -*- +""" +Help sysops to quickly check and/or delete pages listed for speedy deletion. + +This bot trawls through candidates for speedy deletion in a fast +and semi-automated fashion. It displays the contents of each page +one at a time and provides a prompt for the user to skip or delete the page. +Of course, this will require a sysop account. + +Future upcoming options include the ability to untag a page as not being +eligible for speedy deletion, as well as the option to commute its sentence to +Proposed Deletion (see [[en:WP:PROD]] for more details). Also, if the article +text is long, to prevent terminal spamming, it might be a good idea to truncate +it just to the first so many bytes. + +WARNING: This tool shows the contents of the top revision only. It is possible +that a vandal has replaced a perfectly good article with nonsense, which has +subsequently been tagged by someone who didn't realize it was previously a good +article. The onus is on you to avoid making these mistakes. + +NOTE: This script currently only works for the Wikipedia project. +""" +# +# (C) Pywikibot team, 2007-2020 +# +# Distributed under the terms of the MIT license. +# +import time + +from textwrap import fill, wrap + +import pywikibot +from pywikibot import i18n, pagegenerators +from pywikibot.bot import ExistingPageBot, SingleSiteBot +from pywikibot.tools.formatter import color_format + + +class SpeedyBot(SingleSiteBot, ExistingPageBot): + """Bot to delete pages which are tagged as speedy deletion. + + This bot will load a list of pages from the category of candidates for + speedy deletion on the language's wiki and give the user an interactive + prompt to decide whether each should be deleted or not. + """ + + LINES = 22 + + csd_cat_item = 'Q5964' + + csd_cat_title = { + 'wikiversity': { + 'beta': 'Category:Candidates for speedy deletion', + }, + 'wikibooks': { + 'en': 'Category:Candidates for speedy deletion', + }, + 'incubator': {'incubator': 'Category:Maintenance:Delete'}, + } + + # If the site has several templates for speedy deletion, it might be + # possible to find out the reason for deletion by the template used. + # _default will be used if no such semantic template was used. + deletion_messages = { + 'wikipedia': { + 'ar': { + '_default': + 'حذف مرشح للحذف السريع حسب ' + '[[وProject:حذف سريع|معايير الحذف السريع]]', + }, + 'cs': { + '_default': + 'Bylo označeno k ' + '[[Wikipedie:Rychlé smazání|rychlému smazání]]', + }, + 'de': { + '_default': + 'Lösche Artikel nach ' + '[[Wikipedia:Schnelllöschantrag|Schnelllöschantrag]]', + }, + 'en': { + '_default': + 'Deleting candidate for speedy deletion per ' + '[[WP:CSD|CSD]]', + 'db-author': + 'Deleting page per [[WP:CSD|CSD]] G7: ' + 'Author requests deletion and is its only editor.', + 'db-nonsense': + 'Deleting page per [[WP:CSD|CSD]] G1: ' + 'Page is patent nonsense or gibberish.', + 'db-test': 'Deleting page per [[WP:CSD|CSD]] G2: Test page.', + 'db-nocontext': + 'Deleting page per [[WP:CSD|CSD]] A1: ' + 'Short article that provides little or no context.', + 'db-empty': + 'Deleting page per [[WP:CSD|CSD]] A1: Empty article.', + 'db-attack': + 'Deleting page per [[WP:CSD|CSD]] G10: ' + 'Page that exists solely to attack its subject.', + 'db-catempty': + 'Deleting page per [[WP:CSD|CSD]] C1: Empty category.', + 'db-band': + 'Deleting page per [[WP:CSD|CSD]] A7: ' + 'Article about a non-notable band.', + 'db-banned': + 'Deleting page per [[WP:CSD|CSD]] G5: ' + 'Page created by a banned user.', + 'db-bio': + 'Deleting page per [[WP:CSD|CSD]] A7: ' + 'Article about a non-notable person.', + 'db-notenglish': + 'Deleting page per [[WP:CSD|CSD]] A2: ' + "Article isn't written in English.", + 'db-copyvio': + 'Deleting page per [[WP:CSD|CSD]] G12: ' + 'Page is a blatant copyright violation.', + 'db-repost': + 'Deleting page per [[WP:CSD|CSD]] G4: ' + 'Recreation of previously deleted material.', + 'db-vandalism': + 'Deleting page per [[WP:CSD|CSD]] G3: Blatant vandalism.', + 'db-talk': + 'Deleting page per [[WP:CSD|CSD]] G8: ' + 'Talk page of a deleted or non-existent page.', + 'db-spam': + 'Deleting page per [[WP:CSD|CSD]] G11: ' + 'Blatant advertising.', + 'db-disparage': + 'Deleting page per [[WP:CSD|CSD]] T1: ' + 'Divisive or inflammatory template.', + 'db-r1': + 'Deleting page per [[WP:CSD|CSD]] R1: ' + 'Redirect to a deleted or non-existent page.', + 'db-experiment': + 'Deleting page per [[WP:CSD|CSD]] G2: ' + 'Page was created as an experiment.', + }, + 'fa': { + '_default': + 'حذف مرشَّح للحذ' + 'ف السريع حسب [[ويكيبيديا:حذف سريع|معايير الحذف السريع]]', + }, + 'he': { + '_default': + 'מחיקת מועמד למחיקה מהירה לפי [[ויקיפדיה:מדיניות ' + 'המחיקה|מדיניות המחיקה]]', + 'גם בוויקישיתוף': 'הקובץ זמין כעת בוויקישיתוף.', + }, + 'ja': { + '_default': '[[WP:CSD|即時削除の方針]]に基づい削除', + }, + 'pt': { + '_default': + 'Apagando página por ' + '[[Wikipedia:Páginas para eliminar|eliminação rápida]]', + }, + 'pl': { + '_default': + 'Usuwanie artykułu zgodnie z zasadami ' + '[[Wikipedia:Ekspresowe kasowanko|' + 'ekspresowego kasowania]]', + }, + 'it': { + '_default': + 'Rimuovo pagina che rientra nei casi di ' + '[[Wikipedia:IMMEDIATA|cancellazione immediata]].', + }, + 'zh': { + '_default': '[[WP:CSD]]', + 'advert': 'ad', + 'db-blanked': 'auth', + 'db-spam': '[[WP:CSD#G11|CSD G11]]: 廣告、宣傳頁面', + 'db-rediruser': '[[WP:CSD#O1|CSD O6]] 沒有在使用的討論頁', + 'notchinese': '[[WP:CSD#G7|CSD G7]]: 非中文條目且長時間未翻譯', + 'db-vandalism': 'vand', + '翻译': 'oprj', + '翻譯': 'oprj', + 'notmandarin': 'oprj', + 'no source': + '[[WP:CSD#I3|CSD I3]]: 沒有來源連結，無法確認來源與版權資訊', + 'no license': + '[[WP:CSD#I3|CSD I3]]: 沒有版權模板，無法確認版權資訊', + 'unknown': '[[WP:CSD#I3|CSD I3]]: 沒有版權模板，無法確認版權資訊', + 'temppage': '[[WP:CSD]]: 臨時頁面', + 'nowcommons': 'commons', + 'roughtranslation': 'mactra', + }, + }, + 'wikinews': { + 'en': { + '_default': '[[WN:CSD]]', + }, + 'zh': { + '_default': '[[WN:CSD]]', + }, + }, + } + + # Default reason for deleting a talk page. + talk_deletion_msg = { + 'wikipedia': { + 'ar': 'صفحة نقاش يتيمة', + 'cs': 'Osiřelá diskusní stránka', + 'de': 'Verwaiste Diskussionsseite', + 'en': 'Orphaned talk page', + 'fa': 'بحث یتیم', + 'fr': 'Page de discussion orpheline', + 'he': 'דף שיחה של ערך שנמחק', + 'it': 'Rimuovo pagina di discussione di una pagina già cancellata', + 'pl': 'Osierocona strona dyskusji', + 'pt': 'Página de discussão órfã', + 'zh': '[[WP:CSD#O1|CSD O1 O2 O6]] 沒有在使用的討論頁', + }, + 'wikinews': { + 'en': 'Orphaned talk page', + 'zh': '[[WN:CSD#O1|CSD O1 O2 O6]] 沒有在使用的討論頁', + } + } + + # A list of often-used reasons for deletion. Shortcuts are keys, and + # reasons are values. If the user enters a shortcut, the associated reason + # will be used. + delete_reasons = { + 'wikipedia': { + 'de': { + 'asdf': 'Tastaturtest', + 'egal': 'Eindeutig irrelevant', + 'ka': 'Kein Artikel', + 'mist': 'Unsinn', + 'move': 'Redirectlöschung, um Platz für Verschiebung zu ' + 'schaffen', + 'nde': 'Nicht in deutscher Sprache verfasst', + 'pfui': 'Beleidigung', + 'redir': 'Unnötiger Redirect', + 'spam': 'Spam', + 'web': 'Nur ein Weblink', + 'wg': 'Wiedergänger (wurde bereits zuvor gelöscht)', + }, + 'it': { + 'test': 'Si tratta di un test', + 'vandalismo': 'Caso di vandalismo', + 'copyviol': 'Violazione di copyright', + 'redirect': 'Redirect rotto o inutile', + 'spam': 'Spam', + 'promo': 'Pagina promozionale', + }, + 'ja': { + 'cont': '[[WP:CSD]] 全般1 意味不明な内容のページ', + 'test': '[[WP:CSD]] 全般2 テスト投稿', + 'vand': '[[WP:CSD]] 全般3 荒らしand/orいたずら', + 'ad': '[[WP:CSD]] 全般4 宣伝', + 'rep': '[[WP:CSD]] 全般5 削除されたページの改善なき再作成', + 'cp': '[[WP:CSD]] 全般6 コピペ移動or分割', + 'sh': '[[WP:CSD]] 記事1 短すぎ', + 'nd': '[[WP:CSD]] 記事1 定義なし', + 'auth': '[[WP:CSD]] 記事3 投稿者依頼or初版立項者による白紙化', + 'nr': '[[WP:CSD]] リダイレクト1 無意味なリダイレクト', + 'nc': '[[WP:CSD]] リダイレクト2 [[WP:NC]]違反', + 'ren': + '[[WP:CSD]] ' + 'リダイレクト3 改名提案を経た曖昧回避括弧付きの移動の残骸', + 'commons': '[[WP:CSD]] マルチメディア7 コモンズの画像ページ', + 'tmp': '[[WP:CSD]] テンプレート1 初版投稿者依頼', + 'uau': '[[WP:CSD]] 利用者ページ1 本人希望', + 'nuu': + '[[WP:CSD]] ' + '利用者ページ2 利用者登録されていない利用者ページ', + 'ipu': '[[WP:CSD]] 利用者ページ3 IPユーザの利用者ページ', + }, + 'zh': { + 'empty': '[[WP:CSD#G1]]: 沒有實際內容或歷史記錄的文章。', + 'test': '[[WP:CSD#G2]]: 測試頁', + 'vand': '[[WP:CSD#G3]]: 純粹破壞', + 'rep': '[[WP:CSD#G5]]: 經討論被刪除後又重新創建的內容', + 'repa': '[[WP:CSD#G5]]: 重複的文章', + 'oprj': '[[WP:CSD#G7]]: 內容來自其他中文計劃', + 'move': + '[[WP:CSD#G8]]: ' + '依[[Wikipedia:移動請求|移動請求]]' + '暫時刪除以進行移動或合併頁面之工作', + 'auth': '[[WP:CSD#G10]]: 原作者請求', + 'ad': '[[WP:CSD#G11]]: 明顯的以廣告宣傳為目而建立的頁面', + 'adc': '[[WP:CSD#G11]]: 只有條目名稱中的人物或團體之聯絡資訊', + 'bio': '[[WP:CSD#G12]]: 未列明來源及語調負面的生者傳記', + 'mactra': '[[WP:CSD#G13]]: 明顯的機器翻譯', + 'notrans': '[[WP:CSD#G14]]: 未翻譯的頁面', + 'isol': '[[WP:CSD#G15]]: 孤立頁面', + 'isol-f': '[[WP:CSD#G15]]: 孤立頁面-沒有對應檔案的檔案頁面', + 'isol-sub': '[[WP:CSD#G15]]: 孤立頁面-沒有對應母頁面的子頁面', + 'tempcp': '[[WP:CSD#G16]]: 臨時頁面依然侵權', + 'cont': '[[WP:CSD#A1]]: 非常短，而且沒有定義或內容。', + 'nocont': + '[[WP:CSD#A2]]: ' + '內容只包括外部連接、參見、圖書參考、類別標籤、模板標籤、' + '跨語言連接的條目', + 'nc': '[[WP:CSD#A3]]: 跨計劃內容', + 'cn': '[[WP:CSD#R2]]: 跨空間重定向', + 'wr': '[[WP:CSD#R3]]: 錯誤重定向', + 'slr': '[[WP:CSD#R5]]: 指向本身的重定向或循環的重定向', + 'repi': '[[WP:CSD#F1]]: 重複的檔案', + 'lssd': + '[[WP:CSD#F3]]: ' + '沒有版權或來源資訊，無法確認圖片是否符合方針要求', + 'nls': '[[WP:CSD#F3]]: 沒有版權模板，無法確認版權資訊', + 'svg': '[[WP:CSD#F5]]: 被高解析度與SVG檔案取代的圖片', + 'ui': '[[WP:CSD#F6]]: 圖片未使用且不自由', + 'commons': + '[[WP:CSD#F7]]: ' + '此圖片已存在於[[:commons:|維基共享資源]]', + 'urs': '[[WP:CSD#O1]]: 用戶請求刪除自己的用戶頁子頁面', + 'anou': '[[WP:CSD#O3]]: 匿名用戶的用戶討論頁，其中的內容不再有用', + 'uc': '[[WP:CSD#O4]]: 空類別', + 'tmp': '[[WP:CSD]]: 臨時頁面', + }, + }, + } + + def __init__(self, **kwargs): + """Initializer. + + @keyword site: the site to work on + @type site: pywikibot.APISite + """ + super().__init__(**kwargs) + csd_cat = i18n.translate(self.site, self.csd_cat_title) + if csd_cat is None: + self.csd_cat = self.site.page_from_repository(self.csd_cat_item) + if self.csd_cat is None: + raise pywikibot.Error( + 'No category for speedy deletion found for {}' + .format(self.site)) + else: + self.csd_cat = pywikibot.Category(self.site, csd_cat) + self.saved_progress = None + + def guess_reason_for_deletion(self, page): + """Find a default reason for speedy deletion.""" + # TODO: The following check loads the page 2 times. + # Find a better way to do it. + if page.isTalkPage() and (page.toggleTalkPage().isRedirectPage() + or not page.toggleTalkPage().exists()): + # This is probably a talk page that is orphaned because we + # just deleted the associated article. + reason = i18n.translate(self.site, self.talk_deletion_msg, + fallback=self.site.code != 'zh') + else: + # Try to guess reason by the template used + templates = page.templatesWithParams() + reasons = i18n.translate(self.site, self.deletion_messages) + + for template, params in templates: + template_name = template.title().lower() + if template_name in reasons: + if not isinstance(reasons[template_name], str): + # Make alias to delete_reasons + reason = i18n.translate( + self.site, + self.delete_reasons)[reasons[template_name]] + else: + reason = reasons[template_name] + break + else: + reason = None + if not reason: + # Unsuccessful in guessing the reason. Use a default message. + reason = reasons['_default'] + return reason + + def get_reason_for_deletion(self, page): + """Get a reason for speedy deletion from operator.""" + suggested_reason = self.guess_reason_for_deletion(page) + pywikibot.output(color_format( + 'The suggested reason is: {lightred}{}{default}', + suggested_reason)) + + # We don't use i18n.translate() here because for some languages the + # entry is intentionally left out. + if self.site.family.name in self.delete_reasons \ + and page.site.lang in self.delete_reasons[self.site.family.name]: + local_reasons = i18n.translate(page.site.lang, + self.delete_reasons) + pywikibot.output('') + for key in sorted(local_reasons.keys()): + pywikibot.output((key + ':').ljust(8) + local_reasons[key]) + pywikibot.output('') + reason = pywikibot.input(fill( + 'Please enter the reason for deletion, choose a default ' + 'reason, or press enter for the suggested message:')) + if reason.strip() in local_reasons: + reason = local_reasons[reason] + else: + reason = pywikibot.input( + 'Please enter the reason for deletion,\n' + 'or press enter for the suggested message:') + + return reason or suggested_reason + + def exit(self): + """Just call teardown after current run.""" + self.teardown() + + def run(self): + """Start the bot's action.""" + start_ts = pywikibot.Timestamp.now() + self.saved_progress = None + self.quitting = False + while True: + super().run() + if self.quitting: + break + + if not self.saved_progress: + pywikibot.output( + '\nThere are no (further) pages to delete.\n' + 'Waiting for 30 seconds or press Ctrl+C to quit...') + try: + time.sleep(30) + except KeyboardInterrupt: + break + + self._start_ts = start_ts + super().exit() + + def treat_page(self): + """Process one page.""" + page = self.current_page + + page_text = [] + for text in page.text.split('\n'): + page_text += wrap(text, width=79) or [''] + + pywikibot.output(color_format('{blue}{}{default}', '_' * 80)) + if len(page_text) > self.LINES: + pywikibot.output(color_format( + '{blue}The page detail is too many lines, ' + 'only output first {} lines:{default}', self.LINES)) + pywikibot.output( + '\n'.join(page_text[:min(self.LINES, len(page_text))])) + pywikibot.output(color_format('{blue}{}{default}', '_' * 80)) + + choice = pywikibot.input_choice( + 'Input action?', + [('delete', 'd'), ('skip', 's'), ('update', 'u'), ('quit', 'q')], + default='s', automatic_quit=False) + # quit the bot + if choice == 'q': + self.quitting = True + self.quit() + + # stop the generator and restart from current title + elif choice == 'u': + pywikibot.output('Updating from CSD category.') + self.saved_progress = page.title() + self.stop() + + # delete the current page + elif choice == 'd': + reason = self.get_reason_for_deletion(page) + pywikibot.output(color_format( + 'The chosen reason is: {lightred}{}{default}', + reason)) + page.delete(reason, prompt=False) + + # skip this page + else: + pywikibot.output('Skipping page {}'.format(page)) + + def setup(self): + """Refresh generator.""" + generator = pagegenerators.CategorizedPageGenerator( + self.csd_cat, start=self.saved_progress) + # wrap another generator around it so that we won't produce orphaned + # talk pages. + generator = pagegenerators.PageWithTalkPageGenerator(generator) + self.generator = pagegenerators.PreloadingGenerator(generator, + groupsize=20) + self.saved_progress = None + + +def main(*args): + """Script entry point.""" + pywikibot.handle_args(args) # No local args yet + site = pywikibot.Site() + if site.has_right('delete'): + bot = SpeedyBot(site=site) + bot.run() + elif site.logged_in(): + pywikibot.output("{} does not have 'delete' right for site {}" + .format(site.username(), site)) + else: + pywikibot.output('Login first.') + + +if __name__ == '__main__': + main() diff --git a/tests/script_tests.py b/tests/script_tests.py index 2802c12..37cb7c8 100644 --- a/tests/script_tests.py +++ b/tests/script_tests.py @@ -131,6 +131,7 @@ 'replicate_wiki': 'error: the following arguments are required: destination', 'shell': ('>>> ', 'Welcome to the'), + 'speedy_delete': "does not have 'delete' right for site", 'transferbot': 'Target site not different from source site', 'unusedfiles': ('Working on', None), 'version': 'Pywikibot: [', -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/148646 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I49733820fe77bfef95475d4e2e39c2cd9d974226 Gerrit-Change-Number: 148646 Gerrit-PatchSet: 30 Gerrit-Owner: Ricordisamoa <ricordisamoa(a)disroot.org> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm(a)member.fsf.org> Gerrit-Reviewer: Liuxinyu970226 <541329866(a)qq.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Ricordisamoa <ricordisamoa(a)disroot.org> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 8 months

[Gerrit] ...core[master]: [IMPR] Use ThreadList with weblinkchecker.py

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/511326 ) Change subject: [IMPR] Use ThreadList with weblinkchecker.py ...................................................................... [IMPR] Use ThreadList with weblinkchecker.py Also simplify ignore url checking Change-Id: I714bebfb1221ec73dc2893de5c6592257ec27e78 --- M scripts/weblinkchecker.py 1 file changed, 9 insertions(+), 17 deletions(-) Approvals: Xqt: Looks good to me, approved Matěj Suchánek: Looks good to me, but someone else must approve jenkins-bot: Verified diff --git a/scripts/weblinkchecker.py b/scripts/weblinkchecker.py index 8d4fd46..62f2413 100755 --- a/scripts/weblinkchecker.py +++ b/scripts/weblinkchecker.py @@ -133,7 +133,7 @@ from pywikibot.pagegenerators import ( XMLDumpPageGenerator as _XMLDumpPageGenerator, ) -from pywikibot.tools import deprecated +from pywikibot.tools import deprecated, ThreadList from pywikibot.tools.formatter import color_format try: @@ -857,32 +857,24 @@ self.HTTPignore = HTTPignore self.day = day + # Limit the number of threads started at the same time + self.threads = ThreadList(limit=config.max_external_links, + wait_time=config.retry_wait) + def treat_page(self): """Process one page.""" page = self.current_page - text = page.get() - for url in weblinksIn(text): + for url in weblinksIn(page.text): for ignoreR in ignorelist: if ignoreR.match(url): break - else: # not ignore url - # Limit the number of threads started at the same time. Each - # thread will check one page, then die. - while threading.activeCount() >= config.max_external_links: - pywikibot.sleep(config.retry_wait) + else: + # Each thread will check one page, then die. thread = LinkCheckThread(page, url, self.history, self.HTTPignore, self.day) # thread dies when program terminates thread.setDaemon(True) - try: - thread.start() - except threading.ThreadError: - pywikibot.warning( - "Can't start a new thread.\nPlease decrease " - 'max_external_links in your user-config.py or use\n' - "'-max_external_links:' option with a smaller value. " - 'Default is 50.') - raise + self.threads.append(thread) def RepeatPageGenerator(): -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/511326 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I714bebfb1221ec73dc2893de5c6592257ec27e78 Gerrit-Change-Number: 511326 Gerrit-PatchSet: 9 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com> Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com> Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz> Gerrit-Reviewer: Framawiki <framawiki(a)tools.wmflabs.org> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 8 months

[Gerrit] ...core[master]: [IMPR] Simplify understandBlock in blockpageschecker.py

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/628344 ) Change subject: [IMPR] Simplify understandBlock in blockpageschecker.py ...................................................................... [IMPR] Simplify understandBlock in blockpageschecker.py Change-Id: Ib11562bdaf0dc08238bf62572080d85b91943c6b --- M scripts/blockpageschecker.py 1 file changed, 7 insertions(+), 15 deletions(-) Approvals: Hazard-SJ: Looks good to me, but someone else must approve Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/blockpageschecker.py b/scripts/blockpageschecker.py index 85b5cbf..e76277b 100755 --- a/scripts/blockpageschecker.py +++ b/scripts/blockpageschecker.py @@ -45,8 +45,6 @@ # # Distributed under the terms of the MIT license. # -from __future__ import absolute_import, division, unicode_literals - import re import time import webbrowser @@ -165,21 +163,15 @@ def understandBlock(text, TTP, TSP, TSMP, TTMP, TU): """Understand if the page is blocked and if it has the right template.""" - if TTP: - for catchRegex in TTP: # TTP = templateTotalProtection + results = 'sysop-total', 'autoconfirmed-total', 'unique' + for index, template in enumerate((TTP, TSP, TU)): + if not template: + continue + for catchRegex in template: resultCatch = re.findall(catchRegex, text) if resultCatch: - return ('sysop-total', catchRegex) - if TSP: - for catchRegex in TSP: - resultCatch = re.findall(catchRegex, text) - if resultCatch: - return ('autoconfirmed-total', catchRegex) - if TU: - for catchRegex in TU: - resultCatch = re.findall(catchRegex, text) - if resultCatch: - return ('unique', catchRegex) + return results[index], catchRegex + if TSMP and TTMP and TTP != TTMP and TSP != TSMP: for catchRegex in TTMP: resultCatch = re.findall(catchRegex, text) -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/628344 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Ib11562bdaf0dc08238bf62572080d85b91943c6b Gerrit-Change-Number: 628344 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com> Gerrit-Reviewer: Hazard-SJ <hazardsjwiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 8 months

[Gerrit] ...core[master]: [4.0] remove Python 2 related code in weblinkchecker.py

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/616214 ) Change subject: [4.0] remove Python 2 related code in weblinkchecker.py ...................................................................... [4.0] remove Python 2 related code in weblinkchecker.py - also print a FutureWarning for the deprecated LinkChecker class which is not used inside a script - decrease nested flow statements in DeadLinkReportThread.run - speed up inner loop of treat_page Change-Id: I8c601223ef31974bf78becfec06cccb276dd3bd4 --- M scripts/weblinkchecker.py 1 file changed, 100 insertions(+), 126 deletions(-) Approvals: Hazard-SJ: Looks good to me, but someone else must approve Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/weblinkchecker.py b/scripts/weblinkchecker.py index 9b5e9de..8d4fd46 100755 --- a/scripts/weblinkchecker.py +++ b/scripts/weblinkchecker.py @@ -109,17 +109,32 @@ # # Distributed under the terms of the MIT license. # -from __future__ import absolute_import, division, unicode_literals - import codecs import datetime +import http.client as httpclient import pickle import re import socket import threading import time +from contextlib import suppress from functools import partial +from typing import Optional, Tuple +from urllib.parse import urlsplit +from urllib.request import quote + +import requests + +import pywikibot + +from pywikibot import comms, i18n, config, pagegenerators, textlib, config2 +from pywikibot.bot import ExistingPageBot, SingleSiteBot, suggest_help +from pywikibot.pagegenerators import ( + XMLDumpPageGenerator as _XMLDumpPageGenerator, +) +from pywikibot.tools import deprecated +from pywikibot.tools.formatter import color_format try: import memento_client @@ -127,29 +142,6 @@ except ImportError as e: memento_client = e -import pywikibot - -from pywikibot import ( - comms, i18n, config, pagegenerators, textlib, config2, -) - -from pywikibot.bot import ExistingPageBot, SingleSiteBot, suggest_help -from pywikibot.pagegenerators import ( - XMLDumpPageGenerator as _XMLDumpPageGenerator, -) -from pywikibot.tools import deprecated, PY2 -from pywikibot.tools.formatter import color_format - -import requests - -if not PY2: - import http.client as httplib - import urllib.parse as urlparse - import urllib.request as urllib -else: - import httplib - import urllib - import urlparse docuReplacements = {'&params;': pagegenerators.parameterHelp} # noqa: N816 @@ -288,8 +280,8 @@ """The link is not an URL.""" -@deprecated('requests', since='20160120') -class LinkChecker(object): +@deprecated('requests', since='20160120', future_warning=True) +class LinkChecker: """ Check links. @@ -342,16 +334,16 @@ def getConnection(self): """Get a connection.""" if self.scheme == 'http': - return httplib.HTTPConnection(self.host) + return httpclient.HTTPConnection(self.host) elif self.scheme == 'https': - return httplib.HTTPSConnection(self.host) + return httpclient.HTTPSConnection(self.host) else: raise NotAnURLError(self.url) def getEncodingUsedByServer(self): """Get encodung used by server.""" if not self.serverEncoding: - try: + with suppress(Exception): pywikibot.output( 'Contacting server %s to find out its default encoding...' % self.host) @@ -359,8 +351,7 @@ conn.request('HEAD', '/', None, self.header) self.response = conn.getresponse() self.readEncodingFromResponse(self.response) - except Exception: - pass + if not self.serverEncoding: # TODO: We might also load a page, then check for an encoding # definition in a HTML meta tag. @@ -373,20 +364,18 @@ def readEncodingFromResponse(self, response): """Read encoding from response.""" if not self.serverEncoding: - try: + with suppress(Exception): ct = response.getheader('Content-Type') charsetR = re.compile('charset=(.+)') charset = charsetR.search(ct).group(1) self.serverEncoding = charset - except Exception: - pass def changeUrl(self, url): """Change url.""" self.url = url # we ignore the fragment (self.scheme, self.host, self.path, self.query, - self.fragment) = urlparse.urlsplit(self.url) + self.fragment) = urlsplit(self.url) if not self.path: self.path = '/' if self.query: @@ -399,17 +388,15 @@ self.query.encode('ascii') except UnicodeEncodeError: encoding = self.getEncodingUsedByServer() - self.path = urllib.quote(self.path.encode(encoding)) - self.query = urllib.quote(self.query.encode(encoding), '=&') + self.path = quote(self.path.encode(encoding)) + self.query = quote(self.query.encode(encoding), '=&') - def resolveRedirect(self, useHEAD=False): + def resolveRedirect(self, useHEAD=False) -> Optional[str]: """ Return the redirect target URL as a string, if it is a HTTP redirect. If useHEAD is true, uses the HTTP HEAD method, which saves bandwidth by not downloading the body. Otherwise, the HTTP GET method is used. - - @rtype: str or None """ conn = self.getConnection() try: @@ -422,7 +409,7 @@ self.response = conn.getresponse() # read the server's encoding, in case we need it later self.readEncodingFromResponse(self.response) - except httplib.BadStatusLine: + except httpclient.BadStatusLine: # Some servers don't seem to handle HEAD requests properly, # e.g. http://www.radiorus.ru/ which is running on a very old # Apache server. Using GET instead works on these (but it uses @@ -467,18 +454,14 @@ else: return False # not a redirect - def check(self, useHEAD=False): - """ - Return True and the server status message if the page is alive. - - @rtype: tuple of (bool, unicode) - """ + def check(self, useHEAD=False) -> Tuple[bool, str]: + """Return True and the server status message if the page is alive.""" try: wasRedirected = self.resolveRedirect(useHEAD=useHEAD) except UnicodeError as error: return False, 'Encoding Error: {0} ({1})'.format( error.__class__.__name__, error) - except httplib.error as error: + except httpclient.error as error: return False, 'HTTP Error: {}'.format(error.__class__.__name__) except socket.error as error: # https://docs.python.org/3/library/socket.html : @@ -542,7 +525,7 @@ else: try: conn = self.getConnection() - except httplib.error as error: + except httpclient.error as error: return False, 'HTTP Error: {0}'.format( error.__class__.__name__) try: @@ -573,7 +556,7 @@ def __init__(self, page, url, history, HTTPignore, day): """Initializer.""" - threading.Thread.__init__(self) + super().__init__() self.page = page self.url = url self.history = history @@ -625,7 +608,7 @@ config.weblink_dead_days) -class History(object): +class History: """ Store previously found dead links. @@ -736,15 +719,11 @@ """ if url in self.historyDict: with self.semaphore: - try: + with suppress(KeyError): # Not sure why this can happen del self.historyDict[url] - except KeyError: - # Not sure why this can happen, but I guess we can - # ignore this. - pass return True - else: - return False + + return False def save(self): """Save the .dat file to disk.""" @@ -791,65 +770,64 @@ break else: time.sleep(0.1) - else: - with self.semaphore: - url, errorReport, containingPage, archiveURL = \ - self.queue[0] - self.queue = self.queue[1:] - talkPage = containingPage.toggleTalkPage() - pywikibot.output(color_format( - '{lightaqua}** Reporting dead link on ' - '{0}...{default}', - talkPage.title(as_link=True))) - try: - content = talkPage.get() + '\n\n\n' - if url in content: - pywikibot.output(color_format( - '{lightaqua}** Dead link seems to have ' - 'already been reported on {0}{default}', - talkPage.title(as_link=True))) - continue - except (pywikibot.NoPage, pywikibot.IsRedirectPage): - content = '' + continue - if archiveURL: - archiveMsg = '\n' + \ - i18n.twtranslate( - containingPage.site, - 'weblinkchecker-archive_msg', - {'URL': archiveURL}) - else: - archiveMsg = '' - # The caption will default to "Dead link". But if there - # is already such a caption, we'll use "Dead link 2", - # "Dead link 3", etc. - caption = i18n.twtranslate(containingPage.site, - 'weblinkchecker-caption') - i = 1 - count = '' - # Check if there is already such a caption on - # the talk page. - while re.search('= *{0}{1} *='.format(caption, count), - content) is not None: - i += 1 - count = ' ' + str(i) - caption += count - content += '== {0} ==\n\n{1}\n\n{2}{3}\n--~~~~'.format( - caption, i18n.twtranslate(containingPage.site, - 'weblinkchecker-report'), - errorReport, archiveMsg) - - comment = '[[{0}#{1}|→]] {2}'.format( - talkPage.title(), caption, - i18n.twtranslate(containingPage.site, - 'weblinkchecker-summary')) - try: - talkPage.put(content, comment) - except pywikibot.SpamblacklistError as error: + with self.semaphore: + url, errorReport, containingPage, archiveURL = self.queue[0] + self.queue = self.queue[1:] + talkPage = containingPage.toggleTalkPage() + pywikibot.output(color_format( + '{lightaqua}** Reporting dead link on {}...{default}', + talkPage)) + try: + content = talkPage.get() + '\n\n\n' + if url in content: pywikibot.output(color_format( - '{lightaqua}** SpamblacklistError while trying to ' - 'change {0}: {1}{default}', - talkPage.title(as_link=True), error.url)) + '{lightaqua}** Dead link seems to have ' + 'already been reported on {}{default}', + talkPage)) + continue + except (pywikibot.NoPage, pywikibot.IsRedirectPage): + content = '' + + if archiveURL: + archiveMsg = '\n' + \ + i18n.twtranslate( + containingPage.site, + 'weblinkchecker-archive_msg', + {'URL': archiveURL}) + else: + archiveMsg = '' + # The caption will default to "Dead link". But if there + # is already such a caption, we'll use "Dead link 2", + # "Dead link 3", etc. + caption = i18n.twtranslate(containingPage.site, + 'weblinkchecker-caption') + i = 1 + count = '' + # Check if there is already such a caption on + # the talk page. + while re.search('= *{0}{1} *=' + .format(caption, count), content) is not None: + i += 1 + count = ' ' + str(i) + caption += count + content += '== {0} ==\n\n{3}\n\n{1}{2}\n--~~~~'.format( + caption, errorReport, archiveMsg, + i18n.twtranslate(containingPage.site, + 'weblinkchecker-report')) + + comment = '[[{0}#{1}|→]] {2}'.format( + talkPage.title(), caption, + i18n.twtranslate(containingPage.site, + 'weblinkchecker-summary')) + try: + talkPage.put(content, comment) + except pywikibot.SpamblacklistError as error: + pywikibot.output(color_format( + '{lightaqua}** SpamblacklistError while trying to ' + 'change {0}: {1}{default}', + talkPage, error.url)) class WeblinkCheckerRobot(SingleSiteBot, ExistingPageBot): @@ -862,8 +840,7 @@ def __init__(self, generator, HTTPignore=None, day=7, site=True): """Initializer.""" - super(WeblinkCheckerRobot, self).__init__( - generator=generator, site=site) + super().__init__(generator=generator, site=site) if config.report_dead_links_on_talk: pywikibot.log('Starting talk page thread') @@ -885,11 +862,10 @@ page = self.current_page text = page.get() for url in weblinksIn(text): - ignoreUrl = False for ignoreR in ignorelist: if ignoreR.match(url): - ignoreUrl = True - if not ignoreUrl: + break + else: # not ignore url # Limit the number of threads started at the same time. Each # thread will check one page, then die. while threading.activeCount() >= config.max_external_links: @@ -921,12 +897,11 @@ yield page -def countLinkCheckThreads(): +def countLinkCheckThreads() -> int: """ Count LinkCheckThread threads. @return: number of LinkCheckThread threads - @rtype: int """ i = 0 for thread in threading.enumerate(): @@ -935,11 +910,10 @@ return i -@deprecated('requests', since='20160120') +@deprecated('requests', since='20160120', future_warning=True) def check(url): """DEPRECATED: Use requests instead. Perform a check on URL.""" - c = LinkChecker(url) - return c.check() + return LinkChecker(url).check() def main(*args): -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/616214 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I8c601223ef31974bf78becfec06cccb276dd3bd4 Gerrit-Change-Number: 616214 Gerrit-PatchSet: 7 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com> Gerrit-Reviewer: Hazard-SJ <hazardsjwiki(a)gmail.com> Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com> Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-CC: Zhuyifei1999 <zhuyifei1999(a)gmail.com> Gerrit-MessageType: merged

3 years, 8 months

[Gerrit] ...core[master]: [bugfix] Re-enable -sleep parameter in checkimages.py

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631961 ) Change subject: [bugfix] Re-enable -sleep parameter in checkimages.py ...................................................................... [bugfix] Re-enable -sleep parameter in checkimages.py - improvement for option handling; use arg.partition instead of static length operations - fix all parameter for site.namespace method - shorten skip/skip_number exception Bug: T264521 Change-Id: I17cb49b0191b93a3cf6899a8ec54f56166c66c80 --- M scripts/checkimages.py 1 file changed, 54 insertions(+), 91 deletions(-) Approvals: JJMC89: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/checkimages.py b/scripts/checkimages.py index 1d612fa..635e31d 100755 --- a/scripts/checkimages.py +++ b/scripts/checkimages.py @@ -1542,108 +1542,71 @@ site = pywikibot.Site() # Here below there are the local parameters. for arg in local_args: - if arg.startswith('-limit'): - if len(arg) == 6: - limit = int(pywikibot.input( - 'How many files do you want to check?')) - else: - limit = int(arg[7:]) - elif arg.startswith('-sleep'): - length = len(arg) - if length == len('-sleep'): - time_sleep = int(pywikibot.input( - 'How many seconds do you want runs to be apart?')) - else: - time_sleep = int(arg[length + 1:]) - elif arg == '-break': + option, _, value = arg.partition(':') + if option == '-limit': + limit = int(value or pywikibot.input( + 'How many files do you want to check?')) + elif option == '-sleep': + time_sleep = int(value or pywikibot.input( + 'How many seconds do you want runs to be apart?')) + elif option == '-break': repeat = False - elif arg == '-nologerror': + elif option == '-nologerror': logFullError = False - elif arg == '-commons': + elif option == '-commons': commonsActive = True - elif arg == '-duplicatesreport': + elif option == '-duplicatesreport': duplicatesReport = True - elif arg.startswith('-duplicates'): + elif option == '-duplicates': duplicatesActive = True - if len(arg) == 11: - duplicates_rollback = 1 - elif len(arg) > 11: - duplicates_rollback = int(arg[12:]) - elif arg.startswith('-maxusernotify'): - if len(arg) == 13: - max_user_notify = int(pywikibot.input( - 'What should be the maximum number of notifications per ' - 'user per check?')) - elif len(arg) > 13: - max_user_notify = int(arg[14:]) - elif arg == '-sendemail': + duplicates_rollback = int(value or 1) + elif option == '-maxusernotify': + max_user_notify = int(value or pywikibot.input( + 'What should be the maximum number of notifications per user ' + 'per check?')) + elif option == '-sendemail': sendemailActive = True - elif arg.startswith('-skip'): - if len(arg) == 5: - skip_number = int(pywikibot.input( - 'How many files do you want to skip?')) - elif len(arg) > 5: - skip_number = int(arg[6:]) - elif arg.startswith('-wait'): - if len(arg) == 5: - waitTime = int(pywikibot.input( - 'How many time do you want to wait before checking the ' - 'files?')) - elif len(arg) > 5: - waitTime = int(arg[6:]) - elif arg.startswith('-start'): - if len(arg) == 6: - firstPageTitle = pywikibot.input( - 'From which page do you want to start?') - elif len(arg) > 6: - firstPageTitle = arg[7:] - namespaces = tuple(ns + ':' - for ns in site.namespace(Namespace.FILE, all)) + elif option == '-skip': + skip_number = int(value or pywikibot.input( + 'How many files do you want to skip?')) + elif option == '-wait': + waitTime = int(value or pywikibot.input( + 'How many time do you want to wait before checking the ' + 'files?')) + elif option == '-start': + firstPageTitle = value or pywikibot.input( + 'From which page do you want to start?') + namespaces = tuple( + ns + ':' for ns in site.namespace(Namespace.FILE, True)) if firstPageTitle.startswith(namespaces): - firstPageTitle = firstPageTitle.split(':')[1] + firstPageTitle = firstPageTitle.split(':', 1)[1] generator = site.allimages(start=firstPageTitle) repeat = False - elif arg.startswith('-page'): - if len(arg) == 5: - regexPageName = str(pywikibot.input( - 'Which page do you want to use for the regex?')) - elif len(arg) > 5: - regexPageName = str(arg[6:]) + elif option == '-page': + regexPageName = value or pywikibot.input( + 'Which page do you want to use for the regex?') repeat = False regexGen = True - elif arg.startswith('-url'): - if len(arg) == 4: - regexPageUrl = str(pywikibot.input( - 'Which url do you want to use for the regex?')) - elif len(arg) > 4: - regexPageUrl = str(arg[5:]) + elif option == '-url': + regexPageUrl = value or pywikibot.input( + 'Which url do you want to use for the regex?') urlUsed = True repeat = False regexGen = True - elif arg.startswith('-regex'): - if len(arg) == 6: - regexpToUse = str(pywikibot.input( - 'Which regex do you want to use?')) - elif len(arg) > 6: - regexpToUse = str(arg[7:]) + elif option == '-regex': + regexpToUse = value or pywikibot.input( + 'Which regex do you want to use?') generator = 'regex' repeat = False - elif arg.startswith('-cat'): - if len(arg) == 4: - catName = str(pywikibot.input('In which category do I work?')) - elif len(arg) > 4: - catName = str(arg[5:]) - catSelected = pywikibot.Category(site, - 'Category:{}'.format(catName)) - generator = catSelected.articles(namespaces=[6]) + elif option == '-cat': + cat_name = value or pywikibot.input('In which category do I work?') + cat = pywikibot.Category(site, 'Category:' + cat_name) + generator = cat.articles(namespaces=[6]) repeat = False - elif arg.startswith('-ref'): - if len(arg) == 4: - refName = str(pywikibot.input( - 'The references of what page should I parse?')) - elif len(arg) > 4: - refName = str(arg[5:]) - ref = pywikibot.Page(site, refName) + elif option == '-ref': + ref_name = value or pywikibot.input( + 'The references of what page should I parse?') + ref = pywikibot.Page(site, ref_name) generator = ref.getReferences(namespaces=[6]) repeat = False else: @@ -1652,8 +1615,6 @@ if not generator: normal = True - skip = skip_number > 0 - # Ensure that the bot is localized and right command args are given if site.code not in project_inserted: additional_text = ('Your project is not supported by this script.\n' @@ -1703,18 +1664,20 @@ for image in generator: # Setting the image for the main class Bot.setParameters(image) - if skip: - skip = Bot.skipImages(skip_number, limit) - if skip: - continue + + if skip_number and Bot.skipImages(skip_number, limit): + continue + # Check on commons if there's already an image with the same name if commonsActive and site.family.name != 'commons': if not Bot.checkImageOnCommons(): continue + # Check if there are duplicates of the image on the project if duplicatesActive: if not Bot.checkImageDuplicated(duplicates_rollback): continue + Bot.checkStep() if repeat: -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631961 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I17cb49b0191b93a3cf6899a8ec54f56166c66c80 Gerrit-Change-Number: 631961 Gerrit-PatchSet: 5 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com> Gerrit-Reviewer: JJMC89 <JJMC89.Wikimedia(a)gmail.com> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 8 months

[Gerrit] ...core[master]: [doc] Update ROADMAP.rst

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/630579 ) Change subject: [doc] Update ROADMAP.rst ...................................................................... [doc] Update ROADMAP.rst Change-Id: I03b5bdf770ff10209f0eb29e787f2b61324c4c90 --- M ROADMAP.rst 1 file changed, 27 insertions(+), 6 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/ROADMAP.rst b/ROADMAP.rst index bd90185..b57609f 100644 --- a/ROADMAP.rst +++ b/ROADMAP.rst @@ -1,6 +1,24 @@ Current release changes ~~~~~~~~~~~~~~~~~~~~~~~ +* Prevent circular imports in config2.py and http.py (T264500) +* version.get_module_version() is deprecated and gives no meaningfull result +* Fix version.get_module_filename() and update log lines (T264235) +* Re-enable printing log header (T264235) +* Fix result of tools.intersect_generators() (T263947) +* Only show _GLOBAL_HELP options if explicitly wanted +* Deprecated Family.version() methods were removed +* Unused parameters of page methods like forceReload, insite, throttle, step was removed +* Raise RuntimeError instead of AttributeError for old wikis (T263951) +* Deprecated script options were removed +* lyricwiki_family was removed (T245439) +* RecentChangesPageGenerator parameters has been synced with APISite.recentchanges +* APISite.recentchanges accepts keyword parameters only +* LoginStatus enum class was moved from site to login.py +* WbRepresentation derives from abstract base class abc.ABC +* Update characters in the Cf category to Unicode version 12.1.0 +* Update __all__ variable in pywikibot (T122879) +* Use api.APIGenerator through site._generator (T129013) * Support of MediaWiki releases below 1.19 has been dropped (T245350) * Page.get_best_claim () retrieves preferred Claim of a property referring to the given page (T175207) * Check whether _putthead is current_thread() to join() (T263331) @@ -22,15 +40,18 @@ Future release notes ~~~~~~~~~~~~~~~~~~~~ -* 4.4.0: Property.getTspe() method will be removed -* 4.4.0: Request.http_params() method will be removed -* 4.4.0: DataSite.get_item() method will be removed -* 4.4.0: date.MakeParameter() function will be removed -* 4.4.0: pagegenerators.ReferringPageGenerator is desupported and will be removed +* 5.0.0: version.getfileversion() is desupported and will be removed +* 5.0.0: Methods deprecated for 5 years or longer will be removed +* 5.0.0: Outdated recentchanges parameter will be removed +* 5.0.0: site.LoginStatus will be removed in favour of login.LoginStatus +* 5.0.0: Property.getType() method will be removed +* 5.0.0: Request.http_params() method will be removed +* 5.0.0: DataSite.get_item() method will be removed +* 5.0.0: date.MakeParameter() function will be removed +* 5.0.0: pagegenerators.ReferringPageGenerator is desupported and will be removed * 4.3.0: Unsused UserBlocked exception will be removed * 4.3.0: Deprecated Page.contributingUsers() will be removed * 4.2.0: tools.StringTypes will be removed * 4.1.0: Deprecated editor.command will be removed * 4.1.0: tools.open_compressed, tools.UnicodeType and tools.signature will be removed * 4.1.0: comms.PywikibotCookieJar and comms.mode_check_decorator will be removed -* 4.0.0: Unused parameters of page methods like forceReload, insite, throttle, step will be removed -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/630579 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I03b5bdf770ff10209f0eb29e787f2b61324c4c90 Gerrit-Change-Number: 630579 Gerrit-PatchSet: 7 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 8 months

[Gerrit] ...core[master]: [doc] Update scripts/README.rst

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631971 ) Change subject: [doc] Update scripts/README.rst ...................................................................... [doc] Update scripts/README.rst Change-Id: I07279736c9367123943e12036b2876518554c733 --- M scripts/README.rst 1 file changed, 24 insertions(+), 11 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/scripts/README.rst b/scripts/README.rst index 7bf070c..5b009e5 100644 --- a/scripts/README.rst +++ b/scripts/README.rst @@ -61,6 +61,8 @@ | | articles), if other language wikipedia already has such | | | a template | +------------------------+---------------------------------------------------------+ + | coordinate_import.py | Coordinate importing script. | + +------------------------+---------------------------------------------------------+ | #copyright.py | This robot check copyright text in Google, Yahoo! and | | | Live Search. | +------------------------+---------------------------------------------------------+ @@ -69,8 +71,6 @@ +------------------------+---------------------------------------------------------+ | #copyright_put.py | Put reports of copyright.py on wiki pages. | +------------------------+---------------------------------------------------------+ - | coordinate_import.py | Coordinate importing script. | - +------------------------+---------------------------------------------------------+ | cosmetic_changes.py | Can do slight modifications to a wiki page source code | | | such that the code looks cleaner. | +------------------------+---------------------------------------------------------+ @@ -135,7 +135,7 @@ +------------------------+---------------------------------------------------------+ | listpages.py | listpages: report number of pages found | +------------------------+---------------------------------------------------------+ - | login.py | [IMPROV] Reduce maximum line length to 130 | + | login.py | Script to log the bot in to a wiki account. | +------------------------+---------------------------------------------------------+ | lonelypages.py | Place a template on pages which are not linked to by | | | other pages, and are therefore lonely | @@ -217,7 +217,7 @@ | #spellcheck.py | This bot spellchecks wiki pages. | +------------------------+---+-----------------------------------------------------+ | #standardize_notes.py | Converts external links and notes/references to | - | | : Footnote3 ref/note format. Rewrites References. | + | | Footnote3 ref/note format. Rewrites References. | +----------------------------+-----------------------------------------------------+ | #statistics_in_wikitable.py| This bot renders statistics provided by | | | [[Special:Statistics]] in a table on a wiki page. | @@ -257,7 +257,7 @@ +------------------------+---------------------------------------------------------+ - +----------------------------------------------------------------------------------+ + +------------------------+---------------------------------------------------------+ | archive | Scripts no longer maintained. | +========================+=========================================================+ | cfd.py | Processes the categories for discussion working page. | @@ -268,25 +268,38 @@ +------------------------+---------------------------------------------------------+ + +------------------------+---------------------------------------------------------+ + | maintenance | Framework helper scripts. | + +========================+=========================================================+ + | cache.py | Script for showing and deleting API cache. | + +------------------------+---------------------------------------------------------+ + | colors.py | Utility to show pywikibot colors. | + +------------------------+---------------------------------------------------------+ + | compat2core.py | Helper script to convert compat 1.0 scripts to the core | + | | 3.0 framework. Also works for newer Pywikibot releases. | + +------------------------+---------------------------------------------------------+ + | make_i18n_dict.py | Generate a i18n file from a given script. | + +------------------------+---------------------------------------------------------+ + | wikimedia_sites.py | Updates the language lists in Wikimedia family files. | + +------------------------+---------------------------------------------------------+ + + +----------------------------------------------------------------------------------+ | Others | +========================+=========================================================+ | i18n (folder) | Contains i18n translations for bot edit summaries. | +------------------------+---------------------------------------------------------+ - | maintenance (folder) | Contains maintenance scripts for the developing team. | - +------------------------+---------------------------------------------------------+ | userscripts (folder) | Empty folder for user scripts. | +------------------------+---------------------------------------------------------+ | README.rst | This file (Short info of all scripts). | +------------------------+---------------------------------------------------------+ -**External software can be used with Pywikibot:** - * PyGoogle to access Google Web API and PySearch to access Yahoo! Search - Web Services for use with copyright.py. +**External packages could be required with Pywikibot:** +The pwb.py wrapper scripts informs about the requirement and how to install. More precise information, and a list of the options that are available for the various programs, can be retrieved by running the bot with the -help -parameter, e.g. +parameter, e.g.:: python pwb.py interwiki -help -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631971 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I07279736c9367123943e12036b2876518554c733 Gerrit-Change-Number: 631971 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: D3r1ck01 <xsavitar.wiki(a)aol.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 8 months

[Gerrit] ...core[master]: [L10N] update languages_by_size

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631968 ) Change subject: [L10N] update languages_by_size ...................................................................... [L10N] update languages_by_size Change-Id: I659c42c4160615cc5bebf190d95683610fd0e195 --- M pywikibot/families/wikibooks_family.py M pywikibot/families/wikinews_family.py M pywikibot/families/wikipedia_family.py M pywikibot/families/wikiquote_family.py M pywikibot/families/wikisource_family.py M pywikibot/families/wikivoyage_family.py M pywikibot/families/wiktionary_family.py 7 files changed, 24 insertions(+), 24 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/families/wikibooks_family.py b/pywikibot/families/wikibooks_family.py index b972168..295a811 100644 --- a/pywikibot/families/wikibooks_family.py +++ b/pywikibot/families/wikibooks_family.py @@ -30,7 +30,7 @@ ] languages_by_size = [ - 'en', 'hu', 'de', 'fr', 'pt', 'it', 'ja', 'es', 'nl', 'pl', 'id', 'he', + 'en', 'hu', 'de', 'fr', 'it', 'pt', 'ja', 'es', 'nl', 'pl', 'id', 'he', 'fi', 'zh', 'sq', 'az', 'fa', 'ru', 'vi', 'ca', 'eu', 'da', 'ko', 'ba', 'sv', 'th', 'gl', 'sr', 'hi', 'cs', 'hr', 'no', 'tr', 'sa', 'ar', 'ta', 'uk', 'eo', 'sk', 'is', 'ro', 'bn', 'si', 'mk', 'ka', 'bg', 'ms', 'tt', diff --git a/pywikibot/families/wikinews_family.py b/pywikibot/families/wikinews_family.py index f5ee8f4..ff3524d 100644 --- a/pywikibot/families/wikinews_family.py +++ b/pywikibot/families/wikinews_family.py @@ -21,7 +21,7 @@ ] languages_by_size = [ - 'ru', 'sr', 'fr', 'en', 'pt', 'pl', 'de', 'zh', 'es', 'it', 'ar', 'cs', + 'ru', 'sr', 'fr', 'en', 'pt', 'pl', 'zh', 'de', 'es', 'it', 'ar', 'cs', 'ca', 'nl', 'el', 'ta', 'sv', 'uk', 'fa', 'fi', 'ro', 'li', 'tr', 'ja', 'eo', 'sq', 'no', 'ko', 'bg', 'bs', 'he', ] diff --git a/pywikibot/families/wikipedia_family.py b/pywikibot/families/wikipedia_family.py index 4fbfa03..8ddfb0c 100644 --- a/pywikibot/families/wikipedia_family.py +++ b/pywikibot/families/wikipedia_family.py @@ -33,28 +33,28 @@ 'no', 'ko', 'fi', 'hu', 'cs', 'sh', 'ro', 'zh-min-nan', 'tr', 'eu', 'ms', 'ce', 'eo', 'he', 'hy', 'bg', 'da', 'azb', 'sk', 'kk', 'min', 'hr', 'et', 'lt', 'be', 'el', 'az', 'simple', 'sl', 'gl', 'ur', 'nn', - 'hi', 'uz', 'ka', 'th', 'tt', 'la', 'cy', 'ta', 'vo', 'mk', 'ast', + 'hi', 'ka', 'tt', 'th', 'uz', 'la', 'cy', 'ta', 'vo', 'mk', 'ast', 'lv', 'zh-yue', 'tg', 'bn', 'af', 'mg', 'oc', 'bs', 'sq', 'ky', 'nds', 'new', 'be-tarask', 'ml', 'te', 'br', 'tl', 'vec', 'pms', 'mr', 'su', - 'ht', 'sw', 'lb', 'jv', 'sco', 'pnb', 'ba', 'ga', 'szl', 'is', 'my', - 'fy', 'cv', 'lmo', 'wuu', 'an', 'pa', 'ne', 'yo', 'ku', 'diq', 'bar', - 'io', 'gu', 'als', 'ckb', 'kn', 'scn', 'bpy', 'qu', 'ia', 'mn', - 'bat-smg', 'nv', 'si', 'or', 'cdo', 'ilo', 'gd', 'xmf', 'yi', 'am', - 'nap', 'wa', 'bug', 'sd', 'hsb', 'mai', 'map-bms', 'fo', 'mzn', 'li', + 'ht', 'sw', 'lb', 'jv', 'pnb', 'ba', 'ga', 'szl', 'is', 'my', 'sco', + 'fy', 'cv', 'lmo', 'wuu', 'an', 'pa', 'ne', 'diq', 'ku', 'yo', 'bar', + 'io', 'gu', 'ckb', 'als', 'kn', 'scn', 'bpy', 'qu', 'ia', 'mn', + 'bat-smg', 'si', 'nv', 'or', 'cdo', 'ilo', 'gd', 'xmf', 'yi', 'am', + 'nap', 'wa', 'bug', 'sd', 'hsb', 'mai', 'fo', 'map-bms', 'mzn', 'li', 'sah', 'eml', 'os', 'ps', 'sa', 'frr', 'bcl', 'zh-classical', 'ace', - 'mrj', 'mhr', 'hif', 'avk', 'hak', 'roa-tara', 'gor', 'pam', 'hyw', - 'km', 'nso', 'rue', 'crh', 'se', 'bh', 'shn', 'as', 'vls', 'mi', - 'nds-nl', 'nah', 'sc', 'vep', 'gan', 'myv', 'ab', 'glk', 'sn', 'bo', - 'co', 'tk', 'so', 'lrc', 'fiu-vro', 'csb', 'kv', 'ha', 'ie', 'gv', - 'udm', 'pcd', 'ay', 'kab', 'zea', 'sat', 'nrm', 'ug', 'lij', 'kw', - 'zu', 'lez', 'frp', 'stq', 'lfn', 'ban', 'gn', 'mwl', 'gom', 'rm', + 'mrj', 'mhr', 'hif', 'avk', 'hak', 'gor', 'roa-tara', 'pam', 'hyw', + 'km', 'nso', 'rue', 'crh', 'se', 'bh', 'as', 'shn', 'vls', 'mi', + 'nds-nl', 'nah', 'sc', 'vep', 'gan', 'myv', 'sn', 'ab', 'glk', 'bo', + 'co', 'lrc', 'so', 'tk', 'fiu-vro', 'csb', 'kv', 'ha', 'ie', 'gv', + 'udm', 'pcd', 'ay', 'kab', 'zea', 'sat', 'nrm', 'ug', 'lij', 'zu', + 'kw', 'lez', 'frp', 'ban', 'stq', 'lfn', 'gn', 'mwl', 'gom', 'rm', 'mt', 'lo', 'lad', 'koi', 'fur', 'olo', 'ang', 'dty', 'dsb', 'bjn', - 'ext', 'ln', 'cbk-zam', 'dv', 'ksh', 'tyv', 'gag', 'pfl', 'pag', 'ary', + 'ext', 'ln', 'cbk-zam', 'dv', 'ksh', 'tyv', 'ary', 'gag', 'pfl', 'pag', 'pi', 'av', 'awa', 'haw', 'bxr', 'xal', 'krc', 'pap', 'za', 'pdc', - 'kaa', 'rw', 'szy', 'arc', 'to', 'nov', 'jam', 'tpi', 'kbp', 'kbd', - 'ig', 'na', 'inh', 'tet', 'wo', 'tcy', 'ki', 'atj', 'jbo', 'roa-rup', + 'rw', 'kaa', 'szy', 'arc', 'to', 'nov', 'jam', 'tpi', 'kbp', 'kbd', + 'ig', 'inh', 'na', 'tet', 'wo', 'tcy', 'ki', 'atj', 'jbo', 'roa-rup', 'bi', 'lbe', 'kg', 'ty', 'lg', 'mdf', 'fj', 'srn', 'xh', 'gcr', 'ltg', - 'chr', 'lld', 'ak', 'sm', 'om', 'kl', 'got', 'pih', 'st', 'cu', 'ny', + 'lld', 'chr', 'ak', 'om', 'sm', 'kl', 'got', 'pih', 'st', 'cu', 'ny', 'nqo', 'tn', 'tw', 'ts', 'rmy', 'bm', 'mnw', 'chy', 'rn', 'tum', 'ss', 'ch', 'iu', 'pnt', 'ks', 'ady', 've', 'ee', 'ik', 'ff', 'sg', 'dz', 'ti', 'cr', 'din', diff --git a/pywikibot/families/wikiquote_family.py b/pywikibot/families/wikiquote_family.py index 722651f..bd6ae55 100644 --- a/pywikibot/families/wikiquote_family.py +++ b/pywikibot/families/wikiquote_family.py @@ -31,7 +31,7 @@ 'en', 'it', 'pl', 'ru', 'cs', 'fa', 'de', 'pt', 'es', 'fr', 'uk', 'he', 'sk', 'bs', 'tr', 'ca', 'fi', 'az', 'sl', 'lt', 'eo', 'zh', 'bg', 'ar', 'hr', 'hy', 'et', 'el', 'su', 'nn', 'id', 'sv', 'li', 'hu', 'ko', 'nl', - 'ja', 'la', 'ta', 'sah', 'sr', 'gu', 'gl', 'ur', 'th', 'te', 'be', + 'ja', 'la', 'ta', 'sah', 'sr', 'gu', 'gl', 'th', 'ur', 'te', 'be', 'cy', 'no', 'ml', 'sq', 'kn', 'ro', 'ku', 'eu', 'uz', 'hi', 'ka', 'da', 'sa', 'is', 'vi', 'br', 'mr', 'af', 'wo', 'ky', ] diff --git a/pywikibot/families/wikisource_family.py b/pywikibot/families/wikisource_family.py index 6381460..760c9db 100644 --- a/pywikibot/families/wikisource_family.py +++ b/pywikibot/families/wikisource_family.py @@ -30,7 +30,7 @@ 'mul', 'pt', 'gu', 'sr', 'fa', 'hu', 'sv', 'ml', 'ko', 'bn', 'sa', 'sl', 'hy', 'te', 'ta', 'el', 'ro', 'fi', 'ja', 'th', 'nap', 'vi', 'tr', 'az', 'br', 'la', 'ca', 'nl', 'kn', 'hr', 'eo', 'no', 'is', - 'vec', 'pms', 'et', 'be', 'da', 'id', 'mk', 'lij', 'hi', 'yi', 'mr', + 'vec', 'pms', 'et', 'be', 'id', 'da', 'mk', 'lij', 'hi', 'yi', 'mr', 'bg', 'as', 'li', 'lt', 'eu', 'or', 'pa', 'gl', 'bs', 'sah', 'sk', 'cy', 'zh-min-nan', 'fo', ] diff --git a/pywikibot/families/wikivoyage_family.py b/pywikibot/families/wikivoyage_family.py index 54ea424..d99caa9 100644 --- a/pywikibot/families/wikivoyage_family.py +++ b/pywikibot/families/wikivoyage_family.py @@ -17,7 +17,7 @@ languages_by_size = [ 'en', 'de', 'pl', 'it', 'fa', 'fr', 'ru', 'zh', 'nl', 'pt', 'es', 'he', - 'vi', 'fi', 'sv', 'el', 'ro', 'uk', 'bn', 'ps', 'hi', 'ja', + 'vi', 'fi', 'sv', 'el', 'ro', 'uk', 'bn', 'ps', 'ja', 'hi', ] category_redirect_templates = { diff --git a/pywikibot/families/wiktionary_family.py b/pywikibot/families/wiktionary_family.py index cf0f5c7..4cdf2ec 100644 --- a/pywikibot/families/wiktionary_family.py +++ b/pywikibot/families/wiktionary_family.py @@ -32,11 +32,11 @@ 'en', 'mg', 'fr', 'ru', 'de', 'sh', 'es', 'zh', 'ku', 'el', 'nl', 'sv', 'pl', 'lt', 'it', 'ca', 'fi', 'ta', 'hu', 'tr', 'hy', 'io', 'ko', 'kn', 'pt', 'ja', 'vi', 'sr', 'th', 'chr', 'hi', 'ro', 'no', 'et', 'id', - 'ml', 'my', 'cs', 'uz', 'li', 'or', 'te', 'eo', 'fa', 'ar', 'jv', 'gl', + 'ml', 'cs', 'my', 'uz', 'li', 'or', 'te', 'eo', 'fa', 'ar', 'gl', 'jv', 'az', 'oc', 'eu', 'uk', 'br', 'da', 'lo', 'hr', 'is', 'simple', 'ast', - 'fj', 'la', 'tg', 'ky', 'sk', 'bg', 'ur', 'wa', 'ps', 'cy', 'vo', 'af', + 'fj', 'la', 'tg', 'ky', 'sk', 'bg', 'wa', 'ur', 'ps', 'cy', 'vo', 'af', 'zh-min-nan', 'he', 'shn', 'scn', 'tl', 'pa', 'sw', 'fy', 'bn', 'nn', - 'lv', 'ka', 'sq', 'sl', 'co', 'mn', 'pnb', 'min', 'lb', 'nds', 'bs', + 'lv', 'sl', 'ka', 'sq', 'co', 'mn', 'pnb', 'min', 'lb', 'nds', 'bs', 'nah', 'sa', 'kk', 'yue', 'ms', 'km', 'vec', 'tk', 'mk', 'be', 'sm', 'hsb', 'shy', 'gd', 'ga', 'an', 'wo', 'gom', 'ang', 'ia', 'tt', 'mt', 'sd', 'gn', 'mr', 'fo', 'ie', 'so', 'csb', 'ug', 'st', 'roa-rup', 'si', -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631968 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: I659c42c4160615cc5bebf190d95683610fd0e195 Gerrit-Change-Number: 631968 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 8 months

[Gerrit] ...core[master]: [bugfix] Check for a valid logtype in TestLogentries

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631964 ) Change subject: [bugfix] Check for a valid logtype in TestLogentries ...................................................................... [bugfix] Check for a valid logtype in TestLogentries Bug: T264523 Change-Id: Ib4efde59e91058667c5b8eb4171ced3b609e0f4f --- M tests/logentries_tests.py 1 file changed, 4 insertions(+), 2 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/logentries_tests.py b/tests/logentries_tests.py index 942e5b0..a4d9a67 100644 --- a/tests/logentries_tests.py +++ b/tests/logentries_tests.py @@ -132,8 +132,10 @@ def test_method(logtype): def test_logevent(self, key): """Test a single logtype entry.""" - if key == 'old' and logtype == 'thanks': - self.skipTest('Thanks extension not on old.') + site = self.sites[key]['site'] + if logtype not in site.logtypes: + self.skipTest('{}: "{}" logtype not available on {}.' + .format(key, logtype, site)) self._test_logevent(logtype) return test_logevent -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631964 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: Ib4efde59e91058667c5b8eb4171ced3b609e0f4f Gerrit-Change-Number: 631964 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 8 months

[Gerrit] ...core[master]: [bugfix] ensure bot is logged in for api_tests.TestAPIMWException test

by jenkins-bot (Code Review)

jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631957 ) Change subject: [bugfix] ensure bot is logged in for api_tests.TestAPIMWException test ...................................................................... [bugfix] ensure bot is logged in for api_tests.TestAPIMWException test - tests fails if test account is not logged in. Force to be logged in with user attribute - use a separate test method for all TestAPIMWException tests Bug: T264096 Change-Id: If5f845382fc7793f37e5ca30e0120b81e9520825 --- M tests/api_tests.py 1 file changed, 13 insertions(+), 18 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/api_tests.py b/tests/api_tests.py index 02be034..98684d3 100644 --- a/tests/api_tests.py +++ b/tests/api_tests.py @@ -35,6 +35,8 @@ """Test raising an APIMWException.""" + user = True + data = {'error': {'code': 'internal_api_error_fake', 'info': 'Fake error message'}, 'servedby': 'unittest', @@ -88,17 +90,20 @@ self.error_patcher.stop() super(TestAPIMWException, self).tearDown() + def _test_assert_called_with(self, req): + self.assertRaises(api.APIMWException, req.submit) + pywikibot.warning.assert_called_with( + 'API error internal_api_error_fake: Fake error message') + pywikibot.error.assert_called_with( + 'Detected MediaWiki API exception internal_api_error_fake: ' + 'Fake error message\n[servedby: unittest]; raising') + def test_API_error(self): """Test a static request.""" req = api.Request(site=self.site, parameters={'action': 'query', 'fake': True}) with PatchedHttp(api, self.data): - self.assertRaises(api.APIMWException, req.submit) - pywikibot.warning.assert_called_with( - 'API error internal_api_error_fake: Fake error message') - pywikibot.error.assert_called_with( - 'You have no API read permissions. ' - 'Seems you are not logged in.') + self._test_assert_called_with(req) def test_API_error_encoding_ASCII(self): """Test a Page instance as parameter using ASCII chars.""" @@ -108,12 +113,7 @@ 'titles': page}) self.assert_parameters = {'fake': ''} with PatchedHttp(api, self._dummy_request): - self.assertRaises(api.APIMWException, req.submit) - pywikibot.warning.assert_called_with( - 'API error internal_api_error_fake: Fake error message') - pywikibot.error.assert_called_with( - 'Detected MediaWiki API exception internal_api_error_fake: ' - 'Fake error message\n[servedby: unittest]; raising') + self._test_assert_called_with(req) def test_API_error_encoding_Unicode(self): """Test a Page instance as parameter using non-ASCII chars.""" @@ -123,12 +123,7 @@ 'titles': page}) self.assert_parameters = {'fake': ''} with PatchedHttp(api, self._dummy_request): - self.assertRaises(api.APIMWException, req.submit) - pywikibot.warning.assert_called_with( - 'API error internal_api_error_fake: Fake error message') - pywikibot.error.assert_called_with( - 'Detected MediaWiki API exception internal_api_error_fake: ' - 'Fake error message\n[servedby: unittest]; raising') + self._test_assert_called_with(req) class TestApiFunctions(DefaultSiteTestCase): -- To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/631957 To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Change-Id: If5f845382fc7793f37e5ca30e0120b81e9520825 Gerrit-Change-Number: 631957 Gerrit-PatchSet: 1 Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot Gerrit-MessageType: merged

3 years, 8 months

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits October 2020