[Pywikibot-commits] [Gerrit] pywikibot/core[master]: Do not exceed API limit of pageids in BaseSite.preloadpages

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/348233 ) Change subject: Do not exceed API limit of pageids in BaseSite.preloadpages ...................................................................... Do not exceed API limit of pageids in BaseSite.preloadpages AllpagesGenerator (and possibly others) returns pages with their ids. When reused in PreloadingGenerator, this generates a warning and truncates the series of pages to the limit. Bug: T78333 Bug: T161783 Change-Id: I240b0f998d5ef9c6438338da9572bdde265a3699 --- M pywikibot/site.py M scripts/category_redirect.py 2 files changed, 10 insertions(+), 4 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved Objections: Mpaa: There's a problem with this change, please improve diff --git a/pywikibot/site.py b/pywikibot/site.py index a16bdea..e5259c0 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -3257,7 +3257,14 @@ next_prio = 0 rvgen = api.PropertyGenerator(props, site=self) rvgen.set_maximum_items(-1) # suppress use of "rvlimit" parameter - if len(pageids) == len(sublist): + + parameter = self._paraminfo.parameter('query+info', 'prop') + if self.logged_in() and self.has_right('apihighlimits'): + max_ids = int(parameter['highlimit']) + else: + max_ids = int(parameter['limit']) # T78333, T161783 + + if len(pageids) == len(sublist) and len(set(pageids)) <= max_ids: # only use pageids if all pages have them rvgen.request['pageids'] = set(pageids) else: diff --git a/scripts/category_redirect.py b/scripts/category_redirect.py index 65aae8d..d616d4f 100755 --- a/scripts/category_redirect.py +++ b/scripts/category_redirect.py @@ -223,9 +223,8 @@ comment = i18n.twtranslate(self.site, self.redir_comment) # generator yields all hard redirect pages in namespace 14 - for page in pagegenerators.PreloadingGenerator( - self.site.allpages(namespace=14, filterredir=True), - groupsize=250): + for page in self.site.allpages(namespace=14, filterredir=True, + content=True): if page.isCategoryRedirect(): # this is already a soft-redirect, so skip it (for now) continue -- To view, visit https://gerrit.wikimedia.org/r/348233 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I240b0f998d5ef9c6438338da9572bdde265a3699 Gerrit-PatchSet: 5 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Matěj Suchánek &lt;matejsuchanek97(a)gmail.com&gt; Gerrit-Reviewer: Dalba &lt;dalba.wiki(a)gmail.com&gt; Gerrit-Reviewer: Magul &lt;tomasz.magulski(a)gmail.com&gt; Gerrit-Reviewer: Matěj Suchánek &lt;matejsuchanek97(a)gmail.com&gt; Gerrit-Reviewer: Mpaa &lt;mpaa.wiki(a)gmail.com&gt; Gerrit-Reviewer: Xqt &lt;info(a)gno.de&gt; Gerrit-Reviewer: jenkins-bot <>