[Pywikibot-commits] [Gerrit] api.py: improve QueryGenerator efficiency - change (pywikibot/core)

jenkins-bot has submitted this change and it was merged. Change subject: api.py: improve QueryGenerator efficiency ...................................................................... api.py: improve QueryGenerator efficiency Limit management is currently done on the first module, while the other limited modules are left with default limits. Limit for such modules is set to max possible value, in order to reduce the number of requests and make queries faster. Change-Id: I1c9d96b7bfb121a1b58bd6361dee69691ec5703c --- M pywikibot/data/api.py 1 file changed, 15 insertions(+), 0 deletions(-) Approvals: John Vandenberg: Looks good to me, approved XZise: Looks good to me, but someone else must approve jenkins-bot: Verified diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py index 01d36d9..98217c3 100644 --- a/pywikibot/data/api.py +++ b/pywikibot/data/api.py @@ -1230,21 +1230,36 @@ limited_modules = ( set(self.modules) & self.site._paraminfo.query_modules_with_limits ) + if not limited_modules: self.limited_module = None elif len(limited_modules) == 1: self.limited_module = limited_modules.pop() else: # Select the first limited module in the request. + # Query will continue as needed until limit (if any) for this module + # is reached. for module in self.modules: if module in self.site._paraminfo.query_modules_with_limits: self.limited_module = module + limited_modules.remove(module) break pywikibot.log('%s: multiple requested query modules support limits' "; using the first such module '%s' of %r" % (self.__class__.__name__, self.limited_module, self.modules)) + # Set limits for all remaining limited modules to max value. + # Default values will only cause more requests and make the query + # slower. + for module in limited_modules: + param = self.site._paraminfo.parameter(module, 'limit') + prefix = self.site._paraminfo[module]['prefix'] + if self.site.logged_in() and self.site.has_right('apihighlimits'): + self.request[prefix + 'limit'] = int(param['highmax']) + else: + self.request[prefix + 'limit'] = int(param["max"]) + self.api_limit = None if self.limited_module: -- To view, visit https://gerrit.wikimedia.org/r/173630 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1c9d96b7bfb121a1b58bd6361dee69691ec5703c Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa &lt;mpaa.wiki(a)gmail.com&gt; Gerrit-Reviewer: John Vandenberg &lt;jayvdb(a)gmail.com&gt; Gerrit-Reviewer: Ladsgroup &lt;ladsgroup(a)gmail.com&gt; Gerrit-Reviewer: Merlijn van Deen &lt;valhallasw(a)arctus.nl&gt; Gerrit-Reviewer: Mpaa &lt;mpaa.wiki(a)gmail.com&gt; Gerrit-Reviewer: XZise &lt;CommodoreFabianus(a)gmx.de&gt; Gerrit-Reviewer: jenkins-bot <>