On Mar 28, 2023, at 2:01 PM, Roy Smith <roy@panix.com> wrote:

Hmmm.  What I'm doing requires Page.expand_text(), which looks like it does a Page.get() followed by a Site.expand_text(), and it's the later which actually takes most of the time.  That becomes an action=expandtemplates API call, which I don't see any way to batch.

<Screen Shot 2023-03-28 at 1.55.55 PM.png>

On Mar 28, 2023, at 1:04 PM, Kunal Mehta <legoktm@debian.org> wrote:

Hi,

On 3/27/23 15:57, Roy Smith wrote:

I need to issue a bunch of Page.get() requests in parallel.

Please don't. From <https://www.mediawiki.org/wiki/API:Etiquette#Request_limit>:

"Making your requests in series rather than in parallel, by waiting for one request to finish before sending a new request, should result in a safe request rate."

Instead of making parallel requests, you should make batched requests, which is how the preloading stuff Xqt mentioned works.

-- Kunal / Legoktm
_______________________________________________
pywikibot mailing list -- pywikibot@lists.wikimedia.org
Public archives at https://lists.wikimedia.org/hyperkitty/list/pywikibot@lists.wikimedia.org/message/RXANREWJYNQ3AI2MDU4K4WR67D5W4KCH/
To unsubscribe send an email to pywikibot-leave@lists.wikimedia.org