Pywikibot-commits

pywikibot-commits@lists.wikimedia.org

1 participants
13314 discussions

[Gerrit] Make sure all variables passed to request are str - change (pywikibot/compat)

by jenkins-bot (Code Review)

jenkins-bot has submitted this change and it was merged. Change subject: Make sure all variables passed to request are str ...................................................................... Make sure all variables passed to request are str All parameters passed to request should be bytestrings - str. If any of them is accidentally unicode, this will try to convert all bytestrings to unicode via str.decode(sys.defaultencoding). This is problematic, because non-ascii data will then throw an UnicodeDecodeError as shown in [1]. [1] http://lists.wikimedia.org/pipermail/pywikipedia-l/2013-August/008218.html Change-Id: I1b66a31c8752dee1f950d425d532479f1b671926 --- M wikipedia.py 1 file changed, 4 insertions(+), 4 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/wikipedia.py b/wikipedia.py index 976a310..4e3a543 100644 --- a/wikipedia.py +++ b/wikipedia.py @@ -6488,12 +6488,12 @@ address = address[:-1] headers = { - 'User-agent': useragent, + 'User-agent': str(useragent), 'Content-Length': str(len(data)), - 'Content-type':contentType, + 'Content-type': str(contentType), } if cookies: - headers['Cookie'] = cookies + headers['Cookie'] = str(cookies) if compress: headers['Accept-encoding'] = 'gzip' @@ -6507,7 +6507,7 @@ retry_attempt = 0 while True: try: - request = urllib2.Request(url, data, headers) + request = urllib2.Request(str(url), str(data), headers) f = MyURLopener.open(request) # read & info can raise socket.error -- To view, visit https://gerrit.wikimedia.org/r/80228 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1b66a31c8752dee1f950d425d532479f1b671926 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/compat Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

10 years, 9 months

[Gerrit] hostname and paths should be str in generated family files - change (pywikibot/compat)

by jenkins-bot (Code Review)

jenkins-bot has submitted this change and it was merged. Change subject: hostname and paths should be str in generated family files ...................................................................... hostname and paths should be str in generated family files All parameters passed to request should be bytestrings - str. If any of them is accidentally unicode, this will try to convert all bytestrings to unicode via str.decode(sys.defaultencoding). This is problematic, because non-ascii data will then throw an UnicodeDecodeError as shown in [1]. The url will be converted to str on the postData level, but it is also good to prevent it from being unicode in the first place. See also: I1b66a31c8752dee1f950d425d532479f1b671926 [1] http://lists.wikimedia.org/pipermail/pywikipedia-l/2013-August/008218.html Change-Id: I5eb13d8853b6ad08b48f40b5ad881616a2f4fd2a --- M generate_family_file.py 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/generate_family_file.py b/generate_family_file.py index 06464c9..aef4d11 100644 --- a/generate_family_file.py +++ b/generate_family_file.py @@ -171,7 +171,7 @@ """.lstrip() % {'url': self.base_url, 'name': self.name}) for w in self.wikis.itervalues(): - f.write(" '%(lang)s': u'%(hostname)s',\n" % {'lang': w.lang, 'hostname': urlparse(w.server).netloc}) + f.write(" '%(lang)s': '%(hostname)s',\n" % {'lang': w.lang, 'hostname': urlparse(w.server).netloc}) f.write(" }\n\n") @@ -182,7 +182,7 @@ f.write(" return {\n") for w in self.wikis.itervalues(): - f.write(" '%(lang)s': u'%(path)s',\n" % {'lang': w.lang, 'path': w.scriptpath}) + f.write(" '%(lang)s': '%(path)s',\n" % {'lang': w.lang, 'path': w.scriptpath}) f.write(" }[code]\n") f.write("\n") -- To view, visit https://gerrit.wikimedia.org/r/80229 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I5eb13d8853b6ad08b48f40b5ad881616a2f4fd2a Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/compat Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

10 years, 9 months

[Gerrit] Start with a fresh list in html2unicode every time - change (pywikibot/compat)

by jenkins-bot (Code Review)

jenkins-bot has submitted this change and it was merged. Change subject: Start with a fresh list in html2unicode every time ...................................................................... Start with a fresh list in html2unicode every time def x(..., something=[]): something.extend([1,2,3]) means something becomes [1,2,3] on the first call, but [1,2,3,1,2,3] on the *second* call. This meant html2unicode got a longer list of replacements every time it is called. This commit changes it to the standard def x(..., something=None): if something is None: something = [] which means it's always an empty list, instead of what's left from the last call. Change-Id: Ie490b575a8a0cc4b5d45bbb97c0606e0fd64d4f9 --- M wikipedia.py 1 file changed, 5 insertions(+), 2 deletions(-) Approvals: Ladsgroup: Looks good to me, approved Malafaya: Checked; Looks good to me, but someone else must approve jenkins-bot: Verified diff --git a/wikipedia.py b/wikipedia.py index 976a310..f304932 100644 --- a/wikipedia.py +++ b/wikipedia.py @@ -5657,13 +5657,16 @@ # Utility functions for parsing page titles -def html2unicode(text, ignore = []): +def html2unicode(text, ignore = None): """Return text, replacing HTML entities by equivalent unicode characters.""" + + if ignore is None: + ignore = [] # This regular expression will match any decimal and hexadecimal entity and # also entities that might be named entities. entityR = re.compile( r'&(?:amp;)?(#(?P<decimal>\d+)|#x(?P<hex>[0-9a-fA-F]+)|(?P<name>[A-Za-z]+));') - + ignore.extend((38, # Ampersand (&) 39, # Bugzilla 24093 60, # Less than (<) -- To view, visit https://gerrit.wikimedia.org/r/79811 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ie490b575a8a0cc4b5d45bbb97c0606e0fd64d4f9 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/compat Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: DrTrigon <dr.trigon(a)surfeu.ch> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Malafaya <malafaya(a)clix.pt> Gerrit-Reviewer: Multichill <maarten(a)mdammers.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

10 years, 9 months

[Gerrit] Make generate_user_files.py work for other directories - change (pywikibot/core)

by Xqt (Code Review)

Xqt has submitted this change and it was merged. Change subject: Make generate_user_files.py work for other directories ...................................................................... Make generate_user_files.py work for other directories Use os.path.abspath(__file__) instead of os.path.abspath(sys.argv[0]) Change-Id: I3172c29352f9e7bddc8de173e6057fd9172e167a --- M generate_user_files.py 1 file changed, 1 insertion(+), 4 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/generate_user_files.py b/generate_user_files.py index b8a0064..8f4f7dd 100644 --- a/generate_user_files.py +++ b/generate_user_files.py @@ -176,11 +176,8 @@ if choice in "SE": break - # - # I don't like this solution. Temporary for me. - # # determine what directory this script (generate_user_files.py) lives in - install = os.path.dirname(os.path.abspath(sys.argv[0])) + install = os.path.dirname(os.path.abspath(__file__)) # config2.py will be in the pywikibot/ directory f = codecs.open(os.path.join(install, "pywikibot", "config2.py"), "r", "utf-8") -- To view, visit https://gerrit.wikimedia.org/r/79577 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3172c29352f9e7bddc8de173e6057fd9172e167a Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

10 years, 9 months

[Gerrit] Change title whitelist to title blacklist - change (pywikibot/core)

by Xqt (Code Review)

Xqt has submitted this change and it was merged. Change subject: Change title whitelist to title blacklist ...................................................................... Change title whitelist to title blacklist Titles with characters outside the BMP [1] (>\uFFFF) are now no longer detected as illegal. See this thread: [2] [1] https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane [2] http://thread.gmane.org/gmane.comp.python.pywikipediabot.general/13197/ This list of characters was generated by using the old re and by enumerating characters: import re m = re.compile(u'''[^ %!\"$&'()*,\\-.\\/0-9:;=?@A-Z\\\\^_`a-z~\u0080-\uFFFF+]''') for x in range(0,0x80): if m.match(unichr(x)): print "%x" % x, 0 1 2 3 4 5 6 7 8 9 a b c d e f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 23 3c 3e 5b 5d 7b 7c 7d 7f Change-Id: I02c26be9ad814ce11d9adf2f997d3d1e05764fd1 --- M pywikibot/page.py 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/page.py b/pywikibot/page.py index e51977c..58debb7 100644 --- a/pywikibot/page.py +++ b/pywikibot/page.py @@ -2853,8 +2853,8 @@ """ illegal_titles_pattern = re.compile( - # Matching titles will be held as illegal. - u'''[^ %!\"$&'()*,\\-.\\/0-9:;=?@A-Z\\\\^_`a-z~\u0080-\uFFFF+]''' + # Matching titles will be held as illegal. + ur'''[\x00-\x1f\x23\x3c\x3e\x5b\x5d\x7b\x7c\x7d\x7f]''' # URL percent encoding sequences interfere with the ability # to round-trip titles -- you can't link to them consistently. u'|%[0-9A-Fa-f]{2}' -- To view, visit https://gerrit.wikimedia.org/r/78525 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I02c26be9ad814ce11d9adf2f997d3d1e05764fd1 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

10 years, 9 months

[Gerrit] pep8 fixes - change (pywikibot/core)

by Xqt (Code Review)

Xqt has submitted this change and it was merged. Change subject: pep8 fixes ...................................................................... pep8 fixes Change-Id: Ibfd9a513ee265f9fec8dccfe24e967c051fed7e4 --- M pywikibot/comms/threadedhttp.py 1 file changed, 25 insertions(+), 25 deletions(-) Approvals: Xqt: Looks good to me, approved diff --git a/pywikibot/comms/threadedhttp.py b/pywikibot/comms/threadedhttp.py index f998b53..0b6afbc 100644 --- a/pywikibot/comms/threadedhttp.py +++ b/pywikibot/comms/threadedhttp.py @@ -48,6 +48,7 @@ u"Error: You need the python module httplib2 to use this module") sys.exit(1) + class ConnectionPool(object): """A thread-safe connection pool.""" @@ -67,7 +68,7 @@ self.lock.acquire() try: pywikibot.debug(u"Closing connection pool (%s connections)" - % len(self.connections), + % len(self.connections), _logger) for key in self.connections: for connection in self.connections[key]: @@ -92,7 +93,7 @@ if identifier in self.connections: if len(self.connections[identifier]) > 0: pywikibot.debug(u"Retrieved connection from '%s' pool." - % identifier, + % identifier, _logger) return self.connections[identifier].pop() return None @@ -113,7 +114,7 @@ if len(self.connections[identifier]) == self.maxnum: pywikibot.debug(u"closing %s connection %r" - % (identifier, connection), + % (identifier, connection), _logger) connection.close() del connection @@ -197,9 +198,8 @@ headers['connection'] = headers.pop('connection', 'keep-alive') # determine connection pool key and fetch connection - (scheme, authority, request_uri, defrag_uri) = httplib2.urlnorm( - httplib2.iri2uri(uri)) - conn_key = scheme+":"+authority + (scheme, authority, request_uri, defrag_uri) = httplib2.urlnorm(httplib2.iri2uri(uri)) + conn_key = scheme + ":" + authority connection = self.connection_pool.pop_connection(conn_key) if connection is not None: @@ -209,15 +209,16 @@ follow_redirects = self.follow_redirects self.follow_redirects = False pywikibot.debug(u"%r" % ( - (uri.replace("%7C","|"), method, body, - headers, max_redirects, - connection_type),), - _logger) + (uri.replace("%7C", "|"), method, body, + headers, max_redirects, + connection_type), + ), _logger) try: (response, content) = httplib2.Http.request( - self, uri, method, body, headers, - max_redirects, connection_type) - except Exception, e: # what types? + self, uri, method, body, headers, + max_redirects, connection_type + ) + except Exception, e: # what types? # return exception instance to be retrieved by the calling thread return e self.follow_redirects = follow_redirects @@ -237,19 +238,17 @@ # Check for possible redirects redirectable_response = ((response.status == 303) or (response.status in [300, 301, 302, 307] and - method in ["GET", "HEAD"])) - if self.follow_redirects and (max_redirects > 0) \ - and redirectable_response: + method in ["GET", "HEAD"])) + if self.follow_redirects and (max_redirects > 0) and redirectable_response: (response, content) = self._follow_redirect( uri, method, body, headers, response, content, max_redirects) - return (response, content) + return response, content def _follow_redirect(self, uri, method, body, headers, response, content, max_redirects): """Internal function to follow a redirect recieved by L{request}""" - (scheme, authority, absolute_uri, defrag_uri) = httplib2.urlnorm( - httplib2.iri2uri(uri)) + (scheme, authority, absolute_uri, defrag_uri) = httplib2.urlnorm(httplib2.iri2uri(uri)) if self.cache: cachekey = defrag_uri else: @@ -264,12 +263,11 @@ # Fix-up relative redirects (which violate an RFC 2616 MUST) if "location" in response: location = response['location'] - (scheme, authority, path, query, fragment) = httplib2.parse_uri( - location) - if authority == None: + (scheme, authority, path, query, fragment) = httplib2.parse_uri(location) + if authority is None: response['location'] = httplib2.urlparse.urljoin(uri, location) pywikibot.debug(u"Relative redirect: changed [%s] to [%s]" - % (location, response['location']), + % (location, response['location']), _logger) if response.status == 301 and method in ["GET", "HEAD"]: response['-x-permanent-redirect-url'] = response['location'] @@ -336,7 +334,7 @@ # The Queue item is expected to either an HttpRequest object # or None (to shut down the thread) pywikibot.debug(u"Thread started, waiting for requests.", _logger) - while (True): + while True: item = self.queue.get() if item is None: pywikibot.debug(u"Shutting down thread.", _logger) @@ -420,6 +418,7 @@ # request is the result of a redirect return False + class DummyResponse(object): """Simulated urllib2.Request object for httplib2 implements only what's necessary for cookielib.CookieJar to work @@ -429,6 +428,7 @@ def info(self): return DummyMessage(self.response) + class DummyMessage(object): """Simulated mimetools.Message object for httplib2 @@ -448,5 +448,5 @@ # using ','. but the netscape cookie format uses ',' # as part of the expires= date format. so we have # to split carefully here - header.split(',') won't do it. - HEADERVAL= re.compile(r'\s*(([^,]|(,\s*\d))+)') + HEADERVAL = re.compile(r'\s*(([^,]|(,\s*\d))+)') return [h[0] for h in HEADERVAL.findall(self.response[k])] -- To view, visit https://gerrit.wikimedia.org/r/80169 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ibfd9a513ee265f9fec8dccfe24e967c051fed7e4 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

10 years, 9 months

[Gerrit] pep8 fixes - change (pywikibot/core)

by jenkins-bot (Code Review)

10 years, 9 months

[Gerrit] pep8 fixes - change (pywikibot/core)

by jenkins-bot (Code Review)

10 years, 9 months

[Gerrit] pep8 fixes - change (pywikibot/core)

by jenkins-bot (Code Review)

jenkins-bot has submitted this change and it was merged. Change subject: pep8 fixes ...................................................................... pep8 fixes Change-Id: I20a9ee539e8bd8c1c8eb02fbd902388ede8c65f5 --- M pywikibot/data/api.py 1 file changed, 58 insertions(+), 65 deletions(-) Approvals: Xqt: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py index 4849f7a..237d433 100644 --- a/pywikibot/data/api.py +++ b/pywikibot/data/api.py @@ -38,6 +38,7 @@ lagpattern = re.compile(r"Waiting for [\d.]+: (?P<lag>\d+) seconds? lagged") + class APIError(pywikibot.Error): """The wiki site returned an error message.""" def __init__(self, code, info, **kwargs): @@ -131,14 +132,14 @@ raise ValueError("'action' specification missing from Request.") self.update(**kwargs) self.write = self.params["action"] in ( - "edit", "move", "rollback", "delete", "undelete", - "protect", "block", "unblock", "watch", "patrol", - "import", "userrights", "upload", "wbeditentity", - "wbsetlabel", "wbsetdescription", "wbsetaliases", - "wblinktitles", "wbsetsitelink", "wbcreateclaim", - "wbremoveclaims", "wbsetclaimvalue", "wbsetreference", - "wbremovereferences" - ) + "edit", "move", "rollback", "delete", "undelete", + "protect", "block", "unblock", "watch", "patrol", + "import", "userrights", "upload", "wbeditentity", + "wbsetlabel", "wbsetdescription", "wbsetaliases", + "wblinktitles", "wbsetsitelink", "wbcreateclaim", + "wbremoveclaims", "wbsetclaimvalue", "wbsetreference", + "wbremovereferences" + ) if self.params["action"] == "edit": pywikibot.debug(u"Adding user assertion", _logger) self.params["assert"] = "user" # make sure user is logged in @@ -201,19 +202,17 @@ try: self.params[key] = "|".join(self.params[key]) if isinstance(self.params[key], unicode): - self.params[key] = self.params[key].encode( - self.site.encoding()) + self.params[key] = self.params[key].encode(self.site.encoding()) except Exception: pywikibot.error( -u"http_params: Key '%s' could not be encoded to '%s'; params=%r" - % (key, self.site.encoding(), self.params[key])) + u"http_params: Key '%s' could not be encoded to '%s'; params=%r" + % (key, self.site.encoding(), self.params[key])) return urllib.urlencode(self.params) def __str__(self): return urllib.unquote(self.site.scriptpath() + "/api.php?" - + self.http_params() - ) + + self.http_params()) def _simulate(self, action): if action and config.simulate and action in config.actions_to_block: @@ -252,7 +251,7 @@ if key == "file": local_filename = self.params[key] filetype = mimetypes.guess_type(local_filename)[0] \ - or 'application/octet-stream' + or 'application/octet-stream' file_content = file(local_filename, "rb").read() submsg = MIMENonMultipart(*filetype.split("/")) submsg.add_header("Content-disposition", @@ -272,18 +271,17 @@ container.attach(submsg) # strip the headers to get the HTTP message body body = container.as_string() - marker = "\n\n" # separates headers from body + marker = "\n\n" # separates headers from body eoh = body.find(marker) - body = body[ eoh + len(marker): ] + body = body[eoh + len(marker):] # retrieve the headers from the MIME object mimehead = dict(container.items()) rawdata = http.request(self.site, uri, ssl, method="POST", headers=mimehead, body=body) else: rawdata = http.request(self.site, uri, ssl, method="POST", - headers={'Content-Type': - 'application/x-www-form-urlencoded'}, - body=paramstring) + headers={'Content-Type': 'application/x-www-form-urlencoded'}, + body=paramstring) ## import traceback ## traceback.print_stack() ## print rawdata @@ -309,8 +307,8 @@ # if the result isn't valid JSON, there must be a server # problem. Wait a few seconds and try again pywikibot.warning( -"Non-JSON response received from server %s; the server may be down." - % self.site) + "Non-JSON response received from server %s; the server may be down." + % self.site) pywikibot.debug(rawdata, _logger) # there might also be an overflow, so try a smaller limit for param in self.params: @@ -329,7 +327,7 @@ if type(result) is not dict: raise APIError("Unknown", "Unable to process query response of type %s." - % type(result), + % type(result), {'data': result}) if self['action'] == 'query': if 'userinfo' in result.get('query', ()): @@ -338,11 +336,10 @@ else: self.site._userinfo = result['query']['userinfo'] status = self.site._loginstatus # save previous login status - if ( ("error" in result - and result["error"]["code"].endswith("limit")) - or (status >= 0 - and self.site._userinfo['name'] - != self.site._username[status])): + if (("error" in result + and result["error"]["code"].endswith("limit")) + or (status >= 0 + and self.site._userinfo['name'] != self.site._username[status])): # user is no longer logged in (session expired?) # reset userinfo, then make user log in again del self.site._userinfo @@ -387,9 +384,9 @@ # raise error try: pywikibot.log(u"API Error: query=\n%s" - % pprint.pformat(self.params)) + % pprint.pformat(self.params)) pywikibot.log(u" response=\n%s" - % result) + % result) raise APIError(code, info, **result["error"]) except TypeError: raise RuntimeError(result) @@ -508,7 +505,7 @@ kwargs["indexpageids"] = "" # always ask for list of pageids self.request = Request(**kwargs) self.prefix = None - self.update_limit() # sets self.prefix + self.update_limit() # sets self.prefix if self.api_limit is not None and "generator" in kwargs: self.prefix = "g" + self.prefix self.limit = None @@ -595,16 +592,15 @@ for mod in self.module.split('|'): for param in self._modules[mod].get("parameters", []): if param["name"] == "limit": - if (self.site.logged_in() - and self.site.has_right('apihighlimits')): + if self.site.logged_in() and self.site.has_right('apihighlimits'): self.api_limit = int(param["highmax"]) else: self.api_limit = int(param["max"]) if self.prefix is None: self.prefix = self._modules[mod]["prefix"] pywikibot.debug(u"%s: Set query_limit to %i." - % (self.__class__.__name__, - self.api_limit), + % (self.__class__.__name__, + self.api_limit), _logger) return @@ -621,7 +617,7 @@ for mod in self.module.split('|'): for param in self._modules[mod].get("parameters", []): if param["name"] == "namespace": - self.request[self.prefix+"namespace"] = namespaces + self.request[self.prefix + "namespace"] = namespaces return def __iter__(self): @@ -647,7 +643,7 @@ # so use 250 as a safer limit new_limit = min(new_limit, self.api_limit // 10, 250) if new_limit is not None: - self.request[self.prefix+"limit"] = str(new_limit) + self.request[self.prefix + "limit"] = str(new_limit) if not hasattr(self, "data"): try: self.data = self.request.submit() @@ -662,13 +658,13 @@ if not self.data or not isinstance(self.data, dict): pywikibot.debug( u"%s: stopped iteration because no dict retrieved from api." - % self.__class__.__name__, + % self.__class__.__name__, _logger) return if "query" not in self.data: pywikibot.debug( -u"%s: stopped iteration because 'query' not found in api response." - % (self.__class__.__name__, self.resultkey), + u"%s: stopped iteration because 'query' not found in api response." + % (self.__class__.__name__, self.resultkey), _logger) pywikibot.debug(unicode(self.data), _logger) return @@ -676,9 +672,9 @@ resultdata = self.data["query"][self.resultkey] if isinstance(resultdata, dict): pywikibot.debug(u"%s received %s; limit=%s" - % (self.__class__.__name__, - resultdata.keys(), - self.limit), + % (self.__class__.__name__, + resultdata.keys(), + self.limit), _logger) if "results" in resultdata: resultdata = resultdata["results"] @@ -692,14 +688,14 @@ for k in sorted(resultdata.keys())] else: pywikibot.debug(u"%s received %s; limit=%s" - % (self.__class__.__name__, - resultdata, - self.limit), + % (self.__class__.__name__, + resultdata, + self.limit), _logger) if "normalized" in self.data["query"]: self.normalized = dict((item['to'], item['from']) - for item in - self.data["query"]["normalized"]) + for item in + self.data["query"]["normalized"]) else: self.normalized = {} for item in resultdata: @@ -712,7 +708,7 @@ if not self.continuekey in self.data["query-continue"]: pywikibot.log( u"Missing '%s' key in ['query-continue'] value." - % self.continuekey) + % self.continuekey) return update = self.data["query-continue"][self.continuekey] for key, value in update.iteritems(): @@ -720,7 +716,7 @@ if isinstance(value, int): value = str(value) self.request[key] = value - del self.data # a new request with query-continue is needed + del self.data # a new request with query-continue is needed def result(self, data): """Process result data as needed for particular subclass.""" @@ -768,7 +764,7 @@ else: kwargs['iiprop'] = 'timestamp|user|comment|url|size|sha1|metadata' QueryGenerator.__init__(self, generator=generator, **kwargs) - self.resultkey = "pages" # element to look for in result + self.resultkey = "pages" # element to look for in result def result(self, pagedata): """Convert page dict entry from api to Page object. @@ -882,13 +878,12 @@ if datetime.datetime.now() < self._waituntil: diff = self._waituntil - datetime.datetime.now() pywikibot.warning(u"Too many tries, waiting %s seconds before retrying." - % diff.seconds) + % diff.seconds) time.sleep(diff.seconds) login_request = Request(site=self.site, action="login", lgname=self.username, - lgpassword=self.password - ) + lgpassword=self.password) self.site._loginstatus = -2 while True: login_result = login_request.submit() @@ -900,7 +895,7 @@ for key in ('Token', 'UserID', 'UserName'): cookies.append("%s%s=%s" % (prefix, key, - login_result['login']['lg'+key.lower()])) + login_result['login']['lg' + key.lower()])) self.username = login_result['login']['lgusername'] return "\n".join(cookies) elif login_result['login']['result'] == "NeedToken": @@ -908,10 +903,8 @@ login_request["lgtoken"] = token continue elif login_result['login']['result'] == "Throttled": - self._waituntil = datetime.datetime.now() \ - + datetime.timedelta(seconds=int( - login_result["login"]["wait"]) - ) + self._waituntil = datetime.datetime.now() + datetime.timedelta( + seconds=int(login_result["login"]["wait"])) break else: break @@ -937,8 +930,7 @@ page._pageid = 0 # Non-existent page else: raise AssertionError( - "Page %s has neither 'pageid' nor 'missing' attribute" - % pagedict['title']) + "Page %s has neither 'pageid' nor 'missing' attribute" % pagedict['title']) page._isredir = 'redirect' in pagedict if 'touched' in pagedict: page._timestamp = pagedict['touched'] @@ -953,10 +945,10 @@ timestamp=pywikibot.Timestamp.fromISOformat(rev['timestamp']), user=rev.get('user', u''), anon='anon' in rev, - comment=rev.get('comment', u''), + comment=rev.get('comment', u''), minor='minor' in rev, text=rev.get('*', None) - ) + ) page._revisions[revision.revid] = revision if 'lastrevid' in pagedict: @@ -968,8 +960,8 @@ page._catinfo = pagedict["categoryinfo"] if "templates" in pagedict: - page._templates = [ pywikibot.Page(page.site, tl['title']) - for tl in pagedict['templates'] ] + page._templates = [pywikibot.Page(page.site, tl['title']) + for tl in pagedict['templates']] if "langlinks" in pagedict: links = [] @@ -999,6 +991,7 @@ logging.getLogger("pywiki.data.api").setLevel(logging.DEBUG) mysite = Site("en", "wikipedia") pywikibot.output(u"starting test....") + def _test(): import doctest doctest.testmod() -- To view, visit https://gerrit.wikimedia.org/r/80172 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I20a9ee539e8bd8c1c8eb02fbd902388ede8c65f5 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot

10 years, 9 months

[Gerrit] Revert "Finish implementation of must_be decorator" - change (pywikibot/core)

by jenkins-bot (Code Review)

jenkins-bot has submitted this change and it was merged. Change subject: Revert "Finish implementation of must_be decorator" ...................................................................... Revert "Finish implementation of must_be decorator" Russblau reported that something broke with this and prevented his bot from editing. Reverting until we can debug and figure out what went wrong. This reverts commit 650ea9d2c319d5a58b708c87ad9048ad84491240. Change-Id: I80dd97a75b909f9d2db2485f3681af444ebcd9b9 --- M pywikibot/site.py 1 file changed, 18 insertions(+), 38 deletions(-) Approvals: Legoktm: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/site.py b/pywikibot/site.py index 943e82b..bca3feb 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -637,43 +637,28 @@ def globalusers_address(self, target='', limit=500, offset='', group=''): raise NotImplementedError - def must_be(group=None,right=None): + """ Decorator to require a certain user status. For now, only the values + group = 'user' and group = 'sysop' are supported. The right property + will be ignored for now. + + @param group: the group the logged in user should belong to + legal values: 'user' and 'sysop' + @param right: the rights the logged in user hsould have + not supported yet and thus ignored. + @returns: a decorator to make sure the requirement is statisfied when + the decorated function is called. """ - Decorator to require a certain user status. - You can use the group and right independently or together. - Example: - @must_be(group='user', right='edit) - def edit_page(...): - - @param group: any arbitrary group the user should belong to - @param right: the rights the logged in user should have. - @return: a decorator to make sure the requirement is statisfied when - the decorated function is called. - """ - - if group: - if group == 'user': - grp = lambda self: self.login(False) - elif group == 'sysop': - grp = lambda self: self.login(True) - else: - grp = lambda self: self.has_group(group) + if group == 'user': + run = lambda self: self.login(False) + elif group == 'sysop': + run = lambda self: self.login(True) else: - grp = lambda self: True # No group provided - - if right: - rht = lambda self: self.has_right(group) - else: - rht = lambda self: True # No right provided - - run = lambda self: grp(self) and rht(self) + raise Exception("Not implemented") def decorator(fn): def callee(self, *args, **kwargs): - ok = run(self) - if not ok: - raise NoUsername('') # FIXME: Pick a better error + run(self) return fn(self, *args, **kwargs) callee.__name__ = fn.__name__ callee.__doc__ = fn.__doc__ @@ -848,11 +833,6 @@ self._loginstatus = LoginStatus.NOT_LOGGED_IN # failure if not hasattr(self, "_siteinfo"): self._getsiteinfo() - - if sysop: - return self._loginstatus == LoginStatus.AS_SYSOP - else: - return self._loginstatus == LoginStatus.AS_USER forceLogin = login # alias for backward-compatibility @@ -2563,7 +2543,7 @@ "editconflict": "Page %(title)s not saved due to edit conflict.", } - @must_be(group='user', right='edit') + @must_be(group='user') def editpage(self, page, summary, minor=True, notminor=False, bot=True, recreate=True, createonly=False, watch=None): """Submit an edited Page object to be saved to the wiki. @@ -2921,7 +2901,7 @@ #TODO: implement patrol - @must_be(right='block') + @must_be(group='sysop') def blockuser(self, user, expiry, reason, anononly=True, nocreate=True, autoblock=True, noemail=False, reblock=False): -- To view, visit https://gerrit.wikimedia.org/r/80173 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I80dd97a75b909f9d2db2485f3681af444ebcd9b9 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com> Gerrit-Reviewer: jenkins-bot

10 years, 9 months

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits