-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello all
As I was creating new repos, I did a look at [1]:
pywikibot
pywikibot/compat
pywikibot/core
pywikibot/externals
pywikibot/externals/httplib2
pywikibot/i18n
pywikibot/sf-export
pywikibot/spelling
[1] https://gerrit.wikimedia.org/r/#/admin/projects/?filter=pywikibot
So in my opinion WE HAVE TO BE VERY CAREFUL NOT TO MESS UP THE CLEAN
STRUCTURE here! At the moment everything is still ok - but we might
have to write down some rules like e.g. external code belong to
pywikibot/externals/...
all other code is maintained by us. (By the way how is this supposed
to work? *wonder*)
I also requested a personal repo for me and my bot called
pywikibot/bots/drtrigonbot
I would strongly encourage or even force to use this or a similar repo
structure in order to keep everything nice and clean. We just have to
agree on something BEFORE we create a mess. The structure given in [2]
is e.g. out-dated now and needs and update.
[2] https://www.mediawiki.org/wiki/Git/Conversion/pywikipedia
Greetings
DrTrigon
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlIczWgACgkQAXWvBxzBrDDw3gCfbyZY2hiWa8692AWpLmChkrXU
1poAn1x9SHMEyJDtA014EV0eqqtqb5yz
=ZAmR
-----END PGP SIGNATURE-----
Hello folks
We had some discussions in the IRC channel about BS. There are so many
problems about using BS in different systems. So many people don't
want to run patch.exe and so many people are using servers for running
bots and don't have permission to do that.
My suggestion is using the old system (an internal folder as an
external like i18n but I don't think we need to make another external)
but if you think we can do something else I'll be glad to discuss
Please keep in mind compatibility is one of the most important issues of PWB
Best
--
Amir
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello all!
Hello amir!
I have created and maintained some other additional SVN repositiories
on TS in the past [1].
[1] https://fisheye.toolserver.org/browse/drtrigon/externals
Since it is not possible to include SVN repos into GIT as submodule
(or else?) I fear I have to convert them (by svn2git?) to GIT repos in
order to re-integrate them again into pywikibot/compat. Would it be
possible to create those repos on gerrit in pywikibot too? E.g.:
pywikibot/haartraining
pywikibot/opencv
pywikibot/pycolorname
pywikibot/archive(s) (for .tar.gz files - other suggestions welcome)
in order to copy them
or pywikibot/other or pywikibot/thirdparty
as general repos to create several such additional codes (also for
future). Though later is not optimal since it can be included just as
1 submodule only.
Thanks a lot for your thought and help
Greetins
DrTrigon
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlH4420ACgkQAXWvBxzBrDC5KwCghIA4WXfFogvMW9DZA69bfP47
Yg8An2YpPRTR+ANrqzApHw2cbPRMHtmU
=BYOr
-----END PGP SIGNATURE-----
Hi,
I am Chinmay Naik, a Gsoc intern and operator of ProteinBoxBot(runs on
pywikipedia - formerly rewrite branch). When i run the bot, there exists a
sleep time of around 10 s (7s average) between two successive writes.
The bot will handle around 40,000 wikidata items corresponding to gene
wiki articles from http://en.wikipedia.org/wiki/Category:Human_proteins .
Also , i will be uploading around 30 (claims + sources) for each wikidata
item.
If u take a look at the recent edits (
https://www.wikidata.org/w/index.php?title=Special:Contributions/ProteinBox…),
it takes around 5 mins to completely write all claims+sources to each
wikidata item. This is a very large time lag and will hugely affect
deployment. Is there any way to reduce this sleep time? Any pointers on
this would be helpful.
Thanks,
Chinmay
Hi, I was pointed here from #pywikipediabot on Freenode IRC. I am using
Pywikipediabot (trunk) revision 11781 under Ubuntu 12.04. I'm having no
success using the upload.py script. It keeps failing with the following
error message...
----------
augur@ubuntu:/home/augur/pywikipedia# python upload.py -family:tropical
-keep -filename:Pwbicon.png -noverify Pwbicon.png "The Pywikipediabot
icon"
No handlers could be found for logger "pywiki"
Reading file Pwbicon.png
The suggested description is:
The Pywikipediabot icon
Uploading file to tropical:en via API....
ERROR: UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in
position 654: ordinal not in range(128)
WARNING: Could not open 'http://www.tropicalwikis.com/w/api.php'. Maybe
the server or
your connection is down. Retrying in 1 minutes...
----------
I can perform other actions on www.tropicalwikis.com with my bot, such as
using login.py, replace.py etc. I have also tried using a different
-family: and logging in to www.thetestwiki.org (where I also have a valid
bot account) and tried uploading the image file there, with the same
result.
I'm not a python expert, in fact I'm barely a python novice. I came across
this information which may be relevant though...
http://wiki.python.org/moin/UnicodeDecodeError
Thanks.
AugurNZ
ps. sorry if this is a repeat of a previous message. I wasn't registered
on the list when I first sent this, but I am now.
The file I'm trying to upload is the *.png thumbnail of the Pywikipediabot
logo from this page... http://www.mediawiki.org/wiki/File:Pwb_icon.svg
The thumbnail is called
http://upload.wikimedia.org/wikipedia/commons/thumb/d/d2/Pwb_icon.svg/128px…
but I have shortened the filename, as per the example below, to
"Pwbicon.png" (without the quotes). So the filename should not contain any
non-ASCII chars. As for the file itself, I don't know, and wouldn't even
know how to find out. ;)
> For me it seems that you have an HTTP error on uploading and the error
> message contains a non-ASCII character. This is an old problem, nobody
> could solve it by this time.
> Does your uploaded file contain none-ASCII chars?
>
>
>
> 2013/8/21 AugurNZ <augurnz(a)orcon.net.nz>
>
>> Hi, I was pointed here from #pywikipediabot on Freenode IRC. I am using
>> Pywikipediabot (trunk) revision 11781 under Ubuntu 12.04. I'm having no
>> success using the upload.py script. It keeps failing with the following
>> error message...
>>
>> ----------
>> augur@ubuntu:/home/augur/pywikipedia# python upload.py -family:tropical
>> -keep -filename:Pwbicon.png -noverify Pwbicon.png "The Pywikipediabot
>> icon"
>> No handlers could be found for logger "pywiki"
>> Reading file Pwbicon.png
>> The suggested description is:
>> The Pywikipediabot icon
>> Uploading file to tropical:en via API....
>> ERROR: UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in
>> position 654: ordinal not in range(128)
>> WARNING: Could not open 'http://www.tropicalwikis.com/w/api.php'. Maybe
>> the server or
>> your connection is down. Retrying in 1 minutes...
>> ----------
>>
>> I can perform other actions on www.tropicalwikis.com with my bot, such
>> as
>> using login.py, replace.py etc. I have also tried using a different
>> -family: and logging in to www.thetestwiki.org (where I also have a
>> valid
>> bot account) and tried uploading the image file there, with the same
>> result.
>>
>> I'm not a python expert, in fact I'm barely a python novice. I came
>> across
>> this information which may be relevant though...
>> http://wiki.python.org/moin/UnicodeDecodeError
>>
>> Thanks.
>> AugurNZ
>>
>> ps. sorry if this is a repeat of a previous message. I wasn't registered
>> on the list when I first sent this, but I am now.
>>
>>
>> _______________________________________________
>> Pywikipedia-l mailing list
>> Pywikipedia-l(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
>>
>
>
>
> --
> Bináris
>
While first comparing piwikipedia and wikitools, I have been astonished to
see how much light and simple are scripts to run a bot and "do basic
things", ti: reading and writing pages, access to API, and so on.
I'll try - consider that I'm far from skilled or "professional" - to prune
off from pywikipedia any script but the basic ones, covered by wikitools,
just to have some personal fun; but I encourage developers to do something
similar, writing a real "pywikipedia core" containing nothing but really
needed modules to run basics with personal scripts.
I presume that the result will exclude 95% or more of present pwykipedia
content.
Alex brollo
Op 20-8-2013 3:01, Chi Hong Lee schreef:
> I hope that someone may help to teach me how to create a bot
You could start by not sending the email to -owner address, but to the
actual list and to read the manual at
https://www.mediawiki.org/wiki/Pywikipediabot :-)
Maarten
There is a remaining bug. Try the following e.g. from idle:
>>> import Wikipedia
you are asked to install patch.exe. If you decline this request you are asked a second time. If declined a second time the script raises a fatal error while importing beautifullsoup. Beautifullsoup is a central part if the screen scraping compat framework and we should not prohibit using the framework for a cosmetic patch. Please give us back that part as e.g. externals or a library script. Anyway it is not a good idea to ask for installation (twice) and fall into a fatal error if you answer is NO. There should be alternatives to patching the working copy (e.g. nightly dump etc.)
Greetings
xqt
----- Ursprüngliche Nachricht -----
Von: Dr. Trigon
Gesendet: 03.07.2013 20:14
An: pywikipedia-l(a)lists.wikimedia.org
Betreff: Re: [Pywikipedia-l] Question about externals and patches
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello to all (again)!
Quit some time has elapsed since my first mail. Today I tested
successfully 'patch.exe' [1] in win and in linux wine. This is another
one than mentioned before, but/and as I can see works quite well even
with LF (unix-style) line endings.
[1] http://gnuwin32.sourceforge.net/packages/patch.htm
Since I will commit these changes now the time has come to test it on
other machines also. After that was successful too I will remove
BeautifulSoup.py from externals because it will be downloaded and
patched automatically.
In order to test this on your windows machine you can e.g.:
1.) create a directory, e.g. '[...]/Desktop/patch-test'
2.) download and extract 'patch.exe' from
http://downloads.sourceforge.net/project/gnuwin32/patch/2.5.9-7/patch-2.5.9…
to this directory
3.) download and extract 'BeautifulSoup.py' from
https://pypi.python.org/packages/source/B/BeautifulSoup/BeautifulSoup-3.2.0…
to this directory
4.) download the patch 'patch-BeautifulSoup' from
http://svn.wikimedia.org/viewvc/pywikipedia/trunk/pywikipedia/externals/pat…
to this directory
5.) open 'cmd', cd to '[...]/Desktop/patch-test' and enter "patch.exe
- -p0 -d [...]/Desktop/patch-test/ -i
[...]/Desktop/patch-test/patch-BeautifulSoup --binary"
6.) check the output for error messages and compare the resulting
'BeautifulSoup.py' to the one in your pywikipedia/externals directory,
they should be exactly the same! IF NOT AND/OR ERRORS OCCURED PLEASE
REPORT THIS TO ME!
Thanks a lot to everybody helping and Greetings
DrTrigon
On 25.05.2013 11:40, Dr. Trigon wrote:
> Hello all!
>
> Following issue: I am up re-organizing the whole "externals" part
> in trunk as you might have recognized already. In fact this is done
> now with the single exception of a generic patching system, e.g.
> needed for BeautifulSoup.py. (As usual) this is no problem under
> linux, but becomes a major issue under win.
>
> The mechanism I want to use is the well known diff-patch duo.
> Therefore a "patch" executable/binary (OR python script) is needed
> (for every OS). While this is kind of "built-in" in linux, win
> needs extra-attention. This is what I found so far:
>
> * http://sourceforge.net/projects/unxutils/ The executables do only
> depend on the Microsoft C-runtime (msvcrt.dll) and not on an
> emulation layer like that provided by Cygwin tools - windows ONLY
> not multi OS
>
> * http://python-patch.googlecode.com/svn/trunk/patch.py Python
> script therefore multi OS - but does not support the full
> diff-patch "command set" e.g. cannot create new files
>
> * https://code.google.com/p/google-diff-match-patch/ Not a
> command-line tool like "patch" but a python library/module. Multi
> OS.
>
> So I am stuck here and need some further knowledge, experience and
> personal preferences from your side in order to make a good
> decicion.
>
> In my opinion we should also keep further os (than just linux,
> mac, win) in our mind, becuase they are very close to what we
> already have too.
>
> Thanks for any help and Greetings DrTrigon
>
> _______________________________________________ Pywikipedia-l
> mailing list Pywikipedia-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlHUacQACgkQAXWvBxzBrDCwOACgst3qYcS1YTqZvUvtmy8+2TOD
jVkAnjBYVcMCNLSY5zBBFVG7LGg9pTDD
=fDqI
-----END PGP SIGNATURE-----
_______________________________________________
Pywikipedia-l mailing list
Pywikipedia-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
As you probably know We had a workshop
http://wikimania2013.wikimedia.org/wiki/DevCamp/Schedule on Wikimania's
DevCamp about using PWB for people who are interested.
It went very well thanks Legoktm (a BIG THANK YOU) but It was a very big
problem that installation of PWB is not user-friendly (and It's worse when
people try to install rewrite instead of trunk) The main cause as far as we
faced during the workshop (on different computers) is dependencies.
Maarten (multichill) has suggested we change setup.py and
generate_user_config.py and make a auto-generated list of needed
dependencies (based on OS or other things) and ask user when he/she wants
to install that which one you need! and install it right away. another
suggestion (my idea) is an example:
When someone doesn't install i18n submodule. Codes breaks (error that says
"import i18n, there is no module named "i18n"" or something like that) but
we have to catch this error and ask a question and ask do you want to
install i18n submodule? and maybe a user doesn't want to install it (just
wants to run it in English WP) We have to let the user run the code.
Best
--
Amir
