Hello,
Pretty much everyone who's dealt with creating views for new wikis on the
labs hosts have experienced issues with "Access denied" sometimes.
This was usually due to the MariaDB grant role being missed. We tried to
workaround this by including the grant addition on the maintain-views
script.
Unfortunately, we ran into very weird problems when doing so and this is an
example: https://phabricator.wikimedia.org/T193187#4273281
After lots of back and forth we decided to create a bug to MariaDB (
https://jira.mariadb.org/browse/MDEV-16466) which was confirmed by MariaDB
yesterday and pointed to a similar issue (
https://jira.mariadb.org/browse/MDEV-14732).
The expected fix will come in 10.4 (we are in 10.1), so quite long ahead of
us.
So, for now, the workaround before adding new views is to manually add the
GRANT on the DB and then run the script:
GRANT SELECT, SHOW VIEW ON `newiki\_p`.* to labsdbuser';
Hopefully with this email everyone is on the same page now.
Thanks everyone (specially Brooke for helping me out with the
troubleshooting!)
Manuel.
Hello,
I've deployed change 463611 which converts the DNSBL checks
introduced in 462472 (thanks Keith!) from just warnings to actually
taking action against spammers.
Also enabled sender verification checks (on top of the already
enabled recipient checks). This should help with bounce emails that
can't get delivered (frozen in Exim's queue).
I tested email delivery to/from various hosts, tool's email, watched
emails going to other tools, etc. It seems to be working fine but I'm
sure there will be some false positives.
Infamous @qq.com spam being blocked:
2018-09-29 14:03:38 H=(0voc.cn) [122.237.40.138] F=<5281141(a)qq.com>
rejected RCPT <Mailer-Daemon(a)tools.wmflabs.org>: Blocked by DNSBL (see
https://www.spamhaus.org/query/ip/122.237.40.138)
Sender addresses check:
2018-09-29 14:07:07 H=(ABC) [138.99.254.197] sender verify fail for
<noreply(a)i.dont.exist>: Unrouteable address
2018-09-29 14:07:07 H=(ABC) [138.99.254.197]
F=<noreply(a)i.dont.exist> rejected RCPT
<gtirloni-sandbox.anything(a)tools.wmflabs.org>: Sender verify failed
Hopefully, this decreases the amount of spam we're getting today
until the new SMTP servers are in place. Please let us know if
something broke due to this change.
Thanks
--
Giovanni Tirloni
Operations Engineer
Wikimedia Cloud Services
Hello,
I want to deploy a VM instance on a project to be the test NFS server
we have been talking about it.
According to the server naming conventions [0], I should name this
"cloudstore-XXXX-dev", right?
Also, are we increasing the numbers from the latest labstore servers
or starting fresh for cloudstore?
Is there some other VM naming convention I should follow (or that our
tenants are encouraged to follow)?
Thanks
0 - https://wikitech.wikimedia.org/wiki/Infrastructure_naming_conventions#Serve…
--
Giovanni Tirloni
Operations Engineer
Wikimedia Cloud Services
- Lots of folks are absorbed in the DC migration test coming up on 10-10
and 10-11 . That's unlikely to affect us but it would be good for some
of us to be available in case of unexpected messes.
- Moritz is a bit interested in the NFS vs. Kernel issues; he's been
messing with the 4.14 kernel and might want to give that a try in case
it happens to resolve the issue.
- Procurement people have a new scheme to get our new labvirts working
(swapping in older hardware) but it's not yet a sure thing; and even if
that works Moritz may have to backport some things to Jessie.
- Arzhel will make the cross-region routing network changes in the next
few hours.
- Giovanni's root approved!