[Labs-l] Cannot connect to DB

Tue Jan 27 19:55:52 UTC 2015

Labs-wide restarts are happening today due to security issues, so
expect some instabilities as we work things through.

On Tue, Jan 27, 2015 at 11:53 AM, Nuria Ruiz <nuria at wikimedia.org> wrote:
> Also  we get "too many connections".
>
> Yuvi: (or anyone) what are appropriate settings for pool size for db
> connections for these hosts?
>
> Thanks again,
>
> Nuria
>
> On Tue, Jan 27, 2015 at 11:21 AM, Nuria Ruiz <nuria at wikimedia.org> wrote:
>>
>> Seems like something is missing cause despite having changed the config
>> our wikimetrics1 instance is still throwing a bunch of errors like:
>>
>>
>>  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py",
>> line 238, in _revalidate_connection
>>     "Can't reconnect until invalid "
>> StatementError: Can't reconnect until invalid transaction is rolled back
>> (original cause: InvalidRequestError: Can't reconnect until invalid
>> transaction is rolled back) u'SELECT user.user_id AS user_u
>>
>>
>> Restating the instance fixes things but somehow it gets "blocked" again.
>> Is there something special about how these connections need to be managed?
>>
>>
>> Config is set up as follows (user removed)
>>
>> SQL_ECHO                        : False
>> WIKIMETRICS_ENGINE_URL          : 'mysql://<******>@localhost/wikimetrics'
>> WIKIMETRICS_POOL_SIZE           : 100
>> MEDIAWIKI_ENGINE_URL_TEMPLATE   :
>> 'mysql://<****>. at labsdb1002.eqiad.wmnet/{0}_p'
>> MEDIAWIKI_POOL_SIZE             : 200
>> CENTRALAUTH_ENGINE_URL          :
>> 'mysql://<*****>@labsdb1002.eqiad.wmnet/centralauth_p'
>> REPLICATION_LAG_MW_PROJECTS     : [ 'enwiki', 'eowiki', 'elwiki',
>> 'commonswiki', 'dewiki', 'frwiki', 'eswiki' ]
>> REPLICATION_LAG_THRESHOLD       : 3 # (measured in hours)
>>
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Jan 26, 2015 at 5:09 PM, Yuvi Panda <yuvipanda at gmail.com> wrote:
>>>
>>> Yes they should!
>>>
>>> On Jan 26, 2015 5:04 PM, "Nuria" <nuria at wikimedia.org> wrote:
>>>>
>>>> Do these hosts also workfor the centralauth
>>>> database?
>>>>
>>>>
>>>>
>>>> > On Jan 21, 2015, at 8:20 PM, Yuvi Panda <yuvipanda at gmail.com> wrote:
>>>> >
>>>> > the DNAT rules are no longer required. Just connect directly to one of
>>>> > labsdb1001.eqiad.wmnet, labsdb1002.eqiad.wmnet, or
>>>> > labsdb1003.eqiad.wmnet. That should work.
>>>> >
>>>> > I tried this out on wikimetrics1 instance and that seems to work...
>>>> > --
>>>> > Yuvi Panda T
>>>> > http://yuvi.in/blog
>>>> >
>>>> > _______________________________________________
>>>> > Labs-l mailing list
>>>> > Labs-l at lists.wikimedia.org
>>>> > https://lists.wikimedia.org/mailman/listinfo/labs-l
>>
>>
>

-- 
Yuvi Panda T
http://yuvi.in/blog