[Engineering] Reverted to 1.28.0-wmf.8 once more.

Fri Jan 27 18:47:48 UTC 2017

Status update:

Anomie has a solid theory about the cause of
https://phabricator.wikimedia.org/T156364 and we are going ahead with
deploying wmf.9 today ASAP.  Assuming everything goes as planned then this
will be resolved shortly.

On Fri, Jan 27, 2017 at 11:02 AM, Mukunda Modell <mmodell at wikimedia.org>
wrote:

> Correction: It looks like I only reverted group2 to wmf.8
>
> I'm not sure whether to go ahead and revert all wikis or leave it like it
> is for now.
>
> On Fri, Jan 27, 2017 at 10:24 AM, Mukunda Modell <mmodell at wikimedia.org>
> wrote:
>
>> The deployment of MediaWiki 1.29.0-wmf.9 is currently blocked and all
>> wikis have been reverted to wmf.8.  This latest revert is due to excessive
>> warnings which appeared in logs immediately after deploying to group2. The
>> warnings all read as follows:
>>
>> Warning: Empty regular expression in /srv/mediawiki/php-1.29.0-wmf.
>> 9/includes/parser/DateFormatter.php on line 200
>>
>> This issue is tracked in https://phabricator.wikimedia.org/T156364 and
>> currently blocks further deployments.
>>
>> More background:
>>
>> A similar flood of log spam occurred after deploying the branch to
>> group1, however, the problem subsided after a while and did not reappear
>> until Thursday's deployment.
>>
>> There have been recent changes to DateFormatter.php, however, the changes
>> do not appear to be the direct culprit. In fact, after rolling back the
>> changes, a different message appears in the log, however, with less
>> frequency than the line 200 error.
>>
>> Several engineers have looked at this issue but so far we have not
>> isolated the root cause. Additional eyes on this issue would be welcomed
>> and further discussion should be directed to Phabricator. In the interest
>> of keeping everyone in the loop, I will follow up in this thread if/when
>> there is a resolution.
>>
>> Regards,
>> Mukunda
>>
>> ---
>> Mukunda Modell
>> Wikimedia Release Engineering
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/engineering/attachments/20170127/b14266d2/attachment.html>