[Engineering] [Analytics] Eventlogging Mysql consumers downtime

Thu Jan 28 12:34:15 UTC 2016

Yes, it has been completely backfilled.

Cheers!

On Thu, Jan 28, 2016 at 8:31 AM, Amir Aharoni <aaharoni at wikimedia.org>
wrote:

> Is the backfilling done? I see some data, but I can't be sure whether it's
> complete.
>
> 2016-01-25 20:34 GMT+02:00 Nuria Ruiz <nuria at wikimedia.org>:
>
>> We are working at this time on backfilling data for the scheduled
>> maintenance, we shall report when we think backlogging has finished.
>>
>> Thanks,
>>
>> Nuria
>>
>> On Mon, Jan 25, 2016 at 5:34 AM, Oliver Keyes <okeyes at wikimedia.org>
>> wrote:
>>
>>> Awesome; thanks :). Anything on the switchon time? (I appreciate it's
>>> only barely Monday ;p)
>>>
>>> On 21 January 2016 at 11:36, Andrew Otto <otto at wikimedia.org> wrote:
>>> > FYI, This has started.  I have turned off the MySQL processes.  No new
>>> data
>>> > will be inserted into MySQL until we turn them back on.
>>> >
>>> > On Wed, Jan 20, 2016 at 4:43 PM, Andrew Otto <otto at wikimedia.org>
>>> wrote:
>>> >>
>>> >> (+Jaime)
>>> >>
>>> >> It’s hard to tell how long it will take, but I imagine for quite a
>>> while.
>>> >> I guess a day max, but probably much less.  If Jaime let’s me know
>>> that he’s
>>> >> finished early, and I am close enough to a computer, I’ll turn the
>>> >> EventLogging MySQL processes back on asap.
>>> >>
>>> >>
>>> >>
>>> >> On Wed, Jan 20, 2016 at 4:12 PM, Oliver Keyes <okeyes at wikimedia.org>
>>> >> wrote:
>>> >>>
>>> >>> Thanks Madhu! Awesome to see the transparency and announces around
>>> this
>>> >>> :D.
>>> >>>
>>> >>> Do you know how long the reimport should take on Monday? I want to
>>> >>> make sure I know when to expect we can safely launch all our scripts.
>>> >>>
>>> >>> On 20 January 2016 at 13:08, Madhumitha Viswanathan
>>> >>> <mviswanathan at wikimedia.org> wrote:
>>> >>> > Hi all,
>>> >>> >
>>> >>> > In order to convert tables on db1046 to the TokuDB engine - we
>>> have to
>>> >>> > schedule some downtime on the Eventlogging databases between
>>> tomorrow,
>>> >>> > Thursday Jan 21, 2016 at 16:00 UTC to Monday Jan 25, 2016 16:00
>>> UTC.
>>> >>> >
>>> >>> > What this means for EL users:
>>> >>> >
>>> >>> > 1. Eventlogging will still receive data and it will be available in
>>> >>> > Kafka.
>>> >>> > The data will continue to be imported into Hadoop and files.
>>> >>> > 2. The Mysql consumers of Eventlogging will be stopped, so no data
>>> will
>>> >>> > get
>>> >>> > imported into the master(db1046 or m4-master) and by extension to
>>> the
>>> >>> > analytics-store.
>>> >>> > 3. Querying existing data from analytics-store will still be
>>> available,
>>> >>> > but
>>> >>> > data for the next 4 days won't be available.
>>> >>> > 4. On Monday, after the maintenance window we'll restart the Mysql
>>> >>> > consumers, and all the data should get reimported from Kafka.
>>> >>> >
>>> >>> > Analytics and Ops(DBA) will work on this together.
>>> >>> >
>>> >>> > Feel free to reach out to us here or on #wikimedia-analytics if you
>>> >>> > have any
>>> >>> > concerns/questions.
>>> >>> >
>>> >>> >
>>> >>> > -- Madhu Viswanathan
>>> >>> > Software Engineer, Analytics
>>> >>> >
>>> >>> > _______________________________________________
>>> >>> > Engineering mailing list
>>> >>> > Engineering at lists.wikimedia.org
>>> >>> > https://lists.wikimedia.org/mailman/listinfo/engineering
>>> >>> >
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Oliver Keyes
>>> >>> Count Logula
>>> >>> Wikimedia Foundation
>>> >>>
>>> >>> _______________________________________________
>>> >>> Engineering mailing list
>>> >>> Engineering at lists.wikimedia.org
>>> >>> https://lists.wikimedia.org/mailman/listinfo/engineering
>>> >>
>>> >>
>>> >
>>>
>>>
>>>
>>> --
>>> Oliver Keyes
>>> Count Logula
>>> Wikimedia Foundation
>>>
>>> _______________________________________________
>>> Analytics mailing list
>>> Analytics at lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/analytics
>>>
>>
>>
>> _______________________________________________
>> Engineering mailing list
>> Engineering at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/engineering
>>
>>
>
>
> --
> Amir Elisha Aharoni‏ ። אָמִיר אֱלִישָׁע אַהֲרוֹנִי
> Language Engineering‏ ። הַנְדָּסָה לְשׁוֹנִית
> Wikimedia Foundation‏ ። קֶרֶן וִיקִימֶדְיָה
>
> _______________________________________________
> Analytics mailing list
> Analytics at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>
>

-- 
*Marcel Ruiz Forns*
Analytics Developer
Wikimedia Foundation
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/engineering/attachments/20160128/5a257ce6/attachment.html>