[Engineering] [Analytics] [Eventlogging] Dropping Client IPs from EventCapsule

Wed Mar 2 19:53:01 UTC 2016

Thanks Madhu -- it's great to see the analytics team working proactively on
things like this.

-Toby

On Wed, Mar 2, 2016 at 10:18 AM, Madhumitha Viswanathan <
mviswanathan at wikimedia.org> wrote:

> Hi all,
>
> The analytics team, in an effort to collect sensitive data less, plans to
> drop the clientIP field from the EventCapsule(
> https://meta.wikimedia.org/wiki/Schema:EventCapsule), which is the
> wrapper for all events flowing into Eventlogging (Currently IPs and User
> Agents get purged after the 90 days mark). The field was originally meant
> only for debugging, but has served some research usecases. Most of these
> cases have been wrapped up at this point. It has also been used as a proxy
> to count number of devices visiting sites like our blog - and since IP's
> are not a good measure of that anyway - we plan to move such cases to use
> Piwik.
>
> The rollout of the change will happen in stages (Drop clientIPs first on
> the EL end, then the EventCapsule in meta, and finally on the VarnishKafka
> end). It should be a clean deployment and there's no scheduled downtime -
> EL will keep working as is. What does change? ClientIP's will start being
> set as NULL in your mysql tables. If you update the Eventlogging schema you
> maintain - causing new tables to be created, the new tables will not have
> the clientIp field in them. The change is planned to be rolled out the week
> of 11th or 18th March '16, pending the completion of data collection for
> the ongoing QuickSurveys based research work.
>
> Let us know if you have any questions/concerns on the list or on
> #wikimedia-analytics. The related phab ticket is here -
> https://phabricator.wikimedia.org/T128407.
>
> Thanks,
> Madhu Viswanathan
> Software Engineer, Analytics
>
> _______________________________________________
> Analytics mailing list
> Analytics at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/engineering/attachments/20160302/579d5e6d/attachment-0001.html>