ouch, that is not at all the result we were hoping for. Just goes to show
why we have to test these things and not just take a few examples that
perform badly in one set and look to do a better job with some different
options. Thanks for putting this together!
On Thu, Feb 25, 2016 at 3:06 PM, Dmitry Brant <dbrant(a)wikimedia.org> wrote:
Hello all,
As mentioned previously, the current version of the Android app contains
an A/B test where it presents "read more" suggestions to the user, based on
(a) the standard "morelike" query, or (b) the new "opening_text"
query.
Here are the results from the last ~10 days of the test[0]:
- The clickthrough rate using the default morelike query is (and has been)
around 15%.
- With the new opening_text query, the clickthrough rate decreases to
about 12%:
[image: Inline image 1]
Therefore, it seems that the new query has a nontrivial negative effect on
CTR :(
We'll plan on removing this test in the next release of the app, but we'll
be happy to plug in a different or updated query, if it will be of further
use to Discovery.
[0]
https://docs.google.com/a/wikimedia.org/spreadsheets/d/1BFsrAcPgexQyNVemmJ3…
(queries embedded as comments in the headers)
--
Dmitry Brant
Senior Software Engineer / Product Owner (Android)
Wikimedia Foundation
https://www.mediawiki.org/wiki/Wikimedia_mobile_engineering
_______________________________________________
Mobile-l mailing list
Mobile-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l
