I can see this becoming very useful as we query across projects,
languages, etc to not block on anything explicitly.
Nice work Erik. Great to see us experimenting for future use.
--tomasz
On Tue, Sep 8, 2015 at 12:01 PM, Erik Bernhardson
<ebernhardson(a)wikimedia.org> wrote:
The php engine used in prod by the wmf, hhvm, has
built in support for
shared (non-preemptive) concurrency via async/await keywords[1][2]. Over the
weekend i spent some time converting the Elastica client library we use to
work asynchronously, which would essentially let us continue on performing
other calculations in the web request while network requests are processing.
I've only ported over the client library[3], not the CirrusSearch code. Also
this is not a complete port, there are a couple code paths that work but
most of the test suite still fails.
The most obvious place we could see a benefit from this is when multiple
queries are issued to elasticsearch from a single web request. If the second
query doesn't depend on the results of the first it can be issued in
parallel. This is actually somewhat common use case, for example doing a
full text and a title search in the same request. I'm wary of making much of
a guess in terms of actual latency reduction we could expect, but maybe on
the order of 50 to 100 ms in cases which we currently perform requests
serially and we have enough work to process. Really its hard to say at this
point.
In addition to making some existing code faster, having the ability to do
multiple network operations in an async manner opens up other possibilities
when we are implementing things in the future. In closing, this currently
isn't going anywhere it was just something interesting to toy with. I think
it could be quite interesting to investigate further.
[1]
http://docs.hhvm.com/manual/en/hack.async.php
[2]
https://phabricator.wikimedia.org/T99755
[2]
https://github.com/ebernhardson/Elastica/tree/async
_______________________________________________
Wikimedia-search mailing list
Wikimedia-search(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimedia-search