[Labs-l] Problems when setting up an instance of Wikidata

Tue Apr 17 15:15:34 UTC 2012

Hi,

I'm sure Ryan will have more input, but I've got responses to a few of
these questions that might help.  Inline below.

On Tue, Apr 17, 2012 at 7:14 AM, Daniel Kinzler <daniel at brightbyte.de>wrote:

> Hi all
>
> We have today tried to set up an instance for wikidata, namely
> "wikidata-dev-1"
> in the "wikidata" project. We have run into several issues - some are
> merely
> things that appear unclear, others seem genuinely misswing or broken. I'll
> try
> to be brief in my descriptions in order to provide an overview. You can
> ask me
> or reedy for details.
>
> So, here goes:
>
> * UNCLEAR: Which image (ubuntu version) should be used (when trying to be
> close
> to the live environment)? Why are the different versions there? And why is
> there
> both "oneric" and "oneiric"?
>
> * MISSING: apparently, the security groups (firewall rule sets) for an
> instance
> can not be changed after it was created.
>

This is by design, and is the same on EC2.

> * MISSING/UNCLEAR: We will want to create our own puppet scripts for
> setting up
> wikidata repos for testing, etc. Where do we need to submit them? How
> would we
> apply them to instances?
>

This is a section of the docs that are missing.  In the absence of real
docs, I wrote down what I had to do to get a new project up and running
with my puppet stuff:
https://labsconsole.wikimedia.org/wiki/User:Bhartshorne/Path_to_a_New_Project

I would love it if someone would take that and turn it into something that
can be a part of the general documentation.

>
> * MISSING: it would be very convenient to be save "profiles" for setting
> up new
> instances - profiles would just be the puppet config for that instance.
> Alternatively, allow a new instances to be created "configured just like
> that
> one over there".
>

For the swift project I set up, once I got my puppet stuff right, all I
added to specific nodes was 'swift storage' or 'swift proxy'.  I think that
most of our nodes wind up with just a small set of classes, so for me at
least, this wasn't too important.  If there are a large number of puppet
classes you're installing on a host, perhaps that's an indication that the
abstraction is wrong somewhere?  I'm not very good with puppet, so I may be
way off base here.

>
> * MISSING: foll VM snapshots. But that's just "nice to have". Puppet
> templates
> would be much more useful.
>
> * BROKEN: We were unable to install a mysql server. Trying to apply db:core
> failed with an error. The error ocurrs when trying ot restart the ganglia
> monitoring daemon (gmon or whatever). It complains about a syntax error
> (unexpected token "}") in the config file.
>

Try using the generic::mysql::server class instead of the db::core class.
The core class is specifically tuned for our production DBs.  You might
have to cherrypick the generic mysql stuff over; I'm not sure it's in the
test branch.
https://gerrit.wikimedia.org/r/#patch,sidebyside,4599,2,manifests/generic-definitions.pp

>
> * BROKEN: Actually, installing mysql (db:core) initially failed with a
> different
> error: it's missing the directory /a. If that dir is needed, the puppet
> script
> should create it, no?
>

same as above.

>
> * UNCLEAR: after changing the puppet config for an instance, when are the
> changes applied? We ended up running puppetd -tv manually from the shell...
> shouldn't the console just trigger that whenever the config is changes?
>

I've been running puppetd -tv as well.

>
> * MISSING/UNCLEAR: some puppet bundles are incompatible with each other.
> Trying
> to install all of apache2 and apache2:php5 and apache2:php5-mysql fails
> with an
> unhelpful error message about conflicting declarations. So, dependencies
> and
> conflicts between bundles should somehow be indicated (or at least there
> should
> be meaningful messages on the console).
>

it's true.  I don't know a good way around this.  For example, the
generic::mysql::server class conflicts with the db::core class.  They both
try and install (different versions of) the mysql server in different ways
with different configurations.  I think that's just a fact of having a
large environment; we need different configs for stuff, so we have to
choose the right one for a given task.

>
> * UNCLEAR: which are the bundles needed to install a standard LAMP stack?
>
> * MISSING: we were not able to find out reliably when an instance has
> finished
> booting. It would be helpful to at least have a big message in the console
> output. Just put a big "READY TO ROLL" in there. Well, ideally, the status
> in
> the instance list should be "booting", not "running", until it's fully up.
>

You can check the console output but instead I usually just go make a cup
of tea.  ;)

>
> We'd be very grateful for any input on the above issues. As it stands,
> we'll be
> using labs as a plain VM hoster, and install everything by hand. Which
> kind of
> sucks...
>

HTH,

-ben

>
> Thanks
> Daniel
>
>
> _______________________________________________
> Labs-l mailing list
> Labs-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20120417/9f4d6981/attachment.html>