[Labs-l] RFC: Webtools setup

Thu Feb 14 22:01:26 UTC 2013

On 14/02/13 19:45, Tim Landscheidt wrote:
> Hi,
> 
> some brainstorming about how to set up Webtools
> (https://labsconsole.wikimedia.org/wiki/Nova_Resource:Webtools).
> Please chime in.
> 
> A tool for the purpose of Webtools is a set of files and
> scripts that form a logic unit that can't be split up sanely
> any more, i. e. some PHP files accessed online, some PHP
> scripts for maintenance, some data in files and/or a data-
> base, perhaps some (static) icons, etc.
+1

> Each tool is separated from other tools so they cannot
> change each other's data and any intrusion is limited to one
> tool.
+1

> Tools have one or more developers/maintainers.  A developer
> can work on several tools and needs more (or other) access
> rights than his tool(s).
+1

> Dependencies of tools on other software are specified ex-
> plicitely so that tools can be moved to other servers or
> servers can be split by other software needed (i. e., a
> server that only handles PHP, Ruby on Rails, etc.).  Depen-
> dencies can be different for development (command line) and
> deployment (web).

This could be more complex to organise, since cli and web may still need
to be grouped in the same tool.

> As much configuration as possible should be maintained with
> Puppet and in Gerrit.
> 
> So my proposal is:
> 
> - Each tool has one user under which its web scripts (and
>   perhaps cron jobs) are run.  That user's name should be
>   identical to the tool name used in URLs & Co.
+1

> - Each tool has a user group that consists of the tool user
>   and the developers.
+1

> - Each tool has a directory under /data/project/web, owned
>   by the user and the group, writable by the user and the
>   group.
+1

>   In it, the subdirectory "htdocs" contains the web
>   stuff ("htdocs/cgi-bin" for CGI), the rest of the direc-
>   tory (structure) can be used for private data (including
>   bot credentials), log files & Co.

I used public_html, keeping the same name as used in the toolserver, but
the name isn't really important.

> - Each tool has a Puppet module à la:
> 
>   - webtools::TOOL::someuniqueserver:
> 
>     - ensure that the directory structure under
>       /data/project/web/TOOL is set up
> 
>   - webtools::TOOL::loginserver:
> 
>     - user TOOL exists
>     - group TOOL with members TOOL and developers exists
>     - development dependencies for TOOL exist
> 
>   - webtools::TOOL::webserver:
> 
>     - user TOOL exists
>     - group TOOL with members TOOL and developers exists
>     - deployment dependencies for TOOL exist
>     - configuration in /etc/apache2/conf.d/TOOL for
>       URL "/TOOL/" -> directory
>       "/data/project/web/TOOL/htdocs/" (plus CGI directory)

This looks good, although I wonder if it's practical to ask users to add
puppet configuration for each tool. Maybe it could be templated to the
point where you only provide the tool name as a parameter to get all
those points.

> My (first :-)) questions are:
> 
> - Can glusterfs handle local users and groups on
>   /data/project, or do we need to synchronize uids/gids?

I'm not sure about what you mean. The tools uids should not collision
with the LDAP users, and we should have a central store of them. We
talked about this in irc some time ago, with no clear results. Although
I think it would be safe to start tool uids with eg. 50000.

> - It's probable that some file beneath the "htdocs" direc-
>   tory (or "public_html" or whatever) will at one point be
>   owned by a developer, but they shouldn't be executed as
>   his account.  Can we configure Apache to execute all
>   scripts beneath "/data/project/web/TOOL/htdocs/" as TOOL?

Good idea, I will take it into account. I don't know if you noticed that
I configured webtools-apache-1 to read the tools space, but I didn't set
the setuid yet (all the tools we don't have yet run as apache). It needs
to be changed from mod_php5 to fcgi.