Guides
These are the Guides that the SRCF has produced for its services.
Changes in 20.04 Focal
These notes relate to the migration from Ubuntu 16.04 to 20.04,
scheduled for 3rd/4th April 2021.
Server changes
The following shell-accessible servers, currently on Ubuntu 16.04
'Xenial Xerus', will be upgraded to 20.04 'Focal Fossa':
pip (aka. shell.srcf.net), the main shell serversinkhole (aka. webserver.srcf.net), the web server
As we already have doom, a 20.04 server for games and long-running
processes, we'll be decommissioning the old games server cavein
shortly after the upgrade. Please migrate your servers and scheduled
tasks ahead of the upgrade.
We'll also be upgrading other servers hosting various SRCF services,
which will experience some downtime:
blizzard, the DNS server (though access to other services on the
srcf.net domain will be unaffected by this)flame, the news serversquirrel, the MySQL server
Software updates
If you’d like to check what version of a particular package will be
installed after the upgrade, you can search for it on the Ubuntu
package directory. If a package is listed
for xenial, but not for focal, then it is liable to be removed
following the upgrade -- contact the
sysadmins if there's a package you need
that's slated for removal, and we can see what can be done.
If you've built or compiled your own software linked to libraries on
the system, you'll likely need to recompile them following the upgrade.
Python 2
The majority of Python 2 packages have been dropped by Ubuntu, leaving
only a handful of libraries acting as dependencies for packaged software
that doesn't run on Python 3. If you're running your own software with
py2, you should seek to make it compatible with py3 -- official support
for py2 ended in January 2020, with setuptools, pip and other core
components becoming py3-only.
Failing that, the core interpreter is still available, along with the
ability to create py2 virtual environments if you're currently relying
on any system-installed py2 packages that will no longer be available.
Beware of upgrading the build tools in such environments -- the last
version of pip to support py2 is 20.3, and for setuptools the last
compatible release was 44.0.0.
Python 3
Python 3 itself will receive a minor version bump from 3.5 to 3.8 --
the What's New section of the
Python documentation lists all the changes between versions.
The change in version also changes the library path, which means any
packages user-installed to ~/.local/lib/python3.5 (with
pip install --user or otherwise) will no longer be accessible and will
need reinstalling following the upgrade. In addition, any virtual Python
environments using the system copy of py3 will stop working, and will
need recreating. Custom installations of Python (e.g. those installed
with tools like pyenv) may also need
reinstalling following system library updates.
PHP
PHP will be updated from 7.0 to 7.4. A list of new features and
backwards-incompatible changes can be found in their
documentation.
Notably, this introduces a number of new syntax features, such as arrow
function shorthand and the null-safe ?. operator. The majority of
backwards-incompatible changes seem to cover previously unspecified or
erroneous behaviour, though consult the Backward Incompatible Changes
sections for all the details.
Node.js
Node is being brought into the (relatively) modern times, to be updated
from major version 4 to 10. The canonical binary name will change from
nodejs to node, though the former will remain as a symlink.
Unsurprisingly there will have been a number of additions and changes
-- Node's
changelogs
cover all the details but are quite verbose; there's also this list of
breaking changes
if you just want to check your code for compatibility.
Upgrading a Drupal installation
Overview
Version 6 releases are no longer supported by Drupal, and are liable to
have compatibility issues with PHP 7.
The notes below assume your Drupal site is at drupal.soc.srcf.net --
substitute this for your own site’s URL.
Drupal provide their own extensive guide for
upgrades,
which covers the below information in more detail.
Upgrading files in-place
This method is suitable for minor version upgrades (e.g. from an old 7.x
to the latest 7.x release). See also: Update procedure (minor version
change)
Take backups of your files and database before attempting an
upgrade. Then put the site into maintenance mode.
Download the latest release from https://www.drupal.org/download. The
ZIP file contains a folder called drupal-<version>, which contains the
core files.
If you're working over SSH, copy the URL of the ZIP download, use
wget to download it, then unzip to unpack it in the current
directory.
For SCP, you'll want to extract the files on your local machine, then
upload them to the server.
In either case, the contents of the drupal folder are to be moved into
your existing site (for drupal.soc.srcf.net, this would be
/societies/drupal/public_html). You should delete all files under the
site directory, '''except''' the sites folder which holds all
customisations. If you've made any changes to .htaccess or
robots.txt, you'll also need to merge these into the updated files.
Once the new files are in place, head over to
https://drupal.soc.srcf.net/update.php to prepare the database for the
new version. When complete, take the site out of maintenance mode.
Migrating a site between instances
This is needed for major release upgrades (e.g. from 7.x to 8.x).
Similar to the above, download the latest version, but extract/upload it
to a different location than the current site. Configure this as a new
installation. It still needs to be under your account's public_html
root, so you may need to use a subdirectory. This page assumes you have
https://drupal.soc.srcf.net/new/.
Go to the ''Modules'' section on the new site, and enable the
''Migrate'', ''Migrate Drupal'', and ''Migrate Drupal UI''
modules.
Now head over to https://drupal.soc.srcf.net/new/upgrade, which will
prompt for database credentials -- here, you can specify those of the
old site, which will import all its content into the new installation.
Once you're happy the new site is working, you can move the old one out
of the way and make the new site take its place. You'll need to update
the $base_url in settings.php with the new path.
Upgrading WordPress
Version 3.7 of WordPress introduced an automatic updater, which should
keep your instance up-to-date (as long as someone visits every so often
to kick off the update process).
Versions below 4.4 are likely not compatible with PHP 7, meaning the
admin pages and automatic updates won't work. In this case, such an
instance would need to be upgraded manually.
The notes below assume your WordPress site is at wordpress.soc.srcf.net
-- substitute this for your own site's URL.
WordPress provide their own extensive guide for
upgrades, which covers
the below information in more detail.
Allow WordPress to write its own files
WordPress includes an update utility, but this can be quite tedious
about not correctly detecting that it can update itself; it may fail
without even trying, especially on society accounts.
To make this work, you can add the following to your site's
configuration (wp-config.php), near the bottom but above the line that
says "stop editing":
/**
* Force wordpress to use direct filesystem access so that upgrades work
* properly. See: https://core.trac.wordpress.org/ticket/10205
* https://codex.wordpress.org/Editing_wp-config.php
*/
define('FS_METHOD', 'direct');
define('FS_CHMOD_DIR', (02775 & ~ umask()));
define('FS_CHMOD_FILE', (0664 & ~ umask()));
You must ensure all WordPress files have the correct group
permissions. For society accounts, this means all files should have
the society's group and be group-writable. WordPress will not check
this in advance -- if it fails to write some files during an upgrade,
you may end up with a broken instance.
This should be the case by default, but some SFTP clients may override
it. In addition, files moved/copied from your home directory may still
have your personal group set, rather than that of the target society.
Upgrade through the admin panel
Head over to https://wordpress.soc.srcf.net/wp-admin/, log in with your
WordPress username and password if you haven't already (remember, these
are distinct from any SRCF-provided accounts).
Select ''Updates'', under ''Dashboard'' in the side menu.
If an update is available, you should have a button to perform the
upgrade in one step. If you've installed a non-US-English version, make
sure to pick the one with the correct language.
If you're prompted for SFTP credentials, see the section above for how
to grant direct file access.
Upgrade files manually
Take backups of your files and database before attempting an
upgrade.
Download the latest release from https://wordpress.org/download/. The
ZIP file contains a folder called wordpress, which contains the core
files.
If you're working over SSH, you can just
wget https://wordpress.org/latest.zip to download the ZIP file, then
unzip latest.zip to unpack it in the current directory.
For SCP, you'll want to extract the files on your local machine, then
upload them to the server.
In either case, the contents of the wordpress folder are to be moved
into your existing site (for wordpress.soc.srcf.net, this would be
/societies/wordpress/public_html). You should delete the wp-admin
and wp-includes folders ('''not''' wp-content) first, as the
new files provide complete copies of these folders. The contents of
wp-content should be merged with the existing directory, in order to
retain plugins and themes.
Once the new files are in place, head over to
https://wordpress.soc.srcf.net/wp-admin/ and log in with your WordPress
username and password to check if things are working. You may be
prompted to complete a database upgrade -- this is normally a one-click
step, just follow the instructions it provides.
Web server migration
These notes relate to a previous migration from Ubuntu 12.04 to 16.04,
completed in summer 2017.
Apache
Upgraded from 2.2 to 2.4. See Apache's own upgrade
documentation for a
full list of changes.
The semantics of Allow/Deny directives have changed -- we have
''mod_access_compat'' enabled which should restore the legacy
syntax, though we recommend you migrate to the new Require syntax.
Logs
You'll now need to SSH to webserver.srcf.net to access logs -- you
will find access.log and error.log in
/var/log/apache2/user/<crsid> for your personal website, or
/var/log/apache2/soc/<name> for a society website. Note that CGI error
output should now consistently go to the corresponding error.log file
(previously some types of output ended up in the main log at
/var/log/apache2/error.log).
Proxy webservers
If you run your own webserver (e.g. gunicorn, nginx) and have Apache
proxy to it via htaccess, these server processes need to be run from the
webserver rather than the shell server. As above, SSH to
webserver.srcf.net and run them from there.
PHP
Upgraded from 5.3 to 7.0. This jumps multiple minor versions -- the
bulk of the changes can be found in PHP's own migration guide for 5.6
to 7.0, though see
also the guides for
5.4,
5.5,
5.6.
MySQL
The original MySQL library, which provides methods with names starting
mysql_, has been deprecated since 5.5, and removed completely in 7.x.
MySQLi provides a
close replacement, with most original methods present but prefixed with
mysqli_. You'll also need to pass a connection handle (the return
value of mysqli_connect()) as the first argument to most of the
remaining methods.
Sample MySQL library code:
mysql_connect("localhost", "<user>", "<password>");
$result = mysql_query("...");
while ($row = mysql_fetch_assoc($result)) { ... }
Equivalent MySQLi code:
$conn = mysqli_connect("localhost", "<user>", "<password>");
$result = mysqli_query($conn, "...");
while ($row = mysqli_fetch_assoc($result)) { ... }
ucam-webauth-php
Versions prior to 0.53 include a function call with an argument ignored
in PHP 5.x but removed in 7.x. Error logs will contain something like:
PHP Warning: gmmktime() expects at most 6 parameters, 7 given in .../ucam_webauth.php on line 388
The latest version can be obtained from
GitHub.
php_override.ini
PHP is now run as an Apache module rather than CGI, so override files
are no longer supported. You can however override settings using
htaccess.
Example php_override.ini setting:
Equivalent htaccess setting:
PostgreSQL
The Postgres server has also moved to the new webserver. For websites
connecting to a database, it's still accessible locally (socket
connection) so access should be unaffected.
If you have scripts on the main shell server that access a Postgres
database, these need updating to connect over TCP to hostname
postgres. Ident auth is available so you still shouldn't need to
provide a password.
The webserver is not a shell server
Whilst we provide SSH access to read logs and start servers (see below),
it lacks the resources of the main shell server, and should therefore
not be used as one.
Securing WordPress
WordPress is a regular hackers’ target and instances of WordPress on
the SRCF are regularly compromised in various ways, leading to private
data leaks, tampered files, and website irregularities. Most of these
risks can be minimized by following the hardening steps below.
- Ensure your
wp-config.php is not world-readable, as that will
contain your database credentials. You can set permissions in your
FTP client or in the console by typing chmod 0660 wp-config.php.
If you are installing Wordpress under web space that belongs to a
group or society then your wp-config.php file will be owned by
that group or society’s user account rather than your own personal
account, in which case you will need to run the previous console
command as that user: sudo -u socname chmod 0660 wp-config.php.
Unfortunately FTP clients won’t let you do this so you will need to
use the console.
- It is advised to lock down WordPress’ admin panel at /wp-admin/ by
putting that directory behind Raven authentication – see an example
at
/public/societies/sample/public_html/wordpress/wp-admin/.htaccess.
- We also recommend you disable Allow link notifications from other
blogs (pingbacks and trackbacks) on new posts, under
Settings >
Discussion in the admin panel.
- Activate a spam filtering plugin like Akismet and a capatcha system
like reCAPTCHA. Akismet is installed by default and just needs
activiating. Go to
.../wp-admin/plugins.php to install and
activate plugins.
- You may optionaly want to modify your theme so that it no longer
puts the Wordpress version into the html - this may help stop
hackers finding that you installation is outdated but it does not
protect against problems caused by the version you are using being
compromised.
- Regularly check up on the status of your installation and keep an
eye on any vulnerabilities in the plugins you use.
Society Handovers
Overview
Ensuring that a proper handover occurs will reduce headaches for the new committee in the long term. This guide explains a few simple pointers to keep in mind during the transition period.
This guide is written with societies in mind as they tend to have regular annual handovers but is applicable to any users of our group accounts.
New webmasters
It is becoming increasingly common that the role of “webmaster” is merged into the job of the secretary or the publicity officer. This often means that modifying the website becomes a daunting task due to unfamiliarity with the relevant technical aspects and/or the SRCF. We’ve tried to lower the barriers to entry by writing helpful tutorials with beginners in mind. If you are new to this, please do not be discouraged as we are here to help! We offer several ways of getting help, including a live chat.
As a quick introduction, the Student-Run Computing Facility is best known for providing free web hosting and email forwarding for students in Cambridge.
Granting access
The most important part of the handover is for the incumbent webmaster to add you as an admin to the society account. This can easily be done by:
- Navigating to our control panel
- Under “Your group accounts”, click “Manage” on your society’s card
- Under the “Administrators” card, click “Add new administrator”
- Type in the CRSid of the new admin
This process implies two things: first, the new webmaster must have an SRCF account beforehand (easily done by going to our home page and clicking “Create an account”) and second, any society admins must have Raven access, or in other words be member of Cambridge University.
If possible, you may consider keeping the old webmaster as an admin on the account. We recommend two or more admins to ensure the longevity of the society account in case one or more admins become unreachable.
If you are using a content mangement system like WordPress, this is also the time to add the new webmaster via the web interface as an administrator of the website.
Check details
This is also a good time to check if all society information is correct, such as the role address. This email is used in case the society has no remaining admins and will be our last attempt before permanently deactivating the account. Admins are automatically removed if their email bounces, ie. we can no longer reach them by email.
Files
Once the new webmaster is added, the incumbent webmaster should explain how the website works and how it can be updated. A few written notes on this can go a long way towards facilitating the handover.
In the majority of cases where a content management system is used (like WordPress), “updating” can be done via the handy web interface. If something goes wrong though, it is necessary to log in and poke around at the files. We’ve written tutorials for this.
At this point, the new webmaster should also read our reference material on group accounts, especially on file permissions and how our web services work. The gist of it is that all content gets served from /public/societies/<socname>/public_html and private society files are located at /societies/<socname>/. Shortcutes (symbolic links) to these are automatically added to your home directory.
Email forwarding
A new committee means new emails! If you are making use of email forwarding in the form of socname-chair@srcf.net then you need to update where these emails forward to. This is done in the .forward file present in the society’s home directory and we have more information on that here.
Mailing lists
If you are making use of mailing lists of the form listname@lists.srcf.net then you need to:
- Share the admin password with the new webmaster
- Add the new webmaster as a mailing list admin
This will allow them to, for example, send email to the list members without verification, manage requests from other committee members to the mailing list, and most importantly grant them access to the mailing list admin interface available at https://lists.srcf.net.
Other steps
There are other steps involved in a society handover, like social media handovers, chat groups (check out our Mattermost!), file sharing and more. These are beyond the scope of the SRCF but we mention it here to stress the importance of having a proper handover document (even if brief!) as these details easily get lost over time.
As we have experience with many societies, we are happy to advise on any further steps – just reach out to us!
Getting Started: Groups
Note for beginners
It is becoming increasingly common that the role of “webmaster” is merged into the job of the secretary or the publicity officer. This often means that modifying the website becomes a daunting task due to unfamiliarity with the relevant technical aspects and/or the SRCF. We’ve tried to lower the barriers to entry by writing helpful tutorials with beginners in mind. If you are new to this, please do not be discouraged as we are here to help! We offer several ways of getting help, including a live chat.
As a quick introduction, the Student-Run Computing Facility is best known (though we do other things!) for providing free web hosting and email forwarding for students in Cambridge.
The above is copied from our society handovers guide.
Users vs. groups
Even as a society webmaster, or someone looking to setup a website for a group, you are first and foremost a user. This means that our getting started guide for users is entirely applicable (and useful) to you. The main difference is that your public_html is now a shared between all the admins of your group.
In your home directory, a shortcut (symbolic link) to your group’s file space is automatically created, and in it is the public_html you will want. This is also a shortcut to your group’s public file space, which contains the actual public_html directory. Here is a useful breakdown that also includes the absolute file paths:
spqr2@pip:~$ ls -l
mysociety1 -> /societies/mysociety1
mysociety2 -> /societies/mysociety2
public_html -> /public/home/spqr2/public_html
spqr2@pip:~$ cd mysociety1
spqr2@pip:~$ pwd -P
/societies/mysociety1
spqr2@pip:~$ ls -l
public_html -> /public/societies/mysociety1
Your options
The fact that you are given full access to our webserver means that you have many many different possibilities in the types of websites you can host: dynamic or static, custom made vs. a standard content management system (CMS), PHP vs. Python, the list of options goes on.
There are different usecases, and different levels of technical skill required by each, so choose the one the makes the most sense for you and your group. A good chunk of societies tend to favor the PHP-powered WordPress CMS, which is why we’ve written a special guide for it to make sure you install it properly and securely.
Getting Started: Users
Introduction
If you are a new user and wondering what to do, this guide is your first port of call. Here, we go through the process of getting familiar with the main services the SRCF offers you and how to work with them. It is worth mentioning that we have various tutorials that break down the steps shown here into much more detail, with a beginner-first approach in mind.
Create an account
To start with, you’ll need to create a personal
account. This will allow you to use
the control panel to manage our various services.
Connect to the shell server
The primary way to connect to our public facing servers is via a protocol known as SSH. Make sure to read our tutorial on that, and come back once you’ve finished!
Upload your first website
Now, let’s create your first website on the SRCF. Remember the public_html directory you saw in your home directory when you first logged in? Any file in that directory is served by sinkhole, our webserver, on the domain <yourCRSid>.user.srcf.net. By default, webservers look for an index.html page to serve, otherwise they’ll list the files in your public_html directory, which isn’t very user friendly.
As indicated by the name, you should not place any private files, for example files that contain secrets, in the public_html directory. They will be publicly accessible under your default SRCF domain.
First, ensure that you are now in the public_html directory by changing into it: cd public_html. To create a simple file, we use the text editor “nano”, by typing the command followed by the name of the file we want to create: nano index.html. You are now freely able to type any text you wish, so let’s add the following HTML by copying and pasting into your terminal.
If you are not familiar with HTML and CSS and how websites work, then don’t fret. There are plenty of very good resources on the web for this. Simply typing into a search engine should help you go a long way! The content below is sourced
this page.
<!DOCTYPE html>
<html lang="en">
<meta charset="UTF-8">
<title>I love the SRCF</title>
<meta name="viewport" content="width=device-width,initial-scale=1">
<body>
<h1>Hello there!</h1>
</body>
</html>
Now press “ctrl-o” then enter to confirm the file name, followed by “ctrl-x” to exit the editor. You should now be able to see a “Hello there!” on your SRCF domain.
Now what?
Believe it or not, the basic skills shown here cover most of what you should need to make use of our shell server and our webserver. You are welcome to browse our other tutorials for more advanced information, or simply play around with the Linux environment we offer you!
Reference
This is the technical reference for the SRCF. It contains essential information on our services.
The SRCF is run by volunteers who help to maintain its services for
hundreds of societies. Our services are free and therefore we rely on
donations. If you can, please
consider
donating.
Before using any of our services, you should consult our rules and policies on the left.
Administrators
Each group account has a list of admins: SRCF users who manage the
account and its services. This includes access to the account in the
control panel, permissions to edit files inside
/societies/<groupname>, and receiving administrative email sent to
<groupname>-admins@srcf.net. It may also be used by an account’s
website to limit page access to current admins.
This list is the canonical definition of who is responsible for a group
account, and is independent of any committees or roles in a University
club or society that a group account on the SRCF may happen to
represent.
Handover
We recommend that you keep your own handover documentation, covering
what services and software a group account is using.
It is the responsibility of SRCF group account admins to keep their list
of admins up-to-date, meaning both the appointment of their successors,
and the removal of predecessors that should no longer have access.
SRCF system administrators will not process requests from users wishing
to gain access to a group account, instead directing them to the
account’s current admins to make their request.
Unmaintained group accounts
In order to maintain continuity of group accounts, we recommend keeping
at least two admins at all times.
When a group account admin leaves the University without taking steps to
retain access to their SRCF account (which includes emails to their
membership address bouncing, or otherwise not responding to emails from
the system administrators), they may be dropped from the list of
admins, and their account suspended.
A group with no admins is considered unmaintained, at which point some
services (such as website hosting) will be suspended. If the group
account is configured with a role email address, it may be contacted as
a last resort, to inform the recipients of the situation and for them to
nominate a stand-in admin and reactivate the site.
The SRCF has the responsibility as a data processor to prevent
unauthorised access to personal data. If we’re unable to verify the
handover of an unmaintained group account (for example, if a group
account has no contactable admins and no role email to fall back to), we
will not be able to provide access to its contents. As a user seeking
access to such an account, generally this means you will have to
register a new group account and start from scratch.
Administrators
Society accounts have been renamed to group accounts. This page has been
moved.
After graduation
SRCF membership is for life, but you can also retain access to our
services provided we have a valid contact address for you.
If you’re leaving the university and losing your @cam.ac.uk email
address, make sure to update the email address we have on record for you
to something external. You can login to the SRCF control panel and
change your registered email address on your member page. If you use
mail forwarding using a .forward file or via Hades, check that your mail
is routed to a working address too.
The system administrators need to be able to contact you about your
account, and may suspend accounts without warning if they are unable to
reach you.
Apache (web server)
Overview
We run a central Apache 2.4 for all personal and group websites.
As our server handles hundreds of websites, we don’t allow users to
change server-level configuration, including Apache configuration in
/etc/apache2, as various online tutorials may lead you to.
Unlike most hosting providers, websites are run using your own (or a
group’s) user account on the server. This means your site has the
permissions needed to write its own files, for example with content
editors or automatic updates.
Directory root
Websites are served out of public_html directories, present in your
public file space (/public/home/<crsid>/public_html or
/public/societies/<groupname>/public_html).
To use Apache directives, you should use a .htaccess file, which
provides a subset of functionality at a per-site level.
Logs
Web server logs (including PHP errors) can be found in
/var/log/apache2/user/<crsid> for personal users, and
/var/log/apache2/soc/<groupname> for group accounts. You can access
them over SSH or with a file transfer program.
CGI and PHP scripts
Overview
On the SRCF’s webserver, you can run CGI and PHP scripts! What’s more,
we have a brilliant system in operation which allows even your PHP
scripts to run as ‘you’ rather than as the webserver. Read on for
details.
Location of your CGI files
For personal users, PHP scripts must have a filename ending with
‘.php’ and can be placed anywhere in your public_html directory. CGI
scripts must have filenames ending ‘.cgi’ and can also go anywhere in
public_html.
For group accounts, both of the above methods also work, however
there is an additional method of running CGI scripts which is to place
them in the cgi-bin directory in your group account user area (i.e. not
in public_html). If you do this, the script names do not need to end
‘.cgi’. The URL to access CGI scripts in the cgi-bin directory is
https://<groupname>.soc.srcf.net/cgi-bin, where <groupname> is the
group account name.
Users
For personal users, both PHP and CGI scripts will run as you, not as
the web server’s user id.
For group accounts, rather than running CGI/PHP scripts as an
individual, we have introduced a UNIX user for each group account. This
user cannot log in, however it is used as the user under which all group
account PHP/CGI scripts are run. Outgoing email generated by group
account CGI/PHP scripts will, by default, appear to come from
<groupname>-webmaster@srcf.net. See Group email
addresses.
Note that any CGI/PHP script which hogs the CPU for more than 2 minutes
will be terminated by the system. This should not affect anything other
than out-of-control scripts, as CGI/PHP scripts typically execute in a
few seconds. It is a measure to prevent the server being excessively
slowed down by buggy scripts which go into an infinite loop.
Permissions
For personal users, CGI scripts must be readable and executable by you,
and must be owned by you. PHP scripts must be world-readable (but to
keep database passwords secret, see the next question). For example:
spqr2@pip$ ls -l
-rwx------ 1 saw27 saw27 238 May 5 19:33 env.cgi
-rw-r--r-- 1 saw27 saw27 265 May 13 19:34 phptest.php
CGI scripts and PHP scripts belonging to a group account must have their
system (UNIX) group owner set to that of the group account. CGI scripts
must be system group readable and executable. Group account PHP scripts
must additionally be world readable (but to keep database passwords
secret, see the next question). For example:
spqr2@pip$ ls -l
-rwxrwx--- 1 saw27 casi 238 May 7 23:49 env.cgi
-rw-rw-r-- 1 saw27 casi 265 May 14 23:06 phptest.php
We recommend that you ensure that group account files are
group-writable, so that other admins of the group account can edit them
(not least, when your own personal account expires).
Managing secrets
PHP scripts must be world-readable. This requirement is artificially
imposed because we felt that if we didn’t require world readability,
users might be caught out by assuming that if something (other than a
CGI script) is not world readable then it’s not accessible on the web,
which wouldn’t be the case for PHP scripts. But it’s easy to get
round: put your secret information in a separate file which is not world
readable (but is system group readable), and include that file from your
main PHP script.
Managing memory
By default we have a maximum memory limit of 64MB set for PHP scripts.
You can override this by placing a file called php_override.ini at the
top level of your site containing, for example, memory_limit = 128M.
Please consider the memory requirement of other users, system processes,
etc. before doing this.
Turn CGI off
Several kinds of file will automatically be interpreted as CGI scripts,
and so the CGI handler will try to run them when you visit their URL,
even if you just wanted to download them. The following will turn off
CGI handling for Python scripts, displaying them as plain text instead:
AddHandler default-handler .py
AddType text/plain .py
Put those lines in a .htaccess file in the same directory as your
python files, and they will no longer be considered CGI scripts. You can
do a similar thing for other file types by changing the .py to, for
example, .php.
Common requests
This is a list of all queries and issues we receive often. Please check we haven’t already answered your question here before contacting the sysadmins.
The SRCF hosts user websites and mailing lists on its systems, and we
provide domains of the form CRSid.user.srcf.net
or acronym.soc.srcf.net, along with matching
email addresses like CRSid@srcf.net and
acronym-role@srcf.net.
The SRCF itself is not immediately responsible for content under these
domains. In the first instance, you should direct queries to the
owners of the relevant account.
-
For websites like spqr2.user.srcf.net
or email addresses like spqr2@srcf.net,
the responsible individual can be reached directly via
spqr2@srcf.net.
-
For websites like sample.soc.srcf.net
or email addresses like
sample-role@srcf.net,
the responsible group of people who manage the account (its “admins”)
can be reached collectively via
sample-webmaster@srcf.net for queries about
its website specifically, or
sample-admins@srcf.net for anything else.
Make sure to substitute the example owning account (spqr2 or
sample above) with the matching account name that you’re looking to
contact.
If your communication relates to infringing copyrighted material or
other abuse, please also CC the SRCF system
administrators.
We also host websites on custom domains, which do not end in
.user.srcf.net or .soc.srcf.net. Please contact the system
administrators if you’re getting in
touch about a website hosted by us under a custom domain and are
unsure how to reach the site’s owners.
Gain access to an existing group account
Read more about group admins and unmaintained
accounts
As a prospective admin seeking access to manage a group account, you
should first contact its existing admins and request access from them.
See the answer above for how to identify the short name of the account.
If you don’t know the short name, you can check this list of group
accounts.
If you’re unable to make contact with them, you may ask the system
administrators for assistance, who will
then attempt their own contact, and may remove inactive admins or
transfer ownership as needed.
Group accounts are considered unmaintained if they have no active
admins remaining. Accounts in this state will display an Unmaintained
group website page if you visit their SRCF-hosted website.
Grant group admin access to someone else
As an existing admin of a group account, you can use the control
panel to promote other users. Each new admin
will need to create a personal SRCF account first if they don’t
currently have one.
If you are stepping down from managing a group account, you can remove
yourself as long as there’s at least one additional admin present.
Permission errors writing files in a group account space
Read more about group permissions
You can try to fix permissions yourself using the srcf-soc-permfix
tool over SSH, which can fix your own files and those of the group
account:
srcf-soc-permfix <groupname>
sudo -u <groupname> srcf-soc-permfix <groupname>
However, this won’t be able to fix files owned by other admins. If you
continue to get permission errors, ask the system
administrators to run this for you
instead.
Problem with an @lists.cam.ac.uk mailing list
Whoa, stop right there! That’s the University Information Services
mailing list system, not the SRCF one, and there’s nothing we can do
to help you. Try going to
lists.cam.ac.uk instead.
Update forwarding addresses on a domain
The SRCF does not (and cannot) provide email forwarding outside of
@srcf.net addresses, so we don’t have control over these. You’ll
most likely need to consult your domain registrar to do this.
Can you design/upload/run/fix my website?
Unfortunately, our support team does not have the capacity to maintain
both our servers and the thousands of websites hosted on it. If you have
a specific technical question (e.g. an error message you don’t
understand), be sure to seek help.
You may upload a website designed elsewhere to be hosted on the SRCF.
However, it must be you who performs the upload – you must not share
access to your SRCF account with anyone else, even if they are acting on
your behalf.
We recommend societies and groups have appointed webmasters that are in charge of maintaining the website.
Can you help me debug a program?
Only if we have time - our support team are all volunteers, however we
generally like hacking code, so if you’re having problems drop us a line and we might be able to help you out.
Custom domains
Overview
As part of our web hosting, we provide free domains for both individuals
(of the form <crsid>.user.srcf.net) and group accounts
(<groupname>.soc.srcf.net). These are configured as standard, and will
serve any web content placed in the public_html directory of the
respective personal or group account (subject to a 20-minute delay when
publishing a new website).
We also support external domains purchased from a domain registrar. If
you are looking to buy a domain for yourself or your society then we can
recommend Mythic
Beasts but there are plenty of other registrars out there with varied pricing.
You’ll need to make sure the domain resolves to us
– this is controlled by DNS records, which your registrar should allow
you to configure. If you’ve delegated DNS from your registrar to a
third-party service such as Cloudflare, you’ll need to configure it
there.
The base domain (e.g. example.com) is generally represented by @,
and subdomains (e.g. bubbles.example.com) would just be bubbles.
Assuming you want to serve your site from the base domain example.com,
your DNS records should look something like the following:
Subdomain Type Value
----------- ------- ------------------------------------
@ A 131.111.179.82
@ AAAA 2001:630:212:700:2::1
www CNAME webserver.srcf.societies.cam.ac.uk
The first record (A) makes your base domain point to our webserver’s
IPv4 address; the second (AAAA) does the same but for IPv6. The third
(CNAME) sets an alias from www.example.com to our webserver.
If instead you want to use the subdomain bubbles.example.com:
Subdomain Type Value
------------- ------- ------------------------------------
bubbles CNAME webserver.srcf.societies.cam.ac.uk
www.bubbles CNAME webserver.srcf.societies.cam.ac.uk
The preferred method is a CNAME record, which acts as an alias from
your domain to ours. The value of this record should be
webserver.srcf.societies.cam.ac.uk – this means your domain will
continue to work even if we move servers, as our hostname will stay the
same. Note that you must remove any other non-CNAME records for this
particular (sub)domain.
CNAMEs should not be used on base domains (example.com, sometimes
called the zone apex), only on subdomains. The alternative is to use
A (for IPv4) and AAAA (IPv6) records to specify an IP address.
The A record should be set to 131.111.179.82, whilst AAAA should be
2001:630:212:700:2::1. Make sure to remove all other A/AAAA records
for that domain, otherwise browsers and other software may arbitrarily
choose between our web server and whatever else is configured.
You can confirm if the domain is resolving to us correctly by visiting
it in your browser – you should be greeted with a ‘non-existent
site’ page on our site. Once this is visible to you, use the SRCF
control panel to assign the domain to your (group) account.
DNS takes time to propagate. Each record has an associated time-to-live
(TTL) – this is how long the record is valid for, so a long TTL means
browsers and computers will cache the resulting location for a longer
period. If you had a previous DNS record with a TTL of 1 day, you may
need to wait that long before the domain starts resolving correctly. For
testing purposes, it may be helpful to set your TTLs to the lowest value
that your DNS provider permits so that changes to preexisting records
propagate as quickly as possible.
Document roots and multiple websites
You may wish to run more than one website under a single account. This
can be done if you have your own domain, by delegating different
(sub)domains to different sites.
<crsid>.user.srcf.net and <groupname>.soc.srcf.net domains are fixed
at serving the root of your public_html directory. When adding custom
domains to your account, you can optionally set a Document root –
this is a subdirectory of public_html which acts as the root of your
domain.
For example, you may wish to serve example.com from
/societies/<groupname>/public_html, and bubbles.example.com from
/societies/<groupname>/public_html/bubbles. This means that
/societies/<groupname>/public_html/bubbles/index.html would be
served as the index page of bubbles.example.com.
HTTPS
HTTPS provides secure communication between our servers and your website
visitors, and is recommended for any site that handles personal
information or provides password authentication. In comparison, HTTP
transmits in plain text, and is susceptible to eavesdropping over an
untrusted network (such as public wi-fi or other shared networks).
Websites served from .user.srcf.net and .soc.srcf.net addresses come
with working HTTPS by default, but custom domains must be opted-in
manually. This can be done using the Let’s Encrypt
form. Requests will be
queued, and processed together at the end of each day.
Note that your domains need to be actively resolving to us in order to
have certificates issued – if not, they will be dropped from the
queue. The resulting certificate we’ll serve will list each of your
domains as a Subject Alternative Name (SAN). Once opted in, no further
action is required – certificates will automatically be renewed close
to their expiry.
Custom domains are shown with
www and
non-www variants. We serve
your site on both of these domains, even if only one of the variants
actually resolves to us. If
via wildcard is shown, we are actually
accepting requests for any subdomain of the listed domain, though we are
currently unable to issue wildcard certificates due to limitations with
the issuer. Please
contact the sysadmins
if you wish to request certificates for specific subdomains other than
www.
Websites that are not opted-in will not be served over HTTPS, so will
only respond to plain HTTP. Requests for an HTTPS version will show a
‘non-existent site’ page.
Note that we only handle serving your website over HTTPS, and will still
accept HTTP connections. If your site uses a CMS that stores a canonical
domain, you’ll need to update that to use HTTPS if desired. You can
also use a .htaccess file to require secure connections, and redirect
plain HTTP.
What about email?
The SRCF cannot process mail for your domain – we are prevented from
doing so by the university. This means you must not point MX records
at SRCF machines, as any mail sent there will be dropped before it
enters the CUDN.
You should check what services your domain registrar provides, which may
include forwarding addresses (so that mail sent to bubbles@example.com
is redirected to a mailbox elsewhere) or real mailboxes (so that you
connect to your registrar’s mail server to read mail).
Email accounts
Overview
Every member of the SRCF has an email address on the srcf.net domain.
If your SRCF username is spqr2, your SRCF email address is
spqr2@srcf.net.
In fact, you have as many email addresses as you want, by appending a
suffix of your choosing: spqr2-<something>@srcf.net. You can use
this for filtering different types of email in different ways. You also
have equivalent addresses in our old domain, srcf.ucam.org.
Incoming mail
You can choose via the control
panel one of three possible ways for
delivering email sent to your SRCF email addresses.
Simple forwarding
All of your email is forwarded to another address. For this we use the
contact address you provided when you signed up, which you can also
change on the control panel.
SRCF Hades email service
A modern email service. If you want to read your email on the SRCF,
rather than forwarding it to another email service, this is what we
recommend.
.forward file and/or legacy mailbox
This is a primitive and somewhat unfriendly system for hosting a mailbox
or forwarding mail, but does offer some programmability via the SRCF
shell server; by writing an Exim filter file you can, for example, pass
email messages to a custom program. This is recommended only for
advanced users and people who already have a mailbox on pip which
predates Hades.
Outgoing mail
You can send email from your personal SRCF email account over SMTP. This service is part of Hades, but is
available whether your email is hosted on Hades or not.
Files and permissions
Overview
Each group account has a corresponding system (UNIX) group that manages
access permissions for files and directories in the group account space.
Both /societies/<groupname> and /public/societies/<groupname> are
writable by the system group. Each admin of the account is then added to
that system group, which grants the admin permissions to manage files.
When creating new files or directories, you should ensure the system
group permissions are correct – for each file the system group should
match the group account name (and not your personal system group), and
permissions should include group-write. Directories should also be
group-sticky, so that subdirectories will inherit the correct
permissions. Example output from ls -l:
drwxrwsr-x 2 <crsid> <groupname> 4.0K Jan 1 2020 directory
-rw-rw-r-- 2 <crsid> <groupname> 0 Jan 1 2020 file
The important fields to note here are rw appearing in both of the
first two sets of permissions (user and group), and s in the directory
group column for sticky.
Ownership
Files in a group account space will generally be owned by the admin that
created them, but with the file’s group set to match the corresponding
group account, any admin can modify or delete these files as needed.
Some files may be owned by a user named after the group. This is an
internal system user: websites and group account services are run as
this user, meaning any files they create will have this ownership.
Again, ensuring group permissions are set correctly will mean that any
admin can manage such files.
Fixing bad permissions
You can run srcf-soc-permfix <groupname> over SSH, which will add any
missing group permissions to files owned by the user that run it. This
means you can fix files that you own, as well as files created by the
internal group account’s user with
sudo -u <groupname> srcf-soc-permfix <groupname>. If you encounter
files owned by another admin with incorrect permissions, they will need
to run this command themselves in order to fix them. In the case of
ex-admins, you can contact the SRCF
sysadmins to fix permissions for you.
Files and transfer
Basics
Your home directory on SRCF servers is /home/<crsid> and is private to
you. Group account files can be found at /societies/<groupname> and
can only be accessed by group account admins.
All accounts also come with a ‘public’ directory (these are located at
/public/home/<crsid> and /public/societies/<groupname>
respectively). This includes your website root public_html, but
you’re free to use them as you wish.
You can use SFTP or SCP to transfer files between your SRCF account and
your personal computer. Mac and Linux come with scp for per-file
transfer; you may prefer a graphical client – options include WinSCP
on Windows, or Cyberduck for Mac/Windows.
Quotas
We provide users with 2GB of space for their personal files, and each
group account also comes with 2GB of its own space. If you run out, you
may request additional quota from the sysadmins. For larger increases,
you may be asked to justify the need, and the SRCF committee may need to
approve it.
Consider searching for cache files in your home directory that may be
taking up unnecessary space, or compressing large files (e.g.
high-resolution photos on websites).
You can use the srcf-quota command over SSH to check your current
quota and usage. You’ll also receive email notifications when you
approach your quota. Once the hard limit is reached, you’ll no longer
be able to create or append to files.
Ownership and permissions
Each file or directory stored on the SRCF have two ownership fields
associated with it: an owner (a system user) and a group (a system
group; this is distinct from SRCF group accounts). Every user has an
associated personal group (e.g. the user spqr2 also has a
corresponding group spqr2), whilst admins of a group account will be
members of an additional system group.
By default, your home directory /home/<crsid> is private to you, and
group account home directories like /societies/<groupname> are private
to their current admins. In contrast, the public directories
/public/home/<crsid> and /public/societies/<groupname> are
world-readable, meaning their contents are visible to any SRCF user.
Because websites are served from here, you should ensure any files
containing sensitive information (such as credentials for databases) are
individually set to not be world-readable.
See also group permissions.
World-writable files
World writable files are files that anybody on the system can write to
(edit). Whilst in general you can trust other SRCF users not to modify
your files, there are several reasons why world-writable files are a
problem:
- People make mistakes. For instance, if you have a world-writable
directory, and somebody runs
rm -r / by mistake (this has happened
at least once) then all the files in that directory will be deleted.
- Users’ accounts may have been compromised. We have had one incident
where a worm entered the computer via an insecure society website
and proceeded to overwrite every world-writable file on the
computer.
- World writable files make things easier for attackers. If for
instance a directory underneath your public_html directory is
world-writable then an attacker able to write files on the system
could place a script there containing commands that he could execute
as you.
To avoid problems like this you should avoid creating world-writable
files and directories, and if you have created them then you change them
to be non-world-writable. You can do this using the chmod command –
chmod o-w filename will remove world-writable permissions from a file
and chmod -R o-w ~ will do the same for all world-writable files in
your home directory.
Some CGI scripts will tell you that they need to have world-writable
files / directories to work. This is almost certainly not the case on
the SRCF system where CGI / PHP scripts run as the user that owns them
rather than the webserver. For society accounts it is often necessary to
make the files group-writable rather than world-writable (presuming that
the intended effect is to allow multiple members of the society to write
to them). If you can’t get a script to work without world-writable
files / directories then get in touch
with the support team and we’ll see what we can do to help.
Snapshots
Although the SRCF does not guarantee to take backups of users’ data,
snapshots of /home, /public and /societies are generally taken
regularly for disaster recovery purposes. Snapshots may be accessed
using the hidden (it will not show up in ls -a or in shell autocomplete)
directory .snapshot, available at any level of the file hierarchy. For
example:
spqr2@pip:~$ ls -lut .snapshot
total 336
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 18:00 sv_hourly.0
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 17:00 sv_hourly.1
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 16:00 sv_hourly.2
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 15:00 sv_hourly.3
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 14:00 sv_hourly.4
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 13:00 sv_hourly.5
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 12:00 sv_hourly.6
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 11:00 sv_hourly.7
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 10:00 sv_hourly.8
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 09:00 sv_hourly.9
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 08:00 sv_hourly.10
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 07:00 sv_hourly.11
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 06:00 sv_hourly.12
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 05:00 sv_hourly.13
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 04:00 sv_hourly.14
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 03:00 sv_hourly.15
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 02:00 sv_hourly.16
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 01:00 sv_hourly.17
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 01:00 sv_daily.0
drwxr-x--- 2 spqr2 spqr2 8192 Jul 6 00:00 sv_hourly.18
drwxr-x--- 2 spqr2 spqr2 8192 Jul 5 23:00 sv_hourly.19
drwxr-x--- 2 spqr2 spqr2 8192 Jul 5 22:00 sv_hourly.20
drwxr-x--- 2 spqr2 spqr2 8192 Jul 5 21:00 sv_hourly.21
drwxr-x--- 2 spqr2 spqr2 8192 Jul 5 20:00 sv_hourly.22
drwxr-x--- 2 spqr2 spqr2 8192 Jul 5 19:00 sv_hourly.23
drwxr-x--- 2 spqr2 spqr2 8192 Jul 5 01:00 sv_daily.1
drwxr-x--- 2 spqr2 spqr2 8192 Jul 4 01:00 sv_weekly.0
drwxr-x--- 2 spqr2 spqr2 8192 Jul 4 01:00 sv_daily.2
drwxr-x--- 2 spqr2 spqr2 8192 Jul 3 01:00 sv_daily.3
drwxr-x--- 2 spqr2 spqr2 8192 Jul 2 01:00 sv_daily.4
drwxr-x--- 2 spqr2 spqr2 8192 Jul 1 01:00 sv_daily.5
drwxr-x--- 2 spqr2 spqr2 8192 Jun 2 01:00 sv_daily.6
drwxr-x--- 2 spqr2 spqr2 8192 Jun 29 01:00 sv_daily.7
drwxr-x--- 2 spqr2 spqr2 8192 Jun 28 01:00 sv_daily.8
drwxr-x--- 2 spqr2 spqr2 8192 Jun 27 01:00 sv_weekly.1
drwxr-x--- 2 spqr2 spqr2 8192 Jun 27 01:00 sv_daily.9
drwxr-x--- 2 spqr2 spqr2 8192 Jun 26 01:00 sv_daily.10
drwxr-x--- 2 spqr2 spqr2 8192 Jun 25 01:00 sv_daily.11
drwxr-x--- 2 spqr2 spqr2 8192 Jun 24 01:00 sv_daily.12
drwxr-x--- 2 spqr2 spqr2 8192 Jun 23 01:00 sv_daily.13
drwxr-x--- 2 spqr2 spqr2 8192 Jun 20 01:00 sv_weekly.2
drwxr-x--- 2 spqr2 spqr2 8192 Jun 13 01:00 sv_weekly.3
Note that snapshots are named sv_[type].[index], with index 0
indicating the most recent snapshot of that type. The listing above
shows 24 hourly, 14 daily and 4 weekly snapshots; you may see fewer or
more than this.
Snapshots preserve file permissions and are read-only, so if you wish to
retrieve something from a snapshot, you must have had permission to
access it at the time the snapshot was taken, and must copy (rather than
move) it out of the snapshot.
Snapshots going further back in time may be available on an off-site
disaster-recovery replica; if you need access to these, contact the
sysadmins (but please don’t count on them being available; you should
take your own backups regardless).
Game servers
Overview
If you want to run a server for your favourite online game (or any other
long-running application) you should use doom.srcf.net, as described
here.
Please limit your server to around 1GB of RAM and set it to nice level
19 to avoid causing problems for other users. For example, if you are
running a Minecraft server, then the following should work:
nice -n 19 java -Xms1024m -Xmx1024m -jar minecraft_server.jar
It is possible (and probable, in the case of Minecraft) that the default
port for your application will be in use. We don’t have any specific
rules for picking a port for your application, so just pick
high-numbered ports at random until you find one that isn’t in use.
Running at startup
If you want your server application to run at system startup, you can
create a cron job for it as described
here.
Gopher
Overview
Gopher is a protocol for distributing, searching and retrieving
documents over the Internet as an alternative to the Hypertext Transfer
Protocol (HTTP). In contrast to the World Wide Web, Gopherspace is
navigated using a menu-based interface.
Your normal web browser likely won’t be able to access Gopher sites so
you will need to install some other software or browser plugin first.
Once you do this you should be able to find the SRCF’s presence at
gopher://gopher.srcf.net/.
Publishing your Gopher site
To get started with a Gopher site you’ll want to create a subdirectory
in your public home directory called public_gopher:
> /public/home/spqr2/public_gopher # For individual users
> /public/societies/foosoc/public_gopher # For groups and societies
After a short delay your site will be available via Gopher at:
> gopher://gopher.srcf.net/1/users/spqr2
> gopher://gopher.srcf.net/1/societies/foosoc
Customising your directory listing
Gopher sites consist of a directory listing of files and a description
(or abstract) for each file. To attach an abstract to a file, say
myfile.txt, create another file called myfile.txt.abstract with a
line or two of descriptive text. This will be displayed with any links
to that file. To attach an abstract to a directory, create a .abstract
file inside that directory.
The directory listing of a Gopher site can be customised using a
gophermap file in the directory in question. For full details on how
to do this take a look at the /var/gopher/gophermap-info.txt file on
pip.
Group account email
Overview
Group accounts, like personal accounts, can receive email to either
their base name or a suffixed name. This can be useful for setting up
‘role’ addresses for the group account, for example to pass mail sent
to <groupname>-president@srcf.net.
Default role addresses
All group accounts come with the role address
<groupname>-admins@srcf.net, which can’t be overwritten. Mail
sent here will be distributed to the individual SRCF mail accounts of
all the users who are admins of the group account.
By default, we also define <groupname>-webmaster@srcf.net as an
alias to <groupname>-admins@srcf.net. This will be used as the
default sender address for email sent by the group account, and may be
displayed in some error messages to visitors, but you can change where
mail ends up as described below.
Customising mail handling
You can set up a .forward file in your group account’s directory at
/societies/<groupname>/.forward. This will be used to process all
email sent to addresses of the form <groupname>@srcf.net or
<groupname>-<anything>@srcf.net, with the exception of
<groupname>-admins@srcf.net which is always forwarded to admins.
See pip forwarding for more information.
Unlike personal accounts, mail sent to <groupname>@srcf.net will
be bounced by default – no default mail handling is configured for
group accounts.
Hades email service
Overview
SRCF Hades is a modern, standards-compliant email service, introduced in
October 2018 for two reasons:
For @srcf.net addresses
Use the control panel to switch to
Hades: from your member page, select Change email configuration under
Email. When you choose to switch your email to Hades, the control
panel will explain the consequences which depend upon your current
configuration. You will have a chance to cancel the migration after
seeing this information.
This applies equally to our old domain @srcf.ucam.org.
For @cam.ac.uk addresses
If you miss Hermes, you can set your @cam.ac.uk email address to
deliver mail into your SRCF Hades inbox, thus making Hades the service
you use to read your University email.
You need to first enable Hades for your @srcf.net address as described
above. It is very important that you make sure that your SRCF email
does not forward to your @cam.ac.uk address before you proceed!
Go to your page on the University Lookup
service. Edit it, and change your
‘@cam delivery address’ to your SRCF email address (for example
spqr2@srcf.net). If you have a University Exchange Online account then
by default email sent by other Exchange Online users to your @cam.ac.uk
address will bypass the setting in University Lookup and be delivered to
you Exchange Online inbox regardless. To setup forwarding in ExOL, click
on the settings cog icon and search for ‘Forwarding’.
Reading email
Webmail
The easy way to read your email:
webmail.hades.srcf.net
For apps: IMAP
To configure any standards-compliant mail app to read your email, you
will need the following details:
- Server type: IMAP
- IMAP server: hades.srcf.net
- Security: STARTTLS (or SSL/TLS)
- Username: your SRCF username, e.g. spqr2 (not your email
address)
- Password: your SRCF password (not your Raven password)
Some email apps such as Thunderbird will automatically configure this
for you after you fill in your SRCF email address — provided that you
have already enabled Hades email via the control
panel.
If you’re looking for an app to get your Hades email on an Android
phone, we suggest K-9 Mail.
Sending email
Outside of webmail, you can use SMTP to send emails from your
@srcf.net address. Add these details to your client of choice:
- SMTP server: smtp.srcf.net
- Security: STARTTLS
- Port: 587
- Username: your SRCF username, e.g. spqr2 (not your email
address)
- Password: your SRCF password (not your Raven password)
Mail forwarding
You have two options:
- Bypassing Hades: On the control
panel you can configure your
@srcf.net email address to forward mail to another address. This
is the simplest option.
- Using Hades filters: You can add a filtering rule (see below)
which forwards all your email elsewhere, with or without keeping a
copy in your Hades mailbox.
Mail filtering
- Log into Hades Webmail
- Click ‘Settings’
- Click ‘Filters’
- Add a filter by clicking the ‘+’ button at the bottom of the
‘Filters’ panel
- Choose a name for the filter, a set of conditions to match (e.g.
‘all messages’) and an action (e.g. ‘Move message to folder’)
- Click ‘Save’
You can add as many filters as you want, and drag them around to change
the order in which they are run. You can optionally have multiple filter
sets, although only one of those can be active at once (for example you
could have an alternate filter set, normally disabled, to handle your
email when you are on holiday).
Alternatively, you can write your own Sieve script
and upload it as a filter set: click the ‘+’ button at the bottom of
the ‘Filter sets’ panel. (Note that Sieve filters use a different
syntax to the .forward Exim filters available on pip, although the
capabilities are similar.)
Backing up email
The SRCF does not guarantee to have backups of user data. You must
take your own backups.
You might find an IMAP synchronisation tool such as isync (a.k.a.
mbsync) useful. The Arch Wiki has a
handy guide to setting up
isync.
At times we might have disaster-recovery backups which include a
snapshot of your mailbox. If you’ve lost data and your own backups have
failed, try asking the sysadmins if we can help — but we might not be
able to.
Internet Relay Chat (IRC)
Overview
Internet relay chat is a protocol for real-time text messaging between
internet-connected computers. While it is dated in comparison to some
other messaging protocols, it continues to be used regularly by SRCF
members and as the emergency messaging platform of choice, with several
servers located around the world.
We have written a tutorial on getting started with IRC,
available here.
Much like you will find in other open source and similar communities, we
have a few guidelines that we want to make clear:
- There are no technical prerequisites
- Anyone can contribute
- By students for students
We have a code of conduct to foster the above aims: inclusivity,
collaboration and teamwork (https://www.srcf.net/code-of-conduct)
Channels
- #welcome — a place for newcomers to say hello
- #general — chat about anything here, we’re humans after all :)
- #society — questions and topics about the SRCF as a society
- #support — tech questions, SRCF-specific or not
- #hackday — home of volunteers, future home of interested
volunteers
- #ops — infra-specific discussion, invite only (interested
sysadmins encouraged to email sysadmins@srcf.net to join!)
- #srcf — alumni
Server details
You need to connect your client to
irc.srcf.net. If you have a
suitably-configured IRC client then clicking on that link might work, or
if you want to try out IRC without downloading and installing a client
then the SRCF provides an IRC web interface.
Setting up a persistent connection
Some people like to leave an IRC client connected to the network even
when they are away from their computer, or have turned it off, so that
people can continue to leave them messages for when they return. There
are two main ways: a bouncer or a tmux/screen instance.
For tmux/screen, a common choice is irssi,
although there are many others.
If you want to use a bouncer, we recommend ZNC.
Troubleshooting
If you are seeing ‘nick [~username@my-irc-client-is-misconfigured]’ then this is because
you are not connecting to irc.srcf.net (as indicated above) but instead
to some other hostname – perhaps kern, pip, shell or www. This is not
recommended, since if the IRC server moves at a later date those
addresses will no longer work, whereas irc.srcf.net will always be
updated to point at the new location.
Legacy mail on pip
This information applies only to users of the ‘legacy’ or ‘advanced’
SRCF mail server. It might apply to you if you had a SRCF account before
October 2018, or you deliberately chose to handle your mail this way.
To see which email system you are currently using, see the control
panel.
Reading mail
Legacy forwarding
You can forward incoming mail to some other address. For old accounts,
when your account was first set up it was configured to forward mail to
the address you gave us when you applied. See
mail forwarding below.
Local client
Read your mail using an email client running on the server, for example
mutt or alpine.
For apps: IMAP
To configure any standards-compliant mail app to read your email, you
will need the following details:
- Server type: IMAP
- IMAP server: pip.srcf.net
- Port: 993
- Security: SSL
- Username: your SRCF username, e.g. spqr2 (not your email
address)
- Password: your SRCF password (not your Raven password)
Note that we do not run a POP3 server, as it is an older protocol and
provides no additional functionality over IMAP.
Webmail
Use our pip webmail service:
webmail.pip.srcf.net
Hades email offers a significantly superior
webmail and IMAP experience for most users. You can switch to Hades on
the
control panel.
Sending mail
You cannot use pip itself as a SMTP server, except from applications
running on pip itself.
You can send email from your personal SRCF email account over
SMTP. This service is part of Hades, but is
available whether your email is hosted on Hades or not.
Mail forwarding
Create a file called .forward (note the leading full-stop) in your
home directory containing the email address to which you would like your
mail to be forwarded.
The SRCF offers a simple interactive tool to create this file for you.
Run srcf-MailForward over SSH, which will prompt you to enter an
e-mail address where you would like all your SRCF mail forwarded to.
Auto-forward
We also provide srcf-autoforward, a tool to generate role-like
addresses with a simpler syntax. You need to create a file called
autoforward in a group account’s directory (or your home directory,
for your own roles), where each line consists of a role name, a colon,
and a comma-separated list of target addresses:
<role>:<email>,<email>...
For example:
treasurer:spqr2@cam.ac.uk,treasurer@example.com
Once you’ve defined your roles, you need to build the actual .forward
file:
srcf-autoforward <groupname>
Mail filtering
We have the Exim mail transfer agent installed
– see the Exim filtering
documentation
for how to write filters in your .forward file.
Filters in Exim are more complicated than Hades’ Sieve filters, though there are
a few things supported which cannot be done on Hades, such as piping
your email into a custom program.
Here are a few examples of basic forwarding for a group account. Note
that the # Exim filter line is required.
# Exim filter
# Forward emails for <groupname>-webmaster@srcf.net to spqr2@cam.ac.uk:
if ($local_part_suffix is "-webmaster") then
deliver spqr2@cam.ac.uk
endif
# Forward <groupname>-treasurer@srcf.net to spqr and an external address:
if ($local_part_suffix is "-treasurer") then
deliver spqr2@cam.ac.uk
deliver treasurer@example.com
endif
# Forward anything not yet processed to a lists.cam mailing list:
if not delivered then
deliver group-example-committee@lists.cam.ac.uk
endif
Known quirks
pip’s mail service dates from a time long ago when most UNIX systems
stored email in Mbox format. Mbox
is a primitive file format which concatenates all emails in a mailbox
into one long file. It is slow and inefficient to use, and does not
support concurrent access (you can’t read your inbox whilst a message
is arriving, for example). Our IMAP server tries its best by using
caches and indices but it is fundamentally not an efficient data storage
system.
Furthermore, if you create folders (besides INBOX) in which to file your
emails, those are placed in your home directory along with your other
files. This leads to confusion, and also can lead to your entire home
directory being served over IMAP, which has been known to upset email
software (unsurprisingly).
And for added confusion, Mbox is incapable of storing any email containg
a line of text starting with the word ‘From’. Yes, really. (Mail
software has to rewrite it as >From as a workaround.)
We cannot easily convert pip to a more modern email storage format,
though, as we have some people with extensively curated mbox hierarchies
and we don’t want to break email for those people.
So (in September 2018) we created a brand new email service called
Hades which runs alongside the “legacy”
email service on pip. Hades was implemented with the benefit of another
20+ years of advancements in email technology.
We encourage you to take a look at Hades. If you’re used to the setup
on pip, you may find it quite different (better, we hope!), but if you
have a complex Exim filter it may be time consuming to redefine your
email filtering requirements in Sieve on Hades.
We have no immediate plans to discontinue pip’s mail service, although
eventually we hope to be able to shut down the IMAP server on pip as our
IMAP users migrate to Hades. We know that there are some things that
advanced users might want to do which are only possible on a traditional
UNIX mail system, and we intend to keep running the mail service on pip
for these advanced use cases.
Mailing lists
We run Mailman mailing lists for both personal
and group accounts. New lists can be created via the control panel.
The default list administrator for personal accounts is
<crsid>@srcf.net. For group accounts the default list administrator is
the group account’s admin address <groupname>-admins@srcf.net. To
avoid complicating group account handovers, we recommend you keep this
default for group-owned lists, and manage list admins by adding to the
group account admin list. You are free to use the list moderator role to
add additional users for approving postings and subscriptions.
To delete a mailing list, you’ll need to contact the
sysadmins. Unless asked otherwise, we
will retain the list archives (though the archives must be public for
them to remain accessible).
Mattermost
Overview
Mattermost is an open source messaging platform, similar to Slack. See
the official user
guide
for more about its service.
The SRCF’s mattermost instance can be found at
https://mattermost.srcf.net, and is the
Team Edition. This is open to use for any members of the SRCF as well as
their guests.
Policies and features
-
A user must sign up with an @srcf.net, @cam.ac.uk or
@*.cam.ac.uk email, unless they have an invitation from an
existing user. A user can change their email address to their
personal email after signing up.
-
Only team administrators are allowed to create and delete channels
and modify their descriptions. (The button to make these changes
will still be present in the user interface, but they will not
work)
-
Open teams (teams that do not require invitations to join) are not
available on our server, since open teams are advertised to all
users on the server. If you want to invite a large number of
people (e.g. all members of your Facebook group), you can share an
invitation link.
-
If you want to delete all messages in a channel older than n
days, you can use the following two-step procedure:
- Get the channel id by typing
/channel_id
- Run the command
/delete_posts <channel_id> <n> where
<channel_id> is the channel id you obtained from the
previous step, and <n> is the number of days.
This an only be run be a team administrator.
The SRCF privacy policy applies. This
section describes details pertinent to the Mattermost service.
Email
Your email is used to send notifications to you. The types of
notifications you receive can be configured in the web interface.
Data retention
By default, any person who joins your channels can see all past
messages. Messages that are deleted in the user interface are not
accessible by end users anymore, but remain in the database. Contact the
sysadmins if you wish to have
your data purged. It is also possible to permanently delete all posts by
a user.
User lookup
Your username and name are visible and searchable by other users in the
invitation menu. Your email remains private.
Clients
The SRCF is only responsible for the web client. When using other
clients (e.g. a mobile client), be sure to read their privacy policy.
MySQL and PostgreSQL databases
Overview
Both user and group accounts can make use of databases, either in MySQL
or PostgreSQL. We don’t create accounts for these by default – you
will need to request them separately via the control panel, which will
give you a distinct password for authentication (don’t use your
personal SRCF password).
You will receive a default database named after your account. In MySQL,
you can create additional databases of the form
<username>/<something>. For PostgreSQL, you can make use of schemas to
organise your tables if needed.
Connecting to your database
The canonical hostnames for database servers are mysql and postgres,
which are valid across all user-accessible servers.
To connect to MySQL from an SRCF shell:
mysql -h mysql -p <database>
Here, -h specifies the hostname, -p prompts for your password, and
you’re connected to the database specified at the end – either your
CRSid for your personal database, or a group database name.
Similarly for PostgreSQL:
psql -h postgres [database]
Providing a database is optional; by default, you’ll be connected to
the one matching your username. Ident authentication is enabled for
PostgreSQL, which will authenticate you based on the underlying system
account, so you usually don’t need to provide a password.
You can also use phpMyAdmin for
MySQL, and phpPgAdmin PostgreSQL, to
manage your databases in a browser.
Group account access
If logged in using your personal database account, you can also access
databases owned by group accounts you administrate.
When configuring software or websites in a group account, you must use
the group database account to access its databases – do not use your
personal credentials as that would give other group account admins
access to your personal account and all the databases (both personal and
group) to which you have access!
Passwords
When your personal account is first created, you’ll receive a randomly
generated password by email. You can change it by connecting over SSH
and running passwd.
This password is valid for SSH, file transfer and email. It’s not
valid for database access – both MySQL and PostgreSQL accounts must be
requested separately and come with their own password. It’s also
not valid for Mailman mailing lists, as each list comes with its own
owner and moderator passwords.
Group accounts do not come with passwords, and can not be logged in as
directly. If you need to manage files in a group account, connect using
your personal account and navigate to the group account’s files. You
should avoid storing your personal credentials in any group applications
or config files.
Permissions
Society accounts have been renamed to group accounts. This page has been
moved.
Personal vs. group accounts
All SRCF users, whether they are looking for web hosting for themselves,
or wish to administrate a group account, must first get a personal SRCF
account. It is completely fine if you apply for SRCF membership when all
you really want is to run the website of a society (or other group),
rather than your own.
Raven authentication
Overview
You can configure your site, or a subset of pages, to require user
authentication with Raven, the university’s
authentication system.
With .htaccess
The SRCF has the
mod_ucam_webauth
module installed, which makes it easy to do basic authentication using Raven.
Full documentation
is available, though here are a few common cases.
To protect a directory (whether public_html for your entire site, or a
subdirectory of it), create or edit a .htaccess file in that
directory, and add the following:
AuthType Ucam-WebAuth
Require valid-user
This will permit access to anyone with a ‘current’ Raven account, i.e.
active students and staff. To permit access to any Raven account
(including graduated students), add a Ptags directive:
Alternatively, you may want to limit access to Raven-authenticated users
or visitors within the cam.ac.uk domain:
Order allow,deny
Allow from .cam.ac.uk
AuthType Ucam-WebAuth
Require valid-user
Satisfy any
To limit page access to group account admins only, add a unix-group
Require directive:
Require unix-group <groupname>
You can also list specific users:
Require user <crsid> <crsid>...
To create a ‘logout’ link, add the following to your .htaccess file
(which will create /logout relative to the directory containing the
.htaccess file):
<FilesMatch "logout">
SetHandler AALogout
</FilesMatch>
You can access a Raven-authenticated user’s CRSid using the
REMOTE_USER (or AAPRINCIPAL) environment variables. For example,
adding the following to a PHP page like index.php will display a
customised welcome message on login:
<?php
echo "Hello {$_SERVER['REMOTE_USER']}!"
?>
Example configuration:
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=permanent]
AuthType Ucam-WebAuth
Require user CRSID
Replace CRSID with your CRSID.
The Rewrite section makes all connections to .../wp-admin/ use SSL
which will protect your password, the AuthType section uses Raven to
restrict access to the directory, you probably want to use your CRSID on
the Require line.
A file beginning with a
. is a hidden file on unix. If you are using
the gnome
graphical desktop then pressing Ctrl-h in
nautilus (the default file browser) will show hidden files. (If you are
using something else you should be able to work out what to do.)
Within an application
If you’re writing or maintaining a webapp that needs to authenticate
users for certain pages, you’ll likely need a Raven or UCamWebAuth
library. The Raven developer
documentation may be useful for finding a
suitable integration.
Regular hosting
Overview
All accounts include regular web hosting, which is suitable for
static content, PHP (WordPress, Joomla, etc.), and CGI/FastCGI. For
those wishing to host more advanced web apps with the SRCF (Django,
Flask, Rails, Node.js, etc.) and leverage their full flexibility, see
this page.
This page lists examples of applications and frameworks you can use to
build a website. They can also be seen at
https://sample.soc.srcf.net,
and you can take a look at their configuration inside
/public/societies/sample.
PHP applications
Scripts in public_html using .php extensions will be executed by PHP
by default.
Follow
our full WordPress installation tutorial .
Static site generators
You can use a generator of your choice to manage your site. Just have it
output into your account’s public_html directory.
To create the skeleton project:
jekyll new jekyll
cd jekyll
For a root site (i.e. top of public_html):
ln -s /public/societies/sample/public_html _site
..or for a site in a subdirectory, edit _config.yaml to set
baseurl to e.g. /jekyll, then:
mkdir /public/societies/sample/public_html/jekyll
ln -s /public/societies/sample/public_html/jekyll _site
To (re)build the site:
Rules and principles
Commercial hosting
Our servers are connected to the Internet via the Cambridge University
Data Network (CUDN), and as such all users of the SRCF are bound by its
rules, which explicitly forbids any use of the CUDN for commercial or
for-profit activities. Therefore it is not possible for commercial
websites to be hosted on our machines. For more details, please refer to
the Authorisation for use of the CUDN
and the Rules made by the Information Services Committee.
Our interpretation of these rules is that including a sponsor’s logo on
a group account website is acceptable, however selling web space or
advertising is not.
Furthermore, the SRCF server is maintained by a group of volunteer
sysadmins who donate their time freely to ensure that our services can
continue to operate smoothly. We do not feel that it is appropriate to
ask that the sysadmins give up their time in this way so that others can
make money.
Inappropriate content
Our policy
Inappropriate content is deemed to be any content that is in
contravention of the CUDN rules and guidelines, or JANET AUP. Amongst
other things these rules prohibit using the CUDN to store or transmit
material that is obscene, defamatory, offensive or violates a third
party’s copyright. If such material comes to the attention of the
sysadmins or committee the responsible user will be required to remove
it. In the case of gross and flagrant breaches of the rules the
sysadmins may remove the material themselves. The account of a user
responsible for prohibited material may be suspended and they may be
reported to the Computing Service.
So in summary, don’t do it!
Reporting a breach
Please let the user or group that controls the content know that you
consider it inappropriate, and explain why with reference to the SRCF’s
Terms of Service, CUDN rules and guidelines, or JANET AUP. For pages
with URLs of the form:
https://spqr2.user.srcf.net/ or https://www.srcf.ucam.org/~spqr2/
you should email spqr2@srcf.net
For pages with URLs of the form:
https://groupname.soc.srcf.net/ or
https://www.srcf.ucam.org/groupname/
you should email **groupname**-webmaster@srcf.net
If you do not get a response from the owner of the page or feel unable
to approach them yourselves, then you should forward your concerns to
the committee: committee@srcf.net.
Using resources wisely
Please be considerate of others when running long and CPU-intensive
tasks. Make sure that you use ‘nice’ to give them a low priority and
double check your code to make sure that there aren’t memory leaks or
other bugs which might impair the performance of the system for the
other users.
If you need to use a lot of computing time for academic reasons your
first port of call should be your department or college as they should
provide you with the resources you need for your studies. If your
reasons are non-academic,
get in touch with us.
Scheduled tasks
Overview
You can make use of cron or systemd timers to run programs on a
schedule. In either case, you will need to configure it on the right
server (e.g. tasks relating to web applications should probably run on
sinkhole, game servers on cavein or doom and anything else
most likely on pip).
Cron
Connect over SSH and run crontab -e, which will open an editor and an
explanation of the file format. You can add entries to run something at
a fixed interval, or on server reboot:
# Run at 03:21 every day:
21 3 * * * /home/<crsid>/daily.sh
# Run at server startup:
@reboot /home/<crsid>/startup.sh
If you want to setup cron jobs for a group or society account then
you’ll need to run sudo -u SOCNAME crontab -e.
A tool like this is helpful for mastering syntax.
systemd
Unit files
You can use user unit files to manage services, which should be placed
inside ~/.config/systemd/user/ (where ~ is your home directory, e.g.
/home/<crsid> or /societies/<name>).
A sample personal account unit file, porridge.service, might look like
the following:
[Unit]
Description=spqr2 Porridge app
ConditionHost=pip
[Install]
WantedBy=default.target
[Service]
ExecStart=/home/spqr2/porridge/run.sh
Restart=on-failure
ExecStart defines the command to be run. For group account services,
ensure any paths to programs or directories are absolute and not via
personal account paths (i.e. /societies/foosoc, not
/home/spqr2/foosoc). You may also need to define WorkingDirectory as
the directory path where your service is run from.
ConditionHost is required to ensure your service only runs on a single
desired server, as your home directory and all your unit files will be
present on all servers. Valid values are pip, sinkhole, doom or
cavein. In order for services to run without you being logged in, your
user account needs to have lingering enabled. This is a per-server
state, and is controlled by the existence of unit files with a valid
ConditionHost line – it may take up to 20 minutes for lingering to
be enabled on creation of your first service.
By default, your services’ standard output and error streams are sent
to systemd’s journal, however only the root user can access these. You
should make your service write to their own log file rather than stdout
or stderr.
Consult the systemd
documentation
for all available options. Note that some options are not available for
user units (as opposed to system-wide units).
Timers
The above unit will give you a long-running service, one that starts
your program at system startup. To schedule execution, drop the
[Install] section from the unit file and instead create a
corresponding timer file porridge.timer:
[Unit]
Description=spqr2 Porridge daily update
ConditionHost=pip
[Install]
WantedBy=timers.target
[Timer]
OnCalendar=daily
This will execute your program at midnight each day. The OnCalendar
line accepts phrases like daily and weekly, or time strings of the
form DayOfWeek Year-Month-Day Hour:Minute:Second (for example,
Sat *-*-1..7 18:00:00 gives you 6pm on the first Saturday of each
month).
Service control
To control your app, you can use the systemctl command over SSH. Basic
commands are of the form systemctl --user <action> <unit>. Actions
include start, stop and restart to run your service now, enable
and disable to set whether the unit launches on system startup, and
status to show information about it. The unit parameter is the name of
the unit file itself, e.g. porridge.service or porridge.timer.
If you make a change to a unit file, you need to run
systemctl --user daemon-reload on the affected server in order to pick
up the change.
Due to an implementation detail, you must call systemctl as follows
when interacting with a group account’s services (here using foosoc
for the account name):
sudo -Hu foosoc XDG_RUNTIME_DIR=/run/user/$(id -u foosoc) systemctl --user ...
You might like to add a function to your ~/.bashrc to make this easier
to remember:
socsudo () {
soc=$1
shift
sudo -Hu $soc XDG_RUNTIME_DIR=/run/user/$(id -u $soc) "$@"
}
..and then run socsudo foosoc systemctl --user ....
If you receive an error like this setting up your group account’s first
service:
Failed to connect to bus: No such file or directory
…then you may need to wait up to 20 minutes for lingering to be
switched on – this will happen automatically once a service file is
detected with a valid ConditionHost line.
Soc-SRCF mailing lists
Overview
The SRCF needs to be able to keep in contact with our users at all times
(a requirement imposed upon us). We therefore automatically subscribe
you to Soc-SRCF, Soc-SRCF-users and Soc-SRCF-maintenance shortly after
creating your account. We will also subscribe you to Soc-SRCF-social if
you requested this during the signup process.
These are very low-traffic mailing lists—just a few messages per
year—and we do our best to ensure that the messages sent are relevant
to as many people as possible. We hope that you will not consider this
to be an inconvenience.
How we use these lists
Soc-SRCF
This contains members of the SRCF (membership is for life), and
receives messages from the committee about such things as:
- major society event announcements (e.g. AGMs, garden parties) –
usually about three events per year
- rare but important society administrative information (e.g.
constitutional amendments) which affect all members
Soc-SRCF-users
This contains users of the SRCF (which only applies whilst you have an
active shell account), and receives messages from the sysadmins which
are important to account holders, for example:
- urgent messages which require the attention of every SRCF user (e.g.
security compromises) – very infrequent, we hope!
- rare but important system administrative information (e.g. terms of
service changes) which affect all users
Soc-SRCF-maintenance
This is used for announcements about planned and ongoing system
unavailability (downtime), and announcements about important changes to
service provision (e.g. increased quotas, software changes which may
require your action). If you are not interested in receiving such
messages, you are welcome to unsubscribe, but many users find these
announcements useful.
Soc-SRCF-social
This is a more open mailing list for announcement and discussion of
social events (pub meets, LAN parties, garden parties and so on). Any
SRCF member is welcome to join or leave this list at any time.
Soc-SRCF-careers
The SRCF has in the past provided career opportunities informally. Join
this new list to keep up to date with any employment opportunities,
internships and news directly from companies we trust.
Unsubscribing from Soc-SRCF and/or -users
Because we must be able to contact everyone with an SRCF account, you
may only unsubscribe from Soc-SRCF or Soc-SRCF-users if you are
willing to give up your membership of the SRCF and access to its
services. If this is what you want, please contact the
sysadmins saying you want to terminate
your membership.
If you wish to do so, you may give up your shell account (and so cease
to be an SRCF user) while remaining a member of the SRCF. In this
case, you will be unsubscribed from Soc-SRCF-users, but not Soc-SRCF,
and you are still welcome to attend SRCF events. As above, please
contact the sysadmins to arrange this.
Unsubscribing or rejoining other lists
You can subscribe to or unsubscribe from a list from its information
page:
List archives
Previous messages sent to these lists can be seen here:
Societies
Society accounts have been renamed to group accounts. This page has been moved.
Software and installation
Overview
Our main shell server includes a plethora of Linux software, some of
which can be used graphically over a VNC session.
If something is missing that you’d like to use, we may be able to
install it for you. The usual constraints are that it should be
available in Ubuntu’s repositories, and
suitable for a multi-user environment. You may contact the
sysadmins to request an installation.
As we run long-term support releases of Ubuntu, you may find that
the versions we have installed are rather outdated (typically several
years behind). As packages are managed by Ubuntu maintainers, security
fixes are backported and applied to these versions, but new features are
not. This means some custom software may incorrectly detect our versions
as insecure, as Ubuntu-patched releases add a version suffix (for example, PHP is 7.4.3-4ubuntu2.4).
Whilst some software can be updated through the use of third-party
distributions, or by building the latest version from source, this would
add significantly more complexity to system maintenance and upgrade
considerations, and would require trusting the additional authors or
distribution maintainers.
In some cases, you may be able to set up or install your own copy of
software within your (or a group account’s) home directory. However, it
is then your responsibility to keep it updated – accounts with security
vulnerabilities may be suspended without notice.
System-wide upgrades
Every few years, we need to move to a newer version of Ubuntu in order
to continue receiving security updates. As a side-effect, this will
introduce significantly newer versions of most installed software, which
is liable to break any unusual software configurations of our users. In
the past we’ve seen significant changes with Apache and PHP, which many
users use for their personal or group websites.
We are aware that newer versions of Ubuntu are available and waiting for
us – upgrades are not taken lightly, and will be carefully planned to
minimise disruption to our users.
Program-specific notes
Node.js
Ubuntu comes with node version 10.19.0. If you need a newer version, consider
using Node Version Manager.
PHP
Ubuntu provides version 7.4 of PHP, along with a handful of modules.
We’re aware that various web applications will flag this version as
outdated or insecure – see above for how this is catered for.
Python
We have Python versions 2.7 and 3.8. Note that the python and pip
binaries are Python 2, use python3 and pip3 for Python 3. If you
need a newer version of Python, consider using pyenv.
Assorted Python modules (as packaged by Ubuntu) are installed for both, and
others can be installed for Python 3 on request (subject to the notes above). Module support for Python 2 is limited – if you’re unable to migrate code to Python 3, you’ll likely need to use a virtualenv and manage package installation yourself.
Ruby
The servers have version 2.7 of Ruby installed together with a handful
of useful gems and development dependencies. If you need a more recent
version then you might consider using
ruby-build and/or
rbenv.
Install a package
Probably. Feel free to email us at support@srcf.net and be sure to
provide the name of the Debian
package you
want us to install. Keep in mind we’ll probably be installing the
stable version of the package, so it might be old.
You might prefer to install the package locally. See below.
Update a package
Probably not. Our servers run Ubuntu stable, so it’s expected that
system packages aren’t current (indeed, they’re often a few years
old). We almost never make exceptions or install backported packages.
For developing and deploying your app, you should almost certainly be
using your platform’s version manager (rvm, venv, nvm, gvm, etc.). This
will allow you to run the exact versions you want, and install any
necessary dependencies, all without coordinating with us (or forcing the
rest of our users to switch versions).
The pages above provide instructions on doing this with popular
programming languages.
Port binding
When running programs that bind to ports, please ensure you only bind to
localhost so they’re not publicly accessible, and choose a random
high-numbered port to try and avoid conflicts with other users.
If you’re able to bind to a UNIX socket instead of a port, this has several benefits: the socket is subject to filesystem permissions so only
you or a society account can access it, and you are free to choose a
unique path for it.
SSH and servers
Overview
You can connect to our servers over SSH in order to run programs, manage
files, and change your password.
Clients
You will need an SSH client – Mac and Linux come with ssh in the
terminal; for Windows, you can enable OpenSSH in Optional Features, use
a Linux distro in WSL, or install a Windows client like PuTTY.
Alternatively, you can access a web-based
terminal in your browser.
Passwordless SSH
Passwordless SSH is often a point we get asked about.
There are a number of reasons why your passwordless ssh setup might not
be working but by far the most common (and the hardest to detect on the
client side!) is incorrect permissions on your .ssh directory. Because
the authorized_keys file contains sensitive information it must not be
readable by any other user. Therefore you should ensure that the file
$HOME/.ssh/authorized_keys have permissions -rw------- (i.e. 600).
660 (-rw-rw----) will not work even if you are the only member of the
group that owns the authorized_keys file. The same caution about
permissions applies to the .ssh directory itself.
Available hosts
Shell
shell.srcf.net
Our main server is pip. This is a general-purpose machine for
running software. It has the following SSH fingerprints:
DSA MD5:12:8f:8a:1c:e4:f7:a2:9d:80:a3:ef:85:f8:79:a4:ed
DSA SHA256:fshymFC90Vd0BrlFnAdilNKSWNsoQVI7qa8/GIMBqtw
RSA MD5:4d:da:7f:b2:99:9d:42:8b:10:b4:e5:37:d5:bd:43:b4
RSA SHA256:ud4XwVhosGW3oHZ2POzW4oz0zGC2y7JWLGOM1dbIqZw
Web
webserver.srcf.net
Web applications should be run from sinkhole, our webserver. This
includes Apache, our server software of choice, along with custom
backend servers which Apache can proxy to. Sinkhole has SSH fingerprints
as follows:
DSA MD5:20:28:75:a4:c5:0b:08:47:c5:ae:f8:8c:21:01:1a:00
DSA SHA256:iyt8GRgVBQSNyaH/yX7QZuFODMzfpsO9xV7UnaxICLI
ECDSA MD5:90:78:92:7b:1e:4e:51:22:30:3b:e0:dd:97:9c:aa:e4
ECDSA SHA256:G1glvFC2H4VR9mA1RkJCRVVyLB6e/LarI+8ZUFRGds8
ED25519 MD5:55:2e:b8:bb:53:79:9a:62:0f:62:24:e3:8d:7f:87:62
ED25519 SHA256:WalyNfnZm6yuIhMhe6kj17NikSK8LuC2I6iggWYTKcY
RSA MD5:76:aa:f1:63:ec:ca:ac:97:fb:05:35:3c:c7:8f:55:ff
RSA SHA256:DSC30UdUoQNYWhwU4icuPOyrDC9xVjsnKbRC3Mz9RZM
Games & others
doom.srcf.net
If you’d like to run a game server or more CPU-intensive application,
please use doom, which has fingerprints:
ECDSA MD5:74:52:6e:52:d7:07:51:9e:93:12:51:82:8c:b4:ec:0a
ECDSA SHA256:QdTm7ECl0KObTodBBpY6Qs57PFmr3MHvQfT50IbKa3U
ED25519 MD5:49:d8:e6:67:19:1c:c5:ed:12:de:3a:64:91:95:4c:9d
ED25519 SHA256:kh1Sr6Nrlp/vK9ijKZ43/IQ2tqdPzY/fnZdnGBIKgIM
RSA MD5:f1:ee:8c:a7:7a:cb:f7:c7:dc:c5:7e:56:9a:83:f5:bc
RSA SHA256:c1dlaFnPyJ44CnjZIeV6zLHQCPlIH9Og0K3dL16XGfo
Troubleshooting
Unsupported term-type environment variable
If the terminal behaves incorrectly on certain keypresses - such as by
outputting weird glyphs, or by not removing characters when inputting a
backspace - then it is possible ssh has misconfigured the TERM
environment variable.
This can be checked by running toe -a | grep $TERM. (toe -a lists
all supported values for the TERM variable, and | grep $TERM
performs a search within toe's output to try to find your session’s
value of $TERM). If running the command shows no output, then your
value of TERM is unsupported.
You can temporarily select a more general terminal type
(xterm-256color should work) for your session by setting the TERM
environment variable when running ssh:
TERM=xterm-256color ssh ...
To correct this persistently, run the following command (on your
device):
infocmp -a $TERM | ssh <Your CRSid>@shell.srcf.net tic -x -
This will install your device’s terminal-type to your account’s home
directory (inside ~/.terminfo/). You will not need to make any manual
changes to contents of this directory. Your terminal should now work
correctly for all subsequent sessions.
Usenet (NNTP)
The SRCF offers a Usenet server for our users. To connect, you will need
to point your newsreader at news.srcf.net and connect using a secure
connection (SSL) to port 563. In order to combat spam, you will need to
login with your SRCF username and password.
If you wish to use Alpine as your newsreader then you will find that our
shell server pip has a default configuration ready to go. Otherwise
you will need to configure it by going to (S)etup, (C)onfig and then
setting nntp-server to news.srcf.net/ssl/user=pqrs2 where pqrs2 is
your CRSid.
Web applications
Introduction
All accounts include regular web hosting, which is suitable for
static content, PHP (WordPress, Joomla, etc.), and CGI/FastCGI. For
those wishing to host more advanced web apps with the SRCF (Django,
Flask, Rails, Node.js, etc.) and leverage their full flexibility, we
provide these overview docs and
more custom tutorials too:
We also provide a number of sample websites for you. You can explore
them live at https://sample.soc.srcf.net/ and view the backend at
/public/societies/sample.
You can host basically any kind of web application that can bind to a
socket, including all of the popular frameworks like Flask, Django, Ruby on Rails, Express, and beyond. We provide tips and information on getting your apps to run on our web server.
Adding our badge
We encourage you to add the SRCF badge to your website so that
others can find us. It’s a great way to help us out as it costs little
to you but helps us out immensely, in turn allowing us to provide better
services to you. You can find the badges
here.
Connecting
We provide a webserver (currently named sinkhole), for hosting
applications. You should connect to this server, not to the public
login server, pip.
You connect to this server via SSH using your normal SRCF account name
and password.
- Host: webserver.srcf.net
- Port: 22
Routing traffic to your app
The SRCF uses Apache to serve websites so if you need to run a backend
web app, for example a Django, Rails or Express server, then you will
need to forward web requests. See also our guidance on port binding.
Using UNIX sockets
You will need to configure your application to use a UNIX socket. The
socket should only be accessible to you, which can be done by picking a
path in your home directory (e.g. /home/ab123/myapp/web.sock) or by
using appropriate file modes.
Then add the following to your .htaccess file, replacing
<path-to-socket> with the path to your socket (e.g.
/home/ab123/myapp/web.sock) and <url> with your domain name (e.g.
ab123.user.srcf.net) :
RequestHeader set Host expr=%{HTTP_HOST}
RequestHeader set X-Forwarded-For expr=%{REMOTE_ADDR}
RequestHeader set X-Forwarded-Proto expr=%{REQUEST_SCHEME}
RequestHeader set X-Real-IP expr=%{REMOTE_ADDR}
RewriteRule ^(.*)$ unix:<path-to-socket>|http://<url>/$1 [P,NE,L,QSA]
Using TCP ports
You will need to pick a port (we’ve used 999 here but you should pick a
different one above 1024) and configure your application to bind to that
port. Be aware that port-based forwarding offers less security than UNIX
socket-based forwarding and that any other user will be able to forward
requests to the same port you are using. For that reason, we don’t set
the headers we do above as they can easily be forged by another user.
Those things being considered, you can put the following in your
.htaccess file to enable forwarding requests to a port:
RewriteRule "^(.*)$" http://localhost:999/$1 [P,NE,L,QSA]
If you are serving a page on your root URL, you will have to add DirectoryIndex disabled to your .htaccess file to prevent the default Apache file listing behavior.
Databases
MySQL and PostgreSQL accounts are included with your SRCF account.
The source for these docs are generously provided by the Open Computing
Facility. They have been amended to include SRCF-specific information.
Read more about them
here.
Tutorials
So you want to learn how to make cool stuff on SRCF systems? We can help
you with that! This is a growing list of tutorials that go into the
various things you can do aimed at all levels of skill.
These tutorials show you how to do a variety of things on the SRCF
systems. They may assume some prior knowledge in the relevant sections.
Where to start?
The amount of content and tutorials here can be overwhelming, so here’s a few ideas to start:
- SSHing into an SRCF server, like
pip
- Knowing your way around the terminal and some basic commands
- Creating an
index.html file in your public web directory and viewing it
Simple Minecraft Server
Overview
In this tutorial, you will learn how to set up a vanilla Minecraft Java server hosted by the SRCF.
Introduction
Game servers and other CPU-intensive applications are hosted at doom.srcf.net.
Pre-requisites
To follow this tutorial you will need:
- An SRCF account
- Basic familiarity with the terminal
Logging in
Before starting you must log in to doom.srcf.net via SSH. You will have to use your CRSid and your SRCF password.
Log in using your computer’s terminal (PowerShell on Windows) with the following command.
Setting up the Minecraft server
I suggest you create a directory for the server using these commands.
The machine already has java set up, so that you can avoid the hassle of getting it working.
The JAR file for the server can be found here. To download the JAR
file on the server you must copy the download link from the page and then run the wget command, replacing the url
with the address you copied from the download page.
wget https://piston-data.mojang.com/.../server.jar
You can now run the server using the command
nice -n 19 java -Xms1024m -Xmx1024m -jar server.jar nogui
It will set up some directories and then quit, saying that you need to accept the end user license agreement. To do this,
open the eula.txt file that was created with a text editor such as
Change the line that says eula=false to eula=true and quit using CTRL+X. When asked Save modified buffer?
press Y to save your changes.
It’s also necessary to change the port number the server will run on. To do this, open the server.properties file
and find the lines query.port=25565 and server-port=25565. Replace 25565 with a random number between 1024 and 65535.
Remember the number you use, because you will need it later to connect to the server.
If you want to change the gamemode you can do this here by amending the line gamemode=survival.
You can now run the server again using
nice -n 19 java -Xms1024m -Xmx1024m -jar server.jar nogui
This time you should find that the server starts successfully. If you open your Minecraft client and connect to
doom.srcf.net:<port> (where is your random number) you should spawn into your world!
Running the server in the background
At the moment you have to log into the machine and start the minecraft server manually every time you want to
use it. You can set it up as a systemd service to allow it to run in the background and start up automatically
when the machine restarts.
To do this you need to create a service unit file using the commands
mkdir -p ~/.config/systemd/user/
nano ~/.config/systemd/user/mcserver.service
and paste the following into it, replacing both instances of CRSID with your CRSid.
[Unit]
Description=Minecraft Server
ConditionHost=doom
After=network.target
[Service]
Type=simple
WorkingDirectory=/home/CRSID/mc
ExecStart=/usr/bin/nice -n 19 /usr/bin/java -Xms1024m -Xmx1024m -jar /home/CRSID/mc/server.jar nogui
Restart=always
RestartSec=20
[Install]
WantedBy=default.target
Get systemctl to reload the service unit files, enable your service so that it starts up on reboot and start the
service using the commands
systemctl --user daemon-reload
systemctl --user enable mcserver.service
systemctl --user start mcserver.service
You should now have a Java Minecraft server ready to play!
If you ever want to stop your server or check its status you can use
systemctl --user stop mcserver.service
systemctl --user status mcserver.service
To learn more about running systemd services with the SRCF see
here.
Now that the setup is finished you can log out of doom.srcf.net using logout.
{{ < alert type="warning” > }}
You may notice that in the command to start the server the memory it can use is limited to 1GB and its priority is set
using nice. Please do not change the priority or memory usage of your server in order to be considerate to other
users of the SRCF’s systems!
{{ < /alert > }}
Did you like this or find this cool? We invite you to check out
more tutorials
or get in touch to tell us what you thought!
If you have any suggestions for how we could improve this documentation
please send us an email at support@srcf.net or submit a Pull Request
on GitHub!
Deploy a web app
Overview
Nowadays, most content served to users via the web is dynamic and via web applications. This tutorial will show you how to set up a simple web application with Flask in Python, but the process is much the same for any other web framework and we give tips for other languages.
Introduction
The SRCF hosts a general purpose web server that can serve all types of content. While the existence of a public_html folder might lead users to think that only static content can be served, Apache’s .htaccess file allows for arbitrary reverse proxying to a TCP port or UNIX socket. This gives users great freedom to do whatever they wish, including hosting applications written in popular frameworks like Django, Flask, Sinatra, Ruby on Rails, Express, etc.
We make available a sample group that contains examples of various web frameworks running on the SRCF:
https://sample.soc.srcf.net,
and you can take a look at their configuration inside
/public/societies/sample.
Pre-requisites
Before starting,
- Ensure that you are comfortable in some capacity with the terminal
- Ensure that you are logged into (and performing any subsequent commands on) our web server,
webserver.srcf.net
Environments
Python: set up a venv
You will want to deploy your application in a Python venv so that you
can easily install and manage dependencies and versions.
A venv is a
virtual Python environment that contains its own Python binary as well
as all of your dependencies. When you activate your venv, PATH variables
like python and pip will use the correct versions specified when you
created your venv.
-
Create a directory for your app to live in:
mkdir -p ~/myapp
cd ~/myapp
-
Set up a venv
-
Activate the venv
You should do this step every time before running your app or
managing installed packages.
-
Done! Your Python venv is now installed and hooked into your shell.
Node: installing nvm
You will want to deploy your application using nvm so that you can
easily install and manage dependencies and versions.
nvm stands for Node Version Manager. Ubuntu only provides a
significantly outdated version of Node.js (v4.2.6 at the time of
writing) in its repositories. Using nvm allows you to choose any
version of Node to use in your environment. Note that it is your
responsibility to keep your node installations and nvm itself updated.
-
Create a directory for your app to live in:
mkdir -p ~/myapp
cd ~/myapp
-
Install nvm in your home directory. You’ll need to find the
latest version from the NVM GitHub,
and copy the one-liner straight into your shell to install it. At
the time of writing, it looks like this:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
Piping a script straight into your shell is potentially dangerous as you are allowing arbitrary code to be executed. Make sure you trust any script you run or vet it beforehand.
Make sure you read the installation instructions, then go ahead and run it, and close/re-open your terminal as it suggests.
-
Install whatever version of node.js you want.
nvm install 12
nvm alias default 12
-
Done! The version of node.js you specified is now installed and
hooked into your shell.
Ruby: install rbenv
You will want to deploy your application using rbenv so that you can
easily install and manage dependencies and versions.
-
Create a directory for your app to live in:
mkdir -p ~/myapp
cd ~/myapp
-
Install rbenv in your home directory:
git clone https://github.com/rbenv/rbenv.git ~/.rbenv
cd ~/.rbenv && src/configure && make -C src
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
~/.rbenv/bin/rbenv init
-
Follow the printed instructions on appending to your ~/.bashrc
file:
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
-
Install ruby-build as a plugin:
mkdir -p "$(rbenv root)"/plugins
git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build
-
Install whichever version of Ruby you want.
rbenv install 2.6.6
rbenv local 2.6.6
-
Done! You now have a working Ruby installation that’s hooked into
your shell.
Install your web framework
You should not rely on the system-wide versions of web frameworks present on our webserver as they are LTS releases and may be out of date. Instead, use the version manager that comes with your framework/language to install a more up to date version.
- Node:
npm i express
- Python:
pip install flask or pip install django
- Ruby:
gem install rails
Create your app
If you have an existing node.js app you can simply copy the code to a
directory of your choosing, ~/myapp/src for example. You can use
scp/sftp to upload it or clone it using git or some other source
control tool.
If you’re new to web apps, though, it’s recommended you follow the getting started guide (or equivalent) for your language/framework of interest:
- Node: https://nodejs.org/en/docs/guides/getting-started-guide/
- Express: https://expressjs.com/en/starter/installing.html
- Ruby on Rails: https://guides.rubyonrails.org/getting_started.html
- Flask: https://flask.palletsprojects.com/en/1.1.x/quickstart/
We do have a few lines of code in a few languages that can springboard your app below. For real examples, you should visit the content of the sample group at /public/societies/sample/
Flask
There is a demo flask app setup at https://sample.soc.srcf.net/flask/.
If you’re logged in via ssh, you can find the code at
/public/societies/sample/flask/app.py.
Django
To create a skeleton Django project:
django-admin startproject example
mv example python
cd python
Now take a look at django/example/settings.py for how to configure
your site. There is also a demo available at
https://sample.soc.srcf.net/django/.
Ruby on Rails
Ruby on Rails is arguably one of the most popular and influential web
frameworks. To create a skeleton Rails project:
gem install rails
rails new myapp
You should now have a new Ruby on Rails project setup in the myapp
directory.
Sinatra
Sinatra is a lightweight web microframework that’s well suited to
simple projects. To install it run gem install sinatra and then put
the following in myapp.rb:
require 'sinatra'
get '/' do
'Hello world!'
end
You can then run this by typing ruby myapp.rb and going to
http://localhost:4567 in your web browser. For futher information See
the Sinatra documentation.
Express
Sample site at https://sample.soc.srcf.net/nodejs/
and see nodejs/app.js for a minimum base application.
Deploy your app
In a production environment, your app will need to “boot” (ie. start up) in a well-controlled manner. You will also want it to run in the background and ensure it restart when our web server reboots. systemd is a popular choice for “supervising” a service.
Don’t run development servers on the SRCF – these typically
provide remote code execution via debug consoles, which grants any
visitor full access to your SRCF account. Ensure your site runs in a
production mode if configurable.
Boot scripts
First create a bash script that will run your web app at
~/myapp/run.sh with the following content:
The scripts provided here are just examples and won’t necessarily be perfect for your set-up. For example, they assume you’re using a UNIX socket over a TCP port. More than likely, the getting started guides referred to above will have their own boot scripts too.
Python
#!/bin/bash -e
. ~/myapp/venv/bin/activate
exec gunicorn -w 2 -b unix:/home/crsid/myapp/web.sock \
--log-file - main:app
Replace main:app with the module containing the app, and name of your
app.
Node
#!/bin/bash -e
USER="$(whoami)"
[ -e "/home/crsid/myapp/web.sock" ] && rm "/home/crsid/myapp/web.sock"
umask 0
. ~/.nvm/nvm.sh
NODE_ENV=production PORT="/home/crsid/myapp/web.sock" \
exec ~/myapp/src/bin/www
Replace ~/myapp/src/bin/www with the path to your app.
Ruby
#!/bin/bash -e
eval "$(rbenv init -)"
cd ~/myapp/src
RAILS_ENV=production \
exec bin/rails server -b /home/crsid/myapp/web.sock
Replace ~/myapp/src with the path to your app.
Now, for all frameworks, make the run.sh script executable:
You should now be able to execute the script and access your website (or
see any errors in your terminal).
Using systemd
The SRCF may restart any of its servers as part of regular or emergency
maintenance. When this happens, you’ll likely want your app to start up
again. Similarly you may want your app to automatically restart if it
crashes. We highly recommend using systemd to supervise your app.
-
Create a directory for your app ~/myapp.
-
Place a startup script at ~/myapp/run.sh. Your script should end by
execing the server process. If you followed one of our tutorials
for Django, Node or Rails, you’ve already created this file, so can
move on to the next step. Otherwise, an example would be:
#!/bin/sh -e
exec ~/myapp/run-server
Your server should run in the foreground (it should not
daemonise), and the run.sh script should end with an exec line so
that signals are sent to the server (and not to the shell that
started it).
Once you’ve written the script, make it executable
(chmod +x ~/myapp/run.sh). Test it by executing it in your terminal
before moving on; it will be easier to debug problems.
-
Write a systemd service file so your app will be supervised on
startup.
For applications in your personal account, create the unit directory
if it doesn’t exist:
mkdir -p ~/.config/systemd/user
For a group account, substitute ~ for /societies/foosoc, where
foosoc is the short name of the account.
Then, save the following to the file
~/.config/systemd/user/myapp.service (or
/societies/foosoc/.config/systemd/user/myapp.service for groups):
[Unit]
Description={YOUR USER, SOCIETY OR GROUP NAME} Webapp
ConditionHost=sinkhole
[Install]
WantedBy=default.target
[Service]
ExecStart=/home/{CRSid}/myapp/run
Restart=always
-
Tell systemd to start your app on startup, by running
systemctl --user enable myapp.
-
You’ll need to start your app manually once (on future reboots, it
will be started for you). To do that, run
systemctl --user start myapp.
To control your app, you can use the systemctl tool. See
man systemctl for full details.
In summary,
- Restart an app.
systemctl --user restart myapp
- Bring an app offline.
systemctl --user stop myapp
- Bring an app back online.
systemctl --user start myapp
- Check the status of an app.
systemctl --user status myapp
By default, your app’s standard output and error streams are sent
to systemd’s journal however only the root user can access these.
You will want to make your app write to a log file rather than stdout
or stderr.
We have further more general reading on systemd services.
Notes for Python
Installing gunicorn
We recommend using gunicorn to serve your application. After activating
your venv, install it with pip install gunicorn.
Note that you may see a warning about a syntax error. As long as the
output ends in ‘Successfully installed gunicorn’, it’s safe to
ignore this.
Reloading your app
Gunicorn will reload your app if you send it SIGHUP. You can teach
systemd that fact by adding the following line under [Service] in your
systemd unit file:
ExecReload=/bin/kill -HUP $MAINPID
and then running systemctl --user daemon-reload. After that, you can
use systemctl to reload your app:
systemctl --user reload myapp
Notes for Ruby
- You may need to migrate your database first.
- Make sure you’ve set secret keys for the app and any gems that need
them (e.g. Devise).
- Static file serving is off by default in production, but you’ll
want to turn it on: set both
config.assets.compile and
config.public_file_server.enabled to true in
config/environments/production.rb.
Forwarding requests
You then need to configure Apache to forward web requests to the
web.sock socket specified in the run.sh script. We explain how to do
this in the reference for web apps.
Did you like this or find this cool? We invite you to check out
more tutorials
or get in touch to tell us what you thought!
If you have any suggestions for how we could improve this documentation
please send us an email at support@srcf.net or submit a Pull Request
on GitHub!
Getting to know the terminal
Overview
This tutorial will walk you through some more advanced commands that you
can perform in a UNIX-like terminal, which the SRCF provides you with.
You will learn many of the basic commands that are used and why in
conjunction with other tools they can be very powerful.
This tutorial assumes you have an SRCF personal account (with or without
group account admin permissions) and can SSH. See our
other tutorials for that.
Introduction
The SRCF offers you much more than just a web server - everyone who has
an account has full shell access to our Linux servers. This allows you
to manipulate and edit the contents of your filespace directly without
having to worry about downloading and uploading files, as well as
running all sorts of other software which you might not have available
to you under your usual operating system.
Basic commands
After successfully logging in, you will see a few lines of text welcoming
you to the system and telling you any recent news, followed by a
command prompt:
You are now logged in and can give the system commands by typing them
after the prompt. If you wish to start a particular program, the command
is just the name of the program you wish to run. For example typing
‘date’ and pressing return will tell you the current date and time:
spqr2@pip:~$ date
Wed Apr 5 22:29:11 BST 2020
spqr2@pip:~$
After each command has finished, a new prompt will be displayed ready
for the next command. Some interactive programmes, such as the email
reader pine will fill the whole screen, but will still return to the
command prompt afterwards.
Commands with arguments
Some commands require arguments, that is additional parameters which
alter the way that they run, for example the command ‘ping’, which is
used to check the speed of an Internet connection, requires the name of
a host as an argument:
spqr2@pip:~$ ping www.bbc.co.uk
PING www.bbc.net.uk (212.58.224.35): 56 data bytes
64 bytes from 212.58.224.35: icmp_seq=0 ttl=247 time=4.9 ms
64 bytes from 212.58.224.35: icmp_seq=1 ttl=247 time=4.6 ms
--- www.bbc.net.uk ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 4.6/4.7/4.9 ms
spqr2@pip:~$
(To stop the ‘ping’ program, press control-c)
To find information about the arguments for a command, use the ‘man’
command, for example ‘man ping’ will tell you about the ‘ping’
command. Use the arrow keys to scroll through the man text and press
‘q’ when you want to return to the command prompt.
File system
All files stored on the system reside in directories (this is the
equivalent of Windows folders). The directories are arranged in a
hierachical structure - directories may contain subdirectories and so
on. The top of the hierarchy is called the root directory, and is
represented by /. Other directories are refered to by their path,
for example, /home/abc45/ represents the directory abc45 which is a
subdirectory of ‘home’, which in turn is a subdirectory of the root
directory.
Files can be specified by adding the file name to the end of
the path, for example /home/abc45/my_file.txt is a reference to the
file my_file.txt which is stored in the directory /home/abc45/.
At any point during your session you will be ‘in’ a particular
directory called the working directory. Rather than using full paths
as above, it is possible to refer to the location of files relative to
working directory by omitting the leading /, for example
‘my_dir/my_file.txt’ represents the file ‘my_file.txt’ within the
directory ‘my_dir’, which is a subdirectory of the working directory.
Filenames given without a preceding path are assumed to be in the
working directory. All directories have a special subdirectory called
.. which refers to the directory one level higher up in the hierachy
(the parent direcvtory), so ../some_file.txt is the file
some_file.txt in the parent directory of the working directory.
To find out your current directory, use the command pwd:
spqr2@pip:~$ pwd
/users/abc45
spqr2@pip:~$
You can change the working directory by using the cd command, for
example:
spqr2@pip:~$ cd my_dir
spqr2@pip:~/my_dir$
Notice how the working directory is displayed as part of the command
prompt. The symbol ~ is an abbreviation for your home directory,
in other words the directory you start off in when you first log in.
To view the contents of the working directory, use the command ‘ls’:
spqr2@pip:~$ ls
public_html mygroup my_file.txt
spqr2@pip:~$
Alternatively, use ls -alF to give more detailed information. The
above example shows that the working directory has two directories
public_html and mygroup, the first of which contains your
web space and the latter of which is a link to the mygroup group
account file space.
There is also one file, ‘my_file.txt’ (there is no
way of differentiating between files and directories in the above
example - you need to use the ls -alF form to show that
information).
More commands
The following table gives some common commands used to manipulate files:
+-----------------------------------+-----------------------------------+
| Command | Meaning |
+-----------------------------------+-----------------------------------+
| cp <file1> <file2> | Creates a copy of 'file1' at |
| | the location specified by |
| | 'file2' |
+-----------------------------------+-----------------------------------+
| mv <file1> <file2> | Moves 'file1' to the location |
| | specified by 'file2'. Note that |
| | if 'file1' and 'file2' are in |
| | the same directory you can use |
| | this command to rename the file. |
+-----------------------------------+-----------------------------------+
| rm <file> | Deletes 'file' |
+-----------------------------------+-----------------------------------+
| mkdir <dir> | Creates a directory called |
| | 'dir' |
+-----------------------------------+-----------------------------------+
| rmdir <dir> | Removes the directory 'dir' (it |
| | must be empty first) |
+-----------------------------------+-----------------------------------+
| nano <file> | Edits 'file' using the 'nano' |
| | text editor. |
+-----------------------------------+-----------------------------------+
Note that the last command starts the nano text editor. This is a simple editor
which is sufficient for most tasks. If you get stuck in it, pressing control-x
will return to the command prompt. If you are looking for a more powerful
editor, albeit with a steeper learning curve, you might want to try vim.
Exiting
To log out of the system, type ‘exit’ at the command prompt.
Did you like this or find this cool? We invite you to check out
more tutorials
or get in touch to tell us what you thought!
If you have any suggestions for how we could improve this documentation
please send us an email at support@srcf.net or submit a Pull Request
on GitHub!
Passwordless login: SSH keys
Overview
Passwords aren’t very safe. Instead of remembering multiple passwords for multiple servers (nowadays made easier with password managers), it is common practice to use so-called SSH keys to login to a remote server. This tutorial will show you how to set-up an SSH key pair so you can log in to any of our servers without the need of a pesky password!
Introduction
A well-established method of authentication is via passwords. This, however, has been shown to be quite vulnerable, time and time again. An alternative solution to logging in to servers via SSH that offers lots of flexibility, is to use SSH keys.
SSH keys always exist in pairs: there is a private key and a public key. You will keep the private key while placing your public key in any servers you want to access. Just like you shouldn’t use one password for everything, you should use multiple private keys if you have access to a reasonable quantity of servers.
When logging in to a server, the SSH client on your computer requests the public key for the account you’re trying to log in to and checks to see if your private key is compatible with that one. This works because Math.
Generating a key pair
-
Open your terminal of choice
-
Paste in the following command with an email of your choice
ssh-keygen -t ed25519 -C "your_email@example.com"
This creates an SSH key pair, using your email as a label. You should see the following output
Generating public/private ed25519 key pair.
-
When you’re prompted to “Enter a file in which to save the key,” press Enter. This accepts the default file location, which is in the .ssh directory in your home directory.
-
Optionally enter a password when prompted.
As before, if someone gains access to your computer, they also gain access to every system that uses that key. To add an extra layer of security, you can add a passphrase to your SSH key.
Adding your keys to ssh-agent
You can use ssh-agent to securely save your passphrase so you don’t have to reenter it. You can find further guidance on this here.
Adding your key to the server
You now need to add your public key to one of our servers. Since all of our user-facing server share file systems, you only need to do this once.
A handy utility exists for this:
ssh-copy-id -i ~/.ssh/mykey spqr2@pip.srcf.net
More useful information on that here.
If that doesn’t work, you can always upload your public key to the server as you would a normal file. Your public keys should go in the ~/.ssh/authorized_keys file, separated by line breaks (press “enter”).
Make sure you upload your public key, not your private key. To check, you can always open the file in question and if it contains something like the following then you know it’s the private key.
-----BEGIN PRIVATE KEY-----
BASE64 ENCODED DATA
-----END PRIVATE KEY-----
Logging in
With your SSH keys in place, you should now be able to ssh sqpr2@pip.srcf.net without being prompted for your password!
You can even make this process simpler by setting up an SSH configuration file, to, for example, be able to type ssh srcf-webserver and log into the web server!
Did you like this or find this cool? We invite you to check out
more tutorials
or get in touch to tell us what you thought!
If you have any suggestions for how we could improve this documentation
please send us an email at support@srcf.net or submit a Pull Request
on GitHub!
Setting up IRC
Introduction
Internet relay chat is a protocol for real-time text messaging between
internet-connected computers. While it is dated in comparison to some
other messaging protocols, it continues to be used regularly by SRCF
members and as the emergency messaging platform of choice, with several
servers located around the world.
In this tutorial, you’ll be setting up a persistent IRC connection
using irssi and screen. It’s a great entry point to the terminal and
beginners are encouraged to follow this tutorial!
If at any point you encounter some parts that don’t make sense, reading
material from the references section towards the bottom of this page is
highly recommended. They’re also there if anything here sparks your
curiosity.
Installing
The SRCF already has
irssi and
screen installed so you can skip this
step but we include it here for completeness in case folks want to
venture out and set up their own machine. As they are standard versions
that come with the Long Term Release of Ubuntu they can be a bit
outdated, so feel free to build the latest version in your directory
from source:
https://github.com/irssi/irssi.
sudo pacman -S irssi # Arch
sudo yum install irssi # Red Hat derivatives
sudo apt-get install irssi # Debian derivatives
Install screen
sudo pacman -S screen # Arch
sudo yum install screen # Red Hat derivatives
sudo apt-get install screen # Debian derivatives
Configuring
Create logging directory
Getting this ready for later…
Check if screen is installed
If there’s more than one screen session, you’ll have to enter its name
at the end of the next command in Step 5.
Start a screen session
Consult the GNU screen manual
(https://www.gnu.org/software/screen/manual/screen.html).
Having done this you should be presented with an empty irssi window.
Essential settings
Enter these commands in your irssi prompt.
/SET nick <nick>
/SET alternate_nick <alternate nick>
/SET user_name <username> # not necessary if ident enabled
/SET real_name <real name>
/SET use_msgs_window OFF
# Highlight lines containing your nick
/HILIGHT <nick>
# Set timestamp to something useful
/SET timestamp_format %d.%H:%M:%S
Set up logging
Logging is useful because the IRC connection is not persistent over
system restarts (logging will allow you to see any messages you missed
between the last time you checked IRC and the time the system
restarted). It’s also useful if you’re receiving e.g. tech support, so
you have a record of any instructions you were given!
/SET autolog ON
/SET autolog_level ALL -CRAP -CLIENTCRAP -CTCPS
/SET autolog_path ~/irclogs/%Y/$tag/$0.%m-%d.log
Set quit message
/SET quit_message <message>
Default is ‘leaving,’ mine is ‘Scotty, beam me up!’ (which is
actually the closest Kirk ever came to saying the commonly misquoted
‘beam me up Scotty!’ He never said anything remotely like this in the
TV series but said the above in Star Trek IV).
Add network(s)
For example, ‘srcf’.
/NETWORK ADD [-nick <nick>] <Network name (your choice)>
Add one or more servers
The FQDN in our case is irc.srcf.net.
/SERVER ADD [-auto] -network <network name> <server FQDN>
Add one or more channels
Check out
an overview of the channels
on our server to decide which ones to add.
/CHANNEL ADD -auto #<channel name> <network name>
Connect to a server
Once on a server you can use Alt+num, where num is 0-9, to change channel.
Joining channels whilst on a server
Window Positioning
Move around windows until you get them in an order that you like (first
go to the window that you wish to move):
/wm <position to move to>
(or)
/window move <position to move to>
Save settings and layout
Restart irssi and confirm everything is set correctly
You should now arrive at a terminal, where you can type irssi to restart the client and check that everything is working.
Scheduled restarts
You will also need to create the screen session every time the SRCF
reboots, unless you set up an @reboot cron job containing
screen -Udm irssi using crontab -e.
To disconnect from screen without quitting IRC press Ctrl-A, and then
press D. This will place you back onto the command line, where you can
just type Ctrl-D to quit.
When you want to reconnect later use screen -rd after you have logged
in, and your should find your irssi exactly as you left it.
References & recommended reading
- https://www.irchelp.org/irchelp/new2irc.html
- https://www.irchelp.org/irchelp/changuide.html
- https://www.srcf.net/faq/irc#persistent
- https://irssi.org/documentation/manual/
- https://irssi.org/beginner/#c3
- https://quadpoint.org/articles/irssi/
- http://www.kuro5hin.org/story/2004/3/9/16838/14935
- https://www.gnu.org/software/screen/manual/screen.html
Appendix A: useful irssi commands
Whilst nick, server names, networks names, etc are case sensitive,
commands are not, so /NETWORK is precisely equivalent to /network.
-
List current networks: /NETWORK
-
Remove servers (especially useful for getting rid of irssi’s
default servers):
-
See all settings (to verify that there’s nothing unexpected in
there): switch to status window and type /set.
-
Indent (useful with very long lines, so messages can be
distinguished; experiment with exact value):
-
Quit irssi:
-
Leave channel (after having gone to the channel’s window):
(or)
An alternative, but less elegant way, is to just close the window. This
method is needed if it is a split or hidden (e.g. for closing a query
window): /wc or, in full, /WINDOW CLOSE
-
List all channels on IRC server that are public to join (not
good for large networks): /list
-
Set topic: /topic
-
Switch between networks: Ctrl+X
-
List channels known to irssi: /CHANNEL LIST
-
Reload configuration:
/RELOAD
/RELOAD <filename> # for loading a different config file
-
List users in the current channel:
-
Display user information for a given <nick>:
Joins/Parts
A number of useful things can be done here.
If you are in relatively few channels that have problems with people
joining/leaving frequently, then just ignore for that channel:
/ignore #<channel> MODES JOINS PARTS QUITS
/ignore -except -pattern <yourNick> #<channel>
Replace # with the wildcard operator (*) to do this for all channels.
An alternative way for doing this for all channels is to add the
following to ~/.irssi/conf:
ignores = ( { level = "JOINS PARTS QUITS"; } );
For a more in depth discussion of levels, and how to put all
join/part/quit messages into their own ‘junk’ window (freeing up other
windows for discussion), see
https://pthree.org/2010/03/12/irssi-handling-joinspartsquits/
Appendix B: useful screen commands
Screen is an incredibly useful tool for keeping any process (not just
IRC) running after you terminate an ssh section. There are lots of
useful commands listed in the screen manual, and a few are listed below
https://www.gnu.org/software/screen/manual/screen.html
ctrl+a x locks screen.ctrl+a c creates a new window.ctrl+a n switches to the next window.
Suggestions/improvements?
Did you like this or find this cool? We invite you to check out
more tutorials
If you have a better way to join the SRCF IRC server (or any other
suggestions for how we could improve this documentation), send us an
email at support@srcf.net or submit a Pull Request on
GitHub!
This tutorial was based largely on content provided to us by Matthew
Ireland, former member of the SRCF sysadmin team. Thanks Matthew!
Uploading files
Introduction
This tutorial will help you upload
your first files to the SRCF so you can serve them up from your web page
or use them in whatever project you’re working on.
In this tutorial, you will be using SFTP to upload your files. Let’s
get started!
You may or may not have heard of another file transfer protocol called
SCP. Most clients like FileZilla and WinSCP are equally compatible with
this.
Background
FTP stands for ‘file transfer protocol’ and is another method of
connecting to a web server (different to HTTP). It’s the quickest and
easiest way to transfer files between the computer you’re working on
and the web server where they’ll be supplied to people viewing your
website.
Get an SFTP/SCP client
We recommend one of the following:
Some of these might only be available for macOS or Windows or vice
versa.
If you use WinSCP then
this
documentation and also the
win-built documentation (in the ‘Help’ menu) will be handy.
Connecting
If you have any questions about the user interface, we recommend
checking the documentation of the SFTP client you chose in the first
instance. We’ll be more broadly discussing here what you should do, but
won’t focus on one particular client.
- Use
files.srcf.net as the server name (sometimes referred to as
the hostname)
- Give the username and password that were sent to you when your SRCF
account was created.
Uploading a file
When you first log in, you should see the contents of your home
directory on our server. Inside your home directory you will find a
public_html directory — this is where you can put your personal
website.
If you want to upload a group website, you need to connect to the server
in the same way, still using your personal username and password.
However, rather than putting your files in the public_html directory
in your filespace, you should also find a directory named after the
short name used for the group’s account. Inside here you’ll find
another public_html directory which is where the group website files
go.
Please note that it can take up to 20 minutes between uploading a new
site and that site being published on the web. Please be patient.
Did you like this or find this cool? We invite you to check out
more tutorials
or get in touch to tell us what you thought!
If you have any suggestions for how we could improve this documentation
please send us an email at support@srcf.net or submit a Pull Request
on GitHub!
Using SSH to log in
Overview
Just as a butterfly emerges from its coccoon to explore the great
wonders of the world, you, too, will also uncover the great wonders of
Linux with the SRCF by your side. In this tutorial, we’ll show you how
to log in to a UNIX-like shell and use the basics of the command line.
Background
Much of the functionality offered by the SRCF is accessible via a
text-mode ‘shell’ interface on our server — using this interface you
can manipulate files on your website directly, as well as running other
programs such as email software. When you get an SRCF account, you can
connect to the server and issue commands by typing them. If you’re not
used to this, don’t worry — it’s much less primitive than it sounds
and very powerful!
Because you are remotely accessing our servers, as opposed to your own computer, we will need a communication protocol. The most widely used one is called SSH, secure shell protocol.
Logging in
Linux or macOS
If you’re running a Unix-based computer (including macOS) then logging
in is easy as it’s built into the system:
- Start your terminal
- Type in
ssh <username>@shell.srcf.net
- Enter your password when prompted
You will not see your password as you type it. This is intentional!
Windows
If you’re running a system without a native console, such as MS
Windows, then things are slightly trickier. Not much though, you just
have to go and get a console program. We recommend the PuTTY
application, downloadable for free
here.
Once downloaded,
- Run PuTTY
- Enter the server name as
shell.srcf.net, the protocol as SSH
(port 22) and click to connect
- Enter your username and SRCF password when prompted
Note that for obvious reasons your password will not be printed on the
screen. The password is case-sensitive, so make sure that caps lock is
not on if you are having problems.
First steps
Now you’re in! Let’s see what this terminal is capable of!
Try running ls. You should see a bunch of text printed in the
terminal. These are your directories and files. By default, you start
out in your home directory once you log in. The absolute path is
/home/CRSid. You should, at minimum, see a public_html folder. As
you may have already discovered ls stands for list! You can pass in
additional flags to change its output: ls -la.
Now, try runnng pwd. This stands for print working directory. Think
of this as a street sign. Right now, you’re somewhere in the middle of
complex and multi-layered file system on our shell server, which happens
to be your home directory. pwd tells you where you are in this
organized mess!
The last command we’ll try is cd public_html. This changes your
directory to public_html. You’re now on a different street!
The public_html directory is special. Any content placed here will be
served by our web server, Apache. Check out the tutorials to learn what
to do
You can use cd with either absolute or relative paths.
. represents your current directory. So cd . should not do anything,
you can verify with ls or pwd to see if you’re in the same
directory. If you want to move up a directory you can use cd ...
If you want to go to your home directory you can use cd ~.
~ has a special meaning like .. does, it points to your home
directory. Can you guess what cd ~/public_html does? Try it out and
see if you were right!
If you’re interested in what else you can do with the shell, check out our more advanced tutorial.
Did you like this or find this cool? We invite you to check out
more tutorials
or get in touch to tell us what you thought!
If you have any suggestions for how we could improve this documentation
please send us an email at support@srcf.net or submit a Pull Request
on GitHub!
Virtual desktops over VNC
Introduction
The SRCF desktop server shell.srcf.net has VNC server software
installed (we’re using TightVNC). This
allows you run a graphical desktop on our server, and connect to it
remotely.
However, TightVNC does not encrypt sessions, so you must connect through
SSH in order to avoid possible eavesdropping over the internet.
To start it up, you’ll first need to be able to connect over SSH to the
SRCF desktop server. Linux and Mac should already provide the ssh
terminal command; Windows users may download
PuTTY for a
native SSH client.
Setting a VNC password
SSH to shell.srcf.net using your SRCF username and password (SSH keys
for your account are also valid), then run the vncpasswd command:
spqr2@pip:~$ vncpasswd
Using password file /home/spqr2/.vnc/passwd
Password:
Verify:
Would you like to enter a view-only password (y/n)? n
You can say no to a view-only password, for this use case it’s unlikely
to be useful.
Starting a new session
To start a new VNC server process in the background:
spqr2@pip:~$ vncserver -geometry 1920x1080
New 'X' desktop is pip:12
Starting applications specified in /home/spqr2/.vnc/xstartup
Log file is /home/spqr2/.vnc/pip:12.log
Geometry is of the form <width>x<height> and may be customised as
desired (the VNC window cannot be resized after you have started the VNC
server).
Take a note of the :12 in the output; this is the display number
assigned automatically to your VNC server and could be any two-digit
number.
Add 5900 to the display number to obtain the port number (in this
example, the port number is 5912).
Note that our configuration imposes some additional options upon your
VNC server for security: specifically, -localhost -nolisten tcp, to
prevent insecure direct connections to your VNC server from outside of a
SSH connection.
Tunneling over SSH
As noted above, VNC server connections are insecure. Instead, you should
use an SSH tunnel – this provides encryption over the wire for free,
and allows you to securely authenticate to, and make use of, your VNC
server instance.
The aim is to forward your VNC port on the desktop server, to a port of
your choosing on your local machine.
Linux and Mac
Back on your local machine, SSH to shell.srcf.net again, but request
forwarding for your chosen port:
you@home:~$ ssh -N -L 5901:localhost:5912 shell.srcf.net
Replace 5912 with the port number for your VNC server (the display
number shown when you started vncserver, plus 5900).
(5901 is your local port on your home machine; leave that set as
shown. Adding -N is optional; it tells SSH not to give you a shell
prompt.)
Windows
In PuTTY’s main window, go to ‘‘Connection’’ > ‘‘SSH’’ >
‘‘Tunnels’'.
Under ‘‘Add new forwarded port’', set the source port to 5901 (the
local port on your home machine), and the destination to
localhost:5912 replacing 5912 with the port number for your VNC
server (the display number shown when you started vncserver, plus
5900).
Connecting with a VNC client
At this stage, you should now have port 5901 on your local machine
acting as a tunnel to your VNC server instance on shell.srcf.net. The
last step is to actually point your VNC client to it.
If you don’t have a VNC client installed, this Ubuntu
page lists some
possibilities, though may be out of date.
Start your client and connect to localhost:5901 (this is relative to
your home machine). If all is well, you should be prompted for the VNC
password you created at the start, after which you’ll be greeted with
your new desktop.
You can disconnect and later reconnect, and the state should be
preserved.
Ending the session
Back over SSH to shell.srcf.net:
spqr2@pip:~$ vncserver -kill :12
Replace :12 with the port number for your VNC server.
Make sure to close any running programs over VNC beforehand, as this
will forcefully kill them too.
Statically generated sites
Introduction
Broadly speaking, there are two types of content that can be served by a web server: dynamic and static. Some websites such as social media or online banking are inherently dynamic by nature; the pages and content you see there are constantly changing. But other sites often don’t have any dynamic content and simply deliver static information to users. Using a web application or fully-fledged content management system for the latter can lead to poor performance, for example slow page load times and unnecessary CPU load on the web server.
Overview
Static site generators have emerged as powerful tools which let you turn simple markup into professional and performant websites without having to write cumbersome or repetitive HTML. They allow you to leverage a variety of markup formats, from the simple to the powerful, to produce websites with consistent styling that load quickly. And you’re not totally precluded from having dynamic content in your site; Gatsby is an example that has good integrations which can insert new content into your pages.
A few reputable SSGs out there include:
Did you like this or find this cool? We invite you to check out more tutorials or get in touch to tell us what you thought!
If you have any suggestions for how we could improve this documentation please send us an email at support@srcf.net or submit a Pull Request on GitHub!
WordPress from scratch
Overview
In this tutorial, you’ll learn how to create a fully functional
blog/personal website using WordPress. This is a great first project if
you’re a user starting out with the SRCF, websites in general or want
to use a tried-and-tested Content Management System.
This tutorial assumes you have an SRCF personal account (with or without
group account admin permissions) and can SSH and transfer files. See our
other tutorials for that.
WordPress has huge user and
development bases meaning that there is a lot of help available, as well
as a plugin to handle just about any desired function. WordPress powers
35% of the web!
That said, it also means that it is a frequently targeted platform for hacks and malicious attacks. Take the appropriate steps we mention here to secure your installation and if you are simply serving static content, consider using a static site generator.
Setting up your database
WordPress needs a MySQL database to run. This works a bit like Excel,
storing all of the data for your website in tables and loading this into
templates when each page is loaded. You can create your MySQL database
here.
Sanity check
Now that you’ve got everything setup, let’s check if you can log in
with an FTP client. You’ll want to connect to files.srcf.net via SFTP
(the S stands for secure). You should see a folder called public_html
which we’ll be uploading all of our files to.
Next up, check that you can access your database: log in to phpMyAdmin.
This is a web-based control panel where you can view and edit your
database manually. You should see your personal database on the left
along with any group account databases. They should all look pretty
empty at this point.
If everything is working so far and you have all of your login details
and passwords, we’re good to go! Congratulations, that was the hardest
bit of the whole process…
Installing WordPress
Head over to wordpress.org and hit
the big blue download button. Make sure you don’t get confused with
wordpress*.com* - that’s where you can set up a blog directly with
the WordPress organisation using their hosting. We want to host our own
copy of WordPress on the SRCF.
Once your .zip file has finished downloading, we need to get the files
from your computer onto the SRCF hosting. Unzip the file on your
computer and upload the files into your public_html directory, making
sure that they’re not in a subfolder called wordpress.
You can do this alternatively in the terminal by typing
wget https://wordpress.org/latest and extracting the files, e.g.
tar zxvf wordpress-5.5.1.tar.gz.
Great! Now if you head to your website in a browser
https://**crsid**.user.srcf.net for an indvidual account or
https://**groupname**.soc.srcf.net for a group account) you should see
a friendly welcome screen. Fill in all of the details that it asks for -
leave localhost as it is, try not to use wp_ as the database prefix
or admin as the administrator username (this makes it harder for
hackers to infiltrate your site).
Set up automatic updates
WordPress will likely refuse to do automatic updates, instead requesting
FTP credentials – this is because it assumes it can’t write its own
files on the server. To make this work you can add the following few
lines to your wp-config.php, near the bottom but above the line that
says ‘stop editing’:
# Force WordPress to use direct filesystem access so that upgrades work properly.
# https://core.trac.wordpress.org/ticket/10205
# https://codex.wordpress.org/Editing_wp-config.php
define('FS_METHOD', 'direct');
define('FS_CHMOD_DIR', (02775 & ~ umask()));
define('FS_CHMOD_FILE', (0664 & ~ umask()));
If configured correctly, the updates page should include text similar to
Future security updates will be applied automatically.
If you have a low-traffic or private site, WordPress’ cron service
(which handles updates and other background tasks) may not run regularly
enough. You can invoke wp-cron.php manually using cron or systemd
timers – see crontab or wordpress-cron.timer in
/public/societies/sample.
If you are installing WordPress for a group or society then you will
need to ensure that the permissions on files in your group account
directory are group-writable (and therefore writable by the user that
WordPress will run as). WordPress will not check in advance – if some
of your files are writable and some are not, you will end up with a
half-upgraded WordPress. You can ensure the permissions are correct by
running srcf-soc-permfix socname.
Secure your install
There are crucial steps you must take to secure your WordPress installation. As this applies to existing WordPress installations, we’ve written a separate guide for it.
Customising WordPress
You’ll probably want to make your site look a little different and
certainly contain some content, so the first thing to do is log in to
the WordPress administration panel. There’s usually a link saying
something obvious, but if you can’t find it the URL should look like
https://csrid.user.srcf.net/wp-admin. Log in with the username and
password set up in the previous step.
Content
How you create content will depend on how you intend to use the site.
Posts are designed for small regular news updates, Pages are for
larger static content. You don’t have to use either, you can use both.
If you don’t want to use Posts at all, you can display a page on your
homepage in (Settings > Reading).
Visual changes
WordPress websites use themes to customise the way that they look.
Thankfully lots of people are nice enough to write these for you and
give them away for free (there are plenty of paid-for themes a google
search away too). You can browse these directly through the WordPress
admin back end (Appearance > Themes) or find one online.
Features
If you want to extend the functionality of WordPress, you need to
install a Plugin. Millions of plugins have been written and are dead
easy to install. The chances are that if you want your site to do
something then someone has probably written a plugin to do it.
Accounts
- Log in to WP as
admin using the password given at the end of the
installation process.
- Click on
My Profile and change the admin password using the form
provided.
- Click on
Users and create an account for yourself.
Optional steps
Custom domains
You might be thinking that the *.user.srcf.net or *.soc.srcf.net web
address of your spangly new site isn’t particularly inspiring. Don’t
panic! You can register any domain you like (or even multiple domains)
and point them to your new site by following
our custom domains documentation.
Raven
You can put any site (or parts of your site) behind Raven by following
our Raven docs.
Did you like this or find this cool? We invite you to check out
more tutorials
or get in touch to tell us what you thought!
If you have any suggestions for how we could improve this documentation
please send us an email at support@srcf.net or submit a Pull Request
on GitHub!
A big thanks to Phil Ewels for writing this
on his own blog and to hmw26 for some of their tips. This tutorial
contains some adaptations to their work.
Recommended learning resources
For more general questions, including how to build a website or use the
Linux command line, a number of tutorials are available on the web.
The UIS provide a number of relevant training
courses and
online
resources.