SRCF Documentation

This is a slimmed-down, reader friendly version of the SRCF docs ideal for searching or printing to a PDF.

Last built on August 25, 2024

Table of Contents

Guides

These are the Guides that the SRCF has produced for its services.

Changes in 20.04 Focal

Server changes

The following shell-accessible servers, currently on Ubuntu 16.04 'Xenial Xerus', will be upgraded to 20.04 'Focal Fossa':

  • pip (aka. shell.srcf.net), the main shell server
  • sinkhole (aka. webserver.srcf.net), the web server

As we already have doom, a 20.04 server for games and long-running processes, we'll be decommissioning the old games server cavein shortly after the upgrade. Please migrate your servers and scheduled tasks ahead of the upgrade.

We'll also be upgrading other servers hosting various SRCF services, which will experience some downtime:

  • blizzard, the DNS server (though access to other services on the srcf.net domain will be unaffected by this)
  • flame, the news server
  • squirrel, the MySQL server

Software updates

If you’d like to check what version of a particular package will be installed after the upgrade, you can search for it on the Ubuntu package directory. If a package is listed for xenial, but not for focal, then it is liable to be removed following the upgrade -- contact the sysadmins if there's a package you need that's slated for removal, and we can see what can be done.

Name Current version New version Links
Apache 2.4.18 2.4.41 CHANGES file
MySQL 5.5 8.0 Release Notes
Node.js 4.2.6 10.19.0 Changelogs
PHP 7.0 7.4 Appendices (Migrating from…)
PostgreSQL 9.5 12 Release Notes
Python 3.5 3.8 What's New in Python
Ruby 2.3 2.7 Ruby Releases

If you've built or compiled your own software linked to libraries on the system, you'll likely need to recompile them following the upgrade.

Python 2

The majority of Python 2 packages have been dropped by Ubuntu, leaving only a handful of libraries acting as dependencies for packaged software that doesn't run on Python 3. If you're running your own software with py2, you should seek to make it compatible with py3 -- official support for py2 ended in January 2020, with setuptools, pip and other core components becoming py3-only.

Failing that, the core interpreter is still available, along with the ability to create py2 virtual environments if you're currently relying on any system-installed py2 packages that will no longer be available. Beware of upgrading the build tools in such environments -- the last version of pip to support py2 is 20.3, and for setuptools the last compatible release was 44.0.0.

Python 3

Python 3 itself will receive a minor version bump from 3.5 to 3.8 -- the What's New section of the Python documentation lists all the changes between versions.

The change in version also changes the library path, which means any packages user-installed to ~/.local/lib/python3.5 (with pip install --user or otherwise) will no longer be accessible and will need reinstalling following the upgrade. In addition, any virtual Python environments using the system copy of py3 will stop working, and will need recreating. Custom installations of Python (e.g. those installed with tools like pyenv) may also need reinstalling following system library updates.

PHP

PHP will be updated from 7.0 to 7.4. A list of new features and backwards-incompatible changes can be found in their documentation.

Notably, this introduces a number of new syntax features, such as arrow function shorthand and the null-safe ?. operator. The majority of backwards-incompatible changes seem to cover previously unspecified or erroneous behaviour, though consult the Backward Incompatible Changes sections for all the details.

Node.js

Node is being brought into the (relatively) modern times, to be updated from major version 4 to 10. The canonical binary name will change from nodejs to node, though the former will remain as a symlink.

Unsurprisingly there will have been a number of additions and changes -- Node's changelogs cover all the details but are quite verbose; there's also this list of breaking changes if you just want to check your code for compatibility.

Upgrading a Drupal installation

Overview

The notes below assume your Drupal site is at drupal.soc.srcf.net -- substitute this for your own site’s URL.

Drupal provide their own extensive guide for upgrades, which covers the below information in more detail.

Upgrading files in-place

This method is suitable for minor version upgrades (e.g. from an old 7.x to the latest 7.x release). See also: Update procedure (minor version change)

Take backups of your files and database before attempting an upgrade. Then put the site into maintenance mode.

Download the latest release from https://www.drupal.org/download. The ZIP file contains a folder called drupal-<version>, which contains the core files.

If you're working over SSH, copy the URL of the ZIP download, use wget to download it, then unzip to unpack it in the current directory.

For SCP, you'll want to extract the files on your local machine, then upload them to the server.

In either case, the contents of the drupal folder are to be moved into your existing site (for drupal.soc.srcf.net, this would be /societies/drupal/public_html). You should delete all files under the site directory, '''except''' the sites folder which holds all customisations. If you've made any changes to .htaccess or robots.txt, you'll also need to merge these into the updated files.

Once the new files are in place, head over to https://drupal.soc.srcf.net/update.php to prepare the database for the new version. When complete, take the site out of maintenance mode.

Migrating a site between instances

This is needed for major release upgrades (e.g. from 7.x to 8.x).

Similar to the above, download the latest version, but extract/upload it to a different location than the current site. Configure this as a new installation. It still needs to be under your account's public_html root, so you may need to use a subdirectory. This page assumes you have https://drupal.soc.srcf.net/new/.

Go to the ''Modules'' section on the new site, and enable the ''Migrate'', ''Migrate Drupal'', and ''Migrate Drupal UI'' modules.

Now head over to https://drupal.soc.srcf.net/new/upgrade, which will prompt for database credentials -- here, you can specify those of the old site, which will import all its content into the new installation.

Once you're happy the new site is working, you can move the old one out of the way and make the new site take its place. You'll need to update the $base_url in settings.php with the new path.

Upgrading WordPress

The notes below assume your WordPress site is at wordpress.soc.srcf.net -- substitute this for your own site's URL.

WordPress provide their own extensive guide for upgrades, which covers the below information in more detail.

Allow WordPress to write its own files

WordPress includes an update utility, but this can be quite tedious about not correctly detecting that it can update itself; it may fail without even trying, especially on society accounts.

To make this work, you can add the following to your site's configuration (wp-config.php), near the bottom but above the line that says "stop editing":

/**
* Force wordpress to use direct filesystem access so that upgrades work
* properly. See: https://core.trac.wordpress.org/ticket/10205
* https://codex.wordpress.org/Editing_wp-config.php
*/
define('FS_METHOD', 'direct');
define('FS_CHMOD_DIR', (02775 & ~ umask()));
define('FS_CHMOD_FILE', (0664 & ~ umask()));

You must ensure all WordPress files have the correct group permissions. For society accounts, this means all files should have the society's group and be group-writable. WordPress will not check this in advance -- if it fails to write some files during an upgrade, you may end up with a broken instance.

This should be the case by default, but some SFTP clients may override it. In addition, files moved/copied from your home directory may still have your personal group set, rather than that of the target society.

Upgrade through the admin panel

Head over to https://wordpress.soc.srcf.net/wp-admin/, log in with your WordPress username and password if you haven't already (remember, these are distinct from any SRCF-provided accounts).

Select ''Updates'', under ''Dashboard'' in the side menu.

If an update is available, you should have a button to perform the upgrade in one step. If you've installed a non-US-English version, make sure to pick the one with the correct language.

If you're prompted for SFTP credentials, see the section above for how to grant direct file access.

Upgrade files manually

Take backups of your files and database before attempting an upgrade.

Download the latest release from https://wordpress.org/download/. The ZIP file contains a folder called wordpress, which contains the core files.

If you're working over SSH, you can just wget https://wordpress.org/latest.zip to download the ZIP file, then unzip latest.zip to unpack it in the current directory.

For SCP, you'll want to extract the files on your local machine, then upload them to the server.

In either case, the contents of the wordpress folder are to be moved into your existing site (for wordpress.soc.srcf.net, this would be /societies/wordpress/public_html). You should delete the wp-admin and wp-includes folders ('''not''' wp-content) first, as the new files provide complete copies of these folders. The contents of wp-content should be merged with the existing directory, in order to retain plugins and themes.

Once the new files are in place, head over to https://wordpress.soc.srcf.net/wp-admin/ and log in with your WordPress username and password to check if things are working. You may be prompted to complete a database upgrade -- this is normally a one-click step, just follow the instructions it provides.

Web server migration

Apache

Upgraded from 2.2 to 2.4. See Apache's own upgrade documentation for a full list of changes.

The semantics of Allow/Deny directives have changed -- we have ''mod_access_compat'' enabled which should restore the legacy syntax, though we recommend you migrate to the new Require syntax.

Logs

You'll now need to SSH to webserver.srcf.net to access logs -- you will find access.log and error.log in /var/log/apache2/user/<crsid> for your personal website, or /var/log/apache2/soc/<name> for a society website. Note that CGI error output should now consistently go to the corresponding error.log file (previously some types of output ended up in the main log at /var/log/apache2/error.log).

Proxy webservers

If you run your own webserver (e.g. gunicorn, nginx) and have Apache proxy to it via htaccess, these server processes need to be run from the webserver rather than the shell server. As above, SSH to webserver.srcf.net and run them from there.

PHP

Upgraded from 5.3 to 7.0. This jumps multiple minor versions -- the bulk of the changes can be found in PHP's own migration guide for 5.6 to 7.0, though see also the guides for 5.4, 5.5, 5.6.

MySQL

The original MySQL library, which provides methods with names starting mysql_, has been deprecated since 5.5, and removed completely in 7.x.

MySQLi provides a close replacement, with most original methods present but prefixed with mysqli_. You'll also need to pass a connection handle (the return value of mysqli_connect()) as the first argument to most of the remaining methods.

Sample MySQL library code:

mysql_connect("localhost", "<user>", "<password>");
$result = mysql_query("...");
while ($row = mysql_fetch_assoc($result)) { ... }

Equivalent MySQLi code:

$conn = mysqli_connect("localhost", "<user>", "<password>");
$result = mysqli_query($conn, "...");
while ($row = mysqli_fetch_assoc($result)) { ... }

ucam-webauth-php

Versions prior to 0.53 include a function call with an argument ignored in PHP 5.x but removed in 7.x. Error logs will contain something like:

PHP Warning:  gmmktime() expects at most 6 parameters, 7 given in .../ucam_webauth.php on line 388

The latest version can be obtained from GitHub.

php_override.ini

PHP is now run as an Apache module rather than CGI, so override files are no longer supported. You can however override settings using htaccess.

Example php_override.ini setting:

key = value

Equivalent htaccess setting:

php_value key value

PostgreSQL

The Postgres server has also moved to the new webserver. For websites connecting to a database, it's still accessible locally (socket connection) so access should be unaffected.

If you have scripts on the main shell server that access a Postgres database, these need updating to connect over TCP to hostname postgres. Ident auth is available so you still shouldn't need to provide a password.

Securing WordPress

WordPress is a regular hackers’ target and instances of WordPress on the SRCF are regularly compromised in various ways, leading to private data leaks, tampered files, and website irregularities. Most of these risks can be minimized by following the hardening steps below.

  1. Ensure your wp-config.php is not world-readable, as that will contain your database credentials. You can set permissions in your FTP client or in the console by typing chmod 0660 wp-config.php. If you are installing Wordpress under web space that belongs to a group or society then your wp-config.php file will be owned by that group or society’s user account rather than your own personal account, in which case you will need to run the previous console command as that user: sudo -u socname chmod 0660 wp-config.php. Unfortunately FTP clients won’t let you do this so you will need to use the console.
  2. It is advised to lock down WordPress’ admin panel at /wp-admin/ by putting that directory behind Raven authentication – see an example at /public/societies/sample/public_html/wordpress/wp-admin/.htaccess.
  3. We also recommend you disable Allow link notifications from other blogs (pingbacks and trackbacks) on new posts, under Settings > Discussion in the admin panel.
  4. Activate a spam filtering plugin like Akismet and a capatcha system like reCAPTCHA. Akismet is installed by default and just needs activiating. Go to .../wp-admin/plugins.php to install and activate plugins.
  5. You may optionaly want to modify your theme so that it no longer puts the Wordpress version into the html - this may help stop hackers finding that you installation is outdated but it does not protect against problems caused by the version you are using being compromised.
  6. Regularly check up on the status of your installation and keep an eye on any vulnerabilities in the plugins you use.

Society Handovers

Overview

Ensuring that a proper handover occurs will reduce headaches for the new committee in the long term. This guide explains a few simple pointers to keep in mind during the transition period.

New webmasters

It is becoming increasingly common that the role of “webmaster” is merged into the job of the secretary or the publicity officer. This often means that modifying the website becomes a daunting task due to unfamiliarity with the relevant technical aspects and/or the SRCF. We’ve tried to lower the barriers to entry by writing helpful tutorials with beginners in mind. If you are new to this, please do not be discouraged as we are here to help! We offer several ways of getting help, including a live chat.

As a quick introduction, the Student-Run Computing Facility is best known for providing free web hosting and email forwarding for students in Cambridge.

Granting access

The most important part of the handover is for the incumbent webmaster to add you as an admin to the society account. This can easily be done by:

  1. Navigating to our control panel
  2. Under “Your group accounts”, click “Manage” on your society’s card
  3. Under the “Administrators” card, click “Add new administrator”
  4. Type in the CRSid of the new admin

This process implies two things: first, the new webmaster must have an SRCF account beforehand (easily done by going to our home page and clicking “Create an account”) and second, any society admins must have Raven access, or in other words be member of Cambridge University.

If possible, you may consider keeping the old webmaster as an admin on the account. We recommend two or more admins to ensure the longevity of the society account in case one or more admins become unreachable.

If you are using a content mangement system like WordPress, this is also the time to add the new webmaster via the web interface as an administrator of the website.

Check details

This is also a good time to check if all society information is correct, such as the role address. This email is used in case the society has no remaining admins and will be our last attempt before permanently deactivating the account. Admins are automatically removed if their email bounces, ie. we can no longer reach them by email.

Files

Once the new webmaster is added, the incumbent webmaster should explain how the website works and how it can be updated. A few written notes on this can go a long way towards facilitating the handover.

In the majority of cases where a content management system is used (like WordPress), “updating” can be done via the handy web interface. If something goes wrong though, it is necessary to log in and poke around at the files. We’ve written tutorials for this.

At this point, the new webmaster should also read our reference material on group accounts, especially on file permissions and how our web services work. The gist of it is that all content gets served from /public/societies/<socname>/public_html and private society files are located at /societies/<socname>/. Shortcutes (symbolic links) to these are automatically added to your home directory.

Email forwarding

A new committee means new emails! If you are making use of email forwarding in the form of socname-chair@srcf.net then you need to update where these emails forward to. This is done in the .forward file present in the society’s home directory and we have more information on that here.

Mailing lists

If you are making use of mailing lists of the form listname@lists.srcf.net then you need to:

  1. Share the admin password with the new webmaster
  2. Add the new webmaster as a mailing list admin

This will allow them to, for example, send email to the list members without verification, manage requests from other committee members to the mailing list, and most importantly grant them access to the mailing list admin interface available at https://lists.srcf.net.

Other steps

There are other steps involved in a society handover, like social media handovers, chat groups (check out our Mattermost!), file sharing and more. These are beyond the scope of the SRCF but we mention it here to stress the importance of having a proper handover document (even if brief!) as these details easily get lost over time.

As we have experience with many societies, we are happy to advise on any further steps – just reach out to us!

Getting Started: Groups

Note for beginners

It is becoming increasingly common that the role of “webmaster” is merged into the job of the secretary or the publicity officer. This often means that modifying the website becomes a daunting task due to unfamiliarity with the relevant technical aspects and/or the SRCF. We’ve tried to lower the barriers to entry by writing helpful tutorials with beginners in mind. If you are new to this, please do not be discouraged as we are here to help! We offer several ways of getting help, including a live chat.

As a quick introduction, the Student-Run Computing Facility is best known (though we do other things!) for providing free web hosting and email forwarding for students in Cambridge.

The above is copied from our society handovers guide.

Users vs. groups

Even as a society webmaster, or someone looking to setup a website for a group, you are first and foremost a user. This means that our getting started guide for users is entirely applicable (and useful) to you. The main difference is that your public_html is now a shared between all the admins of your group.

In your home directory, a shortcut (symbolic link) to your group’s file space is automatically created, and in it is the public_html you will want. This is also a shortcut to your group’s public file space, which contains the actual public_html directory. Here is a useful breakdown that also includes the absolute file paths:

spqr2@pip:~$ ls -l
mysociety1 -> /societies/mysociety1
mysociety2 -> /societies/mysociety2
public_html -> /public/home/spqr2/public_html
spqr2@pip:~$ cd mysociety1
spqr2@pip:~$ pwd -P
/societies/mysociety1
spqr2@pip:~$ ls -l
public_html -> /public/societies/mysociety1

Your options

The fact that you are given full access to our webserver means that you have many many different possibilities in the types of websites you can host: dynamic or static, custom made vs. a standard content management system (CMS), PHP vs. Python, the list of options goes on.

There are different usecases, and different levels of technical skill required by each, so choose the one the makes the most sense for you and your group. A good chunk of societies tend to favor the PHP-powered WordPress CMS, which is why we’ve written a special guide for it to make sure you install it properly and securely.

Getting Started: Users

Introduction

If you are a new user and wondering what to do, this guide is your first port of call. Here, we go through the process of getting familiar with the main services the SRCF offers you and how to work with them. It is worth mentioning that we have various tutorials that break down the steps shown here into much more detail, with a beginner-first approach in mind.

Create an account

To start with, you’ll need to create a personal account. This will allow you to use the control panel to manage our various services.

Connect to the shell server

The primary way to connect to our public facing servers is via a protocol known as SSH. Make sure to read our tutorial on that, and come back once you’ve finished!

Upload your first website

Now, let’s create your first website on the SRCF. Remember the public_html directory you saw in your home directory when you first logged in? Any file in that directory is served by sinkhole, our webserver, on the domain <yourCRSid>.user.srcf.net. By default, webservers look for an index.html page to serve, otherwise they’ll list the files in your public_html directory, which isn’t very user friendly.

First, ensure that you are now in the public_html directory by changing into it: cd public_html. To create a simple file, we use the text editor “nano”, by typing the command followed by the name of the file we want to create: nano index.html. You are now freely able to type any text you wish, so let’s add the following HTML by copying and pasting into your terminal.

<!DOCTYPE html>
<html lang="en">
    <meta charset="UTF-8">
    <title>I love the SRCF</title>
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <body>
        <h1>Hello there!</h1>
    </body>
</html>

Now press “ctrl-o” then enter to confirm the file name, followed by “ctrl-x” to exit the editor. You should now be able to see a “Hello there!” on your SRCF domain.

Now what?

Believe it or not, the basic skills shown here cover most of what you should need to make use of our shell server and our webserver. You are welcome to browse our other tutorials for more advanced information, or simply play around with the Linux environment we offer you!

Internal

This is the internal documentation for the SRCF. It contains non user-facing information that is more relevant to current and future volunteers.

How it works

Overview

These docs use the Hugo static site generator (SSG) and are hosted on our web server sinkhole. The content is split into folders that are logically organized based on the nature of their content.

History

These docs evolved from the original “Frequently Asked Questions” pages that were available from the mid 2000s to 2020. Most of the dated information has been removed but some historical artifacts remain.

Around March of 2020, the FAQs were re-organized into a series of pages that addressed technical aspects of our services. These now form the bulk of what is now the “technical reference”. They were generated with Sphinx and accordingly written in reStructuredText (the choice for this was nothing other than an academic exercise at the time). Over time, though, as the docs were expanded, it became apparent that:

  • certain user issues arose more than others and consequently needed more detail in the docs
  • step-by-step instructions were beginning to emerge in some pages, mixed in with other content
  • there was a growing need for tutorials and guides after a few were written up and positive feedback was received

These and a few other reasons like the ability to write in Markdown, the flexibility of SSGs, and efforts to lower barriers to entry, led to the creation of these docs (a.k.a “docs v2”).

Project structure

An important step to the second iteration of our documentation was addressing different types of information, nicely summarized by this philosophy. Accordingly, the content folder has subfolders to match the type of content it houses.

It helps to be familiar with Hugo’s directory structure, but the two main folders of interest are content and layouts, seen below.

docs
┣ archetypes
┣ content
┃ ┣ guides
┃ ┣ internal
┃ ┃ ┣ documentation
┃ ┃ ┃ ┗ updating-a-page.md
┃ ┃ ┗ _index.md
┃ ┣ reference
┃ ┣ tutorials
┃ ┗ _index.md
┣ data
┃ ┗ sidebar.yaml
┣ layouts
┃ ┣ _default
┃ ┣ partials
┃ ┣ shortcodes
┃ ┗ index.html
┣ resources
┣ static
┃ ┣ css
┃ ┃ ┗ style.css
┃ ┣ vendor
┃ ┗ .htaccess
┗ config.toml

An overview of the purpose of all folders and files:

  • archetypes

    An archetype is essentially a template for a new section entry, like reference, tutorials, and other root-level dirs below content. Hugo looks in this folder when using the hugo new command like so:

    hugo new internal/documentation/another-page.md
    

    and creates an empty page with the pre-filled boilerplate in the archetype.

  • content

    The key is in the name. All direct child directories of content are sections. A section is a special grouping of content in Hugo. All written content goes in here and is automatically turned into html during build. By default, the directory structure here mirrors that of the final build.

    Our content also has a root-level _index.md that sets special properties in its front matter (the yaml enclosed in dashes at the top of each Markdown file). We have a few single .md pages here that are also directly converted as-is into html.

  • data

    Useful for extracting text from repetitive/iterative partials or renders. The sidebar is a good example of this.

  • layouts

    Location for all HTML templates, including partials and shortcodes. Hugo has a strict template look up order.

  • static

    Everything here is copied as-is into the final build.

Content is grouped in the sidebar according to its group property. Sidebar logic then links a page to an entry in the sidebar with this group property. This is because there are times when it makes sense for the file name (and consequently URL) to be different from the sidebar text. In most cases though, a best effort should be made to keep the file name and sidebar entry consistent (urlized versions of each other).

The sidebar

The sidebar sources its text from a yaml data file in the data folder. At a glance, this file has a top-level list of category objects.

# category object
- category: Internal
  # list of groups
  groups:
    # a group called "documentation"
    - title: Documentation
      # pages under this group
      pages:
        - title: How it works
        - title: Building
        - title: Managing content
        - title: Reference

Each category object has a name, which must correspond to the Hugo section name (when urlized), eg. “tutorials”, “reference”, essentially the name of the folder.

Children of the groups list form the “dropdowns” in the sidebar. Children of a group under the pages object, form the individual pages in each group. All names (for groups and individual pages) are mapped to the title attribute.

The sidebar logic uses three pieces of information to match any page to its text data in this file: the slug (in most cases the file name, or when overridden in front matter), the urlized group name (provided as a parameter in front matter), the section (mapped to the category attribute).

Design decisions

  • redirects are added in .htaccess for the old docs version
  • “ugly URLs”
    • initially investigated this option to reduce redirects needed
    • real tests resulted in undesired behavior for index files
    • to avoid hacks that might break in the future, we use the default (uglyURLs: false) and live with the trailing slash

Managing content

Overview

Any contribution to this documentation is welcome. Helping to fix/correct a section is often a good way for newcomers to get around using Git and the terminal.

The below contains instructions for creating or updating content.

Obtaining the source

You should first fork our repository into your own GitHub space and make the desired changes there.

Depending on the complexity of your change, you can either:

  • use the GitHub web UI (click “edit this file”) to make your change. GitHub automatically forks the repository you’re trying to edit.
  • clone your forked repository, edit locally, optionally test your changes, push to your repository and finally make a pull request back to ours

The above process is known as the “GitHub flow” and is standard practice across many distributed projects under version control.

If you are creating pages, then you will need to clone the repository locally.

Changing content

Our docs are written in Markdown, a standard markup format. A quick tutorial is available here.

Editing is an important step, and for that you will need a good text editor. VS Code, Atom, Notepad++ are all reliable choices with good extensions to make typing Markdown even easier. For those interested in using the terminal, nano works great for small edits and vim for a full-fledged text editor.

If you just want to fix a typo or add a few sentences, editing text should be sufficient. To do fancier things with Hugo and build upon our docs, you should check out their documentation. You should also read the additional available pages on these docs.

Creating a page

Use the standard hugo new command to create your page so that the right template (archetype) is used. The template is populated with data like the current date when you create a new page.

To create a tutorial called “How to feed your dog”, you would type hugo new tutorials/animals/how-to-feed-your-dog.md" into your terminal. The title and file name do not need to match, but it’s good to keep things consistent and informative for other contributors.

Commit, push and PR

Now it is time to commit your changes, push them to your repository and make a pull request into our repository. If any of those words are foreign to you, you might want to visit a Git tutorial or browse a quick primer.

Before you commit, make sure your message is informative by following these tips. After you’ve pushed your changes to your local repository, navigating to your repository on GitHub should show a prompt asking you to make a pull request back to upstream (us). If not, then you can create your own pull request manually. Be sure to add informative details as to what the nature of your PR is and what issues it solves (if any).

Updating the sidebar

  1. Add an entry in the sidebar data file

    • If you are adding a standalone page, just add a - title: <something> to the groups list.
    • If you are adding a page in a group, add its title under the pages list.
    • If you are adding a new group with child pages, add a title for your group and a pages list attribute.
  2. Add the group: parameter in the page’s front matter

    This must correspond to the urlized group title in the data file.

Test building

See the building section.

Reference

Overview

This is a reference of all quirks and features specific to our documentation.

Shortcodes

Pre-defined bits of HTML that the user can easily inject into the Markdown.

Alert

{{< alert type="info" >}}
    some content in here
{{<  /alert >}}

type can be any valid Bootstrap alert type. The above produces

Tables

Default Markdown tables don’t come out looking very nice, so we use a shortcode to add Bootstrap classes to them.

Shortcode borrowed from here.

Example:

{{< table table table-striped table-bordered" >}}
|———-|———-|———-|
| Item 1   | Item 2   | Item 3   |
| Item 1a  | Item 2a  | Item 3a  |
{{< /table */ >}}

Front matter params

Front page favs

highlight: true adds a page to the lists on the home page.

A quick glance at the structure of the yaml file should reveal most of the complexity.

  • divider: true within any group (ie. direct children of the groups list) adds a divider
  • override: /path/to/page changes the path used to search for a page’s permalink when the sidebar is generated. Used on pages like section home page where the slug cannot be changed.

Other tips

Cross references

Use a regular Markdown link but with the relref shortcode as such:

[mail forwarding]({{</* relref "#mail-forwarding" >}})

Syntax highlighting

Fenced code blocks should use triple back ticks ``` and the language always included after the first set of back ticks. A full list of valid languages for special highlighting available here.

Building the docs

To build our docs, you will need the Hugo binary. A binary is the bundled up version of a program which you can run as a command, eg. hugo config.

Building on pip

We have hugo binaries available on pip, so if you’re building there, there’s no need to install a binary and you can run hugo as per usual.

Building locally

Sometimes it is useful to preview changes locally, either for convenience or necessity (lack of internet).

  1. Grab Hugo with these instructions
  2. Extract the tarball: tar -xvf filename.tar.gz
  3. Run: ./hugo

The above is easy if you’re on a UNIX-like system, such as any Linux distro or macOS. If you’re on Windows, then Windows Subsystem for Linux or any terminal emulator for Windows (Microsoft Terminal is good) will do just fine but can be fiddly.

Building in production

The production version of these docs is available at https://docs.srcf.net.

There is a Makefile to facilitate generating the docs and associated files.

Reference

This is the technical reference for the SRCF. It contains essential information on our services.

Before using any of our services, you should consult our rules and policies on the left.

Administrators

Each group account has a list of admins: SRCF users who manage the account and its services. This includes access to the account in the control panel, permissions to edit files inside /societies/<groupname>, and receiving administrative email sent to <groupname>-admins@srcf.net. It may also be used by an account’s website to limit page access to current admins.

This list is the canonical definition of who is responsible for a group account, and is independent of any committees or roles in a University club or society that a group account on the SRCF may happen to represent.

Handover

We recommend that you keep your own handover documentation, covering what services and software a group account is using.

It is the responsibility of SRCF group account admins to keep their list of admins up-to-date, meaning both the appointment of their successors, and the removal of predecessors that should no longer have access.

SRCF system administrators will not process requests from users wishing to gain access to a group account, instead directing them to the account’s current admins to make their request.

Unmaintained group accounts

In order to maintain continuity of group accounts, we recommend keeping at least two admins at all times.

When a group account admin leaves the University without taking steps to retain access to their SRCF account (which includes emails to their membership address bouncing, or otherwise not responding to emails from the system administrators), they may be dropped from the list of admins, and their account suspended.1

A group with no admins is considered unmaintained, at which point some services (such as website hosting) will be suspended. If the group account is configured with a role email address, it may be contacted as a last resort, to inform the recipients of the situation and for them to nominate a stand-in admin and reactivate the site.

The SRCF has the responsibility as a data processor to prevent unauthorised access to personal data. If we’re unable to verify the handover of an unmaintained group account (for example, if a group account has no contactable admins and no role email to fall back to), we will not be able to provide access to its contents. As a user seeking access to such an account, generally this means you will have to register a new group account and start from scratch.


  1. If your account has been suspended for being uncontactable, you can reactivate it from the Control Panel. Note that you won’t automatically regain access to your group accounts – please contact the sysadmins to request these. ↩︎

Administrators

Society accounts have been renamed to group accounts. This page has been moved.

After graduation

Membership of the SRCF as a society is for life. Access to our services is initially granted to current members of the university, but accounts will remain active, including after graduation or otherwise leaving the university, as long as we have a valid contact address for their owners.

If you’re leaving the university and losing your @cam.ac.uk email address, make sure to update the email address we have on record for you to something external. You can login to the SRCF control panel and change your registered email address on your member page. If you use mail forwarding using a .forward file or via Hades, check that your mail is routed to a working address too.

The system administrators need to be able to contact you about your account, and may suspend accounts without warning if they are unable to reach you.

Apache (web server)

Overview

We run a central Apache 2.4 for all personal and group websites.

As our server handles hundreds of websites, we don’t allow users to change server-level configuration, including Apache configuration in /etc/apache2, as various online tutorials may lead you to.

Unlike most hosting providers, websites are run using your own (or a group’s) user account on the server. This means your site has the permissions needed to write its own files, for example with content editors or automatic updates.

Directory root

Websites are served out of public_html directories, present in your public file space (/public/home/<crsid>/public_html or /public/societies/<groupname>/public_html).

To use Apache directives, you should use a .htaccess file, which provides a subset of functionality at a per-site level.

Logs

Web server logs (including PHP errors) can be found in /var/log/apache2/user/<crsid> for personal users, and /var/log/apache2/soc/<groupname> for group accounts. You can access them over SSH or with a file transfer program.

CGI and PHP scripts

Overview

On the SRCF’s webserver, you can run CGI and PHP scripts! What’s more, we have a brilliant system in operation which allows even your PHP scripts to run as ‘you’ rather than as the webserver. Read on for details.

Location of your CGI files

For personal users, PHP scripts must have a filename ending with ‘.php’ and can be placed anywhere in your public_html directory. CGI scripts must have filenames ending ‘.cgi’ and can also go anywhere in public_html.

For group accounts, both of the above methods also work, however there is an additional method of running CGI scripts which is to place them in the cgi-bin directory in your group account user area (i.e. not in public_html). If you do this, the script names do not need to end ‘.cgi’. The URL to access CGI scripts in the cgi-bin directory is https://<groupname>.soc.srcf.net/cgi-bin, where <groupname> is the group account name.

Users

For personal users, both PHP and CGI scripts will run as you, not as the web server’s user id.

For group accounts, rather than running CGI/PHP scripts as an individual, we have introduced a UNIX user for each group account. This user cannot log in, however it is used as the user under which all group account PHP/CGI scripts are run. Outgoing email generated by group account CGI/PHP scripts will, by default, appear to come from <groupname>-webmaster@srcf.net. See Group email addresses.

Note that any CGI/PHP script which hogs the CPU for more than 2 minutes will be terminated by the system. This should not affect anything other than out-of-control scripts, as CGI/PHP scripts typically execute in a few seconds. It is a measure to prevent the server being excessively slowed down by buggy scripts which go into an infinite loop.

Permissions

For personal users, CGI scripts must be readable and executable by you, and must be owned by you. PHP scripts must be world-readable (but to keep database passwords secret, see the next question). For example:

spqr2@pip$ ls -l
-rwx------  1 saw27  saw27  238 May  5 19:33 env.cgi
-rw-r--r--  1 saw27  saw27  265 May 13 19:34 phptest.php

CGI scripts and PHP scripts belonging to a group account must have their system (UNIX) group owner set to that of the group account. CGI scripts must be system group readable and executable. Group account PHP scripts must additionally be world readable (but to keep database passwords secret, see the next question). For example:

spqr2@pip$ ls -l
-rwxrwx--- 1 saw27  casi  238 May  7 23:49 env.cgi
-rw-rw-r-- 1 saw27  casi  265 May 14 23:06 phptest.php

We recommend that you ensure that group account files are group-writable, so that other admins of the group account can edit them (not least, when your own personal account expires).

Managing secrets

PHP scripts must be world-readable. This requirement is artificially imposed because we felt that if we didn’t require world readability, users might be caught out by assuming that if something (other than a CGI script) is not world readable then it’s not accessible on the web, which wouldn’t be the case for PHP scripts. But it’s easy to get round: put your secret information in a separate file which is not world readable (but is system group readable), and include that file from your main PHP script.

Managing memory

By default we have a maximum memory limit of 64MB set for PHP scripts. You can override this by placing a file called php_override.ini at the top level of your site containing, for example, memory_limit = 128M.

Please consider the memory requirement of other users, system processes, etc. before doing this.

Turn CGI off

Several kinds of file will automatically be interpreted as CGI scripts, and so the CGI handler will try to run them when you visit their URL, even if you just wanted to download them. The following will turn off CGI handling for Python scripts, displaying them as plain text instead:

AddHandler default-handler .py
AddType text/plain .py

Put those lines in a .htaccess file in the same directory as your python files, and they will no longer be considered CGI scripts. You can do a similar thing for other file types by changing the .py to, for example, .php.

Common requests

This is a list of all queries and issues we receive often. Please check we haven’t already answered your question here before contacting the sysadmins.

Contact owners of a personal or group website or account

The SRCF hosts user websites and mailing lists on its systems, and we provide domains of the form CRSid.user.srcf.net or acronym.soc.srcf.net, along with matching email addresses like CRSid@srcf.net and acronym-role@srcf.net.

The SRCF itself is not immediately responsible for content under these domains. In the first instance, you should direct queries to the owners of the relevant account.

  • For websites like spqr2.user.srcf.net or email addresses like spqr2@srcf.net, the responsible individual can be reached directly via spqr2@srcf.net.

  • For websites like sample.soc.srcf.net or email addresses like sample-role@srcf.net, the responsible group of people who manage the account (its “admins”) can be reached collectively via sample-webmaster@srcf.net for queries about its website specifically, or sample-admins@srcf.net for anything else.

If your communication relates to infringing copyrighted material or other abuse, please also CC the SRCF system administrators.

We also host websites on custom domains, which do not end in .user.srcf.net or .soc.srcf.net. Please contact the system administrators if you’re getting in touch about a website hosted by us under a custom domain and are unsure how to reach the site’s owners.

Gain access to an existing group account

Read more about group admins and unmaintained accounts

As a prospective admin seeking access to manage a group account, you should first contact its existing admins and request access from them. See the answer above for how to identify the short name of the account. If you don’t know the short name, you can check this list of group accounts.

If you’re unable to make contact with them, you may ask the system administrators for assistance, who will then attempt their own contact, and may remove inactive admins or transfer ownership as needed.

Group accounts are considered unmaintained if they have no active admins remaining. Accounts in this state will display an Unmaintained group website page if you visit their SRCF-hosted website.

Grant group admin access to someone else

As an existing admin of a group account, you can use the control panel to promote other users. Each new admin will need to create a personal SRCF account first if they don’t currently have one.

If you are stepping down from managing a group account, you can remove yourself as long as there’s at least one additional admin present.

Permission errors writing files in a group account space

Read more about group permissions

You can try to fix permissions yourself using the srcf-soc-permfix tool over SSH, which can fix your own files and those of the group account:

srcf-soc-permfix <groupname>
sudo -u <groupname> srcf-soc-permfix <groupname>

However, this won’t be able to fix files owned by other admins. If you continue to get permission errors, ask the system administrators to run this for you instead.

Problem with an @lists.cam.ac.uk mailing list

Whoa, stop right there! That’s the University Information Services mailing list system, not the SRCF one, and there’s nothing we can do to help you. Try going to lists.cam.ac.uk instead.

Update forwarding addresses on a domain

The SRCF does not (and cannot) provide email forwarding outside of @srcf.net addresses, so we don’t have control over these. You’ll most likely need to consult your domain registrar to do this.

Can you design/upload/run/fix my website?

Unfortunately, our support team does not have the capacity to maintain both our servers and the thousands of websites hosted on it. If you have a specific technical question (e.g. an error message you don’t understand), be sure to seek help.

You may upload a website designed elsewhere to be hosted on the SRCF. However, it must be you who performs the upload – you must not share access to your SRCF account with anyone else, even if they are acting on your behalf.

We recommend societies and groups have appointed webmasters that are in charge of maintaining the website.

Can you help me debug a program?

Only if we have time - our support team are all volunteers, however we generally like hacking code, so if you’re having problems drop us a line and we might be able to help you out.

Custom domains

Overview

As part of our web hosting, we provide free domains for both individuals (of the form <crsid>.user.srcf.net) and group accounts (<groupname>.soc.srcf.net). These are configured as standard, and will serve any web content placed in the public_html directory of the respective personal or group account (subject to a 20-minute delay when publishing a new website).

We also support external domains purchased from a domain registrar. If you are looking to buy a domain for yourself or your society then we can recommend Mythic Beasts but there are plenty of other registrars out there with varied pricing.

You’ll need to make sure the domain resolves to us – this is controlled by DNS records, which your registrar should allow you to configure. If you’ve delegated DNS from your registrar to a third-party service such as Cloudflare, you’ll need to configure it there.

Assuming you want to serve your site from the base domain example.com, your DNS records should look something like the following:

  Subdomain   Type    Value
  ----------- ------- ------------------------------------
  @           A       131.111.179.82
  @           AAAA    2001:630:212:700:2::1
  www         CNAME   webserver.srcf.societies.cam.ac.uk

The first record (A) makes your base domain point to our webserver’s IPv4 address; the second (AAAA) does the same but for IPv6. The third (CNAME) sets an alias from www.example.com to our webserver.

If instead you want to use the subdomain bubbles.example.com:

  Subdomain     Type    Value
  ------------- ------- ------------------------------------
  bubbles       CNAME   webserver.srcf.societies.cam.ac.uk
  www.bubbles   CNAME   webserver.srcf.societies.cam.ac.uk

More information

The preferred method is a CNAME record, which acts as an alias from your domain to ours. The value of this record should be webserver.srcf.societies.cam.ac.uk – this means your domain will continue to work even if we move servers, as our hostname will stay the same. Note that you must remove any other non-CNAME records for this particular (sub)domain.

CNAMEs should not be used on base domains (example.com, sometimes called the zone apex), only on subdomains. The alternative is to use A (for IPv4) and AAAA (IPv6) records to specify an IP address. The A record should be set to 131.111.179.82, whilst AAAA should be 2001:630:212:700:2::1. Make sure to remove all other A/AAAA records for that domain, otherwise browsers and other software may arbitrarily choose between our web server and whatever else is configured.

You can confirm if the domain is resolving to us correctly by visiting it in your browser – you should be greeted with a ‘non-existent site’ page on our site. Once this is visible to you, use the SRCF control panel to assign the domain to your (group) account.

Document roots and multiple websites

You may wish to run more than one website under a single account. This can be done if you have your own domain, by delegating different (sub)domains to different sites.

<crsid>.user.srcf.net and <groupname>.soc.srcf.net domains are fixed at serving the root of your public_html directory. When adding custom domains to your account, you can optionally set a Document root – this is a subdirectory of public_html which acts as the root of your domain.

For example, you may wish to serve example.com from /societies/<groupname>/public_html, and bubbles.example.com from /societies/<groupname>/public_html/bubbles. This means that /societies/<groupname>/public_html/bubbles/index.html would be served as the index page of bubbles.example.com.

HTTPS

HTTPS provides secure communication between our servers and your website visitors, and is recommended for any site that handles personal information or provides password authentication. In comparison, HTTP transmits in plain text, and is susceptible to eavesdropping over an untrusted network (such as public wi-fi or other shared networks).

Websites served from .user.srcf.net and .soc.srcf.net addresses come with working HTTPS by default, but custom domains must be opted-in manually. This can be done using the Let’s Encrypt form. Requests will be queued, and processed together at the end of each day.

Note that your domains need to be actively resolving to us in order to have certificates issued – if not, they will be dropped from the queue. The resulting certificate we’ll serve will list each of your domains as a Subject Alternative Name (SAN). Once opted in, no further action is required – certificates will automatically be renewed close to their expiry.

Websites that are not opted-in will not be served over HTTPS, so will only respond to plain HTTP. Requests for an HTTPS version will show a ‘non-existent site’ page.

Note that we only handle serving your website over HTTPS, and will still accept HTTP connections. If your site uses a CMS that stores a canonical domain, you’ll need to update that to use HTTPS if desired. You can also use a .htaccess file to require secure connections, and redirect plain HTTP.

What about email?

The SRCF cannot process mail for your domain – we are prevented from doing so by the university. This means you must not point MX records at SRCF machines, as any mail sent there will be dropped before it enters the CUDN.

You should check what services your domain registrar provides, which may include forwarding addresses (so that mail sent to bubbles@example.com is redirected to a mailbox elsewhere) or real mailboxes (so that you connect to your registrar’s mail server to read mail).

Email accounts

Overview

Every member of the SRCF has an email address on the srcf.net domain. If your SRCF username is spqr2, your SRCF email address is spqr2@srcf.net.

In fact, you have as many email addresses as you want, by appending a suffix of your choosing: spqr2-<something>@srcf.net. You can use this for filtering different types of email in different ways. You also have equivalent addresses in our old domain, srcf.ucam.org.

Incoming mail

You can choose via the control panel one of three possible ways for delivering email sent to your SRCF email addresses.

Simple forwarding

All of your email is forwarded to another address. For this we use the contact address you provided when you signed up, which you can also change on the control panel.

SRCF Hades email service

A modern email service. If you want to read your email on the SRCF, rather than forwarding it to another email service, this is what we recommend.

.forward file and/or legacy mailbox

This is a primitive and somewhat unfriendly system for hosting a mailbox or forwarding mail, but does offer some programmability via the SRCF shell server; by writing an Exim filter file you can, for example, pass email messages to a custom program. This is recommended only for advanced users and people who already have a mailbox on pip which predates Hades.

Outgoing mail

You can send email from your personal SRCF email account over SMTP. This service is part of Hades, but is available whether your email is hosted on Hades or not.

Files and permissions

Overview

Each group account has a corresponding system (UNIX) group that manages access permissions for files and directories in the group account space. Both /societies/<groupname> and /public/societies/<groupname> are writable by the system group. Each admin of the account is then added to that system group, which grants the admin permissions to manage files.

When creating new files or directories, you should ensure the system group permissions are correct – for each file the system group should match the group account name (and not your personal system group), and permissions should include group-write. Directories should also be group-sticky, so that subdirectories will inherit the correct permissions. Example output from ls -l:

drwxrwsr-x  2 <crsid> <groupname> 4.0K Jan  1  2020 directory
-rw-rw-r--  2 <crsid> <groupname>    0 Jan  1  2020 file

The important fields to note here are rw appearing in both of the first two sets of permissions (user and group), and s in the directory group column for sticky.

Ownership

Files in a group account space will generally be owned by the admin that created them, but with the file’s group set to match the corresponding group account, any admin can modify or delete these files as needed.

Some files may be owned by a user named after the group. This is an internal system user: websites and group account services are run as this user, meaning any files they create will have this ownership. Again, ensuring group permissions are set correctly will mean that any admin can manage such files.

Fixing bad permissions

You can run srcf-soc-permfix <groupname> over SSH, which will add any missing group permissions to files owned by the user that run it. This means you can fix files that you own, as well as files created by the internal group account’s user with sudo -u <groupname> srcf-soc-permfix <groupname>. If you encounter files owned by another admin with incorrect permissions, they will need to run this command themselves in order to fix them. In the case of ex-admins, you can contact the SRCF sysadmins to fix permissions for you.

Files and transfer

Basics

Your home directory on SRCF servers is /home/<crsid> and is private to you. Group account files can be found at /societies/<groupname> and can only be accessed by group account admins.

All accounts also come with a ‘public’ directory (these are located at /public/home/<crsid> and /public/societies/<groupname> respectively). This includes your website root public_html, but you’re free to use them as you wish.

You can use SFTP or SCP to transfer files between your SRCF account and your personal computer. Mac and Linux come with scp for per-file transfer; you may prefer a graphical client – options include WinSCP on Windows, or Cyberduck for Mac/Windows.

Quotas

We provide users with 2GB of space for their personal files, and each group account also comes with 2GB of its own space. If you run out, you may request additional quota from the sysadmins. For larger increases, you may be asked to justify the need, and the SRCF committee may need to approve it.

Consider searching for cache files in your home directory that may be taking up unnecessary space, or compressing large files (e.g. high-resolution photos on websites).

You can use the srcf-quota command over SSH to check your current quota and usage. You’ll also receive email notifications when you approach your quota. Once the hard limit is reached, you’ll no longer be able to create or append to files.

Ownership and permissions

Each file or directory stored on the SRCF have two ownership fields associated with it: an owner (a system user) and a group (a system group; this is distinct from SRCF group accounts). Every user has an associated personal group (e.g. the user spqr2 also has a corresponding group spqr2), whilst admins of a group account will be members of an additional system group.

By default, your home directory /home/<crsid> is private to you, and group account home directories like /societies/<groupname> are private to their current admins. In contrast, the public directories /public/home/<crsid> and /public/societies/<groupname> are world-readable, meaning their contents are visible to any SRCF user. Because websites are served from here, you should ensure any files containing sensitive information (such as credentials for databases) are individually set to not be world-readable.

See also group permissions.

World-writable files

World writable files are files that anybody on the system can write to (edit). Whilst in general you can trust other SRCF users not to modify your files, there are several reasons why world-writable files are a problem:

  • People make mistakes. For instance, if you have a world-writable directory, and somebody runs rm -r / by mistake (this has happened at least once) then all the files in that directory will be deleted.
  • Users’ accounts may have been compromised. We have had one incident where a worm entered the computer via an insecure society website and proceeded to overwrite every world-writable file on the computer.
  • World writable files make things easier for attackers. If for instance a directory underneath your public_html directory is world-writable then an attacker able to write files on the system could place a script there containing commands that he could execute as you.

To avoid problems like this you should avoid creating world-writable files and directories, and if you have created them then you change them to be non-world-writable. You can do this using the chmod command – chmod o-w filename will remove world-writable permissions from a file and chmod -R o-w ~ will do the same for all world-writable files in your home directory.

Some CGI scripts will tell you that they need to have world-writable files / directories to work. This is almost certainly not the case on the SRCF system where CGI / PHP scripts run as the user that owns them rather than the webserver. For society accounts it is often necessary to make the files group-writable rather than world-writable (presuming that the intended effect is to allow multiple members of the society to write to them). If you can’t get a script to work without world-writable files / directories then get in touch with the support team and we’ll see what we can do to help.

Snapshots

Although the SRCF does not guarantee to take backups of users’ data, snapshots of /home, /public and /societies are generally taken regularly for disaster recovery purposes. Snapshots may be accessed using the hidden (it will not show up in ls -a or in shell autocomplete) directory .snapshot, available at any level of the file hierarchy. For example:

spqr2@pip:~$ ls -lut .snapshot
total 336
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 18:00 sv_hourly.0
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 17:00 sv_hourly.1
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 16:00 sv_hourly.2
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 15:00 sv_hourly.3
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 14:00 sv_hourly.4
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 13:00 sv_hourly.5
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 12:00 sv_hourly.6
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 11:00 sv_hourly.7
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 10:00 sv_hourly.8
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 09:00 sv_hourly.9
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 08:00 sv_hourly.10
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 07:00 sv_hourly.11
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 06:00 sv_hourly.12
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 05:00 sv_hourly.13
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 04:00 sv_hourly.14
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 03:00 sv_hourly.15
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 02:00 sv_hourly.16
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 01:00 sv_hourly.17
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 01:00 sv_daily.0
drwxr-x--- 2 spqr2 spqr2 8192 Jul  6 00:00 sv_hourly.18
drwxr-x--- 2 spqr2 spqr2 8192 Jul  5 23:00 sv_hourly.19
drwxr-x--- 2 spqr2 spqr2 8192 Jul  5 22:00 sv_hourly.20
drwxr-x--- 2 spqr2 spqr2 8192 Jul  5 21:00 sv_hourly.21
drwxr-x--- 2 spqr2 spqr2 8192 Jul  5 20:00 sv_hourly.22
drwxr-x--- 2 spqr2 spqr2 8192 Jul  5 19:00 sv_hourly.23
drwxr-x--- 2 spqr2 spqr2 8192 Jul  5 01:00 sv_daily.1
drwxr-x--- 2 spqr2 spqr2 8192 Jul  4 01:00 sv_weekly.0
drwxr-x--- 2 spqr2 spqr2 8192 Jul  4 01:00 sv_daily.2
drwxr-x--- 2 spqr2 spqr2 8192 Jul  3 01:00 sv_daily.3
drwxr-x--- 2 spqr2 spqr2 8192 Jul  2 01:00 sv_daily.4
drwxr-x--- 2 spqr2 spqr2 8192 Jul  1 01:00 sv_daily.5
drwxr-x--- 2 spqr2 spqr2 8192 Jun  2 01:00 sv_daily.6
drwxr-x--- 2 spqr2 spqr2 8192 Jun 29 01:00 sv_daily.7
drwxr-x--- 2 spqr2 spqr2 8192 Jun 28 01:00 sv_daily.8
drwxr-x--- 2 spqr2 spqr2 8192 Jun 27 01:00 sv_weekly.1
drwxr-x--- 2 spqr2 spqr2 8192 Jun 27 01:00 sv_daily.9
drwxr-x--- 2 spqr2 spqr2 8192 Jun 26 01:00 sv_daily.10
drwxr-x--- 2 spqr2 spqr2 8192 Jun 25 01:00 sv_daily.11
drwxr-x--- 2 spqr2 spqr2 8192 Jun 24 01:00 sv_daily.12
drwxr-x--- 2 spqr2 spqr2 8192 Jun 23 01:00 sv_daily.13
drwxr-x--- 2 spqr2 spqr2 8192 Jun 20 01:00 sv_weekly.2
drwxr-x--- 2 spqr2 spqr2 8192 Jun 13 01:00 sv_weekly.3

Note that snapshots are named sv_[type].[index], with index 0 indicating the most recent snapshot of that type. The listing above shows 24 hourly, 14 daily and 4 weekly snapshots; you may see fewer or more than this.

Snapshots preserve file permissions and are read-only, so if you wish to retrieve something from a snapshot, you must have had permission to access it at the time the snapshot was taken, and must copy (rather than move) it out of the snapshot.

Snapshots going further back in time may be available on an off-site disaster-recovery replica; if you need access to these, contact the sysadmins (but please don’t count on them being available; you should take your own backups regardless).

Game servers

Overview

If you want to run a server for your favourite online game (or any other long-running application) you should use doom.srcf.net, as described here.

Please limit your server to around 1GB of RAM and set it to nice level 19 to avoid causing problems for other users. For example, if you are running a Minecraft server, then the following should work:

nice -n 19 java -Xms1024m -Xmx1024m -jar minecraft_server.jar

It is possible (and probable, in the case of Minecraft) that the default port for your application will be in use. We don’t have any specific rules for picking a port for your application, so just pick high-numbered ports at random until you find one that isn’t in use.

Running at startup

If you want your server application to run at system startup, you can create a cron job for it as described here.

Gopher

Overview

Gopher is a protocol for distributing, searching and retrieving documents over the Internet as an alternative to the Hypertext Transfer Protocol (HTTP). In contrast to the World Wide Web, Gopherspace is navigated using a menu-based interface.

Your normal web browser likely won’t be able to access Gopher sites so you will need to install some other software or browser plugin first. Once you do this you should be able to find the SRCF’s presence at gopher://gopher.srcf.net/.

Publishing your Gopher site

To get started with a Gopher site you’ll want to create a subdirectory in your public home directory called public_gopher:

>     /public/home/spqr2/public_gopher         # For individual users
>     /public/societies/foosoc/public_gopher   # For groups and societies

After a short delay your site will be available via Gopher at:

>     gopher://gopher.srcf.net/1/users/spqr2
>     gopher://gopher.srcf.net/1/societies/foosoc

Customising your directory listing

Gopher sites consist of a directory listing of files and a description (or abstract) for each file. To attach an abstract to a file, say myfile.txt, create another file called myfile.txt.abstract with a line or two of descriptive text. This will be displayed with any links to that file. To attach an abstract to a directory, create a .abstract file inside that directory.

The directory listing of a Gopher site can be customised using a gophermap file in the directory in question. For full details on how to do this take a look at the /var/gopher/gophermap-info.txt file on pip.

Group account email

Overview

Group accounts, like personal accounts, can receive email to either their base name or a suffixed name. This can be useful for setting up ‘role’ addresses for the group account, for example to pass mail sent to <groupname>-president@srcf.net.

Default role addresses

All group accounts come with the role address <groupname>-admins@srcf.net, which can’t be overwritten. Mail sent here will be distributed to the individual SRCF mail accounts of all the users who are admins of the group account.

By default, we also define <groupname>-webmaster@srcf.net as an alias to <groupname>-admins@srcf.net. This will be used as the default sender address for email sent by the group account, and may be displayed in some error messages to visitors, but you can change where mail ends up as described below.

Customising mail handling

You can set up a .forward file in your group account’s directory at /societies/<groupname>/.forward. This will be used to process all email sent to addresses of the form <groupname>@srcf.net or <groupname>-<anything>@srcf.net, with the exception of <groupname>-admins@srcf.net which is always forwarded to admins.

See pip forwarding for more information.

Hades email service

Overview

SRCF Hades is a modern, standards-compliant email service, introduced in October 2018 for two reasons:

For @srcf.net addresses

Use the control panel to switch to Hades: from your member page, select Change email configuration under Email. When you choose to switch your email to Hades, the control panel will explain the consequences which depend upon your current configuration. You will have a chance to cancel the migration after seeing this information.

This applies equally to our old domain @srcf.ucam.org.

For @cam.ac.uk addresses

If you miss Hermes, you can set your @cam.ac.uk email address to deliver mail into your SRCF Hades inbox, thus making Hades the service you use to read your University email.

Go to your page on the University Lookup service. Edit it, and change your ‘@cam delivery address’ to your SRCF email address (for example spqr2@srcf.net). If you have a University Exchange Online account then by default email sent by other Exchange Online users to your @cam.ac.uk address will bypass the setting in University Lookup and be delivered to you Exchange Online inbox regardless. To setup forwarding in ExOL, click on the settings cog icon and search for ‘Forwarding’.

Reading email

Webmail

The easy way to read your email: webmail.hades.srcf.net

For apps: IMAP

To configure any standards-compliant mail app to read your email, you will need the following details:

  • Server type: IMAP
  • IMAP server: hades.srcf.net
  • Security: STARTTLS (or SSL/TLS)
  • Username: your SRCF username, e.g. spqr2 (not your email address)
  • Password: your SRCF password (not your Raven password)

Some email apps such as Thunderbird will automatically configure this for you after you fill in your SRCF email address — provided that you have already enabled Hades email via the control panel.

If you’re looking for an app to get your Hades email on an Android phone, we suggest K-9 Mail.

Sending email

Outside of webmail, you can use SMTP to send emails from your @srcf.net address. Add these details to your client of choice:

  • SMTP server: smtp.srcf.net
  • Security: STARTTLS
  • Port: 587
  • Username: your SRCF username, e.g. spqr2 (not your email address)
  • Password: your SRCF password (not your Raven password)

Mail forwarding

You have two options:

  • Bypassing Hades: On the control panel you can configure your @srcf.net email address to forward mail to another address. This is the simplest option.
  • Using Hades filters: You can add a filtering rule (see below) which forwards all your email elsewhere, with or without keeping a copy in your Hades mailbox.

Mail filtering

  1. Log into Hades Webmail
  2. Click ‘Settings’
  3. Click ‘Filters’
  4. Add a filter by clicking the ‘+’ button at the bottom of the ‘Filters’ panel
  5. Choose a name for the filter, a set of conditions to match (e.g. ‘all messages’) and an action (e.g. ‘Move message to folder’)
  6. Click ‘Save’

You can add as many filters as you want, and drag them around to change the order in which they are run. You can optionally have multiple filter sets, although only one of those can be active at once (for example you could have an alternate filter set, normally disabled, to handle your email when you are on holiday).

Alternatively, you can write your own Sieve script and upload it as a filter set: click the ‘+’ button at the bottom of the ‘Filter sets’ panel. (Note that Sieve filters use a different syntax to the .forward Exim filters available on pip, although the capabilities are similar.)

Backing up email

The SRCF does not guarantee to have backups of user data. You must take your own backups.

You might find an IMAP synchronisation tool such as isync (a.k.a. mbsync) useful. The Arch Wiki has a handy guide to setting up isync.

At times we might have disaster-recovery backups which include a snapshot of your mailbox. If you’ve lost data and your own backups have failed, try asking the sysadmins if we can help — but we might not be able to.

Internet Relay Chat (IRC)

Overview

Internet relay chat is a protocol for real-time text messaging between internet-connected computers. While it is dated in comparison to some other messaging protocols, it continues to be used regularly by SRCF members and as the emergency messaging platform of choice, with several servers located around the world.

We have written a tutorial on getting started with IRC, available here.

Important information

Channels

  • #welcome — a place for newcomers to say hello
  • #general — chat about anything here, we’re humans after all :)
  • #society — questions and topics about the SRCF as a society
  • #support — tech questions, SRCF-specific or not
  • #hackday — home of volunteers, future home of interested volunteers
  • #ops — infra-specific discussion, invite only (interested sysadmins encouraged to email sysadmins@srcf.net to join!)
  • #srcf — alumni

Server details

You need to connect your client to irc.srcf.net. If you have a suitably-configured IRC client then clicking on that link might work, or if you want to try out IRC without downloading and installing a client then the SRCF provides an IRC web interface.

Setting up a persistent connection

Some people like to leave an IRC client connected to the network even when they are away from their computer, or have turned it off, so that people can continue to leave them messages for when they return. There are two main ways: a bouncer or a tmux/screen instance.

For tmux/screen, a common choice is irssi, although there are many others.

If you want to use a bouncer, we recommend ZNC.

Troubleshooting

Misconfigured client

If you are seeing ‘nick [~username@my-irc-client-is-misconfigured]’ then this is because you are not connecting to irc.srcf.net (as indicated above) but instead to some other hostname – perhaps kern, pip, shell or www. This is not recommended, since if the IRC server moves at a later date those addresses will no longer work, whereas irc.srcf.net will always be updated to point at the new location.

Legacy mail on pip

Reading mail

Legacy forwarding

You can forward incoming mail to some other address. For old accounts, when your account was first set up it was configured to forward mail to the address you gave us when you applied. See mail forwarding below.

Local client

Read your mail using an email client running on the server, for example mutt or alpine.

For apps: IMAP

To configure any standards-compliant mail app to read your email, you will need the following details:

  • Server type: IMAP
  • IMAP server: pip.srcf.net
  • Port: 993
  • Security: SSL
  • Username: your SRCF username, e.g. spqr2 (not your email address)
  • Password: your SRCF password (not your Raven password)

Note that we do not run a POP3 server, as it is an older protocol and provides no additional functionality over IMAP.

Webmail

Use our pip webmail service: webmail.pip.srcf.net

Sending mail

You cannot use pip itself as a SMTP server, except from applications running on pip itself.

You can send email from your personal SRCF email account over SMTP. This service is part of Hades, but is available whether your email is hosted on Hades or not.

Mail forwarding

Create a file called .forward (note the leading full-stop) in your home directory containing the email address to which you would like your mail to be forwarded.

The SRCF offers a simple interactive tool to create this file for you. Run srcf-MailForward over SSH, which will prompt you to enter an e-mail address where you would like all your SRCF mail forwarded to.

Auto-forward

We also provide srcf-autoforward, a tool to generate role-like addresses with a simpler syntax. You need to create a file called autoforward in a group account’s directory (or your home directory, for your own roles), where each line consists of a role name, a colon, and a comma-separated list of target addresses:

<role>:<email>,<email>...

For example:

treasurer:spqr2@cam.ac.uk,treasurer@example.com

Once you’ve defined your roles, you need to build the actual .forward file:

srcf-autoforward <groupname>

Mail filtering

We have the Exim mail transfer agent installed – see the Exim filtering documentation for how to write filters in your .forward file.

Filters in Exim are more complicated than Hades’ Sieve filters, though there are a few things supported which cannot be done on Hades, such as piping your email into a custom program.

Here are a few examples of basic forwarding for a group account. Note that the # Exim filter line is required.

# Exim filter

# Forward emails for <groupname>-webmaster@srcf.net to spqr2@cam.ac.uk:
if ($local_part_suffix is "-webmaster") then
   deliver spqr2@cam.ac.uk
endif

# Forward <groupname>-treasurer@srcf.net to spqr and an external address:
if ($local_part_suffix is "-treasurer") then
   deliver spqr2@cam.ac.uk
   deliver treasurer@example.com
endif

# Forward anything not yet processed to a lists.cam mailing list:
if not delivered then
   deliver group-example-committee@lists.cam.ac.uk
endif

Known quirks

pip’s mail service dates from a time long ago when most UNIX systems stored email in Mbox format. Mbox is a primitive file format which concatenates all emails in a mailbox into one long file. It is slow and inefficient to use, and does not support concurrent access (you can’t read your inbox whilst a message is arriving, for example). Our IMAP server tries its best by using caches and indices but it is fundamentally not an efficient data storage system.

Furthermore, if you create folders (besides INBOX) in which to file your emails, those are placed in your home directory along with your other files. This leads to confusion, and also can lead to your entire home directory being served over IMAP, which has been known to upset email software (unsurprisingly).

And for added confusion, Mbox is incapable of storing any email containg a line of text starting with the word ‘From’. Yes, really. (Mail software has to rewrite it as >From as a workaround.)

We cannot easily convert pip to a more modern email storage format, though, as we have some people with extensively curated mbox hierarchies and we don’t want to break email for those people.

So (in September 2018) we created a brand new email service called Hades which runs alongside the “legacy” email service on pip. Hades was implemented with the benefit of another 20+ years of advancements in email technology.

We encourage you to take a look at Hades. If you’re used to the setup on pip, you may find it quite different (better, we hope!), but if you have a complex Exim filter it may be time consuming to redefine your email filtering requirements in Sieve on Hades.

We have no immediate plans to discontinue pip’s mail service, although eventually we hope to be able to shut down the IMAP server on pip as our IMAP users migrate to Hades. We know that there are some things that advanced users might want to do which are only possible on a traditional UNIX mail system, and we intend to keep running the mail service on pip for these advanced use cases.

Mailing lists

We run Mailman mailing lists for both personal and group accounts. New lists can be created via the control panel.

The default list administrator for personal accounts is <crsid>@srcf.net. For group accounts the default list administrator is the group account’s admin address <groupname>-admins@srcf.net. To avoid complicating group account handovers, we recommend you keep this default for group-owned lists, and manage list admins by adding to the group account admin list. You are free to use the list moderator role to add additional users for approving postings and subscriptions.

To delete a mailing list, you’ll need to contact the sysadmins. Unless asked otherwise, we will retain the list archives (though the archives must be public for them to remain accessible).

Mattermost

Overview

Mattermost is an open source messaging platform, similar to Slack. See the official user guide for more about its service.

The SRCF’s mattermost instance can be found at https://mattermost.srcf.net, and is the Team Edition. This is open to use for any members of the SRCF as well as their guests.

Policies and features

  • A user must sign up with an @srcf.net, @cam.ac.uk or @*.cam.ac.uk email, unless they have an invitation from an existing user. A user can change their email address to their personal email after signing up.

  • Only team administrators are allowed to create and delete channels and modify their descriptions. (The button to make these changes will still be present in the user interface, but they will not work)

  • Open teams (teams that do not require invitations to join) are not available on our server, since open teams are advertised to all users on the server. If you want to invite a large number of people (e.g. all members of your Facebook group), you can share an invitation link.

  • If you want to delete all messages in a channel older than n days, you can use the following two-step procedure:

    1. Get the channel id by typing /channel_id
    2. Run the command /delete_posts <channel_id> <n> where <channel_id> is the channel id you obtained from the previous step, and <n> is the number of days.

    This an only be run be a team administrator.

Privacy information

The SRCF privacy policy applies. This section describes details pertinent to the Mattermost service.

Email

Your email is used to send notifications to you. The types of notifications you receive can be configured in the web interface.

Data retention

By default, any person who joins your channels can see all past messages. Messages that are deleted in the user interface are not accessible by end users anymore, but remain in the database. Contact the sysadmins if you wish to have your data purged. It is also possible to permanently delete all posts by a user.

User lookup

Your username and name are visible and searchable by other users in the invitation menu. Your email remains private.

Clients

The SRCF is only responsible for the web client. When using other clients (e.g. a mobile client), be sure to read their privacy policy.

MySQL and PostgreSQL databases

Overview

Both user and group accounts can make use of databases, either in MySQL or PostgreSQL. We don’t create accounts for these by default – you will need to request them separately via the control panel, which will give you a distinct password for authentication (don’t use your personal SRCF password).

You will receive a default database named after your account. In MySQL, you can create additional databases of the form <username>/<something>. For PostgreSQL, you can make use of schemas to organise your tables if needed.

Connecting to your database

The canonical hostnames for database servers are mysql and postgres, which are valid across all user-accessible servers.

To connect to MySQL from an SRCF shell:

mysql -h mysql -p <database>

Here, -h specifies the hostname, -p prompts for your password, and you’re connected to the database specified at the end – either your CRSid for your personal database, or a group database name.

Similarly for PostgreSQL:

psql -h postgres [database]

Providing a database is optional; by default, you’ll be connected to the one matching your username. Ident authentication is enabled for PostgreSQL, which will authenticate you based on the underlying system account, so you usually don’t need to provide a password.

You can also use phpMyAdmin for MySQL, and phpPgAdmin PostgreSQL, to manage your databases in a browser.

Group account access

If logged in using your personal database account, you can also access databases owned by group accounts you administrate.

When configuring software or websites in a group account, you must use the group database account to access its databases – do not use your personal credentials as that would give other group account admins access to your personal account and all the databases (both personal and group) to which you have access!

Passwords

When your personal account is first created, you’ll receive a randomly generated password by email. You can change it by connecting over SSH and running passwd.

This password is valid for SSH, file transfer and email. It’s not valid for database access – both MySQL and PostgreSQL accounts must be requested separately and come with their own password. It’s also not valid for Mailman mailing lists, as each list comes with its own owner and moderator passwords.

Group accounts do not come with passwords, and can not be logged in as directly. If you need to manage files in a group account, connect using your personal account and navigate to the group account’s files. You should avoid storing your personal credentials in any group applications or config files.

Permissions

Society accounts have been renamed to group accounts. This page has been moved.

Personal vs. group accounts

All SRCF users, whether they are looking for web hosting for themselves, or wish to administrate a group account, must first get a personal SRCF account. It is completely fine if you apply for SRCF membership when all you really want is to run the website of a society (or other group), rather than your own.

Raven authentication

Overview

You can configure your site, or a subset of pages, to require user authentication with Raven, the university’s authentication system.

With .htaccess

The SRCF has the mod_ucam_webauth module installed, which makes it easy to do basic authentication using Raven. Full documentation is available, though here are a few common cases.

To protect a directory (whether public_html for your entire site, or a subdirectory of it), create or edit a .htaccess file in that directory, and add the following:

AuthType Ucam-WebAuth
Require valid-user

This will permit access to anyone with a ‘current’ Raven account, i.e. active students and staff. To permit access to any Raven account (including graduated students), add a Ptags directive:

AARequiredPtags none

Alternatively, you may want to limit access to Raven-authenticated users or visitors within the cam.ac.uk domain:

Order allow,deny
Allow from .cam.ac.uk
AuthType Ucam-WebAuth
Require valid-user
Satisfy any

To limit page access to group account admins only, add a unix-group Require directive:

Require unix-group <groupname>

You can also list specific users:

Require user <crsid> <crsid>...

To create a ‘logout’ link, add the following to your .htaccess file (which will create /logout relative to the directory containing the .htaccess file):

<FilesMatch "logout">
    SetHandler AALogout
</FilesMatch>

You can access a Raven-authenticated user’s CRSid using the REMOTE_USER (or AAPRINCIPAL) environment variables. For example, adding the following to a PHP page like index.php will display a customised welcome message on login:

<?php
echo "Hello {$_SERVER['REMOTE_USER']}!"
?>

Example configuration:

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=permanent]

AuthType Ucam-WebAuth
Require user CRSID

Replace CRSID with your CRSID.

The Rewrite section makes all connections to .../wp-admin/ use SSL which will protect your password, the AuthType section uses Raven to restrict access to the directory, you probably want to use your CRSID on the Require line.

Within an application

If you’re writing or maintaining a webapp that needs to authenticate users for certain pages, you’ll likely need a Raven or UCamWebAuth library. The Raven developer documentation may be useful for finding a suitable integration.

Regular hosting

Overview

All accounts include regular web hosting, which is suitable for static content, PHP (WordPress, Joomla, etc.), and CGI/FastCGI. For those wishing to host more advanced web apps with the SRCF (Django, Flask, Rails, Node.js, etc.) and leverage their full flexibility, see this page.

PHP applications

Scripts in public_html using .php extensions will be executed by PHP by default.

WordPress

Follow our full WordPress installation tutorial .

Static site generators

You can use a generator of your choice to manage your site. Just have it output into your account’s public_html directory.

Jekyll

To create the skeleton project:

jekyll new jekyll
cd jekyll

For a root site (i.e. top of public_html):

ln -s /public/societies/sample/public_html _site

..or for a site in a subdirectory, edit _config.yaml to set baseurl to e.g. /jekyll, then:

mkdir /public/societies/sample/public_html/jekyll
ln -s /public/societies/sample/public_html/jekyll _site

To (re)build the site:

jekyll build

Rules and principles

Commercial hosting

Our servers are connected to the Internet via the Cambridge University Data Network (CUDN), and as such all users of the SRCF are bound by its rules, which explicitly forbids any use of the CUDN for commercial or for-profit activities. Therefore it is not possible for commercial websites to be hosted on our machines. For more details, please refer to the Authorisation for use of the CUDN and the Rules made by the Information Services Committee.

Our interpretation of these rules is that including a sponsor’s logo on a group account website is acceptable, however selling web space or advertising is not.

Furthermore, the SRCF server is maintained by a group of volunteer sysadmins who donate their time freely to ensure that our services can continue to operate smoothly. We do not feel that it is appropriate to ask that the sysadmins give up their time in this way so that others can make money.

Inappropriate content

Our policy

Inappropriate content is deemed to be any content that is in contravention of the CUDN rules and guidelines, or JANET AUP. Amongst other things these rules prohibit using the CUDN to store or transmit material that is obscene, defamatory, offensive or violates a third party’s copyright. If such material comes to the attention of the sysadmins or committee the responsible user will be required to remove it. In the case of gross and flagrant breaches of the rules the sysadmins may remove the material themselves. The account of a user responsible for prohibited material may be suspended and they may be reported to the Computing Service.

So in summary, don’t do it!

Reporting a breach

Please let the user or group that controls the content know that you consider it inappropriate, and explain why with reference to the SRCF’s Terms of Service, CUDN rules and guidelines, or JANET AUP. For pages with URLs of the form:

https://spqr2.user.srcf.net/ or https://www.srcf.ucam.org/~spqr2/

you should email spqr2@srcf.net

For pages with URLs of the form:

https://groupname.soc.srcf.net/ or https://www.srcf.ucam.org/groupname/

you should email **groupname**-webmaster@srcf.net

If you do not get a response from the owner of the page or feel unable to approach them yourselves, then you should forward your concerns to the committee: committee@srcf.net.

Using resources wisely

Please be considerate of others when running long and CPU-intensive tasks. Make sure that you use ‘nice’ to give them a low priority and double check your code to make sure that there aren’t memory leaks or other bugs which might impair the performance of the system for the other users.

If you need to use a lot of computing time for academic reasons your first port of call should be your department or college as they should provide you with the resources you need for your studies. If your reasons are non-academic, get in touch with us.

Scheduled tasks

Overview

You can make use of cron or systemd timers to run programs on a schedule. In either case, you will need to configure it on the right server (e.g. tasks relating to web applications should probably run on sinkhole, game servers on cavein or doom and anything else most likely on pip).

Cron

Connect over SSH and run crontab -e, which will open an editor and an explanation of the file format. You can add entries to run something at a fixed interval, or on server reboot:

# Run at 03:21 every day:
21 3 * * * /home/<crsid>/daily.sh

# Run at server startup:
@reboot /home/<crsid>/startup.sh

If you want to setup cron jobs for a group or society account then you’ll need to run sudo -u SOCNAME crontab -e.

A tool like this is helpful for mastering syntax.

systemd

Unit files

You can use user unit files to manage services, which should be placed inside ~/.config/systemd/user/ (where ~ is your home directory, e.g. /home/<crsid> or /societies/<name>).

A sample personal account unit file, porridge.service, might look like the following:

[Unit]
Description=spqr2 Porridge app
ConditionHost=pip

[Install]
WantedBy=default.target

[Service]
ExecStart=/home/spqr2/porridge/run.sh
Restart=on-failure

ExecStart defines the command to be run. For group account services, ensure any paths to programs or directories are absolute and not via personal account paths (i.e. /societies/foosoc, not /home/spqr2/foosoc). You may also need to define WorkingDirectory as the directory path where your service is run from.

ConditionHost is required to ensure your service only runs on a single desired server, as your home directory and all your unit files will be present on all servers. Valid values are pip, sinkhole, doom or cavein. In order for services to run without you being logged in, your user account needs to have lingering enabled. This is a per-server state, and is controlled by the existence of unit files with a valid ConditionHost line – it may take up to 20 minutes for lingering to be enabled on creation of your first service.

By default, your services’ standard output and error streams are sent to systemd’s journal, however only the root user can access these. You should make your service write to their own log file rather than stdout or stderr.

Consult the systemd documentation for all available options. Note that some options are not available for user units (as opposed to system-wide units).

Timers

The above unit will give you a long-running service, one that starts your program at system startup. To schedule execution, drop the [Install] section from the unit file and instead create a corresponding timer file porridge.timer:

[Unit]
Description=spqr2 Porridge daily update
ConditionHost=pip

[Install]
WantedBy=timers.target

[Timer]
OnCalendar=daily

This will execute your program at midnight each day. The OnCalendar line accepts phrases like daily and weekly, or time strings of the form DayOfWeek Year-Month-Day Hour:Minute:Second (for example, Sat *-*-1..7 18:00:00 gives you 6pm on the first Saturday of each month).

Service control

To control your app, you can use the systemctl command over SSH. Basic commands are of the form systemctl --user <action> <unit>. Actions include start, stop and restart to run your service now, enable and disable to set whether the unit launches on system startup, and status to show information about it. The unit parameter is the name of the unit file itself, e.g. porridge.service or porridge.timer.

If you make a change to a unit file, you need to run systemctl --user daemon-reload on the affected server in order to pick up the change.

Due to an implementation detail, you must call systemctl as follows when interacting with a group account’s services (here using foosoc for the account name):

sudo -Hu foosoc XDG_RUNTIME_DIR=/run/user/$(id -u foosoc) systemctl --user ...

..and then run socsudo foosoc systemctl --user ....

If you receive an error like this setting up your group account’s first service:

Failed to connect to bus: No such file or directory

…then you may need to wait up to 20 minutes for lingering to be switched on – this will happen automatically once a service file is detected with a valid ConditionHost line.

Soc-SRCF mailing lists

Overview

The SRCF needs to be able to keep in contact with our users at all times (a requirement imposed upon us). We therefore automatically subscribe you to Soc-SRCF, Soc-SRCF-users and Soc-SRCF-maintenance shortly after creating your account. We will also subscribe you to Soc-SRCF-social if you requested this during the signup process.

These are very low-traffic mailing lists—just a few messages per year—and we do our best to ensure that the messages sent are relevant to as many people as possible. We hope that you will not consider this to be an inconvenience.

How we use these lists

Soc-SRCF

This contains members of the SRCF (membership is for life), and receives messages from the committee about such things as:

  • major society event announcements (e.g. AGMs, garden parties) – usually about three events per year
  • rare but important society administrative information (e.g. constitutional amendments) which affect all members

Soc-SRCF-users

This contains users of the SRCF (which only applies whilst you have an active shell account), and receives messages from the sysadmins which are important to account holders, for example:

  • urgent messages which require the attention of every SRCF user (e.g. security compromises) – very infrequent, we hope!
  • rare but important system administrative information (e.g. terms of service changes) which affect all users

Soc-SRCF-maintenance

This is used for announcements about planned and ongoing system unavailability (downtime), and announcements about important changes to service provision (e.g. increased quotas, software changes which may require your action). If you are not interested in receiving such messages, you are welcome to unsubscribe, but many users find these announcements useful.

Soc-SRCF-social

This is a more open mailing list for announcement and discussion of social events (pub meets, LAN parties, garden parties and so on). Any SRCF member is welcome to join or leave this list at any time.

Soc-SRCF-careers

The SRCF has in the past provided career opportunities informally. Join this new list to keep up to date with any employment opportunities, internships and news directly from companies we trust.

Unsubscribing from Soc-SRCF and/or -users

Because we must be able to contact everyone with an SRCF account, you may only unsubscribe from Soc-SRCF or Soc-SRCF-users if you are willing to give up your membership of the SRCF and access to its services. If this is what you want, please contact the sysadmins saying you want to terminate your membership.

If you wish to do so, you may give up your shell account (and so cease to be an SRCF user) while remaining a member of the SRCF. In this case, you will be unsubscribed from Soc-SRCF-users, but not Soc-SRCF, and you are still welcome to attend SRCF events. As above, please contact the sysadmins to arrange this.

Unsubscribing or rejoining other lists

You can subscribe to or unsubscribe from a list from its information page:

List archives

Previous messages sent to these lists can be seen here:

Societies

Society accounts have been renamed to group accounts. This page has been moved.

Software and installation

Overview

Our main shell server includes a plethora of Linux software, some of which can be used graphically over a VNC session.

If something is missing that you’d like to use, we may be able to install it for you. The usual constraints are that it should be available in Ubuntu’s repositories, and suitable for a multi-user environment. You may contact the sysadmins to request an installation.

As we run long-term support releases of Ubuntu1, you may find that the versions we have installed are rather outdated (typically several years behind). As packages are managed by Ubuntu maintainers, security fixes are backported and applied to these versions, but new features are not. This means some custom software may incorrectly detect our versions as insecure, as Ubuntu-patched releases add a version suffix (for example, PHP is 7.4.3-4ubuntu2.4).

Whilst some software can be updated through the use of third-party distributions, or by building the latest version from source, this would add significantly more complexity to system maintenance and upgrade considerations, and would require trusting the additional authors or distribution maintainers.

In some cases, you may be able to set up or install your own copy of software within your (or a group account’s) home directory. However, it is then your responsibility to keep it updated – accounts with security vulnerabilities may be suspended without notice.

System-wide upgrades

Every few years, we need to move to a newer version of Ubuntu in order to continue receiving security updates. As a side-effect, this will introduce significantly newer versions of most installed software, which is liable to break any unusual software configurations of our users. In the past we’ve seen significant changes with Apache and PHP, which many users use for their personal or group websites.

We are aware that newer versions of Ubuntu are available and waiting for us – upgrades are not taken lightly, and will be carefully planned to minimise disruption to our users.

Program-specific notes

Node.js

Ubuntu comes with node version 10.19.0. If you need a newer version, consider using Node Version Manager.

PHP

Ubuntu provides version 7.4 of PHP, along with a handful of modules. We’re aware that various web applications will flag this version as outdated or insecure – see above for how this is catered for.

Python

We have Python versions 2.7 and 3.8. Note that the python and pip binaries are Python 2, use python3 and pip3 for Python 3. If you need a newer version of Python, consider using pyenv.

Assorted Python modules (as packaged by Ubuntu) are installed for both, and others can be installed for Python 3 on request (subject to the notes above). Module support for Python 2 is limited – if you’re unable to migrate code to Python 3, you’ll likely need to use a virtualenv and manage package installation yourself.

Ruby

The servers have version 2.7 of Ruby installed together with a handful of useful gems and development dependencies. If you need a more recent version then you might consider using ruby-build and/or rbenv.

Install a package

Probably. Feel free to email us at support@srcf.net and be sure to provide the name of the Debian package you want us to install. Keep in mind we’ll probably be installing the stable version of the package, so it might be old.

You might prefer to install the package locally. See below.

Update a package

Probably not. Our servers run Ubuntu stable, so it’s expected that system packages aren’t current (indeed, they’re often a few years old). We almost never make exceptions or install backported packages.

For developing and deploying your app, you should almost certainly be using your platform’s version manager (rvm, venv, nvm, gvm, etc.). This will allow you to run the exact versions you want, and install any necessary dependencies, all without coordinating with us (or forcing the rest of our users to switch versions).

The pages above provide instructions on doing this with popular programming languages.

Port binding

When running programs that bind to ports, please ensure you only bind to localhost so they’re not publicly accessible, and choose a random high-numbered port to try and avoid conflicts with other users.

If you’re able to bind to a UNIX socket instead of a port, this has several benefits: the socket is subject to filesystem permissions so only you or a society account can access it, and you are free to choose a unique path for it.


  1. At the time of writing, the SRCF shell and web servers both run Ubuntu 20.04. ↩︎

SSH and servers

Overview

You can connect to our servers over SSH in order to run programs, manage files, and change your password.

Clients

You will need an SSH client – Mac and Linux come with ssh in the terminal; for Windows, you can enable OpenSSH in Optional Features, use a Linux distro in WSL, or install a Windows client like PuTTY.

Alternatively, you can access a web-based terminal in your browser.

Passwordless SSH

Passwordless SSH is often a point we get asked about.

There are a number of reasons why your passwordless ssh setup might not be working but by far the most common (and the hardest to detect on the client side!) is incorrect permissions on your .ssh directory. Because the authorized_keys file contains sensitive information it must not be readable by any other user. Therefore you should ensure that the file $HOME/.ssh/authorized_keys have permissions -rw------- (i.e. 600). 660 (-rw-rw----) will not work even if you are the only member of the group that owns the authorized_keys file. The same caution about permissions applies to the .ssh directory itself.

Available hosts

Shell

shell.srcf.net

Our main server is pip. This is a general-purpose machine for running software. It has the following SSH fingerprints:

DSA  MD5:12:8f:8a:1c:e4:f7:a2:9d:80:a3:ef:85:f8:79:a4:ed
DSA  SHA256:fshymFC90Vd0BrlFnAdilNKSWNsoQVI7qa8/GIMBqtw
RSA  MD5:4d:da:7f:b2:99:9d:42:8b:10:b4:e5:37:d5:bd:43:b4
RSA  SHA256:ud4XwVhosGW3oHZ2POzW4oz0zGC2y7JWLGOM1dbIqZw

Web

webserver.srcf.net

Web applications should be run from sinkhole, our webserver. This includes Apache, our server software of choice, along with custom backend servers which Apache can proxy to. Sinkhole has SSH fingerprints as follows:

DSA      MD5:20:28:75:a4:c5:0b:08:47:c5:ae:f8:8c:21:01:1a:00
DSA      SHA256:iyt8GRgVBQSNyaH/yX7QZuFODMzfpsO9xV7UnaxICLI
ECDSA    MD5:90:78:92:7b:1e:4e:51:22:30:3b:e0:dd:97:9c:aa:e4
ECDSA    SHA256:G1glvFC2H4VR9mA1RkJCRVVyLB6e/LarI+8ZUFRGds8
ED25519  MD5:55:2e:b8:bb:53:79:9a:62:0f:62:24:e3:8d:7f:87:62
ED25519  SHA256:WalyNfnZm6yuIhMhe6kj17NikSK8LuC2I6iggWYTKcY
RSA      MD5:76:aa:f1:63:ec:ca:ac:97:fb:05:35:3c:c7:8f:55:ff
RSA      SHA256:DSC30UdUoQNYWhwU4icuPOyrDC9xVjsnKbRC3Mz9RZM

Games & others

doom.srcf.net

If you’d like to run a game server or more CPU-intensive application, please use doom, which has fingerprints:

ECDSA    MD5:74:52:6e:52:d7:07:51:9e:93:12:51:82:8c:b4:ec:0a
ECDSA    SHA256:QdTm7ECl0KObTodBBpY6Qs57PFmr3MHvQfT50IbKa3U
ED25519  MD5:49:d8:e6:67:19:1c:c5:ed:12:de:3a:64:91:95:4c:9d
ED25519  SHA256:kh1Sr6Nrlp/vK9ijKZ43/IQ2tqdPzY/fnZdnGBIKgIM
RSA      MD5:f1:ee:8c:a7:7a:cb:f7:c7:dc:c5:7e:56:9a:83:f5:bc
RSA      SHA256:c1dlaFnPyJ44CnjZIeV6zLHQCPlIH9Og0K3dL16XGfo

Troubleshooting

Unsupported term-type environment variable

If the terminal behaves incorrectly on certain keypresses - such as by outputting weird glyphs, or by not removing characters when inputting a backspace - then it is possible ssh has misconfigured the TERM environment variable.

This can be checked by running toe -a | grep $TERM. (toe -a lists all supported values for the TERM variable, and | grep $TERM performs a search within toe's output to try to find your session’s value of $TERM). If running the command shows no output, then your value of TERM is unsupported.

You can temporarily select a more general terminal type (xterm-256color should work) for your session by setting the TERM environment variable when running ssh:

TERM=xterm-256color ssh ...

To correct this persistently, run the following command (on your device):

infocmp -a $TERM | ssh <Your CRSid>@shell.srcf.net tic -x -

This will install your device’s terminal-type to your account’s home directory (inside ~/.terminfo/). You will not need to make any manual changes to contents of this directory. Your terminal should now work correctly for all subsequent sessions.

Usenet (NNTP)

The SRCF offers a Usenet server for our users. To connect, you will need to point your newsreader at news.srcf.net and connect using a secure connection (SSL) to port 563. In order to combat spam, you will need to login with your SRCF username and password.

If you wish to use Alpine as your newsreader then you will find that our shell server pip has a default configuration ready to go. Otherwise you will need to configure it by going to (S)etup, (C)onfig and then setting nntp-server to news.srcf.net/ssl/user=pqrs2 where pqrs2 is your CRSid.

Web applications

Introduction

All accounts include regular web hosting, which is suitable for static content, PHP (WordPress, Joomla, etc.), and CGI/FastCGI. For those wishing to host more advanced web apps with the SRCF (Django, Flask, Rails, Node.js, etc.) and leverage their full flexibility, we provide these overview docs and more custom tutorials too:

We also provide a number of sample websites for you. You can explore them live at https://sample.soc.srcf.net/ and view the backend at /public/societies/sample.

You can host basically any kind of web application that can bind to a socket, including all of the popular frameworks like Flask, Django, Ruby on Rails, Express, and beyond. We provide tips and information on getting your apps to run on our web server.

Adding our badge

We encourage you to add the SRCF badge to your website so that others can find us. It’s a great way to help us out as it costs little to you but helps us out immensely, in turn allowing us to provide better services to you. You can find the badges here.

Connecting

We provide a webserver (currently named sinkhole), for hosting applications. You should connect to this server, not to the public login server, pip.

You connect to this server via SSH using your normal SRCF account name and password.

  • Host: webserver.srcf.net
  • Port: 22

Routing traffic to your app

The SRCF uses Apache to serve websites so if you need to run a backend web app, for example a Django, Rails or Express server, then you will need to forward web requests. See also our guidance on port binding.

Using UNIX sockets

You will need to configure your application to use a UNIX socket. The socket should only be accessible to you, which can be done by picking a path in your home directory (e.g. /home/ab123/myapp/web.sock) or by using appropriate file modes.

Then add the following to your .htaccess file, replacing <path-to-socket> with the path to your socket (e.g. /home/ab123/myapp/web.sock) and <url> with your domain name (e.g. ab123.user.srcf.net) :

RequestHeader set Host expr=%{HTTP_HOST}
RequestHeader set X-Forwarded-For expr=%{REMOTE_ADDR}
RequestHeader set X-Forwarded-Proto expr=%{REQUEST_SCHEME}
RequestHeader set X-Real-IP expr=%{REMOTE_ADDR}
RewriteRule ^(.*)$ unix:<path-to-socket>|http://<url>/$1 [P,NE,L,QSA]

Using TCP ports

You will need to pick a port (we’ve used 999 here but you should pick a different one above 1024) and configure your application to bind to that port. Be aware that port-based forwarding offers less security than UNIX socket-based forwarding and that any other user will be able to forward requests to the same port you are using. For that reason, we don’t set the headers we do above as they can easily be forged by another user. Those things being considered, you can put the following in your .htaccess file to enable forwarding requests to a port:

RewriteRule "^(.*)$" http://localhost:999/$1 [P,NE,L,QSA]

Databases

MySQL and PostgreSQL accounts are included with your SRCF account.

Tutorials

So you want to learn how to make cool stuff on SRCF systems? We can help you with that! This is a growing list of tutorials that go into the various things you can do aimed at all levels of skill.

These tutorials show you how to do a variety of things on the SRCF systems. They may assume some prior knowledge in the relevant sections.

Where to start?

The amount of content and tutorials here can be overwhelming, so here’s a few ideas to start:

Simple Minecraft Server

Overview

In this tutorial, you will learn how to set up a vanilla Minecraft Java server hosted by the SRCF.

Introduction

Game servers and other CPU-intensive applications are hosted at doom.srcf.net.

Pre-requisites

To follow this tutorial you will need:

  • An SRCF account
  • Basic familiarity with the terminal

Logging in

Before starting you must log in to doom.srcf.net via SSH. You will have to use your CRSid and your SRCF password.

Log in using your computer’s terminal (PowerShell on Windows) with the following command.

ssh CRSid@doom.srcf.net

Setting up the Minecraft server

I suggest you create a directory for the server using these commands.

mkdir mc
cd mc

The machine already has java set up, so that you can avoid the hassle of getting it working.

The JAR file for the server can be found here. To download the JAR file on the server you must copy the download link from the page and then run the wget command, replacing the url with the address you copied from the download page.

wget https://piston-data.mojang.com/.../server.jar

You can now run the server using the command

nice -n 19 java -Xms1024m -Xmx1024m -jar server.jar nogui

It will set up some directories and then quit, saying that you need to accept the end user license agreement. To do this, open the eula.txt file that was created with a text editor such as

nano eula.txt

Change the line that says eula=false to eula=true and quit using CTRL+X. When asked Save modified buffer? press Y to save your changes.

It’s also necessary to change the port number the server will run on. To do this, open the server.properties file

nano server.properties

and find the lines query.port=25565 and server-port=25565. Replace 25565 with a random number between 1024 and 65535. Remember the number you use, because you will need it later to connect to the server.

If you want to change the gamemode you can do this here by amending the line gamemode=survival.

You can now run the server again using

nice -n 19 java -Xms1024m -Xmx1024m -jar server.jar nogui

This time you should find that the server starts successfully. If you open your Minecraft client and connect to doom.srcf.net:<port> (where is your random number) you should spawn into your world!

Running the server in the background

At the moment you have to log into the machine and start the minecraft server manually every time you want to use it. You can set it up as a systemd service to allow it to run in the background and start up automatically when the machine restarts.

To do this you need to create a service unit file using the commands

mkdir -p ~/.config/systemd/user/
nano ~/.config/systemd/user/mcserver.service

and paste the following into it, replacing both instances of CRSID with your CRSid.

[Unit]
Description=Minecraft Server
ConditionHost=doom
After=network.target

[Service]
Type=simple
WorkingDirectory=/home/CRSID/mc
ExecStart=/usr/bin/nice -n 19 /usr/bin/java -Xms1024m -Xmx1024m -jar /home/CRSID/mc/server.jar nogui
Restart=always
RestartSec=20

[Install]
WantedBy=default.target

Get systemctl to reload the service unit files, enable your service so that it starts up on reboot and start the service using the commands

systemctl --user daemon-reload
systemctl --user enable mcserver.service
systemctl --user start mcserver.service

You should now have a Java Minecraft server ready to play!

If you ever want to stop your server or check its status you can use

systemctl --user stop mcserver.service
systemctl --user status mcserver.service

To learn more about running systemd services with the SRCF see here.

Now that the setup is finished you can log out of doom.srcf.net using logout.

{{ < alert type="warning” > }}

You may notice that in the command to start the server the memory it can use is limited to 1GB and its priority is set using nice. Please do not change the priority or memory usage of your server in order to be considerate to other users of the SRCF’s systems!

{{ < /alert > }}

Closing remarks

Did you like this or find this cool? We invite you to check out more tutorials or get in touch to tell us what you thought!

If you have any suggestions for how we could improve this documentation please send us an email at support@srcf.net or submit a Pull Request on GitHub!

Deploy a web app

Overview

Nowadays, most content served to users via the web is dynamic and via web applications. This tutorial will show you how to set up a simple web application with Flask in Python, but the process is much the same for any other web framework and we give tips for other languages.

Introduction

The SRCF hosts a general purpose web server that can serve all types of content. While the existence of a public_html folder might lead users to think that only static content can be served, Apache’s .htaccess file allows for arbitrary reverse proxying to a TCP port or UNIX socket. This gives users great freedom to do whatever they wish, including hosting applications written in popular frameworks like Django, Flask, Sinatra, Ruby on Rails, Express, etc.

Pre-requisites

Before starting,

  1. Ensure that you are comfortable in some capacity with the terminal
  2. Ensure that you are logged into (and performing any subsequent commands on) our web server, webserver.srcf.net

Environments

Python: set up a venv

You will want to deploy your application in a Python venv so that you can easily install and manage dependencies and versions.

A venv is a virtual Python environment that contains its own Python binary as well as all of your dependencies. When you activate your venv, PATH variables like python and pip will use the correct versions specified when you created your venv.

  1. Create a directory for your app to live in:

    mkdir -p ~/myapp
    cd ~/myapp
    
  2. Set up a venv

    python3 -m venv venv
    
  3. Activate the venv

    . venv/bin/activate
    

    You should do this step every time before running your app or managing installed packages.

  4. Done! Your Python venv is now installed and hooked into your shell.

Node: installing nvm

You will want to deploy your application using nvm so that you can easily install and manage dependencies and versions.

nvm stands for Node Version Manager. Ubuntu only provides a significantly outdated version of Node.js (v4.2.6 at the time of writing) in its repositories. Using nvm allows you to choose any version of Node to use in your environment. Note that it is your responsibility to keep your node installations and nvm itself updated.

  1. Create a directory for your app to live in:

    mkdir -p ~/myapp
    cd ~/myapp
    
  2. Install nvm in your home directory. You’ll need to find the latest version from the NVM GitHub, and copy the one-liner straight into your shell to install it. At the time of writing, it looks like this:

    curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
    

    Make sure you read the installation instructions, then go ahead and run it, and close/re-open your terminal as it suggests.

  3. Install whatever version of node.js you want.

    nvm install 12
    nvm alias default 12
    
  4. Done! The version of node.js you specified is now installed and hooked into your shell.

Ruby: install rbenv

You will want to deploy your application using rbenv so that you can easily install and manage dependencies and versions.

  1. Create a directory for your app to live in:

    mkdir -p ~/myapp
    cd ~/myapp
    
  2. Install rbenv in your home directory:

    git clone https://github.com/rbenv/rbenv.git ~/.rbenv
    cd ~/.rbenv && src/configure && make -C src
    echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
    ~/.rbenv/bin/rbenv init
    
  3. Follow the printed instructions on appending to your ~/.bashrc file:

    echo 'eval "$(rbenv init -)"' >> ~/.bashrc
    
  4. Install ruby-build as a plugin:

    mkdir -p "$(rbenv root)"/plugins
    git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build
    
  5. Install whichever version of Ruby you want.

    rbenv install 2.6.6
    rbenv local 2.6.6
    
  6. Done! You now have a working Ruby installation that’s hooked into your shell.

Install your web framework

You should not rely on the system-wide versions of web frameworks present on our webserver as they are LTS releases and may be out of date. Instead, use the version manager that comes with your framework/language to install a more up to date version.

  • Node: npm i express
  • Python: pip install flask or pip install django
  • Ruby: gem install rails

Create your app

If you have an existing node.js app you can simply copy the code to a directory of your choosing, ~/myapp/src for example. You can use scp/sftp to upload it or clone it using git or some other source control tool.

If you’re new to web apps, though, it’s recommended you follow the getting started guide (or equivalent) for your language/framework of interest:

  • Node: https://nodejs.org/en/docs/guides/getting-started-guide/
  • Express: https://expressjs.com/en/starter/installing.html
  • Ruby on Rails: https://guides.rubyonrails.org/getting_started.html
  • Flask: https://flask.palletsprojects.com/en/1.1.x/quickstart/

We do have a few lines of code in a few languages that can springboard your app below. For real examples, you should visit the content of the sample group at /public/societies/sample/

Flask

There is a demo flask app setup at https://sample.soc.srcf.net/flask/. If you’re logged in via ssh, you can find the code at /public/societies/sample/flask/app.py.

Django

To create a skeleton Django project:

django-admin startproject example
mv example python
cd python

Now take a look at django/example/settings.py for how to configure your site. There is also a demo available at https://sample.soc.srcf.net/django/.

Ruby on Rails

Ruby on Rails is arguably one of the most popular and influential web frameworks. To create a skeleton Rails project:

gem install rails
rails new myapp

You should now have a new Ruby on Rails project setup in the myapp directory.

Sinatra

Sinatra is a lightweight web microframework that’s well suited to simple projects. To install it run gem install sinatra and then put the following in myapp.rb:

require 'sinatra'
get '/' do
    'Hello world!'
end

You can then run this by typing ruby myapp.rb and going to http://localhost:4567 in your web browser. For futher information See the Sinatra documentation.

Express

Sample site at https://sample.soc.srcf.net/nodejs/ and see nodejs/app.js for a minimum base application.

Deploy your app

In a production environment, your app will need to “boot” (ie. start up) in a well-controlled manner. You will also want it to run in the background and ensure it restart when our web server reboots. systemd is a popular choice for “supervising” a service.

Boot scripts

First create a bash script that will run your web app at ~/myapp/run.sh with the following content:

Python

#!/bin/bash -e

. ~/myapp/venv/bin/activate
cd ~/myapp
exec gunicorn -w 2 -b unix:/home/crsid/myapp/web.sock \
    --log-file - main:app

Replace main:app with the module containing the app, and name of your app.

Node

#!/bin/bash -e

USER="$(whoami)"
[ -e "/home/crsid/myapp/web.sock" ] && rm "/home/crsid/myapp/web.sock"
umask 0

. ~/.nvm/nvm.sh
NODE_ENV=production PORT="/home/crsid/myapp/web.sock" \
    exec ~/myapp/src/bin/www

Replace ~/myapp/src/bin/www with the path to your app.

Ruby

#!/bin/bash -e

eval "$(rbenv init -)"
cd ~/myapp/src
RAILS_ENV=production \
      exec bin/rails server -b /home/crsid/myapp/web.sock

Replace ~/myapp/src with the path to your app.

Now, for all frameworks, make the run.sh script executable:

chmod +x ~/myapp/run.sh

You should now be able to execute the script and access your website (or see any errors in your terminal).

Using systemd

The SRCF may restart any of its servers as part of regular or emergency maintenance. When this happens, you’ll likely want your app to start up again. Similarly you may want your app to automatically restart if it crashes. We highly recommend using systemd to supervise your app.

  1. Create a directory for your app ~/myapp.

  2. Place a startup script at ~/myapp/run.sh. Your script should end by execing the server process. If you followed one of our tutorials for Django, Node or Rails, you’ve already created this file, so can move on to the next step. Otherwise, an example would be:

    #!/bin/sh -e
    exec ~/myapp/run-server
    

    Your server should run in the foreground (it should not daemonise), and the run.sh script should end with an exec line so that signals are sent to the server (and not to the shell that started it).

    Once you’ve written the script, make it executable (chmod +x ~/myapp/run.sh). Test it by executing it in your terminal before moving on; it will be easier to debug problems.

  3. Write a systemd service file so your app will be supervised on startup.

    For applications in your personal account, create the unit directory if it doesn’t exist:

    mkdir -p ~/.config/systemd/user
    

    For a group account, substitute ~ for /societies/foosoc, where foosoc is the short name of the account.

    Then, save the following to the file ~/.config/systemd/user/myapp.service (or /societies/foosoc/.config/systemd/user/myapp.service for groups):

    [Unit]
    Description={YOUR USER, SOCIETY OR GROUP NAME} Webapp
    ConditionHost=sinkhole
    
    [Install]
    WantedBy=default.target
    
    [Service]
    WorkingDirectory=/home/{CRSid}/myapp
    ExecStart=/home/{CRSid}/myapp/run.sh
    Restart=always
    
  4. Tell systemd to start your app on startup, by running systemctl --user enable myapp.

  5. You’ll need to start your app manually once (on future reboots, it will be started for you). To do that, run systemctl --user start myapp.

    To control your app, you can use the systemctl tool. See man systemctl for full details.

    In summary,

    • Restart an app. systemctl --user restart myapp
    • Bring an app offline. systemctl --user stop myapp
    • Bring an app back online. systemctl --user start myapp
    • Check the status of an app. systemctl --user status myapp

    By default, your app’s standard output and error streams are sent to systemd’s journal however only the root user can access these. You will want to make your app write to a log file rather than stdout or stderr.

We have further more general reading on systemd services.

Notes for Python

Installing gunicorn

We recommend using gunicorn to serve your application. After activating your venv, install it with pip install gunicorn.

Note that you may see a warning about a syntax error. As long as the output ends in ‘Successfully installed gunicorn’, it’s safe to ignore this.

Reloading your app

Gunicorn will reload your app if you send it SIGHUP. You can teach systemd that fact by adding the following line under [Service] in your systemd unit file:

ExecReload=/bin/kill -HUP $MAINPID

and then running systemctl --user daemon-reload. After that, you can use systemctl to reload your app:

systemctl --user reload myapp

Notes for Ruby

  • You may need to migrate your database first.
  • Make sure you’ve set secret keys for the app and any gems that need them (e.g. Devise).
  • Static file serving is off by default in production, but you’ll want to turn it on: set both config.assets.compile and config.public_file_server.enabled to true in config/environments/production.rb.

Forwarding requests

You then need to configure Apache to forward web requests to the web.sock socket specified in the run.sh script. We explain how to do this in the reference for web apps.

Closing remarks

Did you like this or find this cool? We invite you to check out more tutorials or get in touch to tell us what you thought!

If you have any suggestions for how we could improve this documentation please send us an email at support@srcf.net or submit a Pull Request on GitHub!

Getting to know the terminal

Overview

This tutorial will walk you through some more advanced commands that you can perform in a UNIX-like terminal, which the SRCF provides you with. You will learn many of the basic commands that are used and why in conjunction with other tools they can be very powerful.

This tutorial assumes you have an SRCF personal account (with or without group account admin permissions) and can SSH. See our other tutorials for that.

Introduction

The SRCF offers you much more than just a web server - everyone who has an account has full shell access to our Linux servers. This allows you to manipulate and edit the contents of your filespace directly without having to worry about downloading and uploading files, as well as running all sorts of other software which you might not have available to you under your usual operating system.

Basic commands

After successfully logging in, you will see a few lines of text welcoming you to the system and telling you any recent news, followed by a command prompt:

sqpr2@pip:~$

You are now logged in and can give the system commands by typing them after the prompt. If you wish to start a particular program, the command is just the name of the program you wish to run. For example typing ‘date’ and pressing return will tell you the current date and time:

spqr2@pip:~$ date
Wed Apr  5 22:29:11 BST 2020
spqr2@pip:~$

After each command has finished, a new prompt will be displayed ready for the next command. Some interactive programmes, such as the email reader pine will fill the whole screen, but will still return to the command prompt afterwards.

Commands with arguments

Some commands require arguments, that is additional parameters which alter the way that they run, for example the command ‘ping’, which is used to check the speed of an Internet connection, requires the name of a host as an argument:

spqr2@pip:~$ ping www.bbc.co.uk
PING www.bbc.net.uk (212.58.224.35): 56 data bytes
64 bytes from 212.58.224.35: icmp_seq=0 ttl=247 time=4.9 ms
64 bytes from 212.58.224.35: icmp_seq=1 ttl=247 time=4.6 ms

--- www.bbc.net.uk ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 4.6/4.7/4.9 ms
spqr2@pip:~$

(To stop the ‘ping’ program, press control-c)

File system

All files stored on the system reside in directories (this is the equivalent of Windows folders). The directories are arranged in a hierachical structure - directories may contain subdirectories and so on. The top of the hierarchy is called the root directory, and is represented by /. Other directories are refered to by their path, for example, /home/abc45/ represents the directory abc45 which is a subdirectory of ‘home’, which in turn is a subdirectory of the root directory.

Files can be specified by adding the file name to the end of the path, for example /home/abc45/my_file.txt is a reference to the file my_file.txt which is stored in the directory /home/abc45/.

At any point during your session you will be ‘in’ a particular directory called the working directory. Rather than using full paths as above, it is possible to refer to the location of files relative to working directory by omitting the leading /, for example ‘my_dir/my_file.txt’ represents the file ‘my_file.txt’ within the directory ‘my_dir’, which is a subdirectory of the working directory.

Filenames given without a preceding path are assumed to be in the working directory. All directories have a special subdirectory called .. which refers to the directory one level higher up in the hierachy (the parent direcvtory), so ../some_file.txt is the file some_file.txt in the parent directory of the working directory.

To find out your current directory, use the command pwd:

spqr2@pip:~$ pwd
/users/abc45
spqr2@pip:~$

You can change the working directory by using the cd command, for example:

spqr2@pip:~$ cd my_dir
spqr2@pip:~/my_dir$

Notice how the working directory is displayed as part of the command prompt. The symbol ~ is an abbreviation for your home directory, in other words the directory you start off in when you first log in.

To view the contents of the working directory, use the command ‘ls’:

spqr2@pip:~$ ls
public_html  mygroup  my_file.txt
spqr2@pip:~$

Alternatively, use ls -alF to give more detailed information. The above example shows that the working directory has two directories public_html and mygroup, the first of which contains your web space and the latter of which is a link to the mygroup group account file space.

There is also one file, ‘my_file.txt’ (there is no way of differentiating between files and directories in the above example - you need to use the ls -alF form to show that information).

More commands

The following table gives some common commands used to manipulate files:

+-----------------------------------+-----------------------------------+
| Command                           | Meaning                           |
+-----------------------------------+-----------------------------------+
|     cp <file1> <file2>            | Creates a copy of 'file1' at      |
|                                   | the location specified by         |
|                                   | 'file2'                           |
+-----------------------------------+-----------------------------------+
|     mv <file1> <file2>            | Moves 'file1' to the location     |
|                                   | specified by 'file2'. Note that   |
|                                   | if 'file1' and 'file2' are in     |
|                                   | the same directory you can use    |
|                                   | this command to rename the file.  |
+-----------------------------------+-----------------------------------+
|     rm <file>                     | Deletes 'file'                    |
+-----------------------------------+-----------------------------------+
|     mkdir <dir>                   | Creates a directory called        |
|                                   | 'dir'                             |
+-----------------------------------+-----------------------------------+
|     rmdir <dir>                   | Removes the directory 'dir' (it   |
|                                   | must be empty first)              |
+-----------------------------------+-----------------------------------+
|     nano <file>                   | Edits 'file' using the 'nano'     |
|                                   | text editor.                      |
+-----------------------------------+-----------------------------------+

Note that the last command starts the nano text editor. This is a simple editor which is sufficient for most tasks. If you get stuck in it, pressing control-x will return to the command prompt. If you are looking for a more powerful editor, albeit with a steeper learning curve, you might want to try vim.

Exiting

To log out of the system, type ‘exit’ at the command prompt.

Closing remarks

Did you like this or find this cool? We invite you to check out more tutorials or get in touch to tell us what you thought!

If you have any suggestions for how we could improve this documentation please send us an email at support@srcf.net or submit a Pull Request on GitHub!

Passwordless login: SSH keys

Overview

Passwords aren’t very safe. Instead of remembering multiple passwords for multiple servers (nowadays made easier with password managers), it is common practice to use so-called SSH keys to login to a remote server. This tutorial will show you how to set-up an SSH key pair so you can log in to any of our servers without the need of a pesky password!

Introduction

A well-established method of authentication is via passwords. This, however, has been shown to be quite vulnerable, time and time again. An alternative solution to logging in to servers via SSH that offers lots of flexibility, is to use SSH keys.

SSH keys always exist in pairs: there is a private key and a public key. You will keep the private key while placing your public key in any servers you want to access. Just like you shouldn’t use one password for everything, you should use multiple private keys if you have access to a reasonable quantity of servers.

When logging in to a server, the SSH client on your computer requests the public key for the account you’re trying to log in to and checks to see if your private key is compatible with that one. This works because Math.

Generating a key pair

  1. Open your terminal of choice

  2. Paste in the following command with an email of your choice

    ssh-keygen -t ed25519 -C "your_email@example.com"
    

    This creates an SSH key pair, using your email as a label. You should see the following output

    Generating public/private ed25519 key pair.
    
  3. When you’re prompted to “Enter a file in which to save the key,” press Enter. This accepts the default file location, which is in the .ssh directory in your home directory.

  4. Optionally enter a password when prompted.

Adding your keys to ssh-agent

You can use ssh-agent to securely save your passphrase so you don’t have to reenter it. You can find further guidance on this here.

Adding your key to the server

You now need to add your public key to one of our servers. Since all of our user-facing server share file systems, you only need to do this once.

A handy utility exists for this:

ssh-copy-id -i ~/.ssh/mykey spqr2@pip.srcf.net

More useful information on that here.

If that doesn’t work, you can always upload your public key to the server as you would a normal file. Your public keys should go in the ~/.ssh/authorized_keys file, separated by line breaks (press “enter”).

Logging in

With your SSH keys in place, you should now be able to ssh sqpr2@pip.srcf.net without being prompted for your password!

You can even make this process simpler by setting up an SSH configuration file, to, for example, be able to type ssh srcf-webserver and log into the web server!

Closing remarks

Did you like this or find this cool? We invite you to check out more tutorials or get in touch to tell us what you thought!

If you have any suggestions for how we could improve this documentation please send us an email at support@srcf.net or submit a Pull Request on GitHub!

Setting up IRC

Introduction

Internet relay chat is a protocol for real-time text messaging between internet-connected computers. While it is dated in comparison to some other messaging protocols, it continues to be used regularly by SRCF members and as the emergency messaging platform of choice, with several servers located around the world.

In this tutorial, you’ll be setting up a persistent IRC connection using irssi and screen. It’s a great entry point to the terminal and beginners are encouraged to follow this tutorial!

Installing

Install irssi

sudo pacman -S irssi # Arch
sudo yum install irssi # Red Hat derivatives
sudo apt-get install irssi # Debian derivatives

Install screen

sudo pacman -S screen # Arch
sudo yum install screen # Red Hat derivatives
sudo apt-get install screen # Debian derivatives

Configuring

Create logging directory

mkdir ~/irclogs

Getting this ready for later…

Check if screen is installed

screen -list

If there’s more than one screen session, you’ll have to enter its name at the end of the next command in Step 5.

Start a screen session

screen -RD

Consult the GNU screen manual (https://www.gnu.org/software/screen/manual/screen.html).

Start irssi

irssi

Having done this you should be presented with an empty irssi window.

Essential settings

Enter these commands in your irssi prompt.

/SET nick <nick>
/SET alternate_nick <alternate nick>
/SET user_name <username>   # not necessary if ident enabled
/SET real_name <real name>
/SET use_msgs_window OFF

# Highlight lines containing your nick
/HILIGHT <nick>

# Set timestamp to something useful
/SET timestamp_format %d.%H:%M:%S

Set up logging

Logging is useful because the IRC connection is not persistent over system restarts (logging will allow you to see any messages you missed between the last time you checked IRC and the time the system restarted). It’s also useful if you’re receiving e.g. tech support, so you have a record of any instructions you were given!

/SET autolog ON
/SET autolog_level ALL -CRAP -CLIENTCRAP -CTCPS
/SET autolog_path ~/irclogs/%Y/$tag/$0.%m-%d.log

Set quit message

/SET quit_message <message>

Default is ‘leaving,’ mine is ‘Scotty, beam me up!’ (which is actually the closest Kirk ever came to saying the commonly misquoted ‘beam me up Scotty!’ He never said anything remotely like this in the TV series but said the above in Star Trek IV).

Add network(s)

For example, ‘srcf’.

/NETWORK ADD [-nick <nick>] <Network name (your choice)>

Add one or more servers

The FQDN in our case is irc.srcf.net.

/SERVER ADD [-auto] -network <network name> <server FQDN>

Add one or more channels

Check out an overview of the channels on our server to decide which ones to add.

/CHANNEL ADD -auto #<channel name> <network name>

Connect to a server

/CONNECT <network name>

Once on a server you can use Alt+num, where num is 0-9, to change channel.

Joining channels whilst on a server

/JOIN #<channel name>

Window Positioning

Move around windows until you get them in an order that you like (first go to the window that you wish to move):

/wm <position to move to>

(or)

/window move <position to move to>

Save settings and layout

/SAVE
/LAYOUT SAVE

Restart irssi and confirm everything is set correctly

/QUIT

You should now arrive at a terminal, where you can type irssi to restart the client and check that everything is working.

Scheduled restarts

You will also need to create the screen session every time the SRCF reboots, unless you set up an @reboot cron job containing screen -Udm irssi using crontab -e.

To disconnect from screen without quitting IRC press Ctrl-A, and then press D. This will place you back onto the command line, where you can just type Ctrl-D to quit.

When you want to reconnect later use screen -rd after you have logged in, and your should find your irssi exactly as you left it.

  1. https://www.irchelp.org/irchelp/new2irc.html
  2. https://www.irchelp.org/irchelp/changuide.html
  3. https://www.srcf.net/faq/irc#persistent
  4. https://irssi.org/documentation/manual/
  5. https://irssi.org/beginner/#c3
  6. https://quadpoint.org/articles/irssi/
  7. http://www.kuro5hin.org/story/2004/3/9/16838/14935
  8. https://www.gnu.org/software/screen/manual/screen.html

Appendix A: useful irssi commands

  • List current networks: /NETWORK

  • Remove servers (especially useful for getting rid of irssi’s default servers):

    /NETWORK REMOVE <server>
    
  • See all settings (to verify that there’s nothing unexpected in there): switch to status window and type /set.

  • Indent (useful with very long lines, so messages can be distinguished; experiment with exact value):

    /SET indent 4
    
  • Quit irssi:

    /quit
    
  • Leave channel (after having gone to the channel’s window):

    /part
    

    (or)

    /part <message>
    

An alternative, but less elegant way, is to just close the window. This method is needed if it is a split or hidden (e.g. for closing a query window): /wc or, in full, /WINDOW CLOSE

  • List all channels on IRC server that are public to join (not good for large networks): /list

  • Set topic: /topic

  • Switch between networks: Ctrl+X

  • List channels known to irssi: /CHANNEL LIST

  • Reload configuration:

    /RELOAD
    /RELOAD <filename>   # for loading a different config file
    
  • List users in the current channel:

    /names
    /n
    
  • Display user information for a given <nick>:

    /whois
    /wi
    

Joins/Parts

A number of useful things can be done here.

If you are in relatively few channels that have problems with people joining/leaving frequently, then just ignore for that channel:

/ignore #<channel> MODES JOINS PARTS QUITS
/ignore -except -pattern <yourNick> #<channel>

Replace # with the wildcard operator (*) to do this for all channels.

An alternative way for doing this for all channels is to add the following to ~/.irssi/conf:

ignores = ( { level = "JOINS PARTS QUITS"; } );

For a more in depth discussion of levels, and how to put all join/part/quit messages into their own ‘junk’ window (freeing up other windows for discussion), see https://pthree.org/2010/03/12/irssi-handling-joinspartsquits/

Appendix B: useful screen commands

Screen is an incredibly useful tool for keeping any process (not just IRC) running after you terminate an ssh section. There are lots of useful commands listed in the screen manual, and a few are listed below https://www.gnu.org/software/screen/manual/screen.html

  • ctrl+a x locks screen.
  • ctrl+a c creates a new window.
  • ctrl+a n switches to the next window.

Suggestions/improvements?

Did you like this or find this cool? We invite you to check out more tutorials

If you have a better way to join the SRCF IRC server (or any other suggestions for how we could improve this documentation), send us an email at support@srcf.net or submit a Pull Request on GitHub!

Uploading files

Introduction

This tutorial will help you upload your first files to the SRCF so you can serve them up from your web page or use them in whatever project you’re working on.

In this tutorial, you will be using SFTP to upload your files. Let’s get started!

Background

FTP stands for ‘file transfer protocol’ and is another method of connecting to a web server (different to HTTP). It’s the quickest and easiest way to transfer files between the computer you’re working on and the web server where they’ll be supplied to people viewing your website.

Get an SFTP/SCP client

We recommend one of the following:

Some of these might only be available for macOS or Windows or vice versa.

Connecting

  1. Use files.srcf.net as the server name (sometimes referred to as the hostname)
  2. Give the username and password that were sent to you when your SRCF account was created.

Uploading a file

When you first log in, you should see the contents of your home directory on our server. Inside your home directory you will find a public_html directory — this is where you can put your personal website.

If you want to upload a group website, you need to connect to the server in the same way, still using your personal username and password. However, rather than putting your files in the public_html directory in your filespace, you should also find a directory named after the short name used for the group’s account. Inside here you’ll find another public_html directory which is where the group website files go.

Please note that it can take up to 20 minutes between uploading a new site and that site being published on the web. Please be patient.

Closing remarks

Did you like this or find this cool? We invite you to check out more tutorials or get in touch to tell us what you thought!

If you have any suggestions for how we could improve this documentation please send us an email at support@srcf.net or submit a Pull Request on GitHub!

Using SSH to log in

Overview

Just as a butterfly emerges from its coccoon to explore the great wonders of the world, you, too, will also uncover the great wonders of Linux with the SRCF by your side. In this tutorial, we’ll show you how to log in to a UNIX-like shell and use the basics of the command line.

Background

Much of the functionality offered by the SRCF is accessible via a text-mode ‘shell’ interface on our server — using this interface you can manipulate files on your website directly, as well as running other programs such as email software. When you get an SRCF account, you can connect to the server and issue commands by typing them. If you’re not used to this, don’t worry — it’s much less primitive than it sounds and very powerful!

Because you are remotely accessing our servers, as opposed to your own computer, we will need a communication protocol. The most widely used one is called SSH, secure shell protocol.

Logging in

Linux or macOS

If you’re running a Unix-based computer (including macOS) then logging in is easy as it’s built into the system:

  1. Start your terminal
  2. Type in ssh <username>@shell.srcf.net
  3. Enter your password when prompted

Windows

If you’re running a system without a native console, such as MS Windows, then things are slightly trickier. Not much though, you just have to go and get a console program. We recommend the PuTTY application, downloadable for free here.

Once downloaded,

  1. Run PuTTY
  2. Enter the server name as shell.srcf.net, the protocol as SSH (port 22) and click to connect
  3. Enter your username and SRCF password when prompted

First steps

Now you’re in! Let’s see what this terminal is capable of!

Try running ls. You should see a bunch of text printed in the terminal. These are your directories and files. By default, you start out in your home directory once you log in. The absolute path is /home/CRSid. You should, at minimum, see a public_html folder. As you may have already discovered ls stands for list! You can pass in additional flags to change its output: ls -la.

Now, try runnng pwd. This stands for print working directory. Think of this as a street sign. Right now, you’re somewhere in the middle of complex and multi-layered file system on our shell server, which happens to be your home directory. pwd tells you where you are in this organized mess!

The last command we’ll try is cd public_html. This changes your directory to public_html. You’re now on a different street!

The public_html directory is special. Any content placed here will be served by our web server, Apache. Check out the tutorials to learn what to do

If you’re interested in what else you can do with the shell, check out our more advanced tutorial.

Closing remarks

Did you like this or find this cool? We invite you to check out more tutorials or get in touch to tell us what you thought!

If you have any suggestions for how we could improve this documentation please send us an email at support@srcf.net or submit a Pull Request on GitHub!

Virtual desktops over VNC

Introduction

The SRCF desktop server shell.srcf.net has VNC server software installed (we’re using TightVNC). This allows you run a graphical desktop on our server, and connect to it remotely.

However, TightVNC does not encrypt sessions, so you must connect through SSH in order to avoid possible eavesdropping over the internet.

To start it up, you’ll first need to be able to connect over SSH to the SRCF desktop server. Linux and Mac should already provide the ssh terminal command; Windows users may download PuTTY for a native SSH client.

Setting a VNC password

SSH to shell.srcf.net using your SRCF username and password (SSH keys for your account are also valid), then run the vncpasswd command:

spqr2@pip:~$ vncpasswd
Using password file /home/spqr2/.vnc/passwd
Password:
Verify:
Would you like to enter a view-only password (y/n)? n

You can say no to a view-only password, for this use case it’s unlikely to be useful.

Starting a new session

To start a new VNC server process in the background:

spqr2@pip:~$ vncserver -geometry 1920x1080
New 'X' desktop is pip:12

Starting applications specified in /home/spqr2/.vnc/xstartup
Log file is /home/spqr2/.vnc/pip:12.log

Geometry is of the form <width>x<height> and may be customised as desired (the VNC window cannot be resized after you have started the VNC server).

Take a note of the :12 in the output; this is the display number assigned automatically to your VNC server and could be any two-digit number.

Add 5900 to the display number to obtain the port number (in this example, the port number is 5912).

Tunneling over SSH

As noted above, VNC server connections are insecure. Instead, you should use an SSH tunnel – this provides encryption over the wire for free, and allows you to securely authenticate to, and make use of, your VNC server instance.

The aim is to forward your VNC port on the desktop server, to a port of your choosing on your local machine.

Linux and Mac

Back on your local machine, SSH to shell.srcf.net again, but request forwarding for your chosen port:

you@home:~$ ssh -N -L 5901:localhost:5912 shell.srcf.net

Replace 5912 with the port number for your VNC server (the display number shown when you started vncserver, plus 5900).

Windows

In PuTTY’s main window, go to ‘‘Connection’’ > ‘‘SSH’’ > ‘‘Tunnels’'.

Under ‘‘Add new forwarded port’', set the source port to 5901 (the local port on your home machine), and the destination to localhost:5912 replacing 5912 with the port number for your VNC server (the display number shown when you started vncserver, plus 5900).

Connecting with a VNC client

At this stage, you should now have port 5901 on your local machine acting as a tunnel to your VNC server instance on shell.srcf.net. The last step is to actually point your VNC client to it.

If you don’t have a VNC client installed, this Ubuntu page lists some possibilities, though may be out of date.

Start your client and connect to localhost:5901 (this is relative to your home machine). If all is well, you should be prompted for the VNC password you created at the start, after which you’ll be greeted with your new desktop.

You can disconnect and later reconnect, and the state should be preserved.

Ending the session

Back over SSH to shell.srcf.net:

spqr2@pip:~$ vncserver -kill :12

Replace :12 with the port number for your VNC server.

Make sure to close any running programs over VNC beforehand, as this will forcefully kill them too.

Statically generated sites

Introduction

Broadly speaking, there are two types of content that can be served by a web server: dynamic and static. Some websites such as social media or online banking are inherently dynamic by nature; the pages and content you see there are constantly changing. But other sites often don’t have any dynamic content and simply deliver static information to users. Using a web application or fully-fledged content management system for the latter can lead to poor performance, for example slow page load times and unnecessary CPU load on the web server.

Overview

Static site generators have emerged as powerful tools which let you turn simple markup into professional and performant websites without having to write cumbersome or repetitive HTML. They allow you to leverage a variety of markup formats, from the simple to the powerful, to produce websites with consistent styling that load quickly. And you’re not totally precluded from having dynamic content in your site; Gatsby is an example that has good integrations which can insert new content into your pages.

A few reputable SSGs out there include:

Closing remarks

Did you like this or find this cool? We invite you to check out more tutorials or get in touch to tell us what you thought!

If you have any suggestions for how we could improve this documentation please send us an email at support@srcf.net or submit a Pull Request on GitHub!

WordPress from scratch

Overview

In this tutorial, you’ll learn how to create a fully functional blog/personal website using WordPress. This is a great first project if you’re a user starting out with the SRCF, websites in general or want to use a tried-and-tested Content Management System.

This tutorial assumes you have an SRCF personal account (with or without group account admin permissions) and can SSH and transfer files. See our other tutorials for that.

Setting up your database

WordPress needs a MySQL database to run. This works a bit like Excel, storing all of the data for your website in tables and loading this into templates when each page is loaded. You can create your MySQL database here.

Sanity check

Now that you’ve got everything setup, let’s check if you can log in with an FTP client. You’ll want to connect to files.srcf.net via SFTP (the S stands for secure). You should see a folder called public_html which we’ll be uploading all of our files to.

Next up, check that you can access your database: log in to phpMyAdmin. This is a web-based control panel where you can view and edit your database manually. You should see your personal database on the left along with any group account databases. They should all look pretty empty at this point.

If everything is working so far and you have all of your login details and passwords, we’re good to go! Congratulations, that was the hardest bit of the whole process…

Installing WordPress

Head over to wordpress.org and hit the big blue download button. Make sure you don’t get confused with wordpress*.com* - that’s where you can set up a blog directly with the WordPress organisation using their hosting. We want to host our own copy of WordPress on the SRCF.

Once your .zip file has finished downloading, we need to get the files from your computer onto the SRCF hosting. Unzip the file on your computer and upload the files into your public_html directory, making sure that they’re not in a subfolder called wordpress.

Configure WordPress

Great! Now if you head to your website in a browser https://**crsid**.user.srcf.net for an indvidual account or https://**groupname**.soc.srcf.net for a group account) you should see a friendly welcome screen. Fill in all of the details that it asks for - leave localhost as it is, try not to use wp_ as the database prefix or admin as the administrator username (this makes it harder for hackers to infiltrate your site).

Set up automatic updates

WordPress will likely refuse to do automatic updates, instead requesting FTP credentials – this is because it assumes it can’t write its own files on the server. To make this work you can add the following few lines to your wp-config.php, near the bottom but above the line that says ‘stop editing’:

# Force WordPress to use direct filesystem access so that upgrades work properly.
# https://core.trac.wordpress.org/ticket/10205
# https://codex.wordpress.org/Editing_wp-config.php

define('FS_METHOD', 'direct');
define('FS_CHMOD_DIR', (02775 & ~ umask()));
define('FS_CHMOD_FILE', (0664 & ~ umask()));

If configured correctly, the updates page should include text similar to Future security updates will be applied automatically.

If you have a low-traffic or private site, WordPress’ cron service (which handles updates and other background tasks) may not run regularly enough. You can invoke wp-cron.php manually using cron or systemd timers – see crontab or wordpress-cron.timer in /public/societies/sample.

Secure your install

There are crucial steps you must take to secure your WordPress installation. As this applies to existing WordPress installations, we’ve written a separate guide for it.

Customising WordPress

You’ll probably want to make your site look a little different and certainly contain some content, so the first thing to do is log in to the WordPress administration panel. There’s usually a link saying something obvious, but if you can’t find it the URL should look like https://csrid.user.srcf.net/wp-admin. Log in with the username and password set up in the previous step.

Content

How you create content will depend on how you intend to use the site. Posts are designed for small regular news updates, Pages are for larger static content. You don’t have to use either, you can use both. If you don’t want to use Posts at all, you can display a page on your homepage in (Settings > Reading).

Visual changes

WordPress websites use themes to customise the way that they look. Thankfully lots of people are nice enough to write these for you and give them away for free (there are plenty of paid-for themes a google search away too). You can browse these directly through the WordPress admin back end (Appearance > Themes) or find one online.

Features

If you want to extend the functionality of WordPress, you need to install a Plugin. Millions of plugins have been written and are dead easy to install. The chances are that if you want your site to do something then someone has probably written a plugin to do it.

Accounts

  • Log in to WP as admin using the password given at the end of the installation process.
  • Click on My Profile and change the admin password using the form provided.
  • Click on Users and create an account for yourself.

Optional steps

Custom domains

You might be thinking that the *.user.srcf.net or *.soc.srcf.net web address of your spangly new site isn’t particularly inspiring. Don’t panic! You can register any domain you like (or even multiple domains) and point them to your new site by following our custom domains documentation.

Raven

You can put any site (or parts of your site) behind Raven by following our Raven docs.

Closing remarks

Did you like this or find this cool? We invite you to check out more tutorials or get in touch to tell us what you thought!

If you have any suggestions for how we could improve this documentation please send us an email at support@srcf.net or submit a Pull Request on GitHub!

A big thanks to Phil Ewels for writing this on his own blog and to hmw26 for some of their tips. This tutorial contains some adaptations to their work.

For more general questions, including how to build a website or use the Linux command line, a number of tutorials are available on the web.

The UIS provide a number of relevant training courses and online resources.