Set Up Ubuntu Server 架設Ubuntu伺服器

Set Up Ubuntu Server 架設Ubuntu伺服器 KCTang Tue, 28/05/2019 - 02:55

Go to End

Note

28 May 2019: Notes regarding unsuccessful installation added. Showing boot messages added.

7 May 2019: Contents added.

25 Dec 2014: First created as a flysheet without its own contents.

Intro

Ubuntu is a linux server software.

It has a desktop version and a server version, amongst other products.

The kernels of the desktop version and the server version are the same. The installation procedures are slightly different.

The desktop version provides a graphical user interface and the server version has a text based user interface, though a graphical user interface can subsequently be installed on the server version. Once installed with the graphical user interface, the two versions would not look much different to the users. 

For small system, the desktop version should be easier to use.

Download

Download free of charge here

Install

Read installation guides:

Read fuller user guide here.

Set up storage drives

When allocating drive spaces, choose "Something else" with the desktop version and "manual" with the server version if one does not want to adopt the default settings, e.g. if one wants to have more options, such as setting one drive for "/boot" partition and one for "/" root system partition.

Set the computer bios to boot from the selected boot up disk. Remember to save the setting.

It is said that the boot partition needs only be about 300 Mb big. However, experience tells that it gets full easily because of frequent updating of the kernels and retainage of the last few kernels in the boot partition.

To remove kernels no longer needed to be retained, execute:

$ sudo apt autoremove

However, sometimes, the 300Mb boot partition has gone up to 100% full and insufficient to contain the required last few kernels such that there are no unused kernels to be removed to free up space.

Therefore, it is recommended to set up a boot partition of 1Gb.

Our company set-up:

  • boot partition on the smaller SSD disk sold with the computer
  • root system partition on the bigger hard disk relocated from old computer
  • additional hard disks mounted on the system for storage of data

To see disks mounted, execute:

$ df -h

The mounting configuration file is contained in /etc/fstab.

To mount disk using graphical user interface permanently, click "Activities" , enter "disk" and select the "Disks":

Mounting using "Disks" would change /etc/fstab permanently, without the need to change the /etc/fstab file manually.

Install application software packages

Update the software repository:

$ sudo apt update

Install software package:

$ sudo apt install <name of software>

Uninstall software package, keeping its configuration files:

$ sudo apt remove <name of software>

Uninstall software package and its configuration files:

$ sudo apt purge <name of software>

Alternatively, to bring up the graphical software package installation manager, execute:

$ sudo synaptic

or, on the graphical desktop, click "Activities" , enter "synaptic".

Upgrade Ubuntu release

Upgrade Ubuntu release:

$ sudo do-release-upgrade

Re-install

If it is necessary to re-install Ubuntu, back up everything first. 

Ubuntu usually can recognize the existing boot and root systems and data disks when Ubuntu is re-installed.

Try to keep the new co-existing with the old and change later after the system is running.

Keep the existing configurations and re-use the existing configuration files as much as possible.

Most of the configuration files are stored in the /etc directory.

If Ubuntu is freshly installed, copy back the files and directories of the application software to the new system before re-installing the application software, e.g. the following files or directories:

  • /etc/aliases
  • /etc/aliases.db
  • /etc/anacrontab
  • /etc/apache2
  • /etc/ca-certificates
  • /etc/ca-certificates.conf
  • /etc/dovecot
  • /etc/hostname
  • /etc/hosts
  • /etc/openvpn
  • /etc/phpmyadmin
  • /etc/postfix
  • /etc/samba
  • /etc/vsftpd.conf
  • /etc/vsftpd.chroot_list

Do not copy back the /etc/fstab file because the file system configuration there is no longer applicable to the new system and copying back will cause the new system not re-bootable.

If copied back, use "Disk" software mentioned above to make some changes so that the /etc/fstab file reflects the latest configuration, before re-booting.

The information of the previous user names, groups and passwords are contained in the following files:

  • /etc/passwd
  • /etc/group
  • /etc/gshadow
  • /etc/shadow

Use text editor and spreadsheet software to pick out old information in the old files and not superseded by the new information and copy it cover to the new files to enable the users to access their previous data directories, otherwise, re-redefine all the user names and passwords one by one.

Re-install the additional application software after coping back as described above.

The software might have updated the default configuration files over time but usually would tolerate using the old configuration files. Therefore, it is preferred to keep using the old configuration files to ensure that the software is successfully re-installed before making changes.  There may be very slight changes to  new configuration files. The installation process will usually prompt to ask which file to keep and give an opportunity to see a comparison. It may be better to record the changes first and change later. Using the new configuration files in conjunction with the old physical configurations may cause the software not workable. Carefully check for any slight changes to the new configuration files if using the new configuration files to add the old configuration settings.

If re-installation is not successful after :

$ sudo apt install <name of software>

try:

$ sudo apt remove <name of software>

or even:

$ sudo apt purge <name of software>

or even remove the software directory before:

$ sudo apt install <name of software>

Usually, it is the configuration which is causing problems.

(The following added, 28 May 2019)

The "remove" option will leave behind the user modified configuration files.

Rename them.

Re-install the application software.

Use the new configuration file if see if the application software can run successfully.

If yes, copy the user modifications in the old configuration file to the new so far as compatible, and see whether the software application can run successfully.

The "purge" option will remove all configuration files and other related software. Care should be exercised to review that the related software would not be required by other software.

Removing the software directory is the extreme option with little additional effect. 

Show messages when booting

(section added, 28 May 2019)

Execute:

$ sudo gedit /etc/default/grub

Specify:

#GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"    (uncomment the default)
GRUB_CMDLINE_LINUX_DEFAULT=""                 (add)

Execute:

$ sudo update-grub

 

End of Page

Install backintime

Install backintime KCTang Tue, 12/05/2020 - 16:30

Go to End

Note

12 May 2020: Updated.

30 May 2019: Created.

Intro

backintime enables incremental backup of files.

Listing it as the first application software to be installed after installing Ubuntu is an indication of the importance of backing up files.

Install

Execute:

$ sudo apt install backintime*
$ sudo apt install backintime-qt

(updated 12 May 2020)

Bring up

Select Activities.

Type "ba" to bring up some application icons:

Select Back in Time (root) to show the front page:

Alternatively, execute

$ sudo i backintime
or
$ pkexec backintime

However, the terminal commands are not reliable to bring up backintime. Using the graphical desktop is more reliable.

(updated 12 May 2020)

Config

Select the setting button:

Select the General page:

Specify:

  • where to save snapshots
  • host computer name, user name "root" to have widest permission, profile starting with 1
  • scheduled intervals and hours to save snapshots

Select the Include page > Add folder to select the folders to include in backup :

Add file can only include files not a whole folder.

Select Exclude page.

Generally accept the default exclusions.

Select Add folder to select sub-folders to be excluded from the backup when their parent folders have been included:

Select Auto-remove to set time criteria to automatically remove old spanshots:

Select Options page.

Generally accept the default settings.

Select Expert Options page.

Generally accept the default settings.

Select OK to save.

Exit the software. It will run at the specified times.

Take snapshot any time

Bring up the front page:

Select the take snapshot button: 

It would take some time to take a snapshot depending on the extent of file changes since the last snapshot.

Restore

Restore backup files when the present files have been lost or corrupted.

Bring up the front page:

 

Select:

  • the desired snapshot in the left window
  • the desired backup folder in the middle window
  • one or more folders or files in the right window

Select the restore button:

Select whether to backup local files with a trailing suffix before restoring the old files:

Select Yes to proceed if for sure.

Folders and files will be restored.

It would take some time depending on the size to be restored.

Let the process finish with the use permissions set back to the original state.

If only the folders in the middle window is selected before the restore button is selected, this message will appear to indicate that the "/" root folder will be restored:

It is important not to just select the folders in the middle window and select the restore button to proceed, because it will easily restore files to the "/" root directory and overwrite the still valid system and programme files in the root directory to cause problems.

Repair

Sometimes, backintime cannot be brought up to run, even after a removal and re-installation.

This can be caused by the removal of some other programmes which backintime depends on but which have been removed when some other application software is removed. Re-installation of backintime may not bring them back.

The following shows the programmes which backintime depends on:

Try to re-install the missing programmes:

$ sudo apt install <name of programme>

rsync, python3, openssh-client are likely missing programmes.

 

End of Page

 

Install Samba file server for Windows

Install Samba file server for Windows KCTang Tue, 07/05/2019 - 20:16

Go to End

Note

7 May 2019: Slight adjustments. "gksudo gedit" changed to "sudo gedit" as Ubuntu 18.04 dropped "gksudo".

25 Dec 2014: First created.

Intro

Samba file server enables specified directories to be accessible by Windows computers on the same network.

Install

Install the package​:​

$ sudo apt-get update 
$ sudo apt-get install samba

Define a workgroup

Edit the config file:

$ sudo gedit /etc/samba/smb.conf

​​​​​Define workgroup name as "kctcl" in the "[global]" section:

workgroup = kctcl

Uncomment to restrict access to server users only:

security = user

Add the following if OpenVPN used (added 7 May 2019):

hosts allow = 192.168.0. 10.8.0. 127.0.0.

Define directories to be shared

Add a section at the end to share directories:

[<sub-directory name or other short name>] 
pth = /<directory name>/<sub-directory name> 
browseable = yes 
guest ok = yes 
read only = no 
create mask = 0775 
directory mask = 0775 
# do not include the next two lines if access is restricted to the owning user (added 7 May 2019)
force user = nobody 
force group = nogroup

Create the directory to be shared, if not already existing:

$ sudo mkdir -p /<directory name>/<sub-directory name>

Change ownership of the directory:

$ sudo chown nobody:nogroup /<directory name>/<sub-directory name>

Start service

Start or restart Samba service whenever the config file is changed:

$ sudo systemctl restart smbd nmbd

or, if ".service" is not automatically appended when executing the above command:

$ sudo systemctl restart smbd.service nmbd.service

End of Page

Install Postfix + Dovecot e-mail server

Install Postfix + Dovecot e-mail server KCTang Wed, 24/11/2021 - 00:10

Go to End

Note

  • 1 Jul 2021: Adjust tls setting.
  • 18 Jun 2021: Adjust tls setting.
  • 5 Jan 2021: Increase imap-login process limit. List Dovecot full custom settings.
  • 9 Dec 2020: Define cron job to delete filtered mails.
  • 20 Sep 2020: Stop using mail-stack-delivery.
  • 8 May 2020: Correct typo errors.
  • 30 May 2019: Add anti-virus and spam mail filtering.
  • 17 May 2019: Add copying emails to external accounts.
  • 7 May 2019: Change "gksudo gedit" to "sudo gedit" as Ubuntu 18.04 dropped "gksudo". Add header_checks. Add auto creation of Trash folders. Add webmaster.
  • 29 Sep 2018: Add "body_checks" for spam control.
  • 20 Sep 2018: Increase message_size_limit to 20 times the default.
  • 5 Apr 2018: Increase message_size_limit to 10 times the default.
  • 25 Dec 2018: Publish on web.
  • 12 Apr 2014: Specify maximal_queue_lifetime to notify unsuccessful delivery immediately.
  • 2 Apr 2014: Specify fully qualified domain name.

Intro

Postfix is a mail transfer agent (MTA) responsible for sending out and receiving emails between servers.

Dovecot is a mail delivery agent (MDA) responsible for sending out and receiving emails between a server and its users.

Mail-stack-delivery was a combined package containing both Postfix and Dovecot. It is now no longer supported.

Re-direct to server ports​

Set the internet router to re-direct the following connections to server ports:

  • SMTP = port 25 (for receiving or sending emails)
  • secure SMTP = port 465 (for receiving or sending emails securely)
  • IMAP = port 143 (for retrieving emails)
  • secure IMAP = port 993 (for retrieving emails securely)
  • POP3 = port 110 (for retrieving emails)

Install both Postfix and Dovecot​

Ubuntu 20.04 does not support the combined package "mail-stack-delivery" anymore.

Install Dovecot and Postfix individually:

$ sudo apt install dovecot-imapd dovecot-pop3d
$ sudo apt install postfix

However, the previous config file "/etc/dovecot/conf.d/99-mail-stack-delivery.conf" is still retained for use, because "99" represents the last and overriding config file. This eliminates the need to change the individual config files.

(revised to install individually, 20 Sep 2020)

Reconfigure Postfix

Reconfig:

$ sudo dpkg-reconfigure postfix

Use Tab key to change selection.

Select "Internet Site".

Enter the following information:

System mail name: <fully qualified domain name, such as "kctang.com.hk">
Root and postmaster mail recipient: <such as "kctang">
Other destinations to accept mail: <fully qualified domain name, such as "kctang.com.hk">, <server name such as "server1">, localhost.localdomain, localhost
Force synchronous updates on mail queue: No
Local networks: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
Use procmail for local delivery: No
Mailbox size limit (bytes): 0
Local address extension character: +
Internet protocols to use: all

Activate the changes:

$ sudo systemctl reload postfix
or
$ sudo service postfix reload

Edit "main.cf" settings:

$ sudo gedit /etc/postfix/main.cf

Specify:

myhostname = kctang.com.hk
# last line changed from server name to fully qualified domain name,
# otherwise some servers would not accept e-mails sent without fully qualified domain name, 2/4/2014

message_size_limit = 204800000
# last line added to increase the default 10 times, 5/4/2014
# increased to 20 times, 20/9/2018

maximal_queue_lifetime = 0
# last line added to report unsuccessful delivery immediately instead of after the default of 5 days, 12/4/2018

bounce_queue_lifetime = 0
# last line added, this should not be bigger than maximal_queue_lifetime, KCTang 20/5/2019

# smtpd_tls_mandatory_protocols = SSLv3, TLSv1 # commented, replaced below, KCTang 18/6/2021
smtpd_tls_received_header = yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache

# next 2 lines added to enhance security, KCTang 1/7/2021
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1

# spf configuration, added but disabled, KCTang 1/6/2019
#check_policy_service unix:private/policyd-spf
#policyd-spf_time_limit = 3600s

# Milter configuration, added but disabled, KCTang 2/6/2019
#milter_default_action = accept
#milter_protocol = 6
#smtpd_milters = local:/opendkim/opendkim.sock
#non_smtpd_milters = $smtpd_milters
# end

#body_checks = regexp:/etc/postfix/body_checks
# last line added to refer to another file to check contents of email bodies, KCTang 29/9/2018, disabled after using amavis 1/6/2019

#header_checks = regexp:/etc/postfix/header_checks
# last line added to check headers, KCTang 30/9/2018, disabled after using amavis 1/6/2019

# virtual_alias_domains = kctang.com.hk
# last line added on 17/5/2019, but disabled since same domain name already defined as destination above, KCTang 20/5/2019

virtual_alias_maps = hash:/etc/postfix/virtual
# last line added to forward emails to another server, KCTang 17/5/2019

content_filter = smtp-amavis:[127.0.0.1]:10024
# last line added for amavis, KCTang 1/6/2019

Edit "master.cf" settings:

$ sudo gedit /etc/postfix/master.cf

Specify​:

smtp      inet  n       -       y       -       -       smtpd
smtps     inet  n       -       y       -       -       smtpd
# last line uncommented
   -o smtpd_sasl_auth_enable=yes
# last line added to enable STARTTLS authentication
   -o smtpd_client_restrictions=permit_sasl_authenticated,reject
# last line added to reject if not authenticated, no space after ","
   -o smtpd_tls_wrappermode=yes
# last line added to force use of TLS
   -o milter_macro_daemon_name=ORIGINATING
# last line added

Create "body_checks" file, if required:

$ sudo gedit /etc/postfix/body_checks

Specify​ one or more lines of texts within //:

/unique text contained in email you do not want to receive/ DISCARD

"DISCARD" means delete from the server.

Create "header_checks" file, if required:

$ sudo gedit /etc/postfix/header_checks

Specify similarly.

(header checks added 7 May 2019)

Change Dovecot settings

Edit config file:

$ sudo gedit /etc/dovecot/conf.d/99-mail-stack-delivery.conf

Specify:

disable_plaintext_auth = yes

mail_location = maildir:~/Maildir:LAYOUT=fs
# LAYOUT=fs added to last line, to use "/" instead of "." to denote sub-folders

auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@

# IMAP configuration
protocol imap {
        mail_max_userip_connections = 1000
        # 10 in last line increased to 1000
        imap_client_workarounds = delay-newmail
}

# POP3 configuration
protocol pop3 {
        mail_max_userip_connections = 50
        # 10 in last line increased to 50
        pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
}

# LDA configuration
protocol lda {
        postmaster_address = postmaster
        mail_plugins = sieve
        quota_full_tempfail = yes
        deliver_log_format = msgid=%m: %$
        rejection_reason = Your message to <%t> was automatically rejected:%n%r
}

# Plugins configuration
plugin {
        sieve=~/.dovecot.sieve
        sieve_dir=~/sieve
}

# Authentication configuration
auth_mechanisms = plain login

service auth {
  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/dovecot-auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

# The following are additional to those in 15-mailboxes.conf.
# They are to auto create Trash folders.
# Trash folders would not be backed up with back-in-time,
# and therefore would need to be re-created after email recovery is done from back-in-time.
namespace inbox {
  mailbox Trash {
    auto = subscribe
  }
}

# process_limit increased
service imap-login {
  process_limit = 200
}

(trash folder setting added 7 May 2019)

(full custom settings listed 5 Jan 2021)

Activate the changes:

$ sudo systmctl reload postfix
or
$ sudo service postfix reload
and
$ sudo systemctl restart dovecot
or
$ sudo service dovecot reload

Verify success

See whether the Postfix server is running:

$ telnet localhost 25

should display:

220 kctang.com.hk ESMTP Postfix (Ubuntu)

ehlo localhost

should display the following:

250-kctang.com.hk

250-PIPELINING

250-SIZE 102400000

250-VRFY

250-ETRN

250-STARTTLS

250-ENHANCEDSTATUSCODES

250-8BITMIME

250 DSN

Ctrl-]

to exit to "telnet >" prompt.

quit

to exit telnet.

Try also:

$ telnet localhost 995

similarly:

$ telnet localhost 465

should display either one:

Connected to localhost

Connected to kctang.com.hk

"Ctrl-]"

 to exit to "telnet >" prompt.

quit

to exit telnet.

Specify internal email forwarding

Edit "aliases" file:

$ sudo gedit /etc/aliases

Specify​:

postmaster: kctang
webmaster: kctang
kctcl: kctcl, kctclpop

meaning:

  • forwarding e-mails sent to postmaster@localhost and webmaster@localhost to kctang@localhost, no email will be left at postmaster or webmaster

(webmaster added 7 May 2019)

  • forwarding e-mails sent to kctcl@localhost to kctcl@localhost (itself) and to kctclpop@localhst, i.e. making a copy

"localhost" means "kctang.com.hk" in our case. For the email user name before "@", there is no need to create a  file system user account for it if all emails addressed to it are forwarded elsewhere. The name serves as an alias only of the email account forwarded to.

(added 17 May 2019)

 

Activate setting everytime the "aliases" file has been changed:

$ sudo newaliases

 

Copy to external email accounts

(section added, 17 May 2019)

Execute:

$ sudo gedit /etc/postfix/main.cf

Specify​ at the end of the file:

virtual_alias_domains = kctang.com.hk
virtual_alias_maps = hash:/etc/postfix/virtual

Execute:

$ sudo gedit /etc/postfix/virtual

Specify​ to make a copy to itself and a copy to the external email account:

# from               to one or more addresses, separated by a space 
kctang@kctang.com.hk kctang@kctang.com.hk name1@external.account.name
kctcl@kctang.com.hk kctcl@kctang.com.hk name2@external.account.name

Omit making a copy to itself if only email forwarding is required.

Save and exit.

Execute after the "virtual" file is created or changed:

$ sudo postmap /etc/postfix/virtual

Execute:

$ sudo systemctl restart postfix

Test by sending emails.

Install anti-virus Clamav-Daemon

(section added, 30 May 2019)

Install clamav-daemon:

$ sudo apt update
$ sudo apt install clamav-daemon            (clamav-freshclam also automatically installed)
$ sudo dpkg-reconfigure clamav-daemon   (must be done, generally accept all defaults, set yes to scan emails)
$ sudo systemctl start clamav-freshclam
$ sudo systemctl start clamav-daemon
$ sudo systemctl status clamav-freshclam    (check if running)
$ sudo systemctl status clamav-daemon       (check if running, OK if reported "/bin/mkdir /run/clamav (code=exited, status=1/FAILURE" because directory already created)
$ tail -f /var/log/clamav/clamav.log        (see running progress, Ctrl-Z to exit)

(typo corrected, 8 May 2020)

Filter spam mails

(section added, 30 May 2019)

Execute to install various software:

$ sudo apt update
$ sudo apt install amavisd-new spamassassin
$ sudo apt install postfix-policyd-spf-python                                 (optionally required for spf) 
$ sudo apt install opendkim                                                   (optionally required for opendkim)
$ sudo apt install pyzor razor                                                (optional extras)
$ sudo apt install arj cabextract cpio lhasa nomarch pax rar unrar unzip zip  ("lhasa", not "lha")

Configure ClamAV:

$ sudo adduser clamav amavis
$ sudo adduser amavis clamav

Configure SpamAssassin:

$ sudo gedit /etc/default/spamassassin

 Change "ENABLED=0" to:

ENABLED=1

(no longer set here, 8 May 2020)

Start the service:

$ sudo systemctl start spamassassin.service

Configure Amavisd-new:

$ sudo gedit /etc/amavis/conf.d/15-content_filter_mode

Specify:

use strict;

# Uncomment the two lines below to enable antivirus checking mode

@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);

# Uncomment the two lines below to enable SPAM checking mode

@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

1;  # insure a defined return

Execute:

$ sudo gedit /etc/amavis/conf.d/20-debian_defaults

Change "D_PASS" to:

$final_spam_destiny = D_DISCARD

(deleted,  8 May 2020, reinstated, 26 Jul 2021)

Adjust the following values only if desired to flag more messages as spam:

$sa_tag_level_deflt = 2.0; # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.31; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 6.31; # triggers spam evasive actions
$sa_dsn_cutoff_level = 10; # spam level beyond which a DSN is not sent

Execute:

$ sudo gedit /etc/amavis/conf.d/50-user

Specify:

$myhostname = 'kctang.com.hk';

Re-start the service:

$ sudo systemctl restart amavis.service

Edit the following file to specify domains to be whitelisted if necessary:

$ sudo gedit /etc/amavis/conf.d/40-policy_banks

Test that the Amavisd-new SMTP is listening:

telnet localhost 10024
Trying 127.0.0.1...
Connected to kctang.com.hk.
Escape character is '^]'.
220 [127.0.0.1] ESMTP amavisd-new service ready

Press Ctrl-]  and enter "quit" to exit.

Configure Postfix master.cf:

$ sudo gedit /etc/postfix/master.cf

Add the following to the end of the file:

smtp-amavis     unix    -       -       -       -       2       smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20

127.0.0.1:10025 inet    n       -       -       -       -       smtpd
    -o content_filter=
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=reject_unauth_pipelining
    -o smtpd_end_of_data_restrictions=
    -o mynetworks=127.0.0.0/8
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters

Add the following immediately below the "pickup" transport service:

     -o content_filter=
     -o receive_override_options=no_header_body_checks

Configure Postfix main.cf:

$ sudo gedit /etc/postfix/main.cf

Add the following to the end of the file:

content_filter = smtp-amavis:[127.0.0.1]:10024

Comment it with "#" at the start of the line if desired to stop using amavis.

Restart service:

$ sudo systemctl restart postfix.service

Check the hidden header of email received after the above to see the presence of one or more of the following:

X-Spam-Level: 
X-Virus-Scanned: Debian amavisd-new at kctang.com.hk
X-Spam-Status:
X-Spam-Level: 

If present, the spam filter is working.

See https://help.ubuntu.com/lts/serverguide/mail-filtering.html.en for a full explanation of the above.

Delete filtered mails

(section added, 9 Dec 2020)

Filtered files are stored in /var/lib/amavis/virusmails and should be deleted regularly.

Edit to define a scheduled job to delete filtered files:

$ sudo crontab -e -u amavis 

where:

-e = edit

-u amavis = user amavis

Define as follows:

0 0 * * * find /var/lib/amavis/virusmails/ -type f -mtime +15 -delete

which means at

0 = 0 minute 

0 = 0 hour

* = every day of month

* = every month

* = every day of week

find /var/lib/amavis/virusmails = find in that directory

-type f = regular files

-mtime +15 = file modified more than 15 days ago

-delete = delete the file

 

End of Page

Install Ftp server

Install Ftp server KCTang Tue, 07/05/2019 - 20:21

Go to End

Note

26 Apr 2022: "gedit" changed to "nano". Minor error corrected.

5 Sep 2019: "0755" changed to "0775" for "Ftp" directory.

7 May 2019: "gksudo gedit" changed to "sudo gedit" as Ubuntu 18.04 dropped "gksudo".

25 Dec 2014: Created.

Intro

FTP server enables directories to be accessible for downloading or uploading by users outside the local network.

Install

Install the package:

$ sudo apt-get install vsftpd

Edit config file:

$ sudo nano /etc/vsftpd.conf

Uncomment the following line to enable uploading:

write_enable=YES

Define as the following line to change the default directory permissions to 775 (drwxrwxr-x) and default file permissions to 664 (-rw-rw-r--):

local_umask=002

Uncomment the following lines to restrict users to their home except for those listed in the file represented by "chroot_list_file":

chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd.chroot_list

("vsftpd.choot_list" corrected as "vsftpd.chroot_list",  7 May 2019)

Save file after uncommenting.

Specify users who can go outside their home by inserting their user login names one on each line in the file represented by "chroot_list_file":

$ sudo nano /etc/vsftpd.chroot_list

("vsftpd/chroot_list" corrected as "vsftpd.chroot_list",  7 May 2019)

Restart ftp service whenever the config files are changed:

$ sudo systemctl restart vsftpd
or
$ sudo service vsftpd restart

Set the internet router to re-direct ftp connections to server port 21.

Set up a root FTP Directory to contain all FTP job folders

Change directory to the top directory assessible for use by Windows network through Samba:

$ cd /<full directory path from root>​

Make a directory specially for FTP storage, called "Ftp" in this example:

$ sudo mkdir Ftp

Change its ownership so that it can be accessed by Windows network:

$ sudo chown nobody:nogroup Ftp

Change its permissions to "read only" for other users:

$ sudo chmod 0775 Ftp

("0755" changed to "0775" because for unknown reasons sub-directory cannot be created under "Ftp", 5 Sep 2019)

Check setting:

$ ls -ls

should show "drwxrwxr-x" and "nobody nogroup" against the "Ftp" item.

(drwxr-xr-x corrected as drwxrwxr-x, 26 Apr 2022)

Create a ftp user for specific job

Create a new user with authority to download and upload the job ftp directory:

$ sudo adduser <ftp user name>

Change the new user's root directory from /home/<ftp user name> to the job ftp directory:

​$ sudo usermod -d /<full directory path from root>/Ftp/<job name> <ftp user name>
  • ​<ftp user name> and <job name> can be the same or different
  • <job name> will become ftp users' root directory, they will be restricted to see only files at or below the root directory, they will not see the name of <job name> or the directory structure outside the root directory 
  • ​Instead of <job name>, a further sub-directory such as <job name>/<sub job name> may be defined as the root​ directory
  • ​The directory /<full directory path from root>/Ftp/<job name> will still exist but not be used for ftp

Set up a ftp directory for specific job for downloading

Create a job ftp directory under the Ftp directory:

  • using Windows Explorer:
​\\<server name>\<full folder path from server>\Ftp\<job name>
  • or at the server terminal:
$ cd /<full directory path from root>​/Ftp
$ sudo mkdir <job name>
$ sudo chown nobody:nogroup <job name>
$ ls -ls

should show "drwxr-xr-x" or "drwxrwxr-x" and "nobody nogroup" against the <job name> item.

Further sub-directories may be created similarly for downloading purposes.

Set up a ftp directory for specific job for uploading

Create an "upload" sub-directory under the job ftp directory:

  • using Windows Explorer:
​\\<server name>\<full folder path from server>\Ftp\<job name>\upload
  • or at the server terminal:
$ cd /<full directory path from root>​/Ftp/<job name>
$ sudo mkdir upload
$ sudo chown nobody:nogroup upload

Change its permissions on the server to enable "write" for all:

$ cd /<full directory path from root>/Ftp/<job name>
$ sudo chmod a+w upload

Check settings:

$ ls -ls

should show "drwxrwxrwx" and "nobody nogroup" against the "upload" item.

Upload or download

​Internally, use Windows file explorer to copy or move files between the Windows networked computers to the ftp directories:

  • ​​copy files to ​\\< server name>\<full folder path from server >\Ftp\<job name> for downloading
  • copy files from \\<server name>\<full folder path from server >\Ftp\<job name>\upload after uploading by others

Externally, inform external users the ftp user login name i.e. <ftp user name> and password for downloading or uploading.

End of Page

Install Apache2 web server

Install Apache2 web server KCTang Mon, 06/01/2020 - 18:08

Go to End

Note

30 Jul 2020: Updated to use Python3.

6 Jan 2020: Updated.

8 May 2019: More explanation on default configuration given. "gksudo gedit" changed to "sudo gedit" as Ubuntu 18.04 dropped "gksudo".

11 Apr 2018:  "apache2" changed to "apache2.service" when used in conjunction with systemctl.

25 Dec 2014: First created.

Intro

Apache2 web server provides web page services.

Prepare

Define hosts:

$ sudo gedit /etc/hosts

Specify:

127.0.0.1 kctang.com.hk <computer name> localhost
127.0.1.1 <computer name>

Define hostname:

$ sudo gedit /etc/hostname

Specify a line to contain:

<computer name>

Install

Install:

$ sudo apt install apache2

or before Ubuntu 16.04:

$ sudo apt-get install apache2

Start service:

$ sudo systemctl start apache2.service

or:

$ sudo service apache2 start

Set the internet router to re-direct http connections to server port 80.

(The following added on 8 May 2019)

Edit the enabled configuration file:

$ sudo gedit /etc/apache2/sites-enabled/000-default.conf

("ls -ls" changed to "gedit", 6 Jan 2020)

The file is symbolic linked to the actual location at /etc/apache2/sites-available/000-default.conf.

The file includes the following:

#ServerName www.example.com
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html

Leave the ServerName to the HTTPS setting below.

Change the ServerAdmin email address to the correct address, or add that address when setting up the email server.

The DocumentRoot tells that the website directories and files will be stored under /var/www/html.

Note that previously the DocumentRoot was at /var/www. The change from /var/www/html would affect the installation of Drupal as explaned on that web page.

Serf to http://www.kctang.com.hk or http://kctang.com.hk on web browser, the following page (var/www/html/index.html) will be displayed to indicate successful installation:

(end of add)

Configure to use HTTPS

This is optional. Starting to use on 6 April 2018.

When the Apache2 server is configured to use HTTPS, and when "https://" is used as the prefix to the website address URL (Uniform Resource Locator) in the web browser navigation bar, encrypted communications will be used with the Apache2 server. This will enhance security.

To do this, enable the mod_ssl module:

$ sudo a2enmod ssl

In order for Apache2 to use HTTPS service, a certificate and a key file are needed. Use EFF's Certbot to automatically deploy Let's Encrypt certificates and enable HTTPS.

Certbot is downloadable at:

https://certbot.eff.org/

No need to download from there.

(revised 8 May 2019)

Install certbot and configure Apache2:

$ sudo apt update
$ sudo apt install software-properties-common
$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt update
$ sudo apt install python3-certbot-apache
$ sudo certbot --apache
$ sudo certbot renew

(python changed to python3, 30 Jul 2020)

When answering questions

  • enter "kctang.com.hk" for name to activate HTTPS
  • decide whether to re-install certificate or renew and replace
  • select to redirect HTTP traffic to HTTPS.

(added on 8 May 2019).

The following will happen:

  • /etc/letsencrypt directory created to contain certificate obtained from Let's Encrypt.
  • A file /etc/apache2/sites-available/000-default-le-ssl.conf added and enabled.

(revised 8 May 2019)

  • The following lines inserted in /etc/apache2/sites-available/000-default.conf to enforce the use of "https://":
RewriteEngine on
RewriteCond %{SERVER_NAME} =kctang.com.hk
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
  • A cron job created to renew the certificate which lasts for 90 days before expiry.

Enable new module and disable default module if not already automatically done:

$ sudo a2ensite 000-default-le-ssl
$ sudo a2dissite default-ssl
$ sudo systemctl restart apache2.service

 

End of Page

Install MySQL server + PHP + phpMyAdmin

Install MySQL server + PHP + phpMyAdmin KCTang Sat, 08/01/2022 - 04:21

Go to End

Note

  • 8 Jan 2020: Configuring MySQL-server updated. Removing completely MySQL added.
  • 6 Jan 2020: Updated.
  • 8 May 2019: References to PHP5 deleted. Secure installation added. Root user authentication added. automysqlbackup added.
  • 25 Dec 2014: First created.

Intro

MySQL is a database server.

PHP (PHP5 before Ubuntu 16.04) is a web page programming language.

phpMyAdmin is a web interface to administer the MySQL server.

Install MySQL-server

Execute:

$ sudo apt update
$ sudo apt install mysql-server
$ sudo systemctl start mysql.service
$ sudo systemctl status mysql.service

Config MySQL-server

(section added, 8 May 2019)

Execute:

$ sudo mysql_secure_installation

Enter new root password for the first time or the existing root password.

When answering the secure installation questions:

  • validate password component - yes
  • set password validation policy - 2 for MEDIUM
  • set password strength - high
  • change the password for root - yes or no as appropriate
  • set new password - as appropriate
  • re-enter new password -
  • continue with the password - yes or no as appropriate
  • remove anonymous users - yes
  • disallow root login remotely - no
  • remove test database - yes
  • reload privilege table now - yes 

(steps updated, 8 Jan 2022)

(suggested answers added, 6 Jan 2020)

Adjust authentication for use with phpMyAdmin

(section added, 8 May 2019)

If not done, the phpMyAdmin menu would not provide a choice to add users.

Execute:

$ sudo mysql -u root -p

enter the root user's password when prompted.

Execute:

mysql> SELECT user,plugin FROM mysql.user;

It will show that the root user's authentication plugin is "auth_socket".

Execute:

mysql> ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY '<root user password>';
mysql> FLUSH PRIVILEGES;
mysql> SELECT user,plugin FROM mysql.user;

(semi colon added after "PRIVILEGES", 6 Jan 2020)

It should show that the root user's authentication plugin has been changed to "mysql_nature_password".

Install PHP and related apache2 module

Execute:

$ sudo apt install php libapache2-mod-php

Install phpMyAdmin

Execute:

$ sudo apt install phpmyadmin

Set its own password.

Choose whether to keep the existing database whenever phpmyadmin is removed and re-installed. 

Enable Apache2 config:

$ sudo ln -s /etc/phpmyadmin/apache.conf /etc/apache2/conf-available/phpmyadmin.conf
$ sudo a2enconf phpmyadmin.conf

Whenever Apache2 has been purged and re-installed, this enabling must be done again.

Restart Apache2 service:

Execute:

$ sudo systemctl restart apache2.service

Create a database user and a database

Execute:

 http://localhost/phpmyadmin
  or
 http://www.kctang.com.hk/phpmyadmin

Enter login name and password.

Create a user called <name, e.g. Drupal8> together with a database also called <name>:

  • click "Users" at the top menu bar
  • click "Add user" at the page middle
  • enter <name> at the User name entry
  • select "Local" at the Host entry
  • enter and re-type the password
  • click "Create database with same name and grant all privileges"
  • click "Go" at the bottom

Install automysqlbackup

(section added, 8 May 2019)

The software will backup mySQL databases daily, weekly and monthly.

Execute to install:

$ sudo apt install automysqlbackup

Edit the configuration file:

$ sudo gedit /etc/default/automysqlbackup

Specify:

BACKUPDIR="/var/lib/automysqlbackup" as the backup directory

MAILCONTENT="log" to send log email

MAILADDR="root" to send the email to root user, which has been set under Postfix Aliases to re-direct to the appropriate user.

Execute to run for the first time:

$ sudo automysqlbackup

Inspect daily backups:

$ sudo ls /var/lib/automysqlbackup/daily

Remove mySQL completely

(section added, 8 Jan 2022)

sudo systemctl stop mysql.service
sudo apt purge mysql*
sudo apt autoremove                    (optional)
sudo apt autoclean                     (optional)
sudo apt remove dbconfig-mysql
sudo rm -r /var/lib/mysql
sudo rm -r /log/mysql
sudo rm -r /etc/mysql
sudo deluser mysql

End of Page

Install Drupal 9 content management system

Install Drupal 9 content management system KCTang Tue, 21/12/2021 - 02:52

Go to End

Note

  • 21 Dec 2021: Bartik theme used instead of Mayo.
  • 11 Nov 2021: Errors in updating core manually corrected.
  • 5 Jan 2020:  Problems when updating to Drupal 8.8 under php7.4 described. Some text updating.
  • 9 Sep 2019: Composer files also copied when relocating system.
  • 18 July 2019: Composer configuration actions added.
  • 7 July 2019: Original user-defined files and attributes kept when updating manually.
  • 26 May 2019: Composer and Drush installations added. Manual core installation added. Deleting configuration file added.
  • 16 May 2019: Simplified installation step added. Using new shell to restore added.
  • 8 May 2019: Page on Drupal 8 withdrawn.  All "/var/www/web" changed to "/var/www/html/web" because Apache2 prefers to put website files underneath "/www/web/html", and it is easier that way. Relocating Drupal 8 added. Print page stylesheet re-written.
  • 18 Jan 2019: Updating Drupal 8 core using composer added.
  • 30 Sep 2018: Setting CKEditor to use Mayo theme stylesheet added. Setting print page to also use Mayo theme stylesheet added. Adding custom styles added.
  • 23 Sep 2018: "php/7.0" changed to "php/7.2"
  • 3 Sep 2018: First created from the page on Drupal 7 after major upgrading to Drupal 8. Resolution of CKEditor table border added.

Intro

Drupal is a web site content management system serving web pages to the internet.

The latest version is Drupal 9. The main text below is about installing Drupal 8 with supplementary notes on upgrading to Drupal 9. The procedures described have not been proven with a fresh installation of Drupal 9 but may be quite similar.

(paragraph added, 20 Sep 2020)

Prepare

Install Apache2 web server if not already installed.

Enable Apache2 rewrite module (should have been automatically installed when installing HTTPS):

$ sudo a2enmod rewrite

Configure Apache2:

$ sudo gedit /etc/apache2/apache2.conf

Specify, keeping "<" and ">":

<Directory /var/www/>
    Options Indexes FollowSymLinks
    # AllowOverride None # replaced with next line
    AllowOverride All
    Require all granted
</Directory>

Restart Apache2 service:

$ sudo systemctl restart apache2
or
$ sudo service apache2 restart

Install MySQL server + PHP5 + phpMyAdmin, and create a user with database both called "drupal8".

Co-exist with Drupal 7

Change "web" below to "web2" to create another set of directories to co-exist with Drupal 7 currently using "web", otherwise, just use "web".

Install Drupal 8

Change "8.x.x" to "9.x.x" in case of Drupal 9.

(paragraph added, 20 Sep 2020)

Download the latest Drupal 8.x.x (filename drupal-8.x.x.tar.gz) from https://www.drupal.org/project/drupal, usually to own "Downloads" directory.

Use the file manager to extract the contents of the compressed file under the "Downloads" directory as "drupal-8.x.x".

Create a directory for Drupal but using a directory name of "web":

$ sudo mkdir /var/www/html/web
$ cd /var/www/html/web
$ sudo mv /home/<own account name>/Downloads/drupal-8.x.x/* .
$ sudo ls -la /home/<own account name>/Downloads/drupal-8.x.x
$ sudo mv /home/<own account name>/Downloads/drupal-8.x.x/.* .
(move hidden files, which may give an error message)
$ sudo ls -la                                                
(check moved files as intended)
$ sudo ls -la /home/<own account name>/Downloads/drupal-8.x.x
(should be empty)

Do it in one step instead of the above:

$ cd /var/www/html
$ sudo wget https://ftp.drupal.org/files/projects/drupal-8.x.x.tar.gz  
(change "8.x.x" to the version number)
$ tar -xzf drupal-8.x.x.tar.gz 
("sudo" not used so that user instead of "root" will become owner) 
$ mv drupal-8.x.x web

(simplified step added, 15 May 2019)

(revised, 26 May 2019)

("sudo" not used, 7 July 2019)

Create a location for site files:

$ cd /var/www/html/web
$ sudo mkdir sites/default/files

(change directory added, 5 January 2020)

The latest recommendation is to install Drupal using Composer. See https://www.drupal.org/docs/develop/using-composer/starting-a-site-using-drupal-composer-project-templates.

There are two templates:

  • drupal/recommended-project which is recommended by Drupal. This template requires an additional level of directory, such as /var/www/html/drupalroot/web instead of /var/www/html/web. 
  • drupal/legacy-project which uses the same directory structure as the downloadable compressed file. Since the existing installation is based on the downloadable compressed file, this structure is continued to be used, and Composer is only used for updating as described later.

(two paragraphs added, 20 Sep 2020)

Change ownership:

$ sudo chown www-data:www-data sites/default/files

Create the initial configuration file for the default site:

$ sudo cp sites/default/default.settings.php sites/default/settings.php

Change ownership:

$ sudo chown www-data:www-data sites/default/settings.php

Restart Apache2 service:

$ sudo systemctl restart apache2
or
$ sudo service apache2 restart

Complete the Drupal Installation through a Browser by pointing to http://localhost/web, and follow the instructions there, using:

(not .../web/install.php, 8 May 2019)

  • Choose language: English language
  • Choose profile: Standard
  • Setup database: MySQL
  • MySQL database name, user name and password as defined above
  • Site name: www.kctang.com.hk
  • Site e-mail address: <>@kctang.com.hk
  • Site maintenance username: <name of the administrator>
  • Site maintenance user e-mail address: <>@kctang.com.hk
  • Default country: Hong Kong S.A.R., China
  • Default time zone: Asia/Hong Kong
  • Check for updates automatically
  • Receive e-mail notifications

Change directory permissions:

$ sudo chmod 555 sites/default
$ sudo chmod 444 sites/default/settings.php

Permit uploading of files (similar change for modules and themes not required, they use ftp upload)

$ sudo chown www-data:www-data -R sites/default/files

Increase upload file size limit:

$ sudo gedit /etc/php/7.x/apache2/php.ini
(change "7.x" to the current version number)

("7.2" changed to "7.x" which is "7.4" as of now, 5 January 2020)

("7.0" changed to "7.2", 23 Sep 2018)

Change existing to:

post_max_size = 200M
upload_max_filesize = 50M
max_file_uploads = 100

Restart Apache2 service:

$ sudo systemctl restart apache2
or
$ sudo service apache2 restart

Edit "settings.php" file:

$ sudo gedit /var/www/web/sites/default/settings.php

specify trusted hosts:

$settings['trusted_host_patterns'] = array(
   '^www\.kctang\.com\.hk$',
   '^kctang\.com\.hk$',
   '^localhost$',
);

Disable "update.php" file:

$ sudo mv /var/www/web/update.php /var/www/web/<some new name>

Migrate from Drupal 7 (as of 21 February 2018)

Log in as an administrator if not already in.

Choose Manage > Extend > Book module under Core > Install at the bottom, to allow users to create and organize related content in an outline.

Choose Manage > Extend > Statistics module under Core > Install at the bottom, to log content statistics.

Similarly, enable modules in Drupal8 that are enabled in Drupal7.

Click open the explanatory note of "Statistics". Select Configure > Count content views > Save configuration > Continue.

Choose Manage > Extend > Migrate, Migrate Drupal and Migrate Drupal UI modules under Core (Experimental) > Install at the bottom, to allow users to create and organize related content in an outline.

Click open the explanatory note of Migrate Drupal UI. Choose Configure > Continue > maintenance mode > Put site into maintenance mode > Save configuration.

Move web page back to the Upgrade page. Choose Continue.

Under SOURCE DATABASE, enter: 

  • Database host: localhost
  • Database name: drupal7
  • Database username: drupal7
  • Database password: <>

Under SOURCE FILES, assuming under "oldweb" enter:

  • /var/www/oldweb

Resolve the following problems found:

  • Some uploaded files have not been migrated. Execute "sudo cp /var/www/<oldweb>/sites/default/files/* /var/www/html/web/sites/default/files" to copy the files over (not the sub-directories).
  • The Main menu used in Drupal 7 has been migrated with a machine name called "main-menu" but the machine name used in Drupal 8 is called "main". This is incompatible. Although the menu structure is the same, the contents cannot be displayed. There is also an error message saying that the main-menu index has a problem. Add another menu under "Manage > Structure > Menus". Still at there, select "Edit menu" against the migrated Main menu to edit the individual menu items. Change the "Parent link" from the migrated Main menu to the newly added menu so as to empty the migrated Main menu. This may help remove the indexing problem. The contents can be displayed using the newly added menu. If things go fine and it is preferred to use the true Main menu, move the items back from the newly added menu to the Main menu, which should now use the correct machine name. The foregoing is based on memory. During the problem solving endeavour, mySQL has been accessed to delete references to "main-menu", but it is not sure whether this is really essential.
  • The Book menu has not been migrated and will need build-up from scratch.
  • The previously used theme has not been migrated. Download and install the desired theme under Manage > Appearance. Set up with reference to the original setting.
  • Many modules have not been migrated. Download and install them from scratch under Manage > Extend. Not all modules are available for Drupal 8.
  • Some modules (e.g. CKEditor Color Button) require the creation of a "libraries" directory under the Drupal root directory.
  • Some modules require the downloading of files at the server (i.e. not the web) and putting the unzipped file directory under the "libraries" and "vendor" directories. Some even require a sub-directory under "libraries", e.g. "CKEditor Find" module requires "... libraries/ckeditor/plugins".

Re-direct http path to sub-directory "web"

The default DocumentRoot  in the default enabled Apache2 configuration file /etc/apache2/sites-enabled/000-default.conf  is "/var/www/html/".

Web browers can access any permissible directories or files underneath the DocumentRoot. Access outside the Document Root will not be possible.

When the Drupal root directory /web is placed underneath /var/www/html such as /var/www/html/web, then http://kctang.com.hk will access the default index.html file underneath /var/www/html, but in the case of http://kctang.com.hk/web, access to those underneath /web is possible.

To re-direct http://kctang.com.hk to access /var/www/html/web directly, create a new ".htaccess" file under "/var/www/html":

$ sudo gedit /var/www/html/.htaccess

Specify:

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteBase /
  RewriteRule ^(.*)$ web/$1 [R]
</IfModule>

"^(.*)$" represents the full text from start to end after the domain name, e.g. "abc/def" in "kctang.com.hk/abc/def".

"web/$1" represents the substitution text where "$1" represents the text represented by "(.*)" with "web/" inserted before it, i.e. "web/abc/def" based on the above example. This would redirect the path to "web/abc/def".

"[R]" is to redirect the path and will show "web/" as part of the redirected path. When using "[R]", "RewriteBase /" is required to add "/" before "web/$1". Without this, "/var/www/html" will be added before "web/$1" and will cause unexpected results. This effect was discovered after many days of error discovery using "[L]".

Using [L] as suggested by many people may hide "web/" from the displayed path, and does not require the use of "RewriteBase", but some of the web pages will still show "web/" unavoidably. A mixed use with or without "web/" displayed will cause denial of access rights or redirection to external URL in some cases. Therefore, it is better to force to display "web/" using "[R]".

The following has the same effect:

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteBase /web
  RewriteRule ^(.*)$ $1 [R]
</IfModule>

Remove "www." prefix from URL

This is not essential for using Drupal, but is adopted only to simplify.

Insert the following in /var/www/html/.htaccess after the RewriteBase line:

# Remove "www." prefix from URL
RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
RewriteRule ^ http%{ENV:protossl}://%1%{REQUEST_URI} [L,R=301]

Permit redirection to external URL

This section would not be required if the path redirection is properly defined. This appears to be not required because the core has been replaced after this section was written, with the two lines unchanged but without problem caused.

Redirecting between a path with or without "web/" will be treated as a redirection to external URL. Drupal 8 does not allow redirection to external URL by default.

To allow:

$ sudo gedit /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/RedirectResponseSubscriber.php

change lines 7 and 74 from:

use Drupal\Core\Routing\LocalRedirectResponse;

$safe_response = LocalRedirectResponse::createFromRedirectResponse($response);

to:

use Drupal\Core\Routing\TrustedRedirectResponse;

$safe_response = TrustedRedirectResponse::createFromRedirectResponse($response);

Handle CKEditor module

CKEditor is a WYSIWYG editor for Drupal.

For Drupal 7, CKEditor is a single optional module. The module is pre-built to contain the desired plugins under the CKEditor web site before being installed under Drupal.

For Drupal 8, CKEditor is core module but with trimmed down features. To add back those features, individual add-on modules from the Drupal website and their corresponding CKEditor plugin files from CKEditor web site have to be installed. Not all previous features have CKEditor Drupal 8 add-on modules available.

The downloaded plugin file directories need to be unzipped under the "libraries" directory . Some even require more than one sub-directory under "libraries", e.g. "CKEditor Find" module requires "... libraries/ckeditor/plugins".

Install the CKEditor "libraries" module (not the same as the libraries directory mentioned above) because some CKEditor modules require this to work.

Not all CKEditor Drupal 8 add-on modules could be installed successfully. The usual unsuccessful phenomenon is that, after installation of the modules and the related plugin directories, the icons newly made available for the editing menu are blank or do not contain the full text. If they are added to the CKEditor editing menu, the menu cannot be displayed when a page is clicked open for editing. After a lot of trial and errors, it has now been found that the cause of the problem sis actually very simple. when the corresponding plugin directories are downloaded and added to the libraries directory, only the owner is permitted to read and write. By permitting the group and others to read, the problems are solved. The command is: (revised 3 Sep 2018) 

$ sudo chmod 755 /var/www/html/web/libraries/<plugin directory name>

 

Change from Mayo theme to Bartik theme

(section added, 21 Dec 2021)

The Mayo theme has been used for this website from the beginning but has not been maintained for use with Drupal 9. The theme has therefore been changed to the Bartik theme, which is a core theme provided by Drupal.

Log in as an administrator if not already in.

Choose Manage > Appearance > Settings, and define Bartik color scheme as follows:

 

Configure Bartik's css files as described below to match the preferences of this website. The configurations described further below in relation to the Mayo theme are not applicable to the Bartik theme. 

Add custom styles to Bartik theme

(section added, 21 Dec 2021)

Log in the server.

Open with file manager the following file, and edit:

/var/www/html/web/core/themes/bartik/css/base/elements.css

Add the following at the end of the file to override the existing settings:

/* the following settings added, KCTang 18/12/2021 */

body {
  margin: 0;
  color: blue;
  font-family: Arial;
}
p {
  margin: 0 0 0.2em;
}
pre{
  border: 1px solid green;
  font-family: monospace;
  margin: 0.15em 0.2em 0.15em;
  padding:0.2em;
  background-color: #f8f9fa;
  white-space: pre-wrap;
}
.widget-toc{
  display: table;
  border: 1px solid green;
  background-color: #f8f9fa;
  padding:0.5em;
  font-size: 0.9em;
}
.hangtwice {
  margin-left:80px;
  text-indent:-80px;
}
.hang1 {
  margin-left:40px;
  text-indent:-40px;
}
.hang2 {
  margin-left:80px;
  text-indent: -40px;
}
.hang3 {
  margin-left:120px;
  text-indent: -40px;
}
.hang4 {
  margin-left:160px;
  text-indent:-40px;
}
.hang5 {
  margin-left:200px;
  text-indent:-40px;
}
.indent1 {
  margin-left:40px;
}
.indent2 {
  margin-left:80px;
}
.indent3 {
  margin-left:120px;
}
.indent4 {
  margin-left:160px;
}
h1,
.heading-a {
  color: blue;
  margin: 0 0 0.5em;
}
h2 {
  border-bottom: 1px solid;
  color: red;
}
h2.hang {
  margin-left:40px;
  text-indent:-40px;
  color: red;
}
h3 {
  font-weight: bold;
  font-weight: normal;
  color: magenta;
}
h3.hang {
  margin-left:40px;
  text-indent:-40px;
  font-weight: normal;
  color: magenta;
}
h4.hang {
  margin-left:40px;
  text-indent:-40px;
}
a,
a.link {
  border-bottom: none;
}
.site-branding__name {
  font-family: "Times New Roman";
}
img {
  margin: 15px;
}

Save and exit.

Additional text style choices have been added. Add style buttons to CKEditor as described later below.

Open with file manager the following file, and edit:

/var/www/html/web/core/themes/bartik/css/components/main-contents.css

Add the following at the end of the file to override the existing settings:

/* the following settings added, KCTang 18/12/2021*/

.main-content h2 {
  margin: 0.5em 0;
  font-size: 1.429em;
}

Save and exit.

Open with file manager the following file, and edit:

/var/www/html/web/core/themes/bartik/css/components/text-formatted.css

Add the following at the end of the file to override the existing settings:

/* the following settings added, KCTang 18/12/2021*/
.text-formatted ul,
.text-formatted ol {
  margin: 0;
  padding: 0 0 0.25em 15px; /* LTR */
}

Open with file manager the following file, and edit:

/var/www/html/web/core/themes/bartik/css/layout.css

Revise part of the settings as follows:

@media all and (min-width: 851px) {
  .layout-container {
#    max-width: 1290px;
    max-width: 1920px; /* revised by KCTang 18/2/2021 */
  }
}

Save and exit.

Clear cache at the command prompt to reveal the effects of the new settings:

$ vendor/drush/drush/drush cr

Note that the Bartik theme files are provided underneath the "core" directory. The files there will be overwritten when there is an upgrading of the core. This would cancel the change to Bartik.

Create a directory for "misc" outside the "core" directory to contain a backup of the changed files, and re-use them after upgrading of the core.

Execute to create "misc" directory:

$ mkdir /var/www/html/web/misc
$ cd /var/www/html/web/
$ cp core/themes/bartik/css/base/elements.css  misc
$ cp core/themes/bartik/css/components/main-content.css misc
$ cp core/themes/bartik/css/components/text-formatted.css misc
$ cp core/themes/bartik/css/layout.css misc

Log out the server.

Resolve CKEditor table border conflicting Mayo theme

(section added, 3 Sep 2018)

When the Mayo theme is used for web page appearance, a table created with CKEditor and seen while editing can only have the outer border lines displayed on the published page. The inner border lines disappear. To display the inner border lines, execute:

$ sudo gedit /var/www/html/web/themes/mayo/css/style.css

Uncomment the following border-style and border-width lines so as not to interfere with those set by CKEditor:

table tr td {
   padding: 4px 6px;
#  border-style: solid;
#  border-width: 0px;
}

table tr th {
#  border-style: solid;
   padding: 4px 6px;
#  border-width: 0px;
   border-right-width: 0px;
}

When creating or modifying a table with CKEditor, set the border size to 1, 2, ... to see the border lines. Set the border size to 0 to hide the border lines.

Set CKEditor to use Mayo theme stylesheet

(section added, 30 Sep 2018)

Edit Mayo theme's info setting:

$ sudo gedit /var/www/html/web/themes/mayo/mayo.info.yml

Insert a new line "- css/style.css":

ckeditor_stylesheets:
  - css/ckeditor-iframe.css
  - css/style.css

Save and exit.

Set print page to also use Mayo theme stylesheet

(section added, 30 Sep 2018)

(re-written to use symbolic link, 8 May 2019)

Execute to find the text "misc/print.css" which is the name of the stylesheet for the default printer-friendly printing:

$ cd /var/www/html/web
$ grep -R misc/print.css

This will show a number of directories containing files named "book-export-html.html.twig" containing:

<link type="text/css" rel="stylesheet" href="misc/print.css" />

The relevant directory is  /var/www/html/web/core/themes/stable/templates/layout.

"misc/print.css" should be corrected to "core/misc/print.css". However, even if this file is absent, it only means that the stylesheet would not apply. The page can still be displayed though without the desired style. To use Mayo theme stylesheet, the name should be changed.

However, the files underneath the "core" directory will be overwritten when there is an upgrade of the core. This would cancel the change to Mayo.

Instead, create a directory for "misc/print.css" outside the "core" directory so as not to be affected.

Execute to create 

$ mkdir /var/www/html/web/misc
$ sudo ln -s /var/www/html/web/themes/mayo/css/style.css  misc/print.css

This will create a symbolic link to the Mayo theme stylesheet which will then apply.

Add custom styles to Mayo theme

(section added, 30 Sep 2018)

Edit Mayo theme's css setting:

$ sudo gedit /var/www/html/web/themes/mayo/css/style.css

Add the following at the end of the file:

/* the following will provide a margin to CKEditor's editing screen to make it not so tight against the left */
body {
  margin: 10px;
}

/* the following will provide a border around text defined as "Formatted" using CKEditor's "Formatted" icon */
pre {
  border: 1px solid green;
  margin: 10px;
  background-color: #f8f9fa;
  padding: 0.5em;
}

/* the following will format the table of contents inserted using CKEditor's Table of Contents icon */
.widget-toc{
  display: table;
  border: 1px solid green;
  background-color: #f8f9fa;
  padding: 1em;
  font-size: 90%;
}

/* all the following will be needed to set up CKEditor's Style icon before use */

/* the following when used in conjunction with "p" for paragraph will give paragraph hanging indentation */
.hangtwice {
  margin-left: 80px;
  text-indent: -80px;
}
.hang1 {
  margin-left: 40px;
  text-indent: -40px;
}
.hang2 {
  margin-left: 80px;
  text-indent: -40px;
}
.hang3 {
  margin-left: 120px;
  text-indent: -40px;
}
.hang4 {
  margin-left: 160px;
  text-indent: -40px;
}
.hang5 {
  margin-left: 200px;
  text-indent: -40px;
}

/* the following when used in conjunction with "p" for paragraph will give paragraph indentation */ 
.indent1 { 
  margin-left: 40px;
}
.indent2 {
  margin-left: 80px;
}
.indent3 {
  margin-left: 120px;
}
.indent4 {
  margin-left: 160px;
}

/* the following will give hanging indentation to headings */
h2.hang {
  margin-left: 40px;
  text-indent: -40px;
}
h3.hang {
  margin-left: 40px;
  text-indent: -40px;
}
h4.hang {
  margin-left: 40px;
  text-indent: -40px;
}

Save and exit.

After re-defining "pre" as above, CKEditor's Formatted icon will give the following dropdown menu:

Add style buttons to CKEditor

(Separated as a section, 21 Dec 2021)

Log in the website as an administrator.

Choose Manage > Configuration > Text formats and editors > Configure Full HTML.

Move the Styles Icon down to a suitable position in the menu bar.

A blank dropdown text box will appear below the menu bar.

Enter a list of classes and titles as follows:

p.hangtwice|Hang Twice
p.hang1|Hang 1 
p.hang2|Hang 2
p.hang3|Hang 3
p.hang4|Hang 4
p.hang5|Hang 5
p.indent1|Indent 1
p.indent2|Indent 2
p.indent3|Indent 3
p.indent4|Indent 4
h2.hang|Head2Hang
h3.hang|Head3Hang
h4.hang|Head4Hang

"p" stands for paragraph.

".hangtwice" stands for the name of class as defined in the "css/style.css" file.

"Hang Twice" stands for the title seen when the Styles button is pressed to give a drop down menu.

 

Install Composer and Drush

(section added, 26 May 2019)

Drupal.org website describes various methods of installing Composer and Drush, generally referring to other websites for complicated steps which are difficult to follow.

The following simple steps do work.

Execute:

$ sudo apt install composer
$ cd /var/www/html/web
$ composer require drush/drush

Drush is installed under /var/www/html/web/vendor/drush/drush.

Update Drupal 8 using Composer

(section added, 18 Jan 2019)

Go to the root directory of the website:

$ cd /var/www/html/web

Configure "composer.json", if not done before: 

$ gedit composer.json

Move the following from the "replace" section to the beginning of the "require" section, with "," added at the end:

"drupal/core": "^8.7",

(configuration action added, 7 July 2019)

Update with the existing "core/composer.json":

$ cd core
$ composer update --with-dependencies
$ cd ..

Do similar update in case the following message is encountered when updating the "drupal/core" as described later below:

Dependency "..." is also a root requirement, but is not explicitly whitelisted. Ignoring.

(configuration action added, 18 July 2019)

(no longer necessary, 5 January 2020)

Check for outdated modules:

$ composer outdated drupal/*
$ composer show --outdated drupal/* -vvv

(syntax changed, 5 January 2020)

"-vvv" is to display what is going on.

("-vvv" added to all commands, 5 January 2020)

Update all:

$ composer update drupal/core --with-dependencies
$ composer update drupal ---vvv

(simplified command appears also working, 5 January 2020)

Update database:

$ vendor/drush/drush/drush updatedb -vvv

 

("vendor/drush/drush/" added, 26 May 2019)

Clear cache:

$ vendor/drush/drush/drush cr -vvv

Log out the server.

Upgrade Drupal 8 to Drupal 9

(section added, 20 Sep 2020)

Not all modules and themes for Drupal 8 are compatible with Drupal 9. They should be checked for compatibility.

Go to the root directory of the website:

$ cd /var/www/html/web

Install the following extensions:

$ composer require drupal/upgrade_rector drupal/upgrade_status

Choose Admin > Manage > Report > Upgrade Status.

Select projects (modules and themes) and run to check compatibility.

Apply patches as suggested.

If all the modules and themes have been made compatible, do upgrading.

Upgrading procedures have been suggested at https://www.drupal.org/docs/upgrading-drupal/upgrading-from-drupal-8-to-drupal-9-or-higher. However, the following suggested commands did not work because of dependencies errors caused by incompatibility:

$ cd /var/www/html/web
$ composer require drupal/core-recommended:^9.0.6 drupal/core-composer-scaffold:^9.0.6 drupal/core-project-message:^9.0.6 --no-update
$ composer update

The following command did work:

$ cd /var/www/html/web
$ composer require "drupal/core:9.0.6 as 8.9.6" --no-update && composer update

"9.0.6" and "8.9.6" are the current Drupal 9 version and installed Drupal 8 version respectively at the time of writing this.

Update database:

$ vendor/drush/drush/drush updatedb -vvv

Answer "yes" when asked to run specified post-update changes.

Error has been reported for the Mayo theme because the base theme has to be stated for Drupal 9.

Edit:

$ sudo nano themes/mayo/mayo.info.yml

Insert "base theme: stable" after the third line for "description".
Clear cache:

$ vendor/drush/drush/drush cr -vvv

Log out the server.

The Admin > Manage > Report > Upgrade Status page may still report incompatibility but this should not matter anymore.

Update Drupal 9 using Composer

(section added, 20 Sep 2020)

Same as those for Drupal 8:

$ cd /var/www/html/web
$ composer update drupal ---vvv
$ vendor/drush/drush/drush updatedb -vvv
$ vendor/drush/drush/drush cr -vvv

Log out the server.

Update Drupal 9 core manually

(tested with 9.2.8, one error corrected, 11 November 2021)

(updated, 5 January 2020)

(section added, 26 May 2019)

Set the website to maintenance mode.

Backup the existing directories and files to another location, e.g. .../webold:

$ cd /var/www/html
$ sudo mv web webold
$ sudo cp -Ra web webold

Download the latest Drupal version, and extract the files as described for fresh installation, i.e.:

$ sudo wget https://ftp.drupal.org/files/projects/drupal-9.x.x.tar.gz 
(change "9.x.x" to the version number)
$ tar -xzf drupal-9.x.x.tar.gz         
("sudo" not used so that user instead of "root" will become owner)
$ mv drupal-9.x.x web

Change to website root directory, remove two sub-directories and all files in the top level directory, including hidden files:

$ cd /var/www/html/web/
$ rm -rf core vendor
$ rm -f *.* .[a-z]*

Change to downloaded directory, copy updated sub-directories and files to the existing website directories:

$ cd ../drupal-9.x.x
$ cp -R core vendor ../web
$ cp *.* .[a-z]* ../web

Update with Composer and Drush

$ composer update -vvv
$ composer require drush/drush
$ vendor/drush/drush/drush updatedb -vvv
$ vendor/drush/drush/drush cr -vvv

Browse the website to see that the webpages can show up successfully.

Set to release the website from maintenance mode.

In case of a need to re-use and copy some of the old user-defined directories and files to the new directories (keeping original attributes), do some of the following as appropriate:

$ cd /var/www/html
$ sudo cp -Rav webold/libraries web
$ sudo cp -Rav webold/modules web
$ sudo cp -Rav webold/profiles web
$ sudo cp -Rav webold/sites web
$ sudo cp -Rav webold/themes web
$ sudo cp -Rav webold/vendor web   
$ sudo cp -Rav webold/composer.json web   
$ sudo cp -Rav webold/composer.lock web 
$ sudo cp -Rav webold/misc web            (this one is not a standard directory)

 

Relocate or restore Drupal 8

(section added, 8 May 2019)

(revised, 16 May 2019)

Export the relevant MySQL database using phpMyAdmin as a backup.

Backup the whole Drupal directories and files. For example, if Drupal's root directory is /web in /var/www/html/web, then everything from and below /web.

Use old pair of backups if fresh pair not available.

Create a new user and blank database only if new computer or new MySQL-server is used. 

Import the database backup.

Move the whole Drupal directories and files to the new directory, e.g. /var/www/new/web.

Keep "new" as the original name in case of no change, e.g. /var/www/html/web.

Keep the name "/web" unchanged since this is hard-coded in the beginning of links  in the form of "/web/..." to media and files embedded in the web pages.

Execute

$ sudo gedit /var/www/new/web/sites/default/settings.php

For

$database['default']['default'] = array (
    'database' = '<text>'
    'username' = '<text>'
    'password' = '<text>'
    ....
);

change the <text> to match the new database.

Change the DocumentRoot in the new site's currently used Apache2 site configuration file, e.g. /etc/apache2/sites-enabled/000-default.conf,  from "/var/www/html/" to /var/www/new, still one level above /web. Links in the form of "/web/..." to media and files will be interpreted as "/var/www/new/web/...", still accessible.

Web browsing to http://kctang.com.hk/web should be able to access Drupal at the new location.

Go to the next section in case web browsing fails to display.

If the DocumentRoot is set directly to the Drupal root directory "/var/www/new/web, web browsing of Drupal web pages is still possible, but links to media and files will be interpreted as "/var/www/new/web/web/... ", and the media and files will not be displayed.

Move the ".htaccess" file from /var/www/html to /var/www/new to re-direct web browsing to http://kctang.com.hk/web:

$ sudo mv /var/www/html/.htaccess /var/www/new

Execute as necessary:

$ vendor/drush/drush/drush updatedb -vvv
$ vendor/drush/drush/drush cr -vvv

("vendor/drush/drush/" added, 26 May 2019)

Use new shell to relocate and restore

(section added, 16 May 2019)

(revised, 26 May 2019)

If the relocation or restoration is not successful, non-display may happen because of some corruptions in the file settings.

Follow the Drupal core manual installation steps described above, but substitute "/var/www/html" described there with "/var/www/new".

Setting the website to maintenance mode would not be possible.

Before changing the settings file, move the ".htaccess" file from /var/www/html to /var/www/new:

$ sudo mv /var/www/html/.htaccess /var/www/new

Delete left-over module configuration files

(section added, 26 May 2019)

If a left-over configuration setting file is reported when a module is re-installed, execute to delete it:

$ cd /var/www/html/web
$ vendor/drush/drush/drush config:delete '<name of configuration file to delete>'

 

End of Page

 

Install MoinMoin wiki engine

Install MoinMoin wiki engine KCTang Thu, 25/12/2014 - 18:25

Go to End

Note

MoinMoin wiki engine serves wiki web pages.

The following installation instructions are outdated. Check https://help.ubuntu.com/lts/serverguide/moinmoin.html for updated instructions.

Prepare

Install Apache2 web server if not already installed.

Download MoinMoin (filename moin-1.9.7.tar.gz) from http://moinmo.in/MoinMoinDownload, usually to own "Downloads" directory.

Use the file manager to extract the contents of the compressed file under the "Downloads" directory as "moin-1.9.7".

Install

Install MoinMoin:

$ cd /home/< own account name >/Downloads/moin-1.9.7
$ sudo python setup.py install --force --prefix /usr/local --record=install.log

Install "wsgi" module and test:

$ sudo apt-get install libapache2-mod-wsgi
$ cd /usr/local/share/moin/server 
$ sudo python test.wsgi

Point the web browser to the address shown on the terminal, such as, http://localhost:8000/.

Setup wiki sites

Setup files for a wiki called "qswiki":

$ cd /usr/local/share/moin
$ sudo mkdir qswiki
$ sudo cp -R data qswiki
$ sudo cp -R underlay qswiki
$ sudo cp server/moin.wsgi qswiki
$ sudo cp config/wikiconfig.py qswiki

Do similar for "dscwiki".

Configure Apache2 for MoinMoin site:

$ sudo gedit /etc/apache2/sites-available/moinmoin.conf

Specify, keeping "<" and ">" in the following codes:

# MoinMoin WSGI configuration
# invoke moin wiki at the root url, like http://servername/mywiki/FrontPage:
# servername can be www.kctang.com.hk or localhost
# qswiki and dscwiki use FrontScreen instead of FrontPage to avoid changes from being overridden when a new FrontPage is installed with new software
WSGIScriptAlias /qswiki   /usr/local/share/moin/qswiki/moin.wsgi
WSGIScriptAlias /dscwiki   /usr/local/share/moin/dscwiki/moin.wsgi
<Directory /usr/local/share/moin/qswiki>
    AllowOverride None
    Require all granted
</Directory>
<Directory /usr/local/share/moin/dscwiki>
     AllowOverride None
     Require all granted
</Directory>
# create some wsgi daemons - use these parameters for a simple setup
WSGIDaemonProcess moin user=www-data group=www-data processes=5 threads=10 maximum-requests=1000 umask=0007
# use the daemons we defined above to process requests!
WSGIProcessGroup moin

Enable the site:

$ sudo a2ensite moinmoin

which will create a linked file in "/etc/apache2/sites-enabled"

If required to disable:

$ sudo a2dissite moinmoin

Configure wsgi:

$ sudo gedit /usr/local/share/moin/qswiki/moin.wsgi

Add at the end of the a2) paragraph:

sys.path.insert(0, '/usr/local/share/moin/qswiki')

Do the same for "dscwiki".

Enable web access, specify access rights and restart Apache2 service:

$ cd /usr/local/share 
$ sudo chown -R www-data:www-data moin 
$ sudo chmod -R ug+rwX moin 
$ sudo chmod -R o-rwx moin
$ sudo systemctl restart apache2
or
$ sudo service apache2 restart

Configure wiki:

$ sudo gedit /usr/local/share/moin/qswiki/wikiconfig.py

Specify:

# -*- coding: utf-8 -*-
## -*- coding: iso-8859-1 -*-

#url_prefix_static = '/mywiki' + url_prefix_static
url_prefix_static = '/qswiki' + url_prefix_static

# sitename = u'Untitled Wiki'
sitename = u'QS Wiki'

#page_front_page = u"FrontPage"
page_front_page = u"FrontScreen"

# superuser = [u"YourName", ]
#    (note: remove "#" at the front, and replace "YourName" with user id within quotation marks)

#acl_rights_before = u"YourName:read,write,delete,revert,admin"
#    (note: remove "#" at the front, and replace "YourName" with user id within quotation marks)

# the following added at the end of the Security section

textchas = {
    'en': {
        u"Please enter antispam password (email to kctang@kctang.com.hk for password):": ur"[password]",
        },
    }
textchas_disabled_group = u"TrustedGroup"

#    (note: replace [password] with actual password)

# end of addition

#mail_smarthost = ""
mail_smarthost = "kctang.com.hk"

# The return address, e.g u"Jgen Wiki <noreply@mywiki.org>" [Unicode]
#mail_from = u""
mail_from = u"K C Tang <kctang@kctang.com.hk>"

# "user pwd" if you need to use SMTP AUTH
#mail_login = ""
mail_login = u"kctang [email password]"

navi_bar = [
    # If you want to show your page_front_page here:
    #u'%(page_front_page)s',
    u'RecentChanges',
    u'FindPage',
    u'HelpContents',
    u'FrontScreen',
]

# The default theme anonymous or new users get
# theme_default = 'modernized'
theme_default = 'modernized_mobile'

Note MoinMoin Config files are:

  • /etc/apache2/sites-available/moinmoin.conf
  • /etc/apache2/sites-enabled/moinmoin.conf
  • /usr/local/share/moin/qswiki/moin.wsgi
  • /usr/local/share/moin/qswiki/wikiconfig.py
  • /usr/local/share/moin/dscwiki/moin.wsgi
  • /usr/local/share/moin/dscwiki/wikiconfig.py

End of Page

Install Roundcube webmail client

Install Roundcube webmail client KCTang Thu, 16/12/2021 - 11:30

Go to End

Note

16 Dec 2021: Updated installation command.

27 May 2019: Changed to use extracting directly to web directory. Protection measures added.

8 May 2019: All "/var/www/web" changed to "/var/www/html/web". "<version number>" used instead of a specific number. More descriptions on settings added.

25 Dec 2014: Created.

Install new by downloading

The usual software installation command "$ sudo apt install roundcube" does not appear to work.

Download roundcubemail-<version number>-complete.tar.gz by selecting "Complete: <version number>" at http://roundcube.net/download/, usually to own "Downloads" directory.

Extract the compressed file, move the extracted directory and files to underneath "/var/www/html", and rename the directory as "roundcube".

Ensure that the hidden ".htaccess" file is also moved.

A quicker way is to extract directly to "/var/www/html" and rename the directory there.

Install Roundcube under "/var/www/html":

$ cd /var/www/html
$ sudo tar -xzfv /home/<own account name>/Downloads/roundcubemail-<version number>-complete.tar.gz  (extract to current directory)
$ tar -xzf /home/<own account name>/Downloads/roundcubemail-<version number>-complete.tar.gz (extract to current directory) $ ls      (see the existence of the extracted directory)​
$ sudo mv roundcubemail-<version number> roundcube    (change direcotry name)
$ cd roundcube
$ sudo chown www-data:www-data logs
$ sudo chown www-data:www-data temp

"-xzfv" can be remembered as extract zipped file verbose. Use "-xzf" if "v" not working. 

(changed to use "-xzf", added cd roundcube, 16 Dec 2021)

(changed to use "tar" directly to web directory,  27 May 2019)

Install MySQL database + PHP + phpMyAdmin, if not already installed.

Login phpmyadmin at web browser:

http://localhost/phpmyadmin

Create a database user "roundcube" with database "roundcube":

  • click "Users" at the top menu bar
  • click "Add user" at the page middle
  • enter "roundcube" at the User name entry
  • select "Local" at the Host entry
  • enter and re-type the password
  • click "Create database with same name and grant all privileges"
  • click "Go" at the bottom

Enter at web browser:

http://localhost/roundcube/installer

Configure Roundcube:

  • accept all of the defaults
  • enter database user "roundcube", database "roundcube" and password under Database setup
  • click: "CREATE CONFIG"
  • download the configuration file generated and save it as config.inc.php
  • move the downloaded file:
$ mv /home/<own account name>/Downloads/config.inc.php /var/www/html/roundcube/config
  • click: "CONTINUE" on the webpage
  • click: "Initialize database"
  • set to use secure SMTP port 465
  • enter sender and recipient email addresses to test SMTP config
  • enter username and password to test IMAP config

Remove or rename installer directory, and protect logs and temp directories:

$ sudo cd /var/www/html/roundcube
$ sudo rm -R installer                   (remove)
$ sudo mv installer installer.original   (or rename)
$ sudo chown -R $USER:$USER logs
$ sudo chown -R $USER:$USER temp

(revised to add protection, 27 May 2019)

Configure PHP5 only (not necessary for PHP7):

$ sudo gedit /etc/php5/apache2/php.ini
  • define:

date.timezone = Asia/Hong_Kong

otherwise the date column of the web mail would be blank. 

Restart Apache2 service:

$ sudo systemctl reload apache2
or
$ sudo service apache2 reload

Log in at web browser:

http://localhost/roundcube

or

http://www.kctang.com.hk/roundcube

Click Settings > Identity to define some settings:

Click Settings > Preferences > User Interface > Date format > 24/7/2019 > Save.

Click Settings > Preferences > Composing Messages

> Compose HTML messages > always.

> when replying > start new message above the quote.

> deselect Force standard separator in signatures.

> Save.

Click Settings > Contacts > Import to import contacts files, e.g. previously exported from Google.

(more descriptions on settings added, 8 May 2019)

Upgrade

Download and extract the subdirectory of the new version to "Downloads" directory as "roundcubemail-<new version number>".

Upgrade existing directory using the installto.sh script:

$ cd /home/<own account name>/Downloads/roundcubemail-<new version number>/bin
$ ./installto.sh /var/www/html/roundcube

In case of big trouble

Uninstall Roundcube:

$ sudo apt-get remove roundcube

Remove "/var/www/html/roundcube":

$ cd /var/www/html
$ sudo rm -r roundcube

Delete mySQL user "roundcube" and database "roundcube" using phpMyAdmin.

Intall Roundcube as new.

End of Page

Install network file system

Install network file system KCTang Mon, 17/09/2018 - 19:20

Go to End

​​​​​​​Note

NFS network file system enables sharing of directories on a Ubuntu server to another Ubuntu client computer.

Install on server computer

Install network file system:

$ sudo apt-get install nfs-kernel-server

Configure to specify the directory to be exported:

$ sudo gedit /etc/exports

Specify:

/<directory exported> <ip address of computer to export to> (rw, sync, no_root_squash)

where "rw" = read and write.

Start the server, required after any rebooting:

$ sudo systemctl start nfs-kernel-server
or
$ sudo service nfs-kernel-server start

Setup client computer

Install nfs-common on the client side:

$ sudo apt install nfs-common

Create a local directory:

$ sudo mkdir -p /media/<local directory>

Mount the remote directory temporarily:

$ sudo mount <ip address of NFS server>:/<directory exported from the NFS server> /media/<local directory>

but remounting required after every reboot.

To keep the mounting permanently unaffected by reboot, edit "fstab" file: 

$ sudo gedit /etc/fstab

Add a line:

<ip address of NFS server>:/<directory exported from the NFS server> /media/<local directory> nfs auto 0 0

(revised 17/8/2018, "defaults" changed to "auto")

Mount again all devices as defined in the "fstab" file after changes:

$ sudo mount -a

Check devices actually mounted (this would show more than those defined in "fstab"):

$ sudo mount -l

End of Page

Install OpenVPN services

Install OpenVPN services KCTang Tue, 18/01/2022 - 09:26

Go to End

Note

  • 26 Apr 2022: Cipher added.
  • 18 Jan 2022: Installation procedures updated.
  • 11 Apr 2018: Re-direct function added.
  • 2 Sep 2018: Revised to suit Ubuntu 18.04 which requires a change of the network card device name.

Intro

OpenVPN enables remote client computers and smartphones to access VPN server's files and structure, and optionally re-direct clients' IP traffic through the VPN server.

Install VPN server for accessing file server

Switch to root:

$ sudo -s

Install openvpn and easy-rsa:

$ apt-get install openvpn easy-rsa

Set up public key infrastructure:

$ mkdir /etc/openvpn/easy-rsa/
$ cp -r /usr/share/easy-rsa/* /etc/openvpn/easy-rsa/
$ nano /etc/openvpn/easy-rsa/vars

Define in vars:

export KEY_COUNTRY="CN"
export KEY_PROVINCE="HK"
export KEY_CITY="HongKong"
export KEY_ORG="K C Tang Consultants Ltd"
export KEY_EMAIL="kctang@kctang.com.hk"
export KEY_OU=kctclVPN
export KEY_NAME=kctclVPN
# next line added to avoid error when building the certificate and key 
export KEY_ALTNAMES=kctclVPN

Generate master Certificate Authority (CA) certificate and key:

$ cd /etc/openvpn/easy-rsa/
$ source vars
# ./easyrsa init-pki
$ ./easyrsa build-ca

Enter New CA Key Passphrase: <>

Re-enter New CA Key Passphrase: <>

Enter Common Name: kctclVPN

Generate a key pair for the server:

$ ./easyrsa gen-req kctclVPN nopass

Accept Common Name default [kctclVPN]: <enter key>

Generate Diffie Hellman parameters and generate certificate for the server

$ ./easyrsa gen-dh
$ ./easyrsa sign-req server kctclVPN

Check and confirm the Common Name: yes

Enter Passphrase as previously defined: <>

Copy certificates and keys generated in subdirectory pki/ to /etc/openvpn/:

$ cp pki/dh.pem pki/ca.crt pki/issued/kctclVPN.crt pki/private/kctclVPN.key /etc/openvpn/

Config server.conf:

$ cd /
$ cp /usr/share/doc/openvpn/examples/sample-config-files/server.conf /etc/openvpn/
$ nano /etc/openvpn/server.conf

Define as follows:

port 1194
proto udp
dev tun
# Enter the paths if files not in the same directory as the server.conf file
ca </path to file/>ca.crt
cert </path to file/>kctclVPN.crt
key </path to file/>kctclVPN.key
dh </path to file/>dh.pem # not dh2048.pem
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist /var/log/openvpn/ipp.txt
keepalive 10 120
tls-auth ta.key 0
cipher AES-256-GCM
user nobody
group nogroup
# last two lines uncommented for linux system
persist-key
persist-tun
status /var/log/openvpn/openvpn-status.log
verb 3
explicit-exit-notify 1

(cipher added, 26 Apr 2022)

Generate ta.key for tls-auth:

$ cd /etc/openvpn
$ openvpn --genkey tls-auth ta.key
$ openvpn --genkey secret ta.key # not --secret

Config sysctl.conf:

$ nano /etc/sysctl.conf

Uncomment the following line to enable IP forwarding:

net.ipv4.ip_forward=1

Reload sysctl.conf:

$ sysctl -p /etc/sysctl.conf

Start the server:

$ systemctl start openvpn@server

Check if OpenVPN created a tun0 interface:

$ ifconfig tun0

Check syslog if tun0 does not appear:

$ grep -i vpn /var/log/syslog

Exit from root:

$ exit

Set the internet router to re-direct OpenVPN connections to server port 1194.

Extend to re-direct clients' IP traffic through VPN server

(section added 5 April 2018)

Define optionally in server.conf to re-direct clients' IP traffic such as web browsing and DNS lookups to go through the VPN server, i.e. the clients will appear to use the IP of the VPN server instead of the actual IP of the clients for internet traffic:

Config server.conf:

$ sudo nano /etc/openvpn/server.conf

Define by uncommenting the following line:

push "redirect-gateway def1 bypass-dhcp"

Some guide suggests to add the following, but this results in email server not working: (10 April 2018)

push "dhcp-option DNS 10.8.0.1"

Some other guides suggest to uncomment the following, this works: (10 April 2018)

push "dhcp-option DNS 208.67.222.222"
push "dhcp-option DNS 208.67.220.220"

However, it is found that it still works when the above two lines are left commented. Therefore, the only line needing change is the 'redirect-gateway' line. (2 Sep 2018)

Execute to restart the service:

$ sudo systemctl restart openvpn@server

Execute to see the network card device names:

$ ip route

Find the output line beginning with "default", e.g.:

default via 192.168.0.1 dev enp4s0 proto static metro 100

The name "enp4s0" after the word "dev" is the default network card device name. Previously, the default name is "eth0", but this has been changed after Ubuntu 16.04.

(2 Sep 2018)

Execute with the default name inserted after "-o":

$ sudo iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o enp4s0 -j MASQUERADE

Note that the iptables configuration will be lost after reboot.

Store the current iptables configurations:

$ sudo sh -c "iptables-save > /etc/iptables.up.rules"

View and remove any configurations no longer applicable:

$ sudo nano /etc/iptables.up.rules

Do the same whenever the iptables configurations have been changed.

Config file for use on reboot:

$ sudo nano /etc/network/interfaces

Define to reuse the stored configurations:

auto lo
iface lo inet loopback
post-up iptables-restore < /etc/iptables.up.rules

Generate files for each Windows client

Switch to root:

$ sudo -s

Generate a certificate and private key for each client user of <username>:

$ cd /etc/openvpn/easy-rsa/
$ ./easyrsa gen-req <username> nopass
$ ./easyrsa sign-req client <username>

Check and confirm the Common Name: yes

Enter Passphrase as previously defined: <>

Copy or move client's certificate and key to a Samba directory, which is for temporary use only: to enable emailing:

$ cd /etc/openvpn/easy-rsa/
$ cp pki/ca.crt pki/issued/<username>.crt pki/private/<username>.key /<Samba directory>/
$ cd /etc/openvpn/
$ cp ta.key /<Samba directory>/ 

Change the owners of the files:

$ cd /<Samba directory>
$ chown nobody:nogroup ca.crt ta.key <username>.crt <username>.key
$ chmod 644 ca.crt ta.key <username>.crt <username>.key

Create a <username>.txt under the Samba directory and define it to contain:

client
dev tun
proto udp
remote kctang.com.hk 1194
resolv-retry infinite
nobind
persist-key
persist-tun
mute-replay-warnings
ca ca.crt
cert <\\path\\><username>.crt
key <\\path\\><username>.key
remote-cert-tls server
tls-auth ta.key 1
cipher AES-256-GCM
auth-nocache
verb 3

(cipher added, 26 Apr 2022)

Specify path in Windows format if ca.crt, ta.key, <username>.crt, and <username>.key are to be saved in a folder different from <username>.txt.

Change filename from <username>.txt to <username>.ovpn. Creating as a txt file first permits editing by the usual text processors. Watch out that the line breaks are using Windows' linebreak code.

E-mail ca.crt, ta.key<username>.crt, <username>.key, and <username>.ovpn files to the client computer.

Remove the files:

$ rm ca.crt ta.key <username>.crt <username>.key <username>.ovpn

Exit from root:

$ exit

Install on Windows client computer

Download and install the latest OpenVPN Windows Installer.

An OpenVPN GUI icon should appear at the bottom system tray, with no connection yet.

Save ca.crt, ta.key, <username>.crt, <username>.key, and <username>.ovpn files (emailed from the server) under C:\Program Files\OpenVPN\config\

Existing config files will be copied to a new config-auto directory. Delete the directory if the files are no longer used.

Run on Windows client as a service

To start OpenVPN automatically as a service every time after rebooting:

  • Click Windows Start > search for Services.
  • Right-click OpenVPN Interactive Service > Properties > Start or Restart.
  • Change Startup type to Automatic.
  • Click OK.

Open file manager and enter \\10.8.0.1 to access the vpn server. All clients will use the same \\10.8.0.1 to map the actual different ip addresses assigned by the vpn server.

If connection is not successful, restart the computer.

Run on Windows client using GUI

To start OpenVPN manually every time after rebooting, or re-connect after loss of connection after sleep or hibernation::

  • Right-click OpenVPN GUI icon on Desktop.
  • Click Run this program as an administrator > Yes.

Or 

  • Click the OpenVPN GUI icon on the bottom system tray to connect or right-click the icon and click Connect.

Map drive for quick access

Define a drive to be listed in the file manager directory to represent the server:
  • Open file manager and enter \\10.8.0.1 to access the vpn server.
  • Right-click the desired folder.
  • Click Map network drive.
  • Choose a drive name to represent the folder.
  • Click Finish.

End of Page

Install x11vnc

Install x11vnc KCTang Tue, 28/05/2019 - 13:53

Go to End

Note

28 May 2019: Disabling Wayland added.

18 May 2019: Page added.

Intro

x11vnc enables remote access to the graphical desktop of Ubuntu server.

Set router

Set the internet router to permit access to the server through ports 5900, 5901, etc.

Disable Wayland

(section added, 28 May 2019)

Ubuntu-desktop uses gdm3 (Wayland) display manager. Wayland is not compatible with Xorg dispaly server used by x11vnc.

Execute to disable Wayland:

$ sudo gedit /etc/gdm3/custom.conf

Remove "#" to uncomment the following line:

#WaylandEnable=false

Reboot the computer.

Install

Execute:

$ sudo apt install x11vnc
$ sudo apt install xvfb

xvfb provides a virtual X window, which is required for the -create option below.

Start

Execute:

$ x11vnc                    (not requiring a password to connect to port 5900)
or
$ x11vnc -rfbport <port number>
or
$ x11vnc -usepw             (requiring a password to connect)
or
$ x11vnc -create -usepw     (create a virtual desktop)

The default port to connect from client computers is 5900.

Specify 5901 for the <port number> if connection is to be permitted at port 5901.

Connection with or without password is possible.

Execute to set up password:

$ x11vnc -storepasswd

The password will be stored in the file ~/.vnc/passwd, i.e. home/<current user>/.vnc/passwd.

Connection is possible if there is already a graphical desktop logged on at the server. The screen movement at the server and the client computer will be synchronised. This would be good for monitoring the screen movement at the server.

If there is no graphical desktop logged on at the server, then:

  • use the "-create" option
  • connect remotely which should show a terminal window at the server
  • execute at the terminal window to bring up other software, such as:
  • $ firefox
    $ nautilus
  • or the graphical session:
  • $ gnome-session
    
  • click Activities to access other software.

The remote screen movement will not be seen at the server.

Ubuntu-desktop with gdm3 is used above.

x11vnc does not work well with gdm3 (Wayland) desktop. Therefore, just use gdm3 desktop.

Gnome (Wayland) desktop also does not permit the starting of backintime

Stop

Press Ctrl-C to terminate the connection.

End of Page

Install OpenSSH services

Install OpenSSH services KCTang Tue, 26/05/2020 - 16:33

Go to End

Note

26 May 2020: systemctl file command revised.

27 May 2019: Security settings added.

11 Apr 2018: Page added.

Intro

OpenSSH enables remote client computers and smartphones to access the server computer's text based terminal shell in a secured manner. "SSH" stands for secured shell.

Install OpenSSH server

Execute:

$ sudo apt install openssh-server

The software will be installed at /etc/ssh.

In case of complaint of no directory, execute  to make directory first:

$ sudo mkdir /etc/ssh

Edit config file:

$ sudo cd /etc/ssh
$ sudo gedit sshd_config

Specify:

# Port 22                             (which is the default port)
Port 2nnn                             (change to some other 4-digit port, 2nnn)
# PermitRootLogin prohibit-password   (meaning no password required)
PermitRootLogin no                    (meaning no root login)

(security settings added, 27 May 2019)

Restart the service:

$ sudo systemctl restart sshd.service
or
$ sudo systemctl restart ssh.service

Check status:

$ sudo systemctl status sshd.service

If found disabled:

$ sudo systemctl enable ssh

(status check added, 26 May 2020)

Change the internet router to permit the use of port 2nnn.

The above is already sufficient for use. Read https://help.ubuntu.com/lts/serverguide/openssh-server.html for more configurations, if desired.

Install SSH client on Windows

Download Putty from https://www.putty.org/ and install.

Enter the Host Name, change the Port to 2nnn, highlight Default Settings and press Save:

"Only on clean exit" is the default. When the server's terminal window is exited with "exit" or "logoff", the PuTTY screen and connection would only close if other processes using the PuTTY connection have all been closed.

Press Open.

Accept the next screen to confirm the server's security key shown, if trusted. This would be necessary for the first time only.

Log in as the usual command terminal. No graphical interface is provided.

Use PuTTYgen that comes installed with PuTTY to generate key pairs, only if required. Read its Help.

Configure for VNC

If PuTTY is used for VNC connection, config the tunnel by entering the Source port and Destination as follows, then press Add to move the setting to the upper window:

With "Local" selected, the Source port means the port of the client computer. It can be "5900" or any free port. "Localhost:5900" at the Destination means the host computer, not the client computer. "5900" refers to the port number on the host computer providing VNC server service.

Go back to the first screen, highlight Default Settings and press Save again.

End of Page