Computer Application 電腦應用

Computer Application 電腦應用 KCTang Thu, 25/12/2014 - 16:35

Commonly used Ubuntu Linux commands 常用的Ubuntu Linux指令

Commonly used Ubuntu Linux commands 常用的Ubuntu Linux指令 KCTang Thu, 22/03/2018 - 00:59

Go to End

Note

$ = the terminal command prompt against which commands are entered.

sudo = superuser.

<text> = information to be entered, angle brackets themselves are not to be entered.

[abc] = options a, b and c, brackets are not necessary.

Capitals and small letters behave differently.

View system information

Show who am I:

$ whoami

Show current date and time:

$ date

Show this month's calendar:

$ cal

List open ports and their processes id:

$ sudo lsof -i

or

$ sudo netstat -lptu

or

$ sudo netstat -lptun

Display disk space:

$ df

Display disk space in a more readable format:

$ df -h

Display file contents:

$ cat <directory>/<sub-directory>/<filename>

Use path and file names

<directory path> = <directory>/<sub-directory> referring from the current directory, without filename stated.

<filename path> = <directory>/<sub-directory>/<filename> referring from the current directory.

Precede with "/" if referring the path from the root directory. Deleting files under a sub-directory but incorrectly referring to the root directory is dangerous. Better avoid referring from the root directory.

Use "*" as a wildcard to represent texts before or after directory or file name, e.g. "*name", "name*" or "*" for any name.

Use "." alone to represent the current directory.

Use "../" to represent the directory immediately above the current directory.

Use "/." at the end to represent all under the stated <directory path>, e.g. "<sub-directory>/."

Search things

Find files with filenames containing <filename> in <directory path>:

$ find <directory path> -name "<filename>"

Use "-iname" to ignore upper or lower case differences.

Find the files and open for editing:

$ find <directory path> -iname "<filename>" -exec nano '{}' \;

Find text in files:

$ grep -[options] <text> <filename path>

where option:

  • -i = ignore upper or lower case differences
  • -r = search sub-directories recursively
  • -h = hide names of files preceding lines of output
  • -w = search for exact text
  • -c = count the number of matches
  • -n = precede lines of output with the numbers of lines containing the text
  • -v = show only lines not containing the text
  • -l = list filenames only
  • --colour = display the output in colour

Adjust disk space

Find duplicate files across several directories and replace duplicate files with hard links to save space:

$ sudo rdfind -makehardlinks true <directory path 1> <directory path 2>

note: rdfind downloadable at http://rdfind.pauldreik.se/rdfind.1.html#lbAG.

Recover such harddisk space of deleted files not reported by "$ df":

$ cd /<directory name of the harddisk> 
​​$ sudo dd if=/dev/zero of=tempfile 
$ sudo rm tempfile

Edit files

$ sudo nano <filename path>

or

$ gksudo gedit <filename path>

Note that the following can work sometimes but may have problem under some desktop environment:

$ sudo gedit <filename path>

Shutdown and start up

Shutdown the system:

$ sudo shutdown

Shutdown and reboot the system:

$ sudo reboot

Start a service:

$ sudo systemctl start <service name>

or older method:

$ sudo service <service name> start

or even older method:

$ sudo /etc/init.d/<service name> start

Restart a service:

$ sudo systemctl restart <service name>

or older method:

$ sudo service <service name> restart

or even older method:

$ sudo /etc/init.d/<service name> restart

Stop a service:

$ sudo systemctl stop <service name>

or older method:

$ sudo service <service name> stop

or even older method:

$ sudo /etc/init.d/<service name> stop

Mount devices

Mount a single device "/media/newbackupdrive":

$ sudo mount /media/newbackupdrive

Remounting required after reboot.

Umount single device "/media/newbackupdrive":

$ sudo umount /media/newbackupdrive

Check devices defined in the filesystem table file "fstab" to be mounted upon booting:

$ cat /etc/fstab

Check devices actually mounted (this would show more than those defined in "fstab"):

$ sudo mount -l

Add or delete groups and users

Create new group:

$ sudo addgroup <new group name>

Create new user, with a group and a home directory of the same name created if not already existing:

$ sudo adduser <new user name>

To verify:

$ ls -lh /home

shows that the new directory has "drwxr-xr-x" permissions, i.e. "d" for directory with "rwx" owner permissions but "r-x" group and others' permissions.

Add a user to a group:

$ sudo adduser <user name> <group name>

Delete user, and group of the same name, keeping the home directory:

$ sudo deluser <user name>

Delete group:

$ sudo delgroup <group name>

Change ownership

Change file or directory ownership:

$ chown -R <owner name>:<group name> <filename path>

where:

  • -R = recursively from and below if <filename> is a sub-directory

Change permissions

Change file or directory permissions:

$ chmod -R [ugoa][-+=][rwxXst] <filename path>

where:

  • -R = recursively from and below if <filename> is a <sub-directory> 
  • u=owner, g=group, o=others, a=all 
  • -+= mean minus, add or equal permissions
  • r=read, w=write and delete, x=execute file or change directory into; Xst=for more special choices
  • e.g. "u+rw" = add read and write permission to owner

An alternative form is:

$ chmod -R <ugo> <filename path>

where:

  • <ugo> is a 3 digit number where u=owner, g=group, o=others 
  • each digit can be:
    • 0 for nothing
    • 1 for execute "x"
    • 2 for write "w"
    • 3 = 1 + 2 = "wx"
    • 4 for read "r"
    • 5 = 4 + 1 = "rx"
    • 6 = 4 + 2 = "rw"
    • 7 = 4 + 2 + 1 = "rwx"

therefore:

  • 666 = read and write permissions to all
  • 777 = read, write and execute permissions to all

Change password

Change own password:

$ passwd

Change other user 's password:

$ sudo passwd <other user's name>

Create directories

Make new directory:

$ mkdir <directory>

Make new directory and sub-directory in one go, "-p" means making parent directory also:

$ mkdir -p <directory>/<sub-directory>

Remove files and directories

Remove empty sub-directory:
$ rmdir <directory path>

Remove empty sub-directory and its parent directory in one go:

$ rmdir /<parent directory>/<sub-directory>

Remove file:

$ rm <filename path>

Remove files and directories starting from and below sub-directory, even for empty sub-directory:

$ rm -r <directory path>

List and change directories

List names of current directory contents, hiding entries starting with ".":

$ ls

List current directory contents, with more detailed information:

$ ls -[options]

where options

  • l = list also permissions, owners, date and size
  • a = list also entries starting with "."
  • h = to be used in conjunction with "l", show file sizes in "K" or "M"

List other directory contents:

$ ls -[options] <directory path>

Change working directory:

$ cd <directory path>

Copy files and directories

Copy a file within the same directory:

$ cp <source filename> <new filename>

Copy a file across different directories, keeping the same filename:

$ cp <source directory>/<sub-directory>/<filename> <target directory>/<sub-directory>/.​

to give:

<target directory>/<sub-directory>/<filename>

​Copy a file across different directories, to a new filename:

$ cp <source directory>/<sub-directory>/<filename> <target directory>/<sub-directory>/<new filename>

to give:

<target directory>/<sub-directory>/<new filename>

Omit:

<source directory>/<sub-directory>/

if copying files in the current directory.

Copy directories recursively, keeping the directory name:

$ cp -R <source directory>/<sub-directory A> <target directory>/.

to give:

<target directory>/<sub-directory A>

Copy directories recursively:

$ cp -R <source directory>/<sub-directory A> <target directory>/<sub-directory B>

If sub-directory B exists, all files and directories under sub-directory A will be copied under sub-directory B.

If sub-directory B does not exist, it will be created, and all files and directories under sub-directory A will be copied under it.

Move files and directories

Use "mv" instead of "cp" for the above copy commands. No need to use "-R". Recursive move is the default.

Backup a directory of files

Archive (-a) all files under a sub-directory (A) and all sub-sub-directories underneath to the same sub-directory name under another sub-directory (B) keeping all the file attributes, symbolic links and time-stamps unchanged, preserving hard-links (-H) and displaying the progress verbosely (-v) and the numbers in human-readable format (-h):

$ sudo rsync -aHvh <source directory>/<sub-directory A> <target directory>/<sub-directory B>/​

to give:

<target directory>/<sub-directory B>/<sub-directory A>

The command can be used repeatedly to update the files in sub-directory A in the new location. If the source files have not been changed, no over-writing copying will be done. This would save time and is better than the cp command.

 

Page last updated: 21 March 2018

End of Page

Setting Up Ubuntu Server 架設Ubuntu伺服器

Setting Up Ubuntu Server 架設Ubuntu伺服器 KCTang Tue, 28/05/2019 - 02:55

Go to End

28 May 2019: Notes regarding unsuccessful installation added. Showing boot messages added.

7 May 2019: Contents added.

25 Dec 2014: First created as a flysheet without its own contents.

Note

Ubuntu is a linux server software.

It has a desktop version and a server version, amongst other products.

The kernels of the desktop version and the server version are the same. The installation procedures are slightly different.

The desktop version provides a graphical user interface and the server version has a text based user interface, though a graphical user interface can subsequently be installed on the server version. Once installed with the graphical user interface, the two versions would not look much different to the users. 

For small system, the desktop version should be easier to use.

Download

Download free of charge here

Install

Read installation guides:

Read fuller user guide here.

Set up storage drives

When allocating drive spaces, choose "Something else" with the desktop version and "manual" with the server version if one does not want to adopt the default settings, e.g. if one wants to have more options, such as setting one drive for "/boot" partition and one for "/" root system partition.

Set the computer bios to boot from the selected boot up disk. Remember to save the setting.

It is said that the boot partition needs only be about 300 Mb big. However, experience tells that it gets full easily because of frequent updating of the kernels and retainage of the last few kernels in the boot partition.

To remove kernels no longer needed to be retained, execute:

$ sudo apt autoremove

However, sometimes, the 300Mb boot partition has gone up to 100% full and insufficient to contain the required last few kernels such that there are no unused kernels to be removed to free up space.

Therefore, it is recommended to set up a boot partition of 1Gb.

Our company set-up:

  • boot partition on the smaller SSD disk sold with the computer
  • root system partition on the bigger hard disk relocated from old computer
  • additional hard disks mounted on the system for storage of data

To see disks mounted, execute:

$ df -h

The mounting configuration file is contained in /etc/fstab.

To mount disk using graphical user interface permanently, click "Activities" , enter "disk" and select the "Disks":

Mounting using "Disks" would change /etc/fstab permanently, without the need to change the /etc/fstab file manually.

Install application software packages

Update the software repository:

$ sudo apt update

Install software package:

$ sudo apt install <name of software>

Uninstall software package, keeping its configuration files:

$ sudo apt remove <name of software>

Uninstall software package and its configuration files:

$ sudo apt purge <name of software>

Alternatively, to bring up the graphical software package installation manager, execute:

$ sudo synaptic

or, on the graphical desktop, click "Activities" , enter "synaptic".

Upgrade Ubuntu release

Upgrade Ubuntu release:

$ sudo do-release-upgrade

Re-install

If it is necessary to re-install Ubuntu, back up everything first. 

Ubuntu usually can recognize the existing boot and root systems and data disks when Ubuntu is re-installed.

Try to keep the new co-existing with the old and change later after the system is running.

Keep the existing configurations and re-use the existing configuration files as much as possible.

Most of the configuration files are stored in the /etc directory.

If Ubuntu is freshly installed, copy back the files and directories of the application software to the new system before re-installing the application software, e.g. the following files or directories:

  • /etc/aliases
  • /etc/aliases.db
  • /etc/anacrontab
  • /etc/apache2
  • /etc/ca-certificates
  • /etc/ca-certificates.conf
  • /etc/dovecot
  • /etc/hostname
  • /etc/hosts
  • /etc/openvpn
  • /etc/phpmyadmin
  • /etc/postfix
  • /etc/samba
  • /etc/vsftpd.conf
  • /etc/vsftpd.chroot_list

Do not copy back the /etc/fstab file because the file system configuration there is no longer applicable to the new system and copying back will cause the new system not re-bootable.

If copied back, use "Disk" software mentioned above to make some changes so that the /etc/fstab file reflects the latest configuration, before re-booting.

The information of the previous user names, groups and passwords are contained in the following files:

  • /etc/passwd
  • /etc/group
  • /etc/gshadow
  • /etc/shadow

Use text editor and spreadsheet software to pick out old information in the old files and not superseded by the new information and copy it cover to the new files to enable the users to access their previous data directories, otherwise, re-redefine all the user names and passwords one by one.

Re-install the additional application software after coping back as described above.

The software might have updated the default configuration files over time but usually would tolerate using the old configuration files. Therefore, it is preferred to keep using the old configuration files to ensure that the software is successfully re-installed before making changes.  There may be very slight changes to  new configuration files. The installation process will usually prompt to ask which file to keep and give an opportunity to see a comparison. It may be better to record the changes first and change later. Using the new configuration files in conjunction with the old physical configurations may cause the software not workable. Carefully check for any slight changes to the new configuration files if using the new configuration files to add the old configuration settings.

If re-installation is not successful after :

$ sudo apt install <name of software>

try:

$ sudo apt remove <name of software>

or even:

$ sudo apt purge <name of software>

or even remove the software directory before:

$ sudo apt install <name of software>

Usually, it is the configuration which is causing problems.

(The following added, 28 May 2019)

The "remove" option will leave behind the user modified configuration files.

Rename them.

Re-install the application software.

Use the new configuration file if see if the application software can run successfully.

If yes, copy the user modifications in the old configuration file to the new so far as compatible, and see whether the software application can run successfully.

The "purge" option will remove all configuration files and other related software. Care should be exercised to review that the related software would not be required by other software.

Removing the software directory is the extreme option with little additional effect. 

Show messages when booting

(section added, 28 May 2019)

Execute:

$ sudo gedit /etc/default/grub

Specify:

#GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"    (uncomment the default)
GRUB_CMDLINE_LINUX_DEFAULT=""                 (add)

Execute:

$ sudo update-grub

 

End of Page

Install backintime

Install backintime KCTang Tue, 12/05/2020 - 16:30

Go to End

12 May 2020: Updated.

30 May 2019: Page added.

Note

backintime enables incremental backup of files.

Listing it as the first application software to be installed after installing Ubuntu is an indication of the importance of backing up files.

Install

Execute:

$ sudo apt install backintime*
$ sudo apt install backintime-qt

(updated 12 May 2020)

Bring up

Select Activities.

Type "ba" to bring up some application icons:

Select Back in Time (root) to show the front page:

Alternatively, execute

$ sudo i backintime
or
$ pkexec backintime

However, the terminal commands are not reliable to bring up backintime. Using the graphical desktop is more reliable.

(updated 12 May 2020)

Config

Select the setting button:

Select the General page:

Specify:

  • where to save snapshots
  • host computer name, user name "root" to have widest permission, profile starting with 1
  • scheduled intervals and hours to save snapshots

Select the Include page > Add folder to select the folders to include in backup :

Add file can only include files not a whole folder.

Select Exclude page.

Generally accept the default exclusions.

Select Add folder to select sub-folders to be excluded from the backup when their parent folders have been included:

Select Auto-remove to set time criteria to automatically remove old spanshots:

Select Options page.

Generally accept the default settings.

Select Expert Options page.

Generally accept the default settings.

Select OK to save.

Exit the software. It will run at the specified times.

Take snapshot any time

Bring up the front page:

Select the take snapshot button: 

It would take some time to take a snapshot depending on the extent of file changes since the last snapshot.

Restore

Restore backup files when the present files have been lost or corrupted.

Bring up the front page:

 

Select:

  • the desired snapshot in the left window
  • the desired backup folder in the middle window
  • one or more folders or files in the right window

Select the restore button:

Select whether to backup local files with a trailing suffix before restoring the old files:

Select Yes to proceed if for sure.

Folders and files will be restored.

It would take some time depending on the size to be restored.

Let the process finish with the use permissions set back to the original state.

If only the folders in the middle window is selected before the restore button is selected, this message will appear to indicate that the "/" root folder will be restored:

It is important not to just select the folders in the middle window and select the restore button to proceed, because it will easily restore files to the "/" root directory and overwrite the still valid system and programme files in the root directory to cause problems.

Repair

Sometimes, backintime cannot be brought up to run, even after a removal and re-installation.

This can be caused by the removal of some other programmes which backintime depends on but which have been removed when some other application software is removed. Re-installation of backintime may not bring them back.

The following shows the programmes which backintime depends on:

Try to re-install the missing programmes:

$ sudo apt install <name of programme>

rsync, python3, openssh-client are likely missing programmes.

 

End of Page

 

Install Samba file server for Windows

Install Samba file server for Windows KCTang Tue, 07/05/2019 - 20:16

Go to End

7 May 2019: Slight adjustments. "gksudo gedit" changed to "sudo gedit" as Ubuntu 18.04 dropped "gksudo".

25 Dec 2014: First created.

Note

Samba file server enables specified directories to be accessible by Windows computers on the same network.

Install

Install the package​:​

$ sudo apt-get update 
$ sudo apt-get install samba

Define a workgroup

Edit the config file:

$ sudo gedit /etc/samba/smb.conf

​​​​​Define workgroup name as "kctcl" in the "[global]" section:

workgroup = kctcl

Uncomment to restrict access to server users only:

security = user

Add the following if OpenVPN used (added 7 May 2019):

hosts allow = 192.168.0. 10.8.0. 127.0.0.

Define directories to be shared

Add a section at the end to share directories:

[<sub-directory name or other short name>] 
pth = /<directory name>/<sub-directory name> 
browseable = yes 
guest ok = yes 
read only = no 
create mask = 0775 
directory mask = 0775 
# do not include the next two lines if access is restricted to the owning user (added 7 May 2019)
force user = nobody 
force group = nogroup

Create the directory to be shared, if not already existing:

$ sudo mkdir -p /<directory name>/<sub-directory name>

Change ownership of the directory:

$ sudo chown nobody:nogroup /<directory name>/<sub-directory name>

Start service

Start or restart Samba service whenever the config file is changed:

$ sudo systemctl restart smbd nmbd

or, if ".service" is not automatically appended when executing the above command:

$ sudo systemctl restart smbd.service nmbd.service

End of Page

Install Postfix + Dovecot e-mail server

Install Postfix + Dovecot e-mail server KCTang Tue, 05/01/2021 - 01:39

Go to End

5 Jan 2021: Increase imap-login process_limit. List Dovecot full custom settings.

9 Dec 2020: Define cron job to delete filtered mails.

20 Sep 2020: Stop using mail-stack-delivery.

8 May 2020: Correct typo errors.

30 May 2019: Add anti-virus and spam mail filtering.

17 May 2019: Add copying emails to external accounts.

7 May 2019: Change "gksudo gedit" to "sudo gedit" as Ubuntu 18.04 dropped "gksudo". Add header_checks. Add auto creation of Trash folders. Add webmaster.

29 Sep 2018: Add "body_checks" for spam control.

20 Sep 2018: Increase message_size_limit to 20 times the default.

5 Apr 2018: Increase message_size_limit to 10 times the default.

25 Dec 2018: Publish on web.

12 Apr 2014: Specify maximal_queue_lifetime to notify unsuccessful delivery immediately.

2 Apr 2014: Specify fully qualified domain name.

Note

Postfix is a mail transfer agent (MTA) responsible for sending out and receiving emails between servers.

Dovecot is a mail delivery agent (MDA) responsible for sending out and receiving emails between a server and its users.

Mail-stack-delivery was a combined package containing both Postfix and Dovecot. It is now no longer supported.

Re-direct to server ports​

Set the internet router to re-direct the following connections to server ports:

  • SMTP = port 25 (for receiving or sending emails)
  • secure SMTP = port 465 (for receiving or sending emails securely)
  • IMAP = port 143 (for retrieving emails)
  • secure IMAP = port 993 (for retrieving emails securely)
  • POP3 = port 110 (for retrieving emails)

Install both Postfix and Dovecot​

Ubuntu 20.04 does not support the combined package "mail-stack-delivery" anymore.

Install Dovecot and Postfix individually:

$ sudo apt install dovecot-imapd dovecot-pop3d
$ sudo apt install postfix

However, the previous config file "/etc/dovecot/conf.d/99-mail-stack-delivery.conf" is still retained for use, because "99" represents the last and overriding config file. This eliminates the need to change the individual config files.

(revised to install individually, 20 Sep 2020)

Reconfigure Postfix

Reconfig:

$ sudo dpkg-reconfigure postfix

Use Tab key to change selection.

Select "Internet Site".

Enter the following information:

System mail name: <fully qualified domain name, such as "kctang.com.hk">
Root and postmaster mail recipient: <such as "kctang">
Other destinations to accept mail: <fully qualified domain name, such as "kctang.com.hk">, <server name such as "server1">, localhost.localdomain, localhost
Force synchronous updates on mail queue: No
Local networks: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
Use procmail for local delivery: No
Mailbox size limit (bytes): 0
Local address extension character: +
Internet protocols to use: all

Activate the changes:

$ sudo systemctl reload postfix
or
$ sudo service postfix reload

Edit "main.cf" settings:

$ sudo gedit /etc/postfix/main.cf

Specify:

myhostname = kctang.com.hk
# last line changed from server name to fully qualified domain name,
# otherwise some servers would not accept e-mails sent without fully qualified domain name, 2/4/2014
message_size_limit = 204800000
# last line added to increase the default 10 times, 5/4/2014
# increased to 20 times, 20/9/2018
maximal_queue_lifetime = 0
# last line added to report unsuccessful delivery immediately instead of after the default of 5 days, 12/4/2018
body_checks = regexp: /etc/postfix/body_checks
# last line added to refer to another file to check contents of email bodies, 29/9/2018
header_checks = regexp:/etc/postfix/header_checks
# last line added to check headers, KCTang 30/9/2018

Edit "master.cf" settings:

$ sudo gedit /etc/postfix/master.cf

Specify​:

smtp      inet  n       -       y       -       -       smtpd
smtps     inet  n       -       y       -       -       smtpd
# last line uncommented
   -o smtpd_sasl_auth_enable=yes
# last line added to enable STARTTLS authentication
   -o smtpd_client_restrictions=permit_sasl_authenticated,reject
# last line added to reject if not authenticated, no space after ","
   -o smtpd_tls_wrappermode=yes
# last line added to force use of TLS
   -o milter_macro_daemon_name=ORIGINATING
# last line added

Create "body_checks" file:

$ sudo gedit /etc/postfix/body_checks

Specify​ one or more lines of texts within //:

/unique text contained in email you do not want to receive/ DISCARD

"DISCARD" means delete from the server.

Create "header_checks" file:

$ sudo gedit /etc/postfix/header_checks

Specify similarly.

(header_checks added 7 May 2019)

Change Dovecot settings

Edit config file:

$ sudo gedit /etc/dovecot/conf.d/99-mail-stack-delivery.conf

Specify:

disable_plaintext_auth = yes

mail_location = maildir:~/Maildir:LAYOUT=fs
# LAYOUT=fs added to last line, to use "/" instead of "." to denote sub-folders

auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@

# IMAP configuration
protocol imap {
        mail_max_userip_connections = 1000
        # 10 in last line increased to 1000
        imap_client_workarounds = delay-newmail
}

# POP3 configuration
protocol pop3 {
        mail_max_userip_connections = 50
        # 10 in last line increased to 50
        pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
}

# LDA configuration
protocol lda {
        postmaster_address = postmaster
        mail_plugins = sieve
        quota_full_tempfail = yes
        deliver_log_format = msgid=%m: %$
        rejection_reason = Your message to <%t> was automatically rejected:%n%r
}

# Plugins configuration
plugin {
        sieve=~/.dovecot.sieve
        sieve_dir=~/sieve
}

# Authentication configuration
auth_mechanisms = plain login

service auth {
  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/dovecot-auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

# The following are additional to those in 15-mailboxes.conf.
# They are to auto create Trash folders.
# Trash folders would not be backed up with back-in-time,
# and therefore would need to be re-created after email recovery is done from back-in-time.
namespace inbox {
  mailbox Trash {
    auto = subscribe
  }
}

# process_limit increased
service imap-login {
  process_limit = 200
}

(trash folder setting added 7 May 2019)

(full custom settings listed 5 Jan 2021)

Activate the changes:

$ sudo systmctl reload postfix
or
$ sudo service postfix reload
and
$ sudo systemctl restart dovecot
or
$ sudo service dovecot reload

Verify success

See whether the Postfix server is running:

$ telnet localhost 25

should display:

220 kctang.com.hk ESMTP Postfix (Ubuntu)

ehlo localhost

should display the following:

250-kctang.com.hk

250-PIPELINING

250-SIZE 102400000

250-VRFY

250-ETRN

250-STARTTLS

250-ENHANCEDSTATUSCODES

250-8BITMIME

250 DSN

Ctrl-]

to exit to "telnet >" prompt.

quit

to exit telnet.

Try also:

$ telnet localhost 995

similarly:

$ telnet localhost 465

should display either one:

Connected to localhost

Connected to kctang.com.hk

"Ctrl-]"

 to exit to "telnet >" prompt.

quit

to exit telnet.

Specify internal email forwarding

Edit "aliases" file:

$ sudo gedit /etc/aliases

Specify​:

postmaster: kctang
webmaster: kctang
kctcl: kctcl, kctclpop

meaning:

(webmaster added 7 May 2019)

"localhost" means "kctang.com.hk" in our case. For the email user name before "@", there is no need to create a  file system user account for it if all emails addressed to it are forwarded elsewhere. The name serves as an alias only of the email account forwarded to.

(added 17 May 2019)

 

Activate setting everytime the "aliases" file has been changed:

$ sudo newaliases

 

Copy to external email accounts

(section added, 17 May 2019)

Execute:

$ sudo gedit /etc/postfix/main.cf

Specify​ at the end of the file:

virtual_alias_domains = kctang.com.hk
virtual_alias_maps = hash:/etc/postfix/virtual

Execute:

$ sudo gedit /etc/postfix/virtual

Specify​ to make a copy to itself and a copy to the external email account:

# from               to one or more addresses, separated by a space 
kctang@kctang.com.hk kctang@kctang.com.hk name1@external.account.name
kctcl@kctang.com.hk kctcl@kctang.com.hk name2@external.account.name

Omit making a copy to itself if only email forwarding is required.

Save and exit.

Execute after the "virtual" file is created or changed:

$ sudo postmap /etc/postfix/virtual

Execute:

$ sudo systemctl restart postfix

Test by sending emails.

Install anti-virus Clamav-Daemon

(section added, 30 May 2019)

Install clamav-daemon:

$ sudo apt update
$ sudo apt install clamav-daemon            (clamav-freshclam also automatically installed)
$ sudo dpkg-reconfigure clamav-daemon   (must be done, generally accept all defaults, set yes to scan emails)
$ sudo systemctl start clamav-freshclam
$ sudo systemctl start clamav-daemon
$ sudo systemctl status clamav-freshclam    (check if running)
$ sudo systemctl status clamav-daemon       (check if running, OK if reported "/bin/mkdir /run/clamav (code=exited, status=1/FAILURE" because directory already created)
$ tail -f /var/log/clamav/clamav.log        (see running progress, Ctrl-Z to exit)

(typo corrected, 8 May 2020)

Filter spam mails

(section added, 30 May 2019)

Execute to install various software:

$ sudo apt update
$ sudo apt install amavisd-new spamassassin
$ sudo apt install postfix-policyd-spf-python                                 (optionally required for spf) 
$ sudo apt install opendkim                                                   (optionally required for opendkim)
$ sudo apt install pyzor razor                                                (optional extras)
$ sudo apt install arj cabextract cpio lhasa nomarch pax rar unrar unzip zip  ("lhasa", not "lha")

Configure ClamAV:

$ sudo adduser clamav amavis
$ sudo adduser amavis clamav

Configure SpamAssassin:

$ sudo gedit /etc/default/spamassassin

 Change "ENABLED=0" to:

ENABLED=1

(no longer set here, 8 May 2020)

Start the service:

$ sudo systemctl start spamassassin.service

Configure Amavisd-new:

$ sudo gedit /etc/amavis/conf.d/15-content_filter_mode

Specify:

use strict;

# Uncomment the two lines below to enable antivirus checking mode

@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);

# Uncomment the two lines below to enable SPAM checking mode

@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

1;  # insure a defined return

Execute:

$ sudo gedit /etc/amavis/conf.d/20-debian_defaults

Change "D_PASS" to:

$final_spam_destiny = D_DISCARD

(revised,  8 May 2020)

Adjust the following values only if desired to flag more messages as spam:

$sa_tag_level_deflt = 2.0; # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.31; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 6.31; # triggers spam evasive actions
$sa_dsn_cutoff_level = 10; # spam level beyond which a DSN is not sent

Execute:

$ sudo gedit /etc/amavis/conf.d/50-user

Specify:

$myhostname = 'kctang.com.hk';

Re-start the service:

$ sudo systemctl restart amavis.service

Edit the following file to specify domains to be whitelisted if necessary:

$ sudo gedit /etc/amavis/conf.d/40-policy_banks

Test that the Amavisd-new SMTP is listening:

telnet localhost 10024
Trying 127.0.0.1...
Connected to kctang.com.hk.
Escape character is '^]'.
220 [127.0.0.1] ESMTP amavisd-new service ready

Press Ctrl-]  and enter "quit" to exit.

Configure Postfix master.cf:

$ sudo gedit /etc/postfix/master.cf

Add the following to the end of the file:

smtp-amavis     unix    -       -       -       -       2       smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20

127.0.0.1:10025 inet    n       -       -       -       -       smtpd
    -o content_filter=
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=reject_unauth_pipelining
    -o smtpd_end_of_data_restrictions=
    -o mynetworks=127.0.0.0/8
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters

Add the following immediately below the "pickup" transport service:

     -o content_filter=
     -o receive_override_options=no_header_body_checks

Configure Postfix main.cf:

$ sudo gedit /etc/postfix/main.cf

Add the following to the end of the file:

content_filter = smtp-amavis:[127.0.0.1]:10024

Comment it with "#" at the start of the line if desired to stop using amavis.

Restart service:

$ sudo systemctl restart postfix.service

Check the hidden header of email received after the above to see the presence of one or more of the following:

X-Spam-Level: 
X-Virus-Scanned: Debian amavisd-new at kctang.com.hk
X-Spam-Status:
X-Spam-Level: 

If present, the spam filter is working.

See https://help.ubuntu.com/lts/serverguide/mail-filtering.html.en for a full explanation of the above.

Delete filtered mails

(section added, 9 Dec 2020)

Filtered files are stored in /var/lib/amavis/virusmails and should be deleted regularly.

Edit to define a scheduled job to delete filtered files:

$ sudo crontab -e -u amavis 

where:

-e = edit

-u amavis = user amavis

Define as follows:

0 0 * * * find /var/lib/amavis/virusmails/ -type f -mtime +15 -delete

which means at

0 = 0 minute 

0 = 0 hour

* = every day of month

* = every month

* = every day of week

find /var/lib/amavis/virusmails = find in that directory

-type f = regular files

-mtime +15 = file modified more than 15 days ago

-delete = delete the file

 

End of Page

Install Ftp server

Install Ftp server KCTang Tue, 07/05/2019 - 20:21

Go to End

5 Sep 2019: "0755" changed to "0775" for "Ftp" directory.

7 May 2019: "gksudo gedit" changed to "sudo gedit" as Ubuntu 18.04 dropped "gksudo".

25 Dec 2014: First created.

Note

FTP server enables directories to be accessible for downloading or uploading by users outside the local network.

Install

Install the package:

$ sudo apt-get install vsftpd

Edit config file:

$ sudo gedit /etc/vsftpd.conf

Uncomment the following line to enable uploading:

write_enable=YES

Define as the following line to change the default directory permissions to 775 (drwxrwxr-x) and default file permissions to 664 (-rw-rw-r--):

local_umask=002

Uncomment the following lines to restrict users to their home except for those listed in the file represented by "chroot_list_file":

chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd.chroot_list

("vsftpd.choot_list" corrected as "vsftpd.chroot_list",  7 May 2019)

Save file after uncommenting.

Specify users who can go outside their home by inserting their user login names one on each line in the file represented by "chroot_list_file":

$ sudo gedit /etc/vsftpd.chroot_list

("vsftpd/chroot_list" corrected as "vsftpd.chroot_list",  7 May 2019)

Restart ftp service whenever the config files are changed:

$ sudo systemctl restart vsftpd
or
$ sudo service vsftpd restart

Set the internet router to re-direct ftp connections to server port 21.

Set up a root FTP Directory to contain all FTP job folders

Change directory to the top directory assessible for use by Windows network through Samba:

$ cd /<full directory path from root>​

Make a directory specially for FTP storage, called "Ftp" in this example:

$ sudo mkdir Ftp

Change its ownership so that it can be accessed by Windows network:

$ sudo chown nobody:nogroup Ftp

Change its permissions to "read only" for other users:

$ sudo chmod 0775 Ftp

("0755" changed to "0775" because for unknown reasons sub-directory cannot be created under "Ftp", 5 Sep 2019)

Check setting:

$ ls -ls

should show "drwxr-xr-x" and "nobody nogroup" against the "Ftp" item.

Create a ftp user for specific job

Create a new user with authority to download and upload the job ftp directory:

$ sudo adduser <ftp user name>

Change the new user's root directory from /home/<ftp user name> to the job ftp directory:

​$ sudo usermod -d /<full directory path from root>/Ftp/<job name> <ftp user name>
  • ​<ftp user name> and <job name> can be the same or different
  • <job name> will become ftp users' root directory, they will be restricted to see only files at or below the root directory, they will not see the name of <job name> or the directory structure outside the root directory 
  • ​Instead of <job name>, a further sub-directory such as <job name>/<sub job name> may be defined as the root​ directory
  • ​The directory /<full directory path from root>/Ftp/<job name> will still exist but not be used for ftp

Set up a ftp directory for specific job for downloading

Create a job ftp directory under the Ftp directory:

  • using Windows Explorer:
​\\<server name>\<full folder path from server>\Ftp\<job name>
  • or at the server terminal:
$ cd /<full directory path from root>​/Ftp
$ sudo mkdir <job name>
$ sudo chown nobody:nogroup <job name>
$ ls -ls

should show "drwxr-xr-x" or "drwxrwxr-x" and "nobody nogroup" against the <job name> item.

Further sub-directories may be created similarly for downloading purposes.

Set up a ftp directory for specific job for uploading

Create an "upload" sub-directory under the job ftp directory:

  • using Windows Explorer:
​\\<server name>\<full folder path from server>\Ftp\<job name>\upload
  • or at the server terminal:
$ cd /<full directory path from root>​/Ftp/<job name>
$ sudo mkdir upload
$ sudo chown nobody:nogroup upload

Change its permissions on the server to enable "write" for all:

$ cd /<full directory path from root>/Ftp/<job name>
$ sudo chmod a+w upload

Check settings:

$ ls -ls

should show "drwxrwxrwx" and "nobody nogroup" against the "upload" item.

Upload or download

​Internally, use Windows file explorer to copy or move files between the Windows networked computers to the ftp directories:

  • ​​copy files to ​\\< server name>\<full folder path from server >\Ftp\<job name> for downloading
  • copy files from \\<server name>\<full folder path from server >\Ftp\<job name>\upload after uploading by others

Externally, inform external users the ftp user login name i.e. <ftp user name> and password for downloading or uploading.

End of Page

Install Apache2 web server

Install Apache2 web server KCTang Mon, 06/01/2020 - 18:08

Go to End

30 Jul 2020: Updated to use Python3.

6 Jan 2020: Updated.

8 May 2019: More explanation on default configuration given. "gksudo gedit" changed to "sudo gedit" as Ubuntu 18.04 dropped "gksudo".

11 Apr 2018:  "apache2" changed to "apache2.service" when used in conjunction with systemctl.

25 Dec 2014: First created.

Note

Apache2 web server provides web page services.

Prepare

Define hosts:

$ sudo gedit /etc/hosts

Specify:

127.0.0.1 kctang.com.hk <computer name> localhost
127.0.1.1 <computer name>

Define hostname:

$ sudo gedit /etc/hostname

Specify a line to contain:

<computer name>

Install

Install:

$ sudo apt install apache2

or before Ubuntu 16.04:

$ sudo apt-get install apache2

Start service:

$ sudo systemctl start apache2.service

or:

$ sudo service apache2 start

Set the internet router to re-direct http connections to server port 80.

(The following added on 8 May 2019)

Edit the enabled configuration file:

$ sudo gedit /etc/apache2/sites-enabled/000-default.conf

("ls -ls" changed to "gedit", 6 Jan 2020)

The file is symbolic linked to the actual location at /etc/apache2/sites-available/000-default.conf.

The file includes the following:

#ServerName www.example.com
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html

Leave the ServerName to the HTTPS setting below.

Change the ServerAdmin email address to the correct address, or add that address when setting up the email server.

The DocumentRoot tells that the website directories and files will be stored under /var/www/html.

Note that previously the DocumentRoot was at /var/www. The change from /var/www/html would affect the installation of Drupal as explaned on that web page.

Serf to http://www.kctang.com.hk or http://kctang.com.hk on web browser, the following page (var/www/html/index.html) will be displayed to indicate successful installation:

(end of add)

Configure to use HTTPS

This is optional. Starting to use on 6 April 2018.

When the Apache2 server is configured to use HTTPS, and when "https://" is used as the prefix to the website address URL (Uniform Resource Locator) in the web browser navigation bar, encrypted communications will be used with the Apache2 server. This will enhance security.

To do this, enable the mod_ssl module:

$ sudo a2enmod ssl

In order for Apache2 to use HTTPS service, a certificate and a key file are needed. Use EFF's Certbot to automatically deploy Let's Encrypt certificates and enable HTTPS.

Certbot is downloadable at:

https://certbot.eff.org/

No need to download from there.

(revised 8 May 2019)

Install certbot and configure Apache2:

$ sudo apt update
$ sudo apt install software-properties-common
$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt update
$ sudo apt install python3-certbot-apache
$ sudo certbot --apache
$ sudo certbot renew

(python changed to python3, 30 Jul 2020)

When answering questions

  • enter "kctang.com.hk" for name to activate HTTPS
  • decide whether to re-install certificate or renew and replace
  • select to redirect HTTP traffic to HTTPS.

(added on 8 May 2019).

The following will happen:

  • /etc/letsencrypt directory created to contain certificate obtained from Let's Encrypt.
  • A file /etc/apache2/sites-available/000-default-le-ssl.conf added and enabled.

(revised 8 May 2019)

  • The following lines inserted in /etc/apache2/sites-available/000-default.conf to enforce the use of "https://":
RewriteEngine on
RewriteCond %{SERVER_NAME} =kctang.com.hk
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
  • A cron job created to renew the certificate which lasts for 90 days before expiry.

Enable new module and disable default module if not already automatically done:

$ sudo a2ensite 000-default-le-ssl
$ sudo a2dissite default-ssl
$ sudo systemctl restart apache2.service

 

End of Page

Install MySQL server + PHP + phpMyAdmin

Install MySQL server + PHP + phpMyAdmin KCTang Mon, 06/01/2020 - 17:51

Go to End

6 Jan 2020: Updated.

8 May 2019: References to PHP5 deleted. Secure installation added. Root user authentication added. automysqlbackup added.

25 Dec 2014: First created.

Note

MySQL is a database server.

PHP (PHP5 before Ubuntu 16.04) is a web page programming language.

phpMyAdmin is a web interface to administer the MySQL server.

Install MySQL-server

Execute:

$ sudo apt update
$ sudo apt install mysql-server
$ sudo systemctl restart mysql.service
$ sudo systemctl status mysql.service

Config MySQL-server

(section added, 8 May 2019)

Execute:

$ sudo mysql_secure_installation

When answering the secure installation questions:

  • set password strength - high
  • set root password 
  • remove anonymous users - yes
  • allow or disallow root login remotely - no
  • remove test database - yes
  • reload privilege table now - yes
  • set root password - not asked if already done 

(suggested answers added, 6 Jan 2020)

Adjust authentication for use with phpMyAdmin

(section added, 8 May 2019)

If not done, the phpMyAdmin menu would not provide a choice to add users.

Execute:

$ sudo mysql -u root -p

enter the root user's password when prompted.

Execute:

mysql> SELECT user,plugin FROM mysql.user;

It will show that the root user's authentication plugin is "auth_socket".

Execute:

mysql> ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY '<root user password>';
mysql> FLUSH PRIVILEGES;
mysql> SELECT user,plugin FROM mysql.user;

(semi colon added after "PRIVILEGES", 6 Jan 2020)

It should show that the root user's authentication plugin has been changed to "mysql_nature_password".

Install PHP and related apache2 module

Execute:

$ sudo apt install php libapache2-mod-php

Install phpMyAdmin

Execute:

$ sudo apt install phpmyadmin

Set its own password.

Choose whether to keep the existing database whenever phpmyadmin is removed and re-installed. 

Enable Apache2 config:

$ sudo ln -s /etc/phpmyadmin/apache.conf /etc/apache2/conf-available/phpmyadmin.conf
$ sudo a2enconf phpmyadmin.conf

Whenever Apache2 has been purged and re-installed, this enabling must be done again.

Restart Apache2 service:

Execute:

$ sudo systemctl restart apache2.service

Create a database user and a database

Execute:

 http://localhost/phpmyadmin
  or
 http://www.kctang.com.hk/phpmyadmin

Enter login name and password.

Create a user called <name, e.g. Drupal8> together with a database also called <name>:

  • click "Users" at the top menu bar
  • click "Add user" at the page middle
  • enter <name> at the User name entry
  • select "Local" at the Host entry
  • enter and re-type the password
  • click "Create database with same name and grant all privileges"
  • click "Go" at the bottom

Install automysqlbackup

(section added, 8 May 2019)

The software will backup mySQL databases daily, weekly and monthly.

Execute to install:

$ sudo apt install automysqlbackup

Edit the configuration file:

$ sudo gedit /etc/default/automysqlbackup

Specify:

BACKUPDIR="/var/lib/automysqlbackup" as the backup directory

MAILCONTENT="log" to send log email

MAILADDR="root" to send the email to root user, which has been set under Postfix Aliases to re-direct to the appropriate user.

Execute to run for the first time:

$ sudo automysqlbackup

Inspect daily backups:

$ sudo ls /var/lib/automysqlbackup/daily

 

End of Page

Install Drupal 9 content management system

Install Drupal 9 content management system KCTang Sun, 20/09/2020 - 02:01

Go to End

20 Sep 2020: Upgrade to Drupal 9

5 Jan 2020:  Problems when updating to Drupal 8.8 under php7.4 described. Some text updating.

9 Sep 2019: Composer files also copied when relocating system.

18 July 2019: Composer configuration actions added.

7 July 2019: Original user-defined files and attributes kept when updating manually.

26 May 2019: Composer and Drush installations added. Manual core installation added. Deleting configuration file added.

16 May 2019: Simplified installation step added. Using new shell to restore added.

8 May 2019: Page on Drupal 8 withdrawn.  All "/var/www/web" changed to "/var/www/html/web" because Apache2 prefers to put website files underneath "/www/web/html", and it is easier that way. Relocating Drupal 8 added. Print page stylesheet re-written.

18 Jan 2019: Updating Drupal 8 core using composer added.

30 Sep 2018: Setting CKEditor to use Mayo theme stylesheet added. Setting print page to also use Mayo theme stylesheet added. Adding custom styles added.

23 Sep 2018: "php/7.0" changed to "php/7.2"

3 Sep 2018: First created from the page on Drupal 7 after major upgrading to Drupal 8. Resolution of CKEditor table border added.

Note

Drupal is a web site content management system serving web pages to the internet.

The latest version is Drupal 9. The main text below is about installing Drupal 8 with supplementary notes on upgrading to Drupal 9. The procedures described have not been proven with a fresh installation of Drupal 9 but may be quite similar.

(paragraph added, 20 Sep 2020)

Prepare

Install Apache2 web server if not already installed.

Enable Apache2 rewrite module (should have been automatically installed when installing HTTPS):

$ sudo a2enmod rewrite

Configure Apache2:

$ sudo gedit /etc/apache2/apache2.conf

Specify, keeping "<" and ">":

<Directory /var/www/>
    Options Indexes FollowSymLinks
    # AllowOverride None # replaced with next line
    AllowOverride All
    Require all granted
</Directory>

Restart Apache2 service:

$ sudo systemctl restart apache2
or
$ sudo service apache2 restart

Install MySQL server + PHP5 + phpMyAdmin, and create a user with database both called "drupal8".

Co-exist with Drupal 7

Change "web" below to "web2" to create another set of directories to co-exist with Drupal 7 currently using "web", otherwise, just use "web".

Install Drupal 8

Change "8.x.x" to "9.x.x" in case of Drupal 9.

(paragraph added, 20 Sep 2020)

Download the latest Drupal 8.x.x (filename drupal-8.x.x.tar.gz) from https://www.drupal.org/project/drupal, usually to own "Downloads" directory.

Use the file manager to extract the contents of the compressed file under the "Downloads" directory as "drupal-8.x.x".

Create a directory for Drupal but using a directory name of "web":

$ sudo mkdir /var/www/html/web
$ cd /var/www/html/web
$ sudo mv /home/<own account name>/Downloads/drupal-8.x.x/* .
$ sudo ls -la /home/<own account name>/Downloads/drupal-8.x.x
$ sudo mv /home/<own account name>/Downloads/drupal-8.x.x/.* .
(move hidden files, which may give an error message)
$ sudo ls -la                                                
(check moved files as intended)
$ sudo ls -la /home/<own account name>/Downloads/drupal-8.x.x
(should be empty)

Do it in one step instead of the above:

$ cd /var/www/html
$ sudo wget https://ftp.drupal.org/files/projects/drupal-8.x.x.tar.gz  
(change "8.x.x" to the version number)
$ tar -xzf drupal-8.x.x.tar.gz 
("sudo" not used so that user instead of "root" will become owner) 
$ mv drupal-8.x.x web

(simplified step added, 15 May 2019)

(revised, 26 May 2019)

("sudo" not used, 7 July 2019)

Create a location for site files:

$ cd /var/www/html/web
$ sudo mkdir sites/default/files

(change directory added, 5 January 2020)

The latest recommendation is to install Drupal using Composer. See https://www.drupal.org/docs/develop/using-composer/starting-a-site-using-drupal-composer-project-templates.

There are two templates:

  • drupal/recommended-project which is recommended by Drupal. This template requires an additional level of directory, such as /var/www/html/drupalroot/web instead of /var/www/html/web. 
  • drupal/legacy-project which uses the same directory structure as the downloadable compressed file. Since the existing installation is based on the downloadable compressed file, this structure is continued to be used, and Composer is only used for updating as described later.

(two paragraphs added, 20 Sep 2020)

Change ownership:

$ sudo chown www-data:www-data sites/default/files

Create the initial configuration file for the default site:

$ sudo cp sites/default/default.settings.php sites/default/settings.php

Change ownership:

$ sudo chown www-data:www-data sites/default/settings.php

Restart Apache2 service:

$ sudo systemctl restart apache2
or
$ sudo service apache2 restart

Complete the Drupal Installation through a Browser by pointing to http://localhost/web, and follow the instructions there, using:

(not .../web/install.php, 8 May 2019)

  • Choose language: English language
  • Choose profile: Standard
  • Setup database: MySQL
  • MySQL database name, user name and password as defined above
  • Site name: www.kctang.com.hk
  • Site e-mail address: <>@kctang.com.hk
  • Site maintenance username: <name of the administrator>
  • Site maintenance user e-mail address: <>@kctang.com.hk
  • Default country: Hong Kong S.A.R., China
  • Default time zone: Asia/Hong Kong
  • Check for updates automatically
  • Receive e-mail notifications

Change directory permissions:

$ sudo chmod 555 sites/default
$ sudo chmod 444 sites/default/settings.php

Permit uploading of files (similar change for modules and themes not required, they use ftp upload)

$ sudo chown www-data:www-data -R sites/default/files

Increase upload file size limit:

$ sudo gedit /etc/php/7.x/apache2/php.ini
(change "7.x" to the current version number)

("7.2" changed to "7.x" which is "7.4" as of now, 5 Januaray 2020)

("7.0" changed to "7.2", 23 Sep 2018)

Change existing to:

post_max_size = 200M
upload_max_filesize = 50M
max_file_uploads = 100

Restart Apache2 service:

$ sudo systemctl restart apache2
or
$ sudo service apache2 restart

Edit "settings.php" file:

$ sudo gedit /var/www/web/sites/default/settings.php

specify trusted hosts:

$settings['trusted_host_patterns'] = array(
   '^www\.kctang\.com\.hk$',
   '^kctang\.com\.hk$',
   '^localhost$',
);

Disable "update.php" file:

$ sudo mv /var/www/web/update.php /var/www/web/<some new name>

Migrate from Drupal 7 (as of 21 February 2018)

Log in as an administrator if not already in.

Choose "Manage" > "Extend" > "Book" module under Core > "Install" at the bottom, to allow users to create and organize related content in an outline.

Choose "Manage" > "Extend" > "Statistics" module under Core > "Install" at the bottom, to log content statistics.

Similarly, enable modules in Drupal8 that are enabled in Drupal7.

Click open the explanatory note of "Statistics". Select "Configure" > "Count content views" > "Save configuration" > "Continue".

Choose "Manage" > "Extend" > "Migrate", "Migrate Drupal" and "Migrate Drupal UI" modules under Core (Experimental) > "Install" at the bottom, to allow users to create and organize related content in an outline.

Click open the explanatory note of "Migrate Drupal UI". Select "Configure" > "Continue" > "maintenance mode" > "Put site into maintenance mode" > "Save configuration".

Move web page back to the Upgrade page. Choose "Continue".

Under SOURCE DATABASE, enter: 

  • Database host: localhost
  • Database name: drupal7
  • Database username: drupal7
  • Database password: <>

Under SOURCE FILES, assuming under "oldweb" enter:

  • /var/www/oldweb

Resolve the following problems found:

  • Some uploaded files have not been migrated. Execute "sudo cp /var/www/<oldweb>/sites/default/files/* /var/www/html/web/sites/default/files" to copy the files over (not the sub-directories).
  • The Main menu used in Drupal 7 has been migrated with a machine name called "main-menu" but the machine name used in Drupal 8 is called "main". This is incompatible. Although the menu structure is the same, the contents cannot be displayed. There is also an error message saying that the main-menu index has a problem. Add another menu under "Manage > Structure > Menus". Still at there, select "Edit menu" against the migrated Main menu to edit the individual menu items. Change the "Parent link" from the migrated Main menu to the newly added menu so as to empty the migrated Main menu. This may help remove the indexing problem. The contents can be displayed using the newly added menu. If things go fine and it is preferred to use the true Main menu, move the items back from the newly added menu to the Main menu, which should now use the correct machine name. The foregoing is based on memory. During the problem solving endeavour, mySQL has been accessed to delete references to "main-menu", but it is not sure whether this is really essential.
  • The Book menu has not been migrated and will need build-up from scratch.
  • The previously used theme has not been migrated. Download and install the desired theme under "Manage > Appearance". Set up with reference to the original setting.
  • Many modules have not been migrated. Download and install them from scratch under "Manage > Extend". Not all modules are available for Drupal 8.
  • Some modules (e.g. CKEditor Color Button) require the creation of a "libraries" directory under the Drupal root directory.
  • Some modules require the downloading of files at the server (i.e. not the web) and putting the unzipped file directory under the "libraries" and "vendor" directories. Some even require a sub-directory under "libraries", e.g. "CKEditor Find" module requires "... libraries/ckeditor/plugins".

Re-direct http path to sub-directory "web"

The default DocumentRoot  in the default enabled Apache2 configuration file /etc/apache2/sites-enabled/000-default.conf  is "/var/www/html/".

Web browers can access any permissible directories or files underneath the DocumentRoot. Access outside the Document Root will not be possible.

When the Drupal root directory /web is placed underneath /var/www/html such as /var/www/html/web, then http://kctang.com.hk will access the default index.html file underneath /var/www/html, but in the case of http://kctang.com.hk/web, access to those underneath /web is possible.

To re-direct http://kctang.com.hk to access /var/www/html/web directly, create a new ".htaccess" file under "/var/www/html":

$ sudo gedit /var/www/html/.htaccess

Specify:

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteBase /
  RewriteRule ^(.*)$ web/$1 [R]
</IfModule>

"^(.*)$" represents the full text from start to end after the domain name, e.g. "abc/def" in "kctang.com.hk/abc/def".

"web/$1" represents the substitution text where "$1" represents the text represented by "(.*)" with "web/" inserted before it, i.e. "web/abc/def" based on the above example. This would redirect the path to "web/abc/def".

"[R]" is to redirect the path and will show "web/" as part of the redirected path. When using "[R]", "RewriteBase /" is required to add "/" before "web/$1". Without this, "/var/www/html" will be added before "web/$1" and will cause unexpected results. This effect was discovered after many days of error discovery using "[L]".

Using [L] as suggested by many people may hide "web/" from the displayed path, and does not require the use of "RewriteBase", but some of the web pages will still show "web/" unavoidably. A mixed use with or without "web/" displayed will cause denial of access rights or redirection to external URL in some cases. Therefore, it is better to force to display "web/" using "[R]".

The following has the same effect:

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteBase /web
  RewriteRule ^(.*)$ $1 [R]
</IfModule>

Remove "www." prefix from URL

This is not essential for using Drupal, but is adopted only to simplify.

Insert the following in /var/www/html/.htaccess after the RewriteBase line:

# Remove "www." prefix from URL
RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
RewriteRule ^ http%{ENV:protossl}://%1%{REQUEST_URI} [L,R=301]

Permit redirection to external URL

This section would not be required if the path redirection is properly defined. This appears to be not required because the core has been replaced after this section was written, with the two lines unchanged but without problem caused.

Redirecting between a path with or without "web/" will be treated as a redirection to external URL. Drupal 8 does not allow redirection to external URL by default.

To allow:

$ sudo gedit /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/RedirectResponseSubscriber.php

change lines 7 and 74 from:

use Drupal\Core\Routing\LocalRedirectResponse;

$safe_response = LocalRedirectResponse::createFromRedirectResponse($response);

to:

use Drupal\Core\Routing\TrustedRedirectResponse;

$safe_response = TrustedRedirectResponse::createFromRedirectResponse($response);

Handle CKEditor module

CKEditor is a WYSIWYG editor for Drupal.

For Drupal 7, CKEditor is a single optional module. The module is pre-built to contain the desired plugins under the CKEditor web site before being installed under Drupal.

For Drupal 8, CKEditor is core module but with trimmed down features. To add back those features, individual add-on modules from the Drupal website and their corresponding CKEditor plugin files from CKEditor web site have to be installed. Not all previous features have CKEditor Drupal 8 add-on modules available.

The downloaded plugin file directories need to be unzipped under the "libraries" directory . Some even require more than one sub-directory under "libraries", e.g. "CKEditor Find" module requires "... libraries/ckeditor/plugins".

Install the CKEditor "libraries" module (not the same as the libraries directory mentioned above) because some CKEditor modules require this to work.

Not all CKEditor Drupal 8 add-on modules could be installed successfully. The usual unsuccessful phenomenon is that, after installation of the modules and the related plugin directories, the icons newly made available for the editing menu are blank or do not contain the full text. If they are added to the CKEditor editing menu, the menu cannot be displayed when a page is clicked open for editing. After a lot of trial and errors, it has now been found that the cause of the problem sis actually very simple. when the corresponding plugin directories are downloaded and added to the libraries directory, only the owner is permitted to read and write. By permitting the group and others to read, the problems are solved. The command is: (revised 3 Sep 2018) 

$ sudo chmod 755 /var/www/html/web/libraries/<plugin directory name>

Resolve CKEditor table border conflicting Mayo theme

(section added, 3 Sep 2018)

When the Mayo theme is used for web page appearance, a table created with CKEditor and seen while editing can only have the outer border lines displayed on the published page. The inner border lines disappear. To display the inner border lines, execute:

$ sudo gedit /var/www/html/web/themes/mayo/css/style.css

Uncomment the following border-style and border-width lines so as not to interfere with those set by CKEditor:

table tr td {
   padding: 4px 6px;
#  border-style: solid;
#  border-width: 0px;
}

table tr th {
#  border-style: solid;
   padding: 4px 6px;
#  border-width: 0px;
   border-right-width: 0px;
}

When creating or modifying a table with CKEditor, set the border size to 1, 2, ... to see the border lines. Set the border size to 0 to hide the border lines.

Set CKEditor to use Mayo theme stylesheet

(section added, 30 Sep 2018)

Edit Mayo theme's info setting:

$ sudo gedit /var/www/html/web/themes/mayo/mayo.info.yml

Insert a new line "- css/style.css":

ckeditor_stylesheets:
  - css/ckeditor-iframe.css
  - css/style.css

Save and exit.

Set print page to also use Mayo theme stylesheet

(section added, 30 Sep 2018)

(re-written to use symbolic link, 8 May 2019)

Execute to find the text "misc/print.css" which is the name of the stylesheet for the default printer-friendly printing:

$ cd /var/www/html/web
$ grep -R misc/print.css

This will show a number of directories containing files named "book-export-html.html.twig" containing:

<link type="text/css" rel="stylesheet" href="misc/print.css" />

The relevant directory is  /var/www/html/web/core/themes/stable/templates/layout.

"misc/print.css" should be corrected to "core/misc/print.css". However, even if this file is absent, it only means that the stylesheet would not apply. The page can still be displayed though without the desired style. To use Mayo theme stylesheet, the name should be changed.

However, the files underneath the "/core" directory may be changed when there is an update of the core. This would cancel the change to Mayo.

Instead, create a directory for "misc/print.css" outside the "/core" directory so as not to be affected.

Execute to create 

$ mkdir /var/www/html/web/misc
$ sudo ln -s /var/www/html/web/themes/mayo/css/style.css  misc/print.css

This will create a symbolic link to the Mayo theme stylesheet which will then apply.

Add custom styles

(section added, 30 Sep 2018)

Edit Mayo theme's css setting:

$ sudo gedit /var/www/html/web/themes/mayo/css/style.css

Add the following at the end of the file:

/* the following will provide a margin to CKEditor's editing screen to make it not so tight against the left */
body {
  margin: 10px;
}

/* the following will provide a border around text defined as "Formatted" using CKEditor's "Formatted" icon */
pre {
  border: 1px solid green;
  margin: 10px;
  background-color: #f8f9fa;
  padding: 0.5em;
}

/* the following will format the table of contents inserted using CKEditor's Table of Contents icon */
.widget-toc{
  display: table;
  border: 1px solid green;
  background-color: #f8f9fa;
  padding: 1em;
  font-size: 90%;
}

/* all the following will be needed to set up CKEditor's Style icon before use */

/* the following when used in conjunction with "p" for paragraph will give paragraph hanging indentation */
.hangtwice {
  margin-left: 80px;
  text-indent: -80px;
}
.hang1 {
  margin-left: 40px;
  text-indent: -40px;
}
.hang2 {
  margin-left: 80px;
  text-indent: -40px;
}
.hang3 {
  margin-left: 120px;
  text-indent: -40px;
}
.hang4 {
  margin-left: 160px;
  text-indent: -40px;
}
.hang5 {
  margin-left: 200px;
  text-indent: -40px;
}

/* the following when used in conjunction with "p" for paragraph will give paragraph indentation */ 
.indent1 { 
  margin-left: 40px;
}
.indent2 {
  margin-left: 80px;
}
.indent3 {
  margin-left: 120px;
}
.indent4 {
  margin-left: 160px;
}

/* the following will give hanging indentation to headings */
h2.hang {
  margin-left: 40px;
  text-indent: -40px;
}
h3.hang {
  margin-left: 40px;
  text-indent: -40px;
}
h4.hang {
  margin-left: 40px;
  text-indent: -40px;
}

Save and exit.

After re-defining "pre" as above, CKEditor's Formatted icon will give the following dropdown menu:

Log in the website as an administrator.

Select Manage > Configuration > Text formats and editors > Configure Full HTML.

Move the Styles Icon down to a suitable position in the menu bar.

A blank dropdown text box will appear below the menu bar.

Enter a list of classes and titles as follows:

p.hangtwice|Hang Twice
p.hang1|Hang 1 
p.hang2|Hang 2
p.hang3|Hang 3
p.hang4|Hang 4
p.hang5|Hang 5
p.indent1|Indent 1
p.indent2|Indent 2
p.indent3|Indent 3
p.indent4|Indent 4
h2.hang|Head2Hang
h3.hang|Head3Hang
h4.hang|Head4Hang

"p" stands for paragraph.

".hangtwice" stands for the name of class as defined in the "css/style.css" file.

"Hang Twice" stands for the title seen when the Styles button is pressed to give a drop down menu.

 

Install Composer and Drush

(section added, 26 May 2019)

Drupal.org website describes various methods of installing Composer and Drush, generally referring to other websites for complicated steps which are difficult to follow.

The following simple steps do work.

Execute:

$ sudo apt install composer
$ cd /var/www/html/web
$ composer require drush/drush

Drush is installed under /var/www/html/web/vendor/drush/drush.

Update Drupal 8 using Composer

(section added, 18 Jan 2019)

Go to the root directory of the website:

$ cd /var/www/html/web

Configure "composer.json", if not done before: 

$ gedit composer.json

Move the following from the "replace" section to the beginning of the "require" section, with "," added at the end:

"drupal/core": "^8.7",

(configuration action added, 7 July 2019)

Update with the existing "core/composer.json":

$ cd core
$ composer update --with-dependencies
$ cd ..

Do similar update in case the following message is encountered when updating the "drupal/core" as described later below:

Dependency "..." is also a root requirement, but is not explicitly whitelisted. Ignoring.

(configuration action added, 18 July 2019)

(no longer necessary, 5 January 2020)

Check for outdated modules:

$ composer outdated drupal/*
$ composer show --outdated drupal/* -vvv

(syntax changed, 5 January 2020)

"-vvv" is to display what is going on.

("-vvv" added to all commands, 5 January 2020)

Update all:

$ composer update drupal/core --with-dependencies
$ composer update drupal ---vvv

(simplified command appears also working, 5 January 2020)

Update database:

$ vendor/drush/drush/drush updatedb -vvv

 

("vendor/drush/drush/" added, 26 May 2019)

Clear cache:

$ vendor/drush/drush/drush cr -vvv

Log out the server.

Upgrade Drupal 8 to Drupal 9

(section added, 20 Sep 2020)

Not all modules and themes for Drupal 8 are compatible with Drupal 9. They should be checked for compatibility.

Go to the root directory of the website:

$ cd /var/www/html/web

Install the following extensions:

$ composer require drupal/upgrade_rector drupal/upgrade_status

Go to Admin > Manage > Report > Upgrade Status.

Select projects (modules and themes) and run to check compatibility.

Apply patches as suggested.

If all the modules and themes have been made compatible, do upgrading.

Upgrading procedures have been suggested at https://www.drupal.org/docs/upgrading-drupal/upgrading-from-drupal-8-to-drupal-9-or-higher. However, the following suggested commands did not work because of dependencies errors caused by incompatibility:

$ cd /var/www/html/web
$ composer require drupal/core-recommended:^9.0.6 drupal/core-composer-scaffold:^9.0.6 drupal/core-project-message:^9.0.6 --no-update
$ composer update

The following command did work:

$ cd /var/www/html/web
$ composer require "drupal/core:9.0.6 as 8.9.6" --no-update && composer update

"9.0.6" and "8.9.6" are the current Drupal 9 version and installed Drupal 8 version respectively at the time of writing this.

Update database:

$ vendor/drush/drush/drush updatedb -vvv

Answer "yes" when asked to run specified post-update changes.

Error has been reported for the Mayo theme because the base theme has to be stated for Drupal 9.

Edit:

$ sudo nano themes/mayo/mayo.info.yml

Insert "base theme: stable" after the third line for "description".
Clear cache:

$ vendor/drush/drush/drush cr -vvv

Log out the server.

The Admin > Manage > Report > Upgrade Status page may still report incompatibility but this should not matter anymore.

Update Drupal 9 using Composer

(section added, 20 Sep 2020)

Same as those for Drupal 8:

$ cd /var/www/html/web
$ composer update drupal ---vvv
$ vendor/drush/drush/drush updatedb -vvv
$ vendor/drush/drush/drush cr -vvv

Log out the server.

Update Drupal 8 core manually

(updated, 5 January, 2020)

(section added, 26 May 2019)

Set the website to maintenance mode.

Backup the existing directories and files to another location, e.g. .../webold:

$ cd /var/www/html
$ sudo mv web webold

Download the latest Drupal version, and extract the files as described for fresh installation, i.e.:

$ sudo wget https://ftp.drupal.org/files/projects/drupal-8.x.x.tar.gz 
(change "8.x.x" to the version number)
$ tar -xzf drupal-8.x.x.tar.gz         
("sudo" not used so that user instead of "root" will become owner)
$ mv drupal-8.x.x web

Change to website root directory, remove two sub-directories and all files in the top level directory, including hidden files:

$ cd /var/www/html/web/
$ rm -rf core vendor
$ rm -f *.* .[a-z]*

Change to downloaded directory, copy updated sub-directories and files to the existing website directories:

$ cd ../drupal-8.x.x
$ cp -R core vendor ../web
$ cp *.* .[a-z]* ../web

Update with Composer and Drush

$ composer update -vvv
$ composer require drush/drush
$ vendor/drush/drush/drush updatedb -vvv
$ vendor/drush/drush/drush cr -vvv

Browse the website to see that the webpages can show up successfully.

Set to release the website from maintenance mode.

In case of a need to re-use and copy the some of the old user-defined directories and files to the new directories (keeping original attributes), do some of the following as appropriate:

$ cd /var/www/html
$ sudo cp -Rav webold/libraries web
$ sudo cp -Rav webold/modules web
$ sudo cp -Rav webold/profiles web
$ sudo cp -Rav webold/sites web
$ sudo cp -Rav webold/themes web
$ sudo cp -Rav webold/vendor web   
$ sudo cp -Rav webold/composer.json web   
$ sudo cp -Rav webold/composer.lock web 
$ sudo cp -Rav webold/misc web            (this one is not a standard directory)

 

Relocate or restore Drupal 8

(section added, 8 May 2019)

(revised, 16 May 2019)

Export the relevant MySQL database using phpMyAdmin as a backup.

Backup the whole Drupal directories and files. For example, if Drupal's root directory is /web in /var/www/html/web, then everything from and below /web.

Use old pair of backups if fresh pair not available.

Create a new user and blank database only if new computer or new MySQL-server is used. 

Import the database backup.

Move the whole Drupal directories and files to the new directory, e.g. /var/www/new/web.

Keep "new" as the original name in case of no change, e.g. /var/www/html/web.

Keep the name "/web" unchanged since this is hard-coded in the beginning of links  in the form of "/web/..." to media and files embedded in the web pages.

Execute

$ sudo gedit /var/www/new/web/sites/default/settings.php

For

$database['default']['default'] = array (
    'database' = '<text>'
    'username' = '<text>'
    'password' = '<text>'
    ....
);

change the <text> to match the new database.

Change the DocumentRoot in the new site's currently used Apache2 site configuration file, e.g. /etc/apache2/sites-enabled/000-default.conf,  from "/var/www/html/" to /var/www/new, still one level above /web. Links in the form of "/web/..." to media and files will be interpreted as "/var/www/new/web/...", still accessible.

Web browsing to http://kctang.com.hk/web should be able to access Drupal at the new location.

Go to the next section in case web browsing fails to display.

If the DocumentRoot is set directly to the Drupal root directory "/var/www/new/web, web browsing of Drupal web pages is still possible, but links to media and files will be interpreted as "/var/www/new/web/web/... ", and the media and files will not be displayed.

Move the ".htaccess" file from /var/www/html to /var/www/new to re-direct web browsing to http://kctang.com.hk/web:

$ sudo mv /var/www/html/.htaccess /var/www/new

Execute as necessary:

$ vendor/drush/drush/drush updatedb -vvv
$ vendor/drush/drush/drush cr -vvv

("vendor/drush/drush/" added, 26 May 2019)

Use new shell to relocate and restore

(section added, 16 May 2019)

(revised, 26 May 2019)

If the relocation or restoration is not successful, non-display may happen because of some corruptions in the file settings.

Follow the Drupal core manual installation steps described above, but substitute "/var/www/html" described there with "/var/www/new".

Setting the website to maintenance mode would not be possible.

Before changing the settings file, move the ".htaccess" file from /var/www/html to /var/www/new:

$ sudo mv /var/www/html/.htaccess /var/www/new

Delete left-over module configuration files

(section added, 26 May 2019)

If a left-over configuration setting file is reported when a module is re-installed, execute to delete it:

$ cd /var/www/html/web
$ vendor/drush/drush/drush config:delete '<name of configuration file to delete>'

 

End of Page

 

Install MoinMoin wiki engine

Install MoinMoin wiki engine KCTang Thu, 25/12/2014 - 18:25

Go to End

Note

MoinMoin wiki engine serves wiki web pages.

The following installation instructions are outdated. Check https://help.ubuntu.com/lts/serverguide/moinmoin.html for updated instructions.

Prepare

Install Apache2 web server if not already installed.

Download MoinMoin (filename moin-1.9.7.tar.gz) from http://moinmo.in/MoinMoinDownload, usually to own "Downloads" directory.

Use the file manager to extract the contents of the compressed file under the "Downloads" directory as "moin-1.9.7".

Install

Install MoinMoin:

$ cd /home/< own account name >/Downloads/moin-1.9.7
$ sudo python setup.py install --force --prefix /usr/local --record=install.log

Install "wsgi" module and test:

$ sudo apt-get install libapache2-mod-wsgi
$ cd /usr/local/share/moin/server 
$ sudo python test.wsgi

Point the web browser to the address shown on the terminal, such as, http://localhost:8000/.

Setup wiki sites

Setup files for a wiki called "qswiki":

$ cd /usr/local/share/moin
$ sudo mkdir qswiki
$ sudo cp -R data qswiki
$ sudo cp -R underlay qswiki
$ sudo cp server/moin.wsgi qswiki
$ sudo cp config/wikiconfig.py qswiki

Do similar for "dscwiki".

Configure Apache2 for MoinMoin site:

$ sudo gedit /etc/apache2/sites-available/moinmoin.conf

Specify, keeping "<" and ">" in the following codes:

# MoinMoin WSGI configuration
# invoke moin wiki at the root url, like http://servername/mywiki/FrontPage:
# servername can be www.kctang.com.hk or localhost
# qswiki and dscwiki use FrontScreen instead of FrontPage to avoid changes from being overridden when a new FrontPage is installed with new software
WSGIScriptAlias /qswiki   /usr/local/share/moin/qswiki/moin.wsgi
WSGIScriptAlias /dscwiki   /usr/local/share/moin/dscwiki/moin.wsgi
<Directory /usr/local/share/moin/qswiki>
    AllowOverride None
    Require all granted
</Directory>
<Directory /usr/local/share/moin/dscwiki>
     AllowOverride None
     Require all granted
</Directory>
# create some wsgi daemons - use these parameters for a simple setup
WSGIDaemonProcess moin user=www-data group=www-data processes=5 threads=10 maximum-requests=1000 umask=0007
# use the daemons we defined above to process requests!
WSGIProcessGroup moin

Enable the site:

$ sudo a2ensite moinmoin

which will create a linked file in "/etc/apache2/sites-enabled"

If required to disable:

$ sudo a2dissite moinmoin

Configure wsgi:

$ sudo gedit /usr/local/share/moin/qswiki/moin.wsgi

Add at the end of the a2) paragraph:

sys.path.insert(0, '/usr/local/share/moin/qswiki')

Do the same for "dscwiki".

Enable web access, specify access rights and restart Apache2 service:

$ cd /usr/local/share 
$ sudo chown -R www-data:www-data moin 
$ sudo chmod -R ug+rwX moin 
$ sudo chmod -R o-rwx moin
$ sudo systemctl restart apache2
or
$ sudo service apache2 restart

Configure wiki:

$ sudo gedit /usr/local/share/moin/qswiki/wikiconfig.py

Specify:

# -*- coding: utf-8 -*-
## -*- coding: iso-8859-1 -*-

#url_prefix_static = '/mywiki' + url_prefix_static
url_prefix_static = '/qswiki' + url_prefix_static

# sitename = u'Untitled Wiki'
sitename = u'QS Wiki'

#page_front_page = u"FrontPage"
page_front_page = u"FrontScreen"

# superuser = [u"YourName", ]
#    (note: remove "#" at the front, and replace "YourName" with user id within quotation marks)

#acl_rights_before = u"YourName:read,write,delete,revert,admin"
#    (note: remove "#" at the front, and replace "YourName" with user id within quotation marks)

# the following added at the end of the Security section

textchas = {
    'en': {
        u"Please enter antispam password (email to kctang@kctang.com.hk for password):": ur"[password]",
        },
    }
textchas_disabled_group = u"TrustedGroup"

#    (note: replace [password] with actual password)

# end of addition

#mail_smarthost = ""
mail_smarthost = "kctang.com.hk"

# The return address, e.g u"Jgen Wiki <noreply@mywiki.org>" [Unicode]
#mail_from = u""
mail_from = u"K C Tang <kctang@kctang.com.hk>"

# "user pwd" if you need to use SMTP AUTH
#mail_login = ""
mail_login = u"kctang [email password]"

navi_bar = [
    # If you want to show your page_front_page here:
    #u'%(page_front_page)s',
    u'RecentChanges',
    u'FindPage',
    u'HelpContents',
    u'FrontScreen',
]

# The default theme anonymous or new users get
# theme_default = 'modernized'
theme_default = 'modernized_mobile'

Note MoinMoin Config files are:

  • /etc/apache2/sites-available/moinmoin.conf
  • /etc/apache2/sites-enabled/moinmoin.conf
  • /usr/local/share/moin/qswiki/moin.wsgi
  • /usr/local/share/moin/qswiki/wikiconfig.py
  • /usr/local/share/moin/dscwiki/moin.wsgi
  • /usr/local/share/moin/dscwiki/wikiconfig.py

End of Page

Install Roundcube webmail client

Install Roundcube webmail client KCTang Wed, 08/05/2019 - 19:36

Go to End

27 May 2019: Changed to use extracting directly to web directory. Protection measures added.

8 May 2019: All "/var/www/web" changed to "/var/www/html/web". "<version number>" used instead of a specific number. More descriptions on settings added.

25 Dec 2014: First created.

Install new by downloading

The usual software installation command "$ sudo apt install roundcube" does not appear to work.

Download roundcubemail-<version number>-complete.tar.gz by selecting "Complete: <version number>" at http://roundcube.net/download/, usually to own "Downloads" directory.

Extract the compressed file, move the extracted directory and files to underneath "/var/www/html", and rename the directory as "roundcube".

Ensure that the hidden ".htaccess" file is also moved.

A quicker way is to extract directly to "/var/www/html" and rename the directory there.

Install Roundcube under "/var/www/html":

$ cd /var/www/html
$ sudo tar -xzfv /home/<own account name>/Downloads/roundcubemail-<version number>-complete.tar.gz  (extract to current directory)
$ ls      (see the existence of the extracted directory)​
$ sudo mv roundcubemail-<version number> roundcube    (change direcotry name)
$ sudo chown www-data:www-data logs
$ sudo chown www-data:www-data temp

"-xzfv" can be remembered as extract zipped file verbose.

(changed to use "tar" directly to web directory,  27 May 2019)

Install MySQL database + PHP + phpMyAdmin, if not already installed.

Login phpmyadmin at web browser:

http://localhost/phpmyadmin

Create a database user "roundcube" with database "roundcube":

  • click "Users" at the top menu bar
  • click "Add user" at the page middle
  • enter "roundcube" at the User name entry
  • select "Local" at the Host entry
  • enter and re-type the password
  • click "Create database with same name and grant all privileges"
  • click "Go" at the bottom

Enter at web browser:

http://localhost/roundcube/installer

Configure Roundcube:

  • accept all of the defaults
  • enter database user "roundcube", database "roundcube" and password under Database setup
  • click: "CREATE CONFIG"
  • download the configuration file generated and save it as config.inc.php
  • move the downloaded file:
$ mv /home/<own account name>/Downloads/config.inc.php /var/www/html/roundcube/config
  • click: "CONTINUE" on the webpage
  • click: "Initialize database"
  • set to use secure SMTP port 465
  • enter sender and recipient email addresses to test SMTP config
  • enter username and password to test IMAP config

Remove or rename installer directory, and protect logs and temp directories:

$ sudo cd /var/www/html/roundcube
$ sudo rm -R installer
$ sudo mv installer installer.original
$ sudo chown -R $USER:$USER logs
$ sudo chown -R $USER:$USER temp

(revised to add protection, 27 May 2019)

Configure PHP5 only (not necessary for PHP7):

$ sudo gedit /etc/php5/apache2/php.ini
  • define:

date.timezone = Asia/Hong_Kong

otherwise the date column of the web mail would be blank. 

Restart Apache2 service:

$ sudo systemctl reload apache2
or
$ sudo service apache2 reload

Log in at web browser:

http://localhost/roundcube

or

http://www.kctang.com.hk/roundcube

Click Settings > Identity to define some settings:

Click Settings > Preferences > User Interface > Date format > 24/7/2019 > Save.

Click Settings > Preferences > Composing Messages

> Compose HTML messages > always.

> when replying > start new message above the quote.

> deselect Force standard separator in signatures.

> Save.

Click Settings > Contacts > Import to import contacts files, e.g. previously exported from Google.

(more descriptions on settings added, 8 May 2019)

Upgrade

Download and extract the subdirectory of the new version to "Downloads" directory as "roundcubemail-<new version number>".

Upgrade existing directory using the installto.sh script:

$ cd /home/<own account name>/Downloads/roundcubemail-<new version number>/bin
$ ./installto.sh /var/www/html/roundcube

In case of big trouble

Uninstall Roundcube:

$ sudo apt-get remove roundcube

Remove "/var/www/html/roundcube":

$ cd /var/www/html
$ sudo rm -r roundcube

Delete mySQL user "roundcube" and database "roundcube" using phpMyAdmin.

Intall Roundcube as new.

End of Page

Install network file system

Install network file system KCTang Mon, 17/09/2018 - 19:20

Go to End

​​​​​​​Note

NFS network file system enables sharing of directories on a Ubuntu server to another Ubuntu client computer.

Install on server computer

Install network file system:

$ sudo apt-get install nfs-kernel-server

Configure to specify the directory to be exported:

$ sudo gedit /etc/exports

Specify:

/<directory exported> <ip address of computer to export to> (rw, sync, no_root_squash)

where "rw" = read and write.

Start the server, required after any rebooting:

$ sudo systemctl start nfs-kernel-server
or
$ sudo service nfs-kernel-server start

Setup client computer

Install nfs-common on the client side:

$ sudo apt install nfs-common

Create a local directory:

$ sudo mkdir -p /media/<local directory>

Mount the remote directory temporarily:

$ sudo mount <ip address of NFS server>:/<directory exported from the NFS server> /media/<local directory>

but remounting required after every reboot.

To keep the mounting permanently unaffected by reboot, edit "fstab" file: 

$ sudo gedit /etc/fstab

Add a line:

<ip address of NFS server>:/<directory exported from the NFS server> /media/<local directory> nfs auto 0 0

(revised 17/8/2018, "defaults" changed to "auto")

Mount again all devices as defined in the "fstab" file after changes:

$ sudo mount -a

Check devices actually mounted (this would show more than those defined in "fstab"):

$ sudo mount -l

End of Page

Install OpenVPN services

Install OpenVPN services KCTang Sun, 02/09/2018 - 22:35

Go to End

Note

OpenVPN enables remote client computers and smartphones to access VPN server's files and structure, and optionally re-direct clients' IP traffic through the VPN server.

(revised 11 April 2018 to add re-direct function)

(revised 2 Sep 2018 to suit Ubuntu 18.04 which requires a change of the network card device name)

Install VPN server for accessing file server

Switch to root:

$ sudo -s

Install openvpn and easy-rsa:

$ apt-get install openvpn easy-rsa

Set up public key infrastructure:

$ mkdir /etc/openvpn/easy-rsa/
$ cp -r /usr/share/easy-rsa/* /etc/openvpn/easy-rsa/
$ gedit /etc/openvpn/easy-rsa/vars

Define in vars:

export KEY_COUNTRY="CN"
export KEY_PROVINCE="HK"
export KEY_CITY="HongKong"
export KEY_ORG="K C Tang Consultants Ltd"
export KEY_EMAIL="kctang@kctang.com.hk"
export KEY_OU=kctclVPN
export KEY_NAME=kctclVPN
# next line added to avoid error when building the certificate and key 
export KEY_ALTNAMES=kctclVPN

Generate master Certificate Authority (CA) certificate and key:

$ cd /etc/openvpn/easy-rsa/
$ source vars
$ ./clean-all
$ ./build-ca

Generate a certificate and private key for the server:

$ ./build-key-server kctclVPN

Accept defaults for most parameters.

Answer “y” to:

  • "Sign the certificate? [y/n]"
  • "1 out of 1 certificate requests certified, commit? [y/n]"

Generate Diffie Hellman parameters:

$ ./build-dh

Copy certificates and keys generated in subdirectory keys/ to /etc/openvpn/:

$ cd keys/
$ cp kctclVPN.crt kctclVPN.key ca.crt dh2048.pem /etc/openvpn/

Generate a certificate and private key for the client user <client>:

$ cd /etc/openvpn/easy-rsa/
$ source vars
$ ./build-key <client>

Copy or move client's certificate and key to a Samba directory, which is for temporary use only: to enable emailing:

$ cp /etc/openvpn/ca.crt /<Samba directory>/
$ mv /etc/openvpn/easy-rsa/keys/<client>.crt /<Samba directory>/
$ mv /etc/openvpn/easy-rsa/keys/<client>.key /<Samba directory>/

Change the owners of the files:

$ cd /<Samba directory>
$ chown nobody:nogroup ca.crt <client>.crt <client>.key
$ chmod 644 ca.crt <client>.crt <client>.key

​E-mail the files to the client computer.

Remove the files:

$ rm ca.crt <client>.crt <client>.key

Config server.conf:

$ cd /
$ cp /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz /etc/openvpn/
$ gzip -d /etc/openvpn/server.conf.gz
$ gedit /etc/openvpn/server.conf

Define as follows:

port 1194
proto udp
dev tun
ca </path to file/>ca.crt
cert </path to file/>kctclVPN.crt
key </path to file/>kctclVPN.key
dh </path to file/>dh2048.pem
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist /var/log/openvpn/ipp.txt
keepalive 10 120
; tls-auth ta.key 0
# the last line is a new feature and would provide extra security,
# but would require corresponding addition in the form of "tls-auth ta.key 1" to the client.conf
# now commented to avoid that change to the clients until convenient time to change 
# last line commented (2 Sep 2018) 
cipher AES-256-CBC # last line added (30 Dec 2017)
comp-lzo
user nobody
group nogroup
# last two lines for linux system
persist-key
persist-tun
status /var/log/openvpn/openvpn-status.log
verb 3
explicit-exit-notify lcicipher

Config sysctl.conf:

$ gedit /etc/sysctl.conf

Uncomment the following line to enable IP forwarding:

net.ipv4.ip_forward=1

Reload sysctl.conf:

$ sysctl -p /etc/sysctl.conf

Start the server:

$ systemctl start openvpn@server

Check if OpenVPN created a tun0 interface:

$ ifconfig tun0

Check syslog if tun0 does not appear:

$ grep -i vpn /var/log/syslog

Exit from root:

$ exit

Set the internet router to re-direct OpenVPN connections to server port 1194.

Extend to re-direct clients' IP traffic through VPN server

(section added 5 April 2018)

Define optionally in server.conf to re-direct clients' IP traffic such as web browsing and DNS lookups to go through the VPN server, i.e. the clients will appear to use the IP of the VPN server instead of the actual IP of the clients for internet traffic:

Config server.conf:

$ sudo gedit /etc/openvpn/server.conf

Define by uncommenting the following line:

push "redirect-gateway def1 bypass-dhcp"

Some guide suggests to add the following, but this results in email server not working: (10 April 2018)

push "dhcp-option DNS 10.8.0.1"

Some other guides suggest to uncomment the following, this works: (10 April 2018)

push "dhcp-option DNS 208.67.222.222"
push "dhcp-option DNS 208.67.220.220"

However, it is found that it still works when the above two lines are left commented. Therefore, the only line needing change is the 'redirect-gateway' line. (2 Sep 2018)

Execute to restart the service:

$ sudo systemctl restart openvpn@server

Execute to see the network card device names:

$ ip route

Find the output line beginning with "default", e.g.:

default via 192.168.0.1 dev enp4s0 proto static metro 100

The name "enp4s0" after the word "dev" is the default network card device name. Previously, the default name is "eth0", but this has been changed after Ubuntu 16.04. (2 Sep 2018)

Execute with the default name inserted after "-o":

$ sudo iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o enp4s0 -j MASQUERADE

Note that the iptables configuration will be lost after reboot.

Store the current iptables configurations:

$ sudo sh -c "iptables-save > /etc/iptables.up.rules"

View and remove any configurations no longer applicable:

$ sudo gedit /etc/iptables.up.rules

Do the same whenever the iptables configurations have been changed.

Config file for use on reboot:

$ sudo gedit /etc/network/interfaces

Define to reuse the stored configurations:

auto lo
iface lo inet loopback
post-up iptables-restore < /etc/iptables.up.rules

Install on Windows client computer

Download and install the latest OpenVPN Windows Installer.

Save ca.crt <client>.crt <client>.key under C:\Program Files\OpenVPN\config\

Create C:\Program Files\OpenVPN\config\<client>.txt file and define it to contain:

client
dev tun
proto udp
remote kctang.com.hk 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
cert <\\path\\><client>.crt
key <\\path\\><client>.key
ns-cert-type server
comp-lzo
verb 3

Also add the following at the end of the file: (added 30 December 2017)

auth-nocache
cipher AES-256-CBC

Specify path in Windows format if ca.crt <client>.crt <client>.key are saved in a folder different from <client>.txt.

Change filename from <client>.txt to <client>.ovpn.

Run on Windows client as a service

To start OpenVPN automatically as a service every time after rebooting:

  • Go to Microsoft Management Console: Start > Computer > Manage > Services and Applications > Services.
  • Right-click OpenVPN service > Properties > Start.
  • Change Startup type to Automatic.
  • Click OK.

However, running as a service has a bug that the connection may not be re-connected after the computer wakes up after sleep or hibernation.

If there is the case, connecting with the GUI will be the better choice.

Run on Windows client using GUI

Start OpenVPN every time after rebooting:

  • Right-click OpenVPN GUI icon on Desktop.
  • Click Properties > Compatibility > Run this program as an administrator > OK.

Connect after starting or loss of connection after sleep or hibernation:

  • Click OpenVPN GUI icon on Desktop.
  • Click the icon on the system tray to connect or right-click the icon and click Connect.

Map drive for quick access

Define a drive to be listed in the file manager directory to represent the server:
  • Open file manager and enter \\10.8.0.1 to access the vpn server.
  • Right-click the desired folder.
  • Click Map network drive.
  • Choose a drive name to represent the folder.
  • Click Finish.

End of Page

Install x11vnc

Install x11vnc KCTang Tue, 28/05/2019 - 13:53

Go to End

28 May 2019: Disabling Wayland added.

18 May 2019: Page added.

Note

x11vnc enables remote access to the graphical desktop of Ubuntu server.

Set router

Set the internet router to permit access to the server through ports 5900, 5901, etc.

Disable Wayland

(section added, 28 May 2019)

Ubuntu-desktop uses gdm3 (Wayland) display manager. Wayland is not compatible with Xorg dispaly server used by x11vnc.

Execute to disable Wayland:

$ sudo gedit /etc/gdm3/custom.conf

Remove "#" to uncomment the following line:

#WaylandEnable=false

Reboot the computer.

Install

Execute:

$ sudo apt install x11vnc
$ sudo apt install xvfb

xvfb provides a virtual X window, which is required for the -create option below.

Start

Execute:

$ x11vnc                    (not requiring a password to connect to port 5900)
or
$ x11vnc -rfbport <port number>
or
$ x11vnc -usepw             (requiring a password to connect)
or
$ x11vnc -create -usepw     (create a virtual desktop)

The default port to connect from client computers is 5900.

Specify 5901 for the <port number> if connection is to be permitted at port 5901.

Connection with or without password is possible.

Execute to set up password:

$ x11vnc -storepasswd

The password will be stored in the file ~/.vnc/passwd, i.e. home/<current user>/.vnc/passwd.

Connection is possible if there is already a graphical desktop logged on at the server. The screen movement at the server and the client computer will be synchronised. This would be good for monitoring the screen movement at the server.

If there is no graphical desktop logged on at the server, then:

  • use the "-create" option
  • connect remotely which should show a terminal window at the server
  • execute at the terminal window to bring up other software, such as:
  • $ firefox
    $ nautilus
  • or the graphical session:
  • $ gnome-session
    
  • click Activities to access other software.

The remote screen movement will not be seen at the server.

Ubuntu-desktop with gdm3 is used above.

x11vnc does not work well with gdm3 (Wayland) desktop. Therefore, just use gdm3 desktop.

Gnome (Wayland) desktop also does not permit the starting of backintime

Stop

Press Ctrl-C to terminate the connection.

End of Page

Install OpenSSH services

Install OpenSSH services KCTang Tue, 26/05/2020 - 16:33

Go to End

26 May 2020: systemctl file command revised.

27 May 2019: Security settings added.

11 Apr 2018: Page added.

Note

OpenSSH enables remote client computers and smartphones to access the server computer's text based terminal shell in a secured manner. "SSH" stands for secured shell.

Install OpenSSH server

Execute:

$ sudo apt install openssh-server

The software will be installed at /etc/ssh.

In case of complaint of no directory, execute  to make directory first:

$ sudo mkdir /etc/ssh

Edit config file:

$ sudo cd /etc/ssh
$ sudo gedit sshd_config

Specify:

# Port 22                             (which is the default port)
Port 2nnn                             (change to some other 4-digit port, 2nnn)
# PermitRootLogin prohibit-password   (meaning no password required)
PermitRootLogin no                    (meaning no root login)

(security settings added, 27 May 2019)

Restart the service:

$ sudo systemctl restart sshd.service
or
$ sudo systemctl restart ssh.service

Check status:

$ sudo systemctl status sshd.service

If found disabled:

$ sudo systemctl enable ssh

(status check added, 26 May 2020)

Change the internet router to permit the use of port 2nnn.

The above is already sufficient for use. Read https://help.ubuntu.com/lts/serverguide/openssh-server.html for more configurations, if desired.

Install SSH client on Windows

Download Putty from https://www.putty.org/ and install.

Enter the Host Name, change the Port to 2nnn, highlight Default Settings and press Save:

"Only on clean exit" is the default. When the server's terminal window is exited with "exit" or "logoff", the PuTTY screen and connection would only close if other processes using the PuTTY connection have all been closed.

Press Open.

Accept the next screen to confirm the server's security key shown, if trusted. This would be necessary for the first time only.

Log in as the usual command terminal. No graphical interface is provided.

Use PuTTYgen that comes installed with PuTTY to generate key pairs, only if required. Read its Help.

Configure for VNC

If PuTTY is used for VNC connection, config the tunnel by entering the Source port and Destination as follows, then press Add to move the setting to the upper window:

With "Local" selected, the Source port means the port of the client computer. It can be "5900" or any free port. "Localhost:5900" at the Destination means the host computer, not the client computer. "5900" refers to the port number on the host computer providing VNC server service.

Go back to the first screen, highlight Default Settings and press Save again.

End of Page

Using Excel

Using Excel KCTang Wed, 11/03/2020 - 19:21

Go to End

Note

11 Mar 2020: Rounding up to 2 significant digits added

17 Jan 2020: Using macros and Add-ins added.

01 Feb 2019: Bracketing negative numbers and hiding zeros added.

26 Sep 2018: Showing one-digit as two-digits added.

19 Sep 2018: First created.

Insert page number and total number of pages

Select Page Layout > Print Titles > Header/Footer > Custom Footer:

(By the way, click Scale with document and Align with page margins to enable the Header and Footer not to encroach into the margins.)

Select icon to insert Page number:

Enter " / ".

Select icon to insert Number of Pages:

Select OK.

Select Page:

Change First page number from "Auto" to "1".

If it is "Auto", the page number of all Worksheets selected to print will run consecutively.

If it is "1", each Worksheet will start at page 1.

However, the total number of pages will be the total of all Worksheets selected printed, not the total number of the respective Worksheet. This may not be desirable.

The next is a solution.

Show total number of pages of a Worksheet

Select Formulas > Name Manager > New.

Define:

  • Name: NumberOfPages
  • Refers to: GETDOCUMENT(50)

Enter in a cell a formula =NumberOfPages

The displayed value will be the total number of pages of the worksheet.

Enter in a cell a formula ="Total " & NumberOfPages & " Pages" (note spaces) will display a meaningful text.

This cannot be used in the Header or Footer.

Show single-digit whole number as two-digit whole number

Press Ctrl-1 > Number > Custom,

Enter "00" against Type. Select OK.

To make the double-digit number actually become a text for other uses, use the TEXT() formula, where "00" represents the Format Type used above, as follows:

Show negative numbers in brackets

(added 01 Feb 2019)

Use: 

#,##0.00;(#,##0.00)

The part before ";" defines the format of positive numbers.

The part after ";" defines the format of positive numbers.

Show negative numbers in brackets

Right-align positive numbers with negative numbers before the ")"

(added 01 Feb 2019)

Append "_-" to the first part:

#,##0.00_-;(#,##0.00)

 

Show negative numbers in brackets and align

Hide zeros for individual cells

(added 01 Feb 2019)

Append ";;@" to the end:

#,##0.00;(#,##0.00);;@

Hide zeros for individual cells

Hide zeros for the whole worksheet

(added 01 Feb 2019)

Select File > Options > Advanced.

De-select Show a zero in cells that have zero value.

All zeros will be hidden.

Excel-Worksheet-HideZeroForWhole

Round up numerical values to 2 significant digits

(added 11 Mar 2020)

Use formula in the cell to display:

"=IF(INT(Q)>0,ROUNDUP(Q,2-LEN(INT(Q))),ROUNDUP(Q,2))"

where Q = referenced cell address (or formula)

It can be read as: if integer of Q > 0, then round it to 2 significant digits, else round up Q to 2 decimal places.

If Q = 1831.03, then

  • INT(Q) > 0.
  • LEN(INT(Q)) = 4
  • 2 - LEN(INT(Q)) = 2 - 4 = -2 (meaning, reduce length from 4 to 2 by -2)
  • ROUNDUP(Q, -2) = 1900.

If Q = 0.1234, then

  • ROUNDUP(Q, 2) = 0.13.

Application:

  • The unit rates used in Cost Estimates are approximate rates. 
  • It would not make sense to show them in many digits and with decimal places to make them look like very accurate.
  • 2 significant digits should be sufficient.
  • Estimates should include buffers. 
  • Rounding up is appropriate.

Use macros

(added 17 Jan 2020)

Macros can be used to execute keyboard or mouse commands automatically. This will simplify the tasks if the same set of keyboard or mouse commands are repeated frequently.

Record macros: 

  • Select View > Macros > Record Macro
  • Enter <Macro Name>
  • Select OK
  • Do an example of the intended task with keyboards and mouse
  • Select View > Macros > Stop Recording

View macros: 

  • Select View > Macros > View Macros
  • Select the desired <Macro Name>
  • Select Run to run the macro to execute the recorded commands
  • Select Step into to run the macro step by step
  • Select Edit to edit the macro
  • Select Option to assign a shortkey to run the macro

Note:

  • The macro will record based on the cells used when recording the macro.
  • Editing is required when the same set of commands is to be run starting from a cell at a different location.
  • Change the relative addresses of the cells as recorded in the macro, and test run to get the expected results.
  • Once successful, the macro can be used for similar cases when the cells used are of the same relative layout.
  • More complex macro would need to be written if more varying circumstances are to be handled as well.
  • The recorded macro can serve as the base or give code segments for use.

Use macros on different files

To run macros on a different file:

  • Save the file with macros as an macro file with ".xlsm" extension to enable it to be run, otherwise the macros must be enabled each time when they are run
  • Open the macro file
  • Open the target file and display the target worksheet as the active worksheet
  • View macro as described above, and select and run the desired macro

Use Add-in to run macros (standard method)

The above process would need a number of steps before the macros can be run.

A simpler way is to add the macros as commands in the ribbon at the top.

Add macros to a ribbon

  • Save the macro file with ".xlsm" extension as an Add-in file with ".xlam" extension, which will save it by default to the following folder:
C:\Users\<Admin or user name>\AppData\Roaming\Microsoft\AddIns
  • Right-click any empty space on any of the ribbons at the top
  • Select Customize the Ribbon > Add-ins 
  • Select Go against Manage:  Excel Add-ins
  •  Select the desired ".xlas" Add-in file > OK
  • Right-click any empty space on any of the ribbons at the top
  • Select Customize the Ribbon > Customize Ribbon
  • Select Macros under Choose commands from to show the available macros
  • Select New Tab to create a New Tab with a New Group (Custom)
  •  
  • Select the desired macro > Add to add the macro under the New Group (Custom)
  • Select the existing added macro > Remove if removal is desired
  • Select OK 
  • A New Tab with menu items for the macros should be available for use
  • Close Excel without the need to save

Use Add-in to run macros (quick method)

  • Copy the required Add-in file with ".xlam" extension to the folder:
C:\Users\<Admin or user name\AppData\Roaming\Microsoft\AddIns
  • Right-click any empty space on any of the ribbons at the top
  • Select Customize the Ribbon > Developer > OK to add the Developer tab
  • Select Developer > Excel-Add-ins
  • Select the desired ".xlam" Add-in file
  • De-select the superseded Add-in file in case of updating
  • Select OK, which should add the Add-in tab
  • Select Add-ins tab to access the menu items for the macros for use.
  • Close and re-open Excel to see that the Add-in ribbon stays for use.

Note:

  • The Add-in file with ".xlam" extension is previously created by saving the macro file with ".xlsm" extension.
  • The macros in the Add-in files will not be seen in the usual View Macro menu.
  • The Add-in file can be accessed by pressing Alt-F11.
  • The VBA macro codes are contained under the Modules folder of the Add-in file directory structure.
  • Two extra macros are contained the "ThisWorkbook" item under "Microsoft Excel Objects" item of the Add-in file directory structure.
  • One of the two macros is called "Private Sub Workbook_Open()" which will remove all existing Add-in menu items and re-add new Add-in menu items when Excel is opened or when the Add-in file is added through Excel-Add-ins.
  • The other of the two macros is called "Private Sub Workbook_BeforeClose()" to remove all Add-in menu items upon closing Excel.
  • It has been suggested that the two macros should be named as "Private Sub Workbook_AddinInstall()"  and "Private Sub Workbook AddinUnintall()" to be effected upon adding or removing the Add-in file, but the AddinInstall is not very effective because the Add-in tab will disappear after closing Excel.

End of Page

Install Thunderbird

Install Thunderbird KCTang Tue, 29/06/2021 - 17:13

Go to End

29 Jun 2021: Advice against upgrading to release 78 removed because more add-ons become compatible. Some notes added.

16 Jan 2021: TbSync manual configuration for Google added.

15 Jan 2021: Google Calendar Plugin added. PrintingTools attachments per line option added. Header fixed font size revised. CardBook add-on removed because TbSync supports Google again. Lists of add-ons updated. 

19 Nov 2020: Option set to show all folders without subscribing individually. CardBook add-on added because TbSync no longer supports Google. Lists of add-ons updated. 

8 Sep 2020: 'Manually sort folders' add-on added.

12 Aug 2020: Revised to fix font for print headers only.

11 Aug 2020: First Created.

Note

Thunderbird is an email client.

Install Thunderbird

Download the latest release from here and install:

https://www.thunderbird.net/en-US/download/

Download old releases from here and install:

https://archive.mozilla.org/pub/thunderbird/releases

Set up the first email account.

For Gmail account, enter the Gmail address and use automatic setup.

Use the following settings to manually set up email account accessing company's email server:

  • Account Name: kctcl
  • Your Name: K C Tang Consultants Ltd.
  • Email Address: kctcl@kctang.com.hk
  • Incoming Server Type: IMAP Mail Server
  • Server Name: kctang.com.hk
  • Port: 993
  • Connection security: SSL/TLS
  • Authentication method: Normal password
  • Outgoing Server Name: kctang.com.hk
  • Port: 465
  • Connection security: SSL/TLS
  • Authentication method: Normal password

Select Tools > Accounts Setting to further define or adjust the settings:

Uncheck "Show only subscribed folders" to show all folders instead of subscribing individually:

(Added, 11 Nov 2020)

The default is to keep all messages on the local computer as shown below, but it is suggested not to do so to avoid using up local computer's storage space. Subject headings will always be downloaded. The email body and attachments will be downloaded only when being read, and may take a bit longer time, but this should be tolerable.

(Note added, 29 Jun 2021)

​​​​​​​​​​​​​​​​​​​

Select Tools > Accounts Setting > Account Actions > Add Mail Account to add other accounts.

Leave the options under Tools > Options as the default except in the following case (not required now as the Postfix configuration can be raised).

When the tls security level has to be lowered if the email server is not of the level expected by Thunderbird:

Select Tools > Options > General > Config Editor > I accept the risk > Search "security.tls" > security.tls.version.min

Change the default value "3" to "1" and select OK and exit.

(Note to adjust tls security level added, 29 Jun 2021)

Install Add-ons​​​​​​​

Select Tools > Add-ons.

Search at "Find more extensions" for the following add-ons, install them and restart Thunderbird:

  1. AttachmentExtractor Continued - to download attachments (emails must have been fully downloaded before the full attachments can be extracted)
  2. Google Calendar Plugin - to open Google calendar
  3. ImportExport Tools NG - to import or export emails or folders, good for exporting for archive 
  4. Lightning - to use calendars (better use Google calendars directly)
  5. LookOut (fix version) - to open Outlook attachments
  6. Manually sort folders - to re-sequence order of accounts and folders
  7. PrintingTools NG - to set print page layout
  8. Provider for CalDAD & CardDAV - required for TbSync below
  9. Quick Folder Move - to move or copy emails between folders
  10. Remove Duplicate Messages - to remove duplicate emails
  11. SmartTemplate4 - to set templates for composition, reply and forwarding
  12. TbSync - to synchronize contact, task and calendar information.

(Lists updated, 15 Jan 2021)

(Note against AttachmentExtractor added, 29 Jun 2021)

Use Quick Folder Move

Select some messages.

Press the following keys to open a context menu, enter a few characters for the name of the destination folder to display possible choices, move down to select the folder, and press Enter.

Press:

Shift+M to move the currently selected messages to the destination folder.

Ctrl+Shift+M to repeat moving to the previously selected folder.

Shift+Y to copy the currently selected messages to the destination folder.

Ctrl+Shift+Y to repeat copying to the previously selected folder.

Shift+G to jump to the destination folder (without the need to pre-select come messages).

Setup Remove Duplicate Messages

Select Tools > Add-on Options > Remove Duplicate Options:

Select Move to Trash instead of Delete permanently if not sure.

Select Folder if comparison is not across different folder.

Setup Print Page

Select File > Page Setup:

Setup Printing Options

Select File > Printing Options:

(Attachments per line option added, 15 Jan 2021)

(Header fixed font size revised, 15 Jan 2021)

Setup SmartTemplate4

Select Tools > Add-on Options > SmartTemplate4:

Copy and paste the following latest Code in the Quote Header of the Reply and Forward options:

<a blank line first>
Regards,

Mr. K C Tang <replace with your name>
for
K C Tang Consultants Ltd.
T:(852)2866-6451 F:(852)2865-4751 W:www.kctang.com.hk

<hr/>
<SPAN style="FONT-SIZE: 10pt">
<b>Sent: </b>%date% ​​​​​​​<Use this to show: Sent: Mon, 14 Dec 2020 10:47:45 +0800>
<b>Sent: </b>%X:=sent% %datelocal% %date_tz% ​​​​​​​<Or use this to show: Sent:  Monday, Dec 14, 2020, 10:47 +0800>
<b>From: </b>%from(name,bracketMail())%
<b>To: </b>
<SPAN style="FONT-SIZE: 8pt">
%to(name,bracketMail())%</SPAN>
[[<br><b>Cc: </b>
<SPAN style="FONT-SIZE: 8pt">
%cc(name,bracketMail())%</SPAN>]][[<br><b>Bcc: </b>
<SPAN style="FONT-SIZE: 8pt">
%bcc(name,bracketMail())%</SPAN>]]
<b>Subject: </b>%subject%
</SPAN>
<hr/>

(Note against date formats added, 29 Jun 2021)

Insert license key in the next menu.

Setup TbSync

Select Tools > Synchronization Settings (TbSync):

Or select Tools > Add-on Options to go to the following, then select Add new account > CalDAV & CardDAV:

Select Install Provider > download link:

Add CalDAV & CardDAV:

After adding, and re-starting Thunderbird as necessary, go back and select Manual Configuration > Next:

Enter

Account name: friendly account name, e.g. kctangcl

User name: gmail.com account name

CalDAV server address: https://apidata.googleusercontent.com/caldav/v2

CardDAV server address: https://www.googleapis.com/.well-known/carddav

Select Next:

(Manual configuration added, 15 Jan 2021)

The following is an older version of TbSync which provided automatic Google support. In that case, select Google > Next:

Enter a friendly account name and select Next:

After selecting Next in the manual case or automatic case, enter Google account email address and its password (may require doing it twice):

Select Finish:

Select Enable and synchronize this account:

Available resources like Google address book and calendars will be displayed.

Select those to be synchronized > Synchronize now:

The address book and calendars will be available for use.

Change Periodic synchronization (in minutes) to 30, which is the minimum interval. 

 

End of Page