Details

Published: 19 October 2020

• git

A common Git workflow has two main branches, 'development'  and 'master'.  With the common workflow of creating feature branches from ‘development’, merging the feature branch back to ‘development’, and then for a release to production, merging 'development' to 'master'.  Non-trivial conflicts can occur during the merge to 'master' when same/similar changes are made in both 'master' and 'development', or 'development' has had some necessary reverts or other changes.

Instead of having two main or long lived branches, another idea for a Git workflow is to use only one main branch, then create a branch for production pulls.

aka 'trunk-based development workflow' or 'single branch strategy'

Think of the one main branch as the 'development' branch

• flow for development would be the same:

       - branch from 'development' for a Ticket, merge request  to 'development' when done

• when want to do a release to production, instead of a merge request to 'master'

       - create a new 'production' branch at the last or desired 'development' commit

• when want to do another release to production

       - rename current 'production' to 'production-date' ie 'production-20200531'

       - create a new 'production' branch at the last or desired 'development' commit

• 'master' branch would then be unused and eventually removed

       - current purpose of 'master' is: 

          release this code, which will be the purpose of 'production'

• 'development' is never merged into 'production'
• overtime, like feature branches, old 'production-date' branches can be deleted

pros:

• never any conflicts between 'master' eg 'production' and 'development', as only one main branch
• hotfixes can be applied to current 'production' branch without worry of later conflicts
• 'production-date' can be used as a rollback for code in case a release is non usable/broken and code related

cons:

• no 'master' branch; but 'production' and 'development' are more explicit
• no merge request to 'master'; 'production' and 'production-*' can be marked as a protected branches in GitLab
• slightly different

GitLab CI/CD:

• should still work at creation of 'production', 
• and maybe the renaming of 'production-date' and creation of 'production' could be part of the GitLab CI/CD

Tags:

An alternative to creating 'production' branches would be to create 'production' tags.  If a hotfix is needed, then create a branch from the tag.  But just keeping everything a branch simplifies: GitLab CI/CD, Git UIs, merge request for hotfix, etc

And yes, the one branch to rule them all could be named the default Git repo branch name of ‘master’.  Or 'sam', or whatever your group agrees upon, and you can tell others.

-End of Document-
Thanks for reading

Details

Published: 27 September 2020

• security
• sftp
• ssh

If you want to allow a user to upload files securely to a site, and they do not need shell access or know how to use shell, you can grant them SFTP only access.  SFTP, which stands for SSH File Transfer Protocol, or Secure File Transfer Protocol, is a separate protocol packaged with SSH that works in a similar way over a secure connection.  To an end user, SFTP works the same as FTP.  You login, browse directories, and upload/download files, but more securely. 

While FTPS adds a layer to the FTP protocol, SFTP is a different protocol based on the network protocol SSH (Secure Shell). Unlike both FTP and FTPS, SFTP uses only one connection and encrypts both authentication information and data files being transferred.

To add a SFTP only user to Red Hat Enterprise 8 (RHEL8)
Note, of course, should work for other Linux flavors too

1) Create your user appuser1
> sudo useradd -s /sbin/nologin appuser1

Verify 
> grep /etc/passwd appuser1

While the SSH server will be configured to prevent shell access,
setting /sbin/nologin as shell adds another layer to not allow the user to SSH and get shell access

You can optionally create a group of sftp only users
> groupadd sftponly

And add you user to it
> sudo usermod -a -G sftponly appuser1

Note, make sure to add -a to append groups, else you will end up setting to only that one group

Verify 
> grep /etc/groups appuser1

2) Update the SSH server to only allow your SFTP user and/or group
> sudo vim /etc/ssh/sshd_config

Find Subsystem SFTP, and, if needed, change it to
Subsystem sftp  internal-sftp

> sudo vim /etc/ssh/sshd_config
# override default of no subsystems
#Subsystem sftp /usr/libexec/openssh/sftp-server # default
Subsystem sftp  internal-sftp # must use for sftp 'jail'; similar to default

If you do not change the Subsystem SFTP, your SFTP client may report
"Cannot initialize SFTP protocol. Is the host running a SFTP server?"
and you may see errors in 
/var/log/secure
Accepted password for wpsite1 from 172.30.0.160 port 52436 ssh2
pam_unix(systemd-user:session): session opened for user wpsite1 by (uid=0)
pam_unix(sshd:session): session opened for user wpsite1 by (uid=0)
pam_unix(sshd:session): session closed for user wpsite1
https://winscp.net/eng/docs/message_cannot_initialize_sftp_protocol

Toward the bottom, add
> sudo vim /etc/ssh/sshd_config

# web user 
Match User appuser1
    ChrootDirectory /var/www/html 
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no

While you can and should use SSH keys, you can also add 
PasswordAuthentication yes 
for only specific users, groups or ips

# app1 user on vpn network
Match User appuser1 Address 10.10.0.0/16
    PasswordAuthentication yes
    ChrootDirectory /data 
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no

# app1 users
Match Group sftponly
    ChrootDirectory /var/www/html 
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no

3) create the directory where the SFTP user will be restricted to, also know as chroot or 'jail' directory
This directory, as specified by ChrootDirectory, must be a root-owned directory that is not writable by any other user or group.

So if your website is in /var/www/html
> ls -ld /var/www/html
drwxr-xr-x 2 root root /var/www/html

You can use that directory for your SFTP user chroot directory

To create another directory
> sudo mkdir /data
> sudo chmod 755 /data
> ls -ld /data
drwxr-xr-x 2 root root /data

If the permissions for configured ChrootDirectory are not correct, you will see errors in
/var/log/secure
Accepted password for appuser1 from 10.10.0.2 port 52331 ssh2
pam_unix(systemd-user:session): session opened for user appuser1 by (uid=0)
pam_unix(sshd:session): session opened for user appuser1 by (uid=0)
fatal: bad ownership or modes for chroot directory "/home/appuser1" [postauth]
pam_unix(sshd:session): session closed for user appuser1
pam_unix(systemd-user:session): session closed for user appuser1
https://serverfault.com/a/660180/523393

4) Restart SSH
> sudo systemctl restart sshd

Note, if you configuration is incorrect, you will not be bounced from your current SSH session.
!But do fix and test the configuration before exiting!
"bouncing sshd is smart enough to permit existing ssh connections to merrily continue unabated "
https://askubuntu.com/a/462971/708501

5) Verify
Test you new user SFTP login .. should work
Test you new user SSH login .. should not work
Test your existing SSH login .. should still work

You have now created a limited SFTP user.  

Details

Published: 14 September 2020

• security
• ftp

If you want to allow a user to upload files securely to a site, you can grant them FTPS access.

FTPS (also known FTP-SSL, and FTP Secure) is an extension to the commonly used File Transfer Protocol (FTP) that adds support for the Transport Layer Security (TLS) and, formerly, the Secure Sockets Layer (SSL) cryptographic protocols.
https://en.wikipedia.org/wiki/FTPS

While SFTP should be used instead, sometimes apps or users require using FTP.

While FTPS adds a layer to the FTP protocol, SFTP is a different protocol based on the network protocol SSH (Secure Shell). Unlike both FTP and FTPS, SFTP uses only one connection and encrypts both authentication information and data files being transferred.
https://www.keycdn.com/support/ftps-vs-sftp

To add a FTPS only user to Red Hat Enterprise 8 (RHEL8)
Note, of course, this should work for other Linux flavors too

1) Create your user appuser1
> sudo useradd -s /sbin/nologin appuser1

Setting /sbin/nologin as shell prevents the user from using SSH and get shell access

2) Install a FTP server, vsftpd
> sudo yum install vsftpd

3) Update the vsftpd config
> sudo vim /etc/vsftpd/vsftpd.conf

Enable local users

..
# Uncomment this to allow local users to log in.
# When SELinux is enforcing check for SE bool ftp_home_dir
local_enable=YES
..
# Allow virtual users to use the same privileges as local users
virtual_use_local_privs=YES

# Setup the virtual users config folder
user_config_dir=/etc/vsftpd/user_config/
..
 

More logging

..
# more verbose logging, including connections and commands
xferlog_std_format=NO
log_ftp_protocol=YES
vsftpd_log_file=/var/log/vsftpd/vsftpd.log
dual_log_enable=YES
..

Restrict users to a dir

..
# restricted to users home dir /etc/passwd
chroot_local_user=YES
..

Your ISP or router may block the default port 21, so use another port such as 2121
FTP requires another port for data, hence 2120

..
# port 21 blocked by .. modem or router
listen_port=2121
ftp_data_port=2120 # just to match
..

Enable a whitelisted access list

..
# /etc/pam.d/vsftpd tried to use file /etc/vsftpd/ftpusers, default deny, but had to comment out
pam_service_name=vsftpd

# default, do not allow these users, but allow anyone else
# userlist_enable=YES
# userlist_file=/etc/vsftpd/user_list

# allow only these users
userlist_enable=NO
userlist_file=/etc/vsftpd/sci_user_list
userlist_deny=NO
..

Enable passive mode.
In an active mode connection, when the client makes the initial connection and sends PORT, the server initiates the second connection back. In a passive connection, the client connects and sends the PASV command, which functions as a request for a port number to connect to.  Passive mode solves the problem of an FTP client's firewall blocking incoming connections.

..
pasv_enable=YES
pasv_min_port=2124
pasv_max_port=2148
pasv_address=[your public ip]
..

Set the paths to your existing web SSL certs

..
# path of the SSL certificate
# using web certs
rsa_cert_file=/etc/ssl/site.crt
rsa_private_key_file=/etc/ssl/site.key
# enable SSL
ssl_enable=YES
allow_anon_ssl=NO
force_local_data_ssl=YES
force_local_logins_ssl=YES
# TSL is more secure than SSL so enable ssl_tlsv1_2.
ssl_tlsv1=YES
ssl_sslv2=NO
ssl_sslv3=NO
require_ssl_reuse=NO
ssl_ciphers=HIGH
# enable SSL debugging
debug_ssl=YES
..

4) Update pam.d/vsftp authentication
> sudo vim /etc/pam.d/vsftp

#%PAM-1.0
session    optional     pam_keyinit.so    force revoke
# prevented login with valid user
# auth       required pam_listfile.so item=user sense=deny file=/etc/vsftpd/ftpusers onerr=succeed
# /sbin/nologin is not a valid shell, so ignore check
# auth       required pam_shells.so
auth       include  password-auth
account    include  password-auth
session    required     pam_loginuid.so
session    include  password-auth

5) Create the directory where the SFTP user will be restricted to, also know as chroot or 'jail' directory.  This directory must be root-owned directories that are not writable by any other user or group.
Note, enabled via vsftpd.conf chroot_local_user=YES

So if your website is in /var/www/html
> ls -ld /var/www/html
drwxr-xr-x 2 root root /var/www/html

You can use that directory for your SFTP user chroot directory

To create another directory
> sudo mkdir /data
> sudo chmod 755 /data
> ls -ld /data
drwxr-xr-x 2 root root /data

Change the users home directory to the chroot directory
> usermod -d /var/www/html appuser1
> usermod -d /data appuser1

6) add custom config per user
which allows the ftp user to create files as another user
> sudo vim /etc/vsftpd/user_config/appuser1
# also set users home dir in /etc/password
local_root=/var/www/html
write_enable=YES
# create new files as
guest_enable=YES
guest_username=appweb1

7) Restart vsftpd
> sudo systemctl restart vsftpd

8) Update your firewall
If you are using Amazon EC2, configure your Security Group, adding the ports
TCP 2120-2148.  These are the ports vsftpd is listening on and passive mode responding on

9) Test using a FTP client, such as FileZilla
https://filezilla-project.org/
Note, don't forget to change the default port 21 to what you configured
vsftpd.conf listen_port=2121
And enable encryption
Require explicit FTP over TLS

You have now created a limited FTPS user. 

Details

Published: 17 August 2020

• portable
• nginx
• security
• laragon