Details

Published: 16 November 2020

• composer
• php

Details

Published: 26 October 2020

• git

Most Git workflows do not address picking what is released. Once a feature branch has been merged back to 'development', it is in the release pipe, pending QA and Business validation.  Once in 'development', code/branches cannot easily or arbitrarily be plucked out to go directly to production as the code is often intermingled with other branches.  But the code changes in 'development' can be manually re-coded (Git patches help) into a hotfix branch from 'master'/'production' with the risk of not being QA-ed.

The 'Git single branch strategy' primarily removes the pain point of the conflicts between 'master' and 'development', while providing a clearer history of production releases, and an easier rollback with switching branches.

Generally picking what to release is largely mitigated by how, and the order in which Tickets are chosen. However, the release pipe can be slowed down by a Ticket/branch needing time to fix, or be validated by QA or Business.  To remove the blocking branch, depending on the changes, the feature could be hidden, or removed if a small change, or more likely wait for the fix(es) and QA.

Hopefully the upfront choosing of Tickets and quality of specifications somewhat mitigates the blockers.

Some Git strategies to mitigate blockers in the release pipe, which are caused by Tickets that often require feedback once seen.

1) Put less in the release pipe: (Less is more)

If limit releases to one branch per release, then there is no re-picking once in 'development'.  Basically the other feature branches would queue up waiting to be picked and for merge to 'development' and QA-ed.  Which leads to the pros and cons of Staging branches.

2) Staging branches:

Another process to maybe help with picking branches for release is to not merge feature branches back to 'development' until picked.  The feature branches could be deployed to their own directory (devsite.com/branch/123-shortdesc) be QA-ed, reviewed by Business, then if ok, merged to 'development'.  Then when decided to go to production, everything currently in 'development' is QA-ed again, fixes added via branch updates or a new branch, and then released via 'master'/'production'.  

Note, if after being merged to 'development', production release decisions change, well, then we are back to the same problem of doing hotfixes, or hiding not ready functionality, or waiting for the fixes.

Also this approach can be a burden on the developers: fixes enhancements, code cleanups, won't be seen or utilized until the branch is picked, and those fixes/enhancements might be required or desired for another branch, thus duplication of code which probably means conflicts later.  Before merging the feature branch to 'development', 'development' would need to be merged back to the feature branch to handle any changes or conflicts in the branch, so the developer can test again before merging to 'development'.  And as the code won't be fresh on the developers mind, there is a higher risk of mistakes to be made during the merge to 'development'.

pros: 

• able to preview branches before release
• able to choose branches for release

cons:

• more work for Business and/or QA as the merged branches in ‘development’ still need to  be reviewed
• if decisions change to remove a branch or hotfix a branch to production before qa, same problems
• more burden on developers, potential conflicts, developing the branch twice: once orig, then later (days, weeks) when picked
• dev-ops + some app work to make 'their own directory' happen

Committing often, merging often seems to be better for code quality.

3) Feedback branches, Preview branches: (A hybrid of Staging branches)

For branches which require Business or early QA feedback, after development is done, but before QA or merging to 'development', push the branch to a preview location (devsite.com/branch/123-shortdesc).  There it can be previewed for one or two days, before being merged to 'development' and moved to QA; required feedback branches should not be held for a long time, else the cons of Staged branches may become apparent.  The branches that require feedback should be marked as such before development.  Every branch should not be marked as requires feedback, only a few should be.

pros: 

• able to preview branches marked as feedback before release
• should reduce fixes needed when in 'development' for QA
• as only a day or two delay, no large time incurred burden on developers

cons:

• does not allow changing order of released branches
• more work for Business and/or QA as the merged branches in ‘development’ still need to  be reviewed
• if decisions change to remove a branch or hotfix a branch to production before QA, same problems
• dev-ops work  + some app work to make 'preview location' happen

Hopefully some useful Git strategies when dealing with Tickets that often require feedback once seen.

-End of Document-
Thanks for reading

Details

Published: 19 October 2020

• git

A common Git workflow has two main branches, 'development'  and 'master'.  With the common workflow of creating feature branches from ‘development’, merging the feature branch back to ‘development’, and then for a release to production, merging 'development' to 'master'.  Non-trivial conflicts can occur during the merge to 'master' when same/similar changes are made in both 'master' and 'development', or 'development' has had some necessary reverts or other changes.

Instead of having two main or long lived branches, another idea for a Git workflow is to use only one main branch, then create a branch for production pulls.

aka 'trunk-based development workflow' or 'single branch strategy'

Think of the one main branch as the 'development' branch

• flow for development would be the same:

       - branch from 'development' for a Ticket, merge request  to 'development' when done

• when want to do a release to production, instead of a merge request to 'master'

       - create a new 'production' branch at the last or desired 'development' commit

• when want to do another release to production

       - rename current 'production' to 'production-date' ie 'production-20200531'

       - create a new 'production' branch at the last or desired 'development' commit

• 'master' branch would then be unused and eventually removed

       - current purpose of 'master' is: 

          release this code, which will be the purpose of 'production'

• 'development' is never merged into 'production'
• overtime, like feature branches, old 'production-date' branches can be deleted

pros:

• never any conflicts between 'master' eg 'production' and 'development', as only one main branch
• hotfixes can be applied to current 'production' branch without worry of later conflicts
• 'production-date' can be used as a rollback for code in case a release is non usable/broken and code related

cons:

• no 'master' branch; but 'production' and 'development' are more explicit
• no merge request to 'master'; 'production' and 'production-*' can be marked as a protected branches in GitLab
• slightly different

GitLab CI/CD:

• should still work at creation of 'production', 
• and maybe the renaming of 'production-date' and creation of 'production' could be part of the GitLab CI/CD

Tags:

An alternative to creating 'production' branches would be to create 'production' tags.  If a hotfix is needed, then create a branch from the tag.  But just keeping everything a branch simplifies: GitLab CI/CD, Git UIs, merge request for hotfix, etc

And yes, the one branch to rule them all could be named the default Git repo branch name of ‘master’.  Or 'sam', or whatever your group agrees upon, and you can tell others.

-End of Document-
Thanks for reading

Details

Published: 27 September 2020

• security
• sftp
• ssh

If you want to allow a user to upload files securely to a site, and they do not need shell access or know how to use shell, you can grant them SFTP only access.  SFTP, which stands for SSH File Transfer Protocol, or Secure File Transfer Protocol, is a separate protocol packaged with SSH that works in a similar way over a secure connection.  To an end user, SFTP works the same as FTP.  You login, browse directories, and upload/download files, but more securely. 

While FTPS adds a layer to the FTP protocol, SFTP is a different protocol based on the network protocol SSH (Secure Shell). Unlike both FTP and FTPS, SFTP uses only one connection and encrypts both authentication information and data files being transferred.

To add a SFTP only user to Red Hat Enterprise 8 (RHEL8)
Note, of course, should work for other Linux flavors too

1) Create your user appuser1
> sudo useradd -s /sbin/nologin appuser1

Verify 
> grep /etc/passwd appuser1

While the SSH server will be configured to prevent shell access,
setting /sbin/nologin as shell adds another layer to not allow the user to SSH and get shell access

You can optionally create a group of sftp only users
> groupadd sftponly

And add you user to it
> sudo usermod -a -G sftponly appuser1

Note, make sure to add -a to append groups, else you will end up setting to only that one group

Verify 
> grep /etc/groups appuser1

2) Update the SSH server to only allow your SFTP user and/or group
> sudo vim /etc/ssh/sshd_config

Find Subsystem SFTP, and, if needed, change it to
Subsystem sftp  internal-sftp

> sudo vim /etc/ssh/sshd_config
# override default of no subsystems
#Subsystem sftp /usr/libexec/openssh/sftp-server # default
Subsystem sftp  internal-sftp # must use for sftp 'jail'; similar to default

If you do not change the Subsystem SFTP, your SFTP client may report
"Cannot initialize SFTP protocol. Is the host running a SFTP server?"
and you may see errors in 
/var/log/secure
Accepted password for wpsite1 from 172.30.0.160 port 52436 ssh2
pam_unix(systemd-user:session): session opened for user wpsite1 by (uid=0)
pam_unix(sshd:session): session opened for user wpsite1 by (uid=0)
pam_unix(sshd:session): session closed for user wpsite1
https://winscp.net/eng/docs/message_cannot_initialize_sftp_protocol

Toward the bottom, add
> sudo vim /etc/ssh/sshd_config

# web user 
Match User appuser1
    ChrootDirectory /var/www/html 
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no

While you can and should use SSH keys, you can also add 
PasswordAuthentication yes 
for only specific users, groups or ips

# app1 user on vpn network
Match User appuser1 Address 10.10.0.0/16
    PasswordAuthentication yes
    ChrootDirectory /data 
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no

# app1 users
Match Group sftponly
    ChrootDirectory /var/www/html 
    ForceCommand internal-sftp
    AllowTcpForwarding no
    X11Forwarding no

3) create the directory where the SFTP user will be restricted to, also know as chroot or 'jail' directory
This directory, as specified by ChrootDirectory, must be a root-owned directory that is not writable by any other user or group.

So if your website is in /var/www/html
> ls -ld /var/www/html
drwxr-xr-x 2 root root /var/www/html

You can use that directory for your SFTP user chroot directory

To create another directory
> sudo mkdir /data
> sudo chmod 755 /data
> ls -ld /data
drwxr-xr-x 2 root root /data

If the permissions for configured ChrootDirectory are not correct, you will see errors in
/var/log/secure
Accepted password for appuser1 from 10.10.0.2 port 52331 ssh2
pam_unix(systemd-user:session): session opened for user appuser1 by (uid=0)
pam_unix(sshd:session): session opened for user appuser1 by (uid=0)
fatal: bad ownership or modes for chroot directory "/home/appuser1" [postauth]
pam_unix(sshd:session): session closed for user appuser1
pam_unix(systemd-user:session): session closed for user appuser1
https://serverfault.com/a/660180/523393

4) Restart SSH
> sudo systemctl restart sshd

Note, if you configuration is incorrect, you will not be bounced from your current SSH session.
!But do fix and test the configuration before exiting!
"bouncing sshd is smart enough to permit existing ssh connections to merrily continue unabated "
https://askubuntu.com/a/462971/708501

5) Verify
Test you new user SFTP login .. should work
Test you new user SSH login .. should not work
Test your existing SSH login .. should still work

You have now created a limited SFTP user.