Skip to content

armgit / core

Public Imported from GitHub
0

100% open-source IoT Platform - Integrate your devices, add intelligence, build custom apps

CI/CD documentation and minor updates

Rich Turner0cdf4f8ac8c3
diff --git a/.ci_cd/README.md b/.ci_cd/README.md
index dd22cd7ba..d78016d09 100644
--- a/.ci_cd/README.md
+++ b/.ci_cd/README.md
@@ -1,15 +1,79 @@
# CI/CD workflow files
-The files in this directory are used by the [deploy workflow](../.github/workflows/deploy.yml) which provides a standard
-github workflow for deploying this repo on a linux host via SSH.
+The files in this directory are used by the [CI/CD workflow](../.github/workflows/ci_cd.yml) which provides a standard
+github workflow for CI/CD tasks in this repo and custom projects/forks on a linux host via SSH.
-# `/env` Directory
-Environment variable files that will be loaded into the workflow to configure the behaviour of the workflow and are also
-passed through to the host and loaded whenever remote commands are executed via SSH on the host.
+# `ci_cd.json` File
+Configuration file for distribution (pushing openremote/manager docker images) and deploying for this specific
+repository; the file is only required if automatic deployments are desired during push/release and/or openremote/manager
+docker image tag updates. The file layout is:
-The naming convention should be as follows:
+```json
+{
+ // Configure behaviour on push
+ "push": {
+ // Name of branch to react to (exact match)
+ "develop": {
+ // Comma separated list of tags to push to openremote/manager docker hub image $version is replaced with release version
+ // This can only be used on the main openremote repo (openremote/openremote)
+ "distribute": {
+ "tags": "develop"
+ },
+ // Singleton or array of deployments to execute, a deployment consists of (environment and/or managerTag see variables for explanation)
+ "deploy": {
+ "managerTag": "develop", // Can use #ref or exclude this to build COMMIT specific maanger docker image
+ "environment": "staging"
+ }
+ },
+ "master": {
+ "distribute": {
+ "tags": "latest"
+ }
+ }
+ },
+ // Configure behaviour on release
+ "release": {
+ // Comma separated list of tags to push to openremote/manager docker hub image $version is replaced with release version
+ // This can only be used on the main openremote repo (openremote/openremote)
+ "distribute": {
+ "tags": "latest,$version"
+ },
+ // Singleton or array of deployments to execute, a deployment consists of (environment and/or managerTag see variables for explanation)
+ "deploy": {
+ "managerTag": "latest", // Can use #ref or exclude this to build COMMIT specific maanger docker image
+ "environment": "production"
+ }
+ },
+ // Configure behaviour on manager docker image update (must be triggered by schedule event from a custom project or fork)
+ "managerDockerPush": {
+ // Docker tags to monitor can only be used if not the main openremote repo (openremote/openremote)
+ "develop": {
+ // Singleton or array of deployments to execute, a deployment consists of (environment and/or managerTag see variables for explanation)
+ "deploy": {
+ "environment": "staging"
+ }
+ }
+ }
+}
+```
+
+# `deploy.sh` Bash script
+Bash script that handles the actual deployment to a specific host; environment variables should be already loaded into
+the shell or be available in `temp/env` file which will be automatically loaded. If `ssh.env` is found this env file is
+also automatically loaded. See [Variables section below](#variables) for available and required values.
+
+## Docker compose file resolution
+The docker compose file used for the deployment is resolved as follows; if resolved file does not exist then deployment
+will fail:
+* `ENV_COMPOSE_FILE` variable
+* `profile/${ENVIRONMENT}.yml` file
+* `docker-compose.yml` file
-* `env/.env` - File containing environment variables that will be loaded for any deployment
-* `${environment}.env` - File containing environment specific environment variables will be loaded if `environment` input
+# `env` Directory
+Environment variable files that will be loaded when deploying.
+
+The naming convention should be as follows:
+* `env/.env` - File containing environment variables that will be loaded for all deployments
+* `${ENVIRONMENT}.env` - File containing environment specific environment variables will be loaded if `ENVIRONMENT` variable
is set for the workflow run; these will be loaded after the `env` file and so can override any value defined there as well
as being able to add new environment specific values
@@ -17,10 +81,34 @@ as being able to add new environment specific values
Contains scripts/files required to initialise the host ready for running the stack (e.g. download map tiles etc.);
the workflow looks for one of the following bash scripts (in priority order):
-* `host_init/${environment}.sh` - Bash script for environment specific host initialisation
+* `host_init/${ENVIRONMENT}.sh` - Bash script for environment specific host initialisation
* `host_init/init.sh` - Bash script that can be used by any environment as a fallback if no environment specific script
exists
If an initialisation script is found then the entire `host_init` directory will be copied to the host and the script
will be executed; by copying the entire directory additional files can be included and called from the initialisation
script as required.
+
+
+# Variables
+The following variables are supported by deployments; any additional variables can also be used and these are made
+available to the shell where the docker compose stack is brought up; so any variables required in the docker compose
+profile can be specified in any of the places variables are loaded from (github secrets, inputs, and/or env files).
+
+* `ENVIRONMENT` - Used to control which env file(s), docker compose file and github secrets to use for deployment.
+* `MANAGER_TAG` - The docker tag to pull for the manager image (if not specified)
+* `CLEAN_INSTALL` - Indicates if the or_postgresql-data volume should be removed before starting the stack
+* `COMMIT` - Which commit or branch to checkout and use on this repo (default: branch/SHA on which workflow is executed)
+* `HOST` - FQDN for the host (default: env/secrets.HOST)
+* `SSH_USER` - SSH username (default: root)
+* `SSH_KEY` - SSH private key (this is written to `ssh.key` file and loaded by `deploy.sh` for SSH/SCP commands)
+* `SSH_PASSWORD` - SSH password (alternative to private key authentication)
+* `SSH_PORT` - SSH port (default: 22)
+* `SETUP_ADMIN_PASSWORD` - Admin password to be set on deployment; sets the SETUP_ADMIN_PASSWORD env variable
+* `ENV_COMPOSE_FILE` - The full path to the docker compose file to use for the deployment
+
+## Github secrets
+Variables are also loaded from Github secrets using the '_{ENVIRONMENT}_' prefix naming convention (note it doesn't use
+github environment secrets as these only seem to be supported for public repositories); any secrets that begin with `_`
+will only be loaded if they begin with `_{ENVIRONMENT}_` and this prefix will be removed, all other secrets will be
+loaded independent of the `ENVIRONMENT` variable.
diff --git a/.ci_cd/gitlab-ci/aws-cloudformation.template.arm64.yml b/.ci_cd/aws/aws-cloudformation.template.arm64.yml
similarity index 100%
rename from .ci_cd/gitlab-ci/aws-cloudformation.template.arm64.yml
rename to .ci_cd/aws/aws-cloudformation.template.arm64.yml
diff --git a/.ci_cd/aws/aws-cloudformation.template.yml b/.ci_cd/aws/aws-cloudformation.template.yml
new file mode 100644
index 000000000..f646de221
--- /dev/null
+++ b/.ci_cd/aws/aws-cloudformation.template.yml
@@ -0,0 +1,413 @@
+---
+AWSTemplateFormatVersion: '2010-09-09'
+Description: 'Custom project OpenRemote 3 stack on AWS'
+Parameters:
+ OpenRemotePassword:
+ Description: Password for OpenRemote admin login
+ Type: String
+ Default: secret
+ DomainName:
+ Description: DNS domain for created stack (clear it is you want the deployment with an insecure SSL certificate)
+ Type: String
+ Default: developers.openremote.io
+ HostName:
+ Description: Hostname of the Openremote manager
+ Type: String
+ Default: demo
+ HostedZone:
+ Description: Use AWS hosted zone
+ Type: String
+ Default: true
+ AllowedValues:
+ - true
+ - false
+ KeyName:
+ Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
+ Type: AWS::EC2::KeyPair::KeyName
+ Default: openremote
+ ConstraintDescription: must be the name of an existing EC2 KeyPair.
+ InstanceType:
+ Description: EC2 instance type (don't change it unless you know what you are doing)
+ Type: String
+ # AWS Compute Optimizer reports that this instance type is under-provisioned and advise to use
+ # more than double the price instance. However, it works in our test setup.
+ Default: t4g.small
+ ConstraintDescription: must be a valid EC2 instance type.
+ InstanceAMI:
+ Description: Managed AMI ID for EC2 Instance (don't change it unless you know what you are doing)
+ Type : String
+ Default: ami-03ec287fa560a6ccc
+Conditions:
+ DnsRecordCreate: !Equals [!Ref HostedZone, true]
+ DomainName: !Not [ !Equals [!Ref DomainName, ""] ]
+Outputs:
+ InstanceIP:
+ Description: The Instance public IP
+ Value: !GetAtt EC2Instance.PublicIp
+ PublicUrl:
+ Condition: DnsRecordCreate
+ Description: OpenRemote Instance URL
+ Value: !Join [ ".", [ !Ref HostName, !Ref DomainName ] ]
+ UserId:
+ Description: SMTP server username
+ Value: !Ref AccessKeys
+ UserSecret:
+ Description: SMTP server password
+ Value: !GetAtt AccessKeys.SecretAccessKey
+ SmtpServer:
+ Value: !Join [ ".", [ "email-smtp", !Ref "AWS::Region", "amazonaws.com"]]
+Resources:
+ EmailUser:
+ Type: AWS::IAM::User
+ Properties:
+ Policies:
+ - PolicyName: OpenRemoteSendEmail
+ PolicyDocument:
+ Version: '2012-10-17'
+ Statement:
+ - Effect: Allow
+ Action:
+ - ses:SendRawEmail
+ Resource:
+ - '*'
+ AccessKeys:
+ Type: AWS::IAM::AccessKey
+ Properties:
+ UserName: !Ref EmailUser
+ DNSRecord:
+ Condition: DnsRecordCreate
+ Type: AWS::Route53::RecordSet
+ Properties:
+ HostedZoneName: !Join [ "", [ !Ref DomainName, "." ] ]
+ Name: !Join [ ".", [!Ref HostName, !Ref DomainName ] ]
+ ResourceRecords:
+ - !GetAtt EC2Instance.PublicIp
+ TTL: '60'
+ Type: A
+ LogRole:
+ Type: AWS::IAM::Role
+ Properties:
+ AssumeRolePolicyDocument:
+ Version: '2012-10-17'
+ Statement:
+ - Effect: Allow
+ Principal:
+ Service:
+ - ec2.amazonaws.com
+ Action:
+ - sts:AssumeRole
+ Path: /
+ Policies:
+ - PolicyName: LogRolePolicy
+ PolicyDocument:
+ Version: '2012-10-17'
+ Statement:
+ - Effect: Allow
+ Action:
+ - logs:CreateLogGroup
+ - logs:CreateLogStream
+ - logs:PutLogEvents
+ - logs:DescribeLogStreams
+ Resource:
+ - !Join
+ - ''
+ - - 'arn:aws:'
+ - logs:*:*:*
+ - PolicyName: ECREFSPolicy
+ PolicyDocument:
+ Version: '2012-10-17'
+ Statement:
+ - Effect: Allow
+ Action:
+ - ecr:*
+ - elasticfilesystem:*
+ Resource:
+ - '*'
+ LogRoleInstanceProfile:
+ Type: AWS::IAM::InstanceProfile
+ Properties:
+ Path: /
+ Roles:
+ - !Ref 'LogRole'
+ CloudFormationLogs:
+ Type: AWS::Logs::LogGroup
+ Properties:
+ RetentionInDays: 7
+
+ EC2Instance:
+ Type: AWS::EC2::Instance
+ Metadata:
+ AWS::CloudFormation::Init:
+ configSets:
+ install_all:
+ - install_cfn
+ - install_logs
+ install_cfn:
+ files:
+ /etc/cfn/cfn-hup.conf:
+ content: !Join
+ - ''
+ - - "[main]\n"
+ - stack=
+ - !Ref 'AWS::StackId'
+ - "\n"
+ - region=
+ - !Ref 'AWS::Region'
+ - "\n"
+ mode: '000400'
+ owner: root
+ group: root
+ /etc/cfn/hooks.d/cfn-auto-reloader.conf:
+ content: !Join
+ - ''
+ - - "[cfn-auto-reloader-hook]\n"
+ - "triggers=post.update\n"
+ - "path=Resources.WebServerInstance.Metadata.AWS::CloudFormation::Init\n"
+ - 'action=/opt/aws/bin/cfn-init -v '
+ - ' --stack '
+ - !Ref 'AWS::StackName'
+ - ' --resource EC2Instance '
+ - ' --configsets install_all '
+ - ' --region '
+ - !Ref 'AWS::Region'
+ - "\n"
+ - "runas=root\n"
+ mode: '000400'
+ owner: root
+ group: root
+ services:
+ sysvinit:
+ cfn-hup:
+ enabled: 'true'
+ ensureRunning: 'true'
+ files:
+ - /etc/cfn/cfn-hup.conf
+ - /etc/cfn/hooks.d/cfn-auto-reloader.conf
+ install_logs:
+ files:
+ /etc/awslogs/awslogs.conf:
+ content: !Join
+ - ''
+ - - "[general]\n"
+ - "state_file= /var/awslogs/state/agent-state\n"
+ - "[/var/log/cloud-init.log]\n"
+ - "file = /var/log/cloud-init.log\n"
+ - 'log_group_name = '
+ - !Ref 'CloudFormationLogs'
+ - "\n"
+ - "log_stream_name = {instance_id}/cloud-init.log\n"
+ - "datetime_format = \n"
+ - "[/var/log/cloud-init-output.log]\n"
+ - "file = /var/log/cloud-init-output.log\n"
+ - 'log_group_name = '
+ - !Ref 'CloudFormationLogs'
+ - "\n"
+ - "log_stream_name = {instance_id}/cloud-init-output.log\n"
+ - "datetime_format = \n"
+ - "[/var/log/cfn-init.log]\n"
+ - "file = /var/log/cfn-init.log\n"
+ - 'log_group_name = '
+ - !Ref 'CloudFormationLogs'
+ - "\n"
+ - "log_stream_name = {instance_id}/cfn-init.log\n"
+ - "datetime_format = \n"
+ - "[/var/log/cfn-hup.log]\n"
+ - "file = /var/log/cfn-hup.log\n"
+ - 'log_group_name = '
+ - !Ref 'CloudFormationLogs'
+ - "\n"
+ - "log_stream_name = {instance_id}/cfn-hup.log\n"
+ - "datetime_format = \n"
+ - "[/var/log/cfn-wire.log]\n"
+ - "file = /var/log/cfn-wire.log\n"
+ - 'log_group_name = '
+ - !Ref 'CloudFormationLogs'
+ - "\n"
+ - "log_stream_name = {instance_id}/cfn-wire.log\n"
+ - "datetime_format = \n"
+ mode: '000444'
+ owner: root
+ group: root
+ /etc/awslogs/awscli.conf:
+ content: !Join
+ - ''
+ - - "[plugins]\n"
+ - "cwlogs = cwlogs\n"
+ - "[default]\n"
+ - 'region = '
+ - !Ref 'AWS::Region'
+ - "\n"
+ mode: '000444'
+ owner: root
+ group: root
+ commands:
+ '01_create_state_directory':
+ command: mkdir -p /var/awslogs/state
+ services:
+ sysvinit:
+ awslogs:
+ enabled: 'true'
+ ensureRunning: 'true'
+ files:
+ - /etc/awslogs/awslogs.conf
+ Properties:
+ InstanceType:
+ Ref: InstanceType
+ KeyName:
+ Ref: KeyName
+ ImageId:
+ Ref: InstanceAMI
+ SecurityGroups:
+ - Ref: InstanceSecurityGroup
+ Ipv6AddressCount: 1
+ Tags:
+ - Key: "Name"
+ Value: !Join [ ".", [ !Ref HostName, !Ref DomainName ] ]
+ IamInstanceProfile: !Ref 'LogRoleInstanceProfile'
+ UserData:
+ Fn::Base64: !Sub |
+ #!/bin/bash
+ apt update
+ # Install docker newest version
+ groupadd docker
+ gpasswd -a ubuntu docker
+ apt-get update
+ apt-get install apt-transport-https ca-certificates curl gnupg lsb-release -y
+ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
+ echo "deb [arch=arm64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
+ apt-get update
+ apt-get install docker-ce docker-ce-cli containerd.io -y
+ apt install docker-compose -y
+ # SNMP tools
+ apt install snmp snmpd snmptt snmptrapd -y
+ # Install openremote-cli tool
+ apt install python3-pip -y
+ pip3 install -U openremote-cli
+ # Install AWS CLI
+ apt install awscli -y
+ # CFN tools for CloudWatch
+ pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz
+ cp /usr/local/init/ubuntu/cfn-hup /etc/init.d/cfn-hup
+ chmod +x /etc/init.d/cfn-hup
+ update-rc.d cfn-hup defaults
+ service cfn-hup start
+ echo cfn-init -v --stack ${AWS::StackName} --resource EC2Instance --configsets install_all --region ${AWS::Region}
+ cfn-init -v --stack ${AWS::StackName} --resource EC2Instance --configsets install_all --region ${AWS::Region}
+ echo cfn-signal -e $? --stack ${AWS::StackName} --resource EC2Instance --region ${AWS::Region}
+ cfn-signal -e $? --stack ${AWS::StackName} --resource EC2Instance --region ${AWS::Region}
+ # end CFN
+ # awslogs
+ echo AWS Logs
+ curl https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O
+ # Bump Python version (TODO AWS)
+ sed -i 's/3,6/3,9/g' awslogs-agent-setup.py
+ python3 awslogs-agent-setup.py -n -r ${AWS::Region} -c /etc/awslogs/awslogs.conf
+ # awslogs
+ cd /home/ubuntu
+ # AWS EFS for map
+ apt-get -y install git binutils
+ git clone https://github.com/aws/efs-utils
+ cd efs-utils/
+ ./build-deb.sh
+ apt-get -y install ./build/amazon-efs-utils*deb
+ mkdir /deployment.local
+ mount -t efs -o tls fs-0b1f2c29945f8db97:/ /deployment.local
+ cd ..
+ # AWS EFS for map
+ docker swarm init
+ openremote-cli -V -t
+ # create environment
+ export DEPLOYMENT_VERSION=latest
+ export MANAGER_VERSION=latest
+ echo export AWS_DEFAULT_REGION=${AWS::Region} > environment.txt
+ echo export EMAIL_PASSWORD=$(openremote-cli configure_aws --secret ${AccessKeys.SecretAccessKey} --quiet) >> environment.txt
+ echo export EMAIL_USER=${AccessKeys} >> environment.txt
+ echo export EMAIL_HOST=email-smtp.${AWS::Region}.amazonaws.com >> environment.txt
+ echo export AWSLOGS_GROUP=${CloudFormationLogs} >> environment.txt
+ echo export [email protected] >> environment.txt
+ echo export DEPLOYMENT_HOST=${HostName}.${DomainName} >> environment.txt
+ echo export PASSWORD=${OpenRemotePassword} >> environment.txt
+ echo export [email protected]
+ source environment.txt
+ # docker ECR login
+ `aws ecr get-login --no-include-email`
+ # 90 days personal access token (repo) Expires on Tue, Jan 11 2022
+ curl https://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@raw.githubusercontent.com/openremote/k2/main/aws-docker-compose.yml --output docker-compose.yml
+ docker-compose -p custom up -d
+ chown ubuntu:ubuntu -R .
+
+ InstanceSecurityGroup:
+ Type: AWS::EC2::SecurityGroup
+ Properties:
+ GroupDescription: SSH + HTTP + HTTPS + MQTT + ping + NFS
+ SecurityGroupIngress:
+ - IpProtocol: tcp
+ FromPort: '22'
+ ToPort: '22'
+ CidrIp: 0.0.0.0/0
+ Description: SSH
+ - IpProtocol: tcp
+ FromPort: '80'
+ ToPort: '80'
+ CidrIp: 0.0.0.0/0
+ Description: HTTP
+ - IpProtocol: tcp
+ FromPort: '443'
+ ToPort: '443'
+ CidrIp: 0.0.0.0/0
+ Description: HTTPS
+ - IpProtocol: tcp
+ FromPort: '8883'
+ ToPort: '8883'
+ CidrIp: 0.0.0.0/0
+ Description: MQTT
+ - IpProtocol: udp
+ FromPort: '161'
+ ToPort: '162'
+ CidrIp: 0.0.0.0/0
+ Description: SNMP
+ - IpProtocol: icmp
+ FromPort: '8'
+ ToPort: '-1'
+ CidrIp: 0.0.0.0/0
+ Description: Ping
+ - IpProtocol: tcp
+ FromPort: '2049'
+ ToPort: '2049'
+ CidrIp: 0.0.0.0/0
+ Description: NFS
+ - IpProtocol: tcp
+ FromPort: '2049'
+ ToPort: '2049'
+ CidrIpv6: ::/0
+ Description: NFS
+ - IpProtocol: tcp
+ FromPort: '22'
+ ToPort: '22'
+ CidrIpv6: ::/0
+ Description: SSH
+ - IpProtocol: tcp
+ FromPort: '80'
+ ToPort: '80'
+ CidrIpv6: ::/0
+ Description: HTTP
+ - IpProtocol: tcp
+ FromPort: '443'
+ ToPort: '443'
+ CidrIpv6: ::/0
+ Description: HTTPS
+ - IpProtocol: tcp
+ FromPort: '8883'
+ ToPort: '8883'
+ CidrIpv6: ::/0
+ Description: MQTT
+ - IpProtocol: udp
+ FromPort: '161'
+ ToPort: '162'
+ CidrIpv6: ::/0
+ Description: SNMP
+ - IpProtocol: icmpv6
+ FromPort: '-1'
+ ToPort: '-1'
+ CidrIpv6: ::/0
+ Description: Ping
diff --git a/.ci_cd/ci_cd.json b/.ci_cd/ci_cd.json
index f41cef0f3..326b62104 100644
--- a/.ci_cd/ci_cd.json
+++ b/.ci_cd/ci_cd.json
@@ -3,6 +3,10 @@
"develop": {
"distribute": {
"tags": "develop"
+ },
+ "deploy": {
+ "managerTag": "develop",
+ "environment": "staging"
}
},
"master": {
diff --git a/.ci_cd/deploy.sh b/.ci_cd/deploy.sh
index 9fedea5af..6ed263cd7 100644
--- a/.ci_cd/deploy.sh
+++ b/.ci_cd/deploy.sh
@@ -70,7 +70,7 @@ PLATFORM="linux/$PLATFORM"
# Create docker image tarballs as required
if [ "$MANAGER_TAG" != '#ref' ]; then
docker manifest inspect openremote/manager:$MANAGER_TAG > /dev/null 2> /dev/null
- if [ $? != 0 ]; then
+ if [ $? -ne 0 ]; then
echo "Specified manager tag does not exist in docker hub"
exit 1
fi
@@ -133,42 +133,60 @@ $sshCommandPrefix ${hostStr} << EOF
fi
# Run host init
+ hostInitCmd=
if [ "$HOST_INIT_SCRIPT" == 'NONE' -o "$HOST_INIT_SCRIPT" == 'none' ]; then
echo "No host init requested"
elif [ ! -z "$HOST_INIT_SCRIPT" ]; then
- if [ ! -f ".ci_cd/host_init/${HOST_INIT_SCRIPT}.sh" ]; then
- echo "HOST_INIT_SCRIPT (.ci_cd/host_init/${HOST_INIT_SCRIPT}.sh) does not exist"
+ if [ ! -f "temp/host_init/${HOST_INIT_SCRIPT}.sh" ]; then
+ echo "HOST_INIT_SCRIPT (temp/host_init/${HOST_INIT_SCRIPT}.sh) does not exist"
exit 1
fi
- hostInitCmd=".ci_cd/host_init/${HOST_INIT_SCRIPT}.sh"
- elif [ -f ".ci_cd/host_init/$ENVIRONMENT.sh" ]; then
- hostInitCmd=".ci_cd/host_init/$ENVIRONMENT.sh"
- elif [ -f ".ci_cd/host_init/init.sh" ]; then
- hostInitCmd=".ci_cd/host_init/init.sh"
+ hostInitCmd="temp/host_init/${HOST_INIT_SCRIPT}.sh"
+ elif [ -f "temp/host_init/$ENVIRONMENT.sh" ]; then
+ hostInitCmd="temp/host_init/$ENVIRONMENT.sh"
+ elif [ -f "temp/host_init/init.sh" ]; then
+ hostInitCmd="temp/host_init/init.sh"
fi
- if [ ! -z "$hostInitCmd" ]; then
+ if [ ! -z "$hostInitCmd" ]; then
echo "Running host init script: '$hostInitCmd'"
- $hostInitCmd
+ sudo $hostInitCmd
+ else
+ echo "No host init script"
fi
-
- # MAKE SURE WE HAVE CORRECT KEYCLOAK, PROXY AND POSTGRES IMAGES
+
+ # Make sure we have correct keycloak, proxy and postgres images
+ echo "Pulling requested service versions from docker hub"
docker-compose -p or -f temp/docker-compose.yml pull --ignore-pull-failures
-
+
+ if [ \$? -ne 0 ]; then
+ echo "Deployment failed to pull docker images"
+ exit 1
+ fi
+
# Attempt docker compose down
- docker-compose -f temp/docker-compose.yml -p or down
+ echo "Stopping existing stack"
+ docker-compose -f temp/docker-compose.yml -p or down 2> /dev/null
+
+ if [ \$? -ne 0 ]; then
+ echo "Deployment failed to stop the existing stack"
+ exit 1
+ fi
# Delete postgres volume if CLEAN_INSTALL=true
- if [ $CLEAN_INSTALL == 'true' ]; then
- docker volume rm or_postgresql-data
+ if [ "\$CLEAN_INSTALL" == 'true' ]; then
+ echo "Deleting existing postgres data volume"
+ docker volume rm or_postgresql-data 2> /dev/null
fi
# Delete any deployment volume so we get the latest
- docker volume rm or_deployment-data
+ echo "Deleting existing deployment data volume"
+ docker volume rm or_deployment-data 2 > /dev/null
# Start the stack
+ echo "Starting the stack"
docker-compose -f temp/docker-compose.yml -p or up -d
- if [ $? != 0 ];then
+ if [ \$? -ne 0 ]; then
echo "Deployment failed to start the stack"
exit 1
fi
diff --git a/.ci_cd/gitlab-ci/README.md b/.ci_cd/gitlab-ci/README.md
deleted file mode 100644
index 503db5691..000000000
--- a/.ci_cd/gitlab-ci/README.md
+++ /dev/null
@@ -1,314 +0,0 @@
-# GitLab CI/CD pipeline
-
-There is .gitlab-ci.yml configuration file which is used to build/test and deploy stack in the GitLab enviroment.
-If one imports main repo the pipeline can be started.
-
-## Build docker images
-
-You can skip this stage by adding **skip-build** in the git commit message.
-
-## Test the stack
-
-The second major step in the CI/CD pipeline are functional tests. When extending
-the functionality, e.g. by adding a new protocol, tests covering its behaviour should
-be included to the main branch and executed during each commit and check whether no
-legacy functionality is broken. Sometimes this is intended and in this case rgresssion
-tests should be adjusted to the new situation.
-
-In case that no functionaity is changed, e.g. in case of documentation updates, this stage can
-be skipped by adding **skip-test** in the git commit message.
-
-### Inspect failed test report output
-
-When tests fail and break the pipeline, test reports are stored as artifacts. On the right side of the failed pipleine there is a link
-where the reports can be downloaded.
-
-## Deploy
-
-You can skip this stage by adding **skip-deploy** in the git commit message.
-
-# Deploing OpenRemote stack in the cloud
-
-## Deployment on AWS through the web UI
-
-This is slower prodcedure and meant as en excercise for AWS novice users. Advanced users should use [**aws-cli**](#awscli) method described in the next section.
-
-- Sign in into [AWS web console](https://console.aws.amazon.com/console/home?nc2=h_ct&src=header-signin)
-- Select region to eu-west-1 (Ireland). Currently the cloudformation template supports only this region. *Other regions can be added when
- necessary*.
-- Go to [EC2 KeyPairs](https://eu-west-1.console.aws.amazon.com/ec2/v2/home?region=eu-west-1#KeyPairs:)
- + create or import key pair with name **openremote**
- if you create a new key pair don't forget to download it after creation
-- Go to [CloudFormation conslole](https://eu-west-1.console.aws.amazon.com/cloudformation/home?region=eu-west-1)
- + The top right corner -> Create stack -> With new resources (standard)
- + On the Create stack page select *Upload a template file*
- + *Chose file* and select the openremote/gitlab-ci/aws-cloudformation.template.yml, press *Next*
- + Enter *Stack name*, e.g. **openremote3**
- On *Parameters* check if KeyName has a correct value, default is **openremote** which we used for key generation, press *Next*
- + On *Configure stack options* press *Next*
- + On *Review* page you can check all parameters and press *Create stack*
- You should be on a page where the stack is in CREATE_IN_PROGRESS status.
- In less than a mniute the stack should be in CREATE_COMPLETE state
- Press *Outputs* tab where you should find *The instance public IP*. Copy it to web browser. You will need to wait about 7 minutes before the
- instance initializes itself with OpenRemote containers.
-
-- After you are done with the stack please for to the [Cloudformation](https://eu-west-1.console.aws.amazon.com/cloudformation/home) dashboard and
-delete it. **This will remove all resources and prevent from generating unnecessary costs**.
-
-## <a name="awscli"></a>Deployment on AWS with CLI (advanced)
-
-We will use the [AWS CLI](https://aws.amazon.com/cli/) to create the necessary resources on AWS side.
-Following commands should be executed from openremote/gitlab-ci subdirectory.
-
-We are presenting 3 scenarios:
-1. Deploy OpenRemote stack with domain hosted on AWS
-2. Deploy OpenRemote stack without domain
-3. Deploy OpenRemote stack with domain hosted anywhere
-
-### Prepare the environment
-
-You will need to do this only once per AWS account.
-
-Enter the following commands in your terminal:
-
-```bash
-# Configure the AWS CLI with your AWS region (supported eu-west-1<Ireland>)
-aws configure
-
-# Create SSH keys (it's a pity but AWS EC2 does not accept other type than rsa keys here)
-ssh-keygen -f openremote -t rsa -b 4096 -C "[email protected]"
-
-# Import key pair to AWS
-aws ec2 import-key-pair --key-name openremote --public-key-material fileb://openremote.pub
-
-```
-
-### Scenario 1 - Deploy OpenRemote stack with domain hosted on AWS
-
-In this scenario we have an AWS account with Route53 hosted zone equal to DomainName input parameter. This is the fastest way to create a full
-functional OpenRemote stack with valid SSL key. Within less than 10 minutes you can create a full functional deployment with valid URL.
-
-```bash
-# Create stack
-aws cloudformation create-stack --stack-name OpenRemote --template-body file://gitlab-ci/aws-cloudformation.template.yml --parameters ParameterKey=DomainName,ParameterValue=developers.openremote.io ParameterKey=HostedZone,ParameterValue=true ParameterKey=HostName,ParameterValue=myhost
-
-```
-
-Wait about 7 minutes.
-
-Check the stack:
-```bash
-# Check the stack status. If it is different than CREATE_COMPLETE the you can find more info in the AWS console
-# Services -> CloudFormation -> Stacks -> OpenRemote -> Events tab
-aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].StackStatus"
-
-# Get the URL of the stack (it should be DomainName.HostName i.e. myhost.developers.openremote.io in this example)
-export URL=$(aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].Outputs[?OutputKey=='PublicUrl'].OutputValue" --output text)
-
-# Check if it gives valid response
-curl -L -I $URL
-
-# The output should look like this:
-HTTP/1.1 302 Found
-content-length: 0
-location: https://myhost.developers.openremote.io/
-cache-control: no-cache
-
-HTTP/1.1 302 Found
-location: http://myhost.developers.openremote.io/main
-content-length: 0
-date: Sat, 18 Jul 2020 10:41:15 GMT
-
-HTTP/1.1 302 Found
-content-length: 0
-location: https://myhost.developers.openremote.io/main
-cache-control: no-cache
-
-HTTP/1.1 302 Found
-location: /main/
-content-type: text/html;charset=UTF-8
-content-length: 64
-date: Sat, 18 Jul 2020 10:41:15 GMT
-
-HTTP/1.1 200 OK
-expires: Sat, 18 Jul 2020 22:41:15 GMT
-cache-control: public,max-age=43200,must-revalidate
-pragma:
-accept-ranges: bytes
-date: Sat, 18 Jul 2020 10:41:15 GMT
-etag: W/"index.html-1595005800000"
-last-modified: Fri, 17 Jul 2020 17:10:00 GMT
-content-type: text/html
-content-length: 3627
-```
-
-### Scenario 2 - Deploy OpenRemote stack without domain
-
-In this scenario we are deploing stack without valid domain. It will use public IP dynamically assigned by AWS.
-
-To achieve this we create the stack with empty DomainName and HostName set to localhost.
-
-```bash
-# Create stack
-aws cloudformation create-stack --stack-name OpenRemote --template-body file://gitlab-ci/aws-cloudformation.template.yml --parameters ParameterKey=DomainName,ParameterValue= ParameterKey=HostName,ParameterValue=localhost
-```
-
-Wait about 7 minutes.
-
-Check the stack:
-```bash
-# Check the stack status. If it is different than CREATE_COMPLETE the you can find more info in the AWS console
-# Services -> CloudFormation -> Stacks -> OpenRemote -> Events tab
-aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].StackStatus"
-
-# Get public IP of the stack
-export IP=$(aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].Outputs[?OutputKey=='InstanceIP'].OutputValue" --output text)
-
-# Check if it gives valid response
-curl -L -I $IP
-
-# The output should look like this:
-HTTP/1.1 302 Found
-content-length: 0
-location: https://3.250.36.158/
-cache-control: no-cache
-
-HTTP/1.1 302 Found
-location: http://3.250.36.158/main
-content-length: 0
-date: Sat, 18 Jul 2020 09:12:44 GMT
-
-HTTP/1.1 302 Found
-content-length: 0
-location: https://3.250.36.158/main
-cache-control: no-cache
-
-HTTP/1.1 302 Found
-location: /main/
-content-type: text/html;charset=UTF-8
-content-length: 64
-date: Sat, 18 Jul 2020 09:12:44 GMT
-
-HTTP/1.1 200 OK
-expires: Sat, 18 Jul 2020 21:12:44 GMT
-cache-control: public,max-age=43200,must-revalidate
-pragma:
-accept-ranges: bytes
-date: Sat, 18 Jul 2020 09:12:44 GMT
-etag: W/"index.html-1595005800000"
-last-modified: Fri, 17 Jul 2020 17:10:00 GMT
-content-type: text/html
-content-length: 3627
-```
-
-Of course you can point a browser to this endpoint but you have to accept invalid SSL certificate. Sometimes you need to change browser settings to allow for this.
-
-### Scenario 3 - Deploy OpenRemote stack with domain hosted anywhere
-
-In this scenario we are hosting our DNS records outside of AWS, or on AWS but on a different account. It is possible to create a cloudformation template
-which would work automatically with the second case by using IAM roles, however this is beyond this tutorial. Please conntact OpenRemote if this is your case.
-
-This is the most time consuming options as it can take up to 24 hours to propagate the DNS record if you are using a slow provider. [AWS](https://aws.amazon.com/) and [CloudFlare](https://www.cloudflare.com/) do this
-within few seconds, therefore in dynamic environments we advise to use one of them.
-
-```bash
-# Create stack
-aws cloudformation create-stack --stack-name OpenRemote --template-body file://gitlab-ci/aws-cloudformation.template.yml --parameters ParameterKey=DomainName,ParameterValue=mydomain.com ParameterKey=HostName,ParameterValue=example
-```
-
-Wait about 7 minutes.
-
-Check the stack:
-```bash
-# Check the stack status. If it is different than CREATE_COMPLETE the you can find more info in the AWS console
-# Services -> CloudFormation -> Stacks -> OpenRemote -> Events tab
-aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].StackStatus"
-
-# Get public IP and URL of the stack
-export IP=$(aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].Outputs[?OutputKey=='InstanceIP'].OutputValue" --output text)
-export URL=$(aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].Outputs[?OutputKey=='PublicUrl'].OutputValue" --output text)
-
-# Show public IP and URL
-# It should produce something like 34.244.50.94 example.mydomain.com
-echo $IP $URL
-```
-
-With the data of the last command go to your DNS dashboard and create an A record example.mydomain.com with coresponding IP address.
-
-Wait up to 24 hours for DNS propagation.
-
-```bash
-# Check if DNS is updated
-ping $URL
-# you should get response like
-PING example.mydomain.com (34.244.50.94): 56 data bytes
-64 bytes from 34.244.50.94: icmp_seq=0 ttl=41 time=29.731 ms
-64 bytes from 34.244.50.94: icmp_seq=1 ttl=41 time=37.164 ms
-
-# Update SSL ceriticate
-ssh -i ~/.ssh/openremote ubuntu@$URL docker start openremote_proxy_1
-
-# Check if it gives valid response
-curl -L -I $URL
-
-HTTP/1.1 302 Found
-content-length: 0
-location: https://example.mydomain.com/
-cache-control: no-cache
-
-HTTP/1.1 302 Found
-location: http://example.mydomain.com/main
-content-length: 0
-date: Sat, 18 Jul 2020 12:20:54 GMT
-
-HTTP/1.1 302 Found
-content-length: 0
-location: https://example.mydomain.com/main
-cache-control: no-cache
-
-HTTP/1.1 302 Found
-location: /main/
-content-type: text/html;charset=UTF-8
-content-length: 64
-date: Sat, 18 Jul 2020 12:20:54 GMT
-
-HTTP/1.1 200 OK
-expires: Sun, 19 Jul 2020 00:20:54 GMT
-cache-control: public,max-age=43200,must-revalidate
-pragma:
-accept-ranges: bytes
-date: Sat, 18 Jul 2020 12:20:54 GMT
-etag: W/"index.html-1595005800000"
-last-modified: Fri, 17 Jul 2020 17:10:00 GMT
-content-type: text/html
-content-length: 3627
-```
-
-### Add/remove assets (optional)
-
-TODO
-
-### **IMPORTANT** Delete unused stack
-
-When you are done with the stack don't forget to delete it. Depending on the EC2 instance selected leaving it running incures [costs](https://ec2instances.info/?region=eu-west-1). The default one **t3a.medium** in eu-west-1 region (Ireland) currently costs $0.040800 hourly and when reserved $0.025700 hourly.
-
-```bash
-# Delete unused stack
-aws cloudformation delete-stack --stack-name OpenRemote
-```
-
-## Deployment on Alibaba
-
-TODO
-
-## Deployment on Azure
-
-TODO
-
-## Deployment on GCP
-
-TODO
-
-## Deployment on OVH
-
-TODO
diff --git a/.ci_cd/gitlab-ci/aws-mail.py b/.ci_cd/gitlab-ci/aws-mail.py
deleted file mode 100644
index 20f4650d3..000000000
--- a/.ci_cd/gitlab-ci/aws-mail.py
+++ /dev/null
@@ -1,38 +0,0 @@
-#!/usr/bin/python3
-
-import smtplib
-import os
-
-sender = '[email protected]'
-receivers = [os.environ['WHO']]
-subject = 'Gitlab pipeline FAILED on ' + os.environ['STAGE'] + ': ' + os.environ['WHAT']
-
-print(sender,receivers, subject)
-message = '\r\n'.join(['From: %s' % sender,
- 'To: %s' % receivers[0],
- 'Subject: %s' % subject,
- """
-The GitLab CI/CD pipeline FAILED for {repo}
-
-To skip testing and run complete pipeline without errors please include
-'skip-tests' in the git commit message.
-
-{what}
-{when}
-{who}.
-""".format(when=os.environ['WHEN'], who=os.environ['WHO'], what=os.environ['WHAT'], repo=os.environ['REPO'])
- ])
-
-try:
- server = smtplib.SMTP('email-smtp.eu-west-1.amazonaws.com',587)
- server.ehlo()
- server.starttls()
- server.login(os.environ['SMTP_USERNAME'], os.environ['SMTP_PASSWORD'])
-
- server.sendmail(sender, receivers, message)
- print ("Successfully sent email")
-except Exception as ex:
- print ("Error: unable to send email for "+os.environ['SMTP_USERNAME'])
-
- template = "An exception of type {0} occurred. Arguments:\n{1!r}"
- print (template.format(type(ex).__name__, ex.args))
diff --git a/.ci_cd/gitlab-ci/gitlab-ci-submodule.yml b/.ci_cd/gitlab-ci/gitlab-ci-submodule.yml
deleted file mode 100644
index 8f1eeafee..000000000
--- a/.ci_cd/gitlab-ci/gitlab-ci-submodule.yml
+++ /dev/null
@@ -1,327 +0,0 @@
-default:
- interruptible: true
- image: docker:latest
- services:
- - docker:dind # Docker in docker service
-
-variables:
- ## Following 3 variables should be defined as GitLab CI/CD variables
- #PROJECT_NAME:
- #DEPLOYMENT_HOST_USER:
- #DEPLOYMENT_PORT:
- GIT_SUBMODULE_STRATEGY: normal
- DAST_WEBSITE: https://staging.${PROJECT_NAME}.${DOMAIN}
-
-workflow:
- rules:
- - if: $CI_COMMIT_BRANCH != "master"
- when: never
- - when: always
-
-stages:
- - build_staging
- - build_fail
- - test
- - tests_fail
- - deploy
- - notify_deploy_staging_fail
- - dast
- - build_production
- - deploy_production
- - deploy_production_clean
- - notify_deploy_production
-
-.build_template: &build_definition
- variables:
- SCAN_IMAGE_TAG: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA
- MANAGER_IMAGE_TAG: $CI_REGISTRY_IMAGE/manager:$CI_ENVIRONMENT_SLUG
- DEPLOY_IMAGE_TAG: $CI_REGISTRY_IMAGE/deployment:$CI_ENVIRONMENT_SLUG
- PROXY_IMAGE_TAG: $CI_REGISTRY_IMAGE/proxy:$CI_ENVIRONMENT_SLUG
- POSTGRE_IMAGE_TAG: $CI_REGISTRY_IMAGE/postgresql:$CI_ENVIRONMENT_SLUG
- KEYCLO_IMAGE_TAG: $CI_REGISTRY_IMAGE/keycloak:$CI_ENVIRONMENT_SLUG
- before_script:
- - echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin $CI_REGISTRY
- script:
- - docker run -v $(pwd):/or -v $(pwd)/logs:/root/.npm/_logs registry.gitlab.com/openremote/openremote:master
- /bin/sh -c "./gradlew --console plain clean installDist"
- - docker run -v $(pwd):/or
- -v /var/run/docker.sock:/var/run/docker.sock
- -e "CI_REGISTRY_IMAGE=$CI_REGISTRY_IMAGE"
- -e "CI_ENVIRONMENT=$CI_ENVIRONMENT_SLUG"
- registry.gitlab.com/openremote/openremote:master
- /bin/sh -c "docker-compose -f profile/$CI_ENVIRONMENT_SLUG.yml build"
- - docker images
- # Image for container scanning
- - docker tag $MANAGER_IMAGE_TAG $SCAN_IMAGE_TAG
- - docker push $SCAN_IMAGE_TAG
- # Push to registry
- - docker push "$MANAGER_IMAGE_TAG"
- - docker push "$DEPLOY_IMAGE_TAG"
- - docker push "$PROXY_IMAGE_TAG"
- - docker push "$POSTGRE_IMAGE_TAG"
- - docker push "$KEYCLO_IMAGE_TAG"
- # Tag for possible rollback
- - if [ $CI_COMMIT_TAG ]; then
- - docker tag "$CI_REGISTRY_IMAGE/manager:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/manager:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - docker tag "$CI_REGISTRY_IMAGE/deployment:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/deployment:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - docker tag "$CI_REGISTRY_IMAGE/proxy:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/proxy:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - docker tag "$CI_REGISTRY_IMAGE/postgresql:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/postgresql:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - docker tag "$CI_REGISTRY_IMAGE/keycloak:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/keycloak:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - docker push "$CI_REGISTRY_IMAGE/manager:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - docker push "$CI_REGISTRY_IMAGE/deployment:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - docker push "$CI_REGISTRY_IMAGE/proxy:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - docker push "$CI_REGISTRY_IMAGE/postgresql:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - docker push "$CI_REGISTRY_IMAGE/keycloak:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"
- - fi
- artifacts:
- when: on_failure
- paths:
- - "logs/*"
- expire_in: 4 weeks
-
-.notify_whatsapp: &notify_whatsapp
- image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest
- allow_failure: true
- variables:
- GIT_STRATEGY: none
- script:
- - >
- curl --location --request POST $WHATSAPP_API
- --header 'Content-Type: application/javascript'
- --data-raw "{
- \"message\": \"Your $CI_PROJECT_TITLE+$CI_JOB_STAGE code is $CI_JOB_NAME\",
- \"recipient\": \"$WHATSAPP_TEL\",
- \"user_id\": \"1\"
- }"
- retry:
- max: 2
-
-build staging docker images:
- stage: build_staging
- rules:
- - if: $CI_COMMIT_MESSAGE !~ /skip-build/
- environment: staging
- <<: *build_definition
-
-build manually staging docker images:
- stage: build_staging
- rules:
- - if: $CI_COMMIT_MESSAGE =~ /skip-build/
- when: manual
- allow_failure: true
- environment: staging
- <<: *build_definition
-
-.email_template: &notify_email
- image: docker:latest
- # Do not clone git repo as it can fail and sending mail'll fail too
- variables:
- GIT_SUBMODULE_STRATEGY: none
- before_script:
- - cat $AWS_MAIL > aws_mail.py
- script:
- - docker pull registry.gitlab.com/openremote/openremote:master
- - export WHO=$(docker run --rm -v $(pwd):/or registry.gitlab.com/openremote/openremote:master git log --pretty="%ce" -1)
- - export WHEN=$(docker run --rm -v $(pwd):/or registry.gitlab.com/openremote/openremote:master git log --pretty="%ad %ar" -1)
- - export WHAT=$(docker run --rm -v $(pwd):/or registry.gitlab.com/openremote/openremote:master git log --pretty="%h %s" -1)
- - export REPO=$(docker run --rm -v $(pwd):/or registry.gitlab.com/openremote/openremote:master git remote get-url origin|xargs basename)
- - echo $REPO
- - docker run -v $(pwd):/usr/src/myapp -w /usr/src/myapp
- -e "STAGE=$CI_JOB_STAGE"
- -e "WHO=$WHO"
- -e "WHAT=$WHAT"
- -e "WHEN=$WHEN"
- -e "REPO=$REPO"
- -e "SMTP_USERNAME=$SMTP_USERNAME"
- -e "SMTP_PASSWORD=$SMTP_PASSWORD"
- --rm python:3 python aws_mail.py
- retry:
- max: 2
-
-# This step is executed when build fails. The purpose is to e-mail a git commiter about the problem.
-email failed pipeline:
- stage: build_fail
- needs: ["build staging docker images"]
- <<: *notify_email
- when: on_failure
-
-build FAILED:
- stage: build_fail
- needs: ["build staging docker images"]
- <<: *notify_whatsapp
- when: on_failure
-
-test-master:
- stage: test
- needs: []
- allow_failure: false
- before_script:
- - docker run -v $(pwd):/or
- -v /var/run/docker.sock:/var/run/docker.sock registry.gitlab.com/openremote/openremote:master
- /bin/sh -c "docker-compose -f openremote/profile/dev-testing.yml pull"
- - docker run -v $(pwd):/or
- -v /var/run/docker.sock:/var/run/docker.sock registry.gitlab.com/openremote/openremote:master
- /bin/sh -c "docker-compose -f openremote/profile/dev-testing.yml up --no-build -d"
- script:
- #- docker run --network host -v $(pwd):/or registry.gitlab.com/openremote/openremote:master
- # /bin/sh -c "./gradlew clean test"
- - echo Tests moved to github actions
- rules:
- - if: $CI_COMMIT_MESSAGE !~ /skip-test/
- when: on_success
- artifacts:
- when: on_failure
- paths:
- - "openremote/test/build/reports/tests/test/*"
- - "openremote/test/build/reports/tests/test/*/*"
- expire_in: 4 weeks
- timeout: 30m
-
-email failed tests:
- stage: tests_fail
- needs: ["test-master"]
- <<: *notify_email
- rules:
- - if: $CI_COMMIT_MESSAGE !~ /skip-test/
- when: on_failure
-
-test FAILED:
- stage: tests_fail
- needs: ["test-master"]
- <<: *notify_whatsapp
- rules:
- - if: $CI_COMMIT_MESSAGE !~ /skip-test/
- when: on_failure
-
-include:
- - template: Container-Scanning.gitlab-ci.yml
- - template: Dependency-Scanning.gitlab-ci.yml
- - template: License-Scanning.gitlab-ci.yml
- # - template: SAST.gitlab-ci.yml
- - template: DAST.gitlab-ci.yml
-
-.ssh_config: &ssh_config
- - mkdir -p ~/.ssh
- - chmod 600 "$GITLAB_SSH_PRIVATE_KEY"
- - eval "$(ssh-agent -s)"
- - ssh-add "$GITLAB_SSH_PRIVATE_KEY"
- - ssh-keyscan -H -p $DEPLOYMENT_PORT $DEPLOYMENT_HOST >> ~/.ssh/known_hosts
- - alias SSH="ssh -p $DEPLOYMENT_PORT ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}"
- - alias SCP="scp -p -r -P $DEPLOYMENT_PORT"
-
-.deploy_template: &deploy_definition
- before_script:
- - *ssh_config
- script:
- - echo "Deploy to $CI_ENVIRONMENT_SLUG server"
- - echo "export PROJECT_NAME=$PROJECT_NAME" >> deploy.sh
- - echo "export CI_REGISTRY_IMAGE=$CI_REGISTRY_IMAGE" >> deploy.sh
- - if [ ! $DOCKER_IMAGE_TAG ]; then echo "export CI_ENVIRONMENT=$CI_ENVIRONMENT_SLUG" >> deploy.sh; fi
- - if [ $DOCKER_IMAGE_TAG ]; then echo "export CI_ENVIRONMENT=$DOCKER_IMAGE_TAG" >> deploy.sh; fi
- - echo "export SETUP_WIPE_CLEAN_INSTALL=$SETUP_WIPE_CLEAN_INSTALL" >> deploy.sh
- - if [ $SETUP_ADMIN_PASSWORD ]; then echo "export SETUP_ADMIN_PASSWORD=$SETUP_ADMIN_PASSWORD" >> deploy.sh; fi
- - echo "echo ${OR_PASSWORD} | docker login -u openremote --password-stdin $CI_REGISTRY || exit 666" >> deploy.sh
- - echo "docker pull $CI_REGISTRY_IMAGE/manager:\$CI_ENVIRONMENT" >> deploy.sh
- - echo "docker pull $CI_REGISTRY_IMAGE/deployment:\$CI_ENVIRONMENT">> deploy.sh
- - echo "docker pull $CI_REGISTRY_IMAGE/proxy:\$CI_ENVIRONMENT" >> deploy.sh
- - echo "docker pull $CI_REGISTRY_IMAGE/postgresql:\$CI_ENVIRONMENT">> deploy.sh
- - echo "docker pull $CI_REGISTRY_IMAGE/keycloak:\$CI_ENVIRONMENT" >> deploy.sh
- # Dump DB if it exists for possible data restoration
- - echo "if [ \"\$(docker ps --filter name=${PROJECT_NAME}_postgresql_1 -q)\" ]; then docker exec ${PROJECT_NAME}_postgresql_1 pg_dump -U postgres openremote > dump/\$RANDOM.sql; fi || exit 666" >> deploy.sh
- - echo "docker-compose -p $PROJECT_NAME -f profile/$CI_ENVIRONMENT_SLUG.yml down || exit 666" >> deploy.sh
- - echo "docker volume rm ${PROJECT_NAME}_deployment-data" >> deploy.sh
- - echo "docker-compose -p $PROJECT_NAME -f profile/$CI_ENVIRONMENT_SLUG.yml up --no-build -d || exit 666" >> deploy.sh
- - echo "docker cp FIREBASE_CONFIG_FILE ${PROJECT_NAME}_manager_1:/deployment/manager/fcm.json || exit 666" >> deploy.sh
- - echo "docker restart ${PROJECT_NAME}_manager_1 || exit 666" >> deploy.sh
- - chmod +x deploy.sh
- - SSH mkdir -p dump profile openremote/profile deployment openremote/proxy openremote/manager/build/install/manager openremote/postgresql openremote/keycloak
- - SCP ./profile/ deploy.sh ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}:~/
- - SCP ./openremote/profile/* ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}:~/openremote/profile/
- - SCP ${FIREBASE_CONFIG_FILE} ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}:~/FIREBASE_CONFIG_FILE
- - SSH ./deploy.sh
- - SSH "docker system prune -f"
- after_script:
- - *ssh_config
- - SSH "docker-compose -p ${PROJECT_NAME} -f profile/${CI_ENVIRONMENT_SLUG}.yml logs --no-color --timestamp > ${CI_ENVIRONMENT_SLUG}.log"
- - SCP ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}:~/${CI_ENVIRONMENT_SLUG}.log .
- - tail -n 100 ${CI_ENVIRONMENT_SLUG}.log
- - SSH "rm ~/${CI_ENVIRONMENT_SLUG}.log"
- artifacts:
- when: on_failure
- paths:
- - ${CI_ENVIRONMENT_SLUG}.log
- expire_in: 4 weeks
-
-deploy staging:
- stage: deploy
- variables:
- DEPLOYMENT_HOST: ${CI_ENVIRONMENT_SLUG}.${PROJECT_NAME}.${DOMAIN}
- SETUP_WIPE_CLEAN_INSTALL: "true"
- environment:
- name: staging
- url: https://${CI_ENVIRONMENT_SLUG}.${PROJECT_NAME}.${DOMAIN}
- <<: *deploy_definition
-
-deploy staging when tests fail:
- stage: deploy
- needs: ["email failed tests"]
- when: manual
- variables:
- DEPLOYMENT_HOST: ${CI_ENVIRONMENT_SLUG}.${PROJECT_NAME}.${DOMAIN}
- SETUP_WIPE_CLEAN_INSTALL: "true"
- environment:
- name: staging
- url: https://${CI_ENVIRONMENT_SLUG}.${PROJECT_NAME}.${DOMAIN}
- <<: *deploy_definition
-
-email failed deploy staging:
- stage: notify_deploy_staging_fail
- needs: ["deploy staging"]
- <<: *notify_email
- when: on_failure
-
-deploy staging FAILED:
- stage: notify_deploy_staging_fail
- needs: ["deploy staging"]
- <<: *notify_whatsapp
- when: on_failure
-
-build production docker images:
- stage: build_production
- when: manual
- allow_failure: false
- environment: production
- <<: *build_definition
-
-deploy production:
- stage: deploy_production
- needs: ["build production docker images"]
- variables:
- DEPLOYMENT_HOST: ${PROJECT_NAME}.${DOMAIN}
- SETUP_WIPE_CLEAN_INSTALL: "false"
- environment:
- name: production
- url: https://${PROJECT_NAME}.${DOMAIN}
- <<: *deploy_definition
-
-deploy production clean:
- stage: deploy_production_clean
- when: on_failure
- needs: ["build production docker images"]
- variables:
- DEPLOYMENT_HOST: ${PROJECT_NAME}.${DOMAIN}
- SETUP_WIPE_CLEAN_INSTALL: "true"
- environment:
- name: production
- url: https://${PROJECT_NAME}.${DOMAIN}
- <<: *deploy_definition
-
-deploy production PASSED:
- stage: notify_deploy_production
- <<: *notify_whatsapp
-
-deploy production FAILED:
- stage: notify_deploy_production
- needs: ["build production docker images"]
- <<: *notify_whatsapp
- when: on_failure
diff --git a/.ci_cd/host_init/init.sh b/.ci_cd/host_init/init.sh
index ad0db2a9d..050fdb7e9 100644
--- a/.ci_cd/host_init/init.sh
+++ b/.ci_cd/host_init/init.sh
@@ -1,3 +1,6 @@
#!/bin/bash
-sudo rm /etc/cron.d/openremote
-sudo echo "0 5 * * * docker restart or_manager_1" >> /etc/cron.d/openremote
+#
+# Create CRON JOB to restart manager daily at 05:00
+#
+rm /etc/cron.d/openremote
+echo "0 5 * * * docker restart or_manager_1" >> /etc/cron.d/openremote
diff --git a/.github/workflows/ci_cd.yml b/.github/workflows/ci_cd.yml
index 8b99f4dd2..98ee13467 100644
--- a/.github/workflows/ci_cd.yml
+++ b/.github/workflows/ci_cd.yml
@@ -1,3 +1,52 @@
+# This workflow handles all CI/CD related tasks and can be re-used in custom projects
+# (https://github.com/openremote/custom-project i.e. has openremote submodule) or forks of this repo, it handles:
+#
+# - Running tests on push/release
+# - Distributing openremote/manager images to docker hub (DOCKERHUB_USER and DOCKERHUB_PASSWORD must be set) on push/release
+# - Deploying to hosts on push/release and/or manual trigger and/or when certain openremote/manager docker tags are updated
+#
+# By default this workflow will just run tests on push/release; distribution (pushing of openremote/manager docker image)
+# and deployment behaviour is configured via the .ci_cd/ci_cd.json file (see .ci_cd/README.md for more details).
+#
+# DEPLOYMENTS
+#
+# When a deployment is requested the repo is checked for a 'deployment/Dockerfile' and if that exists then also
+# 'deployment/build.gradle' must exist with an installDist task that prepares the deployment image in the
+# 'deployment/build' directory; if this condition is not met the deployment will fail.
+#
+# Secrets, inputs and environment variables are combined for consumption by the '.ci_cd/deploy.sh' bash script, they are
+# combined in the following priority:
+#
+# - '.ci_cd/env/.env'
+# - '.ci_cd/env/{ENVIRONMENT}.env'
+# - secrets without '_' prefix
+# - secrets with '_{ENVIRONMENT}_' prefix (secrets with any other '_' prefix will be excluded)
+# - inputs (from manual trigger using workflow_dispatch)
+#
+# The above variables are output to 'temp/env' file for each requested deployment. If a secret called 'SSH_PASSWORD' is
+# found that is output to 'ssh.env' file (so it is not copied to the host), if a secret called 'SSH_KEY' is
+# found that is output to 'ssh.key' file (so it is not copied to the host). Sensitive credentials should always be stored
+# in github secrets so they are encrypted, do not store within repo files (even private repos as these are not encrypted).
+#
+# SEE https://github.com/openremote/openremote/tree/master/.ci_cd for details of standard variables and handling.
+#
+# MANUAL TRIGGER
+#
+# This workflow can be triggered by push, release, manually or schedule (from any repo other than openremote/openremote)
+# if triggered manually then the following inputs are available and these override any set in env files and/or secrets:
+#
+# Inputs:
+#
+# - ENVIRONMENT - Which environment to deploy (equivalent to deploy/environment in '.ci_cd/ci_cd.json')
+# - MANAGER_TAG - Which manager docker tag to deploy (equivalent to deploy/managerTags in '.ci_cd/ci_cd.json')
+# - CLEAN_INSTALL
+# - COMMIT
+# - HOST
+# - SSH_USER
+# - SSH_PASSWORD
+# - SSH_PORT
+# - SETUP_ADMIN_PASSWORD
+
name: CI/CD
on:
@@ -26,18 +75,18 @@ on:
CLEAN_INSTALL:
description: 'Delete data before starting'
type: boolean
+ COMMIT:
+ description: 'Repo branch or commit SHA to checkout'
HOST:
description: 'Host to deploy to'
- SSH_USER:
- description: 'SSH Username'
SSH_PORT:
description: 'SSH port'
+ SSH_USER:
+ description: 'SSH Username'
SSH_PASSWORD:
description: 'SSH password'
SETUP_ADMIN_PASSWORD:
description: 'Admin password override'
- COMMIT:
- description: 'Repo branch or commit SHA to use'
workflow_call:
inputs: