100% open-source IoT Platform - Integrate your devices, add intelligence, build custom apps
Java asset-managementbuildingdashboarddataplatformdevice-managementedge-computingenergyenergy-managementenergy-monitorfleet-managementfleet-telematicsiotiot-applicationiot-platformmodbusmqttmqtt-brokerota-updaterules-enginesmartcity
CI/CD documentation and minor updates
Rich Turner0cdf4f8ac8c3
diff --git a/.ci_cd/README.md b/.ci_cd/README.mdindex dd22cd7ba..d78016d09 100644--- a/.ci_cd/README.md+++ b/.ci_cd/README.md@@ -1,15 +1,79 @@# CI/CD workflow files-The files in this directory are used by the [deploy workflow](../.github/workflows/deploy.yml) which provides a standard-github workflow for deploying this repo on a linux host via SSH.+The files in this directory are used by the [CI/CD workflow](../.github/workflows/ci_cd.yml) which provides a standard+github workflow for CI/CD tasks in this repo and custom projects/forks on a linux host via SSH.-# `/env` Directory-Environment variable files that will be loaded into the workflow to configure the behaviour of the workflow and are also-passed through to the host and loaded whenever remote commands are executed via SSH on the host.+# `ci_cd.json` File+Configuration file for distribution (pushing openremote/manager docker images) and deploying for this specific+repository; the file is only required if automatic deployments are desired during push/release and/or openremote/manager+docker image tag updates. The file layout is:-The naming convention should be as follows:+```json+{+ // Configure behaviour on push+ "push": {+ // Name of branch to react to (exact match)+ "develop": {+ // Comma separated list of tags to push to openremote/manager docker hub image $version is replaced with release version+ // This can only be used on the main openremote repo (openremote/openremote)+ "distribute": {+ "tags": "develop"+ },+ // Singleton or array of deployments to execute, a deployment consists of (environment and/or managerTag see variables for explanation)+ "deploy": {+ "managerTag": "develop", // Can use #ref or exclude this to build COMMIT specific maanger docker image+ "environment": "staging"+ }+ },+ "master": {+ "distribute": {+ "tags": "latest"+ }+ }+ },+ // Configure behaviour on release+ "release": {+ // Comma separated list of tags to push to openremote/manager docker hub image $version is replaced with release version+ // This can only be used on the main openremote repo (openremote/openremote)+ "distribute": {+ "tags": "latest,$version"+ },+ // Singleton or array of deployments to execute, a deployment consists of (environment and/or managerTag see variables for explanation)+ "deploy": {+ "managerTag": "latest", // Can use #ref or exclude this to build COMMIT specific maanger docker image+ "environment": "production"+ }+ },+ // Configure behaviour on manager docker image update (must be triggered by schedule event from a custom project or fork)+ "managerDockerPush": {+ // Docker tags to monitor can only be used if not the main openremote repo (openremote/openremote)+ "develop": {+ // Singleton or array of deployments to execute, a deployment consists of (environment and/or managerTag see variables for explanation)+ "deploy": {+ "environment": "staging"+ }+ }+ }+}+```++# `deploy.sh` Bash script+Bash script that handles the actual deployment to a specific host; environment variables should be already loaded into+the shell or be available in `temp/env` file which will be automatically loaded. If `ssh.env` is found this env file is+also automatically loaded. See [Variables section below](#variables) for available and required values.++## Docker compose file resolution+The docker compose file used for the deployment is resolved as follows; if resolved file does not exist then deployment+will fail:+* `ENV_COMPOSE_FILE` variable+* `profile/${ENVIRONMENT}.yml` file+* `docker-compose.yml` file-* `env/.env` - File containing environment variables that will be loaded for any deployment-* `${environment}.env` - File containing environment specific environment variables will be loaded if `environment` input+# `env` Directory+Environment variable files that will be loaded when deploying.++The naming convention should be as follows:+* `env/.env` - File containing environment variables that will be loaded for all deployments+* `${ENVIRONMENT}.env` - File containing environment specific environment variables will be loaded if `ENVIRONMENT` variableis set for the workflow run; these will be loaded after the `env` file and so can override any value defined there as wellas being able to add new environment specific values@@ -17,10 +81,34 @@ as being able to add new environment specific valuesContains scripts/files required to initialise the host ready for running the stack (e.g. download map tiles etc.);the workflow looks for one of the following bash scripts (in priority order):-* `host_init/${environment}.sh` - Bash script for environment specific host initialisation+* `host_init/${ENVIRONMENT}.sh` - Bash script for environment specific host initialisation* `host_init/init.sh` - Bash script that can be used by any environment as a fallback if no environment specific scriptexistsIf an initialisation script is found then the entire `host_init` directory will be copied to the host and the scriptwill be executed; by copying the entire directory additional files can be included and called from the initialisationscript as required.+++# Variables+The following variables are supported by deployments; any additional variables can also be used and these are made+available to the shell where the docker compose stack is brought up; so any variables required in the docker compose+profile can be specified in any of the places variables are loaded from (github secrets, inputs, and/or env files).++* `ENVIRONMENT` - Used to control which env file(s), docker compose file and github secrets to use for deployment.+* `MANAGER_TAG` - The docker tag to pull for the manager image (if not specified)+* `CLEAN_INSTALL` - Indicates if the or_postgresql-data volume should be removed before starting the stack+* `COMMIT` - Which commit or branch to checkout and use on this repo (default: branch/SHA on which workflow is executed)+* `HOST` - FQDN for the host (default: env/secrets.HOST)+* `SSH_USER` - SSH username (default: root)+* `SSH_KEY` - SSH private key (this is written to `ssh.key` file and loaded by `deploy.sh` for SSH/SCP commands)+* `SSH_PASSWORD` - SSH password (alternative to private key authentication)+* `SSH_PORT` - SSH port (default: 22)+* `SETUP_ADMIN_PASSWORD` - Admin password to be set on deployment; sets the SETUP_ADMIN_PASSWORD env variable+* `ENV_COMPOSE_FILE` - The full path to the docker compose file to use for the deployment++## Github secrets+Variables are also loaded from Github secrets using the '_{ENVIRONMENT}_' prefix naming convention (note it doesn't use+github environment secrets as these only seem to be supported for public repositories); any secrets that begin with `_`+will only be loaded if they begin with `_{ENVIRONMENT}_` and this prefix will be removed, all other secrets will be+loaded independent of the `ENVIRONMENT` variable.diff --git a/.ci_cd/gitlab-ci/aws-cloudformation.template.arm64.yml b/.ci_cd/aws/aws-cloudformation.template.arm64.ymlsimilarity index 100%rename from .ci_cd/gitlab-ci/aws-cloudformation.template.arm64.ymlrename to .ci_cd/aws/aws-cloudformation.template.arm64.ymldiff --git a/.ci_cd/aws/aws-cloudformation.template.yml b/.ci_cd/aws/aws-cloudformation.template.ymlnew file mode 100644index 000000000..f646de221--- /dev/null+++ b/.ci_cd/aws/aws-cloudformation.template.yml@@ -0,0 +1,413 @@+---+AWSTemplateFormatVersion: '2010-09-09'+Description: 'Custom project OpenRemote 3 stack on AWS'+Parameters:+ OpenRemotePassword:+ Description: Password for OpenRemote admin login+ Type: String+ Default: secret+ DomainName:+ Description: DNS domain for created stack (clear it is you want the deployment with an insecure SSL certificate)+ Type: String+ Default: developers.openremote.io+ HostName:+ Description: Hostname of the Openremote manager+ Type: String+ Default: demo+ HostedZone:+ Description: Use AWS hosted zone+ Type: String+ Default: true+ AllowedValues:+ - true+ - false+ KeyName:+ Description: Name of an existing EC2 KeyPair to enable SSH access to the instance+ Type: AWS::EC2::KeyPair::KeyName+ Default: openremote+ ConstraintDescription: must be the name of an existing EC2 KeyPair.+ InstanceType:+ Description: EC2 instance type (don't change it unless you know what you are doing)+ Type: String+ # AWS Compute Optimizer reports that this instance type is under-provisioned and advise to use+ # more than double the price instance. However, it works in our test setup.+ Default: t4g.small+ ConstraintDescription: must be a valid EC2 instance type.+ InstanceAMI:+ Description: Managed AMI ID for EC2 Instance (don't change it unless you know what you are doing)+ Type : String+ Default: ami-03ec287fa560a6ccc+Conditions:+ DnsRecordCreate: !Equals [!Ref HostedZone, true]+ DomainName: !Not [ !Equals [!Ref DomainName, ""] ]+Outputs:+ InstanceIP:+ Description: The Instance public IP+ Value: !GetAtt EC2Instance.PublicIp+ PublicUrl:+ Condition: DnsRecordCreate+ Description: OpenRemote Instance URL+ Value: !Join [ ".", [ !Ref HostName, !Ref DomainName ] ]+ UserId:+ Description: SMTP server username+ Value: !Ref AccessKeys+ UserSecret:+ Description: SMTP server password+ Value: !GetAtt AccessKeys.SecretAccessKey+ SmtpServer:+ Value: !Join [ ".", [ "email-smtp", !Ref "AWS::Region", "amazonaws.com"]]+Resources:+ EmailUser:+ Type: AWS::IAM::User+ Properties:+ Policies:+ - PolicyName: OpenRemoteSendEmail+ PolicyDocument:+ Version: '2012-10-17'+ Statement:+ - Effect: Allow+ Action:+ - ses:SendRawEmail+ Resource:+ - '*'+ AccessKeys:+ Type: AWS::IAM::AccessKey+ Properties:+ UserName: !Ref EmailUser+ DNSRecord:+ Condition: DnsRecordCreate+ Type: AWS::Route53::RecordSet+ Properties:+ HostedZoneName: !Join [ "", [ !Ref DomainName, "." ] ]+ Name: !Join [ ".", [!Ref HostName, !Ref DomainName ] ]+ ResourceRecords:+ - !GetAtt EC2Instance.PublicIp+ TTL: '60'+ Type: A+ LogRole:+ Type: AWS::IAM::Role+ Properties:+ AssumeRolePolicyDocument:+ Version: '2012-10-17'+ Statement:+ - Effect: Allow+ Principal:+ Service:+ - ec2.amazonaws.com+ Action:+ - sts:AssumeRole+ Path: /+ Policies:+ - PolicyName: LogRolePolicy+ PolicyDocument:+ Version: '2012-10-17'+ Statement:+ - Effect: Allow+ Action:+ - logs:CreateLogGroup+ - logs:CreateLogStream+ - logs:PutLogEvents+ - logs:DescribeLogStreams+ Resource:+ - !Join+ - ''+ - - 'arn:aws:'+ - logs:*:*:*+ - PolicyName: ECREFSPolicy+ PolicyDocument:+ Version: '2012-10-17'+ Statement:+ - Effect: Allow+ Action:+ - ecr:*+ - elasticfilesystem:*+ Resource:+ - '*'+ LogRoleInstanceProfile:+ Type: AWS::IAM::InstanceProfile+ Properties:+ Path: /+ Roles:+ - !Ref 'LogRole'+ CloudFormationLogs:+ Type: AWS::Logs::LogGroup+ Properties:+ RetentionInDays: 7++ EC2Instance:+ Type: AWS::EC2::Instance+ Metadata:+ AWS::CloudFormation::Init:+ configSets:+ install_all:+ - install_cfn+ - install_logs+ install_cfn:+ files:+ /etc/cfn/cfn-hup.conf:+ content: !Join+ - ''+ - - "[main]\n"+ - stack=+ - !Ref 'AWS::StackId'+ - "\n"+ - region=+ - !Ref 'AWS::Region'+ - "\n"+ mode: '000400'+ owner: root+ group: root+ /etc/cfn/hooks.d/cfn-auto-reloader.conf:+ content: !Join+ - ''+ - - "[cfn-auto-reloader-hook]\n"+ - "triggers=post.update\n"+ - "path=Resources.WebServerInstance.Metadata.AWS::CloudFormation::Init\n"+ - 'action=/opt/aws/bin/cfn-init -v '+ - ' --stack '+ - !Ref 'AWS::StackName'+ - ' --resource EC2Instance '+ - ' --configsets install_all '+ - ' --region '+ - !Ref 'AWS::Region'+ - "\n"+ - "runas=root\n"+ mode: '000400'+ owner: root+ group: root+ services:+ sysvinit:+ cfn-hup:+ enabled: 'true'+ ensureRunning: 'true'+ files:+ - /etc/cfn/cfn-hup.conf+ - /etc/cfn/hooks.d/cfn-auto-reloader.conf+ install_logs:+ files:+ /etc/awslogs/awslogs.conf:+ content: !Join+ - ''+ - - "[general]\n"+ - "state_file= /var/awslogs/state/agent-state\n"+ - "[/var/log/cloud-init.log]\n"+ - "file = /var/log/cloud-init.log\n"+ - 'log_group_name = '+ - !Ref 'CloudFormationLogs'+ - "\n"+ - "log_stream_name = {instance_id}/cloud-init.log\n"+ - "datetime_format = \n"+ - "[/var/log/cloud-init-output.log]\n"+ - "file = /var/log/cloud-init-output.log\n"+ - 'log_group_name = '+ - !Ref 'CloudFormationLogs'+ - "\n"+ - "log_stream_name = {instance_id}/cloud-init-output.log\n"+ - "datetime_format = \n"+ - "[/var/log/cfn-init.log]\n"+ - "file = /var/log/cfn-init.log\n"+ - 'log_group_name = '+ - !Ref 'CloudFormationLogs'+ - "\n"+ - "log_stream_name = {instance_id}/cfn-init.log\n"+ - "datetime_format = \n"+ - "[/var/log/cfn-hup.log]\n"+ - "file = /var/log/cfn-hup.log\n"+ - 'log_group_name = '+ - !Ref 'CloudFormationLogs'+ - "\n"+ - "log_stream_name = {instance_id}/cfn-hup.log\n"+ - "datetime_format = \n"+ - "[/var/log/cfn-wire.log]\n"+ - "file = /var/log/cfn-wire.log\n"+ - 'log_group_name = '+ - !Ref 'CloudFormationLogs'+ - "\n"+ - "log_stream_name = {instance_id}/cfn-wire.log\n"+ - "datetime_format = \n"+ mode: '000444'+ owner: root+ group: root+ /etc/awslogs/awscli.conf:+ content: !Join+ - ''+ - - "[plugins]\n"+ - "cwlogs = cwlogs\n"+ - "[default]\n"+ - 'region = '+ - !Ref 'AWS::Region'+ - "\n"+ mode: '000444'+ owner: root+ group: root+ commands:+ '01_create_state_directory':+ command: mkdir -p /var/awslogs/state+ services:+ sysvinit:+ awslogs:+ enabled: 'true'+ ensureRunning: 'true'+ files:+ - /etc/awslogs/awslogs.conf+ Properties:+ InstanceType:+ Ref: InstanceType+ KeyName:+ Ref: KeyName+ ImageId:+ Ref: InstanceAMI+ SecurityGroups:+ - Ref: InstanceSecurityGroup+ Ipv6AddressCount: 1+ Tags:+ - Key: "Name"+ Value: !Join [ ".", [ !Ref HostName, !Ref DomainName ] ]+ IamInstanceProfile: !Ref 'LogRoleInstanceProfile'+ UserData:+ Fn::Base64: !Sub |+ #!/bin/bash+ apt update+ # Install docker newest version+ groupadd docker+ gpasswd -a ubuntu docker+ apt-get update+ apt-get install apt-transport-https ca-certificates curl gnupg lsb-release -y+ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg+ echo "deb [arch=arm64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null+ apt-get update+ apt-get install docker-ce docker-ce-cli containerd.io -y+ apt install docker-compose -y+ # SNMP tools+ apt install snmp snmpd snmptt snmptrapd -y+ # Install openremote-cli tool+ apt install python3-pip -y+ pip3 install -U openremote-cli+ # Install AWS CLI+ apt install awscli -y+ # CFN tools for CloudWatch+ pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz+ cp /usr/local/init/ubuntu/cfn-hup /etc/init.d/cfn-hup+ chmod +x /etc/init.d/cfn-hup+ update-rc.d cfn-hup defaults+ service cfn-hup start+ echo cfn-init -v --stack ${AWS::StackName} --resource EC2Instance --configsets install_all --region ${AWS::Region}+ cfn-init -v --stack ${AWS::StackName} --resource EC2Instance --configsets install_all --region ${AWS::Region}+ echo cfn-signal -e $? --stack ${AWS::StackName} --resource EC2Instance --region ${AWS::Region}+ cfn-signal -e $? --stack ${AWS::StackName} --resource EC2Instance --region ${AWS::Region}+ # end CFN+ # awslogs+ echo AWS Logs+ curl https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O+ # Bump Python version (TODO AWS)+ sed -i 's/3,6/3,9/g' awslogs-agent-setup.py+ python3 awslogs-agent-setup.py -n -r ${AWS::Region} -c /etc/awslogs/awslogs.conf+ # awslogs+ cd /home/ubuntu+ # AWS EFS for map+ apt-get -y install git binutils+ git clone https://github.com/aws/efs-utils+ cd efs-utils/+ ./build-deb.sh+ apt-get -y install ./build/amazon-efs-utils*deb+ mkdir /deployment.local+ mount -t efs -o tls fs-0b1f2c29945f8db97:/ /deployment.local+ cd ..+ # AWS EFS for map+ docker swarm init+ openremote-cli -V -t+ # create environment+ export DEPLOYMENT_VERSION=latest+ export MANAGER_VERSION=latest+ echo export AWS_DEFAULT_REGION=${AWS::Region} > environment.txt+ echo export EMAIL_PASSWORD=$(openremote-cli configure_aws --secret ${AccessKeys.SecretAccessKey} --quiet) >> environment.txt+ echo export EMAIL_USER=${AccessKeys} >> environment.txt+ echo export EMAIL_HOST=email-smtp.${AWS::Region}.amazonaws.com >> environment.txt+ echo export AWSLOGS_GROUP=${CloudFormationLogs} >> environment.txt+ echo export [email protected] >> environment.txt+ echo export DEPLOYMENT_HOST=${HostName}.${DomainName} >> environment.txt+ echo export PASSWORD=${OpenRemotePassword} >> environment.txt+ echo export [email protected]+ source environment.txt+ # docker ECR login+ `aws ecr get-login --no-include-email`+ # 90 days personal access token (repo) Expires on Tue, Jan 11 2022+ curl https://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@raw.githubusercontent.com/openremote/k2/main/aws-docker-compose.yml --output docker-compose.yml+ docker-compose -p custom up -d+ chown ubuntu:ubuntu -R .++ InstanceSecurityGroup:+ Type: AWS::EC2::SecurityGroup+ Properties:+ GroupDescription: SSH + HTTP + HTTPS + MQTT + ping + NFS+ SecurityGroupIngress:+ - IpProtocol: tcp+ FromPort: '22'+ ToPort: '22'+ CidrIp: 0.0.0.0/0+ Description: SSH+ - IpProtocol: tcp+ FromPort: '80'+ ToPort: '80'+ CidrIp: 0.0.0.0/0+ Description: HTTP+ - IpProtocol: tcp+ FromPort: '443'+ ToPort: '443'+ CidrIp: 0.0.0.0/0+ Description: HTTPS+ - IpProtocol: tcp+ FromPort: '8883'+ ToPort: '8883'+ CidrIp: 0.0.0.0/0+ Description: MQTT+ - IpProtocol: udp+ FromPort: '161'+ ToPort: '162'+ CidrIp: 0.0.0.0/0+ Description: SNMP+ - IpProtocol: icmp+ FromPort: '8'+ ToPort: '-1'+ CidrIp: 0.0.0.0/0+ Description: Ping+ - IpProtocol: tcp+ FromPort: '2049'+ ToPort: '2049'+ CidrIp: 0.0.0.0/0+ Description: NFS+ - IpProtocol: tcp+ FromPort: '2049'+ ToPort: '2049'+ CidrIpv6: ::/0+ Description: NFS+ - IpProtocol: tcp+ FromPort: '22'+ ToPort: '22'+ CidrIpv6: ::/0+ Description: SSH+ - IpProtocol: tcp+ FromPort: '80'+ ToPort: '80'+ CidrIpv6: ::/0+ Description: HTTP+ - IpProtocol: tcp+ FromPort: '443'+ ToPort: '443'+ CidrIpv6: ::/0+ Description: HTTPS+ - IpProtocol: tcp+ FromPort: '8883'+ ToPort: '8883'+ CidrIpv6: ::/0+ Description: MQTT+ - IpProtocol: udp+ FromPort: '161'+ ToPort: '162'+ CidrIpv6: ::/0+ Description: SNMP+ - IpProtocol: icmpv6+ FromPort: '-1'+ ToPort: '-1'+ CidrIpv6: ::/0+ Description: Pingdiff --git a/.ci_cd/ci_cd.json b/.ci_cd/ci_cd.jsonindex f41cef0f3..326b62104 100644--- a/.ci_cd/ci_cd.json+++ b/.ci_cd/ci_cd.json@@ -3,6 +3,10 @@"develop": {"distribute": {"tags": "develop"+ },+ "deploy": {+ "managerTag": "develop",+ "environment": "staging"}},"master": {diff --git a/.ci_cd/deploy.sh b/.ci_cd/deploy.shindex 9fedea5af..6ed263cd7 100644--- a/.ci_cd/deploy.sh+++ b/.ci_cd/deploy.sh@@ -70,7 +70,7 @@ PLATFORM="linux/$PLATFORM"# Create docker image tarballs as requiredif [ "$MANAGER_TAG" != '#ref' ]; thendocker manifest inspect openremote/manager:$MANAGER_TAG > /dev/null 2> /dev/null- if [ $? != 0 ]; then+ if [ $? -ne 0 ]; thenecho "Specified manager tag does not exist in docker hub"exit 1fi@@ -133,42 +133,60 @@ $sshCommandPrefix ${hostStr} << EOFfi# Run host init+ hostInitCmd=if [ "$HOST_INIT_SCRIPT" == 'NONE' -o "$HOST_INIT_SCRIPT" == 'none' ]; thenecho "No host init requested"elif [ ! -z "$HOST_INIT_SCRIPT" ]; then- if [ ! -f ".ci_cd/host_init/${HOST_INIT_SCRIPT}.sh" ]; then- echo "HOST_INIT_SCRIPT (.ci_cd/host_init/${HOST_INIT_SCRIPT}.sh) does not exist"+ if [ ! -f "temp/host_init/${HOST_INIT_SCRIPT}.sh" ]; then+ echo "HOST_INIT_SCRIPT (temp/host_init/${HOST_INIT_SCRIPT}.sh) does not exist"exit 1fi- hostInitCmd=".ci_cd/host_init/${HOST_INIT_SCRIPT}.sh"- elif [ -f ".ci_cd/host_init/$ENVIRONMENT.sh" ]; then- hostInitCmd=".ci_cd/host_init/$ENVIRONMENT.sh"- elif [ -f ".ci_cd/host_init/init.sh" ]; then- hostInitCmd=".ci_cd/host_init/init.sh"+ hostInitCmd="temp/host_init/${HOST_INIT_SCRIPT}.sh"+ elif [ -f "temp/host_init/$ENVIRONMENT.sh" ]; then+ hostInitCmd="temp/host_init/$ENVIRONMENT.sh"+ elif [ -f "temp/host_init/init.sh" ]; then+ hostInitCmd="temp/host_init/init.sh"fi- if [ ! -z "$hostInitCmd" ]; then+ if [ ! -z "$hostInitCmd" ]; thenecho "Running host init script: '$hostInitCmd'"- $hostInitCmd+ sudo $hostInitCmd+ else+ echo "No host init script"fi-- # MAKE SURE WE HAVE CORRECT KEYCLOAK, PROXY AND POSTGRES IMAGES++ # Make sure we have correct keycloak, proxy and postgres images+ echo "Pulling requested service versions from docker hub"docker-compose -p or -f temp/docker-compose.yml pull --ignore-pull-failures-++ if [ \$? -ne 0 ]; then+ echo "Deployment failed to pull docker images"+ exit 1+ fi+# Attempt docker compose down- docker-compose -f temp/docker-compose.yml -p or down+ echo "Stopping existing stack"+ docker-compose -f temp/docker-compose.yml -p or down 2> /dev/null++ if [ \$? -ne 0 ]; then+ echo "Deployment failed to stop the existing stack"+ exit 1+ fi# Delete postgres volume if CLEAN_INSTALL=true- if [ $CLEAN_INSTALL == 'true' ]; then- docker volume rm or_postgresql-data+ if [ "\$CLEAN_INSTALL" == 'true' ]; then+ echo "Deleting existing postgres data volume"+ docker volume rm or_postgresql-data 2> /dev/nullfi# Delete any deployment volume so we get the latest- docker volume rm or_deployment-data+ echo "Deleting existing deployment data volume"+ docker volume rm or_deployment-data 2 > /dev/null# Start the stack+ echo "Starting the stack"docker-compose -f temp/docker-compose.yml -p or up -d- if [ $? != 0 ];then+ if [ \$? -ne 0 ]; thenecho "Deployment failed to start the stack"exit 1fidiff --git a/.ci_cd/gitlab-ci/README.md b/.ci_cd/gitlab-ci/README.mddeleted file mode 100644index 503db5691..000000000--- a/.ci_cd/gitlab-ci/README.md+++ /dev/null@@ -1,314 +0,0 @@-# GitLab CI/CD pipeline--There is .gitlab-ci.yml configuration file which is used to build/test and deploy stack in the GitLab enviroment.-If one imports main repo the pipeline can be started.--## Build docker images--You can skip this stage by adding **skip-build** in the git commit message.--## Test the stack--The second major step in the CI/CD pipeline are functional tests. When extending-the functionality, e.g. by adding a new protocol, tests covering its behaviour should-be included to the main branch and executed during each commit and check whether no-legacy functionality is broken. Sometimes this is intended and in this case rgresssion-tests should be adjusted to the new situation.--In case that no functionaity is changed, e.g. in case of documentation updates, this stage can-be skipped by adding **skip-test** in the git commit message.--### Inspect failed test report output--When tests fail and break the pipeline, test reports are stored as artifacts. On the right side of the failed pipleine there is a link-where the reports can be downloaded.--## Deploy--You can skip this stage by adding **skip-deploy** in the git commit message.--# Deploing OpenRemote stack in the cloud--## Deployment on AWS through the web UI--This is slower prodcedure and meant as en excercise for AWS novice users. Advanced users should use [**aws-cli**](#awscli) method described in the next section.--- Sign in into [AWS web console](https://console.aws.amazon.com/console/home?nc2=h_ct&src=header-signin)-- Select region to eu-west-1 (Ireland). Currently the cloudformation template supports only this region. *Other regions can be added when- necessary*.-- Go to [EC2 KeyPairs](https://eu-west-1.console.aws.amazon.com/ec2/v2/home?region=eu-west-1#KeyPairs:)- + create or import key pair with name **openremote**- if you create a new key pair don't forget to download it after creation-- Go to [CloudFormation conslole](https://eu-west-1.console.aws.amazon.com/cloudformation/home?region=eu-west-1)- + The top right corner -> Create stack -> With new resources (standard)- + On the Create stack page select *Upload a template file*- + *Chose file* and select the openremote/gitlab-ci/aws-cloudformation.template.yml, press *Next*- + Enter *Stack name*, e.g. **openremote3**- On *Parameters* check if KeyName has a correct value, default is **openremote** which we used for key generation, press *Next*- + On *Configure stack options* press *Next*- + On *Review* page you can check all parameters and press *Create stack*- You should be on a page where the stack is in CREATE_IN_PROGRESS status.- In less than a mniute the stack should be in CREATE_COMPLETE state- Press *Outputs* tab where you should find *The instance public IP*. Copy it to web browser. You will need to wait about 7 minutes before the- instance initializes itself with OpenRemote containers.--- After you are done with the stack please for to the [Cloudformation](https://eu-west-1.console.aws.amazon.com/cloudformation/home) dashboard and-delete it. **This will remove all resources and prevent from generating unnecessary costs**.--## <a name="awscli"></a>Deployment on AWS with CLI (advanced)--We will use the [AWS CLI](https://aws.amazon.com/cli/) to create the necessary resources on AWS side.-Following commands should be executed from openremote/gitlab-ci subdirectory.--We are presenting 3 scenarios:-1. Deploy OpenRemote stack with domain hosted on AWS-2. Deploy OpenRemote stack without domain-3. Deploy OpenRemote stack with domain hosted anywhere--### Prepare the environment--You will need to do this only once per AWS account.--Enter the following commands in your terminal:--```bash-# Configure the AWS CLI with your AWS region (supported eu-west-1<Ireland>)-aws configure--# Create SSH keys (it's a pity but AWS EC2 does not accept other type than rsa keys here)-ssh-keygen -f openremote -t rsa -b 4096 -C "[email protected]"--# Import key pair to AWS-aws ec2 import-key-pair --key-name openremote --public-key-material fileb://openremote.pub--```--### Scenario 1 - Deploy OpenRemote stack with domain hosted on AWS--In this scenario we have an AWS account with Route53 hosted zone equal to DomainName input parameter. This is the fastest way to create a full-functional OpenRemote stack with valid SSL key. Within less than 10 minutes you can create a full functional deployment with valid URL.--```bash-# Create stack-aws cloudformation create-stack --stack-name OpenRemote --template-body file://gitlab-ci/aws-cloudformation.template.yml --parameters ParameterKey=DomainName,ParameterValue=developers.openremote.io ParameterKey=HostedZone,ParameterValue=true ParameterKey=HostName,ParameterValue=myhost--```--Wait about 7 minutes.--Check the stack:-```bash-# Check the stack status. If it is different than CREATE_COMPLETE the you can find more info in the AWS console-# Services -> CloudFormation -> Stacks -> OpenRemote -> Events tab-aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].StackStatus"--# Get the URL of the stack (it should be DomainName.HostName i.e. myhost.developers.openremote.io in this example)-export URL=$(aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].Outputs[?OutputKey=='PublicUrl'].OutputValue" --output text)--# Check if it gives valid response-curl -L -I $URL--# The output should look like this:-HTTP/1.1 302 Found-content-length: 0-location: https://myhost.developers.openremote.io/-cache-control: no-cache--HTTP/1.1 302 Found-location: http://myhost.developers.openremote.io/main-content-length: 0-date: Sat, 18 Jul 2020 10:41:15 GMT--HTTP/1.1 302 Found-content-length: 0-location: https://myhost.developers.openremote.io/main-cache-control: no-cache--HTTP/1.1 302 Found-location: /main/-content-type: text/html;charset=UTF-8-content-length: 64-date: Sat, 18 Jul 2020 10:41:15 GMT--HTTP/1.1 200 OK-expires: Sat, 18 Jul 2020 22:41:15 GMT-cache-control: public,max-age=43200,must-revalidate-pragma:-accept-ranges: bytes-date: Sat, 18 Jul 2020 10:41:15 GMT-etag: W/"index.html-1595005800000"-last-modified: Fri, 17 Jul 2020 17:10:00 GMT-content-type: text/html-content-length: 3627-```--### Scenario 2 - Deploy OpenRemote stack without domain--In this scenario we are deploing stack without valid domain. It will use public IP dynamically assigned by AWS.--To achieve this we create the stack with empty DomainName and HostName set to localhost.--```bash-# Create stack-aws cloudformation create-stack --stack-name OpenRemote --template-body file://gitlab-ci/aws-cloudformation.template.yml --parameters ParameterKey=DomainName,ParameterValue= ParameterKey=HostName,ParameterValue=localhost-```--Wait about 7 minutes.--Check the stack:-```bash-# Check the stack status. If it is different than CREATE_COMPLETE the you can find more info in the AWS console-# Services -> CloudFormation -> Stacks -> OpenRemote -> Events tab-aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].StackStatus"--# Get public IP of the stack-export IP=$(aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].Outputs[?OutputKey=='InstanceIP'].OutputValue" --output text)--# Check if it gives valid response-curl -L -I $IP--# The output should look like this:-HTTP/1.1 302 Found-content-length: 0-location: https://3.250.36.158/-cache-control: no-cache--HTTP/1.1 302 Found-location: http://3.250.36.158/main-content-length: 0-date: Sat, 18 Jul 2020 09:12:44 GMT--HTTP/1.1 302 Found-content-length: 0-location: https://3.250.36.158/main-cache-control: no-cache--HTTP/1.1 302 Found-location: /main/-content-type: text/html;charset=UTF-8-content-length: 64-date: Sat, 18 Jul 2020 09:12:44 GMT--HTTP/1.1 200 OK-expires: Sat, 18 Jul 2020 21:12:44 GMT-cache-control: public,max-age=43200,must-revalidate-pragma:-accept-ranges: bytes-date: Sat, 18 Jul 2020 09:12:44 GMT-etag: W/"index.html-1595005800000"-last-modified: Fri, 17 Jul 2020 17:10:00 GMT-content-type: text/html-content-length: 3627-```--Of course you can point a browser to this endpoint but you have to accept invalid SSL certificate. Sometimes you need to change browser settings to allow for this.--### Scenario 3 - Deploy OpenRemote stack with domain hosted anywhere--In this scenario we are hosting our DNS records outside of AWS, or on AWS but on a different account. It is possible to create a cloudformation template-which would work automatically with the second case by using IAM roles, however this is beyond this tutorial. Please conntact OpenRemote if this is your case.--This is the most time consuming options as it can take up to 24 hours to propagate the DNS record if you are using a slow provider. [AWS](https://aws.amazon.com/) and [CloudFlare](https://www.cloudflare.com/) do this-within few seconds, therefore in dynamic environments we advise to use one of them.--```bash-# Create stack-aws cloudformation create-stack --stack-name OpenRemote --template-body file://gitlab-ci/aws-cloudformation.template.yml --parameters ParameterKey=DomainName,ParameterValue=mydomain.com ParameterKey=HostName,ParameterValue=example-```--Wait about 7 minutes.--Check the stack:-```bash-# Check the stack status. If it is different than CREATE_COMPLETE the you can find more info in the AWS console-# Services -> CloudFormation -> Stacks -> OpenRemote -> Events tab-aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].StackStatus"--# Get public IP and URL of the stack-export IP=$(aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].Outputs[?OutputKey=='InstanceIP'].OutputValue" --output text)-export URL=$(aws cloudformation describe-stacks --stack-name OpenRemote --query "Stacks[0].Outputs[?OutputKey=='PublicUrl'].OutputValue" --output text)--# Show public IP and URL-# It should produce something like 34.244.50.94 example.mydomain.com-echo $IP $URL-```--With the data of the last command go to your DNS dashboard and create an A record example.mydomain.com with coresponding IP address.--Wait up to 24 hours for DNS propagation.--```bash-# Check if DNS is updated-ping $URL-# you should get response like-PING example.mydomain.com (34.244.50.94): 56 data bytes-64 bytes from 34.244.50.94: icmp_seq=0 ttl=41 time=29.731 ms-64 bytes from 34.244.50.94: icmp_seq=1 ttl=41 time=37.164 ms--# Update SSL ceriticate-ssh -i ~/.ssh/openremote ubuntu@$URL docker start openremote_proxy_1--# Check if it gives valid response-curl -L -I $URL--HTTP/1.1 302 Found-content-length: 0-location: https://example.mydomain.com/-cache-control: no-cache--HTTP/1.1 302 Found-location: http://example.mydomain.com/main-content-length: 0-date: Sat, 18 Jul 2020 12:20:54 GMT--HTTP/1.1 302 Found-content-length: 0-location: https://example.mydomain.com/main-cache-control: no-cache--HTTP/1.1 302 Found-location: /main/-content-type: text/html;charset=UTF-8-content-length: 64-date: Sat, 18 Jul 2020 12:20:54 GMT--HTTP/1.1 200 OK-expires: Sun, 19 Jul 2020 00:20:54 GMT-cache-control: public,max-age=43200,must-revalidate-pragma:-accept-ranges: bytes-date: Sat, 18 Jul 2020 12:20:54 GMT-etag: W/"index.html-1595005800000"-last-modified: Fri, 17 Jul 2020 17:10:00 GMT-content-type: text/html-content-length: 3627-```--### Add/remove assets (optional)--TODO--### **IMPORTANT** Delete unused stack--When you are done with the stack don't forget to delete it. Depending on the EC2 instance selected leaving it running incures [costs](https://ec2instances.info/?region=eu-west-1). The default one **t3a.medium** in eu-west-1 region (Ireland) currently costs $0.040800 hourly and when reserved $0.025700 hourly.--```bash-# Delete unused stack-aws cloudformation delete-stack --stack-name OpenRemote-```--## Deployment on Alibaba--TODO--## Deployment on Azure--TODO--## Deployment on GCP--TODO--## Deployment on OVH--TODOdiff --git a/.ci_cd/gitlab-ci/aws-mail.py b/.ci_cd/gitlab-ci/aws-mail.pydeleted file mode 100644index 20f4650d3..000000000--- a/.ci_cd/gitlab-ci/aws-mail.py+++ /dev/null@@ -1,38 +0,0 @@-#!/usr/bin/python3--import smtplib-import os--sender = '[email protected]'-receivers = [os.environ['WHO']]-subject = 'Gitlab pipeline FAILED on ' + os.environ['STAGE'] + ': ' + os.environ['WHAT']--print(sender,receivers, subject)-message = '\r\n'.join(['From: %s' % sender,- 'To: %s' % receivers[0],- 'Subject: %s' % subject,- """-The GitLab CI/CD pipeline FAILED for {repo}--To skip testing and run complete pipeline without errors please include-'skip-tests' in the git commit message.--{what}-{when}-{who}.-""".format(when=os.environ['WHEN'], who=os.environ['WHO'], what=os.environ['WHAT'], repo=os.environ['REPO'])- ])--try:- server = smtplib.SMTP('email-smtp.eu-west-1.amazonaws.com',587)- server.ehlo()- server.starttls()- server.login(os.environ['SMTP_USERNAME'], os.environ['SMTP_PASSWORD'])-- server.sendmail(sender, receivers, message)- print ("Successfully sent email")-except Exception as ex:- print ("Error: unable to send email for "+os.environ['SMTP_USERNAME'])-- template = "An exception of type {0} occurred. Arguments:\n{1!r}"- print (template.format(type(ex).__name__, ex.args))diff --git a/.ci_cd/gitlab-ci/gitlab-ci-submodule.yml b/.ci_cd/gitlab-ci/gitlab-ci-submodule.ymldeleted file mode 100644index 8f1eeafee..000000000--- a/.ci_cd/gitlab-ci/gitlab-ci-submodule.yml+++ /dev/null@@ -1,327 +0,0 @@-default:- interruptible: true- image: docker:latest- services:- - docker:dind # Docker in docker service--variables:- ## Following 3 variables should be defined as GitLab CI/CD variables- #PROJECT_NAME:- #DEPLOYMENT_HOST_USER:- #DEPLOYMENT_PORT:- GIT_SUBMODULE_STRATEGY: normal- DAST_WEBSITE: https://staging.${PROJECT_NAME}.${DOMAIN}--workflow:- rules:- - if: $CI_COMMIT_BRANCH != "master"- when: never- - when: always--stages:- - build_staging- - build_fail- - test- - tests_fail- - deploy- - notify_deploy_staging_fail- - dast- - build_production- - deploy_production- - deploy_production_clean- - notify_deploy_production--.build_template: &build_definition- variables:- SCAN_IMAGE_TAG: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA- MANAGER_IMAGE_TAG: $CI_REGISTRY_IMAGE/manager:$CI_ENVIRONMENT_SLUG- DEPLOY_IMAGE_TAG: $CI_REGISTRY_IMAGE/deployment:$CI_ENVIRONMENT_SLUG- PROXY_IMAGE_TAG: $CI_REGISTRY_IMAGE/proxy:$CI_ENVIRONMENT_SLUG- POSTGRE_IMAGE_TAG: $CI_REGISTRY_IMAGE/postgresql:$CI_ENVIRONMENT_SLUG- KEYCLO_IMAGE_TAG: $CI_REGISTRY_IMAGE/keycloak:$CI_ENVIRONMENT_SLUG- before_script:- - echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin $CI_REGISTRY- script:- - docker run -v $(pwd):/or -v $(pwd)/logs:/root/.npm/_logs registry.gitlab.com/openremote/openremote:master- /bin/sh -c "./gradlew --console plain clean installDist"- - docker run -v $(pwd):/or- -v /var/run/docker.sock:/var/run/docker.sock- -e "CI_REGISTRY_IMAGE=$CI_REGISTRY_IMAGE"- -e "CI_ENVIRONMENT=$CI_ENVIRONMENT_SLUG"- registry.gitlab.com/openremote/openremote:master- /bin/sh -c "docker-compose -f profile/$CI_ENVIRONMENT_SLUG.yml build"- - docker images- # Image for container scanning- - docker tag $MANAGER_IMAGE_TAG $SCAN_IMAGE_TAG- - docker push $SCAN_IMAGE_TAG- # Push to registry- - docker push "$MANAGER_IMAGE_TAG"- - docker push "$DEPLOY_IMAGE_TAG"- - docker push "$PROXY_IMAGE_TAG"- - docker push "$POSTGRE_IMAGE_TAG"- - docker push "$KEYCLO_IMAGE_TAG"- # Tag for possible rollback- - if [ $CI_COMMIT_TAG ]; then- - docker tag "$CI_REGISTRY_IMAGE/manager:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/manager:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - docker tag "$CI_REGISTRY_IMAGE/deployment:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/deployment:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - docker tag "$CI_REGISTRY_IMAGE/proxy:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/proxy:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - docker tag "$CI_REGISTRY_IMAGE/postgresql:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/postgresql:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - docker tag "$CI_REGISTRY_IMAGE/keycloak:$CI_ENVIRONMENT_SLUG" "$CI_REGISTRY_IMAGE/keycloak:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - docker push "$CI_REGISTRY_IMAGE/manager:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - docker push "$CI_REGISTRY_IMAGE/deployment:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - docker push "$CI_REGISTRY_IMAGE/proxy:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - docker push "$CI_REGISTRY_IMAGE/postgresql:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - docker push "$CI_REGISTRY_IMAGE/keycloak:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_TAG"- - fi- artifacts:- when: on_failure- paths:- - "logs/*"- expire_in: 4 weeks--.notify_whatsapp: ¬ify_whatsapp- image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest- allow_failure: true- variables:- GIT_STRATEGY: none- script:- - >- curl --location --request POST $WHATSAPP_API- --header 'Content-Type: application/javascript'- --data-raw "{- \"message\": \"Your $CI_PROJECT_TITLE+$CI_JOB_STAGE code is $CI_JOB_NAME\",- \"recipient\": \"$WHATSAPP_TEL\",- \"user_id\": \"1\"- }"- retry:- max: 2--build staging docker images:- stage: build_staging- rules:- - if: $CI_COMMIT_MESSAGE !~ /skip-build/- environment: staging- <<: *build_definition--build manually staging docker images:- stage: build_staging- rules:- - if: $CI_COMMIT_MESSAGE =~ /skip-build/- when: manual- allow_failure: true- environment: staging- <<: *build_definition--.email_template: ¬ify_email- image: docker:latest- # Do not clone git repo as it can fail and sending mail'll fail too- variables:- GIT_SUBMODULE_STRATEGY: none- before_script:- - cat $AWS_MAIL > aws_mail.py- script:- - docker pull registry.gitlab.com/openremote/openremote:master- - export WHO=$(docker run --rm -v $(pwd):/or registry.gitlab.com/openremote/openremote:master git log --pretty="%ce" -1)- - export WHEN=$(docker run --rm -v $(pwd):/or registry.gitlab.com/openremote/openremote:master git log --pretty="%ad %ar" -1)- - export WHAT=$(docker run --rm -v $(pwd):/or registry.gitlab.com/openremote/openremote:master git log --pretty="%h %s" -1)- - export REPO=$(docker run --rm -v $(pwd):/or registry.gitlab.com/openremote/openremote:master git remote get-url origin|xargs basename)- - echo $REPO- - docker run -v $(pwd):/usr/src/myapp -w /usr/src/myapp- -e "STAGE=$CI_JOB_STAGE"- -e "WHO=$WHO"- -e "WHAT=$WHAT"- -e "WHEN=$WHEN"- -e "REPO=$REPO"- -e "SMTP_USERNAME=$SMTP_USERNAME"- -e "SMTP_PASSWORD=$SMTP_PASSWORD"- --rm python:3 python aws_mail.py- retry:- max: 2--# This step is executed when build fails. The purpose is to e-mail a git commiter about the problem.-email failed pipeline:- stage: build_fail- needs: ["build staging docker images"]- <<: *notify_email- when: on_failure--build FAILED:- stage: build_fail- needs: ["build staging docker images"]- <<: *notify_whatsapp- when: on_failure--test-master:- stage: test- needs: []- allow_failure: false- before_script:- - docker run -v $(pwd):/or- -v /var/run/docker.sock:/var/run/docker.sock registry.gitlab.com/openremote/openremote:master- /bin/sh -c "docker-compose -f openremote/profile/dev-testing.yml pull"- - docker run -v $(pwd):/or- -v /var/run/docker.sock:/var/run/docker.sock registry.gitlab.com/openremote/openremote:master- /bin/sh -c "docker-compose -f openremote/profile/dev-testing.yml up --no-build -d"- script:- #- docker run --network host -v $(pwd):/or registry.gitlab.com/openremote/openremote:master- # /bin/sh -c "./gradlew clean test"- - echo Tests moved to github actions- rules:- - if: $CI_COMMIT_MESSAGE !~ /skip-test/- when: on_success- artifacts:- when: on_failure- paths:- - "openremote/test/build/reports/tests/test/*"- - "openremote/test/build/reports/tests/test/*/*"- expire_in: 4 weeks- timeout: 30m--email failed tests:- stage: tests_fail- needs: ["test-master"]- <<: *notify_email- rules:- - if: $CI_COMMIT_MESSAGE !~ /skip-test/- when: on_failure--test FAILED:- stage: tests_fail- needs: ["test-master"]- <<: *notify_whatsapp- rules:- - if: $CI_COMMIT_MESSAGE !~ /skip-test/- when: on_failure--include:- - template: Container-Scanning.gitlab-ci.yml- - template: Dependency-Scanning.gitlab-ci.yml- - template: License-Scanning.gitlab-ci.yml- # - template: SAST.gitlab-ci.yml- - template: DAST.gitlab-ci.yml--.ssh_config: &ssh_config- - mkdir -p ~/.ssh- - chmod 600 "$GITLAB_SSH_PRIVATE_KEY"- - eval "$(ssh-agent -s)"- - ssh-add "$GITLAB_SSH_PRIVATE_KEY"- - ssh-keyscan -H -p $DEPLOYMENT_PORT $DEPLOYMENT_HOST >> ~/.ssh/known_hosts- - alias SSH="ssh -p $DEPLOYMENT_PORT ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}"- - alias SCP="scp -p -r -P $DEPLOYMENT_PORT"--.deploy_template: &deploy_definition- before_script:- - *ssh_config- script:- - echo "Deploy to $CI_ENVIRONMENT_SLUG server"- - echo "export PROJECT_NAME=$PROJECT_NAME" >> deploy.sh- - echo "export CI_REGISTRY_IMAGE=$CI_REGISTRY_IMAGE" >> deploy.sh- - if [ ! $DOCKER_IMAGE_TAG ]; then echo "export CI_ENVIRONMENT=$CI_ENVIRONMENT_SLUG" >> deploy.sh; fi- - if [ $DOCKER_IMAGE_TAG ]; then echo "export CI_ENVIRONMENT=$DOCKER_IMAGE_TAG" >> deploy.sh; fi- - echo "export SETUP_WIPE_CLEAN_INSTALL=$SETUP_WIPE_CLEAN_INSTALL" >> deploy.sh- - if [ $SETUP_ADMIN_PASSWORD ]; then echo "export SETUP_ADMIN_PASSWORD=$SETUP_ADMIN_PASSWORD" >> deploy.sh; fi- - echo "echo ${OR_PASSWORD} | docker login -u openremote --password-stdin $CI_REGISTRY || exit 666" >> deploy.sh- - echo "docker pull $CI_REGISTRY_IMAGE/manager:\$CI_ENVIRONMENT" >> deploy.sh- - echo "docker pull $CI_REGISTRY_IMAGE/deployment:\$CI_ENVIRONMENT">> deploy.sh- - echo "docker pull $CI_REGISTRY_IMAGE/proxy:\$CI_ENVIRONMENT" >> deploy.sh- - echo "docker pull $CI_REGISTRY_IMAGE/postgresql:\$CI_ENVIRONMENT">> deploy.sh- - echo "docker pull $CI_REGISTRY_IMAGE/keycloak:\$CI_ENVIRONMENT" >> deploy.sh- # Dump DB if it exists for possible data restoration- - echo "if [ \"\$(docker ps --filter name=${PROJECT_NAME}_postgresql_1 -q)\" ]; then docker exec ${PROJECT_NAME}_postgresql_1 pg_dump -U postgres openremote > dump/\$RANDOM.sql; fi || exit 666" >> deploy.sh- - echo "docker-compose -p $PROJECT_NAME -f profile/$CI_ENVIRONMENT_SLUG.yml down || exit 666" >> deploy.sh- - echo "docker volume rm ${PROJECT_NAME}_deployment-data" >> deploy.sh- - echo "docker-compose -p $PROJECT_NAME -f profile/$CI_ENVIRONMENT_SLUG.yml up --no-build -d || exit 666" >> deploy.sh- - echo "docker cp FIREBASE_CONFIG_FILE ${PROJECT_NAME}_manager_1:/deployment/manager/fcm.json || exit 666" >> deploy.sh- - echo "docker restart ${PROJECT_NAME}_manager_1 || exit 666" >> deploy.sh- - chmod +x deploy.sh- - SSH mkdir -p dump profile openremote/profile deployment openremote/proxy openremote/manager/build/install/manager openremote/postgresql openremote/keycloak- - SCP ./profile/ deploy.sh ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}:~/- - SCP ./openremote/profile/* ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}:~/openremote/profile/- - SCP ${FIREBASE_CONFIG_FILE} ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}:~/FIREBASE_CONFIG_FILE- - SSH ./deploy.sh- - SSH "docker system prune -f"- after_script:- - *ssh_config- - SSH "docker-compose -p ${PROJECT_NAME} -f profile/${CI_ENVIRONMENT_SLUG}.yml logs --no-color --timestamp > ${CI_ENVIRONMENT_SLUG}.log"- - SCP ${DEPLOYMENT_HOST_USER}@${DEPLOYMENT_HOST}:~/${CI_ENVIRONMENT_SLUG}.log .- - tail -n 100 ${CI_ENVIRONMENT_SLUG}.log- - SSH "rm ~/${CI_ENVIRONMENT_SLUG}.log"- artifacts:- when: on_failure- paths:- - ${CI_ENVIRONMENT_SLUG}.log- expire_in: 4 weeks--deploy staging:- stage: deploy- variables:- DEPLOYMENT_HOST: ${CI_ENVIRONMENT_SLUG}.${PROJECT_NAME}.${DOMAIN}- SETUP_WIPE_CLEAN_INSTALL: "true"- environment:- name: staging- url: https://${CI_ENVIRONMENT_SLUG}.${PROJECT_NAME}.${DOMAIN}- <<: *deploy_definition--deploy staging when tests fail:- stage: deploy- needs: ["email failed tests"]- when: manual- variables:- DEPLOYMENT_HOST: ${CI_ENVIRONMENT_SLUG}.${PROJECT_NAME}.${DOMAIN}- SETUP_WIPE_CLEAN_INSTALL: "true"- environment:- name: staging- url: https://${CI_ENVIRONMENT_SLUG}.${PROJECT_NAME}.${DOMAIN}- <<: *deploy_definition--email failed deploy staging:- stage: notify_deploy_staging_fail- needs: ["deploy staging"]- <<: *notify_email- when: on_failure--deploy staging FAILED:- stage: notify_deploy_staging_fail- needs: ["deploy staging"]- <<: *notify_whatsapp- when: on_failure--build production docker images:- stage: build_production- when: manual- allow_failure: false- environment: production- <<: *build_definition--deploy production:- stage: deploy_production- needs: ["build production docker images"]- variables:- DEPLOYMENT_HOST: ${PROJECT_NAME}.${DOMAIN}- SETUP_WIPE_CLEAN_INSTALL: "false"- environment:- name: production- url: https://${PROJECT_NAME}.${DOMAIN}- <<: *deploy_definition--deploy production clean:- stage: deploy_production_clean- when: on_failure- needs: ["build production docker images"]- variables:- DEPLOYMENT_HOST: ${PROJECT_NAME}.${DOMAIN}- SETUP_WIPE_CLEAN_INSTALL: "true"- environment:- name: production- url: https://${PROJECT_NAME}.${DOMAIN}- <<: *deploy_definition--deploy production PASSED:- stage: notify_deploy_production- <<: *notify_whatsapp--deploy production FAILED:- stage: notify_deploy_production- needs: ["build production docker images"]- <<: *notify_whatsapp- when: on_failurediff --git a/.ci_cd/host_init/init.sh b/.ci_cd/host_init/init.shindex ad0db2a9d..050fdb7e9 100644--- a/.ci_cd/host_init/init.sh+++ b/.ci_cd/host_init/init.sh@@ -1,3 +1,6 @@#!/bin/bash-sudo rm /etc/cron.d/openremote-sudo echo "0 5 * * * docker restart or_manager_1" >> /etc/cron.d/openremote+#+# Create CRON JOB to restart manager daily at 05:00+#+rm /etc/cron.d/openremote+echo "0 5 * * * docker restart or_manager_1" >> /etc/cron.d/openremotediff --git a/.github/workflows/ci_cd.yml b/.github/workflows/ci_cd.ymlindex 8b99f4dd2..98ee13467 100644--- a/.github/workflows/ci_cd.yml+++ b/.github/workflows/ci_cd.yml@@ -1,3 +1,52 @@+# This workflow handles all CI/CD related tasks and can be re-used in custom projects+# (https://github.com/openremote/custom-project i.e. has openremote submodule) or forks of this repo, it handles:+#+# - Running tests on push/release+# - Distributing openremote/manager images to docker hub (DOCKERHUB_USER and DOCKERHUB_PASSWORD must be set) on push/release+# - Deploying to hosts on push/release and/or manual trigger and/or when certain openremote/manager docker tags are updated+#+# By default this workflow will just run tests on push/release; distribution (pushing of openremote/manager docker image)+# and deployment behaviour is configured via the .ci_cd/ci_cd.json file (see .ci_cd/README.md for more details).+#+# DEPLOYMENTS+#+# When a deployment is requested the repo is checked for a 'deployment/Dockerfile' and if that exists then also+# 'deployment/build.gradle' must exist with an installDist task that prepares the deployment image in the+# 'deployment/build' directory; if this condition is not met the deployment will fail.+#+# Secrets, inputs and environment variables are combined for consumption by the '.ci_cd/deploy.sh' bash script, they are+# combined in the following priority:+#+# - '.ci_cd/env/.env'+# - '.ci_cd/env/{ENVIRONMENT}.env'+# - secrets without '_' prefix+# - secrets with '_{ENVIRONMENT}_' prefix (secrets with any other '_' prefix will be excluded)+# - inputs (from manual trigger using workflow_dispatch)+#+# The above variables are output to 'temp/env' file for each requested deployment. If a secret called 'SSH_PASSWORD' is+# found that is output to 'ssh.env' file (so it is not copied to the host), if a secret called 'SSH_KEY' is+# found that is output to 'ssh.key' file (so it is not copied to the host). Sensitive credentials should always be stored+# in github secrets so they are encrypted, do not store within repo files (even private repos as these are not encrypted).+#+# SEE https://github.com/openremote/openremote/tree/master/.ci_cd for details of standard variables and handling.+#+# MANUAL TRIGGER+#+# This workflow can be triggered by push, release, manually or schedule (from any repo other than openremote/openremote)+# if triggered manually then the following inputs are available and these override any set in env files and/or secrets:+#+# Inputs:+#+# - ENVIRONMENT - Which environment to deploy (equivalent to deploy/environment in '.ci_cd/ci_cd.json')+# - MANAGER_TAG - Which manager docker tag to deploy (equivalent to deploy/managerTags in '.ci_cd/ci_cd.json')+# - CLEAN_INSTALL+# - COMMIT+# - HOST+# - SSH_USER+# - SSH_PASSWORD+# - SSH_PORT+# - SETUP_ADMIN_PASSWORD+name: CI/CDon:@@ -26,18 +75,18 @@ on:CLEAN_INSTALL:description: 'Delete data before starting'type: boolean+ COMMIT:+ description: 'Repo branch or commit SHA to checkout'HOST:description: 'Host to deploy to'- SSH_USER:- description: 'SSH Username'SSH_PORT:description: 'SSH port'+ SSH_USER:+ description: 'SSH Username'SSH_PASSWORD:description: 'SSH password'SETUP_ADMIN_PASSWORD:description: 'Admin password override'- COMMIT:- description: 'Repo branch or commit SHA to use'workflow_call:inputs: