Commit Graph

169 Commits

Author SHA1 Message Date
Amir Zarrinkafsh
2b627c6c04
[CI] Set concurrency groups at a global level and simplify pipeline (#942) 2020-04-29 14:06:35 +10:00
Amir Zarrinkafsh
f781d63b2c
[CI] Prevent race conditions with appropriate deployment steps (#941)
If we have multiple builds to master that intend to deploy AUR packages or documentation, we must ensure that the jobs are locked and executed sequentially, not simultaneously. If they were to run simultaneously this has the ability to cause a race condition when attempting to commit the respective steps.
2020-04-29 13:05:41 +10:00
Amir Zarrinkafsh
12100d21e2
[CI] Linting optimisations (#940)
* [CI] Lint all builds except tagged commits to satisfy branch protection

* [CI] Add automatic retries for linting failures
This is to treat any issues with the reviewdog API server and occasional failures we are seeing.
2020-04-29 12:30:46 +10:00
Amir Zarrinkafsh
f8bd506326
[FEATURE] Embed static assets in Go binary (#916)
* [FEATURE] Embed static assets in Go binary

* Refactor/consolidate code and specify public_html via configuration

* Update docs and config template for assets

* Update AUR package pre-requisites and systemd unit

* Include static assets as Buildkite and GitHub artifacts

* Remove references to PUBLIC_DIR

* Only serve assets via embedded filesystem and remove configuration references

* Update authelia-scripts helper to build the embedded filesystem

* Mock the embedded filesystem for unit tests
Add to gitignore to ensure this isn't overwritten.

* Move go:generate to satisfy linter
2020-04-29 00:07:20 +10:00
Clément Michaud
bfff9237fa
[RELEASE] v4.15.1 (#919) 2020-04-26 00:29:08 +02:00
James Elliott
8917c98d65
[RELEASE] v4.15.0 (#904)
* [RELEASE] v4.15.0

Co-authored-by: Amir Zarrinkafsh <nightah@me.com>
2020-04-23 12:43:27 +10:00
Amir Zarrinkafsh
0ec3f18b44
[CI] Introduce GitHub checks based linting with reviewdog (#900)
* [CI] Introduce linting for branch commits with reviewdog
This utilises the GitHub checks API and could be a potential candidate instead of in-line PR reviews.

* [CI] Change reporter to `github-check`

* [CI] Adjust linting in-line PR commentary to execute with linting step
2020-04-22 23:10:22 +10:00
James Elliott
d233057b31
[RELEASE] 4.14.2 (#882) 2020-04-19 22:06:59 +10:00
Amir Zarrinkafsh
4b664cf15f
[RELEASE] v4.14.1 (#880) 2020-04-19 04:33:18 +10:00
Amir Zarrinkafsh
02c55580bc
[RELEASE] v4.14.0 (#873) 2020-04-16 10:41:23 +10:00
James Elliott
04d875b101
[RELEASE] v4.13.1 (#856) 2020-04-11 15:29:03 +10:00
Amir Zarrinkafsh
3de1827b21
[Buildkite] Control clean builds with agent metadata (#840)
This is to optimise build times at the sacrifice of disk space/clean up tasks.
2020-04-09 14:46:04 +10:00
Amir Zarrinkafsh
49ac65eb41
[RELEASE] v4.13.0 (#839) 2020-04-09 11:50:34 +10:00
Amir Zarrinkafsh
de2c5836fd
[Buildkite] Introduce CI linting with golangci-lint and reviewdog (#832)
* [Buildkite] Introduce CI linting with golangci-lint and reviewdog

* Initial pass of golangci-lint

* Add gosimple (megacheck) recommendations

* Add golint recommendations

* [BUGFIX] Migrate authentication traces from v3 mongodb

* Add deadcode recommendations

* [BUGFIX] Fix ShortTimeouts suite when run in dev workflow

* Add unused recommendations

* Add unparam recommendations

* Disable linting on unfixable errors instead of skipping files

* Adjust nolint notation for unparam

* Fix ineffectual assignment to err raised by linter.

* Export environment variable in agent hook

* Add ineffassign recommendations

* Add staticcheck recommendations

* Add gocyclo recommendations

* Adjust ineffassign recommendations

Co-authored-by: Clement Michaud <clement.michaud34@gmail.com>
2020-04-09 11:05:17 +10:00
Amir Zarrinkafsh
13712d0f36
[Buildkite] Fine-grained control of build steps for agent allocation (#835) 2020-04-08 11:31:33 +10:00
James Elliott
670eeb2879
[RELEASE] v4.12.0 (#828) 2020-04-05 22:53:26 +10:00
Amir Zarrinkafsh
580152b40b
[FEATURE] Include darwin based binaries for OSX (#814)
Build and publish binary artifacts for Authelia which can be run directly from OSX.
2020-04-03 16:13:24 +11:00
Amir Zarrinkafsh
a71ca1903d
[RELEASE] v4.11.0 (#810) 2020-04-01 10:53:48 +11:00
Amir Zarrinkafsh
6128081e1f
[Buildkite] Adjust CI bypasses (#804) 2020-04-01 09:44:57 +11:00
Amir Zarrinkafsh
95f6c1a893
[Buildkite] Add contents of BREAKING.md for tag to releases (#797)
This will ensure that notes pertaining to a version in the BREAKING.md will be published in each of the respective github releases.

All information from:
'## Breaking in $TAG' until the next '## Breaking in $TAG' is included.
2020-03-31 08:46:23 +11:00
Amir Zarrinkafsh
92023de85d
[Buildkite] Tweak pipeline to conditionally ignore a wait step (#781) 2020-03-25 14:09:35 +11:00
Amir Zarrinkafsh
d95bda8cdc
[Buildkite] Fix pipeline regression for master deployments (#770)
Fixes the regression #766 introduced as when the `build.branch` is `master` `build.tag` is `""`.
2020-03-22 16:36:18 +11:00
Amir Zarrinkafsh
8b64c9aad5
[Buildkite] Optimise pipeline for tagged deployments (#766)
* [Buildkite] Optimise pipeline for tagged deployments
Ensure Unit and Integration testing is bypassed for tagged builds.

* Apply suggestions from code review

Co-Authored-By: Clément Michaud <clement.michaud34@gmail.com>
2020-03-22 11:03:36 +11:00
Amir Zarrinkafsh
93abf699b8
[Buildkite] Fix docs bypass for origin/master divergences (#764)
If origin/master diverges beyond the initial fork-point on branches or external forks, the `git merge-base` command exits unsuccessfully. This will cause commits to incorrectly be recognised as a docs bypass. This change will catch the unsuccessful exit and treat it as a normal CI/CD run.

Examples:
- https://buildkite.com/authelia/authelia/builds/932
- https://buildkite.com/authelia/authelia/builds/933
- https://buildkite.com/authelia/authelia/builds/941
2020-03-21 14:42:29 +01:00
Amir Zarrinkafsh
daba4d789f
[Buildkite] Optimise pipeline for deployments to master (#756)
Prior to this change all PR's which are merged into master would result in another run of the Unit and Integration testing.
This is not necessary because all steps have to pass for a PR to be accepted in to master, this will save significant time for deployments to master and reduce overall load to the Buildkite workers.
2020-03-20 14:03:50 +11:00
Amir Zarrinkafsh
7eddf07155
[Buildkite] Clean Docker environment on exit (#734) 2020-03-19 16:15:54 +11:00
Amir Zarrinkafsh
06e36f89e3
[Buildkite] Fix unbound variable for docs bypass (#711) 2020-03-15 22:01:52 +11:00
Amir Zarrinkafsh
a24ca937cd
[Buildkite] Fix docs bypass for tagged releases (#709) 2020-03-15 21:07:23 +11:00
Amir Zarrinkafsh
7a3d43a12a
[Buildkite] Adjust pipeline for dependabot (#707)
Minor tweak to skip the whole deployment pipeline rather than individual steps.
2020-03-15 14:48:18 +11:00
Amir Zarrinkafsh
cd561ef094
[Buildkite] Fix pipeline to work alongside dependabot (#706)
This change will continue to perform unit and integration testing, however, disables deployment steps in association with dependabot PRs.
Deployment comments on the PR with autheliabot are also disabled.
2020-03-15 13:49:57 +11:00
Amir Zarrinkafsh
8b80be4061
[Buildkite] Utilise annotations for build notifications (#700)
* [Buildkite] Utilise annotations for artifact and doc bypass notifications

* [Buildkite] Add context to annotations

* [Buildkite] Adjust docs annotation to display for PRs
2020-03-11 10:25:47 +11:00
Amir Zarrinkafsh
6af27cb3f9
[Buildkite] Fix CI docs bypass by ensuring HEAD diff for master is correct (#699) 2020-03-09 17:45:15 +11:00
Amir Zarrinkafsh
7a0d217b67
[Buildkite] Reorder git fetch in pipeline (#697)
This will ensure that we always will have up-to-date refs for the repo post-checkout.
2020-03-09 16:53:13 +11:00
Amir Zarrinkafsh
df431b32c8
[Buildkite] Compare to origin/master for docs bypass (#696) 2020-03-09 15:59:58 +11:00
Amir Zarrinkafsh
b70c4a744f
[Buildkite] Ignore unnecessary CI steps for docs/* only based commits (#690)
* [Buildkite] Ignore build and deploy steps for [DOCS] only based commits

* [Buildkite] Convert static pipelines into dynamic pipelines

* [Buildkite] Convert dynamic pipeline steps into heredocs

* [Buildkite] Fix indentation for aurpackages.sh

* [Buildkite] Rename docs bypass env variable

* [Buildkite] Fix automatic retries in integration tests
2020-03-09 12:32:07 +11:00
Amir Zarrinkafsh
ae5533d41b
[Buildkite] Fix always reporting as failure for github artifact step (#673) 2020-03-01 15:56:04 +11:00
Amir Zarrinkafsh
b007953580
[Buildkite] Optimise deployment dependencies (#668)
* Update all dependencies to allow more parallel jobs.
* Remove concurrency limit for non-master deployments to prevent pipeline blocking.
2020-02-29 15:31:10 +01:00
Clément Michaud
9c0e722bd7
[DOCS] Do not let think OAuth won't be supported. (#665)
* [DOCS] Do not let think OAuth won't be supported.

* [Buildkite] Prevent docs commit if there are no changes
2020-02-29 23:07:23 +11:00
Clément Michaud
f821793afb
[Buildkite] Change commit author of commits in gh-pages to autheliabot. (#662) 2020-02-29 22:29:55 +11:00
Clément Michaud
a9f8958187
[BUGFIX] Add jekyll dependency in Gemfile. (#660)
* [BUGFIX] Add jekyll dependency in Gemfile.

* [Buildkite] Optimise documentation sync step

* [DOC] Fix merge conflict for index.md

* [DOC] Fix formatting issues
2020-02-29 16:15:03 +11:00
Clément Michaud
adf7bbaf5b
[DOCS] Bootstrap new documentation website based on just-the-docs (#659) 2020-02-29 01:43:59 +01:00
Amir Zarrinkafsh
150a2e177a
[Buildkite] Enable automatic retries for failed github artifact step (#658)
* [Buildkite] Enable automatic retries for failed github artifact step

This is to handle failures which may occur when attempting to upload assets, per: https://buildkite.com/authelia/authelia/builds/465#537f931f-efc3-4f7b-9527-c927c1425a52.

* [Buildkite] Ensure GitHub artifact step is reported as a failure

When the initial command fails and we remove the release, we need to ensure that the exit status is reported as non-zero to trigger the automatic retry.
2020-02-28 22:58:44 +01:00
Amir Zarrinkafsh
4c09df9868
[Buildkite] Fix AUR version tagging (#645)
Need to fetch all tags prior to extracting the correct version.
2020-02-20 11:04:07 +11:00
Amir Zarrinkafsh
447b2461e4
[Buildkite] Automate CD for AUR packages (#644)
* [Buildkite] Automate continuous deployment for AUR packages

* [Buildkite] Make AUR deploy step conditional
2020-02-20 10:25:28 +11:00
Amir Zarrinkafsh
d80becc343
[FIX] Changelog generation for github releases (#641) 2020-02-19 12:25:41 +11:00
Amir Zarrinkafsh
6530780817
[MISC] Utilise Probot for PR commentary (#633)
Remove Buildkite trigger for commentary.
2020-02-14 18:50:38 +11:00
Amir Zarrinkafsh
5588014ea7 [Buildkite] Fix agent key allocation for build step (#624) 2020-02-06 09:18:56 +01:00
Amir Zarrinkafsh
27b8a1b0fe
[Buildkite] Fix issues with releases in CD pipeline (#617)
* [Buildkite] Fix changelog output for github releases

Fetch is required to grab the latest tag, this will ensure the correct data is generated

* [Buildkite] Only clean tags on pushes to master

Also ensure that master tag is not removed on github API failures.

* [Buildkite] Fix tag publishing for releases

* [Buildkite] Minor tweaks to github changelog output
2020-02-05 23:24:19 +11:00
Amir Zarrinkafsh
9c9d8518eb
[Buildkite] Perform PR commentary in pipeline and remove github action (#614)
* [Buildkite] Perform PR commentary in pipeline and remove github action

* [Buildkite] Optimise deployment post-command hook
2020-02-03 20:07:01 +11:00
Amir Zarrinkafsh
1b478e8f3d
[Buildkite] Do not persist Docker secret in builds (#603) 2020-02-01 19:10:18 +11:00
Amir Zarrinkafsh
275af90137
[Buildkite] Re-order pipeline to improve security (#598) 2020-01-31 15:16:46 +11:00
Amir Zarrinkafsh
49e739d009
[Buildkite] Add automatic deployment and removal of Docker images for Branches and PRs (#592) 2020-01-30 08:37:11 +01:00
Amir Zarrinkafsh
722cbb63a0 [Buildkite] Remove redundant if clause in post-checkout hook 2020-01-28 10:06:03 +11:00
Amir Zarrinkafsh
107126929b Update README.md with AUR references and remove CHANGELOG.md (#576)
* Update README.md
Provide badges and references to the AUR for Arch Linux Authelia packages.
Closes #571 #572.

* Add systemd unit file
Include the unit in future release artifacts.

* Remove CHANGELOG.md
As of future releases Changelog details will dynamically be generated.

* Update README.md
Add badge for authelia-git package.

* Update Changelog to only publish explicit Docker tag
Do not include Major and Minor versions, as these will change over time.
2020-01-24 10:21:17 +01:00
Amir Zarrinkafsh
6f669ec8b7 Package config.template.yml in published artifacts 2020-01-22 08:43:06 +01:00
Amir Zarrinkafsh
1059551133
Optimise deploy artifacts step (#564)
* Optimise deploy artifacts step
authelia-scripts is not required to publish GitHub artifacts as we utilise [Hub](https://hub.github.com/), this should save ~10 seconds in this step.

* Specify release number in pipeline

* Change buildkite and github published artifacts back to gzip

* Update README.md
2020-01-20 10:53:55 +11:00
Amir Zarrinkafsh
a0b79c61d2 Group docker deployment steps to prevent race conditions/conflicts 2020-01-18 11:17:25 +11:00
Clement Michaud
ece4423d33 Add back wait block in buildkite pipeline. 2020-01-18 00:12:36 +01:00
Clément Michaud
ce7b6b8167
Build docker image upfront in CI and use it in integration tests. (#555)
* Build docker image upfront in CI and use it in integration tests.

Previously, the development workflow was broken because the container
generated from Dockerfile.CI was used in dev environments but the binary
was not pre-built as it is on buildkite. I propose to just remove that
image and use the "to be published" image instead in integration tests.

This will have several advantages:
- Fix the dev workflow.
- Remove CI arch from authelia-scripts build command
- Optimize CI time in buildkite since we'll cache a way small artifact
- We don't build authelia more than once for earch arch.

* Fix suites and only build ARM images on master or tagged commits

* Optimise pipeline dependencies and Kubernetes suite to utilise cache

* Run unit tests and docker image build in parallel.

* Fix suite trying to write on read only fs.

Co-authored-by: Amir Zarrinkafsh <nightah@me.com>
2020-01-17 20:46:51 +01:00
Amir Zarrinkafsh
9b8be0fef0 Remove Travis and promote Buildkite (#545)
* Remove Travis and promote Buildkite

* Add Docker Size badge to README.md

* Call MicroBadger webhook to update metadata for shields

Add updateMicroBadger function and refactor publishDockerReadme to be called explicitly instead of on every deployManifest call.
2020-01-16 21:57:44 +01:00
Amir Zarrinkafsh
1b39d28cbe
Optimise Buildkite steps
* Utilise multi-threaded zstd for compression

* Fix container removal on dirty exit

* Optimise build step agents
2020-01-07 22:28:04 +11:00
Amir Zarrinkafsh
e4764ad2cf Separate download and extract steps for reporting 2020-01-07 13:13:41 +11:00
Amir Zarrinkafsh
e97a11a9c1 Utilise zstd for compression 2020-01-07 13:13:41 +11:00
Amir Zarrinkafsh
8b8d0c0037 Shellcheck Buildkite pipeline 2020-01-06 02:20:14 +11:00
Amir Zarrinkafsh
30ddfeab38 Build static Go binary 2020-01-05 16:28:28 +11:00
Amir Zarrinkafsh
7f7a3af60c Make Buildkite wait blocks conditional
This is so they do not appear on the Buildkite interface when their subsequent steps will not be executed.
2020-01-01 13:43:02 +11:00
Amir Zarrinkafsh
e85fc6b1b2 Automatically retry failed integration tests
Default parameters retry on exit_status=* and will retry a single step a maximum of 2 times (3 total with initial failure)
2019-12-29 09:06:25 +11:00
Amir Zarrinkafsh
d037fb2728 Allow authelia-scripts to be called in e2etest setup 2019-12-28 09:08:18 +01:00
Amir Zarrinkafsh
2fb20882d9
Utilise Buildkite for Authelia CI/CD (#507)
Publish steps are currently disabled.
2019-12-27 22:07:53 +11:00