libretic/ansible-role-k3s
6
0
Fork 0
mirror of https://github.com/PyratLabs/ansible-role-k3s synced 2025-01-07 10:10:19 +01:00
Code Issues Projects Releases Packages Wiki Activity

A number of enhancements for v1.19 release.

Browse source 
- Added option to skip validation checks #47
  - Add SELinux support in containerd #48
  - Added check for Etcd member count #46
  - Moved token to a file #50
  - Added Etcd snapshot configuration options #49
This commit is contained in:
ᗪєνιη ᗷυнʟ 2020-09-21 14:38:51 -04:00 committed by Xan Manning
parent 1438ddde69
commit c447fcec39
12 changed files with 146 additions and 60 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

121
README.md
View file

@ -45,59 +45,65 @@ my spare time so I cannot promise a speedy fix delivery.
Below are variables that are set against all of the play hosts for environment Below are variables that are set against all of the play hosts for environment
consistency. consistency.
| Variable | Description | Default Value | | Variable | Description | Default Value |
|------------------------------------------|-------------------------------------------------------------------------------------|-----------------------------------------| |------------------------------------------|-------------------------------------------------------------------------------------|--------------------------------------------|
| `k3s_cluster_state` | State of cluster: installed, started, stopped, restarted, downloaded, uninstalled. | installed | | `k3s_cluster_state` | State of cluster: installed, started, stopped, restarted, downloaded, uninstalled. | installed |
| `k3s_release_version` | Use a specific version of k3s, eg. `v0.2.0`. Specify `false` for stable. | `false` | | `k3s_release_version` | Use a specific version of k3s, eg. `v0.2.0`. Specify `false` for stable. | `false` |
| `k3s_build_cluster` | When multiple `play_hosts` are available, attempt to cluster. Read notes below. | `true` | | `k3s_build_cluster` | When multiple `play_hosts` are available, attempt to cluster. Read notes below. | `true` |
| `k3s_github_url` | Set the GitHub URL to install k3s from. | https://github.com/rancher/k3s | | `k3s_github_url` | Set the GitHub URL to install k3s from. | https://github.com/rancher/k3s |
| `k3s_install_dir` | Installation directory for k3s. | `/usr/local/bin` | | `k3s_skip_validation` | Skip all tasks that validate configuration. | `false` |
| `k3s_install_hard_links` | Install using hard links rather than symbolic links. | `false` | | `k3s_install_dir` | Installation directory for k3s. | `/usr/local/bin` |
| `k3s_server_manifests_dir` | Path for place the `k3s_server_manifests_templates`. | `/var/lib/rancher/k3s/server/manifests` | | `k3s_install_hard_links` | Install using hard links rather than symbolic links. | `false` |
| `k3s_server_manifests_templates` | A list of Auto-Deploying Manifests Templates. | [] | | `k3s_server_manifests_dir` | Path for place the `k3s_server_manifests_templates`. | `/var/lib/rancher/k3s/server/manifests` |
| `k3s_use_experimental` | Allow the use of experimental features in k3s. | `false` | | `k3s_server_manifests_templates` | A list of Auto-Deploying Manifests Templates. | [] |
| `k3s_use_unsupported_config` | Allow the use of unsupported configurations in k3s. | `false` | | `k3s_use_experimental` | Allow the use of experimental features in k3s. | `false` |
| `k3s_non_root` | Install k3s as non-root user. See notes below. | `false` | | `k3s_use_unsupported_config` | Allow the use of unsupported configurations in k3s. | `false` |
| `k3s_cluster_cidr` | Network CIDR to use for pod IPs | 10.42.0.0/16 | | `k3s_non_root` | Install k3s as non-root user. See notes below. | `false` |
| `k3s_service_cidr` | Network CIDR to use for service IPs | 10.43.0.0/16 | | `k3s_cluster_cidr` | Network CIDR to use for pod IPs | 10.42.0.0/16 |
| `k3s_control_node_address` | Use a specific control node address. IP or FQDN. | _NULL_ | | `k3s_service_cidr` | Network CIDR to use for service IPs | 10.43.0.0/16 |
| `k3s_control_token` | Use a specific control token, please read notes below. | _NULL_ | | `k3s_control_node_address` | Use a specific control node address. IP or FQDN. | _NULL_ |
| `k3s_private_registry` | Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") | _NULL_ | | `k3s_control_token` | Use a specific control token, please read notes below. | _NULL_ |
| `k3s_https_port` | HTTPS port listening port. | 6443 | | `k3s_private_registry` | Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") | _NULL_ |
| `k3s_use_docker` | Use Docker rather than Containerd? | `false` | | `k3s_https_port` | HTTPS port listening port. | 6443 |
| `k3s_no_flannel` | Do not use Flannel | `false` | | `k3s_use_docker` | Use Docker rather than Containerd? | `false` |
| `k3s_flannel_backend` | Flannel backend ('none', 'vxlan', 'ipsec', 'host-gw' or 'wireguard') | vxlan | | `k3s_no_flannel` | Do not use Flannel | `false` |
| `k3s_no_coredns` | Do not use CoreDNS | `false` | | `k3s_flannel_backend` | Flannel backend ('none', 'vxlan', 'ipsec', 'host-gw' or 'wireguard') | vxlan |
| `k3s_cluster_dns` | Cluster IP for CoreDNS service. Should be in your service-cidr range. | _NULL_ | | `k3s_no_coredns` | Do not use CoreDNS | `false` |
| `k3s_cluster_domain` | Cluster Domain. | cluster.local | | `k3s_cluster_dns` | Cluster IP for CoreDNS service. Should be in your service-cidr range. | _NULL_ |
| `k3s_resolv_conf` | Kubelet resolv.conf file | _NULL_ | | `k3s_cluster_domain` | Cluster Domain. | cluster.local |
| `k3s_no_traefik` | Do not use Traefik | `false` | | `k3s_resolv_conf` | Kubelet resolv.conf file | _NULL_ |
| `k3s_no_servicelb` | Do not use ServiceLB, necessary for using something like MetalLB. | `false` | | `k3s_no_traefik` | Do not use Traefik | `false` |
| `k3s_no_local_storage` | Do not use Local Storage | `false` | | `k3s_no_servicelb` | Do not use ServiceLB, necessary for using something like MetalLB. | `false` |
| `k3s_default_local_storage_path` | Set Local Storage Path. Specify `false` for default. | `false` | | `k3s_no_local_storage` | Do not use Local Storage | `false` |
| `k3s_no_metrics_server` | Do not deploy metrics server | `false` | | `k3s_default_local_storage_path` | Set Local Storage Path. Specify `false` for default. | `false` |
| `k3s_kube_apiserver_args` | Customized flag for kube-apiserver process | [] | | `k3s_no_metrics_server` | Do not deploy metrics server | `false` |
| `k3s_kube_scheduler_args` | Customized flag for kube-scheduler process | [] | | `k3s_kube_apiserver_args` | Customized flag for kube-apiserver process | [] |
| `k3s_kube_controller_manager_args` | Customized flag for kube-controller-manager process | [] | | `k3s_kube_scheduler_args` | Customized flag for kube-scheduler process | [] |
| `k3s_kube_cloud_controller_manager_args` | Customized flag for kube-cloud-controller-manager process | [] | | `k3s_kube_controller_manager_args` | Customized flag for kube-controller-manager process | [] |
| `k3s_disable_scheduler` | Disable Kubernetes default scheduler | `false` | | `k3s_kube_cloud_controller_manager_args` | Customized flag for kube-cloud-controller-manager process | [] |
| `k3s_disable_cloud_controller` | Disable k3s default cloud controller manager. | `false` | | `k3s_disable_scheduler` | Disable Kubernetes default scheduler | `false` |
| `k3s_disable_network_policy` | Disable k3s default network policy controller. | `false` | | `k3s_disable_cloud_controller` | Disable k3s default cloud controller manager. | `false` |
| `k3s_write_kubeconfig_mode` | Define the file mode from the generated KubeConfig, eg. `644` | _NULL_ | | `k3s_disable_network_policy` | Disable k3s default network policy controller. | `false` |
| `k3s_datastore_endpoint` | Define the database or etcd cluster endpoint for HA. | _NULL_ | | `k3s_write_kubeconfig_mode` | Define the file mode from the generated KubeConfig, eg. `644` | _NULL_ |
| `k3s_datastore_cafile` | Define the database TLS CA file. | _NULL_ | | `k3s_datastore_endpoint` | Define the database or etcd cluster endpoint for HA. | _NULL_ |
| `k3s_datastore_certfile` | Define the database TLS Cert file. | _NULL_ | | `k3s_datastore_cafile` | Define the database TLS CA file. | _NULL_ |
| `k3s_datastore_keyfile` | Define the database TLS Key file. | _NULL_ | | `k3s_datastore_certfile` | Define the database TLS Cert file. | _NULL_ |
| `k3s_become_for_all` | Enable become for all (where value for `k3s_become_for_*` is _NULL_ | `false` | | `k3s_datastore_keyfile` | Define the database TLS Key file. | _NULL_ |
| `k3s_become_for_systemd` | Enable become for systemd commands. | _NULL_ | | `k3s_become_for_all` | Enable become for all (where value for `k3s_become_for_*` is _NULL_ | `false` |
| `k3s_become_for_install_dir` | Enable become for writing to `k3s_install_dir`. | _NULL_ | | `k3s_become_for_systemd` | Enable become for systemd commands. | _NULL_ |
| `k3s_become_for_usr_local_bin` | Enable become for writing to `/usr/local/bin/`. | _NULL_ | | `k3s_become_for_install_dir` | Enable become for writing to `k3s_install_dir`. | _NULL_ |
| `k3s_become_for_package_install` | Enable become for installing prerequisite packages. | _NULL_ | | `k3s_become_for_usr_local_bin` | Enable become for writing to `/usr/local/bin/`. | _NULL_ |
| `k3s_become_for_kubectl` | Enable become for kubectl commands. | _NULL_ | | `k3s_become_for_package_install` | Enable become for installing prerequisite packages. | _NULL_ |
| `k3s_become_for_uninstall` | Enable become for running uninstall scripts. | _NULL_ | | `k3s_become_for_kubectl` | Enable become for kubectl commands. | _NULL_ |
| `k3s_etcd_datastore` | Use Embedded Etcd as the database backend for HA. (EXPERIMENTAL) | `false` | | `k3s_become_for_uninstall` | Enable become for running uninstall scripts. | _NULL_ |
| `k3s_secrets_encryption` | Use secrets encryption at rest. (EXPERIMENTAL) | `false` | | `k3s_etcd_datastore` | Use Embedded Etcd as the database backend for HA. (EXPERIMENTAL) | `false` |
| `k3s_debug` | Enable debug logging on the k3s service | `false` | | `k3s_etcd_disable_snapshots` | Disable Etcd snapshots. | `false` |
| `k3s_etcd_snapshot_schedule_cron` | Etcd snapshot cron schedule. | "`* */12 * * *`" |
| `k3s_etcd_snapshot_retention` | Etcd snapshot retention. | 5 |
| `k3s_etcd_snapshot_directory` | Etcd snapshot directory. | `/var/lib/rancher/k3s/server/db/snapshots` |
| `k3s_secrets_encryption` | Use secrets encryption at rest. (EXPERIMENTAL) | `f alse` |
| `k3s_debug` | Enable debug logging on the k3s service | `false` |
| `k3s_enable_selinux` | Enable SELinux in containerd. (EXPERIMENTAL) | `false` |
#### Important note about `k3s_release_version` #### Important note about `k3s_release_version`
@ -228,7 +234,7 @@ a Production workload.
If multiple hosts have `k3s_control_node` set to true, you must also set If multiple hosts have `k3s_control_node` set to true, you must also set
`k3s_datastore_endpoint` as the connection string to a MySQL or PostgreSQL `k3s_datastore_endpoint` as the connection string to a MySQL or PostgreSQL
database, or etcd cluster else the play will fail. database, or external Etcd cluster else the play will fail.
If using TLS, the CA, Certificate and Key need to already be available on If using TLS, the CA, Certificate and Key need to already be available on
the play hosts. the play hosts.
@ -241,7 +247,12 @@ configuration you will need to set `k3s_use_unsupported_config` to `true`.
Since K3s v1.19.1 it is possible to use Etcd as the backend database, and this Since K3s v1.19.1 it is possible to use Etcd as the backend database, and this
is done by setting `k3s_etcd_datastore` to true. As this is an experimental is done by setting `k3s_etcd_datastore` to true. As this is an experimental
feature you will also need to set `k3s_use_experimental` to true. feature you will also need to set `k3s_use_experimental` to `true`. The best
practice for Etcd is to define at least 3 members to ensure quorum is
established. In addition to this, an odd number of members is recommended to
ensure a majority in the event of a network partition. If you want to use 2
members or an even number of members, please set `k3s_use_unsupported_config`
to `true`.
#### Important note about `k3s_flannel_interface` #### Important note about `k3s_flannel_interface`

20
defaults/main.yml
View file

@ -16,6 +16,9 @@ k3s_build_cluster: true
# URL for GitHub project # URL for GitHub project
k3s_github_url: https://github.com/rancher/k3s k3s_github_url: https://github.com/rancher/k3s
# Skip all tasks that validate configuration
k3s_skip_validation: false
# Installation directory for k3s # Installation directory for k3s
k3s_install_dir: /usr/local/bin k3s_install_dir: /usr/local/bin
@ -53,9 +56,21 @@ k3s_datastore_endpoint: false
# k3s_datastore_certfile: /path/to/cert.crt # k3s_datastore_certfile: /path/to/cert.crt
# k3s_datastore_keyfile: /path/to/key.pem # k3s_datastore_keyfile: /path/to/key.pem
# Use Etcd for HA Datastore? (EXPERIMENTAL) # Use embedded Etcd for HA Datastore? (EXPERIMENTAL)
k3s_etcd_datastore: false k3s_etcd_datastore: false
# Disable Etcd snapshots
k3s_etcd_disable_snapshots: false
# Etcd snapshot cron schedule. Example below is every 12 hours.
# k3s_etcd_snapshot_schedule_cron: "* */12 * * *"
# Etcd snapshot retention
# k3s_etcd_snapshot_retention: 5
# Etcd snapshot directory
# k3s_etcd_snapshot_directory: /server/db/snapshots
# HTTPS Listening port # HTTPS Listening port
k3s_https_port: 6443 k3s_https_port: 6443
@ -107,6 +122,9 @@ k3s_default_local_storage_path: false
# Use secret encryption at rest (EXPERIMENTAL) # Use secret encryption at rest (EXPERIMENTAL)
k3s_secrets_encryption: false k3s_secrets_encryption: false
# Enable SELinux in containerd (EXPERIMENTAL)
k3s_enable_selinux: false
# with become privileges for # with become privileges for
k3s_become_for_all: false k3s_become_for_all: false
k3s_become_for_systemd: null k3s_become_for_systemd: null

1
molecule/highavailability/playbook-etcd.yml
View file

@ -12,6 +12,5 @@
- name: Set each node to be a control node - name: Set each node to be a control node
set_fact: set_fact:
k3s_control_node: true k3s_control_node: true
when: inventory_hostname in ['node2', 'node3']
roles: roles:
- role: xanmanning.k3s - role: xanmanning.k3s

18
tasks/build/configure-k3s-cluster.yml
View file

@ -19,6 +19,24 @@
check_mode: false check_mode: false
when: k3s_control_token is not defined and ansible_check_mode when: k3s_control_token is not defined and ansible_check_mode
- name: Ensure the cluster NODE_TOKEN file location exists
file:
path: "{{ k3s_token_location }}"
state: directory
mode: 0755
become: "{{ k3s_become_for_systemd | ternary(true, false, k3s_become_for_all) }}"
- name: Ensure k3s cluster token file is present on workers and secondary control nodes
template:
src: cluster-token.j2
dest: "{{ k3s_token_location }}/cluster-token"
mode: 0600
become: "{{ k3s_become_for_systemd | ternary(true, false, k3s_become_for_all) }}"
when: (k3s_control_node and not k3s_primary_control_node)
or not k3s_control_node
notify:
- restart k3s
- name: Ensure k3s service unit file is present - name: Ensure k3s service unit file is present
template: template:
src: k3s.service.j2 src: k3s.service.j2

2
tasks/state-installed.yml
View file

@ -1,6 +1,7 @@
--- ---
- import_tasks: validate/check-environment.yml - import_tasks: validate/check-environment.yml
when: not k3s_skip_validation
- import_tasks: build/preconfigure-k3s.yml - import_tasks: build/preconfigure-k3s.yml
@ -12,6 +13,7 @@
or k3s_release_version is not regex('\\+k3s[1-9]$') or k3s_release_version is not regex('\\+k3s[1-9]$')
- import_tasks: validate/main.yml - import_tasks: validate/main.yml
when: not k3s_skip_validation
- import_tasks: build/get-systemd-context.yml - import_tasks: build/get-systemd-context.yml

1
tasks/state-uninstalled.yml
View file

@ -21,3 +21,4 @@
and (k3s_non_root is not defined or not k3s_non_root) and (k3s_non_root is not defined or not k3s_non_root)
- import_tasks: validate/check-uninstalled.yml - import_tasks: validate/check-uninstalled.yml
when: not k3s_skip_validation

1
tasks/validate/check-experimental-variables.yml
View file

@ -10,6 +10,7 @@
or (k3s_dqlite_datastore is defined and k3s_dqlite_datastore) or (k3s_dqlite_datastore is defined and k3s_dqlite_datastore)
or (k3s_etcd_datastore is defined and k3s_etcd_datastore) or (k3s_etcd_datastore is defined and k3s_etcd_datastore)
or (k3s_secrets_encryption is defined and k3s_secrets_encryption) or (k3s_secrets_encryption is defined and k3s_secrets_encryption)
or (k3s_enable_selinux is defined and k3s_enable_selinux)
- name: Check if experimental dqlite is being used and k3s_use_unsupported_config is configured - name: Check if experimental dqlite is being used and k3s_use_unsupported_config is configured
assert: assert:

9
tasks/validate/check-master-count.yml
View file

@ -22,3 +22,12 @@
success_msg: "Control plane configuration is valid." success_msg: "Control plane configuration is valid."
fail_msg: "Control plane configuration is invalid. Please see notes about k3s_control_node and HA in README.md." fail_msg: "Control plane configuration is invalid. Please see notes about k3s_control_node and HA in README.md."
when: k3s_controller_count | length >= 2 when: k3s_controller_count | length >= 2
- name: Check the conditions when embedded etcd is defined
assert:
that:
- (k3s_controller_count | length >= 3)
and (((k3s_controller_count | length) % 2) == 1)
success_msg: "Control plane configuration is valid."
fail_msg: "Etcd should have a minimum of 3 defined members and the number of members should be odd. Please see notes about HA in README.md"
when: k3s_etcd_datastore and not k3s_use_unsupported_config

8
tasks/validate/check-variables.yml
View file

@ -278,3 +278,11 @@
success_msg: "Secrets encryption at rest supported in {{ k3s_release_version }}" success_msg: "Secrets encryption at rest supported in {{ k3s_release_version }}"
fail_msg: "Secrets encryption at rest is not supported in {{ k3s_release_version }}" fail_msg: "Secrets encryption at rest is not supported in {{ k3s_release_version }}"
when: k3s_secrets_encryption is defined and k3s_secrets_encryption when: k3s_secrets_encryption is defined and k3s_secrets_encryption
- name: Check k3s_enable_selinux against k3s version
assert:
that:
- (k3s_release_version | replace('v', '')) is version_compare('1.17.4', '>=')
success_msg: "SELinux supported in {{ k3s_release_version }}"
fail_msg: "SELinux is not supported in {{ k3s_release_version }}"
when: k3s_enable_selinux is defined and k3s_enable_selinux

1
templates/cluster-token.j2 Normal file
View file

@ -0,0 +1 @@
{{ k3s_control_token }}

23
templates/k3s.service.j2
View file

@ -1,7 +1,8 @@
[Unit] [Unit]
Description=Lightweight Kubernetes Description=Lightweight Kubernetes
Documentation=https://k3s.io Documentation=https://k3s.io
After=network.target Wants=network-online.target
After=network-online.target
[Service] [Service]
Type={{ 'notify' if k3s_control_node else 'exec' }} Type={{ 'notify' if k3s_control_node else 'exec' }}
@ -83,7 +84,20 @@ ExecStart={{ k3s_install_dir }}/k3s
--cluster-init --cluster-init
{% else %} {% else %}
--server https://{{ k3s_control_node_address }}:{{ k3s_https_port }} --server https://{{ k3s_control_node_address }}:{{ k3s_https_port }}
--token {{ k3s_control_token }} --token-file {{ k3s_token_location }}/cluster-token
{% endif %}
{% if k3s_etcd_disable_snapshots %}
--etcd-disable-snapshots
{% else %}
{% if k3s_etcd_snapshot_schedule_cron is defined %}
--etcd-snapshot-schedule-cron "{{ k3s_etcd_snapshot_schedule_cron }}"
{% endif %}
{% if k3s_etcd_snapshot_retention is defined %}
--etcd-snapshot-retention {{ k3s_etcd_snapshot_retention }}
{% endif %}
{% if k3s_etcd_snapshot_directory is defined %}
--etcd-snapshot-dir {{ k3s_etcd_snapshot_directory }}
{% endif %}
{% endif %} {% endif %}
{% endif %} {% endif %}
{% if k3s_secrets_encryption is defined and k3s_secrets_encryption %} {% if k3s_secrets_encryption is defined and k3s_secrets_encryption %}
@ -120,7 +134,10 @@ ExecStart={{ k3s_install_dir }}/k3s
{% else %} {% else %}
agent agent
--server https://{{ k3s_control_node_address }}:{{ k3s_https_port }} --server https://{{ k3s_control_node_address }}:{{ k3s_https_port }}
--token {{ k3s_control_token }} --token-file {{ k3s_token_location }}/cluster-token
{% endif %}
{% if k3s_enable_selinux %}
--selinux
{% endif %} {% endif %}
{% if k3s_resolv_conf is defined and k3s_resolv_conf %} {% if k3s_resolv_conf is defined and k3s_resolv_conf %}
--resolv-conf {{ k3s_resolv_conf }} --resolv-conf {{ k3s_resolv_conf }}

1
vars/main.yml
View file

@ -35,3 +35,4 @@ k3s_controller_count: []
k3s_systemd_context: system k3s_systemd_context: system
k3s_systemd_unit_directory: "/etc/systemd/{{ k3s_systemd_context }}" k3s_systemd_unit_directory: "/etc/systemd/{{ k3s_systemd_context }}"
k3s_token_location: "/etc/rancher"