For anybody who has had to manage secure Red Hat Linux servers, the term SELinux (Security-Enhanced Linux) probably sounds familiar. SELinux which was initially developed by the National Security Agency(NSA) gave system administrators a way to issue fine-grained control to a system.

Troubleshooting permission issues often resulted an administrator using audit2allow to parse the Linux audit logs and inform you of the SELinux restrictions that was the cause of pain.

However, in this day of containerized applications and micro-services, fewer and fewer people have to deal with SELinux. And with this evolution also comes an evolution of the tools we would use.

So earlier today, I was getting permission issues while attempting to create resources in an Openshift environment via an operator I was writing. While the errors clearly spelled out I had permission issues, it did not clearly define what the issue was.

So here comes the tool, audit2rbac which appears inspired by the aforementioned audit2allow from SELinux. While reviewing the README on their github repository, I learn I would need to enable the audit logs on the target Kubernetes cluster am using.

Since am using Openshift, I presume the audit logs were probably disabled during installation. A quick search on Google reveals some good information. Openshift API server logs can be retrieved using oc, the Openshift command-line client.


The dirty work

  1. The next step is to get the list of master nodes and the available logs
❯ oc adm node-logs --role=master --path=openshift-apiserver/ 
ip-10-0-125-176.ec2.internal audit-2020-05-06T20-30-42.723.log
ip-10-0-125-176.ec2.internal audit.log

2. Once done, identify and take not of the master node and name of audit log you would like to retrieve and make it available locally.

❯ oc adm node-logs ip-10-0-125-176.ec2.internal --path=openshift-apiserver/audit.log | tee local-audit.log

3.  In the final command, I use audit2rbac command to generate a role and rolebinding for my service account.

❯ audit2rbac -f local-audit.log --user system:serviceaccount:default:service-account-name
Opening audit source...
Loading events........
Evaluating API calls...
Generating roles...
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  annotations:
    audit2rbac.liggitt.net/version: v0.8.0
  labels:
    audit2rbac.liggitt.net/generated: "true"
    audit2rbac.liggitt.net/user: system-serviceaccount-default-service-account-name
  name: audit2rbac:system:serviceaccount:default:service-account-name
  namespace: default
rules:
- apiGroups:
  - route.openshift.io
  resources:
  - routes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations:
    audit2rbac.liggitt.net/version: v0.8.0
  labels:
    audit2rbac.liggitt.net/generated: "true"
    audit2rbac.liggitt.net/user: system-serviceaccount-default-service-account-name
  name: audit2rbac:system:serviceaccount:default:service-account-name
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: audit2rbac:system:serviceaccount:default:service-account-name
subjects:
- kind: ServiceAccount
  name: service-account-name
  namespace: default
Complete!


Step 2 and 3 above can also be combined into a single command as shown below:

❯ audit2rbac -f <(oc adm node-logs ip-10-0-168-58.ec2.internal --path=openshift-apiserver/audit.log) --user system:serviceaccount:default:service-account-name
Opening audit source...
Loading events........
Evaluating API calls...
Generating roles...
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  annotations:
    audit2rbac.liggitt.net/version: v0.8.0
  labels:
    audit2rbac.liggitt.net/generated: "true"
    audit2rbac.liggitt.net/user: system-serviceaccount-default-service-account-name
  name: audit2rbac:system:serviceaccount:default:service-account-name
  namespace: default
rules:
- apiGroups:
  - route.openshift.io
  resources:
  - routes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations:
    audit2rbac.liggitt.net/version: v0.8.0
  labels:
    audit2rbac.liggitt.net/generated: "true"
    audit2rbac.liggitt.net/user: system-serviceaccount-default-service-account-name
  name: audit2rbac:system:serviceaccount:default:service-account-name
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: audit2rbac:system:serviceaccount:default:service-account-name
subjects:
- kind: ServiceAccount
  name: service-account-name
  namespace: default
Complete!

Conclusion


If you are following closely, you would probably notice by now the application in question was running in the default project(Openshift) / namespace (Kubernetes)  and the service account in use is called service-account-name.