ceph-ansible: minimal containerized deployment (docker)

tested with v3.0.26


monitor_interface: eth1
radosgw_interface: eth1
        osd scrub during recovery: false
ceph_docker_image: "ceph/daemon"
ceph_docker_image_tag: latest
containerized_deployment: true


crush_location: true
osd_crush_location: "\"root={{ ceph_crush_root }} rack={{ ceph_crush_rack }} host={{ ansible_hostname }}\""
osd_objectstore: bluestore
osd_scenario: non-collocated
- /dev/sdb
- /dev/sdc
- /dev/sdd
- /dev/sde
- /dev/sdf
- /dev/sdg
- /dev/sdh
- /dev/nvme0n1
- /dev/nvme0n1
- /dev/nvme0n1
- /dev/nvme0n1
- /dev/nvme0n1
- /dev/nvme0n1
- /dev/nvme0n1

group_vars/mons.yml (optional)

openstack_config: true
  name: images
  pg_num: "{{ osd_pool_default_pg_num }}"
  rule_name: ""
  - "{{ openstack_glance_pool }}"

ejabberd + letsencrypt (ssl config)

    port: 5222
    module: ejabberd_c2s
    certfile: "/etc/ejabberd/ejabberd.pem"
    starttls: true
    starttls_required: true
      - "no_sslv2"
      - "no_sslv3"
      - "no_tlsv1"
      - "no_tlsv1_1"
    dhfile: "/etc/ejabberd/dh2048.pem"
    port: 5269
    ip: "::"
    module: ejabberd_s2s_in
      - "no_sslv2"
      - "no_sslv3"
      - "no_tlsv1"
      - "no_tlsv1_1"

s2s_use_starttls: required
s2s_certfile: "/etc/ejabberd/ejabberd.pem"
s2s_dhfile: "/etc/ejabberd/dh2048.pem"

  - "no_sslv2"
  - "no_sslv3"
  - "no_tlsv1"
  - "no_tlsv1_1"

Links https://docs.ejabberd.im/admin/guide/configuration/

RHEV/ovirt – can’t switch SPM role – async_tasks are stucked

On the host with the SPM role

$ vdsClient -s 0 getAllTasksStatuses
{'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {'feb3aaa5-ec1c-42a6-8f17-f7c94891b43f': {'message': '1 jobs completed successfully', 'code': 0, 'taskID': '631fd441-0955-49da-9376-1cba24764aa7', 'taskResult': 'success', 'taskState': 'finished'}, 'b4fe0c6d-d458-4ed2-a9e2-2c0d41914b8f': {'message': '1 jobs completed successfully', 'code': 0, 'taskID': '67e1a2e8-3747-43fa-b0dd-fc469a6f6a02', 'taskResult': 'success',
'taskState': 'finished'}}}

On the RHEV/ovirt manager

$ for i in b4fe0c6d-d458-4ed2-a9e2-2c0d41914b8f feb3aaa5-ec1c-42a6-8f17-f7c94891b43f; do psql --dbname=engine --command="DELETE FROM async_tasks WHERE vdsm_task_id='${i}'"; done
$ for j in b4fe0c6d-d458-4ed2-a9e2-2c0d41914b8f feb3aaa5-ec1c-42a6-8f17-f7c94891b43f; do vdsClient -s 0 clearTask ${j}; done

RHEV/ovirt – find stucked / zombie tasks

Random notes

$ vdsClient -s 0 getAllTasksStatuses
$ vdsClient stopTask <taskid>
$ vdsClient clearTask <taskid>
$ su - postgres
$ psql -d engine -U postgres
> select * from job order by start_time desc;
> select DeleteJob('702e9f6a-e2a3-4113-bd7d-3757ba6bc4ef');


/usr/share/ovirt-engine/dbscripts/engine-psql.sh -c "select * from job;"

ldap initial configuration

A more or less initial configuration for openldap (>2.4)

# to import run:
# ldapmodify -Y EXTERNAL -H ldapi:/// -f $filename
# to verfiy run:
# ldapsearch -Y EXTERNAL -H ldapi:/// -b "olcDatabase={1}hdb,cn=config"
# to create a password:
# slappasswd -h {SSHA} -s admin

dn: olcDatabase={1}hdb,cn=config
changetype: modify
replace: olcSuffix
olcSuffix: dc=example,dc=de
replace: olcAccess
olcAccess: {0}to attrs=userPassword,shadowLastChange by dn="cn=admin,dc=example,dc=de" write by anonymous auth by self write by * none
olcAccess: {1}to dn.base="" by * read
olcAccess: {2}to * by self write by dn="cn=admin,dc=example,dc=de" write by * read
replace: olcRootDN
olcRootDN: cn=admin,dc=example,dc=de
replace: olcRootPW
olcRootPW: {SSHA}4RHgrU6ghLqA21CNI8biQblHtEodToyd

TLS config

dn: cn=config
changetype: modify
add: olcTLSCipherSuite
olcTLSCipherSuite: AES128+EECDH:AES128+EDH
add: olcTLSCACertificateFile
olcTLSCACertificateFile: /etc/ssl/ca.crt
add: olcTLSCertificateFile
olcTLSCertificateFile: /etc/ssl/cert.crt
add: olcTLSCertificateKeyFile
olcTLSCertificateKeyFile: /etc/ssl/keyfile.key
add: olcTLSVerifyClient
# never - allow - try - demand
olcTLSVerifyClient: demand

openldap – tls config
openldap – access

Syncing a fork with git/github

  • Configure a remote
    git remove -v
    # git remote add <name> <url>
    git remote add upstream https://github.com/foo/bar.git
    git remove -v
  • Pull “upstream”
    # git fetch <name>
    git fetch upstream
  • Checkout the master
    git checkout master
  • Merge “upstream” master to local master
    # git merge <name>/<branch>
    git merge upstream/master
  • (optional) Delete old branch
    # git push origin :<branch>
    git push origin :foobar
    git branch -d foobar

Refs https://help.github.com/articles/

Choosing the right scheduler on a virtual maschine (kvm)

The default i/o scheduler is the Completely Fair Queuing (cfq) in the 2.6 kernel. This is not the first choice for a virtual machine/hypervisor. The combination of the noop and the deadline scheduler is much better for a virtualization host.

virtual machine: noop
hypervisor: deadline

Set the scheduler temporarily (vm)

$ echo noop > /sys/block/sda/queue/scheduler

Set the scheduler permanently (vm)

kernel /vmlinuz-3.8.11 root=/dev/vgsystem/lvroot elevator=noop

(For the hypervisor replace noop with deadline!)

And don’t forget to use virtio & raw devices for guest and noatime & nodiratime in fstab wherever it’s possible.

Hint: VMware also recommends the noop scheduler for the guests.

Gentoo and libvirt-0.9.12

Yesterday i’ve started the update process for my system…so far, so good.

After a while emerge finished successfully…of course with a lot of messages, even some messages (from libvirt) that in my kernel config some features are missing e.g.


As usually i’ve ignored these messages 🙁 After a reboot i try’d to start one of my several VMs – without success. Only with a error message

Could not access KVM kernel module: Permission denied 
failed to initialize KVM: Permission denied 
No accelerator found!

Uhm what is this now? I’ve try’d to start qemu-kvm on a shell..that worked. So it must be anything with libvirt and qemu-kvm. After some research on my system i’ve found out that qemu-kvm try’d to start the VMs as the user qemu but /dev/kvm belongs to root:kvm.

Adding the user qemu to the group kvm should help

gpasswd -a qemu kvm

Maybe this is Bug in the ebuild file!?