ceph - down pgs after lost osd

Posted on Wed 19 August 2020 in Ceph • Tagged with ceph, osd, pgs, howto, linux, notepad, work • 1 min read

Initial situation / Issue

  • pool size = 1 (Don't ask why)
  • lost/crashed osd
  • pgs down/incomplete


$ ceph pg <id> query
    "snap_trimq": "[]",
    "snap_trimq_len": 0,
    "state": "active+clean",
    "epoch": 348,
    "acting": [
         "blocked": "peering is blocked due to down osds",
         "down_osds_we_would_probe": [
         "peering_blocked_by": [
               { "osd": 60,
                 "current_lost_at": 0,
                 "comment": "starting or marking this …

Continue reading

ceph - rbd namespaces

Posted on Mon 14 October 2019 in Ceph • Tagged with ceph, rbd, namespace, radosgw, howto, linux, notepad, work • 1 min read

Fun with rbd namespaces


# rbd --namespace $namespace $command $pool/$image
# rbd $command $pool/$namespace/$image


# rbd namespace ls --format=json
# rbd namespace create --namespace newns1
# rbd namespace ls --format=json
# rbd create --namespace newns1 newrbd1 --size 1G
# rbd ls --long
# rbd --namespace newns1 ls --long

Continue reading

gpg - export all (private-)keys

Posted on Sat 04 May 2019 in Linux • Tagged with howto, linux, notepad, gpg • 1 min read

GPG export

for keyid in $( gpg -K --with-colons | grep ^sec | cut -d: -f5 ); do \
  [[ -d ~/.gnupg/export ]] || mkdir ~/.gnupg/export; \
  gpg --export -a --export-options export-minimal --output ~/.gnupg/export/${keyid}.pub $keyid; \
  gpg --export-secret-keys -a --output ~/.gnupg/export/${keyid}.key $keyid;

For a public key without signatures use --export-options export-minimal …

Continue reading

cryptsetup - remote unlocking over ssh

Posted on Fri 03 May 2019 in Linux • Tagged with howto, linux, notepad, work, initrd • 1 min read


Only vaild for Ubuntu/Debian!


  • dropbear-initramfs


ip config

# egrep "(DEVICE|IP)=" /etc/initramfs-tools/initramfs.conf 
   <device>:<autoconf>:<dns0-ip>:<dns1 …

Continue reading

Quickstart guide with kolla-ansible (for rocky)

Posted on Fri 15 February 2019 in openstack • Tagged with openstack, kolla, ansible, howto, linux, notepad, work • 2 min read


  • kolla-ansible (testet with 7.0.1)
  • ansible (testet with 2.6.13)
  • epel-release (for centos)
  • python-pip

And some magic:

  • vagrant
  • vagrant-libvirt plugin
  • vagrant config


We will run all commands on our admin-node (vagrant ssh kolla-admin)

sudo yum install -y epel-release
sudo yum install -y python2-pip
sudo …

Continue reading

ceph/rgw metasearch with elasticsearch 6.x

Posted on Tue 29 January 2019 in Ceph • Tagged with ceph, elasticsearch, radosgw, howto, linux, notepad, work • 1 min read

Houston, we have a problem

Since elasticsearch 6.x requires to provide the correct content-type for a request and the elastic sync module has no option for the content-type....we stucked at elastic 5.x

Error message

{"error":"Content-Type header [] is not supported","status":406}

strace log

strace -e network …

Continue reading

CaaS3: Update hostname

Posted on Tue 18 December 2018 in Linux • Tagged with suse, caas, kubernetes, linux, notepad • 1 min read

linux-q0sf:~ # cat /var/lib/misc/infra-secrets/mariadb-root-password  
linux-q0sf:~ # docker exec -ti d72182149834 bash  
bash-4.3# mysql -u root -p  
Enter password:  
Welcome to the MariaDB monitor. Commands end with ; or \g.  
Your MariaDB connection id is 18  
Server version: 10.0.34-MariaDB SLE …

Continue reading

ceph - sort osds by utilisation

Posted on Thu 20 September 2018 in Ceph • Tagged with ceph, crush, linux, notepad, code • 1 min read

Version 1: keep it simple

$ ceph osd df  | awk '{ print "osd."$1, "size: "$5, "usage: " $8 }' | sort -nk5

OSDs can be listed twice - depends on the crushmap.

Version 2: json + python

$ ceph osd df tree -f json | python sort_hdd_osds.py
osd.28  utilization: 15.278888
osd.15  utilization: 19.700484 …

Continue reading

ceph - wrong osd id with lvm+filestore

Posted on Wed 04 July 2018 in Ceph • Tagged with bluestore, ceph, ceph-ansible, LVM, linux, notepad, howto • 2 min read

Not sure why...but i've found a strange ceph-volume behavior with lvm and filestore.

ceph-volume lvm list shows the wrong osd id while the affected osd is online with a another id.

$ mount | grep ceph-2
/dev/mapper/vg00-datalv1 on /var/lib/ceph/osd/ceph-2 type xfs (rw,relatime,seclabel,attr2 …

Continue reading

selinux - change default type

Posted on Thu 12 April 2018 in Linux • Tagged with selinux, linux • 1 min read

$ matchpathcon /tftpboot
/tftpboot       system_u:object_r:tftpdir_t:s0
$ semanage fcontext -a -t tftpdir_rw_t "/srv/tftp(/.*)?"
$ restorecon -Rv /srv/tftp