Change The OpenStack Glance Image Store

Today I ran into an issue where I ran out of space on my root partition due to multiple ISOs which I have stored in OpenStack Glance. After some tests, I decided to change the Glance image store to an iSCSI volume attached to my controller.

Let’s get started with the basic iSCSI setup (no MPIO), I assume you’ve already created a volume on your storage and set the ACL accordingly :

1. Identify your storage system's iSCSI Discovery IP address

2. Use iscsiadm to discover the volumes:
   [root@TS-Training-OS-01 ~]# iscsiadm -m discovery -t sendtargets -p discovery_IP
   In my case the following volume has been discovered:
   172.21.8.155:3260,2460 iqn.2007-11.com.nimblestorage:jan-openstack-glance-v2057ea2dd8c4465b.00000027.f893ac76

3. Establish a connection to the appropriate volume:
   [root@TS-Training-OS-01 ~]# iscsiadm --mode node --targetname iqn.2007-11.com.nimblestorage:jan-openstack-glance-v2057ea2dd8c4465b.00000027.f893ac76 --portal discovery_ip:3260 --login

4. Once the volume has been connected, use fdisk -l to identify the new disk, in my case it is /dev/sdc. Use mkfs to format the disk in ext4:
   [root@TS-Training-OS-01 ~]# mkfs.ext4 -b 4096 /dev/sdc

5. After the device has been formatted, create a mount-point and change the permissions on it:
   [root@TS-Training-OS-01 ~]# mkdir /mnt/glance
   [root@TS-Training-OS-01 ~]# chmod 777 /mnt/glance

6. Configure fstab to automatically mount /dev/sdc to /mnt/glance after a reboot. Add the following line to /etc/fstab:
   /dev/sdc        /mnt/glance  ext4    defaults        0       0

7. Mount /dev/sdc to /mnt/glance by running the following command:
   [root@TS-Training-OS-01 ~]# mount /dev/sdc /mnt/glance

8. Since we've mounted the new disk to our mount-point, we can go ahead and change the following within /etc/glance/glance-api.conf:
   # ============ Filesystem Store Options ========================
   
   # Directory that the Filesystem backend store
   # writes image data to
   #filesystem_store_datadir=/var/lib/glance/images/
   filesystem_store_datadir=/mnt/glance/

9. Now, restart the glance-api service and any newly uploaded image through glance will be located under /mnt/glance on your controller.
   [root@TS-Training-OS-01 ~]# service openstack-glance-api restart