CentOS 6 ran out of space, need to reclaim it

Hello everyone,

I am having an issue here with CentOS release 6.6 (Final) that shows all of the space used up, but I can’t tell where the space went.

Seemingly I am using up 100%, according to

df -h

Filesystem                     Size  Used Avail Use% Mounted on
/dev/mapper/vg_sugar-lv_root   50G   49G     0 100% /

So that’s 49G out of 50G, fair enough, but when I try to see what exactly is being used (x to remove mounts):

du -xsh /*

1.8G    /usr
2.0G    /var

That’s all, these are the 2 biggest directories, all others combined are under 1G.

Here is info on Disk /dev/mapper/vg_sugar-lv_root, it does show close to 50G (this is a virtual machine):

fdisk -l

Disk /dev/mapper/vg_sugar-lv_root: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Node count is at 7%:

df -i

Filesystem                   Inodes  IUsed     IFree   IUse% Mounted on
/dev/mapper/vg_sugar-lv_root 3276800 200552    3076248    7% /

The system was rebooted several times, so any log files should’ve been deleted. I’d appreciate some tips in resolving this.

find / -xdev -type f -size +100M

returned only a single file. The problem is not in amount of large files, but that the space is hidden or used for something else, or not released.

 du -ahx / | sort -rh | head -20 
5.0G / 
2.0G /var 
1.8G /usr
1.1G /var/lib 
945M /var/lib/mysql 
909M /var/lib/mysql/sugarcrm 
759M /var/www 
745M /var/www/html 
707M /root 
672M /usr/lib64 
642M /var/www/html/sugar 
611M /usr/share 
480M /var/lib/mysql/sugarcrm/emails_text.MYD 
441M /lib 
392M /lib/modules 

228M /usr/lib

What’s the output from

du -hx --max-depth=1 /

run with elevated privileges ?

Also maybe try

du -hx --max-depth=1 /home

Hi, thanks for trying to help, the output is (under elevated privileges)

du -hx --max-depth=1 /
7.6M    /bin
15M     /sbin
4.0K    /media
4.0K    /selinux
31M     /etc
0       /misc
0       /dev
6.0K    /mnt
441M    /lib
4.0K    /home
4.0K    /srv
4.0K    /boot
1.8G    /usr
0       /proc
1.4M    /tmp
27M     /lib64
16K     /lost+found
0       /sys
41M     /opt
707M    /root
0       /net
1.8G    /var
4.8G    /

du -hx --max-depth=1 /home
16K     /home/lost+found
3.9M    /home

I should say that I am trying to troubleshoot MySQL not starting:
190826 11:00:55 InnoDB: Started; log sequence number 0 8747645
190826 11:00:55 [ERROR] /usr/libexec/mysqld: Error writing file ‘/var/run/mysqld/mysqld.pid’ (Errcode: 28)
190826 11:00:55 [ERROR] Can’t start server: can’t create PID file: No space left on device

Ok, so I’ll post the solution. In this case, it was a network location /mnt/backup mounted over a local directory /mnt/backup.
once unmounted with
umount /mnt/backup

it showed the local directory that was taking up 45G, filled with backups:

cd /mnt/backup/
du -h
39G     ./servers-unix-hq/sugar.gnsa.local
39G     ./servers-unix-hq
4.5G    ./db-mysql-hq/sugar.gnsa.local
4.5G    ./db-mysql-hq

I’ve removed some old backups and MySQL is able to start now.

Thank you for assistance!