[ABORTED] HELP! - Where's all my space gone

Linux Support
1

I’ve noticed the partition which I installed Ubuntu on has become rather full but I don’t know why or how. But, saying that, I have an idea…
I have 4 external hard drives, in the BIOS there is a setting to make the computer think the external HD’s are internal. I successfully did this with 2 external HD’s and then later got 2 more. I have been having problems with the 2 new HD’s being assigned different ‘locations’ even after doing the same with the 2 new HD’s.

/dev/sda5: UUID=“b670b54e-542b-43fe-9329-f620887acaa1” TYPE=“ext4”
/dev/sda1: UUID=“a4cc9e70-36d1-4dcd-ad90-ec5c2aaf4b51” TYPE=“ext4”
/dev/sda2: UUID=“718377f7-8759-44ad-888d-e0fb3f91c5e4” TYPE=“swap”
/dev/sda6: LABEL=“Drive5” UUID=“127a7d9d-be7b-4ec8-9a05-661dd763d9e6” TYPE=“ext4”
/dev/sdb1: LABEL=“Drive1” UUID=“d5e1db61-b980-4624-801c-e0546b288b8f” TYPE=“ext4”
/dev/sdc1: LABEL=“Drive3” UUID=“3211899a-4095-4d76-8d34-70c534d52374” TYPE=“ext4”
/dev/sdh1: LABEL=“EXT-02” UUID=“e59d54c9-6b03-4788-94aa-ef1adb4ad2dd” TYPE=“ext4”
/dev/sdi1: LABEL=“EXT-03” UUID=“07056c2f-ff73-45b6-b633-8fee8eda675e” TYPE=“ext4”
/dev/sdj1: LABEL=“EXT-01” UUID=“70364fca-b0a2-4058-a976-1bd435e101a5” TYPE=“ext4”
/dev/sdk1: LABEL=“EXT-04” UUID=“b304532e-1908-47bd-bb54-5bdcc9ee7d39” TYPE=“ext4”

The drives labelled ‘EXT-03’ & ‘EXT-04’ have been given ‘sdl1’ & ‘sdm1’ intermittently. I think even the other 2 HD’s, ‘EXT-01’ & ‘EXT-02’ have been given different ‘sd’s’ too.

When the HD’s have not being mounted properly, though there has been entries in /media/pooky2483 relating to the drives which did not mount at boot. Such as /media/pooky2483/drive2 & /media/pooky2483/EXT-02 & /media/pooky2483/EXT-04.
On last boot I did not have any problems with any of the drives not mounting.

Here are some screenshots to show I have a problem…
I don’t know how or why it shows filesize used of 140TB as the partition is only 158.4Gb and I have not included ‘media’.

http://imagizer.imageshack.us/v2/880x800q90/633/hcIeFQ.jpg

Contents of ‘media’. I deleted the ‘drives’ in ‘pooky2483’ checking that I have the same file(s) in the ‘real’ drive.

http://imagizer.imageshack.us/v2/880x800q90/911/5zYrmb.jpg

Is there anything I can do to find out why I am reported to be using 140Tb while only having a 158Gb partition.

2

Mmm, your file / folder count doesn’t look unreasonable … I’d be looking for an unreasonably large ‘sparse’ file.

Generally when you open a file and write it it, if you write 10Mb, then close the file, you will use 10Mb of disk space and the system will report 10Mb less free space. However, when treat a file as a ‘sparse’ file, the amount of space consumed and the amount of space ‘allocated’ are two different things.

This feature is often used by virtual machines, you can create a virtual machine image file (to emulate a VM’s hard disk) of say 100G, but do it ‘sparsely’, so when you come to look at the file you will get two different size results depending on how you look.

If you do a

ls -lah <file>

, it will read the file’s “metadata” which should report a 100G file.
If on the other hand you do a “du -sh ” , it will report the space consumed, which for a new machine image might be a few tens of Mb.

An easy way to get the ‘real’ usage figure would be to run

du -sh /

from the command line. (this may take a while)

An easy way to looks for large files would be say;

find / -size +10G -ls

I don’t suppose you’ve created any virtual machines on your system recently?

3

How big are
/ffmulticonverter
/lost+found
/root
/srv
/temp

if I had to take a guess, I’d guess you’ve been sending stuff to trash “as root” … which doesn’t get cleared when you empty trash as a regular user.

4

How do I know what to put there?

If on the other hand you do a "du -sh " , it will report the space consumed, which for a new machine image might be a few tens of Mb.

See file ‘du sh.txt’

An easy way to looks for large files would be say; 
find / -size +10G -ls

See file ‘find.txt’

I don't suppose you've created any virtual machines on your system recently?

Nope.

2.3Mb (2,327,890) 51 Files 6 Sub-folders

/lost+found

16.4k (16,384) 0 Files 0 Sub-folders

/root

1.2Mb (1,177,931) 66 Files 46 Sub-folders

/srv

4.1k (4096) 0 Files 0 Sub-folders

/temp

2.7Mb (2,721,257) 55 Files 13 Sub-folders

I’ve done a filesize check on each individual folder and the culprit is ‘proc’ at
140.7Tb (140,737,486,307,584) 128,026 Files 7,598 Sub-folders

if I had to take a guess, I'd guess you've been sending stuff to trash "as root" .. which doesn't get cleared when you empty trash as a regular user.

…So, I’d have to empty the trash as ‘root’!

5

You’re checking the file system usage incorrectly

if you right-click a virtual file system (such as /proc) and select “Properties” it will report all sorts of weird file system sizes … eg. run:

sudo du -bhs /proc

it’ll tell you that it’s 128TB in size :o
yet:

du -sh /proc

will tell you its correct size, which most likely = 0

run:

df -h

and

sudo du -sh /proc

and post the output

6

du: cannot access ‘/proc/25943/task/25943/fd/3’: No such file or directory
du: cannot access ‘/proc/25943/task/25943/fdinfo/3’: No such file or directory
du: cannot access ‘/proc/25943/fd/3’: No such file or directory
du: cannot access ‘/proc/25943/fdinfo/3’: No such file or directory
129T /proc

yet: 
du -sh /proc

will tell you its correct size, which most likely = 0

See attached file ‘proc.txt’

run: 
df -h

Filesystem Size Used Avail Use% Mounted on
/dev/sda1 148G 138G 3.0G 98% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 3.8G 4.0K 3.8G 1% /dev
tmpfs 774M 2.0M 772M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 3.8G 364K 3.8G 1% /run/shm
none 100M 32K 100M 1% /run/user
/dev/sdb1 230G 91G 127G 42% /media/Drive1
/dev/sdc1 1.8T 372G 1.4T 22% /media/Drive3
/dev/sda6 148G 65G 76G 47% /media/Drive5
/dev/sdj1 3.6T 2.7T 779G 78% /media/EXT-01
/dev/sdh1 2.7T 1.8T 801G 70% /media/EXT-02
/dev/sdk1 3.6T 2.0T 1.5T 59% /media/EXT-04
/dev/sdi1 2.7T 2.4T 238G 91% /media/EXT-03

and 
sudo du -sh /proc

and post the output

du: cannot access ‘/proc/26678/task/26678/fd/3’: No such file or directory
du: cannot access ‘/proc/26678/task/26678/fdinfo/3’: No such file or directory
du: cannot access ‘/proc/26678/fd/3’: No such file or directory
du: cannot access ‘/proc/26678/fdinfo/3’: No such file or directory
0 /proc

I’m NOT going to do it unless you tell me it’s safe to do so, can I delete the contents of ‘proc’. What is the purpose of the files in ‘proc’?

7

should have been

sudo du -sh /proc

and also post the output from:

ls -l /proc

Are you SURE that /proc is the only oversized directory ?

/proc is running processes, they are not “real” files … it’s just that Linux represents EVERYTHING as a file (including processes).

Have you tried simply rebooting and seeing if

df -h

gives you a more reasonable usage figure

8

du: cannot access ‘/proc/5423/task/5423/fd/3’: No such file or directory
du: cannot access ‘/proc/5423/task/5423/fdinfo/3’: No such file or directory
du: cannot access ‘/proc/5423/fd/3’: No such file or directory
du: cannot access ‘/proc/5423/fdinfo/3’: No such file or directory
0 /proc

Are you SURE that /proc is the only oversized directory ?

yes

/proc is running processes, they are not "real" files .. it's just that Linux represents EVERYTHING as a file (including processes).

Have you tried simply rebooting and seeing if

df -h

gives you a more reasonable usage figure

/dev/sda1 148G 138G 2.8G 99% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 3.8G 12K 3.8G 1% /dev
tmpfs 774M 2.0M 772M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 3.8G 748K 3.8G 1% /run/shm
none 100M 44K 100M 1% /run/user
/dev/sdb1 230G 91G 127G 42% /media/Drive1
/dev/sdc1 1.8T 372G 1.4T 22% /media/Drive3
/dev/sda6 148G 65G 76G 47% /media/Drive5
/dev/sdj1 3.6T 2.7T 778G 78% /media/EXT-01
/dev/sdh1 2.7T 1.8T 801G 70% /media/EXT-02
/dev/sdk1 3.6T 2.0T 1.5T 59% /media/EXT-04
/dev/sdi1 2.7T 2.4T 240G 91% /media/EXT-03

And, yes, I have rebooted, a couple of times in fact. And it still shows as an almost full Hd.
Could I delete all or some of the files in ‘proc’?

9

They aren’t genuine files, they are just a placeholder for bits of the running kernel and such.
Sda1 is pretty damn full - where is your /home folder mounted?

10

My /home folder is actually spread on several drives, usiny symlinks to direct where the actual folde(s) is/are. I’ve done it that way so as not to fill up the ‘boot’ drive with all ‘my’ stuff and to leave the boot drive for the system to use.

11

You’re worrying about /proc too much

it’s actually 0 bytes in size

what’s the output from:

du -sh /*
12

I did as you asked and then with ‘sudo’ at the begining and got different results, the first one resulted in a large output, which I have uploaded as ‘du -sh01.txt’ and as the 'sudo’version was smaller, here it is…

12K /bacula
12M /bin
46M /boot
4.0K /cdrom
4.0K /dev
16M /etc
2.4M /ffmulticonverter-1.7.1
18G /home
0 /initrd.img
310M /lib
4.0K /lib64
16K /lost+found
9.3T /media
4.0K /mnt
172M /opt
du: cannot access ‘/proc/8216/task/8216/fd/3’: No such file or directory
du: cannot access ‘/proc/8216/task/8216/fdinfo/3’: No such file or directory
du: cannot access ‘/proc/8216/fd/3’: No such file or directory
du: cannot access ‘/proc/8216/fdinfo/3’: No such file or directory
0 /proc
1.4M /root
2.3M /run
20M /sbin
4.0K /srv
0 /sys
1.5M /tmp
11G /usr
2.1G /var
0 /vmlinuz

But WHY does it show as 0 when my boot partition is 99% full with 2.8Gb free. Can I delete any or all of it’s contents safely without harming the running of the system in any way?

13

delete the contents of what ?

what’s the contents of:

gedit /etc/fstab

or if this is KDE

kate /etc/fstab

or whatever text editor you have.

14

/proc


http://imagizer.imageshack.us/v2/850x800q90/901/sEjCDB.jpg

See bottom right… Capacity nearly FULL!

what's the contents of: 
gedit /etc/fstab

or if this is KDE

kate /etc/fstab

or whatever text editor you have.

[spoiler]# /etc/fstab: static file system information.

Use ‘blkid’ to print the universally unique identifier for a

device; this may be used with UUID= as a more robust way to name devices

that works even if disks are added and removed. See fstab(5).

/ was on /dev/sda1 during installation

UUID=a4cc9e70-36d1-4dcd-ad90-ec5c2aaf4b51 / ext4 errors=remount-ro 0 1

swap was on /dev/sda2 during installation

UUID=718377f7-8759-44ad-888d-e0fb3f91c5e4 none swap sw 0 0

/dev/sdb1 /media/Drive1 ext4 rw,user,exec 0 0

UUID=d5e1db61-b980-4624-801c-e0546b288b8f ext4 rw,user,exec 0 0

/dev/sdc1 /media/Drive3 ext4 rw,user,exec 0 0

UUID=3211899a-4095-4d76-8d34-70c534d52374 ext4 rw,user,exec 0 0

/dev/sda6 /media/Drive5 ext4 rw,user,exec 0 0

/dev/sdj1 /media/EXT-01 ext4 rw,user,exec 0 0
#UUID=70364fca-b0a2-4058-a976-1bd435e101a5 ext4 rw,user,exec 0 0
#LABEL=“EXT-01” UUID=“70364fca-b0a2-4058-a976-1bd435e101a5” TYPE=“ext4”

/dev/sdh1 /media/EXT-02 ext4 rw,user,exec 0 0
#UUID=e59d54c9-6b03-4788-94aa-ef1adb4ad2dd ext4 rw,user,exec 0 0
#LABEL=“EXT-02” UUID=“e59d54c9-6b03-4788-94aa-ef1adb4ad2dd” TYPE=“ext4”

/dev/sdi1 /media/EXT-03 ext4 rw,user,exec 0 0
#UUID=07056c2f-ff73-45b6-b633-8fee8eda675e ext4 rw,user,exec 0 0
#LABEL=“EXT-03” UUID=“07056c2f-ff73-45b6-b633-8fee8eda675e” TYPE=“ext4”

/dev/sdk1 /media/EXT-04 ext4 rw,user,exec 0 0
#UUID=b304532e-1908-47bd-bb54-5bdcc9ee7d39 ext4 rw,user,exec 0 0
#LABEL=“EXT-04” UUID=“b304532e-1908-47bd-bb54-5bdcc9ee7d39” TYPE=“ext4”

/dev/sda1: UUID=“a4cc9e70-36d1-4dcd-ad90-ec5c2aaf4b51” TYPE=“ext4”

/dev/sda2: UUID=“718377f7-8759-44ad-888d-e0fb3f91c5e4” TYPE=“swap”

/dev/sda5: UUID=“b670b54e-542b-43fe-9329-f620887acaa1” TYPE=“ext4”

/dev/sda6: LABEL=“Drive5” UUID=“127a7d9d-be7b-4ec8-9a05-661dd763d9e6” TYPE=“ext4”

/dev/sdb1: LABEL=“Drive1” UUID=“d5e1db61-b980-4624-801c-e0546b288b8f” TYPE=“ext4”

/dev/sdc1: LABEL=“Drive3” UUID=“3211899a-4095-4d76-8d34-70c534d52374” TYPE=“ext4”

/dev/sdh1: LABEL=“EXT-02” UUID=“e59d54c9-6b03-4788-94aa-ef1adb4ad2dd” TYPE=“ext4”

/dev/sdi1: LABEL=“EXT-03” UUID=“07056c2f-ff73-45b6-b633-8fee8eda675e” TYPE=“ext4”

/dev/sdj1: LABEL=“EXT-01” UUID=“70364fca-b0a2-4058-a976-1bd435e101a5” TYPE=“ext4”

/dev/sdk1: LABEL=“EXT-04” UUID=“b304532e-1908-47bd-bb54-5bdcc9ee7d39” TYPE=“ext4”

#[/spoiler]

15

There’s no point deleting anything in /proc, they have a zero file size anyway … and I’m fairly sure it’d not be a good idea to do so.

Just as an experiment, comment out all the lines that pertain to your external drives … then reboot so they’re not mounted.

then post the output from:

df -h
16

That’s going to cause problems as I have Xeoma running at boot (a security camera program which starts writing to ‘EXT-04’ as soon as it starts and continues continuously) and for some reason, I can’t un-tick it from running at boot as I am unable to get System Settings running to be able to change ‘run at boot’ settings.

When I previously booted and ‘EXT-04’ was unable to be mounted, Xeoma started writing to it’s folder where it was installed on the boot drive in ‘root/media/pooky2483/EXT-04’. I was unable to access ANY of the folders in ‘root/media/pooky2483/’ that were created when they could not be mounted at boot.

17

I have a feeling that something from your other drives is being reported as part of the other because of the symlinking.

and there’s only one way to find out…

18

Ok, never touch anything in /proc, it’s not a “real” filesystem, it’s a virtual filesystem presented by the kernel to enable you to view and sometimes modify kernel settings. It won’t take up any space.

Check free space with “df -h”.

For the filesystem that’s full, you want;

find <filesystem> -size +10G -ls

As per your previous comments. When you see or it means that you need to put a real file or filesystem name in this position for the command to do something meaningful. or have no meaning as such, they’re just there as placeholders to indicate what you need to put there.

For Example;

$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 7.8G 0 7.8G 0% /dev
tmpfs 1.6G 162M 1.5G 11% /run
/dev/sdc2 92G 50G 38G 57% /
tmpfs 7.9G 368M 7.5G 5% /dev/shm
tmpfs 5.0M 20K 5.0M 1% /run/lock
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
tmpfs 7.9G 85M 7.8G 2% /tmp
/dev/mapper/vols-home 493G 440G 53G 90% /home
cgmfs 100K 0 100K 0% /run/cgmanager/fs
tmpfs 1.6G 64K 1.6G 1% /run/user/1000

In this case, I would be running;

find /home -size +10G -ls

To see what oversize files I have.

19

It’s your /media directory that is huge, and I agree with Mark - I think this has to do with symlinking. I think the “du” command is following the links, and reporting your external drives as part of /media, rather than as their own drives.

This is only a problem if other programs (e.g package manager) start doing the same thing, and complaining about lack of space. Do you know if they are symlinks or hard links?

20

Well, I’ve managed to do a reboot with the Hd’s commented out in fstab and temporarily set Xeoma to not run at boot.
I’ve opened dolphin at root and it was 2.3Gb free and then on a second look, it showed 1.8Gb free.

Also, I am having temporary trouble starting applications, such as System settings, Terminal and Kate, which at the moment, Kate will not start. (which this be a different problem - But I’m unsure).

Filesystem Size Used Avail Use% Mounted on
/dev/sda1 148G 139G 1.4G 100% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 3.8G 4.0K 3.8G 1% /dev
tmpfs 774M 2.0M 772M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 3.8G 260K 3.8G 1% /run/shm
none 100M 28K 100M 1% /run/user
/dev/sdb1 230G 94G 125G 43% /media/Drive1
/dev/sdc1 1.8T 291G 1.5T 17% /media/Drive3
/dev/sda6 148G 65G 76G 47% /media/Drive5

See above

For the filesystem that's full, you want;[/font] 
find <filesystem> -size +10G -ls

I’ve tried

sudo find /root -size +10G -ls

sudo find /dev/sda1 -size +10G -ls

sudo find /home -size +10G -ls

And all three show nothing, no output, just straight to the ‘~$’ waiting for commands.

‘/root’ is STILL showing as almost full but after typing

sudo du -sh /*

it shows as 0.
12K /bacula
12M /bin
46M /boot
4.0K /cdrom
4.0K /dev
16M /etc
2.4M /ffmulticonverter-1.7.1
18G /home
0 /initrd.img
310M /lib
4.0K /lib64
16K /lost+found
557G /media
4.0K /mnt
172M /opt
du: cannot access ‘/proc/5571/task/5571/fd/3’: No such file or directory
du: cannot access ‘/proc/5571/task/5571/fdinfo/3’: No such file or directory
du: cannot access ‘/proc/5571/fd/3’: No such file or directory
du: cannot access ‘/proc/5571/fdinfo/3’: No such file or directory
0 /proc
1.4M /root
2.2M /run
20M /sbin
4.0K /srv
0 /sys
1.6M /tmp
11G /usr
2.0G /var
0 /vmlinuz

I’m thinking as a final drastic measure to wipe the boot Hd and reformat it without making partitions and do a fresh install of Kubuntu.