nonstop HD activity-freezing

[mako6]

UBUNTU 10.4.1 AMD 64 on Acer 5003WLMi Turion64 ML32 1.8Gh, 1Gb 400Mh RAM

I'm convinced I've got some Ubuntu virus or bug (yeah, I know they don't exist) cause I was using unsecured neighbor wifi for 7 months (now got secure Internet): the HD just goes on rants of activity- 5 minutes and system semi-freezes with limited mouse movement and delay of any change, opening or closing of any program. It's so bad and annoying I sometimes REISUB it. May be connected w FF, but has continued quite a while even after closing.

Anyway I scanned it with a new AVIRA Boot AV CD and got these results (didn't complete cause it started doing all the other drives in Media- how to prevent that?). Is this history thing important (50 warnings), what is it, endless recursions bad, and can I delete it without consequences (I LIKE history)?  How about changelog? Full scan file attached.

WARNING: [Archive not completly scanned. Reason: maximum recursion level (10) reached] /media/Devices/hda9/usr/share/clamav-testfiles/clam_cache_emax.tgz --> clam_cache_ema x.tar --> clam02.tgz --> clam02.tar --> clam03.tgz --> clam03.tar --> clam04.tgz --> clam04.tar --> clam05.tgz --> clam05.tar --> clam06.tgz


WARNING: [Archive not completly scanned. Reason: maximum recursion level (10) reached] /media/Devices/hda9/usr/share/doc/libxvidcore4/changelog.gz --> changelog --> object --> object --> object --> object --> object --> object --> object --> object --> object


WARNING: [Archive not completly scanned. Reason: maximum recursion level (10) reached] /media/Devices/hda9/usr/share/doc/w3m/ja/HISTORY.gz --> HISTORY --> object --> object --> object --> object --> object --> object --> object --> object --> object

[Mark Greaves (PCNetSpec)]

All those warnings mean is that Avira can only scan 10 directories deep (maximum recursion level (10) reached) .. it doesn't mean you have a virus.

If you think it's connected with Firefox .. try clearing the Firefox cache.

But at a *guess* I'd say you have either a failing hard drive (bad sectors that need to be re-read several times before the data is read properly), or a screwed up file system (try running fsck).

Does "SMART Data" in "Disk Utility" suggest any errors ?

[mako6]

Well the results are strange. This Samsung 160GB disk was never too good, and initially had such weird numbers here without
being flagged that I didn't believe it + ignored it. So they're right + my disk is crap?- well I bought it in 2008 Ukraine where they dump all kinds of defective crap. Oh, man. Better back up files.

3 bad sectors; only warning in red is: CURRENT PENDING SECTOR COUNT: 3 waiting to be remapped;

UNCORRECTABLE SECTOR COUNT: 3

It says "GOOD" for read-write errors, but with a value of 1271733 Normalized 102, Worst 72, Threshhold 6,

SEEK ERROR RATE: "Good" , Value 8776831664, Norm: 79, Worst 60, Thresh 30

HARDWARE ECC Recovered: Value: 199191785, Norm 73, WOrs: 57, Thresh 0

LOAD/UNLOAD CYCLE COUNT: 96880, NORM 52, Wor: 52, Thresh: 0

POWER OFF RETRACT COUNT: 781, Norm 100, Worst 100, Thresh: 0

All GOOD: Power On 198 days (of almost 2 years); Power Cycle Count 1945: Start-STOP count: 1796

How do I just force it to ignore the bad sectors (how big is that)?

IS instruction: sudo fsck    ? Is that analysis or FIXING IT?

[mako6]

whats the proper FSCK instruction  to analyze and fix disk or file system:  SDA9. Is the "check file system" on Disk Utility any good?  Can I figure out which partition these errors are on?

[Mark Greaves (PCNetSpec)]

DO NOT run fsck on a mounted disk/partition

First try this ..

run:

Code: [Select]

sudo touch /forcefsck

then reboot.

That will run fsck at bootup

To run fsck on particular partitions (and ones not listed in fstab), you'll want to boot to a LiveCD/LiveUSB (so the partitions aren't mounted) .. then let's say you want to run fsck on /dev/sda1

Code: [Select]

sudo fsck /dev/sda1

As I said .. DO NOT run fsck on a mounted partition.


Just a thought, but whilst booted to the hard drive .. what's the output from:

Code: [Select]

free -m

[mako6]

free -m
             total       used       free     shared    buffers     cached
Mem:           870        842         28          0         15        130
-/+ buffers/cache:        696        174
Swap:         1048        457        591
mako4@mak4-laptop:~$

Would it even permit fsck on a mounted disk?

[Mark Greaves (PCNetSpec)]

Would it even permit fsck on a mounted disk?

From memory, I think it will .. but only after warning you not to .. but I could be wrong.

Or maybe it will but only if you use the "force" (-f ?) option .. I can't remember


It looks like you're VERY low on memory .. when you ran free -m was that with a load of applications open ?

Your system was already accessing the swap partition,  which will cause "disk thrashing" and system slowdown as stuff is constantly swapped out of RAM to the hard drives swap partition (which is MUCH slower than RAM).

With only 1GB RAM, you'd probably be better off with a "light" distro such as Peppermint/Lubuntu/Crunchbang/etc.

Or just fit some more RAM

[mako6]

1 gb should be fine for Ubuntu. Even Windows XP runs OK, but Ubuntu was always lighter. Yeah it's been open for a while and I've opened and closed disc burning software (think that eats up all memory). Wonder if have some bad RAM? Other times I ran that free program, it didn't show any swap use. Thought Ubuntu doesn't leak or retain memory loss if you close programs

But what about those horrible disc stats- think maybe these endless writes are trying to access bad sectors. What are risks of fsck- can it destroy big chunks of data, or trash OS? Seem to remember, I had these lousy stats before but other disk analysis programs in Windows said it was fine. What is difference between fsck (cleaner operation?) and using "check (+fix) file system" in disk utility?

[SeZo]

What are risks of fsck- can it destroy big chunks of data, or trash OS?

If you are running fsck as Mark suggested then the risk is minimal (no worst than a failing drive), run:

Code: [Select]

sudo touch /forcefsck

then reboot. This will run the check on the boot up.
Ubuntu should automatically run the check on every 20 or so boot up if you got the flag set in your /etc/fstab:

Code: [Select]

/dev/sda1	/	ext4	rw,errors=remount-ro	0	1

Notice the 1 at the far right. If yours is 0 then change it.

You can also reduce the swappiness:
First see the current setting (default is 60):

Code: [Select]

cat /proc/sys/vm/swappiness

To change the system swappiness value, open /etc/sysctl.conf as root. Then, change or add this line to the file:

Code: [Select]

vm.swappiness = 10

Reboot for the change to take effect

[Mad Penguin]

1 gb should be fine for Ubuntu.

ROFL.

Looks like you have some corruption on your hard drive - it needs fscking. That said that, looking at your SMART output I'd say that drive is fsck'd and needs replacing asap. (Samsung drives in general are great on performance, cheap, but have a terrible average lifespan .. I have ~ 50 drives, failure rate in the first 3 years is probably ~ 30-40%)

Another way to force a fsck on reboot is;

Code: [Select]

tune2fs -c199 -C200 /dev/sda1

Sample output for a Decent (expensive) drive; [WD 600G VelociRaptor]

Code: [Select]

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       1
  3 Spin_Up_Time            0x0027   232   231   021    Pre-fail  Always       -       3375
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       38
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   092   092   000    Old_age   Always       -       6220
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       36
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       28
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       9
194 Temperature_Celsius     0x0022   128   110   000    Old_age   Always       -       22
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

And a Samsung Spinpoint that's starting to show a little wear, note the Multi-Zone error rate.
Note, the more faults you get, the more retries your drive is doing, the slower it will get ....

Code: [Select]

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   100   100   051    Pre-fail  Always       -       1
  2 Throughput_Performance  0x0026   252   252   000    Old_age   Always       -       0
  3 Spin_Up_Time            0x0023   070   068   025    Pre-fail  Always       -       9224
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       45
  7 Seek_Error_Rate         0x002e   252   252   051    Old_age   Always       -       0
  8 Seek_Time_Performance   0x0024   252   252   015    Old_age   Offline      -       0
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       11383
 10 Spin_Retry_Count        0x0032   252   252   051    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   252   252   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       51
191 G-Sense_Error_Rate      0x0022   252   252   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0022   252   252   000    Old_age   Always       -       0
194 Temperature_Celsius     0x0002   064   062   000    Old_age   Always       -       25 (Min/Max 13/39)
195 Hardware_ECC_Recovered  0x003a   100   100   000    Old_age   Always       -       0
196 Reallocated_Event_Count 0x0032   252   252   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x002a   001   001   000    Old_age   Always       -       309314
225 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       51

[Mark Greaves (PCNetSpec)]

Im not going to argue whether 1Gb of RAM is enough .. at this point (as I don't know what you were running at the time, and what you have set to autostart) all I can do is point out that when you ran free -m you had 28MB or RAM left, and your system was using the swapfile.

If that is the "normal" state of your PC .. you need more RAM .. simple as.


Here's an idea .. install iotop:

Code: [Select]

sudo apt-get install iotop

run it with:

Code: [Select]

sudo iotop

now leave that terminal open and watch which process(es) are making all the disk accesses.

iotop displays the top I/O using processes.

You might also want to run:

Code: [Select]

top

in another termiinal, and see what is using all your memory

top displays the top memory/cpu using processes.

[Mark Greaves (PCNetSpec)]

@ Mad Penguin

That's handy to know as (IIRC) Peppermint doesn't take any notice of a /forcefsck file.

But using tune2fs to set the mount count higher than the maximum mount count (before checking) worked a treat

[EDIT]

Turns out Peppermint 3 *does* work with /forcefsck .. maybe it was Peppermint 2 or Linpus Lite 

[Mad Penguin]

. ....  I've not seen the forcefsck option before , tune2fs has been my stock method since Noah emptied out the Arc ...

1m or less is often fine on a server, but typically my Chrome instance eats more than that by itself ...

Sent from my Nexus S using Tapatalk 2

[mako6]

Thanks madmen. I tried the forcefsck (legitimate rape?) as per instructions but it didn't seem to work- maybe 3 seconds longer boot + no output, same number of errors, but honestly I don't trust that standard Disc Manager Ubuntu program- I tested that partition quite a while ago and think other disk check programs weren't showing problems. What's a good one in Ubuntu? Now I'm using Windows since it is working better and have secure Internet.

Yes Mark, seem to have many processes open eating all the RAM, even when I don't open anything- I've studied the process manager quite a bit before- Clam eats alot, sometimes that "sleeping" xxxx (neutral process can't remember but usually several instances open), evolution (would love to just prevent this from opening but can't find start-up control for processes) has 3-5 instances and bizarrely auto-opens in workspace 2 (the phantom instances that I never could close). I'll copy the output of it + send it to you when I mess w Ubuntu again. Which is most crucial- % use of processor, memory, cache or swap?

Thanks for diagnostic things- love to see what the hell is happening before I fix wrong problem. Ever replace a generator for $200 and a day's work because of loose fan belt or cable? Oww Oww. There really is a word: "swappiness" -sounds like something I'd invent. IOTOP is kinda cute too, like OOOTOP of Quake. Think reducing swapiness might help, but better to reduce load by eliminating unnecissary (sp) auto-opening processes. Pretty scrupulous about that in Windows- 15 processes max.

Sorry it is a SEAGATE drive, legendarily crummy, and the Ubun partition is near the end- at 130 gig of 160 gig drive. Initially had hellish problems with it masking 60 gigs of drive by that bizarre spy program that makes all disparates drive an identical length (100gig in my case). Luckily the powerful +_ invaluable HDAT2 analyzed it and made it all writable. What is difference with that "tune 2fs" instruction. Must read the manual for fsck (how does one exit from a manual in terminal- always have to close window) to see if I need any options.

[Mark Greaves (PCNetSpec)]

If you mean you don't trust the SMART data from Disk Utility .. trust it .. it's getting that information directly from the hard drive firmware itself.
(look up what S.M.A.R.T. (Self Monitoring Analysis and Reporting Technology) is)
http://en.wikipedia.org/wiki/S.M.A.R.T.

Effectively, if you say you don't trust SMART, you're saying you don't trust the hard drive to monitor itself .. if you don't trust the hard drive to monitor itself, get a new drive.

Swappiness .. changing swappiness will only change WHEN the swap file/partition begins to be used .. it will make no difference if you are running out of available RAM .. it will still start paging to swap when NECESSARY (which is when you run out of RAM).

Exit from a man page with the "Q" key .. or exit from anything else in a terminal with "Ctrl+C" (be careful with Ctrl+C)

I'm sure there are tutorials about removing Evolution in 10.04 .. but be careful, just uninstalling it in Synaptic removes half your desktop too IIRC.

ClamAV .. dunno why you're running that at all

Seriously, the easiest solution would be to FIT MORE RAM, or move to a "light" distro .. and that HDD will need replacing SOON.

Anything less, and you can expect issues .. Windows may currently be running perfectly because it's not spanning the failing sectors of the HDD, but when drives start to fail it most often spreads .. think about this .. if the read head at some point touched the platter and scraped off some of the magnetic coating, there are now little bits of metal flying around inside your HDD accelerating it demise .. same thing with the spindle bearings, etc.

Code: [Select]

Which is most crucial- % use of processor, memory, cache or swap?

If your CPU is at 100% usage, instruction will start to be queued .. at some point the PC may freeze till the queues are cleared.

Memory and swap are intertwined .. data is loaded to memory from the hard drive when it's going to be needed to be read or changed quickly, but when you run low on memory, rather than your system just crash, it starts using the swap partition of the hard drive as additional memory (which is why its sometimes called virtual memory), but the hard drive is 1000's of times slower than actual RAM .. so again your system will eventually have to pause whilst it waits for data from swap.

Neither are really "more important" .. but there's less you can do about CPU usage, short of replacing the CPU and maybe motherboard, or just not using CPU intensive applications.