Transferring large files via the terminal

Linux Support
1

Hi guys

Over the past week or so I’ve been rebuilding my NAS and changing from Freenas to Openmediavault, I have everything working ok all the shares set up etc but I’m falling at the last hurdle which is trying to transfer 80gb of movies I have on an ext, USB HDD onto a drive in the NAS up to this point, I’ve been transferring the files by booting a live Lubuntu session on the NAS mounting the ext HDD and copy/pasting the files over, but this last lot seems to freeze the session, so maybe it would be better to do it in the OMV terminal, but I’m not sure of the commands or alternatively I’m open to any other suggestions

Any help or advice would be much appreciated

Many thanks

Graeme

2

dunno what you want here ?

but something like:

cp -vr /path/to/source/directory /path/to/destination/directory

?

if you have something that’s refusing to copy … the -v option may help track it down.
and the -r option to also copy directories recursively.

see:

man cp
3

Hi mark

thanks for that, I now think the problems I’m having is due to the drive, I’ve since noticed it’s throwing weird errors when I try to write to it and randomly causing the NAS not to boot,

also I’m not sure if this is due to sata controller or the drive itself,

I’ve been desperately trying everything to get this drive to work, the whole point of this exercise was to use this drive to replace a 160gb drive full of childrens films that’s bursting at the seams, it’s really frustrating because I have 3 other drives installed with the shares set up and working perfectly and this was the last one

The odd thing is I can plug the drive into an all in one dock I have connected to my PC and the drive will mount and I can access it no problem it just seems to be when I try to write to it all hell breaks loose.

I can go back to the 160gb drive again but I would need to add another power socket to accommodate it but I can do that if I have to

Graeme

4

Sounds like it’s putting up one hell of a fight.

5
Sounds like it's putting up one hell of a fight.

Yeah it’s become a battle, no prizes for guessing who’s coming out second best

6

Just out of curiosity are there any utilities that could run any tests on this drive that might give me a clue what’s wrong with it or if it’s something that’s fixable

Graeme

7

What does the SMART data say in the “Disks” or “Disk Utility” ?

8
What does the SMART data say in the "Disks" or "Disk Utility" ?

it says SMART is not supported

9

Is the drive attached by USB ?

10
Is the drive attached by USB ?

yes it’s connected to my PC via a USB all in one dock

11

Ahh … I don’t think USB supports SMART.

12
Ahh .. I don't think USB supports SMART.

Ok well I can connect the drive back into the nas and communicate directly from the root terminal or ssh (which would allow me to copy/paste output) or boot into a live session, I only have Lubuntu 12.04 if that’s any good.

but just to embarras me further I just tried copying the files to the drive again and they’re now copying ok (no connection time outs so far and it’s remarkably fast (40 mins for the whole directory 80gb)

13

Spoke too soon it transfered about 20 or so films then crashed with these error messages

Chicken Run (2000).m4v: Error splicing file: Read-only file system folder.jpg: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/folder.jpg': Permission denied Chicken Run (2000).tbn: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/Chicken Run (2000).tbn': Permission denied cdart.png: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/cdart.png': Permission denied Chicken Run (2000)-fanart.jpg: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/Chicken Run (2000)-fanart.jpg': Permission denied Chicken Run (2000)-poster.jpg: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/Chicken Run (2000)-poster.jpg': Permission denied movie.nfo: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/movie.nfo': Permission denied Chicken Run (2000).nfo: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/Chicken Run (2000).nfo': Permission denied clearart.png: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/clearart.png': Permission denied disc.png: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/disc.png': Permission denied logo.png: Error opening file '/media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2/movies/Childrens/Chicken Run (2000)/logo.png': Permission denied .actors: Error creating directory: Permission denied Cinderella 2 (2002): Error creating directory: Permission denied Cinderella (1950): Error creating directory: Permission denied Despicable Me (2010): Error creating directory: Permission denied Dinosaur (2000): Error creating directory: Permission denied Dumbo (1941): Error creating directory: Permission denied Epic 2013: Error creating directory: No such file or directory Finding Nemo (2003): Error creating directory: No such file or directory George of The Jungle (1997): Error creating directory: No such file or directory Happy Feet 2 (2011): Error creating directory: No such file or directory Happy Feet (2006): Error creating directory: No such file or directory High School Musical (2006): Error creating directory: No such file or directory Hoodwinked Too-Hood V Evil (2011): Error creating directory: No such file or directory Horton Hears A Who (2008): Error creating directory: No such file or directory How The Grinch Stole Christmas (2000): Error creating directory: No such file or directory Ice Age 2-The Meltdown (2006): Error creating directory: No such file or directory Ice Age 3-Dawn of the Dinosaur (2009): Error creating directory: No such file or directory Ice Age 4-Continental Drift (2012): Error creating directory: No such file or directory Ice Age (2002): Error creating directory: No such file or directory Jungle Book (1967): Error creating directory: No such file or directory Kung Fu Panda 2 (2011): Error creating directory: No such file or directory Kung Fu Panda (2008): Error creating directory: No such file or directory Lady and The Tramp 2-Scamps Adventure (2001): Error creating directory: No such file or directory Lady and The Tramp (1955): Error creating directory: No such file or directory Madagascar 2-Escape 2 Africa (2008): Error creating directory: No such file or directory Madagascar 3-Europes Most Wanted (2012): Error creating directory: No such file or directory Madagascar (2005): Error creating directory: No such file or directory Matilda (1996): Error creating directory: No such file or directory Monster Inc (2001): Error creating directory: No such file or directory Monsters University (2013): Error creating directory: No such file or directory Oliver & Company (1988): Error creating directory: No such file or directory Open Season 1 (2006): Error creating directory: No such file or directory Open Season 2 (2008): Error creating directory: No such file or directory Open Season 3 (2010): Error creating directory: No such file or directory Over The Hedge (2006): Error creating directory: No such file or directory Pinocchio (1940): Error creating directory: No such file or directory Pocahontas 2 (1998): Error creating directory: No such file or directory Pocahontas (1995): Error creating directory: No such file or directory Puss in Boots (2009): Error creating directory: No such file or directory Rango (2011): Error creating directory: No such file or directory Ratatouille (2007): Error creating directory: No such file or directory Robin Hood (1973): Error creating directory: No such file or directory Rockin' With Judy Jetson: Error creating directory: No such file or directory Shark Tale (2004): Error creating directory: No such file or directory Shrek (2001): Error creating directory: No such file or directory Shrek-Forever After (2010): Error creating directory: No such file or directory Sleeping Beauty (1959): Error creating directory: No such file or directory Snow White and The Seven Dwarfs (1937): Error creating directory: No such file or directory Stuart Little 2 (2002): Error creating directory: No such file or directory Stuart Little 3 (2005): Error creating directory: No such file or directory Stuart Little (1999): Error creating directory: No such file or directory Surfs Up (2007): Error creating directory: No such file or directory Tangled (2010): Error creating directory: No such file or directory The Aristocats (1970): Error creating directory: No such file or directory The Chronicals of Narnia-Prince Caspian (2008): Error creating directory: No such file or directory The Chronicals of Narnia-The Lion, The Witch and The Wardrobe (2005): Error creating directory: No such file or directory The Fox and The Hound 2 (2006): Error creating directory: No such file or directory The Fox and The Hound (1981): Error creating directory: No such file or directory The Gruffalo (2009): Error creating directory: No such file or directory The Gruffalos Child (2011): Error creating directory: No such file or directory The Hunchback of Notre Dame 2 (2002): Error creating directory: No such file or directory The Hunchback of Notre Dame (1996): Error creating directory: No such file or directory The Land Before Time (1988): Error creating directory: No such file or directory The Legend of The Guardians (2010): Error creating directory: No such file or directory The Lion King 2-Simbas Pride (1998): Error creating directory: No such file or directory The Lion King (1994): Error creating directory: No such file or directory The Little Mermaid 2-Return To The Sea (2000): Error creating directory: No such file or directory The Little Mermaid (1989): Error creating directory: No such file or directory The Little Mermaid-Ariel's Begining (2008): Error creating directory: No such file or directory The Nutcracker Prince (1990): Error creating directory: No such file or directory The Penguins of Madagascar-Operation Get Ducky (2010): Error creating directory: No such file or directory The Princess & The Frog (2009): Error creating directory: No such file or directory The Wizard of Oz (1939): Error creating directory: No such file or directory Tinkerbell (2008): Error creating directory: No such file or directory Tinkerbell and The Great Fairy Rescue (2010): Error creating directory: No such file or directory Tinkerbell And The Lost Treasure (2009): Error creating directory: No such file or directory Top Cat-The Movie (2011): Error creating directory: No such file or directory Toy Story 2 (1999): Error creating directory: No such file or directory Toy Story 3 (2010): Error creating directory: No such file or directory Toy Story (1995): Error creating directory: No such file or directory Up (2009): Error creating directory: No such file or directory Wall-E (2008): Error creating directory: No such file or directory Wreck It Ralph (2012): Error creating directory: No such file or directory Yogi Bear (2010): Error creating directory: No such file
14

OK, you’re not giving enough info here…

a) what’s the location of the files you’re trying to copy

b) what’s the path you’re trying to copy them to

c) what’s the command you’re using

it “looks” like the target drives file system hit an error and was remounted “read only” … what’s the output from:

mount
15
OK, you're not giving enough info here....

sorry

what's the location of the files you're trying to copy

there on my PC on a Windows partition

what's the path you're trying to copy them to

I’m copying to the 500gb hard drive plugged into a USB docking station connected to my PC

what's the command you're using

No command just copy and paste

graeme@Linux1 ~ $ mount
/dev/sdb6 on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/cgroup type tmpfs (rw)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)


none on /run/shm type tmpfs (rw,nosuid,nodev)
none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
gvfsd-fuse on /run/user/graeme/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,user=graeme)
/dev/sdb1 on /media/graeme/5647AEED206B9449 type fuseblk (rw,nosuid,nodev,allow_other,default_permissions,blksize=4096)
/dev/sdh1 on /media/graeme/25428eef-e71f-4e3e-aeef-048568df49a2 type ext4 (rw,nosuid,nodev,uhelper=udisks2)
graeme@Linux1 ~ $

All the files are copying over at the moment and seems stable, but issues will likely occur when I plug the drive into the NAS and boot

16

I’m completely lost here :confused:

is it the source drive that’s failing … or the target drive ?

Does the target drive only fail when attached to the NAS ? … attached how ?

17
is it the source drive that's failing .. or the target drive ?

The target drive (500gb sata) although tonight is the only time I’ve been able to copy files without errors and connection timeouts to that drive, which I was previously getting all the time whether connected to the NAS or my PC via the usb docking station.

When it was connected to the NAS errors were flagging up during boot up referring to that drive and in some cases refusing to boot

So I’ve now managed to transfer one movie directory, but there’s still roughly another 60gb I need to transfer which is currently stored on a an external usb hdd

Sorry if I’m not explaining this well I am trying but this whole operation has been a nightmare from start to finish and my head’s up my arse

Graeme

18

After transferring files over to the drive successfully (while connected to my PC via the usb docking station) I transferred the drive into the NAS and rebooted here’s the output from the NAS terminal

see screenshot

19

If it’s any help, if I boot the NAS with the drive disconnected then connect the drive it will show up in the web gui under “Physical Drives” and online under “Filesystems” but refuses to mount with the error “communication failure”

20

Out of curiosity, is this (troublesome) drive the only sata drive in the NAS?