Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 18 Nov 2017, 12:09
All times are UTC - 4
 Forum index » Taking the Puppy out for a walk » Misc
How to reduce the size of a HDD image?
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 2 [17 Posts]   Goto page: 1, 2 Next
Author Message
John Alien

Joined: 26 May 2017
Posts: 17

PostPosted: Sun 25 Jun 2017, 19:19    Post subject:  How to reduce the size of a HDD image?  

It's a bit of a general linux question.. How can I reduce an image of a 250gb hard drive to write the image to a 128gb ssd, provided that there is enough free space on the 250gb hard disk?

If I write the hard drive as a qcow2 image it will reduce the size? Or does it have to also be written to a device which has TRIM enabled?

Can I use dd to write the qcow2 image to an ssd?
Back to top
View user's profile Send private message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 12699
Location: Arizona USA

PostPosted: Mon 26 Jun 2017, 00:09    Post subject:  

Have you considered defragmenting the 250 GB drive? Done properly, defragmenting should cram all the data on the drive to the beginning of the drive, with no wasted space. Assuming there is less data on that drive than will fill the drive you want to copy it to, that should do the trick if the trick can indeed be done.

I think dd copies the data, bit for bit, including unused areas in a badly fragmented drive. So you'd need to defragment a drive before you used dd to copy everything from it to a smaller drive.
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 649

PostPosted: Mon 26 Jun 2017, 00:15    Post subject: Re: Reducing the size of an image  

John Alien wrote:
It's a bit of a general linux question.. How can I reduce an image of a 250gb hard drive to write the image to a 128gb ssd, provided that there is enough free space on the 250gb hard disk?

If I write the hard drive as a qcow2 image it will reduce the size? Or does it have to also be written to a device which has TRIM enabled?

Can I use dd to write the qcow2 image to an ssd?

I'm not sure. You could try it and see what happens. Here are some instructions.

https://blog.sviluppoweb.eu/2010/07/16/how-to-convert-an-existing-partitiondisk-in-an-image-to-virtualize-it-in-linux-qemukvm/

I think though if you defragment a disk that you shouldn't need to image the whole thing so maybe either "dd" or "safecopy" would be fine.


Once you've created the image you can mount it as a loop device.

Here are three example mount commands:
Code:

losetup /dev/loop3 /mnt/sdg6/part3.dat
mount -o loop -t ntfs-3g /mnt/sdg6/part3.dat /mnt/sdg4/
mount -o loop /mnt/sdg6/part3.dat /mnt/sdg4/


Some cloning tools can create spare images. For example with ntfsclone you can create a sparse image of an ntfs partition.

Some further research tells me some ways that you can create a sparse file:
Code:

dd if=/dev/sdb1 | cp --sparse=always /proc/self/fd/0 new-sparse-file

https://unix.stackexchange.com/questions/25528/is-creating-a-sparse-image-using-dd-appropriate-for-backup-restore-from-a-raid

Code:

$ ddrescue -S -b8M /dev/sda1 /mount/external/backup/sda1.raw

https://unix.stackexchange.com/questions/17087/clone-whole-partition-or-hard-drive-to-a-sparse-file

In the last option the capital "S" is for sparce copy in ddresecule. See the manpage:
https://linux.die.net/man/1/ddrescue
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 649

PostPosted: Mon 26 Jun 2017, 00:19    Post subject:  

Flash wrote:
Have you considered defragmenting the 250 GB drive? Done properly, defragmenting should cram all the data on the drive to the beginning of the drive, with no wasted space. Assuming there is less data on that drive than will fill the drive you want to copy it to, that should do the trick if the trick can indeed be done.

I think dd copies the data, bit for bit, including unused areas in a badly fragmented drive. So you'd need to defragment a drive before you used dd to copy everything from it to a smaller drive.


You beat me to it by 6 seconds! Anyway, I suspect that whatever approach the original post takes that better results will be obtained by defragmenting the disk first.
Back to top
View user's profile Send private message 
p310don

Joined: 19 May 2009
Posts: 1113
Location: Brisbane, Australia

PostPosted: Mon 26 Jun 2017, 00:29    Post subject:  

What is the image of? What is your end goal?

My experience with copying an old XP hard drive to a VM has been to defrag first and then resize the partition to leave almost no free space. Then create the image. Then create a VM from the resultant image.

You can also reduce the size of the image for a VM using tools for the VM program (I use VirtualBox).
Back to top
View user's profile Send private message 
John Alien

Joined: 26 May 2017
Posts: 17

PostPosted: Mon 26 Jun 2017, 00:48    Post subject:  

In case it's unclear, the goal is to copy the bootloader and the operating system from a 250gb harddrive to a 128gb ssd

I just found a possible solution combining these two ideas:

https://serverfault.com/questions/439128/dd-on-entire-disk-but-do-not-want-empty-portion
https://raspberrypi.stackexchange.com/questions/7177/image-of-a-16gb-card-containing-unpartitioned-space-at-the-end-truncating-possi

use gparted to shrink the disk to minimum possible size, leaving rest of the space unallocated
Use dd to create an .img of the disk
Use fdisk to check where the last partition on the disk ends
Use truncate on the .img
dd .img to ssd


I'd say it's very possible resizing a partition defragments the disk... Does anyone know?

I see p310don does something similar. And there is a defrag involved in it's own right.

Not that I have anything against defragging, but is it necessary to do before reducing the partition size?

Also if I'm pretty sure I'll need to truncate after making the disk image. AFAIK dd will always output the whole disk irregardless of where partitions are.

I wonder if qcow2 can be used to make an image only the size the disk's partitions...
Back to top
View user's profile Send private message 
p310don

Joined: 19 May 2009
Posts: 1113
Location: Brisbane, Australia

PostPosted: Mon 26 Jun 2017, 08:34    Post subject:  

What is the original system? Is it linux or windows?

Linux is much more tolerant of being copied. I have had success with simply copying everything over to a new hard drive / SSD. Windows is not tolerant at all of such a thing.

If it is windows, a defrag is necessary (if the drive is fragmented) to compress the partition down.

If linux source, boot from live medium (CD or USB) Puppy is great for this! Compress entire hard drive into single .gz file, uncompress onto bootable SSD.

If windows source, defrag (which essentially compresses and decompresses into one area of the disk), then boot Puppy with live medium, use gparted to resize partition to have no free space. Use PUDD to copy the partition from the old drive to the new SSD (PUDD being a basic front end for DD). Then use gparted again to grow the partition to whatever size you like (probably to fill the disk).

You can use the windows method for Linux of course. YMMV with how tolerant your OS of choice is.
Back to top
View user's profile Send private message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 12699
Location: Arizona USA

PostPosted: Mon 26 Jun 2017, 11:43    Post subject:  

I don't know if resizing a disk defragments it, but I think copying (not using dd) automatically defragments the data in the process of copying it. In other words, I think that copying first assembles entire files before writing them to the destination.
Back to top
View user's profile Send private message 
Burn_IT


Joined: 12 Aug 2006
Posts: 2913
Location: Tamworth UK

PostPosted: Mon 26 Jun 2017, 18:10    Post subject:  

The key here is to stop using the word IMAGE.
What you are trying to do is copy the CONTENTS of the disk to the SSD.
An IMAGE can really only be copied to a file or to a disk with the same architecture (or larger).
In your case you are not only wanting just the actual data, but you need it repacking for the new format of the SSD.
The safest way of doing it is with a disk handling tool that is designed to copy old disks to new ones such as GPARTED or any other stand alone Disk/Prtition tool.
It is also best to use a bootable tool so that both the intended source and target disks are not in use.

_________________
"Just think of it as leaving early to avoid the rush" - T Pratchett
Back to top
View user's profile Send private message 
John Alien

Joined: 26 May 2017
Posts: 17

PostPosted: Mon 26 Jun 2017, 20:25    Post subject:  

p310don wrote:
What is the original system? Is it linux or windows?


The source is puppy.

p310don wrote:
If linux source, boot from live medium (CD or USB) Puppy is great for this! Compress entire hard drive into single .gz file, uncompress onto bootable SSD.


So I can use dd to store the contents of a disk in a gz file and I can point dd towards the entire device ie. /dev/sdb as the input file? And is it compressing with gz the operation which removes the empty space in this case?

p310don wrote:
Use PUDD to copy the partition from the old drive to the new SSD (PUDD being a basic front end for DD).

Puppy is amazingly versatile, good work everyone involved.
Back to top
View user's profile Send private message 
John Alien

Joined: 26 May 2017
Posts: 17

PostPosted: Mon 26 Jun 2017, 20:25    Post subject:  

Burn_IT wrote:
The key here is to stop using the word IMAGE.


Ok.
Back to top
View user's profile Send private message 
p310don

Joined: 19 May 2009
Posts: 1113
Location: Brisbane, Australia

PostPosted: Tue 27 Jun 2017, 06:29    Post subject:  

Quote:
The source is puppy.


Well that opens up a whole other can of worms. Most operating systems aren't made to be portable. Puppy is.

If you used a frugal install, you can simply make the SSD bootable, copy grub or grub4dos or whichever bootloader you have, copy the puppy files from the ISO and then copy over your save file & VOILA puppy is installed on the new drive. No need to much about with DD, PUDD, qcow2 or any other convoluted process. That is one of the million things that makes puppy awesome.

Quote:
So I can use dd to store the contents of a disk in a gz file and I can point dd towards the entire device ie. /dev/sdb as the input file? And is it compressing with gz the operation which removes the empty space in this case?


DD can create an image of the drive. PUDD creates that image, but compresses it into a GZ file, making it about half the size. DD (and PUDD) give you an exact duplication of the drive, including the empty bits.
You can simply compress the files on your drive into an archive of your choice, and then uncompress them to get rid of the empty spaces between files. This is what defrag does. If your source is Puppy, I might assume you have an ext3 or 4 file system, so you don't really need to worry about defrag. You can of course create the archive file and then unpack it onto the SSD, but that is an extra step beyond simply copying the files.


For what you're doing, my method would be:

Install SSD into PC.

Fire it up to boot Puppy. Either from USB, or from old HDD if you have more than one drive installed.

Open Gparted, format the SSD to be bootable. Right Click, select manage flags, tick make bootable.

Click Install on desktop

Go through the install process

The copy your old hard drive's files to the SSD, overwriting everything.

Cross your fingers

Turn off the PC. Take out the old drive. Turn on the PC.

Cross your other fingers.

Marvel at the 40x speed increase from SSD Smile
Back to top
View user's profile Send private message 
John Alien

Joined: 26 May 2017
Posts: 17

PostPosted: Wed 28 Jun 2017, 22:29    Post subject:  

Sounds good. Will I have to also install grub on the ssd as part of the installation process?

I'm a little unsure of how grub works.

Format the SSD to be bootable.
Install puppy, let the installer partition.
Install Grub (or should grub be installed before puppy? Does it matter?)
Copy, essentially something like 'cp -rf /mnt/sda/ /mnt/sdb/'

And then grub fits in the process like: Hardware should see the bootable drive, start grub, grub reads the partition table and then it will know where to find the kernel?

Which, hopefully, boots into the system I'm try to copy?

When grub reads the partition table, does it then read the grub configuration files from the partition with /boot?

Last edited by John Alien on Thu 29 Jun 2017, 19:44; edited 1 time in total
Back to top
View user's profile Send private message 
cthisbear

Joined: 29 Jan 2006
Posts: 4215
Location: Sydney Australia

PostPosted: Thu 29 Jun 2017, 06:53    Post subject:  

My Chinese computer shop techie had a Toshiba laptop that had issues.

I gave him Puppy 4.21 at the time and newer Puppies later as rescue
disks. Plus many others.

He always copied the data with Puppy to a spare drive,
and used the hidden partition to do the Factory re-install.

Then recopy the data back.
Anyway this particular Toshiba refused to boot properly for some reason.
He redid it twice...no go.

I got him to give me a spare hard drive.
Copied his new but failing Windows install...

Program Files
Users
Windows
Documents and Settings
ProgramData

Booted a Windows disc....and installed Windows 7 just one percent,

Then I rebooted in Puppy, deleted any Win files and just copied
all the Win files from the spare hard drive back to the main drive.

Checked GParted that it could boot, and rebooted.

Voila! booted cleanly into Windows.
He could then copy the data back.

He knew Puppy was good...but this was awesome to him.

When I install any Windows system, I always boot Puppy straight after, and clean the new install out, and make a backup.

I then only ever install the drive1%, clean it out as above with Puppy, and recopy the files. Never had a fail yet.

Chris.
Back to top
View user's profile Send private message 
John Alien

Joined: 26 May 2017
Posts: 17

PostPosted: Tue 04 Jul 2017, 01:26    Post subject:  

Very good to know that about Windows 7.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 2 [17 Posts]   Goto page: 1, 2 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Taking the Puppy out for a walk » Misc
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0588s ][ Queries: 13 (0.0037s) ][ GZIP on ]