Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 18 Apr 2015, 11:16
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Filesystem
afo - Aggressive file obliterator
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [10 Posts]  
Author Message
jafadmin

Joined: 19 Mar 2009
Posts: 477

PostPosted: Wed 18 Mar 2015, 13:35    Post subject:  afo - Aggressive file obliterator  

Code change (Mar-23): Massive speed increase. Re-download afo & bcrypt.
Source updated as well...


This utility installs for use via Roxfiler as well as the command line.

It overwrites files/directories with random gibberish then deletes them. Files deleted with this utility won't be recovered using undelete functions, recovery software, or dd/hexedit.

It permanently obliterates your files to the point they are not recoverable.

Installation adds a right-click item in Roxfiler under the "Open With" menu. Use Rox to select files and folders, right-click, select "Open With", then click on "Afo".

You can specify the number of overwrites on the command line. The default is 1 pass, which should suffice. If you wear your tinfoil a little tighter than I do you can specify any number of overwrite passes. That default parameter for rox can be set by editing the /usr/local/apps/Afo/AppRun file and changing the -c1 to your favorite number. i.e.: -c3

This should run on all versions of puppy. If problems, let me know.

-jafadmin
Afo.pet
Description 
pet

 Download 
Filename  Afo.pet 
Filesize  6.11 KB 
Downloaded  42 Time(s) 
afo-source.tar.gz
Description 
gz

 Download 
Filename  afo-source.tar.gz 
Filesize  4.85 KB 
Downloaded  22 Time(s) 

Last edited by jafadmin on Wed 25 Mar 2015, 17:11; edited 7 times in total
Back to top
View user's profile Send private message 
jafadmin

Joined: 19 Mar 2009
Posts: 477

PostPosted: Sun 22 Mar 2015, 12:28    Post subject:  

One of the problems with these kinds of utilities is it's sometimes hard to verify that they actually function as advertised.

I ran into the problem originally with the BCrypt utility that is included with all installs of Puppy Linux. I was trying to verify that the "-sN" switch, which tells bcrypt how many times to overwrite the original file with random garbage before deleting, was working properly.

So here at the Jafadmin Furious Linux Research Grotto & Pasta Bistro™ I set up a lab to do just that. The results of that lab were downright terrifying.

I used dd, hexedit and gparted to do the experimentation. First I created a 1M ext2 partition on a spare thumb drive with gparted. Next I used dd to overwrite the partition with zeros, then reformatted it. I created a text file filled with "Lorem ipsum" fill text on the partition, then encrypted/compressed it with bcrypt using the -s2 switch which should overwrite it twice with random gibberish before deleting. Bcrypt created the .bfe file and deleted the original.

Next I used dd to create an image file of the 1M partition and opened the image file with Hexedit. I searched for the string "Lorem", and found the whole entire text of the file.
The file simply WAS NOT getting overwritten at all!

I located the source code for bcrypt on sourceforge and proceeded to investigate. I found two disturbing things in the "rwfile.c" file; 1) The original algorithm for overwriting the files would only overwrite the first 25% of the file. And, 2) Once that was fixed it still didn't work.

What the heck was going on here? If I changed the code so that it just didn't unlink (delete) the file after overwriting, it worked just fine. I wound up with an original file that was filled with random gibberish.

Here's the deal: The program was using high level I/O functions to access the file. Specifically the fopen(), fread(), fwrite() family of file I/O functions. Why was this bad? Because all modern Linux OS's optimize their file I/O with buffering and prioritizing algorithms. The OS processes the "unlink()" command first, and discards pending write requests for that file.

The fix? Low level I/O functions. I rewrote the file clobbering function using the open(), read(), write(), and fsync() functions that bypass OS buffering and work directly on the disk.

So what's my point with all of this? After fixing my version of bcrypt, I wrote "afo" so I had a tool that could do secure deletes on a Directory/File level instead of having to wipe whole disks/partitions.

A Word To The Wise: None of these following things got rid of the original file data.
1) Reformatting failed.
2) Reformatting to a different filesystem failed
3) Deleting the partition failed.
4) Deleting the partition table failed

The only thing that worked was overwriting the partition with dd
Code:
dd bs=1M if=/dev/zero of=/dev/"partition descriptor", or
dd bs=1M if=/dev/random of=/dev/"partition descriptor"


Here is the original bcrypt source with my documented code changes as well as the original source code. It also contains a new compiled bcrypt binary with the bug fixes:
bcrypt-1.1.(jafa).tar.gz
Description  fixed bcrypt
gz

 Download 
Filename  bcrypt-1.1.(jafa).tar.gz 
Filesize  51.5 KB 
Downloaded  20 Time(s) 

Last edited by jafadmin on Thu 02 Apr 2015, 21:49; edited 4 times in total
Back to top
View user's profile Send private message 
seaside

Joined: 11 Apr 2007
Posts: 889

PostPosted: Sun 22 Mar 2015, 18:00    Post subject:  

Jafadmin,

File deletion utilities are very close to backup utilities. Very rarely does anyone really test them!

Thank you for actually making this test on bcrypt. It's quite revealing to learn that it doesn't behave as anyone thought.

I'm sure your utility Afo works nicely, and the inclusion of the source code for the Afo binary would make it even more comforting.

Cheers, and thanks again,
s
Back to top
View user's profile Send private message 
jafadmin

Joined: 19 Mar 2009
Posts: 477

PostPosted: Sun 22 Mar 2015, 21:27    Post subject:  

seaside wrote:


I'm sure your utility Afo works nicely, and the inclusion of the source code for the Afo binary would make it even more comforting.

Cheers, and thanks again,
s


Agreed. Done.
Back to top
View user's profile Send private message 
Burn_IT


Joined: 12 Aug 2006
Posts: 1407
Location: Tamworth UK

PostPosted: Mon 23 Mar 2015, 08:17    Post subject:  

Quote:
A Word To The Wise: None of these following things got rid of the original file data.
1) Reformatting failed.
2) Reformatting to a different filesystem failed
3) Deleting the partition failed.
4) Deleting the partition table failed

The only thing that worked was overwriting the partition with dd
That is perfectly correct and to be expected.The only one of those that might actually write to the data portion of the file rather than just the "table entries" is the reformat, and even then only a FULL format and it only marks the cluster as formatted and doesn't remove prior data.

Even with data overwrite on magnetic disks it is possible (though expensive) to retrieve prior data after a few passes. That is why DOD standards say seven overwrites with different random data each time.
It is also very difficult to "clean" SSD type storage media because of the randomising. The only sure way is to delete the whole disk and force full garbage collection.

_________________
"Just think of it as leaving early to avoid the rush" - T Pratchett
Back to top
View user's profile Send private message 
jafadmin

Joined: 19 Mar 2009
Posts: 477

PostPosted: Mon 23 Mar 2015, 11:53    Post subject:  

Burn_IT wrote:

Even with data overwrite on magnetic disks it is possible (though expensive) to retrieve prior data after a few passes. That is why DOD standards say seven overwrites with different random data each time.
It is also very difficult to "clean" SSD type storage media because of the randomising. The only sure way is to delete the whole disk and force full garbage collection.


I have an associate that is a DOJ certified Forensic IT Specialist. These guys have really expensive hardware and software designed to do just exactly what you said.

According to this individual, the assertion that data can be retrieved after a disk has been overwritten, even with just one pass of zeros, is largely just a theory.

There aren't any actual proofs of this. Anywhere. Think about that. Someone would have published by now.
Back to top
View user's profile Send private message 
Burn_IT


Joined: 12 Aug 2006
Posts: 1407
Location: Tamworth UK

PostPosted: Mon 23 Mar 2015, 12:23    Post subject:  

That is not entirely true. I have some software that will recover data from a disk that has been overwritten with a simple static pattern.
I have done it. It took some time to rebuild the original file, mainly because there were earlier generations there as well.

What I didn't know at the time was that the person who asked me to do it was employed at the MOD and was testing potential suppliers. They were thinking of buying a product I was selling and were doing security checks.

_________________
"Just think of it as leaving early to avoid the rush" - T Pratchett
Back to top
View user's profile Send private message 
jafadmin

Joined: 19 Mar 2009
Posts: 477

PostPosted: Mon 23 Mar 2015, 12:58    Post subject:  

It would be fascinating see how software by itself would accomplish that.

Without using a different type of read/write head (hardware) on the disk, I have difficulty understanding how software alone could reliably read a different value from a given disk location.

While an abundance of caution is essential for safe computing these days, I think it's incumbent for we technologists to differentiate in our own minds between theory and reality.
Back to top
View user's profile Send private message 
Burn_IT


Joined: 12 Aug 2006
Posts: 1407
Location: Tamworth UK

PostPosted: Mon 23 Mar 2015, 13:51    Post subject:  

I used http://www.lc-tech.com/pc/filerecovery/

It was some years ago and was a pro version.

_________________
"Just think of it as leaving early to avoid the rush" - T Pratchett
Back to top
View user's profile Send private message 
jafadmin

Joined: 19 Mar 2009
Posts: 477

PostPosted: Mon 23 Mar 2015, 14:04    Post subject:  

Burn_IT wrote:
I used http://www.lc-tech.com/pc/filerecovery/

It was some years ago and was a pro version.


Yeah, that's pretty standard file recovery stuff. You can do the same thing with dd and a hex editor as I pointed out above. That kind of software automates file reassembly though, which is nice.

That type of software assumes the disk hasn't been overwritten, though. Using either of the dd commands above will render the software useless.

Give it a try some time. This is why lab work is fun.

[edit: 3-31-2015] I published this here for purposes of peer review. I fully expect technical people here to try this on their own then test/publish their independent results.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [10 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Filesystem
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0775s ][ Queries: 12 (0.0056s) ][ GZIP on ]