Normal Linux commands to Locate your system files, INSTANTLY

[musher0]

Yes, Musher, of course the slocate package also contains the program updatedb. What I was trying to point out is that this:
"SEARCH FOR FILESYSTEM FILE IS INSTANTANEOUS"
is not accurate. slocate does not search the 'filesystem' -it searches the database created by using 'updatedb' or 'slocate -u'

If your run either of those commands and then add a file or files to the filesystem, these new items will not show up in the slocate 'search' since they are not in the database. slocate is only useful and accurate when the database is kept up-to-date.

True. Sorry for the misunderstanding.

[gcmartin]

Our biggest item is how does the command(s) get into PUPs terminal commandset as a standard item. PETs are useful in setup but, if its in the PUP's commandset, then users or distro developers can implement as they see proper need and use.

Not sure how to approach this, now that the community, here, has shown its benefit in use.

This is NOT a special program developed. This is a LINUX command, missing in PUPs

[musher0]

Hi, everyone.

Indeed, it's blazingly fast. Another feature, for me anyway, is that it is accessible
from the initial console. That may sound like nothing, but yesterday, slocate
helped me find my way around on the initial console. (I was wondering where I
had put a new wm I wanted to test.)

In any case, the slocate utility is an all-around plus for PuppyLinux. If it had been
in Puppy when I started, It would have saved me hours of feeling and groping
around the system.

It's worth having slocate in every Puppy and explaining its use to newbies. It'll
save them countless time and frustrations.

BFN.

musher0

[RetroTechGuy]

I decided to take Retrotechguy's item a bit further. I simply call it drive index creator (in menu under "Business") or via terminal typing "index1".

I'm still digesting what these tools do...

I have a lot of USB drives (HDD and flash), and usually don't leave stuff plugged in.

I'm thinking a combination of the indexxing tool here, and a better front-end to access the stored databases. It appears that this and slocate both create a single index file in /root/.

Since any given drive might be plugged in, in a random order, it would make sense for each drive to have its own database, and have those collated by the graphical or command line tool when plugged in (the tool can store the list of the file tree, and identify which mount location when it reports the search results).

[smokey01]

By default it creates a database called slocate.db in /var/lib/slocate

By having drives mounted does not necessarily mean they will be indexed, well not here anyway. The only drive/s that seems to get indexed are / and /mnt/home

How do you make it index all drives?

[musher0]

By default it creates a database called slocate.db in /var/lib/slocate

By having drives mounted does not necessarily mean they will be indexed, well not here anyway. The only drive/s that seems to get indexed are / and /mnt/home

How do you make it index all drives?

Hi, Smokey.

Look in the readme. It explains a way to create secondary db's. You probably can
create a file db for each thumb-drive and have it reside on it. (Or not!)

BFN.

musher0

[Uten]

I used to use locate when I started out on linux. But when I changed to a livecd distro and as my number of removable disks grew I simply did not find locate to be the right tool.

Using find and grep is my choice these days. I do something like this

Code: Select all

time nice find /mnt/hda1/ > /mnt/hda1/locate.db
time nice find /mnt/hdd2/ > /mnt/hdd2/locate.db
time nice find /root/ > /root/locate.db

Building the locate.db files obviously takes some time. The time spent depends on the disk IO speed, but this is also true when you use updatedb. I use nice in front of find to be able to do other work.


And I search with something like this:

Code: Select all

export locatedbs="/mnt/hda1/locate.db /mnt/hdd2/locate.db /root/locate.db"
grep hurt.*[.]mp3$ $locatedbs

It takes about 1sek to search 1.000.000 entries in the locate.db files on my pentium 4 (hmm, when the locate.db are in /tmp, on slow disks it will take a bit longer).


To update the locate.db files we can use find -cnewer and such, but usually I don't bother

Sometimes I wrap it all up in two nice scripts, depending on how much I think I will use it before a reinstall.

)

[slavvo67]

Enhanced Drive Index Creator. I jumped to Drive Index Creator - 3.

Again, just for kicks but I improved to be able to search and then search the sub. I'll probably go one more round and just let the user choose which directory to move the files to.

Best,

Slavvo67

[smokey01]

Hi musher0,

I'm not interested in indexing usb flash drives but I would like to index all of my main drives and partitions into one large.db file. This way I can find files even if the drives are not mounted. I guess drives/partitions would have to be mounted to index them though. It would be easy enough to write a script to mount each drive/partition, index it then unmount it at each reboot. It probably could be re-indexed more often but that would do me.

My problem is I have lots of drives/partitions and often have problems finding files. Having to search through many different .db files would be a pain or less help than seems possible.

I read the read.me but I failed to fully understand it.

Care to share some code to address my requirement.

Thanks

By default it creates a database called slocate.db in /var/lib/slocate

By having drives mounted does not necessarily mean they will be indexed, well not here anyway. The only drive/s that seems to get indexed are / and /mnt/home

How do you make it index all drives?

Hi, Smokey.

Look in the readme. It explains a way to create secondary db's. You probably can
create a file db for each thumb-drive and have it reside on it. (Or not!)

BFN.

musher0

[smokey01]

What is the latest source available for slocate?

The latest I could find was 3.1 with a patch to make it 3.1-1 but slocate --version still reports it at 3.1.

I'm running it of FD700RC and it's verrrrry fast.

[musher0]

Enhanced Drive Index Creator. I jumped to Drive Index Creator - 3.

Again, just for kicks but I improved to be able to search and then search the sub. I'll probably go one more round and just let the user choose which directory to move the files to.

Best,

Slavvo67

Hi, Slavvo67.

Looking at it now. TWYL.

musher0

[musher0]

Hello, Smokey001

In answer to your question, I tested, but I don't think slocate will create a database
of files outside the internal filesystem. It may be able to, but I haven't found how,
or I don't enough yet about this new-to-me utility. Maybe this is why it is called
slocate ("secure locate").

For the time being, I would suggest the following -- which is how I am currently
indexing the files on the drives outside my Puppy system per se.

Code: Select all

ls -lgohR /mnt/$home > /mnt/$home/files.lst

# $home, not 'home'. This will create a list of ALL files in /mnt
# -- in /mnt, under the name 'files.lst'.

ls -lgohR /mnt/sda1 > /mnt/sda1/files.lst

# This command will create a list of all files on drive sda1
# -- on /sda1, under the name 'files.lst'.

# Explanation of parameters, from back to front:
# -R  : recursive, meaning ls will dig until it finds the lowest ("bottomest", in bad 
# English!) folder in the folder tree.

# -h  : size of file is displayed in human readable form, e.g. 234K.

# -g and -o : these parms skip who the file or folder belongs to 
# (be it "root" or other)

# -l  : in long format, including date and type of file.

# All put next to one another as "-lgohR".

# ~~~~~~~~~~~

# If you wish to list also the hidden files, you would insert 'A' to the line of 
# parameters somewhere before the R, e.g.

ls -lAgohR /mnt # or # ls -lAgohR /mnt/sda1

# To save some typing, you can use this form:

Place="/mnt/sda1";ls -lAgohR $Place > $Place/files.lst

# Once sda1 is done, you rotate with the up arrow one line, and you only have to 
# change the drive number and hit the return key to index the next drive. And so 
# on, until you're finished.

# I keep my music files on drive sda1. So, to find all files in sda1 pertaining to, 
# for example, Chopin, I type:

grep Chopin /mnt/sda1/files.lst | more

# Another possibility is to load the files.lst file in less, like so

less -~ /mnt/sda1/files.lst

# and to look for the "Chopin" entries using the "/" function and then the "n" 
# function in less. This process takes longer than using grep, of course, but
some people may like the less presentation format.

If time is not a factor, you will wish to use, in the above, only the line with $home
in it. The files.lst created will have all the files in your /mnt tree.

As for me, I prefer the one-index-per-drive solution because refreshing those lists
takes much less time.

I hope the above is clear enough -- and that it helps a bit.

Best regards.

musher0

[musher0]

What is the latest source available for slocate?

The latest I could find was 3.1 with a patch to make it 3.1-1 but slocate --version still reports it at 3.1.

I'm running it of FD700RC and it's verrrrry fast.

Maybe the "patcher" didn't bother "patching" the version number?

I gather that "FD700RC" is the "Federal Department of 700 Remote
Convulsions" ? A branch of the Revenue service, no doubt.

BFN.

musher0

[musher0]

Hi, folks.



Not bad, eh? But you won't like it! It's not even locate, it runs with ls.
See my post above http://murga-linux.com/puppy/viewtopic. ... ost#828710,
this is a derivative.

In the interest of open source philosophy:

Code: Select all

#!/bin/ash
# MntFilesLst.sh ## A file list on every mounted drive
# musher0, Febr. 16, 2015.
####
# set -xe
echo -e "\e[1;36m\n\t\t      Please be patient, this takes a while."

a="`mount | awk '$3 ~ /mnt\/sd/ { print $3 }' | sort | wc -l`"
echo -e "\t\t\t $a files lists have to be indexed.\n\n"
for i in `mount | awk '$3 ~ /mnt\/sd/ { print $3 }' | sort`
do
	cd $i;ls -lAgohR $i > $i/files.lst
	echo -e "\e[1A\e[1;33m\t\t\t      The index on $i  "
	echo -e "\t\t\t         has been created.  \n"
done

cd /mnt/home;ls -lAgohR . > files.lst
echo -e "\e[3A\t\t\t      The index on /mnt/home  "
echo -e "\t\t\t         has been created.\n"

echo -e "\e[1;36m\t\t\t\t    Finished!\n\e[1;37m"
# set +xe

Anyway, tiny script attached. Unzip in ~/my-applications/bin and make executable.

BFN.

musher0

[slavvo67]

Download +1. I did a copy / paste instead.

Awk and Ash, very nice!

A couple of issues popped up using the copy / paste version:

The second font color comes up as a bright yellow in my terminal, making it difficult to read. I default to a white background which is clearly the issue. It looks great with your black background.

After indexing I was getting multiple errors to the following:

cannot read symbolic link /.proc/10/exe.....
cannot read symbolic link /.proc/11/exe...

With no such file or directory appearing to the right of the exe. I had at least 40 lines of this before I finally hit CTRL - C.

That said, it only took a minute or two to get through almost 400,000 files. Not bad in my book.

[musher0]

Thanks for testing, Slavvo67.

I'll have to rethink the color scheme... Maybe force the default background to black,
and the default foreground to white, just while this script is running. Which would
mean execution in a separate terminal, in order not to touch the user's defaults for
his/her terminal. But that's fine.

I forgot to mention in my previous post that the script will work only on mounted
drives. If a drive is not mounted, it won't appear in the "mount" list on the 1st line,
and its contents will not be indexed. So, Puppyists, mount all the drives you want
indexed before running this script. I don't like routines that fiddle with opening and
closing my partitions, so I didn't write one.

I see an "exe" after your symbolic link, also a /proc directory on that line. Correct
me if I'm wrong, but I believe that the /proc directory cannot be indexed. Also, /proc
is an internal system folder, and my script is for external drives only.

For internal files, one should use slocate, IMO, and AVOID scanning the /proc
folder altogether. I've been told the "files" and "links" in the /proc folder read stuff
pretty close to the bare machine level, so the features of the so-called files in /proc
are constantly changing. The contents of /proc are best accessed by cat, e.g.
cat /proc/meminfo. I would think that an indexing script would be useless on /proc.

Bye for now.

musher0

[gcmartin]

Hello @Smokey01

If I understand correctly, I think I know where you're going with use of this command in FD.

This might help, as I think @Kirk/@JamesBond would also be interested in this.

See this page of useful information

To those who have shared number of files that their systems contain...Thanks . You exemplify what we all are experiencing with content we have in our homes/possessions.

[Puppus Dogfellow]

(...)
Musher0 - Where is the actual Slocate program? Is it a pet somewhere?

Best,

Slavvo67

Hi, Slavvo67.

You can download the slackware package I used from the second URL listed at:
http://murga-linux.com/puppy/viewtopic. ... ost#826647
which is on the 1st page of the current thread.

As I mentioned in that post, I felt no need to create a pet package, at least on this
Puppy slacko, because my slacko-6.0b recognized the archive format and installed it
directly and instantly simply by clicking on the slocate.tgz archive.

Alll Puppies can install tgz packages directly, no? The capacity is built-in, in all
Puppies, like for *.deb archives. (I didn't do any extensive testing, but I think so.)

BFN.

musher0

the txz installed fine in precise 571. precise 5.5 didn't offer to install it (571 gives a popup--do you want to decompress this as an archive or install it as a slackware package?--something like that), but it gave a right click option to convert it, which may be from don570's right click suite or from something i've installed and forgotten. regardless, the 5.5 didn't install the .txz correctly, but stemsee's pet worked fine.

so there you have it. yes and no.

[musher0]

Hello,

@Puppus: Thanks for the tests!

I'm reworking my script above, and I think that I've solved the console colors
problem mentioned by Slavvo67.

I'll be introducing symlinks from the hard drives to a certain folder so that the user
can search one or all indexes conveniently. The links works ok, but...

What I'd like to know is what most users think would be a "convenient" folder for
those links. ~/my-applications/bin ? ~/my-documents? another place? What do you
use, routinely, for similar data files? And where is the slocate database located,
BTW? Sorry to say: I'm using it, it runs fine, but I can't find it anywhere!

Thanks in advance.

musher0

[Puppus Dogfellow]

Hello,

@Puppus: Thanks for the tests!

I'm reworking my script above, and I think that I've solved the console colors
problem mentioned by Slavvo67.

I'll be introducing symlinks from the hard drives to a certain folder so that the user
can search one or all indexes conveniently. The links works ok, but...

What I'd like to know is what most users think would be a "convenient" folder for
those links. ~/my-applications/bin ? ~/my-documents? another place? What do you
use, routinely, for similar data files? And where is the slocate database located,
BTW? Sorry to say: I'm using it, it runs fine, but I can't find it anywhere!

Thanks in advance.

musher0

Code: Select all

slocate slocate.db 

/var/lib/slocate/slocate.db

instant results.